Bash script - for rsync to backup file server running samba.

[James Dugger]

Lisa, Thanks for the code snippets and the direction.

Kevin,

I anticipated that I might get this response and I appreciate the
recommendations.  I will look into backupPC.  There are some pretty narrow
constraints that I am working with that have lead me to this direction.

1.  This project is for a business that has outgrown the home offices of 5
people into an established office location.  They all use laptops that are
currently not apart of any network.  I use the term LAN only in the sense
that their router is providing wireless access and the ability (now that
they are are centrally located) to connect to each other.
2.  These users have little if any understanding or experience with
connecting or working in a network environment where they share files from
a server, and do not want to.
3.  They are averse to spending any additional resources to formalize a LAN.
4.  They still work from home occasionally and each persons tasks are such
the they don't need to share files very much.
5.  The only reason they have been convinced that they need a central
backup system is because of a recent scare to a lost directory on one of
their laptops.
6.  They will most likely not initiate a backup from there laptops and
given personal use of the laptops if I install a system that works from
there laptop there is very little guarantee that they won't inadvertently
shut it down (accidentally).

Sounds like a security nightmare.  At this stage I am only providing a NAS
and a backup solution and trying to make it as locked down as I can.

With the parameters above I have built a NAS backup server that basically
works in the background with only the following tasks

1.  Manage a 4 TB software RAID 10 configuration of disks
2.  Provide a file server environment using Samba
3.  Provide a symlinked share folder (named the same as there My Docs)
folder to access backups if needed
3.  Webmin and Openssh for admin access (server is headless)
4.  Provide backup script using rsync and cron to create a rotating
incremental backup
5.  Perform all this on the server side - I have no control
administratively to do much configuring or lock down the laptops.
6.  Create a log file of backups and notify me by email of any viruses,
failures, or crashes.

All that aside, I have looked at several backup "programs" that are out
there and as I have delved into them more and more I am finding that many
of them are simply scripts written in Perl or Python or bash to run rsync
or a version of it.  I personally think that it is overkill to load a
program that is designed to do 13 things when I need it to only to do 3.  I
have in the past attempted to configure rsnapshot for this very purpose
only to be disappointing in my limitations on naming files, or how the
backup works, or a combination of all of these.

I find it ironic that I should spend time learning first how backupPC works
than learn how to configure it to do something that you can have rsync do
in no more than a handful of lines in a bash script.  Besides I am trying
to leverage my time learning more about Linux, bash scripting, and the
standardized offerings in most distros (rsync).  I spent 20 years "finding"
software in M$ world to make sure that I didn't have to learn anything
about Windows servers.

As far as the load on the network, all of the laptops and router have
gigabit NIC's.  Most of the work that they do is actually updating
their eCommerce site which is hosted offsite.  There are incidental daily
text files and image files optimized (sized) for web that are the primary
items being backup.  rsync should be able by itself to snapshot incremental
file changes in all of their directories after the initial full backup in
seconds simultaneously.  At any rate the only real load is on the backup
server which should not have any other active writes to the drives from
users.

Despite this I do appreciate the insights of any here.  I am not putting
down any programs, and if this was a more standardized network with more
band usage, server services, and user account management I would check
myself in to the nearest loony bin before trying to "reinvent the wheel".


On Fri, Jan 6, 2012 at 11:01 AM, Kevin Fries <kevin at fries-biro.com> wrote:

> On Fri, 2012-01-06 at 08:49 -0700, Lisa Kachold wrote:
> > Hi Kevin:
> > On Fri, Jan 6, 2012 at 8:26 AM, Kevin Fries <kevin at fries-biro.com>
> > wrote:
> >         Just out of curiosity...
> >         Is there a specific use case you are using to explain why you
> >         are reinventing the wheel?
> > Much of our discussion with scripting and systems use is like "Hello
> > World".
> >
> >         http://backuppc.sourceforge.net/faq/BackupPC.html
> >         It would help to know exactly what you are trying to achieve
> >         that BackupPC can't handle so we can find you a better
> >         solution.
> > Great solution: but he is trying to identify all connected samba
> > shares from Windows7 machines only on a DHCP network.
> >         Kevin
>
> Lisa,
>
> That was not what I got out of the OP's original message at all.  I read
> that he is trying to poll Samba to see which Windows based laptops are
> connected, then trying to do a rsync to back up those laptops.
>
> Seems like a lot of work, and a lot of maintenance for something that is
> already built.  It just takes reversing your logic a little.
>
> His original idea would cause all backups to occur at a fixed time
> (since it was created from the same bash script), and rsync the Windows
> boxes connected at that time.  This solution has a few huge holes that
> are not addressed:
>
> All machines are backed up at once, adding a lot of stress on the
> network within a short time period.
>
> Second, What if Laptop A does not check in at the given time for 6
> months?  How would he know?
>
> Third, this only keeps a single backup of a file, not a revisioned
> version of the file.
>
> And Fourth, These are windows users, how do they initiate a restore of a
> file?
>
> The solution I proposed, uses polling and last backed markers to manage
> backups.  If a machine has not been backed up in one week, a notice can
> be sent to the admin notifying them of this situation.  It also handles
> revisioned backups... If a file changes, then changes again, that first
> change is not lost until the backup system rotates files off.  I used to
> use this product specifically in a Windows desktop (Linux Server)
> environment, with daily incremental backup, full backup every 30 days,
> three backup archive.  So, users had 90 days of backups at their
> disposal.  And finally, users initiated their own restores. Including if
> needed full system restores.
>
> For backup of laptops, just increase the polling time to improve the
> chance of catching them online.  It will only back them up once per
> configured period.  So, if you say back up daily, and it polls 10 times
> per day, it will not matter that the machine is there all 10 periods or
> only one, it will still only back up once.
>
> And this solution includes a throttling option that will prevent your
> network from getting saturated.
>
> The OP never mentioned anything about Windows Shares that I saw.  He
> mentioned trying to use Samba for machine detection to then initiate an
> rsync for backup.  Unless there is some underlying reason that I did not
> understand, there are free, FOSS solutions out there that will do a far
> superior job to what he was trying to do.
>
> That is why I was asking what I was asking... What situation are you
> trying to solve that existing software has not already solved.
>
> Kevin
>
> ---------------------------------------------------
> PLUG-discuss mailing list - PLUG-discuss at lists.plug.phoenix.az.us
> To subscribe, unsubscribe, or to change your mail settings:
> http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss
>



-- 
James
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.PLUG.phoenix.az.us/pipermail/plug-discuss/attachments/20120106/3d8c73ef/attachment.html>