Re: rsync

Top Page
Attachments:
Message as email
+ (text/plain)
+ (text/html)
+ (text/plain)
Delete this message
Reply to this message
Author: Michael Havens
Date:  
To: Main PLUG discussion list
Subject: Re: rsync
by compress I thought they meant zip. so there is no way to zip it?

On Wed, Sep 23, 2015 at 1:36 PM, Brian Cluff <> wrote:

> The --compress-level=5 option is for compressing the data over a network.
> I would hope that rsync is smart enough to not actually compress the data
> internally when the data is staying local to the machine, but if it's not,
> telling it to compress will just eat up a bunch of your CPU to accomplish
> nothing.
>
> rsync does indeed have an exclude option that looks like:
>
> --exclude 'string*'
>
> If you want the data compressed but still want to use rsync on it, you
> will probably need to look into using a filesystem that will automatically
> compress your data. Then you can just use the same command to backup your
> data and the compression will be done transparently.
>
> Try looking into formatting your drive with something like the BTRFS
> filesystem that supports transparent compression.
>
> Brian Cluff
>
> On 09/23/2015 06:49 AM, Michael Havens wrote:
>
>> It seems that the compress doesn't work.
>>
>> size of Documents folder:
>> bmike1@c521 ~ $ du -hs Documents/
>> which lists the size of 19G
>>
>> bmike1@c521 /media/bmike1/USB DISK $ du -hs ./
>> 20G./
>>
>> --compress-level=5 made it bigger!
>>
>> Or else do I also need to include '-z' as well?
>>
>> On Wed, Sep 23, 2015 at 9:29 AM, Michael Havens <
>> <mailto:bmike1@gmail.com>> wrote:
>>
>>     got it to backup. But still, please answer my question about the
>>     'exclude' option and how much I can compress it. I got it to backup
>>     by erasing the .Trash folder. Why don't pendrives make the space
>>     available after you delete a file?

>>
>>     rsync -auWq --progress --delete --compress-level=5
>>     /home/bmike1/Documents /media/bmike1/"USB DISK"/
>>     rsync: rename "/media/bmike1/USB DISK/Documents/.forwardspamto
>>       .elhlC4" -> "Documents/forwardspamto    ": Invalid argument (22)
>>     rsync: mkstemp "/media/bmike1/USB

>>
>> DISK/Documents/Business/Receipts/lazycash/.index.html?a_aid=509c458b25b79.sGaqe4"
>>     failed: Invalid argument (22)

>>
>>     rsync: mkstemp "/media/bmike1/USB
>>     DISK/Documents/ComputerInstructions/.ForgotPassword?.xVeqCC" failed:
>>     Invalid argument (22)
>>     rsync: mkstemp "/media/bmike1/USB
>>     DISK/Documents/Education/.Screenshot-LFS101x.2 Progress | edX -
>>     Google Chrome.png.eyN1PK" failed: Invalid argument (22)
>>     rsync error: some files/attrs were not transferred (see previous
>>     errors) (code 23) at main.c(1183) [sender=3.1.0]

>>
>>
>>     On Wed, Sep 23, 2015 at 8:07 AM, Michael Havens <
>>     <mailto:bmike1@gmail.com>> wrote:

>>
>>         I keep running out of space on the flash drive. How tightly can
>>         I compress it?
>>         Could I get it to not copy files like so: --exclude=string* so
>>         that it would exclude all files string<and whatever else comes
>>         after string>
>>         ????????????????????????

>>
>>         On Mon, Sep 21, 2015 at 12:48 PM, Michael Havens
>>         < <mailto:bmike1@gmail.com>> wrote:

>>
>>             thanks!

>>
>>             On Mon, Sep 21, 2015 at 12:40 PM, Brian Cluff
>>             < <mailto:brian@snaptek.com>> wrote:

>>
>>                 Use --delete if you want the destination to have any
>>                 files that have been deleted from the source to also
>>                 deleted in the destination.

>>
>>                 The -q option just suppresses any output that isn't an
>>                 error, I tend to leave it off do that I can see what
>>                 file it's currently working on.  You can add or remove
>>                 it as needed.

>>
>>                 Brian Cluff

>>
>>                 On 09/21/2015 03:42 AM, Michael Havens wrote:

>>
>>                     what about the -q option and the --delete option? i
>>                     noticed that you
>>                     didn't use them in your command 'rsync -auW
>>                     /sort/directory
>>                     /dest/directory/'.

>>
>>                     On Sun, Sep 20, 2015 at 10:34 PM, Brian Cluff
>>                     < <mailto:brian@snaptek.com>
>>                     <mailto:brian@snaptek.com

>>
>>                     <mailto:brian@snaptek.com>>> wrote:

>>
>>                          If you are backing up locally you will want to
>>                     do things a little
>>                          different size as:

>>
>>                          rsync -auW  /sort/directory /dest/directory/

>>
>>                          You will want to skip the -z option and the
>>                     corresponding
>>                          --compress-level option.  Since you are doing
>>                     copying everything
>>                          locally that will only cause the machine to
>>                     compress and immediately
>>                          decompress every file that is copied wasting a
>>                     ton of CPU/power.

>>
>>                          The other thing you will want to do is use the
>>                     -W flag, that tells
>>                          the machine to copy whole files instead of
>>                     looking for what has
>>                          changed between the documents.  That way it can
>>                     look at the time
>>                          and/or size and if it's changed it will just
>>                     copy the whole file.
>>                          Without that flag it would read through both
>>                     the source and
>>                          destination file and then just copy the
>>                     differences by writing a
>>                          whole new file, so with the -W (whole file)
>>                     flag the machine just
>>                          reads/writes the file once and is a lot more
>>                     efficient/faster.

>>
>>                          This can also be a good flag to set on fast
>>                     networks since it can be
>>                          a lot faster just to re-copy the whole file
>>                     than it is to have the
>>                          hard drive reading the file multiple times.

>>
>>                          The progress flag is very nice, but unless you
>>                     are planning on
>>                          closely monitoring your copy, I would skip it
>>                     as I've found that it
>>                          tends to slow down the transfer... or at least
>>                     make it feel that
>>                          way, like a watched pot never boils :)

>>
>>                          On your slash at end end question.  A slash at
>>                     the end tends to mean
>>                          that you want to put the source files/dirs into
>>                     that directory and a
>>                          destination without a slash usually means that
>>                     you want to rename
>>                          your source file/directory to that destination
>>                     file/dir name.

>>
>>                          Lastly the -h option gives you the sizes in
>>                     easily readable terms or
>>                          in other words, instead of just giving you the
>>                     size in bytes it will
>>                          give you size that look like 100K 2.4M 1.8G

>>
>>                          Brian Cluff

>>
>>
>>                          On 09/20/2015 05:34 AM, Michael Havens wrote:

>>
>>                              I know how to use rsync (sorta) to backup a
>>                     disk and go between
>>                              remote
>>                              machines. With my current incarnation of
>>                     the os I'm not so
>>                              worried about
>>                              backing up the whole system as I am a
>>                     directory (and all of the
>>                              directories under it) nor copying between
>>                     remote machines. The
>>                              directory
>>                              is the 'Documents' directory. Would I:

>>
>>                              rsync -aquz --compress-level=5
>>                     /home/bmike1/Documents
>>                              /media/bmike1/USB
>>                              DISK/
>>                              --
>>                              :-)~MIKE~(-:

>>
>>
>>
>>                     ---------------------------------------------------
>>                              PLUG-discuss mailing list -
>>                     
>>                     <mailto:PLUG-discuss@lists.phxlinux.org>
>>                              <mailto:PLUG-discuss@lists.phxlinux.org
>>                     <mailto:PLUG-discuss@lists.phxlinux.org>>
>>                              To subscribe, unsubscribe, or to change
>>                     your mail settings:

>>
>> http://lists.phxlinux.org/mailman/listinfo/plug-discuss
>>
>>
>>  ---------------------------------------------------
>>                          PLUG-discuss mailing list -
>>                     
>>                     <mailto:PLUG-discuss@lists.phxlinux.org>
>>                          <mailto:PLUG-discuss@lists.phxlinux.org
>>                     <mailto:PLUG-discuss@lists.phxlinux.org>>
>>                          To subscribe, unsubscribe, or to change your
>>                     mail settings:

>>
>> http://lists.phxlinux.org/mailman/listinfo/plug-discuss
>>
>>
>>
>>
>>                     --
>>                     :-)~MIKE~(-:

>>
>>
>>                     ---------------------------------------------------
>>                     PLUG-discuss mailing list -
>>                     
>>                     <mailto:PLUG-discuss@lists.phxlinux.org>
>>                     To subscribe, unsubscribe, or to change your mail
>>                     settings:

>>
>> http://lists.phxlinux.org/mailman/listinfo/plug-discuss
>>
>>
>>                 ---------------------------------------------------
>>                 PLUG-discuss mailing list -
>>                 
>>                 <mailto:PLUG-discuss@lists.phxlinux.org>
>>                 To subscribe, unsubscribe, or to change your mail
>> settings:
>>                 http://lists.phxlinux.org/mailman/listinfo/plug-discuss

>>
>>
>>
>>
>>             --
>>             :-)~MIKE~(-:

>>
>>
>>
>>
>>         --
>>         :-)~MIKE~(-:

>>
>>
>>
>>
>>     --
>>     :-)~MIKE~(-:

>>
>>
>>
>>
>> --
>> :-)~MIKE~(-:
>>
>>
>> ---------------------------------------------------
>> PLUG-discuss mailing list -
>> To subscribe, unsubscribe, or to change your mail settings:
>> http://lists.phxlinux.org/mailman/listinfo/plug-discuss
>>
>> ---------------------------------------------------
> PLUG-discuss mailing list -
> To subscribe, unsubscribe, or to change your mail settings:
> http://lists.phxlinux.org/mailman/listinfo/plug-discuss
>




--
:-)~MIKE~(-:
---------------------------------------------------
PLUG-discuss mailing list -
To subscribe, unsubscribe, or to change your mail settings:
http://lists.phxlinux.org/mailman/listinfo/plug-discuss