On Jul 8, 2008, at 2:55 PM, Evren Yurtesen wrote:
> Tony Schreiner wrote:
>> On Jul 3, 2008, at 4:07 PM, Evren Yurtesen wrote:
>>> Renke Brausse wrote:
>>>> Hello Tony,
>>>>> I've written before about backups involving very big files
>>>>> that seem to execute slowly.
>>>>>
>>>>> What can be slowing things down so much? Except for this
>>>>> operation, everything else runs about as I would expect.
>>>> I have no clue what the reason is but I experienced that backups of
>>>> large files are much faster with tar over ssh instead of rsync
>>>> over ssh.
>>>> Not an explanation but maybe this can solve your problem.
>>> I believe the reason for this is how rsync works. It normally
>>> tries to transfer only the changed parts of the file. This is to
>>> save bandwidth, to do this, it has to scan the whole file on
>>> both sides (I guess). This is unnecessary unless you are over
>>> slow links. You might want to try the whole-file option with rsync:
>>>
>>> -W, --whole-file copy files whole (w/o delta-
>>> xfer algorithm)
>>>
>>> Please let us know the results, as a side-note if you still want
>>> to shrink the transferred file size you can use the ssh
>>> compression with -C option of ssh.
>>>
>>> Thanks,
>>> Evren
>> Reporting back on this. Using the -W option did not make much
>> difference. The dumps in question continue to run for a long time
>> and eventually fail with an ALARM.
>> For the time being I am excluding the really large files in question.
>> I may try tar instead of rsync at some point as was suggested.
>> Tony Schreiner
>
> Are you sure that -W option was active for sure? Did you check with
> ps axwww to be sure? or just took it granted that it was active?
>
> Just asking because I have made several times similar mistake
> myself :)
>
> By the way, if you are using compression, also enabling --checksum-
> seed=32761 will make a dramatic speed difference. (although manual
> suggests that this can be visible only on 3rd full backup).
>
> Thanks,
> Evren
Yes, unfortunately, I looked with ps and the --whole-file option was
set. I also have --checksum--seed=32761 set.
I'm willing to try again, in case I was hallucinating.
Tony
-------------------------------------------------------------------------
Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW!
Studies have shown that voting for your favorite open source project,
along with a healthy diet, reduces your potential for chronic lameness
and boredom. Vote Now at http://www.sourceforge.net/community/cca08
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
|
