On Wed, 9 Apr 2008, Les Mikesell wrote:

>> I've been using BackupPC for several years now, but one problem that I've 
>> never come up with a good answer for is when a single large file is too big 
>> to transfer completely in the time the backup can run before timing out. 
>> For example, a 10M local datafile, backing up over a 768k upstream DSL, 
>> ends up stalling the backup because it can never get past that file.
>
> Did you mean 10 gigs?  10M doesn't sound like it should be a problem.

Hehe...yes, meg, gig, they all start to blur together.

>> Does anyone have a workaround or fix for this?  Is it possible to change 
>> BackupPC so it doesn't remove the in-progress file, but instead copies it 
>> into the pool so rsync will pick up where it left off last time?  There 
>> doesn't seem to be any downside to leaving off on the transfer where it 
>> was.
>
> I usually try to get a full over a weekend where it doesn't matter if things 
> run into the next day.  Things shouldn't just stop unless you have a network 
> problem.

Unfortunately, since we don't necessarily control the client endpoint, that 
this 
is an issue.  Even when we try to run for a full day, you just can't transfer 
that many bytes over a slow remote upstream.

>> There is one other problem related to this (and big backups in general) - 
>> sometimes, there is enough delay that one of the rsync connections will 
>> timeout due to the firewall not seeing activity on the socket.  Is there a 
>> way to force some sort of traffic (a bogus keepalive) on the which socket 
>> is idle to make sure it doesn't get prematurely severed?
>> 
>> Any advice would be appreciated.
>
> One approach is to use a VPN connection to the remote site.  Openvpn has an 
> option to do keepalives on the line - and to do lzo compression on the data.

These files are often compressed already - for example, one client has very 
large MP3 files that contain multi-hour recorded conferences.  Being able to 
restart midstream would be a huge help.

David.

-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference 
Don't miss this year's exciting event. There's still time to save $100. 
Use priority code J8TL2D2. 
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/