BackupPC-users

Re: [BackupPC-users] rsync clients run out of memory

2009-08-14 12:11:15
Subject: Re: [BackupPC-users] rsync clients run out of memory
From: Richard Hansen <rhansen AT bbn DOT com>
To: "General list for user discussion, questions and support" <backuppc-users AT lists.sourceforge DOT net>
Date: Fri, 14 Aug 2009 12:07:01 -0400
>> I'm using BackupPC 3.1.0 and I'm having problems with rsync clients 
>> running out of memory and crashing (the Linux kernel's OOM-Killer is 
>> unleashed, wreaking all sorts of havoc).  Apparently this is a known 
>> problem if a lot of files are being synced and one or both ends of the 
>> rsync transfer is older than rsync 3.0 (see 
>> <http://www.samba.org/rsync/FAQ.html#5>).
>> 
>> The client machines have rsync 3.0, but it looks like BackupPC uses an 
>> older version.  I just switched to tar, so we'll see if that makes the 
>> problem go away.
>> 
>> I'd prefer to use rsync.  Is there any chance BackupPC could use a newer 
>> version of File::RsyncP that supports rsync protocol version 30?
> 
> This has been discussed many times on the list - please see the 
> archives.

I'd appreciate any pointers to previous discussions.  SourceForge's 
search is inadequate, and an hour spent at gmane.org did not turn up 
anything of interest.

> Bottom line is that the memory issue seems to be solved in rsync 3.0 
> even though protocol < 30 is used.

Apparently not -- both of my clients already have rsync 3.0.5 installed, 
yet rsync is causing them to run out of memory.  The clients have 4 to 6 
million (largely redundant) files each.  It appears that I need the 
incremental-recursion feature of protocol 30 to back up this many files.

> Upgrading to version 30 is a *big* 
> deal and would likely require significant rewrite of 
> perl-File-RsyncP.

That's unfortunate.  If I can't find an existing backup program that 
suits my needs, I'll try to find the time to update it.

Thanks for the response,
Richard

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day 
trial. Simplify your report design, integration and deployment - and focus on 
what you do best, core application coding. Discover what's new with 
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/