BackupPC-users

Re: [BackupPC-users] rsync out of memory

2014-11-20 15:48:05
Subject: Re: [BackupPC-users] rsync out of memory
From: Christian Völker <chrischan AT knebb DOT de>
To: backuppc-users AT lists.sourceforge DOT net
Date: Thu, 20 Nov 2014 21:46:41 +0100
Hi Les,


> Native rsync 3.x versions can use protocol 30 which can handle the
> directory tree incrementally. Backuppc's version uses protocol 28 (and
> forces the remote to use that also) which must xfer the entire tree
> first and hold it in RAM at both ends while they walk it doing the
> comparisions. 

Is there a way to force protocol version 30 instead? Obviously with v28
the "--no-incremental-recursive" does not have any effect.
> I'm not actually sure from your error message which end was out of
> memory.  How do things look on the target system while the backuppc
> backup runs?
The host to be backed up seems to run out of memory. Has 4GB and while
rsync runs the number of "page cache" increases so just around 100M
remain as free memory. Though, rsync does not use more than 112M.
Swapping apears but I was not able to catch the exact moment when the
oom appears.

> Memory use should relate to the number of files more than the total
> size.  Do any directories have a huge number of tiny files?
>
Not really. See the output of "find DIR -type f | wc -l" for various dirs:
bin: 86
dev: 43
lib: 2999
lib64: 281
media/: 0
opt/: 0
root/: 21
selinux/: 0
store/: 0
tmp/: 1
boot/: 22
etc/: 1222
mnt/: 37
proc/: 37101
sbin/: 190
srv/: 44
sys/: 6831
usr/: 54306
var: 7176

The huge directory is not to be backed up at this stage because it is
mounted and rsync got the "-x" option. But for completenes:
home: 57949

Any further ideas?

Greetings

Christian



------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

<Prev in Thread] Current Thread [Next in Thread>