BackupPC is being surprisingly slow even for incrementals.  This appears
to be tied to the rsync backend in some fashion.

Here's an example.  It took 5 *HOURS* to run an incremental over a 
machine in which the total incremental size was 2383 files of 616MB. 
This is fairly typical.

Examining processes with strace and lsof, it appears that the BackupPC 
unnecessarily is opening every single file from the existing backup sets 
on the server -- even if they hadn't changed a bit.

I am using cpool with rsync checksum caching -- the latter turned on 
hoping it would help, but it didn't.

The most recent full backup of this system took 56 *HOURS*, though that
may be before checksum caching had a chance to kick in.  The first full 
backup of it took only 17 hours, and I didn't do anything like copy tons 
of new stuff to it or anything.  That most recent full backup had 787941 
files which was 444GB.

 From what I have seen to date, tar doesn't have this problem.  However, 
due to the limitations of the tar backups documented at 
http://bit.ly/fbiCyh I really wish to avoid using tar for backups 
wherever I can.

The system being backed up in this example is a Core 2 Duo running 
Debian squeeze, 8GB ram.  The BackupPC server is a dual-core Pentium 
E6500 2.93GHz. The backup disk is a 1TB USB drive, ext4, running through 
LUKS encryption since it goes offsite.  The USB bit seems to contribute 
more to slowness than the LUKS bit on this system.  But the end effect 
is that if BackupPC is being non-smart about how it accesses the disk on 
the server side, this will be magnified by the USB interface.

What can be done to fix this?  Combined with the problem of very slow 
performance for large files with rsync [1] (which did NOT refer to a 
problem with slow disks), I am starting to doubt whether the rsync 
backend is really useable yet.

Thanks,

-- John

[1] http://bit.ly/hCJ7Gj

------------------------------------------------------------------------------
Free Software Download: Index, Search & Analyze Logs and other IT data in 
Real-Time with Splunk. Collect, index and harness all the fast moving IT data 
generated by your applications, servers and devices whether physical, virtual
or in the cloud. Deliver compliance at lower cost and gain new business 
insights. http://p.sf.net/sfu/splunk-dev2dev 
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/