I'm struggling with the performance of BackupPC_nightly.

The system is FreeBSD 10.0 running BackupPC on a zfs pool with 109Tb of 
space across 30 disks. The system has 128Gb of ram and dedicated ssds 
for zfs intent log and L2ARC. Performance is generally good, as expected 
for this hardware.

 From BackupPC cgi:

There are 53 hosts that have been backed up, for a total of:

368 full backups of total size 111871.28GB (prior to pooling and 
compression),
265 incr backups of total size 3520.74GB (prior to pooling and compression).

This is consuming about 19.4 Tb on the zfs pool.

What I see is that BackupPC_nightly runs sometimes for hours or days and 
with 100% CPU consumption. A simple "find pool/ -links 1" also runs a 
long time, but top shows it spending nearly all it's time in zio (disk) 
routines, not in cpu. I would expect walking the backuppc pool to 
involve mostly disk io with relatively little cpu, which makes me 
question BackupPC_nightly spending it's time in cpu.

I currently have:
MaxBackupPCNightlyJobs = 4
BackupPCNightlyPeriod = 8

I wonder if there is something about having such a large pool that's 
causing BackupPC_nightly to get stuck at 100% cpu? Maybe the number of 
files in it's working batch is too big?

Thanks
Russ Poyner

------------------------------------------------------------------------------
Dive into the World of Parallel Programming. The Go Parallel Website,
sponsored by Intel and developed in partnership with Slashdot Media, is your
hub for all things parallel software development, from weekly thought
leadership blogs to news, videos, case studies, tutorials and more. Take a
look and join the conversation now. http://goparallel.sourceforge.net/
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/