BackupPC-users

[BackupPC-users] rsync: full backup more than twice faster than incremental backup

2012-08-16 05:04:11
Subject: [BackupPC-users] rsync: full backup more than twice faster than incremental backup
From: Udo Rader <listudo AT bestsolution DOT at>
To: backuppc-users AT lists.sourceforge DOT net
Date: Thu, 16 Aug 2012 10:47:02 +0200
Hi,

after adding one of our customers' server we see some weird backup
effects with a full backup taking (somewhat expected) 7 hours but
incremental backups taking more than twice that time.

See for yourself:

2012-08-10 01:30:01 full backup started for directory /backup_tree
2012-08-10 08:35:12 full backup 0 complete, 2599979 files, 25649558375
bytes, 0 xferErrs (0 bad files, 0 bad shares, 0 other)
2012-08-11 01:31:22 incr backup started back to 2012-08-10 01:30:01
(backup #0) for directory /backup_tree
2012-08-11 18:05:25 incr backup 1 complete, 11229 files, 464722370
bytes, 0 xferErrs (0 bad files, 0 bad shares, 0 other)
2012-08-12 01:32:23 incr backup started back to 2012-08-10 01:30:01
(backup #0) for directory /backup_tree
2012-08-12 19:33:58 incr backup 2 complete, 16858 files, 351340522
bytes, 0 xferErrs (0 bad files, 0 bad shares, 0 other)

Backups are done using ordinary rsync over ssh, with no "special"
settings except the nice'ing:

$Conf{RsyncClientCmd} = '$sshPath -q -x -l root $host nice -n 9
$rsyncPath $argList+';

One of the reasons I can think of is the file structure on that host. It
serves as a special storage pool for a customer developed application
and as such it has really really many subdirectories with really really
many subdirectories with really really many subdirectories. And by
really really many I mean really really many ... So my best guess would
be that building the file list diff takes much longer than just fetching
the files as they exist.

I've now disabled incremental backups on this server, but maybe someone
has an idea how to enable incremental backups for this host as well.

thanks in advance!

-- 
Udo Rader, CTO
BestSolution.at EDV Systemhaus GmbH
Eduard-Bodem-Gasse 5-7, A-6020 Innsbruck
http://www.bestsolution.at/
Reg. Nr. FN 222302s am Firmenbuchgericht Innsbruck

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/