BackupPC-users

Re: [BackupPC-users] backuppc slow rsync speeds

2012-09-17 08:52:56
Subject: Re: [BackupPC-users] backuppc slow rsync speeds
From: Tim Fletcher <tim AT night-shade.org DOT uk>
To: "General list for user discussion, questions and support" <backuppc-users AT lists.sourceforge DOT net>
Date: Mon, 17 Sep 2012 15:50:39 +0300
You are being hit by disk io speeds, check you dont have atime turned on on the 
fs. Also it's worth considering tar instead of rsync for this sort of work 
load. 

--

Sent from a mobile device

Tim Fletcher

On 17 Sep 2012, at 10:08, Mark Coetser <mark AT tux-edo.co DOT za> wrote:

> Hi
> 
> backuppc                                       3.1.0-9.1
> rsync                                          3.0.7-2
> 
> OK I have a fairly decent spec backup server with 2 gigabit e1000 nics 
> bonned together and running in bond mode 0 all working 100%. If I run 
> plain rsync between the backup server and a backup client both connected 
> on gigabit lan I can get sync speeds of +/- 300mbit/s but using backuppc 
> and rsync the max speed I get is 20mbit and the backup is taking 
> forever. Currently I have a full backup thats been running for 3461:23 
> minutes where as the normal rsync would have taken a few hours to complete.
> 
> The data is users maildirs and its about 2.6Tb and I am not using rsync 
> over ssh, I have the rsync daemon running on the client and have setup 
> the .pl as follows.
> 
> config
> 
> #
> $Conf{ClientTimeout} = 28800;
> 
> # Minimum period in days between full and incremental backups:
> $Conf{FullPeriod} = 6.97;
> $Conf{IncrPeriod} = 0.97;
> 
> # Number of full and incremental backups to keep:
> $Conf{FullKeepCnt} = 2;
> $Conf{IncrKeepCnt} = 10;
> # Note that additional fulls will be kept for as long as is necessary
> # to support remaining incrementals.
> 
> #$Conf{DumpPreUserCmd} = 'sudo /bin/mount -t nfs ns1:/var/mail /var/mail';
> #$Conf{DumpPostUserCmd} = 'sudo /bin/umount /mnt/mail';
> 
> # What transport to use backup the client [smb|rsync|rsyncd|tar|archive]:
> $Conf{XferMethod} = 'rsyncd';
> 
> # The file system path or the name of the rsyncd module to backup when
> # using rsync/rsyncd:
> $Conf{RsyncShareName} = ['backuppc'];
> 
> $Conf{RsyncdAuthRequired} = 0;
> 
> $Conf{RsyncdUserName} = 'xxxxxxxx';
> $Conf{RsyncdPasswd} = 'xxxxxxxx';
> 
> # If this is defined only these files/paths will be included in the backup:
> $Conf{BackupFilesOnly} = undef;
> 
> # These files/paths will be excluded from the backup:
> $Conf{BackupFilesExclude} = [
>                 '/DONOTDELETE',
>                 '/lost+found'
> ];
> 
> # Level of verbosity in Xfer log files:
> $Conf{XferLogLevel} = 1;
> 
> # Commands to run for client backups:
> # Note the use of SSH's -C attribute. This enables compression in SSH.
> $Conf{RsyncClientCmd} = '$rsyncPath $argList+';
> 
> # Commands to run for client direct restores:
> # Note the use of SSH's -C attribute. This enables compression in SSH.
> $Conf{RsyncClientRestoreCmd} = '$rsyncPath $argList+';
> 
> # Compression level to use on files. 0 means no compression. See notes
> # in main config file before changing after backups have already been done.
> $Conf{CompressLevel} = 3;
> 
> 
> 
> -- 
> Thank you,
> 
> Mark Adrian Coetser
> 
> 
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and 
> threat landscape has changed and how IT managers can respond. Discussions 
> will include endpoint security, mobile security and the latest in malware 
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> BackupPC-users mailing list
> BackupPC-users AT lists.sourceforge DOT net
> List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
> Wiki:    http://backuppc.wiki.sourceforge.net
> Project: http://backuppc.sourceforge.net/
> 

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/