BackupPC-users

Re: [BackupPC-users] rsyncd full backup

2013-10-30 22:09:41
Subject: Re: [BackupPC-users] rsyncd full backup
From: Sharuzzaman Ahmat Raslan <sharuzzaman AT gmail DOT com>
To: "General list for user discussion, questions and support" <backuppc-users AT lists.sourceforge DOT net>
Date: Thu, 31 Oct 2013 10:06:18 +0800
Hi Holger,

Based on short session of troubleshooting, I believe the machine actually suffer from low I/O speed to the disk. Average read is about 3 MB/s, which I considered slow for a SATA disk in IDE emulation.

I'm planning to suggest to the customer to have a RAID 1 setup to increase the I/O speed. I'm looking at possibilities to speed things up by not having to change the overall setup.

Thank you for providing new insights to me regarding rsync. Glad to learn new things :)

Thanks.


On Thu, Oct 31, 2013 at 5:15 AM, Holger Parplies <wbppc AT parplies DOT de> wrote:
Hi,

Adam Goryachev wrote on 2013-10-29 15:29:42 +1100 [Re: [BackupPC-users] rsyncd full backup]:
> On 29/10/13 15:14, Sharuzzaman Ahmat Raslan wrote:
> > [...]
> >On Tue, Oct 29, 2013 at 11:33 AM, Les Mikesell <lesmikesell AT gmail DOT com
> ><mailto:lesmikesell AT gmail DOT com>> wrote:
> >    On Mon, Oct 28, 2013 at 10:08 PM, Sharuzzaman Ahmat Raslan
> >    <sharuzzaman AT gmail DOT com <mailto:sharuzzaman AT gmail DOT com>> wrote:
> >    > [...]
> >    > Initially, the backup transport is SMB, but recently, I noticed
> >    > a lot of machine backup (full and incremental) is not able to
> >    > complete in 8 hours, due to large number of file, and big file size.
> >    >
> >    > Last week, I installed DeltaCopy (rsycnd server for Windows) on
> >    > one machine, and change the backup transport to rysncd. The backup
> >    > runs well.
> >    >
> >    > But today, I noticed, when BackupPC is running a full backup on
> >    > the machine that have rsyncd, it still takes 8 hours to do full
> >    > backup. [...]
> >    Rsync will only transfer the changed data, but in full runs the
> >    contents of the files are read at both ends and compared with block
> >    checksums, so it takes some time. [...]
> >
> >In essence, if I enable
> >|--checksum-seed=32761
> >
> >|
> >then the rsync full backup will be faster?
>
> Yes, the third full backup after you enable that option will be faster
> *IF* the slow speed is due to the backup server needing to decompress
> the file and check the content.

let me stress that again: don't expect a speedup on the *first* full backup
after you enable that option. In my limited opinion (I haven't compared speeds
because I don't have any issues with slow backups), the *second* full backup
should be faster, as you have pre-existing full backups, i.e. the next full
can add the checksums. In any case, the *third* full backup should hopefully
be faster :-).

> In the case that your backup client has really slow disk, then there is
> nothing you can do, except maybe modify backuppc for full backups to not
> send the ignore-times option to rsync (ie, every backup is an
> incremental). Or, of course, upgrade the client to improve performance.

Actually, it is worth noting that aside from a possible speed improvement the
switch from smb to rsync(d) gives you far more precise *incremental* backups,
so it might be an option to increase FullPeriod. This may transfer more data
(because the delta is always relative to the reference backup - normally the
previous full backup - and not to the previous incremental backup), but you
can always explore the IncrLevels setting. So, while you might not speed up
the full runs, you might get away with doing them less often. I would not
recommend patching the ignore-times option away altogether.

But Adams point is correct: you need to find out where the problem is, before
you can fix it. While you might be able to find the problem by trying out
fixes, that might not be the most efficient way :-).

Regards,
Holger

------------------------------------------------------------------------------
Android is increasing in popularity, but the open development platform that
developers love is also attractive to malware creators. Download this white
paper to learn more about secure code signing practices that can help keep
Android apps secure.
http://pubads.g.doubleclick.net/gampad/clk?id=65839951&iu=/4140/ostg.clktrk
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/



--
Sharuzzaman Ahmat Raslan
------------------------------------------------------------------------------
Android is increasing in popularity, but the open development platform that
developers love is also attractive to malware creators. Download this white
paper to learn more about secure code signing practices that can help keep
Android apps secure.
http://pubads.g.doubleclick.net/gampad/clk?id=65839951&iu=/4140/ostg.clktrk
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/