BackupPC-users

Re: [BackupPC-users] Using NFS to increase backup speed

2014-10-17 07:27:29
Subject: Re: [BackupPC-users] Using NFS to increase backup speed
From: Holger Parplies <wbppc AT parplies DOT de>
To: Sorin Srbu <Sorin.Srbu AT orgfarm.uu DOT se>
Date: Fri, 17 Oct 2014 13:25:10 +0200
Hi,

Sorin Srbu wrote on 2014-10-16 12:25:53 +0000 [[BackupPC-users] Using NFS to 
increase backup speed]:
> I'm seeing network speeds at about 35-45 Mbps when using BackupPC and rsync
> over ssh.

it is a frequent misconception that you *want* to see anything close to the
speed the network is capable of delivering here. The whole point of the rsync
protocol is to cut down network usage by transferring only data the remote end
does not yet have.

In the context of BackupPC, you may be using rsync for other reasons, but that
does not change how it works. Ideally, your full backups will transfer only a
small part of your data set over the network, while at least the client
machine will still need to read all of it from disk. On a fast network,
network speed will *not* be the bottleneck. Consequentially, you will not be
utilising (almost) all of it. The data rate becomes more of a measure for how
efficient the rsync algorithm is, with lower data rate meaning more
efficiency.

If you actually *are* transferring a lot of new data (e.g. initial backup),
you need to keep in mind that BackupPC will need to compress it (presuming you
are using compression).

Whenever this topic comes up, the question I tend to ask is: are you fixing a
real problem or are you merely trying to get figures that *seem* better to
you, because you are misreading them?

> [...]
> Is the tar transfer method the only one supported by BPC if using NFS for
> example?

The transfer method is totally independant of the location of the data. You
can use FTP to backup localhost (or something mounted there), if you feel so
inclined, or, even worse, smb. There is not much point to that, though.

As for rsync, there is a point, because it makes much more exact incremental
backups than tar does. I use

        $Conf{RsyncClientCmd} = 'sudo $rsyncPath $argList';

and it works fine, just as expected.

> Rsyncing a NFS-mounted remote host supposedly is a lot faster than doing
> rsync over ssh.

Les has a good point that this would read *all* data over the network vs. only
a small fraction. Well, that depends on your data set. If almost everything
changes every day, there won't be much difference.

> Has anybody on this list maybe set up their systems using NFS and can share
> their experience?

I *have* done that in the past, and it just worked. It didn't give me any
issues, but then, I didn't care how long the backups took as long as they were
"fast enough", which they were. You'll almost definitely need a no_root_squash
export, though, and you'll need to think about the error case where your target
file system is not mounted (e.g. modified PingCmd).

I doubt it's worth the effort both of setting it up and of maintaining it.
rsync over ssh is a well-supported standard case. It works well over VPNs,
through firewalls and across administrative domains. It would always be my
first choice of backup method. Unless you have a real *need* for speeding
up your backups, I don't see the point in experimenting.

Regards,
Holger

------------------------------------------------------------------------------
Comprehensive Server Monitoring with Site24x7.
Monitor 10 servers for $9/Month.
Get alerted through email, SMS, voice calls or mobile push notifications.
Take corrective actions from your mobile device.
http://p.sf.net/sfu/Zoho
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/