BackupPC-users

Re: [BackupPC-users] Using NFS to increase backup speed

2014-10-17 06:40:22
Subject: Re: [BackupPC-users] Using NFS to increase backup speed
From: Sorin Srbu <Sorin.Srbu AT orgfarm.uu DOT se>
To: "General list for user discussion, questions and support" <backuppc-users AT lists.sourceforge DOT net>
Date: Fri, 17 Oct 2014 10:38:43 +0000
> -----Original Message-----
> From: Les Mikesell [mailto:lesmikesell AT gmail DOT com]
> Sent: den 16 oktober 2014 18:09
> To: General list for user discussion, questions and support
> Subject: Re: [BackupPC-users] Using NFS to increase backup speed
> 
> > I'm seeing network speeds at about 35-45 Mbps when using BackupPC and rsync 
> > over ssh.
> >
> 
> That sounds pretty good.  But unless you have a lot of new files
> created daily, the bottleneck is usually disk speed, especially
> merging a lot of small changes into a big existing file.

Not too many new files daily, the reading is done from a three- or four-disk 
raid0-array, so should be fairly fast I guess.

The 35-45 Mbps mentioned above is a pretty rought figure I think. I took it 
from the netspeed Gnome applet. The numbers I posted just earlier are probably 
more exact(ish).

But you think this speed is what you'd expect from a setup with a gigabit 
switch, gigabit-NICs on both ends and CPU:s (BPC: single-core Athlon64 3500+, 
host: Intel core 2 Quad Q8200@2,33 GHx) a few years old then?


> > Running iperf below I notice the network should theoretically be capable of
> > a bit more than that. I understand that ssh adds quite a bit of bottleneck.
> 
> There is some overhead for encryption.  Back when cpus were slow
> enough for it to matter I used to set blowfish as the preferred
> cipher.  Now you probably want aes-128 where you have hardware
> support.

Would aes-128 be faster than arcfour, roughly speaking?

I'll need to give aes-128 a go as well it seems.


> > After reading one of the answers at
> <<http://serverfault.com/questions/377598/why-is-my-rsync-so-slow>>, I
> kinda' wonder if the NFS-angle would work for use with BackupPC and how I
> would go about it. Is the tar transfer method the only one supported by BPC
> if using NFS for example?
> > Rsyncing a NFS-mounted remote host supposedly is a lot faster than doing
> rsync over ssh.
> >
> > Any thoughts on this?
> 
> I'd guess there are too many variables to predict but I'd stick to ssh
> because of the forced validation (--ignore-times) on full runs.  In
> the case that will ultimately matter the most (the third and
> subsequent full runs)  you can avoid most of the server-side reads if
> you used checksum caching but you still have to perform a full read of
> the content on the client side.  With ssh, that's a local disk access.
> with nfs you'll copy every file across the network just so rsync can
> compute its block checksum even where everything already matches.

Some of the articles mentioned this, but didn't go too much into depth. 
Your clarification cleared that up. Thanks!


--
//Sorin


------------------------------------------------------------------------------
Comprehensive Server Monitoring with Site24x7.
Monitor 10 servers for $9/Month.
Get alerted through email, SMS, voice calls or mobile push notifications.
Take corrective actions from your mobile device.
http://p.sf.net/sfu/Zoho
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/