BackupPC-users

[BackupPC-users] Still trying to understand reason for extremely slow backup speed...

2011-03-09 15:06:47
Subject: [BackupPC-users] Still trying to understand reason for extremely slow backup speed...
From: "Jeffrey J. Kosowsky" <backuppc AT kosowsky DOT org>
To: General list for user discussion <backuppc-users AT lists.sourceforge DOT net>
Date: Wed, 09 Mar 2011 15:04:32 -0500
I run BackuppPC 3.2.0 under Fedora 12 on a P4 2.8GHz server with 2GB
of RAM and with storage mounted via NFS on a low-end NAS (DNS-323) on
a 100MB/sec LAN. Before making a number of changes to both server and
NAS (I know bad idea), I would get reasonable backup speeds where for
example an incremental would take 20 minutes and a full would take a
couple of hours. Now my incrementals take about 4 1/2 hours with an
effective transfer rate of 0.01 MB/sec while the fulls take almost 8
hours with an effective transfer of 0.74 MB/sec.

I use rsync as the transfer method with rsync checksum caching enabled
at 10% probability.

The server itself is minimally loaded. The NFS transfer speed is about
11 MB/sec on large files and about 4 MB/sec on small files.

I can't understand how incrementals can be so painfully slow with an
effective average transfer speed of just 0.01MB/sec (that is
analog-modem like speeds) where it takes 4 1/2 hours to transfer just
180MB across 271 files (see the specific data below). I thought that
rsync incrementals just really compared time stamps & perms -- so what
could possibly slow down the transfer speeds by so much?

It's true that the NAS has minimal memory (64MB) and a slow processor
(Arm 800MHz) but it is only used as a file server and I can get
reasonable raw NFS file transfer speeds.

The only change on the server was an upgrade from 3.1.x to 3.2.0 with
minimal config changes.

On the NAS, I upgraded the kernel to a debian kernel, changed from
ext2 to lvm2, and changed from RAID1 to non-RAID (which together only
slightly affected NFS transfer speeds) - but again since the NFS
speeds are reasonable, can any of the changes on the NAS even be a
cause?

The following results are for backup of the server itself (using plain
rsync without ssh since it's local):

               Totals                     Existing Files     New Files
Backup# Type   #Files   Size/MB  MB/sec   #Files   Size/MB   #Files  Size/MB
54      full   490469   20582.0  0.74     490376   20438.1   200     143.9 
55              incr   271              180.6    0.01     185      2.0          
 138     178.6 

where the full backup took 465.5 minutes and the level 1 incremental
took 260.4 minutes. I don't get any BackupPC log errors.

------------------------------------------------------------------------------
Colocation vs. Managed Hosting
A question and answer guide to determining the best fit
for your organization - today and in the future.
http://p.sf.net/sfu/internap-sfd2d
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/