BackupPC-users

Re: [BackupPC-users] Backup Data Transfer Speed

2013-04-07 17:27:59
Subject: Re: [BackupPC-users] Backup Data Transfer Speed
From: <backuppc AT kosowsky DOT org>
To: "General list for user discussion, questions and support" <backuppc-users AT lists.sourceforge DOT net>
Date: Sun, 07 Apr 2013 17:26:11 -0400
Gary Roach wrote at about 13:43:28 -0700 on Sunday, April 7, 2013:
 > Hi,
 > 
 > I am modifying my whole backup system for greater volume and speed. The 
 > new system is as follows:
 > 
 >      Software - Backuppc using rsync (without SSH)
 >      Server -    D865PERL Motherboard
 >                      Pentium 4 - 2.4 GHz Processor
 >                      IDE System hard drive
 >                      SATA, 1 TB WD green backup data storage disk. (S/B 
 > 1.5 Gb/S)
 >                      1GB system memory
 >                      PCI bus
 >                      Intel PWLA8391GT PRO/1000 GT PCI Network Adapter
 >                      Cat 6 cable
 > 
 >      Clients (2)
 > Intel DP55KG   motherboard
 >                  Intel i5-750 Processor 4 core, 2.66 GHz
 >                  Hard Drive - Western Digital WD5000AADS -500GB, 32MB 
 > Cache, SATAII (3 Gb/s)
 >                  PCIX bus
 >                  2 GB system memory
 >                  Intel Gigabit CT PCI-E Network Adapter EXPI9301CTBLK
 >                  Cat 6 Cable
 > 
 > I have read all sorts of complaints about this type setup and need some 
 > advice. 

I am not aware of *any* general complaints about any general
hardware. Complaints if any typically reflect a lack of understanding
about how BackupPC works or unrealistic expectations of performance
relative to the size/number of backups on the one hand and network
bandwidth and to lesser extents CPU speeds and disk setup on the
other.

That being said, I am puzzled by your statement that you are 
"modifying my whole backup system for greater volume and speed", yet
your server setup seems to be based on 10 year old technology -
Pentium 4 2.4GHz (presumably single core), 1GB DRAM, IDE system drive
and a WD Green (i.e.., exact opposite of a performance) data disk. In
fact, it would be harder to imagine a *less* suitable hardware setup
if you are interested in greater volume and speed.

Now this doesn't mean that BackupPC won't run on such a setup -- indeed, my
home backup server is approaching 10 years and has only slightly
better hardware (p4 2.8GHz, 2GB DRAM, pair of RAID1 Seagate
(non-green) 1TB drives). It works fine for me but it is at the low end of
performance. I get about 5-6 MB/sec max (on a 100bps ethernet). Of
course, my backup backup server is even lower powered being an 800MHz
ARM plugcomputer with 512MB DRAM and a USB hard drive... but it's slow.

 > It would appear that I should get around 1 Gbit/S transfer rate 
 > for continuous data transfer. If this is not true, why not and what can 
 > I realistically expect.

How would you ever expect to get 1 Gbit/second????

First, while you mention that your ethernet cards are GigE, you don't
say anything about your actual network speed being GigE which would
require a GigE router/switch.

Second, GigE can't even do 1 Gbit/second raw data transfers due to the
ethernet (and other) protocol overheads plus collisions, etc. Even
under ideal conditions, one would get a good fraction less than 1
Gbit/second.

Third, your old IDE-based motherboard (even with onboard SATA
adapters) is unlikely to have more than about 150 mbit/sec capacity.

Fourth, while your drive may have a 1.5Gb/sec interface and may in
burst mode approach such a speed when reading/writing from its buffer,
there is no way it can do sustained transfers anywhere near that
speed -- especially Green drives that are optimized for power usage
and not for performance.

Finally, how do you expect your lowly p4 2.4GHz to keep up with rsync
at anything approaching 1 Gbit/second let alone any of the overhead of
BackupPC (and in particular compression).

My guess is that if you don't have a true GigE network (including your
router/switch), you are unlikely to get more than about 6-8 mbit/sec
(i.e. .006 - .008 Gbit/sec). If you have a GigE network, maybe you can
expect to get a little more before your motherboard and cpu are rate
limiting.

------------------------------------------------------------------------------
Minimize network downtime and maximize team effectiveness.
Reduce network management and security costs.Learn how to hire 
the most talented Cisco Certified professionals. Visit the 
Employer Resources Portal
http://www.cisco.com/web/learning/employer_resources/index.html
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/