BackupPC-users

Re: [BackupPC-users] Backup Data Transfer Speed

2013-04-08 16:01:29
Subject: Re: [BackupPC-users] Backup Data Transfer Speed
From: <backuppc AT kosowsky DOT org>
To: Gary Roach <gary719_list1 AT verizon DOT net>
Date: Mon, 08 Apr 2013 15:59:18 -0400
Gary Roach wrote at about 09:02:22 -0700 on Monday, April 8, 2013:
 > On 04/07/2013 02:26 PM, backuppc AT kosowsky DOT org wrote:
 > > Gary Roach wrote at about 13:43:28 -0700 on Sunday, April 7, 2013:
 > >   >  Hi,
 > >   >
 > >   >  I am modifying my whole backup system for greater volume and speed. 
 > > The
 > >   >  new system is as follows:
 > >   >
 > >   >       Software - Backuppc using rsync (without SSH)
 > >   >       Server -    D865PERL Motherboard
 > >   >                       Pentium 4 - 2.4 GHz Processor
 > >   >                       IDE System hard drive
 > >   >                       SATA, 1 TB WD green backup data storage disk. 
 > > (S/B
 > >   >  1.5 Gb/S)
 > >   >                       1GB system memory
 > >   >                       PCI bus
 > >   >                       Intel PWLA8391GT PRO/1000 GT PCI Network Adapter
 > >   >                       Cat 6 cable
 > >   >
 > >   >       Clients (2)
 > >   >  Intel DP55KG   motherboard
 > >   >                   Intel i5-750 Processor 4 core, 2.66 GHz
 > >   >                   Hard Drive - Western Digital WD5000AADS -500GB, 32MB
 > >   >  Cache, SATAII (3 Gb/s)
 > >   >                   PCIX bus
 > >   >                   2 GB system memory
 > >   >                   Intel Gigabit CT PCI-E Network Adapter EXPI9301CTBLK
 > >   >                   Cat 6 Cable
 > >   >
 > >   >  I have read all sorts of complaints about this type setup and need 
 > > some
 > >   >  advice.
 > >
 > > I am not aware of *any* general complaints about any general
 > > hardware. Complaints if any typically reflect a lack of understanding
 > > about how BackupPC works or unrealistic expectations of performance
 > > relative to the size/number of backups on the one hand and network
 > > bandwidth and to lesser extents CPU speeds and disk setup on the
 > > other.
 > >
 > > That being said, I am puzzled by your statement that you are
 > > "modifying my whole backup system for greater volume and speed", yet
 > > your server setup seems to be based on 10 year old technology -
 > > Pentium 4 2.4GHz (presumably single core), 1GB DRAM, IDE system drive
 > > and a WD Green (i.e.., exact opposite of a performance) data disk. In
 > > fact, it would be harder to imagine a *less* suitable hardware setup
 > > if you are interested in greater volume and speed.
 > >
 > > Now this doesn't mean that BackupPC won't run on such a setup -- indeed, my
 > > home backup server is approaching 10 years and has only slightly
 > > better hardware (p4 2.8GHz, 2GB DRAM, pair of RAID1 Seagate
 > > (non-green) 1TB drives). It works fine for me but it is at the low end of
 > > performance. I get about 5-6 MB/sec max (on a 100bps ethernet). Of
 > > course, my backup backup server is even lower powered being an 800MHz
 > > ARM plugcomputer with 512MB DRAM and a USB hard drive... but it's slow.
 > >
 > >   >  It would appear that I should get around 1 Gbit/S transfer rate
 > >   >  for continuous data transfer. If this is not true, why not and what 
 > > can
 > >   >  I realistically expect.
 > >
 > > How would you ever expect to get 1 Gbit/second????
 > >
 > > First, while you mention that your ethernet cards are GigE, you don't
 > > say anything about your actual network speed being GigE which would
 > > require a GigE router/switch.
 > >
 > > Second, GigE can't even do 1 Gbit/second raw data transfers due to the
 > > ethernet (and other) protocol overheads plus collisions, etc. Even
 > > under ideal conditions, one would get a good fraction less than 1
 > > Gbit/second.
 > >
 > > Third, your old IDE-based motherboard (even with onboard SATA
 > > adapters) is unlikely to have more than about 150 mbit/sec capacity.
 > >
 > > Fourth, while your drive may have a 1.5Gb/sec interface and may in
 > > burst mode approach such a speed when reading/writing from its buffer,
 > > there is no way it can do sustained transfers anywhere near that
 > > speed -- especially Green drives that are optimized for power usage
 > > and not for performance.
 > >
 > > Finally, how do you expect your lowly p4 2.4GHz to keep up with rsync
 > > at anything approaching 1 Gbit/second let alone any of the overhead of
 > > BackupPC (and in particular compression).
 > >
 > > My guess is that if you don't have a true GigE network (including your
 > > router/switch), you are unlikely to get more than about 6-8 mbit/sec
 > > (i.e. .006 - .008 Gbit/sec). If you have a GigE network, maybe you can
 > > expect to get a little more before your motherboard and cpu are rate
 > > limiting.
 > >
 > > ------------------------------------------------------------------------------
 > > Minimize network downtime and maximize team effectiveness.
 > > Reduce network management and security costs.Learn how to hire
 > > the most talented Cisco Certified professionals. Visit the
 > > Employer Resources Portal
 > > http://www.cisco.com/web/learning/employer_resources/index.html
 > > _______________________________________________
 > > BackupPC-users mailing list
 > > BackupPC-users AT lists.sourceforge DOT net
 > > List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
 > > Wiki:    http://backuppc.wiki.sourceforge.net
 > > Project: http://backuppc.sourceforge.net/
 > >
 > Thank you all for you input.
 > 
 > Some clarification:
 >      The router/switch is being replaced with a Verizon M1424 Gegabit 
 > device.
 >      I plan on increasing the memory on the server.
 >      This is a home system and the real need for the re-do is to get 
 > more storage space.
 >      I don't need a high rel system assuming that all  of the computers 
 > are not going to fail at the same time. ( a chance I am willing to take 
 > for cost reduction)
 >      While these are home systems, they still run 24/7 and power 
 > consumption is a consideration.
 >      I have my backups staggered so that full backups happen on 
 > different days.
 >      The amount of file churning is relatively low.
 > 
 >      Above, do you really mean that I could expect transfer rates as low 
 > as 600 to 800 KB/S. I could carry the data in a bucket faster. I don't 
 > have a real good figure for the transfer rates for my present setup but 
 > know they are way faster than that.

I think you could expect 5-10 MB/S on a GigE network (assuming truly
GigE) with the rate limiting step being your CPU and the combined
limitations of non-RAID, Green drives on an old SATA/IDE motherboard.

 > 
 > I am getting the idea that putting in a GigE system isn't going to buy 
 > much speed. Any comments?

See above...


------------------------------------------------------------------------------
Minimize network downtime and maximize team effectiveness.
Reduce network management and security costs.Learn how to hire 
the most talented Cisco Certified professionals. Visit the 
Employer Resources Portal
http://www.cisco.com/web/learning/employer_resources/index.html
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/