BackupPC-users

Re: [BackupPC-users] Backup Data Transfer Speed

2013-04-08 12:03:10
Subject: Re: [BackupPC-users] Backup Data Transfer Speed
From: Gary Roach <gary719_list1 AT verizon DOT net>
To: "General list for user discussion, questions and support" <backuppc-users AT lists.sourceforge DOT net>
Date: Mon, 08 Apr 2013 09:02:22 -0700
On 04/07/2013 02:26 PM, backuppc AT kosowsky DOT org wrote:
> Gary Roach wrote at about 13:43:28 -0700 on Sunday, April 7, 2013:
>   >  Hi,
>   >
>   >  I am modifying my whole backup system for greater volume and speed. The
>   >  new system is as follows:
>   >
>   >       Software - Backuppc using rsync (without SSH)
>   >       Server -    D865PERL Motherboard
>   >                       Pentium 4 - 2.4 GHz Processor
>   >                       IDE System hard drive
>   >                       SATA, 1 TB WD green backup data storage disk. (S/B
>   >  1.5 Gb/S)
>   >                       1GB system memory
>   >                       PCI bus
>   >                       Intel PWLA8391GT PRO/1000 GT PCI Network Adapter
>   >                       Cat 6 cable
>   >
>   >       Clients (2)
>   >  Intel DP55KG   motherboard
>   >                   Intel i5-750 Processor 4 core, 2.66 GHz
>   >                   Hard Drive - Western Digital WD5000AADS -500GB, 32MB
>   >  Cache, SATAII (3 Gb/s)
>   >                   PCIX bus
>   >                   2 GB system memory
>   >                   Intel Gigabit CT PCI-E Network Adapter EXPI9301CTBLK
>   >                   Cat 6 Cable
>   >
>   >  I have read all sorts of complaints about this type setup and need some
>   >  advice.
>
> I am not aware of *any* general complaints about any general
> hardware. Complaints if any typically reflect a lack of understanding
> about how BackupPC works or unrealistic expectations of performance
> relative to the size/number of backups on the one hand and network
> bandwidth and to lesser extents CPU speeds and disk setup on the
> other.
>
> That being said, I am puzzled by your statement that you are
> "modifying my whole backup system for greater volume and speed", yet
> your server setup seems to be based on 10 year old technology -
> Pentium 4 2.4GHz (presumably single core), 1GB DRAM, IDE system drive
> and a WD Green (i.e.., exact opposite of a performance) data disk. In
> fact, it would be harder to imagine a *less* suitable hardware setup
> if you are interested in greater volume and speed.
>
> Now this doesn't mean that BackupPC won't run on such a setup -- indeed, my
> home backup server is approaching 10 years and has only slightly
> better hardware (p4 2.8GHz, 2GB DRAM, pair of RAID1 Seagate
> (non-green) 1TB drives). It works fine for me but it is at the low end of
> performance. I get about 5-6 MB/sec max (on a 100bps ethernet). Of
> course, my backup backup server is even lower powered being an 800MHz
> ARM plugcomputer with 512MB DRAM and a USB hard drive... but it's slow.
>
>   >  It would appear that I should get around 1 Gbit/S transfer rate
>   >  for continuous data transfer. If this is not true, why not and what can
>   >  I realistically expect.
>
> How would you ever expect to get 1 Gbit/second????
>
> First, while you mention that your ethernet cards are GigE, you don't
> say anything about your actual network speed being GigE which would
> require a GigE router/switch.
>
> Second, GigE can't even do 1 Gbit/second raw data transfers due to the
> ethernet (and other) protocol overheads plus collisions, etc. Even
> under ideal conditions, one would get a good fraction less than 1
> Gbit/second.
>
> Third, your old IDE-based motherboard (even with onboard SATA
> adapters) is unlikely to have more than about 150 mbit/sec capacity.
>
> Fourth, while your drive may have a 1.5Gb/sec interface and may in
> burst mode approach such a speed when reading/writing from its buffer,
> there is no way it can do sustained transfers anywhere near that
> speed -- especially Green drives that are optimized for power usage
> and not for performance.
>
> Finally, how do you expect your lowly p4 2.4GHz to keep up with rsync
> at anything approaching 1 Gbit/second let alone any of the overhead of
> BackupPC (and in particular compression).
>
> My guess is that if you don't have a true GigE network (including your
> router/switch), you are unlikely to get more than about 6-8 mbit/sec
> (i.e. .006 - .008 Gbit/sec). If you have a GigE network, maybe you can
> expect to get a little more before your motherboard and cpu are rate
> limiting.
>
> ------------------------------------------------------------------------------
> Minimize network downtime and maximize team effectiveness.
> Reduce network management and security costs.Learn how to hire
> the most talented Cisco Certified professionals. Visit the
> Employer Resources Portal
> http://www.cisco.com/web/learning/employer_resources/index.html
> _______________________________________________
> BackupPC-users mailing list
> BackupPC-users AT lists.sourceforge DOT net
> List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
> Wiki:    http://backuppc.wiki.sourceforge.net
> Project: http://backuppc.sourceforge.net/
>
Thank you all for you input.

Some clarification:
     The router/switch is being replaced with a Verizon M1424 Gegabit 
device.
     I plan on increasing the memory on the server.
     This is a home system and the real need for the re-do is to get 
more storage space.
     I don't need a high rel system assuming that all  of the computers 
are not going to fail at the same time. ( a chance I am willing to take 
for cost reduction)
     While these are home systems, they still run 24/7 and power 
consumption is a consideration.
     I have my backups staggered so that full backups happen on 
different days.
     The amount of file churning is relatively low.

     Above, do you really mean that I could expect transfer rates as low 
as 600 to 800 KB/S. I could carry the data in a bucket faster. I don't 
have a real good figure for the transfer rates for my present setup but 
know they are way faster than that.

I am getting the idea that putting in a GigE system isn't going to buy 
much speed. Any comments?

Gary R.

------------------------------------------------------------------------------
Minimize network downtime and maximize team effectiveness.
Reduce network management and security costs.Learn how to hire 
the most talented Cisco Certified professionals. Visit the 
Employer Resources Portal
http://www.cisco.com/web/learning/employer_resources/index.html
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/