Re: [BackupPC-users] Hardware considerations for building dedicated back

I have been using backuppc for quite some time. Here are some of the thinks I have found.

1)
backup method is super important. If your backup machine is in near the servers it is going to backup, then consider NOT using rsync. The disadvantage is that you will hammer your network but this can be overcome with a coupe gigabit network cards and a basic gigabit switch creating a backup only network. I suggest this because backuppc is IO bound and the way that rsync checks files is more intensive than doing a tar backup. Using a direct file copy method will still check mod times but will not scan the file list rsync does. You will get to take better advantage of sequential reads and writes rather than random reads and writes.
2)
A 2Ghz Dual Core CPU with 4GB of ram seams to be about the peak of efficiency. Other factors will limit backup speed long before you fill up 4GB of ram or use all of the CPU.
3)
hard disk performance is the #1 thing. RAM only helps compensate so much by giving extra cache memory. Get a real RAID card. software RAID is great for some things but the time it takes for data to travel accross the system bus twice (to cpu and from cpu) is much greater than the time it takes to go to the storage controllers CPU and back. Get a controller than has 128MB or more of cache on board and supports whatever RAID mode you want.
4)
RAID1. Skip RAID5, do RAID1. This data is important and the speed that you back it up is important. A few extra dollar on some redundant hard disks is a bargain. If you need more than 2 drives, consider RAID1 pairs with RAID0 on top.
5)
Hot spare. When a drive fails, you want the rebuild to start immediately and get done. That is a window for data loss. The only way around this without moving to a parity based RAID5 or 6 is to do triple drive redundancy(RAID1 with 3 disks)
6)
secondary backup server. You can mirror you whole array with rsync though it will be slow and use a ton of RAM. What I have done on smaller setups is to actually put my RAID1 device md0 in a RAID1 of md1 that is made of md0 and a AoE disk on another server. You can do some scripts to add the AoE disk to the array until it is completely synced up and then remove it. If you dont remove it, then the md1 raid device will be slow as it will stop to sync up the compontent devices every so often. You can use this same setup to sync to an external USB drive by adding it to the array for a sync and removing it afterwards. AoE = ATA over Ethernet and it has great sequential reads and write on ethernet. I have had 93% efficient transfers which means that I can get up to 116MB/s on Gigabit as there is no TCP/IP overhead, just straight ethernet frames. A RAID1 rebuild is sequential but there is some CPU overhead in either the system CPU or the RAID controller. I have managed over 80MB/s rebuild speed on a PCI-Express based server on Intel Pro Gigabit cards and a Cisco catalyst switch with AoE. Expect that to half if you use lesser hardware. AoE only works on local ethernet as ethernet frames do not travel beyond a managed switched ethernet, which means that you cannot router AoE over the internet or over a VPN.

------------------------------------------------------------------------------
Enter the BlackBerry Developer Challenge  
This is your chance to win up to $100,000 in prizes! For a limited time, 
vendors submitting new applications to BlackBerry App World(TM) will have
the opportunity to enter the BlackBerry Developer Challenge. See full prize  
details at: http://p.sf.net/sfu/Challenge

_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] Hardware considerations for building dedicated backuppc server