Re: [BackupPC-users] Centralized storage with multiple hard drives

thorvald <backuppc-forum AT backupcentral DOT com> wrote on 03/19/2014 06:53:19 AM: > Let's say that the storage is not a problem for me and I can have as > many TB or PT as I need. However the main assumption is that every > box has got a separate "disk" to be backed up to. So now I faced the > problem with BackupPC which does use pool or cpool to store files > within :/. I don't need any compression or deduplication. Is there > any way to backup files directly to pc/HOST/ instead ?

I am going to give a flat "no" to this. You may be able to break things within BackupPC to accomplish this (never run the link, for example), but you are *breaking* things. Don't do that if you expect *anyone* to be able to help you.
> I'm not going to backup couple of hundreds servers using one > BackupPC instance of course but I want to back up at least 100 > servers per BackupPC instance. > > Is there something you could advise me ?
Sure: use virtualization. Create your huge datastore (or multiple datastores) and create a VM for each unit that needs its own pool. Each VM will get its own (virtual) disk. Each will be, for better or worse, completely separate from each other.

The cost for this will be you'll need more RAM. You'll have multiple copies of the Linux kernel, Apache web server and BackupPC running. However, I doubt that all of that together is 250MB: I've run *many* entire BackupPC servers in 512MB of RAM. But there will be resources wasted that you wouldn't have with a single instance.

Of course, that doesn't include RAM you'll need for each VM to have enough RAM for caching (to give good performance), but that RAM requirement would be roughly the same with a single instance anyway, so you can't count that as a disadvantage for VM's: that's just the nature of PC technology.

There are other things that you'll have to worry about, no matter whether it's a single instance or multiple VM's. The screamingly obvious one is disk performance. I've found that running more than one BackupPC job at a time destroys performance, even on servers with 3-4 disks, using both hardware and software RAID. I have one server with 6 disks that does OK with a couple of jobs, but not nearly as fast as one at a time. I have one server with 12 disks: that one has so much disk performance that I'm bottlenecked at the other end, so I can do 3 or so jobs simultaneously. You're talking "hundreds". So disk performance will really, really matter here. Your question will be more like, "How many *shelfs* of disks?" rather than, "How many disks?"

Another issue will be network bandwidth. Unless you're running a well-designed 10GbE network, forget backing up "hundreds" (your number) of servers in a timely fashion. I've found that 4-port bonded 1GbE is less than 2 times as fast as a *single* GbE. Maybe I've done something wrong every time I've done it, but I've basically washed my hands with bonding (for performance). You want performance, you're gonna need 10GbE. And if you *do* go 10GbE, the switch that your server is plugged into better either have a 1GbE connection *directly* to each other switch, or those switches better be behind all 10GbE connections upstream. Otherwise, you'll just bottleneck on a downstream 1GbE connection.

Tim Massey


Out of the Box Solutions, Inc. Creative IT Solutions Made Simple! http://www.OutOfTheBoxSolutions.com tmassey AT obscorp DOT com		22108 Harper Ave. St. Clair Shores, MI 48080 Office: (800)750-4OBS (4627) Cell: (586)945-8796

------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and their
applications. Written by three acclaimed leaders in the field,
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/13534_NeoTech

_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/