Mike Dresser wrote:
> Matthias Meyer wrote:
>
>> Dear all,
>>
>> How scalable is backuppc?
>> Where are the limits or what can produce performance bottlenecks?
>>
>> I've heard about hardlinks which can be a problem if theire are millions of
>> it. Is that true?
>>
>>
> The file system can become... interesting to fix or backup when you get
> a few million hard links, especially if you're using XFS.
Amen to that...
[root@archive-1 trash]# df -i /dev/sdb1
Filesystem Inodes IUsed IFree IUse% Mounted on
/dev/sdb1 6442438528 55813203 6386625325 1% /data
[root@archive-1 trash]# mount |grep /dev/sdb1
/dev/sdb1 on /data type xfs
(rw,noatime,nodiratime,nobarrier,logbufs=8,sunit=512,swidth=6656,logbsize=262144)
[root@archive-1 trash]# cat /etc/redhat-release
CentOS release 5.2 (Final)
[root@archive-1 trash]# uname -srvmp
Linux 2.6.18-92.1.18.el5 #1 SMP Wed Nov 12 09:19:49 EST 2008 x86_64 x86_64
> There _appears_ to be some bugs in Debian etch's xfs tools, last time I had
> to
> run an xfs_repair -n on etch it took 6 days.. after upgrading to lenny
> it takes about 10 minutes or so. I'm guessing there were some major
> improvements in the xfs tools from etch to lenny with regards to memory
> usage.
>
>> Anyone would share his experience about CPU usage in relation to bandwidth
>> usage? Particularly by using rsync?
>>
>>
> I see about 2-15m/s on our rsync backups CPU usage is pretty low, it's
> mostly disk i/o that holds it up. The newer faster machines back up
> faster, so there's a bottleneck on the clients as well.
>
Same story here. CPU is, to a close approximation, never a problem.
For an added twist, most of my clients are on the far end of a satellite
link. After the initial backups, even running 16 backups in parallel, I
hardly peak above 4Mbit/sec (per iftop).
>> I have a 2x1GHz Server with 2GB RAM and 4x500GB SATA Disks in Software
>> Raid5.
>>
>>
> server here is a 2x1.8ghz opteron 265, 5GB ram, 8x1TB with 3ware raid5.
> Backup window is set from 18:00 to 23:00, it generally finishes all the
> backups within that 5 hours. Full's every 8 days, incr every day.
>
> * Pool is 1852.64GB comprising 8466234 files and 4369 directories
> (as of 3/12 02:26),
> * Pool hashing gives 9654 repeated files with longest chain 85,
> * Nightly cleanup removed 7189 files of size 35.62GB (around 3/12
> 02:26),
> * Pool file system was recently at 44% (3/12 12:05), today's max is
> 44% (3/12 00:00) and yesterday's max was 44%.
>
> There are 42 hosts that have been backed up, for a total of:
>
> * 789 full backups of total size 10819.30GB (prior to pooling and
> compression),
> * 284 incr backups of total size 913.00GB (prior to pooling and
> compression).
For comparison, I have a Xeon X3320, 8GB RAM and 16 Seagate ES.2 1 TB
drives (ST31000340NS) on a Adaptec 51645 using a RAID6 setup
(effectively 13 data drives, 2 parity, 1 hot spare). Fulls every 7
days, incremental every day. I Started off keeping 4 weeks of fulls and
two weeks of incrementals, and cleaning the whole pool nightly. All
(~125) backups completed within the 20:00-07:00 time frame and
BackupPC_nightly finished in under 5 hours (with a concurrence of 8).
Then I doubled retention on both fulls and incrementals and have been
fighting steadily worsening performance problems since. Dropping the
retention back to initial levels is taking a while (BackupPC_trashClean
seems to finally be gaining some ground).
* Pool is 587.09GB comprising 31187909 files and 4369 directories
(as of 2009-03-11 14:46),
* Pool hashing gives 2294 repeated files with longest chain 944,
* Nightly cleanup removed 1648979 files of size 50.98GB (around
2009-03-11 14:46),
* Pool file system was recently at 11% (2009-03-12 11:00), today's
max is 11% (2009-03-12 07:00) and yesterday's max was 11%.
There are 125 hosts that have been backed up, for a total of:
* 495 full backups of total size 2384.27GB (prior to pooling and
compression),
* 855 incr backups of total size 380.11GB (prior to pooling and
compression).
Chris
------------------------------------------------------------------------------
Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are
powering Web 2.0 with engaging, cross-platform capabilities. Quickly and
easily build your RIAs with Flex Builder, the Eclipse(TM)based development
software that enables intelligent coding and step-through debugging.
Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
|
