BackupPC-users

Re: [BackupPC-users] backuppc for large scale env

2009-08-27 16:59:08
Subject: Re: [BackupPC-users] backuppc for large scale env
From: Chris Robertson <crobertson AT gci DOT net>
To: "General list for user discussion, questions and support" <backuppc-users AT lists.sourceforge DOT net>
Date: Thu, 27 Aug 2009 12:52:53 -0800
Brent Clark wrote:
> Hiya
>
> I got quite a few servers around the world that I need to backup.
>
> The question I would like to ask is, how does backuppc scale to the other 
> backup solutions, in large environments. Also for those running backuppc for 
> large scale environment, would you be so kind as to share your
> experiences, tell tales of problems, obstacles, and how too you over came it.
>
> If anyone can assist or help, and share your story it would be greatly be 
> appreciated.
>
> Kind Regards
> Brent Clark.

Well, I think I have a pretty large setup, so here you go...

[root@archive-1 ~]# cat /etc/redhat-release
CentOS release 5.3 (Final)

[root@archive-1 ~]# uname -a
Linux archive-1.gcimbs.net 2.6.18-128.1.16.el5 #1 SMP Tue Jun 30 
06:07:26 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux

[root@archive-1 ~]# cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 23
model name      : Intel(R) Xeon(R) CPU           X3320  @ 2.50GHz
[SNIP]

[root@archive-1 ~]# cat /proc/meminfo
MemTotal:      8179984 kB
MemFree:         99864 kB
Buffers:         18972 kB
Cached:        4413596 kB
SwapCached:     476208 kB
Active:        2687372 kB
Inactive:      2385616 kB
[SNIP]

/data is an XFS file system (with an external journal) mounted 
(noatime,nodiratime,logdev=/dev/drbd1,logbufs=8,logbsize=262144) on top 
of (as of yet, unsynchronized) DRBD* on  a RAID 6 setup provided by an 
Adaptec 51645 with 16 Seagate ST31000340NS drives (13 data, 2 parity, 1 
hot spare).

[root@archive-1 ~]# df -h /dev/drbd0
Filesystem            Size  Used Avail Use% Mounted on
/dev/drbd0            3.0T  507G  2.6T  17% /data

[root@archive-1 ~]# df -i /dev/drbd0
Filesystem            Inodes   IUsed   IFree IUse% Mounted on
/dev/drbd0           1932728448 36671135 1896057313    2% /data

BackupPC Server Status

General Server Information

    * The servers PID is 22319, on host archive-1.gcimbs.net, version
      3.1.0, started at 2009-08-11 14:43.
    * This status was generated at 2009-08-27 12:12.
    * The configuration was last loaded at 2009-08-18 10:26.
    * PCs will be next queued at 2009-08-27 13:00.
    * Other info:
          o 0 pending backup requests from last scheduled wakeup,
          o 0 pending user backup requests,
          o 0 pending command requests,
          o Pool is 382.63GB comprising 9959365 files and 4369
            directories (as of 2009-08-27 11:58),
          o Pool hashing gives 2986 repeated files with longest chain 623,
          o Nightly cleanup removed 845394 files of size 45.10GB (around
            2009-08-27 11:58),
          o Pool file system was recently at 17% (2009-08-27 12:05),
            today's max is 17% (2009-08-27 09:00) and yesterday's max
            was 17%.


BackupPC: Host Summary

    * This status was generated at 2009-08-27 12:13.
    * Pool file system was recently at 17% (2009-08-27 12:05), today's
      max is 17% (2009-08-27 09:00) and yesterday's max was 17%.

Hosts with good Backups

There are 125 hosts that have been backed up, for a total of:

    * 499 full backups of total size 2512.20GB (prior to pooling and
      compression),
    * 1849 incr backups of total size 466.60GB (prior to pooling and
      compression).


My largest host is almost 90GB, with 1.3 million files.  My average host 
size is around 5GB and 100,000 files.  My blackout period is from 06:00 
to 20:30.    I run 8 concurrent backups, and rarely see a backup running 
past 08:00.  BackupPC_nightly (the pool culling) starts at 09:00 and 
covers 1/8th of the pool at a time with 4 threads.  It is currently 
completing in around 3 hours.  I run a FullPeriod of 6.6 days and an 
incremental period of .6 days.  My FullKeepCnt is 4, and my IncrKeepCnt 
is 14.  IncrLevels is set to 1,2,3.

If you have any questions, or feel there is anything I was not clear 
about, feel free to ask.

Chris

* So far the DRBD abstraction layer seems to be having a minimal effect 
on performance.  One of these days I'll get the mirror server online and 
start synchronizing.  For what it's worth:

[root@archive-1 ~]# rpm -q kmod-drbd82 drbd82
kmod-drbd82-8.2.6-2
drbd82-8.2.6-1.el5.centos


------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day 
trial. Simplify your report design, integration and deployment - and focus on 
what you do best, core application coding. Discover what's new with 
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
BackupPC-users mailing list
BackupPC-users AT lists.sourceforge DOT net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

<Prev in Thread] Current Thread [Next in Thread>