Good Morning --
I'm going to share my backup enterprise configuration as it is now,
but I'm planning on doing massive hardware reorganization in about
a month (a challenge about which I'm not very excited).
Currently, I have a Sun V240 Solaris 9 master server with 4 Sun StorEdge
3511 storage arrays (SATA drives), striped to create 2 6TB disk storage
units and a 3-LTO2 drive tape library. I was not able to use NetBackup's
disk-staging storage units because I was unable to keep them
empty enough to perform more than a couple of night's backups, after
which most of my backups would fail daily.
For me, optimal performance is actually only fair throughput, but
capable of finishing most of my scheduled backups within a 16-hour
window. I have each DSU set for 24 concurrent backups and a
fragment size of 1000 MB (2000 MB was slower and fewer backups
were able to complete for some reason). I am using the following
/usr/openv/netbackup/db/config BUFFER files (with contents shown):
NUMBER_DATA_BUFFERS: 16
NUMBER_DATA_BUFFERS_DISK: 24
SIZE_DATA_BUFFERS: 262144
SIZE_DATA_BUFFERS_DISK: 1048576
Here are my /etc/system settings:
* NetBackup 5.1 Settings
set msgsys:msginfo_msgmap=512
set msgsys:msginfo_msgmax=8192
set msgsys:msginfo_msgmnb=65536
set msgsys:msginfo_msgmni=256
set msgsys:msginfo_msgssz=16
set msgsys:msginfo_msgtql=512
set msgsys:msginfo_msgseg=8192
set semsys:seminfo_semmap=64
set semsys:seminfo_semmni=1024
set semsys:seminfo_semmns=1024
set semsys:seminfo_semmnu=1024
set semsys:seminfo_semmsl=300
set semsys:seminfo_semopm=32
set semsys:seminfo_semume=64
* setting shmmax to 1/4 physical memory for backups to disk - 20050707 - kfh
* set shmsys:shminfo_shmmax=16777216
set shmsys:shminfo_shmmax=536870912
set shmsys:shminfo_shmmin=1
set shmsys:shminfo_shmmni=220
set shmsys:shminfo_shmseg=100
I have a cron script which checks for active backups or duplications
(disk-to-tape staging processes); when there are no active backups
or duplications, I use the bppllinfo to modify ALL of my backup
policies to the empty dsu.
We have a script which builds files containing backupids based on
the backupids' retention so that each duplication process will be
fill up a tape without constantly having to mount and dismount tapes
(tape mount/dismounts eat up about 4 precious minutes).
So, during the day, my 3 tape drives are in use all day long with
staging diskimages to tape from the storage unit used for backups
the day before. These duplications run into the night, usually finishing
anytime between 2 and 5am. Most of my backup windows open at 8pm with
about 10 policies starting between 6pm and 7pm. The backups will be
writing to the storage unit not being read by the duplication processes.
During the day when there are no backups running, the duplication disk-read
speeds fall between 10000-20000 Kbytes/sec. In the evening, the disk-read
speeds drop to as low as 6500 Kbytes/sec, but there are still many that
are between 13000 and 20000 Kbytes/sec. My disk-write speeds during
backups is a dismal 2500-6000 Kbytes/sec because the cpu load on the
server is over 90% (load average rarely drops under 25); a "top"
reveals that each bpdm process is using 1.2+% of the cpu. The bptm
processes (of which there are only 3 ever, consume very little.
Even with these "optimal" settings, I am rarely able to complete all
backups each night.
Hope this helps.
-- kathy
> --__--__--
>
> Message: 4
> To: veritas-bu AT mailman.eng.auburn DOT edu
> Subject: Re: [Veritas-bu] Re: Fast backup to tape but slow backup to
> disk on NBU
> From: william.d.brown AT gsk DOT com
> Date: Tue, 23 Aug 2005 06:59:30 +0100
>
> Kathryn,
>
> You said:
>
> <<However, because I can't stage diskimages to tape while active backups
> are writing to the disks,>>
>
> Is that a product limitation, or due to some practical reality? Is it an
> easy script...presumably you need a quiet point to run the script?
>
> William D L Brown
>
>
>
> --__--__--
>
>
> Message: 6
> Subject: RE: [Veritas-bu] Re: Fast backup to tape but slow backup to disk on
> NBU
> Date: Tue, 23 Aug 2005 07:43:36 -0400
> From: "Paul Keating" <pkeating AT bank-banque-canada DOT ca>
> To: "Veritas List" <veritas-bu AT mailman.eng.auburn DOT edu>
>
> Performance limitation.
>
> Doing your offload to tape is requiring the heads to seek which would
> kill backup performance.
>
> I'm very interested to hear more about Kathryn's 24 concurrant jobs to a
> staging unit.
> Everyone else seems to say 1 or max 2 jobs per!!!
>
> Paul
>
> --__--__--
>
> Message: 8
> Date: Tue, 23 Aug 2005 08:26:19 -0400
> From: Shyam Hazari <shazari AT gmail DOT com>
> To: Paul Keating <pkeating AT bank-banque-canada DOT ca>
> Subject: Re: [Veritas-bu] Re: Fast backup to tape but slow backup to disk on
> NBU
> Cc: Veritas List <veritas-bu AT mailman.eng.auburn DOT edu>
>
> We have 1.2TB disk staging units with max concurrent jobs set to 4 and
> multiplexing is 1
>
> -Shyam
>
> --__--__--
>
> Message: 9
> Subject: RE: [Veritas-bu] Re: Fast backup to tape but slow backup to disk on
> NBU
> Date: Tue, 23 Aug 2005 08:32:00 -0400
> From: "Paul Keating" <pkeating AT bank-banque-canada DOT ca>
> To: "Veritas List" <veritas-bu AT mailman.eng.auburn DOT edu>
>
> Ok...I'm guessing this is your max jobs on your server/clients.
>
> How many staging units do you have?
>
> Problem is, I have a lot of slow clients......if I set mpx to 1, it
> would take forever to back them up, sequentially.
> In some cases, I've got 8+ machines going to one tape drive, and the
> bottleneck is still the clients pushing the data (old sun boxes with
> 100mb/s ethernet and old disk arrays with small slow disks.), but I
> don't want the mpx'ing higher than that.
>
> With the number of these clients, I would need to cut my storage up into
> a LOT of small DSSUs, and the smaller the DSSU, the more difficult it is
> to manage...
>
> Paul
>
|
