All too often I see NBU databases that aren't tuned for performance - you'd be
surprised what striping your NBU DB filesystem (and the use of VXFS instead of
UFS, in the case of Unix master servers) can do for your overall backup
performance...
rob
At 3:34 PM -0700 10/17/00, Jonathan Geibel wrote:
>our current machine is a dual-processor Sun E4000 with 1GB of RAM..
>/usr/openv is sitting on a SCSI-attached Sun RAID box (an older Maxstrat)
>
>we are about to upgrade all of this as soon as some new hardware comes
>in so we'll see if that makes any difference.
>
>My guess is that this flatfile database just does not scale very well and
>better hardware won't make a difference at this point.
>
>the problem with restores is not with how fast the actual data transfer
>is, but just with netbackup munging through it's database trying to find
>out which images to restore from.
>
>also, using bplist or any of the GUI utilities that access the database
>are excessively slow at this point and are almost useless.
>
>we oftentimes find ourselves grepping through the db/images directories to
>try and find data instead of using the netbackup utilities..
>
>the worst is trying to find a single filename that could exist in any of
>our larger backups.. trying to search through everything can take many
>hours to complete. being able to dump this information into a real
>database where we could do fast searches for single filenames should fix
>this problem..
>
>On Tue, 17 Oct 2000, Joshua Fielden wrote:
>
>> 3.4 removes this artificial limitation (db == tape size), as do various
>> posted tricks (search veritas' web site for it). We are an EMC shop, so we
>> have the database backed up to disk units on machines other than the master
>> server. This equals databases on multiple EMC disk arrays, plus the
>> soon-to-be-implemented vaulting. We feel rather safe with our databases.
>>
>> I'm concerned about the time it takes to read the indexes, and the size.
>> We're already at ~20G in just a couple of months in the images directory.
>> What sort of machine/load/disk do you use that it takes that long to search,
>> Jonathan? We have a Marathon (4-proc 420R equivalent, 2G ram) as our master
>> server with 3 dmp connections to an EMC 3870 with 16G cache, and we see some
>> pretty zippy performance in restores. I realize this is overkill for most
>> shops, so I'm wondering what configurations cause performance cramp? We may
>> be forced to downsize, if the load on the machine stays as consistantly low
>> as it is.
>>
>> As to flat files vs databases, our legato indexes were ~70G on one of our
>> data zones, which did 90mm small files to tape on 'full night'. On a 8x400
>> E4500 with 4.5G RAM, on EMC storage, it would take sometimes 90m to
>> 'changetime' in the recover command on an index of around 18G. Once you get
>> to a certain size, response time sucks no matter what you do, AFAIK. :)
>>
>> JF
>>
>> On Tue, Oct 17, 2000 at 01:22:59PM -0600, McMurphy, Tim filled up my inbox
>> with:
>> >
>> > If anyone has any insight please post to the group! I am soon to re-design
>> > and implement our netbackup structure and one of the problems is, you
>> > guessed it. Our db is only 22 gig (small compared to some) but big enough
>> > to
>> > cause me concern for the future.
>> >
>> > I was thinking along the lines of having one database per year (1999, 2000,
>> > 2001 etc) and trying to keep the size in line that way. Alternatively
>> > separate a separate db for different platforms (not my fav by a long
>> > stretch).
>> >
>> > A question for you Jonathan is what do you backup your db to? I was under
>> > the impression that the netbackup db couldn't span multiple tapes. If I am
>> > wrong on that then good but if it is true then you must be near the limit
>> > of
>> > any tape technology that I am aware of.
>> >
>> > -----Original Message-----
>> > From: Jonathan Geibel [mailto:Jonathan.Geibel AT disney DOT com]
>> > Sent: Tuesday, October 17, 2000 12:41 PM
>> > To: veritas-bu AT mailman.eng.auburn DOT edu
>> > Subject: [Veritas-bu] Netbackup Database size and speed
>> >
>> >
>> > Hello,
>> >
>> > Question for you all:
>> >
>> > Has anyone had any troubles with the netbackup database size getting out
>> > of hand and creating excessively bad performance issues? Has anyone come
>> > up with any intuitive solutions for how to solve this problem?
>> >
>> > With 13TBs of data being backed up every week, our netbackup db/images
>> > directory has grown to obscene sizes (60GB of flat-file data compressed).
>> >
>> > Because netbackup doesn't use a real database to hold this information
>> > (simple flatfiles..) the scalability of this design is rather weak..
>> >
>> > these huge flatfile databases are causing us major performance problems
>> > when trying to do simple restores.. the simple act of looking for a
>> > single file can take several hours to complete as it chugs through all of
>> > it's backups files.
>> >
>> > we've indexed everything, but that doesn't seem to help..
>> >
>> > I've heard there are tools to dump this information into an actual SQL
>> > database which could be an extremely useful tool for hunting down the
>> > backups for specific files..
>> >
>> > has anyone done this? we were thinking of possible dumping the data once
>> > a day into a database for doing quick searches on..
>> >
>> > I'd be curious to hear if anyone else has run into these performance
>> > problems..
>> >
>> > Jon
>> >
>> >
>> > =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
>> > Walt Disney Feature Animation The Secret Lab
>> > Senior Systems Engineer 818.526.3051
>> > =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
>> >
>> >
>> >
>> >
>> >
>> > _______________________________________________
>> > Veritas-bu maillist - Veritas-bu AT mailman.eng.auburn DOT edu
>> > http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
>> > _______________________________________________
>> > Veritas-bu maillist - Veritas-bu AT mailman.eng.auburn DOT edu
>> > http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
>>
>> --
>> "Any man page that includes the words "USE AT YOUR OWN RISK. BEWARE OF DOG.
>> SLIPPERY WHEN WET" means trouble" - Michael Lucas
>> Joshua Fielden, Senior Systems Administrator and Backups Team Lead
>> eXcite@Home, Inc. jfielden AT excitecorp DOT com 650-556-3316
>> _______________________________________________
>> Veritas-bu maillist - Veritas-bu AT mailman.eng.auburn DOT edu
>> http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
>>
>
>_______________________________________________
>Veritas-bu maillist - Veritas-bu AT mailman.eng.auburn DOT edu
>http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
|
