Veritas-bu

[Veritas-bu] NB 4.5 catalog data points & experiences

2002-11-13 14:30:44
Subject: [Veritas-bu] NB 4.5 catalog data points & experiences
From: Fabbro.Andrew AT cnf DOT com (Fabbro, Andrew P)
Date: Wed, 13 Nov 2002 11:30:44 -0800
I have been working on upgrading from 3.4.1 to NB 4.5 (4.5.2 specifically).
One of the things I've been considering is the new catalog format.  I hoped
the new binary format would improve several things:

- consume less disk (we have ~80-90GB of catalog)

- speed catalog backups.  At present, our catalog backups take ~2.5 hours.
This is really ~5 hours + some vault processing time to get an offsite and
onsite every day.  I would love to have two offsite catalog backups daily
but just don't have the time in the schedule.  The delay is mostly caused by
having 250,000+ files.

- reduce the "compress old images" housekeeping.  This process, which is not
schedule-able, essentially takes NetBackup down (no backups start during
this time).  The process takes up to two hours (with an 4-proc, 4GB RAM E450
talking to fiber enterprise storage) in our environment.  

After some closer examination, it appears I'll only get one of those things
(the last one) ;) 

I set up a lab environment and using an older catalog, I set up 3.4 with
about 55GB of catalog.  I performed the 4.5 upgrade and subsequently did the
cat_convert catalog conversion.

For 55GB of catalog, the procedure took 37 hours!  This was on a system that
was otherwise completely unused.  Granted, I wasn't using the same
enterprise-grade storage we use in production, but even if the conversion
time was cut in half, that's still a long train ride.  Veritas does not
provide very good tools - I had to write a script to descend into each
/usr/openv/netbackup/db/images/client-name directory and run the cat_convert
command there.

I talked to Veritas and found out some interesting things:

- "binary format" does not mean "real database".  It's not like NB is
bringing up an instance of Postgres or Oracle to run things.  It's still
using a directory-based format...it's just laying down binary files instead
of ASCII ones.  One benefit of this is that the old practice of making
catalog backups at will (and not shutting off bprd first) still works.  You
lose data on any backups "in flight" but otherwise your catalog is fine.
It's not like Oracle where the files have to be completely quiesced or
everything is worthless.  We do this style of "don't turn off bprd" on 3.4.x
because we don't have periods where we can shut down NB for 5+ hours.  I've
restored from these catalog backups without a problem, though shutting off
bprd first is still best practice.

- the new 4.5 and the old 3.4 database entries (read: directories and files)
can coexist together fine.  Once you install 4.5, all entries from that
point forward are binary.

- how much disk space you'll save depends somewhat on your policies.  ASCII
files that are compressed take less disk space than the new binary files
(though of course, you don't have to uncompress the binary files).  So if a
lot of your catalog is past your "compress old images" window, the you won't
save much.  Even with the bulk of our catalog uncompressed, I saw less than
a 10% improvement: 55GB went to 51GB.  The number of files did not change
significantly, so backing up the catalog will still be slow.

I think a likely upgrade strategy for large shops is to do the upgrade and
then either let the new binary format replace the old ASCII over time (as
the latter expires), or selectively do the catalog conversion for clients
when none of its backups are running.  

Good luck ;)

--
 Drew Fabbro [fabbro.andrew AT cnf DOT com]
 Unix Systems Group

<Prev in Thread] Current Thread [Next in Thread>
  • [Veritas-bu] NB 4.5 catalog data points & experiences, Fabbro, Andrew P <=