I'm afraid I'll have to disagree with my esteemed colleague from
water.com. ;)
I believe the question is whether you are prioritizing ease of control
or full utilization of resources. 3rd party schedulers (TPS) give the
former and completely dismiss the latter. I have seen several large
shops use TPS, and they generally have great reporting and control over
how and when their backups run -- and they have CRAP for throughput.
One very large client, for example, averages 1.5 MB/s on their LTO-2
tape drives. Another averages about 5 MB/s. (And in case you DON'T
know, the farther your throughput is from the advertised rate of the
drive in question, the more that drive will fail.)
If you want to fully leverage your hardware, there is no way you can get
anywhere near as close as NBU can with it's auto-start of the next job
as soon as the last job is completed when there's a tape drive that's
supposed to have 5 jobs running on it and it now only has 4. As to CPU
utilization, the only way you can get CLOSE is to have backups kicked
off in advance (queued) so that NBU does its job and assigns it to the
next available drive. --- BUT --- When you run bpbackup -i on the
command line, a queued job takes up CPU resources. With the NBU
scheduler, you can have THOUSANDS of jobs in the queue waiting for the
right time and not a single one takes up any resources.
So back to my point. IF leveraging resources is the priority, then you
deal with the weird hoops that NBU makes you jump through in order to
handle pre-post processing. (I've even done multi-host-dependent
scripting in NBU. It's certainly harder, but totally possible.) But in
the end, your backups run MUCH faster and your tape drives and disk
systems are much more leveraged.
---
W. Curtis Preston
Backup Blog @ www.backupcentral.com
VP Data Protection, GlassHouse Technologies
-----Original Message-----
From: veritas-bu-bounces AT mailman.eng.auburn DOT edu
[mailto:veritas-bu-bounces AT mailman.eng.auburn DOT edu] On Behalf Of Jeff
Lightner
Sent: Wednesday, January 16, 2008 10:18 AM
To: A Darren Dunham; veritas-bu AT mailman.eng.auburn DOT edu
Subject: Re: [Veritas-bu] 3rd party scheduling
I agree it takes a fair amount of work and planning. However, I
disagree that it wasn't worth the effort. In the places where we did it
I much preferred the TWS setup to using NBU scheduling and/or cron for
those jobs that had to be client initiated. The logging it did was a
very good tool for determining where in the process things broke down
and kept me from troubleshooting things that weren't the underlying
issue. Also it saved CPU/memory because if the first job didn't work it
didn't try to kick off the other jobs at all (they would fail because of
the first job's failure if they had kicked off). Finally, since it was
an overall schedule once we'd resolved the issue with the job that
failed we could resume the schedule at the point of failure instead of
having to manually deal with different steps. (I have to do this manual
stuff at my present job because we rely on cron jobs instead.)
As to signaling it is very easy within scripts to set exit status/return
value (at least in UNIX/Linux) and use non-zero status as a "failure" of
the job to prevent the schedule from proceeding to the next job. One
thing to be sure you do within such scripts used for jobs is to be sure
you set an overall exit status rather than take the one for the last
command executed within the script.
e.g. If you had a script that did something like initiate a BCV
establish, monitor until done then split then at the end you did
something like "echo Job Completed" the final exit status for the job
would be 0 so long as the echo worked even if the BCV establish or split
had failed. You have to get the status for each of these and add it to
a return value variable (say $RETVAL) then at the end of the script put
in a statement like "exit $RETVAL) so it would no the overall job had
failed rather than rely on the exit status of that last command.
-----Original Message-----
From: veritas-bu-bounces AT mailman.eng.auburn DOT edu
[mailto:veritas-bu-bounces AT mailman.eng.auburn DOT edu] On Behalf Of A Darren
Dunham
Sent: Wednesday, January 16, 2008 12:53 PM
To: veritas-bu AT mailman.eng.auburn DOT edu
Subject: Re: [Veritas-bu] 3rd party scheduling
On Wed, Jan 16, 2008 at 10:35:46AM -0600, Stump, Bob A wrote:
> I have seen a trend to stop using NetBackup for starting and
monitoring
> backups. The move has been to doing all command line backups from a
3rd
> party scheduler as part of a true enterprise scheduling, monitoring
and
> management implementation. NetBackup is simply one of many products
that
> are controlled by the enterprise 3rd party sheduler/monitor. What are
> the drawbacks to moving in this direction?
You still require a significant configuration setup in Netbackup (pools,
policies, etc.), so you will end up with configuration information in
two places. That can complicate maintenance.
Most sites don't use them, so you may find it harder to find someone
with experience doing this, or have increased training costs when you
hire someone.
I'm not sure that the ways that the scheduler can launch jobs is
completely available to other tools (although maybe it is, since I
haven't investigated this much). Firing off a user backup is pretty
simple, but I'm not sure how I'd do the equivalent for an existing
schedule.
The NB scheduler takes into account various local status information
like previous successful/unsuccessful runs, current streams on a
client/STU, etc. You would have to gather and incorporate all of that
information into the remote scheduler, or you'd have to ignore it.
Depending on your setups, ignoring it can lead to problems.
I've been through this once (but that was with Networker, not
NetBackup). There was a lot of hope that the scheduler could handle a
lot of off-host processing (disk copies and mounts/unmounts) between
machines and coordinate them properly. It took a long time to build
because the folks setting up the scheduler didn't have much expertise
with backups. And I was limited in the ways I could signal Networker
from the scheduler. It also meant that whenever we had a problem, I had
to work on the backup issue and the folks that ran the scheduler and try
to make them understand the dependency chain that the process was
expecting.
I think it can still be a good idea in some situations, but it was a lot
more work than anyone expected.
--
Darren Dunham ddunham AT taos DOT com
Senior Technical Consultant TAOS http://www.taos.com/
Got some Dr Pepper? San Francisco, CA bay area
< This line left intentionally blank to confuse you. >
_______________________________________________
Veritas-bu maillist - Veritas-bu AT mailman.eng.auburn DOT edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
----------------------------------
CONFIDENTIALITY NOTICE: This e-mail may contain privileged or
confidential information and is for the sole use of the intended
recipient(s). If you are not the intended recipient, any disclosure,
copying, distribution, or use of the contents of this information is
prohibited and may be unlawful. If you have received this electronic
transmission in error, please reply immediately to the sender that you
have received the message in error, and delete it. Thank you.
----------------------------------
_______________________________________________
Veritas-bu maillist - Veritas-bu AT mailman.eng.auburn DOT edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
_______________________________________________
Veritas-bu maillist - Veritas-bu AT mailman.eng.auburn DOT edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
|
