Veritas-bu

Re: [Veritas-bu] netbackup master single threaded vs. multithreaded

2009-06-25 18:14:28
Subject: Re: [Veritas-bu] netbackup master single threaded vs. multithreaded
From: william.d.brown AT gsk DOT com
To: veritas-bu AT mailman.eng.auburn DOT edu
Date: Thu, 25 Jun 2009 23:03:38 +0100
I'll admit straight up that no, I have no *experience* of this.  However 
we did have long discussions with Sun and Symantec about this, as Sun 
raised it as a possible issue as the Sybase ASA is single threaded.

We had decided that on balance the T6320 (which is the same as the T5220 
but in a blade format) would be OK as we would be dividing our NetBackup 
environment into a low number of separate 'medium sized' domains, which 
related to the DR design.   The 'balance' is that the multi-threading is 
very suitable for the numerous daemons that the Master Server runs, and we 
can have 10GbE directly off the processor using the XAUI pass-through, so 
it should cope well with the chit-chat on the network.

Our DR plan changed so we went back to a single large domain, and we've 
gone back to an M4000 - faster CPU cores but many fewer threads.  We've 
not implemented it yet so I'll have to take a rain check on how it works 
out.  It is a very expensive solution compared to the T5220, but then one 
M4000 is cheaper than 8 T5220s.

My understanding is that there is a plan to get a multi-threaded Sybase 
'sometime' but I guess a cheap database has cheap features.  As you've got 
the T5220 I'd persevere, but maybe try and get Sun to help look using 
dtrace to make sure there are no tweaks in the OS - there are some potted 
solutions on the net, though none I've seen specific to NetBackup.  I'd 
also look at the database quite closely.  Put your DBA hat on.   We plan 
to make the changes to the cache that are available as we have tons of 
RAM.  We are separating the database, index, and transaction logs onto 3 
separate file systems.  Until we use seriously it I don't know if the EMM 
database disk performance is an issue - but it is worth looking for fast 
performance so use RAID 0+1 not RAID5 etc.  I'd love to try out the Sun 
SSDs on the EMM db!  They say that measured in IOPS they are already 
cheaper than getting the same IOPS with ordinary disks - of course in 
capacity they are still way expensive, but if that was your bottleneck it 
might be cheaper than another new server.

There is also tuning advice to stagger the start windows of your backups, 
as the resource broker has a self-limiting ability to find resources for 
jobs.  That means that if all jobs have a start window say at 18:00, some 
will start way past that just because the scheduler hasn't even thought 
about them.  You can see that if you create a vast pile of jobs eligible 
to be considered, it is going to take ages for the scheduler to go through 
the numerous options to pick the winner.  If you can stagger the windows, 
it will work through the list quicker, and jobs will start more 
predictably near the start of their windows.  Look at page 50 of the 
tuning guide under ?Staggering the submission of jobs for better load 
distribution?.  May mean changing the habits of a lifetime.

William D L Brown


veritas-bu-bounces AT mailman.eng.auburn DOT edu wrote on 25/06/2009 22:07:47:

> Hello,
> Has anyone here had performance problems with 6.5 masters running on
> Solaris with T5220 or T2 Sparc class processors? I have noticed that
> during periods of heavy job load the nbrb evaluation cycle, which 
> appears to handled via a single threaded dbsrv9 process, can take 
> forever and leave jobs stuck between requesting & granted resources 
> states. Sometime for 15 minutes a time, depending on the job load.
> I am familiar with nbrb.conf tweaks, however this doesn't make much 
> difference in my environment where we have many disk-based 
> destinations. Seems like in large environments, the master needs to 
> be on a processor better suited for single threaded performance.
> Thoughts?
> 
> 
> This message is private and confidential. If you have received it in
> error, please notify the sender and remove it from your system.
> 
> _______________________________________________
> Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu
> http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu


-----------------------------------------------------------
This e-mail was sent by GlaxoSmithKline Services Unlimited 
(registered in England and Wales No. 1047315), which is a 
member of the GlaxoSmithKline group of companies. The 
registered address of GlaxoSmithKline Services Unlimited 
is 980 Great West Road, Brentford, Middlesex TW8 9GS.
-----------------------------------------------------------

_______________________________________________
Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu

<Prev in Thread] Current Thread [Next in Thread>