Veritas-bu

[Veritas-bu] running out of memory on Solaris 8 master servers

2005-12-22 05:48:23
Subject: [Veritas-bu] running out of memory on Solaris 8 master servers
From: bobbyrjw AT bellsouth DOT net (Bobby Williams)
Date: Thu, 22 Dec 2005 05:48:23 -0500
We are running 5.1MP2, 3A and 4 (migrating to 4) on 17 master servers.

Smallest server is an old E-250 with 2GB of RAM.  Most are V240's with 8GB
of RAM.  Solaris 9 on all of them.

We are not running out of memory.

Have you tuned your system?

I am hammering my master servers to see how far we can go.  I duplicate
tapes with large amounts of data (I am using mostly LTO Gen2 drives, but my
E250 has DLT7000).

I am getting great performance.  I am seeing some of my bigger NB clients
streaming @ 25MB /sec across some GB network connections.  The master server
just hums along.

(I will admit this, one call that I opened with Veritas support about a
problem that turned out to be an HBA problem, the backline support person
told me that I had allocated to many buffers in my definitions.  However, I
was not getting any "waited for empty buffers" in my bptm logs).

Tune it up.

Also, we are forced to run HP ITO clients on our masters, along with Sun
CST.  It is not a big load generator, but noticeable.

You may want to check your Solaris OS and see if you have wasted processes
running. 




Bobby Williams
2205 Peterson Drive
Chattanooga, Tennessee  37421
423-296-8200

-----Original Message-----
From: veritas-bu-admin AT mailman.eng.auburn DOT edu
[mailto:veritas-bu-admin AT mailman.eng.auburn DOT edu] On Behalf Of Ed Wilts
Sent: Wednesday, December 21, 2005 11:17 AM
To: King, Cheryl
Cc: veritas-bu AT mailman.eng.auburn DOT edu
Subject: Re: [Veritas-bu] running out of memory on Solaris 8 master servers

On Wed, Dec 21, 2005 at 08:51:45AM -0700, King, Cheryl wrote:
> I have two Solaris 8 master servers running NetBackup 5.1 MP3S2.  They 
> keep running out of memory after a few months.  I'm wondering if 
> anyone else has this problem and if they fixed it.  It started some 
> time after going to v5.1 I think.  Maintenance hasn't fixed it.

We've been seeing this as well on Solaris 9 with NBU 5.0.  We're currently
at MP6.

> Any ideas on what's causing it or how I could find out what's causing 
> it?  When it runs out of memory it's always during regular backups.

Backups take memory so they're victims.  We see a nice jump in swap
utilization every time our main windows open at 6pm.

> Since I haven't seen any discussion about this on this list I'm 
> assuming regular scheduled backups isn't the cause.
> 
> The only way to get back swap space is to boot the server.

Add swap :-).  That's our workaround until we can upgrde the server with a
new box (unfortunately our 420 is maxed out for memory already).

We run the rman Oracle instance on our master server and it appears to be
part of the culprit.  We had it off for a while and we were much better.  We
don't know if it's the straw that breaks the camel's back or if it's the
cause.

NetBackup really doesn't like to run out memory (not that I blame it
much) - all the jobs terminate with a 150.

Are other Solaris admins experiencing the same issues?  Does NetBackup just
take that much memory to run?

I have heard that NBU 6 takes a lot of memory than NBU 5 but haven't been
able to validate that yet.

        .../Ed

--
Ed Wilts, Mounds View, MN, USA
mailto:ewilts AT ewilts DOT org
_______________________________________________
Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu