Veritas-bu

[Veritas-bu] NB51MP3 high cpu loads caused by bpdm?

2005-07-11 10:40:26
Subject: [Veritas-bu] NB51MP3 high cpu loads caused by bpdm?
From: Gregory.Geyer AT Avnet DOT com (Geyer, Gregory)
Date: Mon, 11 Jul 2005 07:40:26 -0700
Kathy,

I've noticed to add new storage units I have to bounce Netbackup.
However, to make a change to a storage unit I just have to run these
commands to make them stick.  See if it works for you:

bpschedreq -read_stu_config
bprdreq -rereadconfig

Someone also has posted "bpsched -read_stunits" but I haven't run this
in the past.

G. 

-----Original Message-----
From: veritas-bu-admin AT mailman.eng.auburn DOT edu
[mailto:veritas-bu-admin AT mailman.eng.auburn DOT edu] On Behalf Of Kathryn
Hemness
Sent: Saturday, July 09, 2005 8:43 AM
To: Geyer, Gregory
Cc: veritas-bu AT mailman.eng.auburn DOT edu
Subject: RE: [Veritas-bu] NB51MP3 high cpu loads caused by bpdm?

Well, last night's backups as a test of the new settings was a wash.
I'm still seeing fragment sizes of 2000MB on the storage unit whose
configurations I changed.  It looks like I need to restart the NetBackup
daemons when I modify the storage unit settings. sigh.

I'll be modifying my other storage unit fragment size after all
duplications from it finish this afternoon.

You can be sure I'll stop and restart the daemons this time.
another sigh.


On Fri, 8 Jul 2005, Geyer, Gregory wrote:

> Date: Fri, 8 Jul 2005 11:37:42 -0700
> From: "Geyer, Gregory" <Gregory.Geyer AT Avnet DOT com>
> To: Kathryn Hemness <kfhemness AT ucdavis DOT edu>
> Subject: RE: [Veritas-bu] NB51MP3 high cpu loads caused by bpdm?
>
> Kathy,
>
> Excellent news.  Please let me know how your weekend backups turn out 
> with your new settings.  I'm going to review my setup next week and go

> back to experimenting with these settings.
>
> Thanks!
>
> Greg
>
> -----Original Message-----
> From: cckat AT reality.ucdavis DOT edu [mailto:cckat AT reality.ucdavis DOT 
> edu] On 
> Behalf Of Kathryn Hemness
> Sent: Friday, July 08, 2005 10:22 AM
> To: Geyer, Gregory
> Cc: veritas-bu AT mailman.eng.auburn DOT edu; Bill_Jorgensen AT csgsystems 
> DOT com
> Subject: RE: [Veritas-bu] NB51MP3 high cpu loads caused by bpdm?
>
> Hi Gregory --
>
> I made the following changes on my server and the number of 'waited 
> for full buffer' messages in my bpdm log decreased by half.  Neither 
> the cpu load nor the 'waited for empty buffer'
> showed any significant change:
>
> /usr/openv/netbackup/db/config/NUMBER_DATA_BUFFERS_DISK:     16
> /usr/openv/netbackup/db/config/SIZE_DATA_BUFFERS_DISK:  1048576
> /etc/system: set shmsys:shminfo_shmmax=536870912
>
>
> I left the tape data buffers the same:
> /usr/openv/netbackup/db/config/NUMBER_DATA_BUFFERS 16 
> /usr/openv/netbackup/db/config/SIZE_DATA_BUFFERS 262144
>
> And my storage unit fragment size was still 2000 MB.
>
> For my backups tonight, I plan on increasing the 
> NUMBER_DATA_BUFFERS_DISK to 24 and decreasing the fragment size to 
> 1000 MB.
>
>
> On Wed, 6 Jul 2005, Geyer, Gregory wrote:
>
> > Date: Wed, 6 Jul 2005 15:52:41 -0700
> > From: "Geyer, Gregory" <Gregory.Geyer AT Avnet DOT com>
> > To: Kathryn Hemness <kfhemness AT ucdavis DOT edu>
> > Cc: veritas-bu AT mailman.eng.auburn DOT edu
> > Subject: RE: [Veritas-bu] NB51MP3 high cpu loads caused by bpdm?
> >
> > Kathy,
> >
> > Not sure, I've only experimented with them lately but I think it's a

> > case of trying a setting and watching and then perhaps tweaking them

> > until you've found a good spot.  I'm in your boat as well: still 
> > looking for the best settings.
> >
> > G.
> >
> > -----Original Message-----
> > From: cckat AT reality.ucdavis DOT edu [mailto:cckat AT reality.ucdavis DOT 
> > edu] 
> > On Behalf Of Kathryn Hemness
> > Sent: Wednesday, July 06, 2005 3:45 PM
> > To: Geyer, Gregory
> > Cc: veritas-bu AT mailman.eng.auburn DOT edu
> > Subject: RE: [Veritas-bu] NB51MP3 high cpu loads caused by bpdm?
> >
> > Thanks, Gregory -
> >
> > Is there a good rule of thumb for the number of disk data buffers 
> > based on the size of the storage unit?  I've set the number to 16 
> > for my backups tonight.  Is the number of disk data buffers supposed

> > to be
>
> > a power of 2 like the non-disk NUMBER_DATA_BUFFERS?
> >
> > On Wed, 6 Jul 2005, Geyer, Gregory wrote:
> >
> > > Date: Wed, 6 Jul 2005 15:31:17 -0700
> > > From: "Geyer, Gregory" <Gregory.Geyer AT Avnet DOT com>
> > > To: Kathryn Hemness <kfhemness AT ucdavis DOT edu>
> > > Cc: veritas-bu AT mailman.eng.auburn DOT edu
> > > Subject: RE: [Veritas-bu] NB51MP3 high cpu loads caused by bpdm?
> > >
> > > A trick a Veritas instructor gave me last year was this:
> > >
> > > grep out "waited for full buffer" and "waited for empty buffer" 
> > > out of
> >
> > > your bpdm/bptm logs.
> > >
> > > If you see many "waited for full buffer" in your bpdm/bptm log 
> > > then the storage unit disk/tape is waiting for the data buffer to
fill.
> > > Tune down the size of the data buffer so it fills quicker.
> > >
> > > If you see many "waited for empty buffer" then the server is 
> > > waiting
>
> > > on the storage unit.  Increase the number of buffers.
> > >
> > > G.
> > >
> > > -----Original Message-----
> > > From: cckat AT reality.ucdavis DOT edu [mailto:cckat AT reality.ucdavis 
> > > DOT edu]
> > > On Behalf Of Kathryn Hemness
> > > Sent: Wednesday, July 06, 2005 3:18 PM
> > > To: Geyer, Gregory
> > > Cc: Kathryn Hemness; veritas-bu AT mailman.eng.auburn DOT edu
> > > Subject: RE: [Veritas-bu] NB51MP3 high cpu loads caused by bpdm?
> > >
> > > Thanks.  I'll give the DISK BUFFER settings a try.  I was kind of 
> > > wondering if a smaller buffer size might be better.
> > >
> > >
> > > On Wed, 6 Jul 2005, Geyer, Gregory wrote:
> > >
> > > > Date: Wed, 6 Jul 2005 15:00:09 -0700
> > > > From: "Geyer, Gregory" <Gregory.Geyer AT Avnet DOT com>
> > > > To: Kathryn Hemness <kfhemness AT ucdavis DOT edu>,
> > > >      veritas-bu AT mailman.eng.auburn DOT edu
> > > > Subject: RE: [Veritas-bu] NB51MP3 high cpu loads caused by bpdm?
> > > >
> > > > Kathy,
> > > >
> > > > I believe SIZE_DATA_BUFFERS and NUMBER_DATA_BUFFERS are for bptm
> > only.
> > > > I've only just discovered "SIZE_DATA_BUFFERS_DISK" and 
> > > > "NUMBER_DATA_BUFFERS_DISK" for use by bpdm.
> > > >
> > > > However I also experienced shared memory issues with these as 
> > > > we're a heavy disk-staging shop.  These might help your speeds, 
> > > > though I don't
> > >
> > > > know about offloading your CPU.
> > > >
> > > > Reference Document ID 273532 for the above info.
> > > >
> > > > G.
> > > >
> > > > -----Original Message-----
> > > > From: veritas-bu-admin AT mailman.eng.auburn DOT edu
> > > > [mailto:veritas-bu-admin AT mailman.eng.auburn DOT edu] On Behalf Of 
> > > > Kathryn Hemness
> > > > Sent: Wednesday, July 06, 2005 2:38 PM
> > > > To: veritas-bu AT mailman.eng.auburn DOT edu
> > > > Subject: [Veritas-bu] NB51MP3 high cpu loads caused by bpdm?
> > > >
> > > > Greetings --
> > > >
> > > > I am running NetBackup 5.1 MP3 on a Sun V240 and using Sun 
> > > > StorEdge
> > > > 3511 storage arrays with 6TB capacity for backups to disk (not 
> > > > using
> >
> > > > the disk-staging configuration).  The OS is Solaris 9.
> > > > The backups are first written to the storage array and then 
> > > > staged
>
> > > > to tape using cron-script managed duplications so that I can use

> > > > backupid-files containing backupids having the same retentions 
> > > > to minimize tape mounts.  My tape library is a Qualstar 88264 
> > > > LTO2 library with 3 LTO2 drives.
> > > >
> > > > To optimize read and write speeds the storage arrays, I 
> > > > alternate directing all active backups to one storage array 
> > > > while staging diskimages to tape from the other storage array.  
> > > > I'm able to alternate the active backup storage unit daily.
> > > >
> > > > In order to complete all of the backups during typical 
> > > > non-workday
>
> > > > hours, I've set NUMBER_DATA_BUFFERS to 24 and SIZE_DATA_BUFFERS 
> > > > to
>
> > > > 262144.
> > > >
> > > > The backups are finishing in their configured windows, but the 
> > > > CPU
>
> > > > load is extremely high. The uptime commands shows a load of 
> > > > 25-40 all during my backup windows and top shows the bpdm 
> > > > processes as using
> > > > 1.0-1.4 percent of the CPU.
> > > >
> > > > I'm curious to see if there are others seeing high loads caused 
> > > > by
>
> > > > bpdm whis is used for both disk and disk-staging storage units.
> > > > I'd like to hear about how others are configuring NetBackup for 
> > > > backups to disk, mostly regarding the size of the disk storage 
> > > > units, NUMBER_DATA_BUFFERS, SIZE_DATA_BUFFERS, and number of
> > concurrent jobs.
> > > >
> > > > Thanks for any information you can give me.
> > > >
> > > > --kathy
> > > >
> > > > _______________________________________________
> > > > Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu 
> > > > http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
> > > >
> > >
> > > --kathy
> > >
> >
> > --kathy
> >
>
> --kathy
>
> ======================================================================
> ==
> =======
> Kathryn Hemness                        kfhemness AT ucdavis DOT edu
> System Administrator                   phone: 530.752.6547
> Campus Data Center & Client Services   fax:   530.752.9154
>

--kathy

========================================================================
=======
Kathryn Hemness                        kfhemness AT ucdavis DOT edu
System Administrator                   phone: 530.752.6547
Campus Data Center & Client Services   fax:   530.752.9154
_______________________________________________
Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu


<Prev in Thread] Current Thread [Next in Thread>