Networker

[Networker] Extremely slow, but error-free, backups on a particular drive... ...anyone?

2010-08-03 15:59:06
Subject: [Networker] Extremely slow, but error-free, backups on a particular drive... ...anyone?
From: Len Philpot <Len.Philpot AT CLECO DOT COM>
To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
Date: Tue, 3 Aug 2010 14:58:15 -0500
* Networker 7.5.1 (Sun) server on Solaris 9/SPARC

* Two libraries with six SuperDLT 320 drives in each, 
  all SCSI-attached; slowness is on one library only

* Most of the clients involved are Solaris 8, 9 and 10, all 
  SPARC, as well as a couple of Windows clients

The last couple of nights I've seen two different drive/tape combinations 
produce _EXTREMELY_ slow writes, but completely without error as far as I 
can tell. Once a tape is loaded in the drive it backs up, but at rates 
generally less than 20KB/sec per saveset. There are no errors in the 
daemon.log, none in /var/adm/messages, 'iostat -En' shows nothing wrong, 
'netstat -i 2' shows no collisions, etc. So far it's happened with one 
tape in one drive and a different tape in another over the last two backup 
runs.

Interesting thing on one (only) of these clients...

Since this client is a critical system with limited backup downtime, I 
actually killed its final 'savepnpc' process after it crawled for 6.5 
hours since the customers needed their apps up. When I did so Networker 
tried again _to_the_same_tape/drive_ and saved the filesystem in the 
typical 4 minutes or so - Full normal speed, in other words. Then again, 
the next backup session, that same saveset finished, but took almost two 
hours (on a different tape/drive). Once again, 3-5 minutes is typical. 
There were three other systems on the same tape/drive, so I killed 
savepnpc on each of them, hoping for similar results. They all restarted 
but no faster than before.

I patched the backup server (Solaris recommended cluster) last Wednesday 
afternoon; everything was fine that night and Thursday night, with the 
first issue showing up during Friday night's backups. There have been no 
changes other than the patch cluster.

I was thinking maybe a particular nsrmmd instance might get slow, but that 
doesn't fit with the speedy backup after the retry. Other than the saveset 
I mentioned above, it's all been different clients/savesets each time. No 
pattern other than all being on one library (one that's usually the least 
troublesome, naturally).

Anyone run ever into this? I have a feeling if this continues, it will be 
a troubleshooting nightmare...  :-\

Thanks!

To sign off this list, send email to listserv AT listserv.temple DOT edu and 
type "signoff networker" in the body of the email. Please write to 
networker-request AT listserv.temple DOT edu if you have any problems with this 
list. You can access the archives at 
http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER

<Prev in Thread] Current Thread [Next in Thread>
  • [Networker] Extremely slow, but error-free, backups on a particular drive... ...anyone?, Len Philpot <=