Re: [Networker] adv_file read-only device dismounting itself

We have seen similar behaviour. Strangely in networker 7.22.
Not only did our RO buffers unmount we also have witnessed our read/write
buffers unmount.
Because we use 3 to 5 buffers physically for the same pool luckily our
backups keep running but restores and staging get into trouble in the day..
As of yet we have no solution or even a recognizable cause for this
phenomenon.
For some unexplicable reason we have not witnessed this behaviour for one ot
two months now. We have upgraded our diskbuffer segment sizes to a higher
setting, but can prove no correlation between the two.
Hope you find a solution.

-----Original Message-----
From: EMC NetWorker discussion [mailto:NETWORKER AT LISTSERV.TEMPLE DOT EDU] On
Behalf Of Curtis Preston
Sent: 31 March 2008 23:49
To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
Subject: Re: [Networker] adv_file read-only device dismounting itself

Dude, that's a serious batch file!

---
W. Curtis Preston
Backup Blog @ www.backupcentral.com
VP Data Protection, GlassHouse Technologies 

> -----Original Message-----
> From: EMC NetWorker discussion [mailto:NETWORKER AT LISTSERV.TEMPLE DOT EDU]
On
> Behalf Of Mathew Harvest
> Sent: Thursday, March 27, 2008 6:50 PM
> To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
> Subject: Re: [Networker] adv_file read-only device dismounting itself
> 
> Hey Lee,
> 
> 
> 
> We have been seeing this problem for a while now (running 7.3.x on
> windows) although it has in the past been happening a lot more often
> than every few months ... at the moment its probably once a week ...
for
> a while it was several RO devices a day ...
> 
> 
> 
> we have a case logged with EMC at moment and are waiting on a debug
> binary for NSRD for our version of NetWorker ... we-ve been providing
> them with various debug logs, but the problem has been a little hard
to
> pin down ...
> 
> 
> 
> in the mean time I have a script that runs every half hour that
> re-mounts any RO devices that have unmounted ... it's a dos batch file
> ... but it would be fairly trivial to convert it to a Unix script ...
> 
> 
> 
> its listed below, and it probably overly complex, but it provides
> reporting on status before and after re-mount attempts ...
> 
> 
> 
> @echo off
> 
> set UNIX=c:\Tools\unix\usr\local\wbin
> 
> set EMC=n:\Legato\nsr\bin
> 
> set SCRIPT=D:\Scripts\NSR
> 
> set [email protected]
> 
> 
> 
> del %temp%\AFDMonitor.out 2>&1 >nul
> 
> del %temp%\AFDMonitor2.out 2>&1 >nul
> 
> 
> 
> rem this line will remove the quotes from around the output ... this
> isnt really necessary at the moment
> 
> rem %UNIX%\echo show name;volume name\nprint type:NSR Device;media
type:
> adv_file;read only: Yes | %EMC%\nsradmin -i - | %UNIX%\gawk
> "{name=$2;getline;if ($3==\";\") {print name};getline}" | %UNIX%\sed
> "s/\\\\/\\/g" | %UNIX%\sed  "s/;//g" | gawk -F\" -F\" "{print $2}"
> >%TEMP%\AFDMonitor.out
> 
> 
> 
> %UNIX%\echo show name;volume name\nprint type:NSR Device;media type:
> adv_file;read only: Yes | %EMC%\nsradmin -i - | %UNIX%\gawk
> "{name=$2;getline;if ($3==\";\") {print name};getline}" | %UNIX%\sed
> "s/\\\\/\\/g" | %UNIX%\sed  "s/;//g" >%TEMP%\AFDMonitor.out
> 
> 
> 
> if NOT "%1" == "NOMOUNT" (
> 
>       for /F "usebackq" %%i in (`type %temp%\AFDMonitor.out`) do nsrmm
> -m -f %%i 2>&1 >>%temp%\AFDMonitor2.out
> 
> )
> 
> 
> 
> 
> 
> set lines=0
> 
> type %temp%\AFDMonitor.out | %UNIX%\wc -l >%temp%\AFDMonitor3.out
> 
> for /F "usebackq" %%i in (`type %temp%\AFDMonitor3.out`) do set
> lines=%%i
> 
> del %temp%\AFDMonitor3.out
> 
> 
> 
> 
> 
> if NOT "%1" == "NOMOUNT" (
> 
>       del %TEMP%\AFDMonitor.mail.out
> 
>       echo. >%TEMP%\AFDMonitor.mail.out
> 
>       echo %date% - %time% >>%TEMP%\AFDMonitor.mail.out
> 
>       echo. >>%TEMP%\AFDMonitor.mail.out
> 
>       set heading=Unmounted Devices
> 
> ) else (
> 
>       set heading=POST Check - Unmounted Devices
> 
> )
> 
> 
> 
> if /I %lines% GTR 0 (
> 
>       echo %heading% >>%TEMP%\AFDMonitor.mail.out
> 
>       echo -----------------------------------
> >>%TEMP%\AFDMonitor.mail.out
> 
>       type %TEMP%\AFDMonitor.out >>%TEMP%\AFDMonitor.mail.out
> 
>       echo.
> 
>       if NOT "%1" == "NOMOUNT" (
> 
>             echo. >>%TEMP%\AFDMonitor.mail.out
> 
>             echo Result of Mount Attempt >>%TEMP%\AFDMonitor.mail.out
> 
>             echo -----------------------------------
> >>%TEMP%\AFDMonitor.mail.out
> 
>             type %TEMP%\AFDMonitor2.out >>%TEMP%\AFDMonitor.mail.out
> 
>             echo. >>%TEMP%\AFDMonitor.mail.out
> 
> 
> 
>             call %SCRIPT%\AFDMountMonitor.bat NOMOUNT
> 
>             blat %TEMP%\AFDMonitor.mail.out -to %MailTo% -s
> "AFDMountMonitor" 2>&1 >nul
> 
>       )
> 
> ) else (
> 
>       echo %heading% >>%TEMP%\AFDMonitor.mail.out
> 
>       echo -----------------------------------
> >>%TEMP%\AFDMonitor.mail.out
> 
>       echo No AFD's unmounted >>%TEMP%\AFDMonitor.mail.out
> 
> )
> 
> 
> 
> if NOT "%1" == "NOMOUNT" (
> 
>       type %TEMP%\AFDMonitor.mail.out >>%SCRIPT%\AFDMountMonitor.log
> 
> )
> 
> 
> 
> 
> 
> Mat.
> 
> 
> 
> 
> 
> -----Original Message-----
> From: EMC NetWorker discussion [mailto:NETWORKER AT LISTSERV.TEMPLE DOT EDU]
On
> Behalf Of Lee Roth
> Sent: Friday, 28 March 2008 11:34 AM
> To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
> Subject: [Networker] adv_file read-only device dismounting itself
> 
> 
> 
> Networker for Unix (Solaris) V7.4
> 
> 
> 
> For the 2nd time in six months, I've had a very strange occurrence: My
> 
> "read-only" (RO) adv_file disk device decides to dismount itself in
the
> 
> middle of the night, causing serious havoc (read: logjam next morning)
> on
> 
> the remaining backups, cloning, staging, etc.
> 
> 
> 
> The smoking gun can be seen on the system console:
> 
> 
> 
> [ID 702911 daemon.notice] NetWorker media: (waiting) waiting for
> adv_file
> 
> disk DiskStorage.RO on BackupServ1
> 
> 
> 
> Here is what is seen in the Networker device display when it is
failed:
> 
> 
> 
> Name                   Volume Name
> 
> /bigdisk               DiskStorage
> 
> /bigdisk/_AF_readonly
> 
> 
> 
> Here is what is seen in the Networker device display after I manually
> 
> remount the RO disk device:
> 
> 
> 
> Name                   Volume Name
> 
> /bigdisk               DiskStorage
> 
> /bigdisk/_AF_readonly  DiskStorage.RO
> 
> 
> 
> Its status is properly set to ENABLED and READ ONLY in its properties.
> 
> 
> 
> ------
> 
> Nobody is doing this manually, it is doing it all by itself. I've
> scanned
> 
> the EMC support database and have yet to find any mention of this kind
> of
> 
> behavior.
> 
> 
> 
> Before I report it to EMC support, I was wondering if anyone on this
> list
> 
> had any hints or suggestions as to what might be happening?
> 
> 
> 
> Thanks!
> 
> 
> 
> To sign off this list, send email to listserv AT listserv.temple DOT edu and
> type "signoff networker" in the body of the email. Please write to
> networker-request AT listserv.temple DOT edu if you have any problems with
this
> list. You can access the archives at
> http://listserv.temple.edu/archives/networker.html or
> 
> via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER
> 
> 
> 
> 
> 
> ********************************* DISCLAIMER
> *********************************
> The information contained in the above e-mail message or messages
(which
> includes any attachments) is confidential and may be legally
privileged.
> It is intended only for the use of the person or entity to which it is
> addressed.  If you are not the addressee any form of disclosure,
copying,
> modification, distribution or any action taken or omitted in reliance
on
> the information is unauthorised.  Opinions contained in the message(s)
do
> not necessarily reflect the opinions of the Queensland Government and
its
> authorities.  If you received this communication in error, please
notify
> the sender immediately and delete it from your computer system
network.
> 
> 
> 
> 
> To sign off this list, send email to listserv AT listserv.temple DOT edu and
type
> "signoff networker" in the body of the email. Please write to
networker-
> request AT listserv.temple DOT edu if you have any problems with this list.
You
> can access the archives at
> http://listserv.temple.edu/archives/networker.html or
> via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER

To sign off this list, send email to listserv AT listserv.temple DOT edu and 
type
"signoff networker" in the body of the email. Please write to
networker-request AT listserv.temple DOT edu if you have any problems with this
list. You can access the archives at
http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER

To sign off this list, send email to listserv AT listserv.temple DOT edu and 
type "signoff networker" in the body of the email. Please write to 
networker-request AT listserv.temple DOT edu if you have any problems with this 
list. You can access the archives at 
http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER