Networker

Re: [Networker] Linux Tape Drive Re-Ordering Issue

2010-11-09 07:18:11
Subject: Re: [Networker] Linux Tape Drive Re-Ordering Issue
From: Francis Swasey <Frank.Swasey AT UVM DOT EDU>
To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
Date: Tue, 9 Nov 2010 07:17:19 -0500
Hi Jee,

Sadly, it has everything to do with NetWorker these days.  Starting with 7.5 
NetWorker now
records the serial numbers from the drives in the jukebox and if it finds a 
mismatch (as it
will after you physically replace a drive that has worn out), it refuses to use 
that drive.

You have to perform a scan for new devices from the NMC on the system that sees 
the robotics of
the jukebox and then wait about 5 minutes for NetWorker to figure out that the 
serial number of
the drive has changed.

If you do not perform that scan from the NMC, NetWorker will load a tape into 
the drive and
then yell about the serial number mismatch and refuse to do anything further 
(so you have a
tape in a drive where it belongs and networker will not even unload the tape 
because the serial
number is wrong on the drive and NetWorker thinks you've got a drive ordering 
problem).

As I've recently had a run on physical drive replacements, I've become 
intimately familiar with
this new addition to the drive replacement procedure.

Therefore, NetWorker has to document this process because NetWorker is now 
integrally involved
in it.

EMC also owes us a way to perform the scan from the command line.  There are 
times (like when
you are using a VPN client) that the NMC java applet will not actually work, 
which is a large
pain in the backside when it happens...  My experience with this is using the 
Cisco VPN clients
(including the SSL AnyConnect client which UVM just rolled out).  However, I 
assume that any
daemon on my workstation/laptop that somehow inserts itself between the OS and 
the Hardware NIC
would cause the same problem.  And the only way I can figure this is a problem 
is that the NMC
java applet was written to tickle the Hardware directly instead of going 
through the OS for its
network traffic -- which, as far as I'm concerned is violating programming 101.

Frank

On 11/8/10 2:42 PM, jee wrote:
> Hi Frank,
> 
> If by "replace" you mean "physically replace", then don't look for that in 
> the 
> netwoker documentation. It won't cover that because that has nothing  to do 
> with networker troubleshooting and/or administration.  
> 
> You should be able to find that in formation in the corresponding service 
> (hardware) manuals.
> 
> 
> jee
> 
> 
> 
> On Monday 08 November 2010 13:31:29 Francis Swasey wrote:
>> Brian,
>>
>> Thanks for that pointer.  Sadly, it doesn't cover how to replace a failed
>> tape drive in a library at all (at least there is not one single use of the
>> word "replace" in the entire document).  I'll peruse it in depth later...
>>
>> Frank
>>
>> On 11/5/10 3:19 PM, Brian Narkinsky wrote:
>>> You have to dig through PowerLink a bit to find this but there is a
>>> pretty good paper on how to configure Tape Drives in Networker.  Lot of
>>> good info in here.
>>>
>>> EMC® NetWorker®
>>> Configuring Tape Devices for EMC NetWorker
>>> Technical Note
>>> P/N 300-008-352
>>> REV A04
>>>
>>> On Fri, Nov 5, 2010 at 9:24 AM, Francis Swasey <Frank.Swasey AT uvm DOT 
>>> edu> 
> wrote:
>>>> Have you tested what happens when a tape drive fails and you have to
>>>> replace it with a new one
>>>> (which has a different serial number) doing it that way?
>>>>
>>>> For those of us that created our jukebox back in the 7.2 (or earlier)
>>>> days and have the jukebox
>>>> defined using the rd=<storagenode>:/dev/nstX devices, I strongly urge
>>>> you to consider coercing
>>>> udev into picking which /dev/nstX device the drive is based on the
>>>> serial number (then
>>>> remembering to update that rule when you replace a drive).
>>>>
>>>> I have set it up as /etc/udev/rules.d/20-local.rules -- which is a copy
>>>> of the 50-udev.rules
>>>> file with a line added for each tape drive like:
>>>>
>>>> KERNEL=="nst[0-9]*", SUBSYSTEM=="scsi_tape",
>>>> ENV{ID_SERIAL}=="3500507631203e5d4", NAME="nst0",
>>>> SYMLINK+="tape/by-id/$env{ID_BUS}-$env{ID_SERIAL}-nst", SYMLINK+="tape0"
>>>>
>>>> just in front of the
>>>>
>>>> KERNEL=="nst[0-9]*", SUBSYSTEM=="scsi_tape", ENV{ID_SERIAL}=="?*",
>>>> SYMLINK+="tape/by-id/$env{ID_BUS}-$env{ID_SERIAL}-nst"
>>>>
>>>> line.
>>>>
>>>> Why would I do something this convoluted?  Because I don't like the risk
>>>> of having to perform a
>>>> 25 hour jukebox inventory just because a tape drive failed and suddenly
>>>> I have to re-define the
>>>> jukebox because EMC's code messed up and can't figure out what happened
>>>> to that tape drive that
>>>> broke and was removed.
>>>>
>>>> Frank
>>>>
>>>> On 11/5/10 8:43 AM, Riku Valli wrote:
>>>>> Matthew Huff wrote:
>>>>>> I'm not sure you understand persistent binding. With persistent
>>>>>> binding,
>>>>
>>>> no changes
>>>>
>>>>>> (including SCSI bus resets) will change the names (and therefore the
>>>>
>>>> drive order).
>>>>
>>>>> Yes, tape drives uses serialnumbers for devicenames at RHEL 5 for
>>>>
>>>> persistent bindings. So them
>>>>
>>>>> cannot change.
>>>>> /dev/tape/by-id/scsi-3500308c09fb49000-nst
>>>>>
>>>>> When you made jukebox, you should select use persistent names and NMC:s
>>>>
>>>> graphical gui uses
>>>>
>>>>> automatic this by-id name convention or manual with command jbconfig
>>>>> -p,
>>>>
>>>> but jbconfig is'nt
>>>>
>>>>> recommed to use at newer Networker versions.
>>>>
>>>> To sign off this list, send email to listserv AT listserv.temple DOT edu 
>>>> and
>>>> type "signoff networker" in the body of the email. Please write to
>>>> networker-request AT listserv.temple DOT edu if you have any problems with 
>>>> this
>>>> list. You can access the archives at
>>>> http://listserv.temple.edu/archives/networker.html or
>>>> via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER
>>>
>>> To sign off this list, send email to listserv AT listserv.temple DOT edu and
>>> type "signoff networker" in the body of the email. Please write to
>>> networker-request AT listserv.temple DOT edu if you have any problems with 
>>> this
>>> list. You can access the archives at
>>> http://listserv.temple.edu/archives/networker.html or via RSS at
>>> http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER
> 
> 

-- 
Frank Swasey                    | http://www.uvm.edu/~fcs
Sr Systems Administrator        | Always remember: You are UNIQUE,
University of Vermont           |    just like everyone else.
  "I am not young enough to know everything." - Oscar Wilde (1854-1900)

To sign off this list, send email to listserv AT listserv.temple DOT edu and 
type "signoff networker" in the body of the email. Please write to 
networker-request AT listserv.temple DOT edu if you have any problems with this 
list. You can access the archives at 
http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER

<Prev in Thread] Current Thread [Next in Thread>