cwilloug
ADSM.ORG Senior Member
- Joined
- Sep 13, 2006
- Messages
- 388
- Reaction score
- 11
- Points
- 0
- Location
- North Dakota
- Website
- Visit site
I have 2 IBM N3600 NAS, each with 6 paths via a switch zoned to tape drives in my TS3500 library, backing up with NDMP. Things have been running smoothly since I figured out the NDMP backup configs, until yesterday. Yesterday morning I came into work and found that one of the paths to the library drives had been taken offline. I checked the connections to the NAS, Switch, and library - all ok, so I used the ISC to place the path back on-line.
This morning I came into work, and the same path, same drive, was offline, a quick search of the actlog found......
06/16/2009 19:05:19 ANR8471E Server no longer polling drive DRIVE08 in library
3500LIB - path /dev/rmt13 will be marked off-line.
(SESSION: 61712, PROCESS: 481)
06/16/2009 19:05:19 ANR8873E The path from source N3600FS2 to destination
DRIVE08 (/dev/rmt13) is taken offline. (SESSION: 61712,
PROCESS: 481)
Then a look at the errpt on my TSM AIX box found.....
IDENTIFIER TIMESTAMP T C RESOURCE_NAME DESCRIPTION
476B351D 0616190309 P H rmt13 TAPE DRIVE FAILURE
A7AB4C8F 0616190309 I H rmt13 TAPE SIM/MIM RECORD
476B351D 0616190309 P H rmt13 TAPE DRIVE FAILURE
A7AB4C8F 0616190309 I H rmt13 TAPE SIM/MIM RECORD
476B351D 0616183309 P H rmt13 TAPE DRIVE FAILURE
476B351D 0616183309 P H rmt13 TAPE DRIVE FAILURE
A7AB4C8F 0616052909 I H rmt11 TAPE SIM/MIM RECORD
A7AB4C8F 0616052509 I H rmt11 TAPE SIM/MIM RECORD
E507DCF9 0616052509 I H rmt11 TAPE DRIVE NEEDS CLEANING
A7AB4C8F 0616015709 I H rmt17 TAPE SIM/MIM RECORD
Wow, a tape drive failure,,, hummmm.. but if there was a tape drive failure, why was only the path, and not the drive taken off-line?
tsm: TSM>q path
Source Name Source Type Destination Destination On-Line
Name Type
----------- ----------- ----------- ----------- -------
TSM SERVER 3500LIB LIBRARY Yes
TSM SERVER DRIVE01 DRIVE Yes
TSM SERVER DRIVE02 DRIVE Yes
TSM SERVER DRIVE03 DRIVE Yes
TSM SERVER DRIVE04 DRIVE Yes
TSM SERVER DRIVE05 DRIVE Yes
TSM SERVER DRIVE06 DRIVE Yes
TSM SERVER DRIVE07 DRIVE Yes
TSM SERVER DRIVE08 DRIVE Yes
N3600FS2 DATAMOVER DRIVE02 DRIVE Yes
N3600FS2 DATAMOVER DRIVE03 DRIVE Yes
N3600FS2 DATAMOVER DRIVE04 DRIVE Yes
N3600FS2 DATAMOVER DRIVE06 DRIVE Yes
N3600FS2 DATAMOVER DRIVE07 DRIVE Yes
N3600FS2 DATAMOVER DRIVE08 DRIVE No
N3600_FS1 DATAMOVER DRIVE01 DRIVE Yes
N3600_FS1 DATAMOVER DRIVE02 DRIVE Yes
N3600_FS1 DATAMOVER DRIVE03 DRIVE Yes
N3600_FS1 DATAMOVER DRIVE05 DRIVE Yes
N3600_FS1 DATAMOVER DRIVE06 DRIVE Yes
N3600_FS1 DATAMOVER DRIVE07 DRIVE Yes
tsm: TSM>q drive
Library Name Drive Name Device Type On-Line
------------ ------------ ----------- -------------------
3500LIB DRIVE01 3592 Yes
3500LIB DRIVE02 3592 Yes
3500LIB DRIVE03 3592 Yes
3500LIB DRIVE04 3592 Yes
3500LIB DRIVE05 3592 Yes
3500LIB DRIVE06 3592 Yes
3500LIB DRIVE07 3592 Yes
3500LIB DRIVE08 3592 Yes
Looking at the Library drive errors, I did find an error on Drive08 with a tape that is not used for the NAS Storage Pool at the time the path went off-line.
So why would an error, with a tape not in the NAS pool, cause the NAS drive path to go off-line, and not the drive?
Anyone?
This morning I came into work, and the same path, same drive, was offline, a quick search of the actlog found......
06/16/2009 19:05:19 ANR8471E Server no longer polling drive DRIVE08 in library
3500LIB - path /dev/rmt13 will be marked off-line.
(SESSION: 61712, PROCESS: 481)
06/16/2009 19:05:19 ANR8873E The path from source N3600FS2 to destination
DRIVE08 (/dev/rmt13) is taken offline. (SESSION: 61712,
PROCESS: 481)
Then a look at the errpt on my TSM AIX box found.....
IDENTIFIER TIMESTAMP T C RESOURCE_NAME DESCRIPTION
476B351D 0616190309 P H rmt13 TAPE DRIVE FAILURE
A7AB4C8F 0616190309 I H rmt13 TAPE SIM/MIM RECORD
476B351D 0616190309 P H rmt13 TAPE DRIVE FAILURE
A7AB4C8F 0616190309 I H rmt13 TAPE SIM/MIM RECORD
476B351D 0616183309 P H rmt13 TAPE DRIVE FAILURE
476B351D 0616183309 P H rmt13 TAPE DRIVE FAILURE
A7AB4C8F 0616052909 I H rmt11 TAPE SIM/MIM RECORD
A7AB4C8F 0616052509 I H rmt11 TAPE SIM/MIM RECORD
E507DCF9 0616052509 I H rmt11 TAPE DRIVE NEEDS CLEANING
A7AB4C8F 0616015709 I H rmt17 TAPE SIM/MIM RECORD
Wow, a tape drive failure,,, hummmm.. but if there was a tape drive failure, why was only the path, and not the drive taken off-line?
tsm: TSM>q path
Source Name Source Type Destination Destination On-Line
Name Type
----------- ----------- ----------- ----------- -------
TSM SERVER 3500LIB LIBRARY Yes
TSM SERVER DRIVE01 DRIVE Yes
TSM SERVER DRIVE02 DRIVE Yes
TSM SERVER DRIVE03 DRIVE Yes
TSM SERVER DRIVE04 DRIVE Yes
TSM SERVER DRIVE05 DRIVE Yes
TSM SERVER DRIVE06 DRIVE Yes
TSM SERVER DRIVE07 DRIVE Yes
TSM SERVER DRIVE08 DRIVE Yes
N3600FS2 DATAMOVER DRIVE02 DRIVE Yes
N3600FS2 DATAMOVER DRIVE03 DRIVE Yes
N3600FS2 DATAMOVER DRIVE04 DRIVE Yes
N3600FS2 DATAMOVER DRIVE06 DRIVE Yes
N3600FS2 DATAMOVER DRIVE07 DRIVE Yes
N3600FS2 DATAMOVER DRIVE08 DRIVE No
N3600_FS1 DATAMOVER DRIVE01 DRIVE Yes
N3600_FS1 DATAMOVER DRIVE02 DRIVE Yes
N3600_FS1 DATAMOVER DRIVE03 DRIVE Yes
N3600_FS1 DATAMOVER DRIVE05 DRIVE Yes
N3600_FS1 DATAMOVER DRIVE06 DRIVE Yes
N3600_FS1 DATAMOVER DRIVE07 DRIVE Yes
tsm: TSM>q drive
Library Name Drive Name Device Type On-Line
------------ ------------ ----------- -------------------
3500LIB DRIVE01 3592 Yes
3500LIB DRIVE02 3592 Yes
3500LIB DRIVE03 3592 Yes
3500LIB DRIVE04 3592 Yes
3500LIB DRIVE05 3592 Yes
3500LIB DRIVE06 3592 Yes
3500LIB DRIVE07 3592 Yes
3500LIB DRIVE08 3592 Yes
Looking at the Library drive errors, I did find an error on Drive08 with a tape that is not used for the NAS Storage Pool at the time the path went off-line.
So why would an error, with a tape not in the NAS pool, cause the NAS drive path to go off-line, and not the drive?
Anyone?