Hi all,
Anyone out there had intermittent NDMP backup problems.
Recently moved NAS data from EMC NS704G to new VNX 5500. New switches also using DS300B as opposed to old Mcdata switches.
My setup is as it was before as far as TSM is concerned, using same library/drives, same paths, same TSM NAS data movers, etc, just updating with new IP addresses, etc.
Sometimes backups work ok, sometimes they fail with:-
18-12-2012 00:19:28 ANR9999D_2547733210 (ndlog.c:120) Thread<41>: NDMPLOG: E
Medium error(SESSION: 48445, PROCESS: 1802)
18-12-2012 00:19:28 ANR9999D Thread<41> issued message 9999 from: (SESSION:
48445, PROCESS: 1802)
18-12-2012 00:19:33 ANR0515I Process 1802 closed volume DB1722L3. (SESSION:
48445, PROCESS: 1802)
18-12-2012 00:19:33 ANR8336I Verifying label of NAS volume DB1722L3 in drive
LTO3DRV3 (c48t0l0). (SESSION: 48445, PROCESS: 1802)
18-12-2012 00:20:33 ANR8468I NAS volume DB1722L3 dismounted from drive
LTO3DRV3 (c48t0l0) in library B_3584LTO. (SESSION: 48445,
PROCESS: 1802)
18-12-2012 00:20:33 ANR1104E NAS Backup process 1802 terminated - NDMP session
errors encountered. (SESSION: 48445, PROCESS: 1802)
18-12-2012 00:20:33 ANR0988I Process 1802 for BACKUP NAS (DIFFERENTIAL)
running in the BACKGROUND processed 2,427,453,440 bytes
with a completion state of FAILURE at 00:20:33. (SESSION:
48445, PROCESS: 1802)
If we keep on trying the backup it eventually works. Makes no difference what drive or what tape we use, we get the same problem.
Sometimes the backup fails fairly quickly other times it will backup (for example) 120 GB of 150Gb and then fail.
Get similar problem with backup stgpool of these NDMP backup tapes:-
17-12-2012 15:13:40 ANR1218E BACKUP STGPOOL: Process 1793 terminated -
excessive read errors encountered. (SESSION: 48043,
PROCESS: 1793)
17-12-2012 15:13:40 ANR0986I Process 1793 for BACKUP STORAGE POOL running in
the BACKGROUND processed 2 items for a total of
34,228,563,258 bytes with a completion state of FAILURE
We do not get any problems with other type of 'normal' backups/archives. So we have counted out issues with drives/tapes.
Strange thing is, we have three sites all using the same equipment, all gone through the same NAS migration, yet we do not have any of these issues at those sites!!!
Also we have an old NS80G attached to same tape library (partitioned with each partition having own drives) and when I connect one of the VNX data movers to that via an old Brocade switch I can';t get the backups to fail????.
Any ideas, anyone had similar issues that they have managed to resolve.??
Anyone out there had intermittent NDMP backup problems.
Recently moved NAS data from EMC NS704G to new VNX 5500. New switches also using DS300B as opposed to old Mcdata switches.
My setup is as it was before as far as TSM is concerned, using same library/drives, same paths, same TSM NAS data movers, etc, just updating with new IP addresses, etc.
Sometimes backups work ok, sometimes they fail with:-
18-12-2012 00:19:28 ANR9999D_2547733210 (ndlog.c:120) Thread<41>: NDMPLOG: E
Medium error(SESSION: 48445, PROCESS: 1802)
18-12-2012 00:19:28 ANR9999D Thread<41> issued message 9999 from: (SESSION:
48445, PROCESS: 1802)
18-12-2012 00:19:33 ANR0515I Process 1802 closed volume DB1722L3. (SESSION:
48445, PROCESS: 1802)
18-12-2012 00:19:33 ANR8336I Verifying label of NAS volume DB1722L3 in drive
LTO3DRV3 (c48t0l0). (SESSION: 48445, PROCESS: 1802)
18-12-2012 00:20:33 ANR8468I NAS volume DB1722L3 dismounted from drive
LTO3DRV3 (c48t0l0) in library B_3584LTO. (SESSION: 48445,
PROCESS: 1802)
18-12-2012 00:20:33 ANR1104E NAS Backup process 1802 terminated - NDMP session
errors encountered. (SESSION: 48445, PROCESS: 1802)
18-12-2012 00:20:33 ANR0988I Process 1802 for BACKUP NAS (DIFFERENTIAL)
running in the BACKGROUND processed 2,427,453,440 bytes
with a completion state of FAILURE at 00:20:33. (SESSION:
48445, PROCESS: 1802)
If we keep on trying the backup it eventually works. Makes no difference what drive or what tape we use, we get the same problem.
Sometimes the backup fails fairly quickly other times it will backup (for example) 120 GB of 150Gb and then fail.
Get similar problem with backup stgpool of these NDMP backup tapes:-
17-12-2012 15:13:40 ANR1218E BACKUP STGPOOL: Process 1793 terminated -
excessive read errors encountered. (SESSION: 48043,
PROCESS: 1793)
17-12-2012 15:13:40 ANR0986I Process 1793 for BACKUP STORAGE POOL running in
the BACKGROUND processed 2 items for a total of
34,228,563,258 bytes with a completion state of FAILURE
We do not get any problems with other type of 'normal' backups/archives. So we have counted out issues with drives/tapes.
Strange thing is, we have three sites all using the same equipment, all gone through the same NAS migration, yet we do not have any of these issues at those sites!!!
Also we have an old NS80G attached to same tape library (partitioned with each partition having own drives) and when I connect one of the VNX data movers to that via an old Brocade switch I can';t get the backups to fail????.
Any ideas, anyone had similar issues that they have managed to resolve.??
