How to use tsmtape.h

PREDATAR Control23

Hello Hema.

You have to build (compile) it on a linux machine, then mount the tape in a tape station that tsmtape can use..

Do you have the right Hardware, do youhave a linux machine running and can you hook up
tapedrive to it that can use your tapes?

Are you still in a bad situation?

If you have the HW i can do the compile, I need to know your linux distro and kernel level.

Good luck
/regards Daniel
 
PREDATAR Control23

Hi Master,

The TSM server is on windows and I don't have any linux machine...Is there anyway to use this tsmatape utility in windows box..?
 
PREDATAR Control23

I recently found that some archive tapes (kept in a safe) had been lost from TSM (it seems like a delete filespace command happened to erase archives as well as backups at some point). It was discovered six months later, so no DB backups were available anymore. I found the tsmtape utility which was really what I needed.

I've rewritten it a bit to work on 64-bit Linux, and also added a switch to be able to read data from newer tapes (not sure if the difference is because they were archived on TSM 5.5 or not - it needs to skip past 177 bytes of junk before the file data).

The data on the tapes were archived tar.gz-files, so it was easy to test that the files were not corrupt after restoring.

TSMtape works well, but I haven't tried restoring files split on several tapes yet.. I'll do that when I get the time.
 
PREDATAR Control23

Hi Master,

The TSM server is on windows and I don't have any linux machine...Is there anyway to use this tsmatape utility in windows box..?

Maybe the easiest fix would be to install Linux on some machine, connect it to the appropriate tape drive and restore the contents. You don't need TSM on that machine.
 
PREDATAR Control23

Hi to everybody, I have the same problem.. I need to recover data from an old tape (I haven't TSM DB) I already tried to use tsmtape but standard version didn't work with TSM5.5 tape (i think..). Please staham, can you tell to me how to modify a tsmtape code to read new tape?

Thank you,
Loppo
 
PREDATAR Control23

Hi to everybody, I have the same problem.. I need to recover data from an old tape (I haven't TSM DB) I already tried to use tsmtape but standard version didn't work with TSM5.5 tape (i think..). Please staham, can you tell to me how to modify a tsmtape code to read new tape?

Thank you,
Loppo

I'll send you the updated code. It works for me. Please tell me if it works for you as well.
 
PREDATAR Control23

TSMtape for 5.5

I'll send you the updated code. It works for me. Please tell me if it works for you as well.

I have a similiar issue, do you think you could send me the updated copy as well.
 
PREDATAR Control23

Here's a patch. No guarantees, but it works for me.

Most of the changes are for getting it to work on a 64-bit system (long => int, and so on). I've used it for retrieving archived files from tapes.
 

Attachments

  • tsmtape.patch.gz
    4.9 KB · Views: 152
Last edited:
PREDATAR Control23

Any idea of why I might be getting these errors?

I have a similiar issue, do you think you could send me the updated copy as well.


[root@nagprd06 tsmtape0.1.patch]# ./TSMtape --skipjunk --audit /dev/IBMtape0 /tmp/s00000.csv
Using device /dev/IBMtape0
Opening device /dev/IBMtape0
File descriptor: 5
File 0, Record 1: 80 bytes
ADSM_Magic=f1d3d6e5, expected '5a4d4e50' or '504e4d5a'
*** ADSM_Magic==f1d3d6e5, expected '5a4d4e50' or '504e4d5a' ***
Volume label: S00000
Positioning to data
File 1, Record 1: 262144 bytes
ADSM_TapeRecordHeader->recnum=5, expected 'fffffffe'
ADSM_SEFR=52464553, expected '53454652'
Forcing read of next record
File 1, Record 8: 262144 bytes
ADSM_SEFR=52464553, expected '53454652'
Forcing read of next record
File 1, Record 9: 262144 bytes
ADSM_TapeRecordHeader->recnum=5, expected '6'
ADSM_SEFR=52464553, expected '53454652'
Forcing read of next record

This just keeps repeating itself and gets stuck in the loop until I abort the job. I'm running the IBMtape utility on redhat linux enterprise. My TSM server is version 5.5.2.0. Any help would be appreciated.
 
PREDATAR Control23

[root@nagprd06 tsmtape0.1.patch]# ./TSMtape --skipjunk --audit /dev/IBMtape0 /tmp/s00000.csv
Using device /dev/IBMtape0
Opening device /dev/IBMtape0
File descriptor: 5
File 0, Record 1: 80 bytes
ADSM_Magic=f1d3d6e5, expected '5a4d4e50' or '504e4d5a'
*** ADSM_Magic==f1d3d6e5, expected '5a4d4e50' or '504e4d5a' ***
Volume label: S00000
Positioning to data
File 1, Record 1: 262144 bytes
ADSM_TapeRecordHeader->recnum=5, expected 'fffffffe'
ADSM_SEFR=52464553, expected '53454652'
Forcing read of next record
File 1, Record 8: 262144 bytes
ADSM_SEFR=52464553, expected '53454652'
Forcing read of next record
File 1, Record 9: 262144 bytes
ADSM_TapeRecordHeader->recnum=5, expected '6'
ADSM_SEFR=52464553, expected '53454652'
Forcing read of next record

This just keeps repeating itself and gets stuck in the loop until I abort the job. I'm running the IBMtape utility on redhat linux enterprise. My TSM server is version 5.5.2.0. Any help would be appreciated.

Note the lines reading "ADSM_SEFR=52464553, expected '53454652'". This is the key.

This is byteswapped (or needs swapping). If you read the bytes in reverse order, you see that the numbers are the same.

You probably need to add the line "swapulong(&bh.SEFR);" before the line:
if (c(bh.SEFR,ADSM_SEFR,"ADSM_SEFR")) break;
in the file tsmtapefunc.c

There is a section right before that place that I've commented out:
// If magic is reversed then swap bytes
// if (byteswap) {

You can test if this might be needed on your platform.

I never figured out why some of those are needed and some are not. There are some comments by someone else that says that some of the byteswaps were not needed when he fixed it for TSM 5. For me, even more of those were necessary to comment out, but it might be my specific hardware / OS / TSM version.

If you enable the debug mode, you'll understand more of this.
 
  • Like
Reactions: BBB
PREDATAR Control23

I'm getting the following output:

Code:
# TSMtape -R /dev/st0 / /mnt/raid/tsm/restore/
Using device /dev/st0
Opening device /dev/st0
File descriptor: 5
 File 0, Record 1: 80 bytes
ADSM_Magic=f1d3d6e5, expected '5a4d4e50' or '504e4d5a'
*** ADSM_Magic==f1d3d6e5, expected '5a4d4e50' or '504e4d5a' ***
Volume label: CRC007
Positioning to data
 File 1, Record 1: 262144 bytes
ADSM_TapeRecordHeader->recnum=5, expected 'fffffffe'
ADSM_BlockHeader->bnum=13, expected '1'
Ignoring
OH.c04=d5, expected 'a5'
OH.type=25, expected 'c5' or '92'
OH.pad=12, expected '0'
CP.fspace.c1=9d, expected '1'
Unknown storage 25
 File 1, Record 8: 262144 bytes
 File 1, Record 9: 262144 bytes
ADSM_TapeRecordHeader->recnum=5, expected '6'
 File 1, Record 8: 262144 bytes
 File 1, Record 9: 262144 bytes
ADSM_TapeRecordHeader->recnum=5, expected '6'
 File 1, Record 8: 262144 bytes
 File 1, Record 9: 262144 bytes
ADSM_TapeRecordHeader->recnum=5, expected '6'
 File 1, Record 8: 262144 bytes
 File 1, Record 9: 262144 bytes
ADSM_TapeRecordHeader->recnum=5, expected '6'
 File 1, Record 8: 262144 bytes
 File 1, Record 9: 262144 bytes
ADSM_TapeRecordHeader->recnum=5, expected '6'
I'm looking into the code to see if I can figure it out, but nothing seems to jump out at me as yet... Anyone with more knowledge of this able to give me a few hints?

EDIT: Looking into this further, the second file, I get:
Code:
ADSM_SEFR=0, expected '53454652'
Forcing read of next record
 File 1, Record 8: 262144 bytes
OH.pad=3, expected '0'
Vasize: 297 Inlengt: 136
Storage is backup
Filetype is regular
B           -  6287360 SLinux86STANDARD/mnt/raid/data/MP3s/Songs/(DJ Talla 2XLC
 File 1, Record 9: 262144 bytes
ADSM_TapeRecordHeader->recnum=5, expected '6'
 File 1, Record 8: 262144 bytes
 File 1, Record 9: 262144 bytes
ADSM_TapeRecordHeader->recnum=5, expected '6'
 File 1, Record 8: 262144 bytes
 File 1, Record 9: 262144 bytes
ADSM_TapeRecordHeader->recnum=5, expected '6'

Seems like it's almost there... but it seems that this record header is causing the file not to be written?
 
Last edited:
PREDATAR Control23

Anybody with a clue as to tsmtapes operation care to assist? please? :)
 
PREDATAR Control23

Digging even deeper, I've gone back to the stock TSMtape0.1 (without the patch listed here) and managed to understand some of the following:

Code:
*** Readfromrecord(...,...,32) ***
 File 1, Record 8: 262144 bytes
ADSM_Magic=504e4d5a OK
ADSM_Magic reversed. Will swap bytes.
The value passed was 4 bytes long... value==5000000,returnval==5
The value passed was 4 bytes long... value==0,returnval==0
ADSM_TapeRecordHeader->recnum=5 OK
The value passed was 4 bytes long... value==53454652,returnval==52464553
The value passed was 2 bytes long... value==2,returnval==200
The value passed was 2 bytes long... value==20,returnval==2000
The value passed was 4 bytes long... value==20b7,returnval==b7200000
The value passed was 4 bytes long... value==3ffb0,returnval==b0ff0300
The value passed was 4 bytes long... value==0,returnval==0
The value passed was 4 bytes long... value==7f8d21,returnval==218d7f00
The value passed was 4 bytes long... value==0,returnval==0
The value passed was 4 bytes long... value==0,returnval==0
ADSM_SEFR=52464553, expected '53454652'
Forcing read of next record

Does anyone have a working TSMtape for their system to see if I can compare values at all and see what should and should not be byteswapped?

I can see that the 4 bytes 20b7 seems to be an offset of some sort - or a block number on the tape - as this increments by one each time - and would make sense NOT to be byteswapped - but I'm not too sure...
 
PREDATAR Control23

I'll send you the updated code. It works for me. Please tell me if it works for you as well.

I too seem to be having the exact results as CRCinAU. Im on a redhat machine. If you could possibly forward a copy of the updated code to me, I would appreciate it. Any further suggestions / explanations will also be helpful as I have not performed any coding in a LONG time.
 
PREDATAR Control23

Was this ever resolved

I'm having the EXACT same problem in Redhat Linux 5.4 64 bit. Has this been fixed yet?
 
Last edited:
PREDATAR Control23

I have been talking with the author of tsmtape for a while - but I haven't heard anything back as yet.
 
PREDATAR Control23

Some hints for fixing tsmtape.. At least this is what I did to get it work for us:

The problem with tsmtape and TSM tapes seems to be that the byte order gets changed for some reason. For example, a value like 0xAABBCCDD might read 0xDDCCBBAA.

Look, for example at one output line you sent:

The value passed was 4 bytes long... value==20b7,returnval==b7200000

What this means is that there is a function in tsmtape that changes the order of the bytes. In some cases this is necessary, and in other cases not. In this case, for example, the first value (input to the byte-order function) looks more sane than the other.
I would try commenting out the appropriate function call, so that this doesn't get changed.

Some of the values seem to be completely irrelevant, but others are important. It's been too long since I was hacking this, so I don't remember any details.

Another thing that can be of use is to manually read the content from the tapes, just to see how it looks:

First, turn off TSM, then manually insert the tape into a tape drive.
You need to exchange /dev/nst1 with your tape device.

Check the status of the drive:
mt -f /dev/nst1 status

Read the first file off the tape:
dd if=/dev/nst1 of=file0 bs=256k
Read the second file:
dd if=/dev/nst1 of=file1 bs=256k
...

From what I remember, the first file was only the tape label and stuff. The second file contains the real data (and maybe file 2, 3, .. - I don't remember).

To rewind the tape:
mt -f /dev/nst1 rewind


Our tapes were in EBCDIC encoding, so you need to use conv=ascii to convert them:

dd if=file1 of=file1.conv conv=ascii


Have a look at the contents:

hexdump -C file1.conv | less

There you'll find all values that tsmtape reads. It takes some work to decode all of that, but you'll see which ones need to be byte switched, and which are to be left alone.



The second problem with tsmtape I had is that it is written on a 32-bit system. Some of the variable definitions are wrong. The original uses long and ulong variables in struct definitions, which won't work, as a long is 32 bit on 32 bit systems, and 64 bit on 64 bit systems. I changed these to int and uint instead. You need to carefully check what needs to be fixed, and what should be left a lone. Check my patch to see what I did. These changes should work on all systems. The problem seems to be mainly the byte switch things.

Another thing. I added a -j option to skip some junk. This was needed to make it work. However, last time I had to read back from a tape, the -j option could not be used! If -j was used, it wouldn't work. Had to use it without -j!!!

So:
Different tapes need -j or not.

It could be that different TSM versions write slightly different looking tapes.
 
PREDATAR Control23

ADSM_SEFR=0, expected '53454652'
Forcing read of next record
File 1, Record 8: 262144 bytes
OH.pad=3, expected '0'
Vasize: 297 Inlengt: 136
Storage is backup
Filetype is regular
B - 6287360 SLinux86STANDARD/mnt/raid/data/MP3s/Songs/(DJ Talla 2XLC
File 1, Record 9: 262144 bytes
ADSM_TapeRecordHeader->recnum=5, expected '6'
File 1, Record 8: 262144 bytes
File 1, Record 9: 262144 bytes
ADSM_TapeRecordHeader->recnum=5, expected '6'
File 1, Record 8: 262144 bytes
File 1, Record 9: 262144 bytes
ADSM_TapeRecordHeader->recnum=5, expected '6'
[/code]Seems like it's almost there... but it seems that this record header is causing the file not to be written?

Those recnum lines are there all the time. Don't bother about them. When I run tsmtape, I get loads of those recnum=5, expected '6' lines, but it works anyway. I guess something has changed in the structure on the tapes since tsmtape was written. Nothing to worry about.

Just let it run until it is done. If you get the prompt back, without any results, then investigate. Otherwise, you probably get the files if you let it run for a few hours.
 
Top