General questions about archiving

[flump]

Hello All ! First post here, I've lurked here sometime and now decided to join

I'm quite new to TSM and I have a bunch of unaswerd practical questions that I will be bugging you with over time, and here comes the first one.

Where i work we use TSM 90% of the time for archive purposes, we generate quite a lot of data daily, and we need to keep it stored for various reasons, we use TSM for that, we have some simple scripts that do most of the "hard work", the main issue with TSM are reliability and overall performance but we are going to look into that, this happens for various hardware and configuration reasons, that we are trying to adress.

One of the things that happens from time to time is that one of my archive tasks fail during processing, for X reason.

When that happens I delete the partial data from the archive and start over ... this is becoming a big deal as I'm facing bigger and bigger folders to store.

Is it possible to archive data so that tsm takes advantage of what's already stored and just start from where it was interrupted ?

like the --skip-existing flag of grsync per say.... ?

Thank you very much !

have a good day

 

[ILCattivo]

Hi Flump and welcome to to the forums..

Are you able to provide us with some more information about the archiving issue you're having? What would be particulalry helpful is the following?

-What TSM Server & client versions are in use, including OS's
-The dsm.opt/dsm.sys file content on the clients side
-The dsmerror.log / dsmsched.log entries from the client around the time of the failure?

From here people should have a much better idea of whats going on?

Regards

 

[flump]

Hi ILCattivo ( italian by any chanche ? )

Thank you for the reply, yeah i completely forgot to add some specs,

- tsm is 6.3.something (i think 2 i needo to check but i'm out of office)
- the machine is an ibm 3550 m3 os is Linux centos 5.1.x
3 libraries
2 x ts3200 one with 2 lto5 drives, one with 1 lto7 drive (in the works).
1x ts3310 with 4 lto 3 drives
network via fc 8gb

without sharing the whole log right now i'm having mainly "mechanical problems " like :

8A: HE: slider blocked
61: error during first slider movement
15: move cartridge

or

94: HE: slider outside range
61: error during first slider movement
15: move cartridge

those seem to be related to the picker assembly and movement, but it's not the case, i kept the library on my desk with the top removed to see if there was something that got "locked" or stuck during the movement, nothing relevant was to see and once you just light touch the assembly with a finger it's moving freely, it went fine trough like 300+ cycles without errors.. i really think that's not the case, let me explain better.

Pulling the magazines and inserting the tapes in and out by hand in the magazines feels very "sticky", so reading around the web ( not finding the page right now ) to pull and insert a tape several times in each slot of the magazines for getting the little plastic lock/bevel a little looser, is something that was suggested..

we had that on a ts3100 lately, so we changed the library with a ts3200, but the problem still arises but I'm really starting to think there's something wrong with the batch of cartridges i'm using (ibm, never had a problem before ), also reloading the latest firmware seems to be something to do.. an i'm planning that.

with lto5 i achieve somewhere around 300/330 gigs/hour, and i think that's not bad, but folders are huge, 20 TB a pop, and "complicated", doing smaller sub batches might be a solution, so that i don't have to restart the whole thing but i really wanted to know if my original question is something doable in principle with tsm.

seems odd that this kind of feature is not present, just starting to think i got told wrong ... so i wanted an overlook from the experts.

thanks !

 

[RecoveryOne]

Greetings and welcome!

What you posted about the library's having issues concerns me. If you are archiving directly to tape, and the library can't fulfill the media movement request the tsm process will fail after a short while.
I can't speak for the TS3100 or TS3200 series, but for the TS3310's and larger units like the 3584's having the library properly leveled and calibrated is a must! Other than that, check for worn cables, lose connections and from the management interface, library errors. You may need to have a CE come and service the unit(s).

Now, are you archiving directly to tape, or are you using a disk/file based storage pool to act as a buffer before that data is getting shipped to the tape library?

As to restarting an archive, I do not think that is supported. When you create an archive TSM doesn't care if it has part of the data from a previous run, it will create new entries in the database again and archive everything once more.

Depending on what your archive requirements are, it may be beneficial to use a long term management class and do incremental backups. Then, when you restart the backup, it only needs to backup what changed.
I will point to these two older posts that still hold true today (as far as I know):
https://adsm.org/forum/index.php?threads/difference-between-backup-and-archive.21079/
and
https://adsm.org/forum/index.php?threads/archive-vs-selective-backup.25102/#post-104364

I'm not saying do X instead of Y, as only you know what your requirements are but may be something worth while to think about.

Please bug us as much as you wish!
I've learned a lot from the people here on these forums, and can't give thanks to them enough!

 

[flump]

Hi Recovery one ! thanks for the welcome and the tips !

i'll be reading the post you linked ( thanks ) our libraries are all in good condition and i recently got a refurbished one with 6 months warranty, the problem is the same as the old library we tought was defective, looks like it's not the library the problem, and btw .. the libraries were in two different rack enclosures, mounted on their sleds, the only common point seems to be this batch of tapes....

we use backups on lto3 and that's working fine .. except the normal quirks here and there, it's pretty stable.
i get what you mean with the example you made above, we tried that, we generate too much stuff daily, and a lot is temporary than get's thrown, i don't want to get too much in detail because that would be a loooong discussion , 90 % of jobs is archiving,

anyway .. when we close a project we clean it form tmp and extra not useful data, and then archive it on tape, once done the tapes go into the vault. it's unlikely that we need to get data back, but it happens from time to time

as per your response about the feasibility of the restart, that's what i tough and i was just hoping i was told wrong, would have been great.
Well then we simply split the task in more sub tasks, so that i just need to make smaller parts eventually i have some scripts to prepare me all I need so no big deal.

Please bug us as much as you wish!

i'll take your word for that ! will do no worries about that hehe..

Thank you !!