• Please help support our sponsors by considering their products and services.
    Our sponsors enable us to serve you with this high-speed Internet connection and fast webservers you are currently using at ADSM.ORG.
    They support this free flow of information and knowledge exchange service at no cost to you.

    Please welcome our latest sponsor Tectrade . We can show our appreciation by learning more about Tectrade Solutions
  • Community Tip: Please Give Thanks to Those Sharing Their Knowledge.

    If you receive helpful answer on this forum, please show thanks to the poster by clicking "LIKE" link for the answer that you found helpful.

  • Community Tip: Forum Rules (PLEASE CLICK HERE TO READ BEFORE POSTING)

    Click the link above to access ADSM.ORG Acceptable Use Policy and forum rules which should be observed when using this website. Violators may be banned from this website. This notice will disappear after you have made at least 3 posts.

General TSM automation talk

gentoomaniac

Newcomer
Joined
Oct 29, 2015
Messages
4
Reaction score
1
Points
0
Hi all,

I've had a look around a bit in the forums and on the Internet in general.
One thing I realised is that it's pretty easy to find answers to questions like "how do I query xyz?" but hardly anything about environment designs, tips etc to automate tsm.

What I would like to start in this thread is therefore some sharing of how you tackled certain problems / inconveniences of tsm.
@admins feel free to move this thread if you think it is misplaced.

To make a quick start here some stories from my current position.

Our server environment is completely automated by puppet.
Single servers should ideally be irrelevant and replaceable any time by just installing a new machine with the same puppet configuration.
For backups with tsm this is quite a challenge (registering nodes, setting schedules, ...).

So to get it as good as possible, we currently have the following tools in place and under active development:

1) a python wrapper for easily executing dsmadmc and retrieving the resulting data, errors parsed etc.
2) on top of this we built a REST API (django) which allows the most common operations important for clients: register/remove node, add/remove assocs, add/remove proxy node, various information gathering like query node f=d, query server sessions, processes, restores,...
For authorization we connected it to ldap and use an ssh key based challenge response system.
All requests are logged to elastic search. For statistics and taking.
3) a company specific command line client which executes all these actions and abstracts certain stuff (converts SLAs to TSM domains etc)

Problems we're facing again and again is limitations in the core as we rely on dsmadmc and so far there seems to be no api for that available.
For dsm I started working on Python wrapper for the C API which will hopefully make some progress the next weeks (https://github.com/gentoomaniac/pytivsm-api).

So much for my experience, curious to hear from some others. Pits you fell in and how you solved it.

Cheers
 

moon-buddy

ADSM.ORG Moderator
Joined
Aug 24, 2005
Messages
7,033
Reaction score
405
Points
0
Location
Somewhere in the US
I have done routine TSM work, i.e., node registration, schedules, delete node, delete schedules, automagically via KSH.

The automation covers multiple TSM servers and is TSM server agnostic. The automation just needs a node name, as an example, and it will find the associated TSM server it is registered to and delete its schedule, delete filespace (be it a single one or all), etc.

Registration and scheduling, likewise, can be part of a server build through Powershell for Windows boxes, and through shell scripts (or puppet for mainly Redhat systems in our case) for *nix systems.
 
Last edited:

gentoomaniac

Newcomer
Joined
Oct 29, 2015
Messages
4
Reaction score
1
Points
0
Do you have any experiences regarding multiple simultaneous sessions?
What bugs me the most is that each process needs to open a new dsmadmc session atm. (I don't really feel like remote controlling with expect our similar to keep a single session open.)

When custom monitoring comes into play (no operation center, but passing select queries to our monitoring infra) this might get excessive...
 

moon-buddy

ADSM.ORG Moderator
Joined
Aug 24, 2005
Messages
7,033
Reaction score
405
Points
0
Location
Somewhere in the US
If you need to do queries or run jobs that don't have dependencies on one another, then the only way is to open multiple sessions which is what I do.

If dependencies exist, then the "wait=yes" option is used and will keep one session opened for the reminder of the routine. As you know, not all commands have the "wait=yes" option. You just have to 'play-by-ear' to determine what is best.
 

gentoomaniac

Newcomer
Joined
Oct 29, 2015
Messages
4
Reaction score
1
Points
0
We built everything to be atomic and add dependencies in the higher layers.

I'm afraid that sooner or later we might hit session limits (which we of course can increase, but it feels wrong. ).

Will def start bugging the support for a proper dsmadmc api ;)
 

Jarrod

Newcomer
Joined
Dec 1, 2016
Messages
2
Reaction score
0
Points
0
On Linux:

% dsmadmc < named_pipe1 > named_pipe2

Then build a perl (or whatever) application that writes your dsmcadm commands to named_pipe1, then fork()s and reads/parses output from named_pipe2.

This can be parallelized - just create 2N pipes for N dsmadmc instances and launch your script N times with 1/N of the workload each time. I had 7 going at once on an 8-core box with a RAID of SSDs hosting the db2 database. Got about 5x speedup as I recall.

This seriously saved my posterior on a giant restore that was going to take 6 months using standard TSM commands. Got the job done in 2 weeks.
 

pvarney

Newcomer
Joined
May 6, 2005
Messages
3
Reaction score
1
Points
0
Location
Rogers, AR
In reply to my own note above, I did end up implementing the REST API in Operations Center. It was tricky to get working, but has been quite handy. It returns the data in JSON format, so it's somewhat standardized. I am using the LWP::UserAgent Perl module to issue the requests. Here is a simple example using curl to get a list of servers and associated data in Operations Center (remove space before "pw" - had to add to disable auto-smiley):

curl -X GET -H "OC-API-Version: 1.0" -H "Accept: application/json" -H "Content-Type: text/plain" -k --user admin: pw https://tsmserver1.company.com:11090/oc/api/servers
 

neha19686

ADSM.ORG Member
Joined
Jul 28, 2009
Messages
10
Reaction score
0
Points
0
Location
mumbai
I have done routine TSM work, i.e., node registration, schedules, delete node, delete schedules, automagically via KSH.

The automation covers multiple TSM servers and is TSM server agnostic. The automation just needs a node name, as an example, and it will find the associated TSM server it is registered to and delete its schedule, delete filespace (be it a single one or all), etc.

Registration and scheduling, likewise, can be part of a server build through Powershell for Windows boxes, and through shell scripts (or puppet for mainly Redhat systems in our case) for *nix systems.

hej do you have any of this script we can try to use in our environemnt as well
 

vegivamp

ADSM.ORG Member
Joined
Nov 13, 2018
Messages
14
Reaction score
2
Points
0
I have a couple of ansible roles that handle setting up storage agents, lin_tape drivers and configuration, T10kD drive aliases (through custom scripting, not yet udev) etc.

I intend to eventually publish them on github, but they're currently not in a state I'm comfortable inflicting on others :)

I was intending to also look into libraries/modules to interact with the server in a better way than cli execution, but haven't gotten around to it at all - after all, it works for us now and it's not like I have to install new STAs every week :p May have a look at your work for that, @gentoomaniac .
 

xixao

Newcomer
Joined
Mar 27, 2020
Messages
1
Reaction score
0
Points
0
I have done routine TSM work, i.e., node registration, schedules, delete node, delete schedules, automagically via KSH.

The automation covers multiple TSM servers and is TSM server agnostic. The automation just needs a node name, as an example, and it will find the associated TSM server it is registered to and delete its schedule, delete filespace (be it a single one or all), etc.

Registration and scheduling, likewise, can be part of a server build through Powershell for Windows boxes, and through shell scripts (or puppet for mainly Redhat systems in our case) for *nix systems.
Hi there,
I have been thinking to prepare a TSM script in KSH as you mentioned that you have it but a little example of what you had would be a really helpful to bounce me from the bottom? What do you think to share with us I do appreciate any respond, thanks.
 

Advertise at ADSM.ORG

If you are reading this, so are your potential customer. Advertise at ADSM.ORG right now.

UpCloud high performance VPS at $5/month

Get started with $25 in credits on Cloud Servers. You must use link below to receive the credit. Use the promo to get upto 5 month of FREE Linux VPS.

The Spectrum Protect TLA (Three-Letter Acronym): ISP or something else?

  • Every product needs a TLA, Let's call it ISP (IBM Spectrum Protect).

    Votes: 19 18.3%
  • Keep using TSM for Spectrum Protect.

    Votes: 63 60.6%
  • Let's be formal and just say Spectrum Protect

    Votes: 13 12.5%
  • Other (please comement)

    Votes: 9 8.7%

Forum statistics

Threads
31,828
Messages
135,735
Members
21,789
Latest member
Fred49
Top