Using wildcards in client schedule - Objects definition for Linux clients

[coveymw]

Hi,
feeling rather dumb here, searched high and low and can't get this to work.

I administer a z/OS TSM server which historically has only had Windows clients..... I've just started getting Linux clients and I'm in the process of creating automatic backup schedules for them.

I'm trying to add in a schedule to backup a file system and want to use a wilcard for part of the structure but when looking at various users dsmsched log it shows as file system not found.

Using the GUI I'm adding the following into the Objects field

/home/*/$user/*

Thhis should then work for every linux client to back up a filesystem $ user if they have it, i.e. it should backup

/home/usera/$user/*
/home/userb/$user/*
/home/userb/$user/*
etc
etc
etc

looking at the dsmsched & dsmerror logs I get the following which looks like its trying to find a '*' directory rather than using it as a widcard

13/08/12 16:51:23 Schedule Name: LINUX_USER
13/08/12 16:51:23 Action: Incremental
13/08/12 16:51:23 Objects: /home/*/$user/*
13/08/12 16:51:23 Options:
13/08/12 16:51:23 Server Window Start: 10:14:00 on 13/08/12
13/08/12 16:51:23 ------------------------------------------------------------
13/08/12 16:51:23
Executing scheduled command now.
13/08/12 16:51:23 --- SCHEDULEREC OBJECT BEGIN LINUX_USER 13/08/12 10:14:00
13/08/12 16:51:23 Incremental backup of volume /home/*/$user/*
13/08/12 16:51:23 ANS1076E The specified directory path could not be found.
13/08/12 16:51:24 --- SCHEDULEREC STATUS BEGIN
13/08/12 16:51:24 --- SCHEDULEREC OBJECT END LINUX_USER 13/08/12 10:14:00
13/08/12 16:51:24 ANS1512E Scheduled event 'LINUX_USER' failed. Return code = 12.

[moon-buddy]

Unfortunately, TSM does not process environmental variable $user (I assuming that this is a variable which stands for a USER ID).
You need something like this:

include /home/user?/*

[coveymw]

$user is the directory name.. the problem is with the wildcard character * and TSM not recognising this as a wildcard...... for instance I have 2 other file systems that I need to in include a widlcard

/home/*/Documents/*

and

/home/*/lotus/notes/data/*

In both of these cases the directory wildcard /*/ is not being recognised

[moon-buddy]

/home/*/Documents/* and /home/*/lotus/notes/data/*

The problem is that * one after the other is not recognized or processed by TSM. The right way is:

/home/.../Documents/* and /home/.../data/*

[coveymw]

Hi thanks.... tried setting the Objects as you suggested and still get the failing messages... pulling my hair out here and I have very little left !

16/08/12 22:20:08 Server Version 5, Release 5, Level 4.0
16/08/12 22:20:08 Server date/time: 16/08/12 21:19:26 Last access: 16/08/12 21:04:20

16/08/12 22:20:08 --- SCHEDULEREC QUERY BEGIN
16/08/12 22:20:08 --- SCHEDULEREC QUERY END
16/08/12 22:20:08 Next operation scheduled:
16/08/12 22:20:08 ------------------------------------------------------------
16/08/12 22:20:08 Schedule Name: @70
16/08/12 22:20:08 Action: Incremental
16/08/12 22:20:08 Objects: /home/.../$user/*
16/08/12 22:20:08 Options:
16/08/12 22:20:08 Server Window Start: 19:08:46 on 16/08/12
16/08/12 22:20:08 ------------------------------------------------------------
16/08/12 22:20:08
Executing scheduled command now.
16/08/12 22:20:08 --- SCHEDULEREC OBJECT BEGIN @70 16/08/12 19:08:46
16/08/12 22:20:09 Incremental backup of volume '/home/.../$user/*'
16/08/12 22:20:09 ANS1076E The specified directory path could not be found.
16/08/12 22:20:09 --- SCHEDULEREC STATUS BEGIN
16/08/12 22:20:09 --- SCHEDULEREC OBJECT END @70 16/08/12 19:08:46
16/08/12 22:20:09 ANS1512E Scheduled event '@70' failed. Return code = 12.
16/08/12 22:20:09 Sending results for scheduled event '@70'.
16/08/12 22:20:09 Results sent to server for scheduled event '@70'.

[moon-buddy]

Question: is $user a real searchable value or is it an environmental variable. If it is the latter, TSM WOULD NOT process as such.

[coveymw]

$user is a 'real' directory......I'm having the same problem with schedules for various file structures... i.e

/home/.../Documents/* has the same result

I'm testing another and will update with the results..

1. Adding in -subdir=y in the options field on teh schedule

This shouldn't make a difference as I have subdir=y set in the OptionSet assigned to these schedules but I'll try anything at the moment!

[Harry_Redl]

Hello,

this is going to be a bit longer ...

as far as I know, this approach does not (and should not) work. This is what BA client manual states:

In a command, you can use wildcard characters in the file name or file extension
only.

... and using "object" field in the client schedule is nothing more than passing a parameter to the BA client command.

What is meant by "file name" here? Let's have a file /mnt/whatever/directory1/subdir2/file.txt - where /mnt/whatever is a mountpoint. Complete path to this file is (in TSM) split into three parts:
filespace name (/mnt/whatever)
HL_name (/directory1/subdir2)
LL_name (file.txt)
It seems that you can use the wildcards on the LL_name ONLY and the allowed wildcards are "*" and "?" (not the "\...\" - see below)

What is a bit confusing here is that there is an exception from this rule - "query backup" accepts wildcards even in the path name and "*" can mean one or more subdirectories.

Did few tests and found a difference between UNIX (tried Linux) and Windows clients:

WINDOWS:
tsm> sel c:\test\user*\* -sub=yes
Selective Backup function invoked.
Directory--> 0 \\machine\c$\test\user1 [Sent]
Directory--> 0 \\machine\c$\test\user1\dir1 [Sent]
Normal File--> 5 \\machine\c$\test\user1\dir1\soubor.txt [Sent]

LINUX:
tsm> sel /root/test/user*/soubor.txt -sub=yes
Selective Backup function invoked.

ANS1076E The specified directory path could not be found.

So you can see Windows backed up FIRST occurence matching the wildcard (it DID NOT back up \\machine\c$\test\user2\dir1\soubor.txt) but the LINUX client refused the path completely.
I believe this is a bug (or it falls under "unexpected results may occur").


Another way of using wildcards is include/exclude. This accepts not only the "*" and "?", but also "\...\" for any (or multiple or no) subdirectory or you can use "[]" brackets for pattern matching.
Problem is that you cannot tell TSM "use this incl/excl rules with this backup schedule only" - and if you change the include/exclude list between two backups, you may end up with files being expired before you wanted to ...

So how to tackle your problem?
1) you can use external script to call "dsmc incr" for all directories /home/userXYZ/$user (so the schedule is of "COMMAND" type)
2) you can use "preschedule" command building a list of files to be backed up for use with "-filelist" option (you have to run "ful incremental" from time to time to handle expiration of deleted files)
3) use separate node for these directories (so you can include/exclude list - one node has these directories included the other excludes them)
4) if you need ONLY these folders being backed up (so there is no schedule for /home/userXYZ/data), then you can use "simple" include/exclude list


Hope it helps

Harry

[coveymw]

Harry & Moon-buddy,

Thanks for the updates and assistance. What I'm going to do (and have done) is set up a schedule to backup everything and use a include/exclude list.... set this up on Friday and so far today the Linux clients that have been scheduled and completed have given the results I expect... its a bit down and dirty but it does work.

I think I'll still try and carry on with individual schedules for different filesystems as it 'should' work... it works with Windows clients as I have approx 30+ schedules for various file systems and these all work without any issue..... so no reason why it should't work with Linux..... I'll raise a call with Tivoli I think and update here if I get any joy.

Thanks again.

[Harry_Redl]

Hi,

just check if your Windows clients really work - as I found during my tests, Windows backed up just the first directory matching the wildcard - so it may look it is working - but in fact it is not.

Harry