HajoEhlers
ADSM.ORG Member
- Joined
- Aug 30, 2005
- Messages
- 10
- Reaction score
- 0
- Points
- 0
Hi,
i am testing a logic to backup only directories which have been changed or where a file has been changed. Its to speedup our GPFS backups by using the ILM engine to find all diretories where changes had happen.
Example:
On a given FS we have 40M files, 1M directories but only 30K directories needs processing.
The scan engine needs about 1h and the backup is done with up to 50 parallel "dsmc incr -subdir=no ..." session and where resourceutilisation of 10. The backup time went from 16h down to 3h. ( 1h scan , 2h backup )
For far everything works fine except one (out of two) thing: If a "move" ( mv ) happend in the given FS these part are not picked up since no change has happen on them.
Because of this i have check if a given directory is available on the TSM server or not via a "dsmc q backup /MyDir/"
In case it is not, it will be selected for backup as well.
Problem: Its not very fast to check a few 1000 directories this way since "dsmc q backup " allows only one directory entry per execution.
Thus i am looking for some kind of sql command which is able todo the same job a little bit faster and where the input would be a list of directories .
Input
/gpfs/dir1/dir2
/gpfs/dir99/dir101/dir44
Output ( 0=exist , 1=not exist )
0 /gpfs/dir1/dir2
1 /gpfs/dir99/dir101/dir44
Any hint ?
tia Hajo
i am testing a logic to backup only directories which have been changed or where a file has been changed. Its to speedup our GPFS backups by using the ILM engine to find all diretories where changes had happen.
Example:
On a given FS we have 40M files, 1M directories but only 30K directories needs processing.
The scan engine needs about 1h and the backup is done with up to 50 parallel "dsmc incr -subdir=no ..." session and where resourceutilisation of 10. The backup time went from 16h down to 3h. ( 1h scan , 2h backup )
For far everything works fine except one (out of two) thing: If a "move" ( mv ) happend in the given FS these part are not picked up since no change has happen on them.
Because of this i have check if a given directory is available on the TSM server or not via a "dsmc q backup /MyDir/"
In case it is not, it will be selected for backup as well.
Problem: Its not very fast to check a few 1000 directories this way since "dsmc q backup " allows only one directory entry per execution.
Thus i am looking for some kind of sql command which is able todo the same job a little bit faster and where the input would be a list of directories .
Input
/gpfs/dir1/dir2
/gpfs/dir99/dir101/dir44
Output ( 0=exist , 1=not exist )
0 /gpfs/dir1/dir2
1 /gpfs/dir99/dir101/dir44
Any hint ?
tia Hajo
