ADSM-L

Re: [ADSM-L] Fine tuning INCLUDE/EXCLUDE and EXCLUDE.DIR

2016-08-29 11:47:37
Subject: Re: [ADSM-L] Fine tuning INCLUDE/EXCLUDE and EXCLUDE.DIR
From: Andrew Raibeck <storman AT US.IBM DOT COM>
To: ADSM-L AT VM.MARIST DOT EDU
Date: Mon, 29 Aug 2016 11:46:09 -0400
This kind of question comes up now and then, about how to back up just a
single directory without traversing the entire file system.

A couple of thoughts on this subject, in addition to what others have
written:

1. Schedule the incremental backup with OBJECTS="J:\IMAGENOW\SUBOB_OSM\*"
OPTIONS=-SUBDIR=YES

This is the simplest alternative that will process only the objects in that
directory. However you will not get the last backup start and end times
filled in for the file space, so this option depends on how critical those
fields are for your reporting.

2. Another alternative whose trickiness depends on the similarity of
directory names is to use a series of EXCLUDE.DIR options that eventually
exclude all directories whose name is not J:\IMAGENOW\SUBOB_OSM:

   * Exclude all file objects in the root of J:
   exclude j:\*

   * Exclude all directories that do not begin with 'I'.
   exclude.dir j:\[a-hj-z0-9$_]*

   * Exclude all directories that do not begin with 'IM'.
   exclude.dir j:\i[a-ln-z0-9$_]*

   * etc.
   exclude.dir j:\im[b-z0-9$_]*
   exclude.dir j:\ima[a-fh-z0-9$_]*
   exclude.dir j:\imag[a-df-z0-9$_]*
   exclude.dir j:\image[a-mo-z0-9$_]*
   exclude.dir j:\imagen[a-np-z0-9$_]*
   exclude.dir j:\imageno[a-vx-z0-9$_]*

   * Exclude all directories that begin with J:\IMAGENOW followed by one or
   more characters.
   exclude.dir j:\imagenow?*

   * Exclude all directories that do not begin with J:\IMAGENOW\S
   exclude.dir j:\imagenow\[a-rt-z0-9$_]*

   * Exclude all directories that do not begin with J:\IMAGENOW\SU
   exclude.dir j:\imagenow\s[a-tv-z0-9$_]*

   * etc.
   exclude.dir j:\imagenow\su[ac-z0-9$_]*
   exclude.dir j:\imagenow\sub[a-np-z0-9$_]*
   exclude.dir j:\imagenow\subo[ac-z0-9$_]*
   exclude.dir j:\imagenow\subob[a-z0-9$_]*
   exclude.dir j:\imagenow\subob_[a-np-z0-9$_]*
   exclude.dir j:\imagenow\subob_o[a-rt-z0-9$_]*
   exclude.dir j:\imagenow\subob_os[a-ln-z0-9$_]*

   * Exclude all directories that begin with J:\IMAGENOW\SUBOB_OSM followed
   by one or more characters.
   exclude.dir j:\imagenow\subob_osm?*

I hope I got that right, I typed it up pretty quickly but hopefully it
conveys the "spirit" of the idea. That is, to exclude all directories that
do not begin with 'I', followed by all directories that do not begin with
'IM', followed by all directories that do not begin with "IMA", and so on.

Note that the character classes used in the EXCLUDE.DIR statements assume
your directories are composed of alphanumeric characters (A-Z, a-z, 0-9),
$, or _ (underscore); you might need to adjust this based on other
characters used in the directory names.

The above represents a "worst case" scenario where you have very similar
directory names, e.g., J:\IMAGENOA, J:\IMAGENOB, J:\IMAGENNW, J:\IMAGENOW2,
etc. The best case scenario is if only one directory in J: begins with the
letter 'I', in which case you can use just the first exclude.dir I
mentioned above.

Andy

____________________________________________________________________________

Andrew Raibeck | IBM Spectrum Protect Level 3 | storman AT us.ibm DOT com

IBM Tivoli Storage Manager links:
Product support:
https://www.ibm.com/support/entry/portal/product/tivoli/tivoli_storage_manager

Online documentation:
http://www.ibm.com/support/knowledgecenter/SSGSG7/landing/welcome_ssgsg7.html

Product Wiki:
https://www.ibm.com/developerworks/community/wikis/home/wiki/Tivoli%20Storage%20Manager

"ADSM: Dist Stor Manager" <ADSM-L AT VM.MARIST DOT EDU> wrote on 2016-08-26
14:21:41:

> From: "Vandeventer, Harold [OITS]" <Harold.Vandeventer AT KS DOT GOV>
> To: ADSM-L AT VM.MARIST DOT EDU
> Date: 2016-08-26 14:22
> Subject: Fine tuning INCLUDE/EXCLUDE and EXCLUDE.DIR
> Sent by: "ADSM: Dist Stor Manager" <ADSM-L AT VM.MARIST DOT EDU>
>
> One of my clients has a server running ImageNow.  There are millions
> of files and several thousand folders that take over 24 hours for
> the daily incremental.
>
> My goal is to backup a single ImageNow folder and reduce the elapsed
> time to traverse thousands of folders that don't need to be backed up.
>
> We worked out a setup for a new node in an effort to backup only a
> given path within the ImageNow structure.
>
> The following dsm.opt statements are in place:
> DOMAIN J:
> EXCLUDE J:\...\*
> INCLUDE J:\IMAGENOW\SUBOB_OSM\...\*
>
> That was successful in backing up only the files and subfolders/
> files in SUBOB_OSM.  But it also backed up thousands of directories
> on J:\.  Most of them are in the \IMAGENOW path.
>
> My plan is to add a few EXCLUDE.DIR statements such as:
> EXCLUDE.DIR J:\IMAGENOW\54_OSM
> EXCLUDE.DIR J:\IMAGENOW\DEFAULT_OSM
>
> The question:  will adding EXCLUDE.DIR now, after the node has
> backed up all those paths, result in a performance improvement?
>
> I suspect Spectrum Protect server will send all the directory names
> to the node, but hopefully the work on the node side will be much faster.
>
> Or, will we need to delete the filespace, then start over with a
> fresh "incremental forever" backup to clear out all that directory
structure?
>
> Thanks.
> ------------------------------------------------------------
> Harold Vandeventer
> Office of Information Technology Services
> 2800 SW Topeka Blvd
> Topeka, KS 66611
> (785) 296-0631
> Harold.Vandeventer AT ks DOT gov | www.oits.ks.gov
>
>
>
> CONFIDENTIALITY AND PRIVILEGE NOTICE
> This email message including attachments, if any, is intended for
> the person or entity to which it is addressed and may contain
> confidential and/or privileged information. Any unauthorized review,
> use or disclosure is prohibited. If you are not the intended
> recipient, please contact the sender and destroy the original
> message, including all copies.
>
>
>
> CONFIDENTIALITY AND PRIVILEGE NOTICE
> This e-mail message, including attachments, if any, is intended for
> the person or entity to which it is addressed and may contain
> confidential or privileged information. Any unauthorized review,
> use, or disclosure is prohibited. If you are not the intended
> recipient, please contact the sender and destroy the original
> message, including all copies, Thank you.
>