At the end of my first paragraph below, I left off part of the explanation:
> ... after the file is
> deleted from the live file system, the next incremental backup operation
> will cause the latest backup version to expire (become inactive).
I meant to add:
Because VERDELETED and RETONLY are zero, all versions will no longer be
available for restore.
Best regards,
Andy
Andy Raibeck
IBM Software Group
Tivoli Storage Manager Client Product Development
Level 3 Team Lead
Internal Notes e-mail: Andrew Raibeck/Hartford/IBM@IBMUS
Internet e-mail: storman AT us.ibm DOT com
IBM Tivoli Storage Manager support web page:
http://www.ibm.com/support/entry/portal/Overview/Software/Tivoli/Tivoli_Storage_Manager
The only dumb question is the one that goes unasked.
The command line is your friend.
"Good enough" is the enemy of excellence.
"ADSM: Dist Stor Manager" <ADSM-L AT vm.marist DOT edu> wrote on 2010-08-10
18:56:33:
> From: Andrew Raibeck/Hartford/IBM@IBMUS
> To: ADSM-L AT vm.marist DOT edu
> Date: 2010-08-10 18:58
> Subject: Re: NBU guy in TSM shop and I need help
> Sent by: "ADSM: Dist Stor Manager" <ADSM-L AT vm.marist DOT edu>
>
> I concur with "TSM User" on the policy settings: everything is being kept
> forever while the file exists on the live file system; after the file is
> deleted from the live file system, the next incremental backup operation
> will cause the latest backup version to expire (become inactive).
>
> Proper retention depends on the needs of the business. One very simple
> example might be to express retention needs in terms of restore
> requirements, e.g., we need to be able to restore any file form backups
> taken up to 30 days ago. In this case, you can configure TSM with
settings
> like this:
>
> ======================================================
> *How many different versions of the file should be kept? NOLIMIT
> (corresponds to the VEREXISTS copygroup setting)
>
> * Number of days to keep inactive versions 30
> (corresponds to the RETEXTRA copygroup setting)
>
> For backup files that a user has deleted from the client node:
> * Number of file versions to keep NOLIMIT
> (corresponds to the VERDELETED copygroup setting)
>
> * Amount of time to keep the last file version 30
> ("last" refers to the latest backup copy made before the file was
deleted;
> corresponds to the RETONLY copygroup setting)
> ======================================================
>
> TSM *always* keeps the latest ("active") backup version as long as the
file
> exists on the live file system, regardless of the RETEXTRA setting. When
a
> new backup copy of a file is made, the prior backup copy is made
"inactive"
> and becomes subject to VEREXISTS and RETEXTRA settings. Inactive versions
> are removed depending on which of the VEREXISTS or RETEXTRA criteria are
> reached first.
>
> For example, suppose VEREXITS is 5 and RETEXTRA is 30. You perform daily
> backups. If the file changes every day, it will be backed up every day.
On
> day 6, the 6th backup version will be made. Because VEREXISTS is 5, the
> oldest backup version is expired, regardless of the RETEXTRA setting.
>
> On the other hand, if VEREXISTS is NOLIMIT and RETEXTRA is 30 (like my
> sample settings above), then on day 31, when the 31st backup copy is
made,
> the oldest backup version is expired, regardless of the RETEXTRA setting.
>
> Why restores need multiple tapes is hard to say since we don't have
> specifics on your scenario; that is, whether you are basing this on an
> actual restore where you noted a large number of mounts, or if you are
just
> looking at the number of tapes on which a given client has backup data.
>
> In general, if the number of tapes mounted to perform a restore operation
> seems excessive, then consider enabling collocation, which will optimize
> the number of tapes on which a given node's data is kept (this won't
happen
> overnight, but occurs over time as TSM subsequently places data on the
> tapes). You can collocate by node (all of a given node's data on an
> optimized number of tapes) or collocate by file space (backups for each
> file system are kept on an optimized number of tapes). As new backup
copies
> are made and old ones expire, over time the amount of expired data on the
> tapes will increase, leaving fewer and fewer valid backup copies; thus
the
> tape becomes less and less utilized. Inventory expiration and reclamation
> should be performed regularly to consolidate these volumes. Alternative
> options are available, as others have pointed out, to help optimize
restore
> processing. In addition, you can also improve restore performance by
using
> multisession restores, which allow multiple restore sessions (one tape
per
> session), based on the MAXNUMMP (maximum number of mount points) setting
> for the node and the RESOURCEUTILIZATION setting in the node's dsm.opt
> file. The number of concurrent tape mounts/restore sessions is, of
course,
> also dependent on the number of available tape drives.
>
> Good luck,
>
> Andy
>
> Andy Raibeck
> IBM Software Group
> Tivoli Storage Manager Client Product Development
> Level 3 Team Lead
> Internal Notes e-mail: Andrew Raibeck/Hartford/IBM@IBMUS
> Internet e-mail: storman AT us.ibm DOT com
>
> IBM Tivoli Storage Manager support web page:
> http://www.ibm.com/support/entry/portal/Overview/Software/Tivoli/
> Tivoli_Storage_Manager
>
>
> The only dumb question is the one that goes unasked.
> The command line is your friend.
> "Good enough" is the enemy of excellence.
>
> "ADSM: Dist Stor Manager" <ADSM-L AT vm.marist DOT edu> wrote on 2010-08-10
> 13:35:28:
>
> > From: TSM User <user.tsm AT GMAIL DOT COM>
> > To: ADSM-L AT vm.marist DOT edu
> > Date: 2010-08-10 13:40
> > Subject: Re: NBU guy in TSM shop and I need help
> > Sent by: "ADSM: Dist Stor Manager" <ADSM-L AT vm.marist DOT edu>
> >
> > If I'm reading what you wrote in the first post correctly, then you're
> > keeping every version of every file ever created, so long as there is
> still
> > a version of the file active on a client. And when a file gets deleted
> on
> > the client, you're immediately purging every version of that file from
> the
> > database. (In this case, "immediately" actually means "during the
first
> > expiration once a backup has run and detected that the file is gone.")
> So
> > if you have enough files changing and not being deleted, then overall,
> yes,
> > you could be using a large total quantity of tape space to store all of
> > those versions, especially if there are databases in the mix. At the
> same
> > time, you're not providing any real recoverability for files that are
> > accidentally deleted. If you want to post the output of the "query
> > copygroup" command it might clarify things.
> >
> > How to fix it? First order of business is to decide on what is
> appropriate
> > retention for the data, existing and deleted. It's unlikely that you
> really
> > need to keep every version of every current file forever; you could
> probably
> > reclaim a good bit of space if you adjusted that. And get better
> protection
> > by increasing the retention of the deleted files (to anything other
than
> > zero, really!). If you decide to implement collocation for some or all
> of
> > the data, you can allow it to happen naturally as data expires and
> reclaims,
> > or you can use 'move data' or 'move nodedata' to speed it along. It's
> > definitely fixable.
> >
> >
> >
> > On Tue, Aug 10, 2010 at 1:08 PM, ego3456
> <tsm-forum AT backupcentral DOT com>wrote:
> >
> > > wow, so it really is as clear as mud.. :D
> > >
> > > my assumption of the retention taking that many tapes was understood
by
> me
> > > to be a function of not only the shelf life (retention) but the
> reclamation
> > > (no we don't use collocation, which I'm well aware we should)
> shotgunning
> > > those data bits across a large number of tapes; if all this is true,
> without
> > > a large increase in tape library capacity (we have 252 slots) or a
> large
> > > increase in disk resources, is there an easy way to fix it? If not,
> I'm
> > > inclined to scrap it all and put in NBU for backups going forward.
> saving
> > > the argument on which is better, I'm an NBU guy and a staff of one,
and
> my
> > > inclination is if I'm spending buckets of cash, I'm doing it in a way
> I'm
> > > familiar. My first whack though is to try and save this thing so I'm
> hoping
> > > someone out there can provide a panacea or at least an incremental
> > > improvement that is free and time efficient. am i spitting in to the
> wind?
> > >
> > >
+----------------------------------------------------------------------
> > > |This was sent by ericgosnell AT gmail DOT com via Backup Central.
> > > |Forward SPAM to abuse AT backupcentral DOT com.
> > >
+----------------------------------------------------------------------
> > >
|
