Re: [ADSM-L] DISASTER: How to do a LOT of restores?

I believe that every file has an entry in the database.  Aggregates were
designed to reduce the number of transaction commits when moving stuff
around.  The first place this really helped was when moving files from a
client to the server.  Rather than doing transaction commits after each
file, aggregate a bunch of files (based on txngroupmax or txnbytelimit)
and send them along and then commit all of them.  Then once these
aggregates are on the server move them around as an aggregate, again,
cutting down on the amount db commits required.

In the case of active only storage pools, the discussion is correct: one
would think that aggregate reconstruction becomes very important in
order to consolidate active files into aggregates with only active
files.  This requires reclamation which does do aggregate
reconstruction.

I hope I'm not pedantic.  I had to Google that to know what it meant!

Kelly Lipp
CTO
STORServer, Inc.
485-B Elkton Drive
Colorado Springs, CO 80907
719-266-8777
www.storserver.com


-----Original Message-----
From: ADSM: Dist Stor Manager [mailto:ADSM-L AT VM.MARIST DOT EDU] On Behalf Of
Curtis Preston
Sent: Monday, January 28, 2008 1:14 PM
To: ADSM-L AT VM.MARIST DOT EDU
Subject: Re: [ADSM-L] DISASTER: How to do a LOT of restores?

Allen,

Your "pedantic" (to use your word) response was actually quite helpful.
I knew about aggregates, but did NOT know that they were only tracked in
the database at the aggregate level.

---
W. Curtis Preston
Backup Blog @ www.backupcentral.com
VP Data Protection, GlassHouse Technologies 

-----Original Message-----
From: ADSM: Dist Stor Manager [mailto:ADSM-L AT VM.MARIST DOT EDU] On Behalf Of
Allen S. Rout
Sent: Monday, January 28, 2008 10:10 AM
To: ADSM-L AT VM.MARIST DOT EDU
Subject: Re: [ADSM-L] DISASTER: How to do a LOT of restores?

>> On Sun, 27 Jan 2008 13:59:47 -0600, Roger Deschner <rogerd AT UIC DOT EDU>
said:


> But then again, I got to thinking, how hard would it be to do it my
way?
> Not very. All we'd need is a new option on normal migration to only
> migrate inactive files. Everything else is there.

If I understand correctly, then I know why this is not contemplated,
and won't be.

For starters, back up and think about how the 'active data' pool is
currently thought of: it's an extra copy, sort of like a copy stgpool.
It's not the primary; that's important.  You could blow away all the
ACTIVEDATA stgpools and still have all your data.

When the TSM client sends data, it sends things in 'aggregates'
(Roger, I'm not pretending you're unfamilliar with the concepts here,
I'm just being pedantic) which oddly enough 'aggregate' possibly many
files into one addressable unit in storage.

Since most files are quite small, this permits the TSM database of
what-is-where to have drastically fewer entries than you have files.
The thing that is copied to the copy stgpool is the aggregate, not the
set of files.  Ditto (I bet) with the active-data copy.

Gedankenexperiment: Give me FILE as the first pool data hits, (desired
to be maintained as active-only) and then TAPE as the larger-capacity,
desired to accept the inactive data.  If you migrate some of the data
from your aggregate (i.e. the inactive fraction) there are several
implications

+ you increase the number of low-level storage constructs of which you
  must keep track.  Say you backup an aggregate of 50 files to FILE,
  if they go inactive daily, you could have as many as 50 separate
  aggregates present in TAPE by the time you're done.

+ you invalidate the copypool status.  When you next run a BACK STG
  TAPE OFFSITE-COPY, the newly created aggregate in TAPE has -not-
  been copied.  The file is present in the copystg, because it was
  backed up from FILE, but the unit of account is the aggregate.  So
  you have to re-send the new aggregate.


Expensive in every direction, and particularly in database size.  Ick.

So, I don't think active data stgpools will ever be PRIMARY stgpools;
not without profound shifts elsewhere.  If we get to the point where
we're keeping track of individual files instead of aggregates, that'd
be possible.  But it'd be a freakin' HUGE database increase, first.



- Allen S. Rout