ADSM-L

Re: Longlasting tape-reclamation run

2001-03-08 22:39:32
Subject: Re: Longlasting tape-reclamation run
From: Joe Faracchio <brother AT SOCRATES.BERKELEY DOT EDU>
Date: Thu, 8 Mar 2001 19:40:04 -0800
High and low points of one-drive reclamation:

1) its not move data as far as I know.  It looks like this:
              Storage Pool Name: TAPEPOOL
              Storage Pool Type: Primary
              Device Class Name: 3590
        Estimated Capacity (MB): 4,149,171.5
................................
           Reclaim Storage Pool: RECLPOOL    <<<<<  here's the kicker
         Maximum Size Threshold: No Limit
                         Access: Read/Write
                    Description: the pool for all onsite data
              Overflow Location:
          Cache Migrated Files?:
                     Collocate?: Filespace
          Reclamation Threshold: 19
Maximum Scratch Volumes Allowed: 1
  Delay Period for Volume Reuse: 6 Day(s)

2) there's a 'bug' where migration back to tapepool begins almost
immediately if I don't have RECLPOOL set to HI=100 low=99
So consequently I use a perl script that watches and does the right thing
(changes 100/99 to 0/0 when it sees *any* data stored in RECLPOOL.

3) the perl script proved to be extra helpful because I do other things
like:  1 - set tape to Read/Only if its still FILLING otherwise it chases
its tail in re-writing the data to the same tape until it can't.
       2 - add 1 or two scratch tapes to tape pool maxscratch count
everytime it migrates the data back.  Thereby ensuring a user gets his own
tape if he doesn't have one already.
       3 I use an 'are we busy' algorithm to allow other work and enabling
14 hours for reclamation over the course of the day.

4) You have to have mountret > 1 because in this setup a new tape is
written on a few times if the users don't have data on some other tape.

5) is it faster? I don't know , don't care because with only two tape
drives but using only one for reclamation  I can let it run noon to
midnight. (after expire inventory and db backup in the a.m.)  I average
about 5 or 6 reclams a day.

6) its true about collocation and cluster passes : the more users on a
tape the more passes the system makes.  But I think I'm saving the tape
read time since its coming off disk.


7) Using Wanda's pool cap average SQL select statement (THANKS WANDA!!!) I
get numbers like:

     STGPOOL_NAME           STATUS                 AVG_PCTU
------------------     ------------------     --------
COPYPOOL               FULL                      89.03
COPYPOOL               FULL                      89.03
TAPEPOOL               FILLING                   51.80
TAPEPOOL               FULL                      88.43
with 138 filling and 60 tapes full. ( I hope to get it to 150/50
respectively.

What more can I say?  I'm nuts! :-) but with 1000 1-Gig pc/mac users ....
it works.
                ... joe.f.

On Fri, 9 Mar 2001, Geoff Fitzhardinge wrote:

> Joe Faracchio wrote to me:
>
> >Try using a disk-file area and one tape reclamation.
> >
> >... joe.f.
> >
> >Joseph A Faracchio,  Systems Programmer, UC Berkeley
>
> Hello Joe,
>
> Have you had good results with this technique to speed things up?
>
> It would be nice if there was some documentation to explain the practical
> differences in the behaviour of
>
> (1) Reclaim (tape pool to itself)
> (2) Move Data (tape pool to itself)
> (3) Move Data (tape pool to disk pool) followed by Migrate (disk pool to
> tape pool)
>
> There has been discussion on this list, and some suggestions in Tivoli
> documentation,
> on differences between Reclaim and Move Data with respect to how much space
> is
> recovered - Reclaim removes empty space from within aggregates, Move Data
> just
> copies aggregates without reconstructing them.
>
> What is important to me in the present context (relaiming tapes with Notes
> Agent files
> and large numbers of "collocation clusters") is the elapsed time taken by
> the operation,
> and also to some extent the resource consumption.
>
> I find that (1) and (2) are similar and abysmally slow, but use very little
> CPU or database
> I/O (as I said in my original posting, most of the elapsed time is waiting
> for input tape
> positioning).
>
> (3) on the other hand is MUCH faster (typically an hour or two for the Move
> Data and
> same again for the Migrate, although sometimes the Migrate bogs down a
> bit).   CPU
> consumption is quite significant.  It is noticeable that the Move Data to
> disk does not
> issue the ANR1142I messages (counting "clusters") which are issued both by
> the
> Reclaim and by Move Data to tape.
>
> Why is the Move Data to disk so much faster than the Move Data to tape,
> when the input
> tape is the same?
>
> I take it that performance using your suggestion (4), Reclaim to Disk File
> then Migrate
> back to tape pool, will be quite similar to my (3).
>
> I haven't tried (4) because I am reluctant to invest more disk space for a
> housekeeping
> function, and my normal disk pool (uncached) is mostly free during the day.
> Unfortunately
> it is a bit manual, I initially hoped I could automate it by defining the
> disk pool as the reclaim
> pool, but found that this is not allowed to be a random access pool.  Your
> method gets
> around this.  On the other hand, I can overlap multiple Move Datas, but
> only one tape will
> Reclaim at a time.
>
> Cheers,
> Geoff
>