I'm all for a Consolidate command. One should be able to Consolidate
an entire node and/or individual file spaces. The suggestion of being able
to specify a trigger value "x tapes", is a good one. I would like to
suggest one additional parameter: ACTIVE=YES/NO If set the YES, only
the active files would be consolidated onto the new set of tapes.
With a command like this, I would have to have mulitple disk pools just to
support different consolidation rules
On Wed, 17 Mar 1999, Kelly J. Lipp wrote:
> Geez!
>
> Keep your fingers crossed hoping not to have to restore this guy.
>
> What about changing the node name? This would force new filespaces and
> effectively a full backup. Then wait 60 days (retonly) and delete the old
> filespaces. Probably easier than and export/import.
>
> Let's kick around a better way to do reclamation that will help to ease
> this "feature".
>
> What about setting collocation on before a reclaim? What about changing
> the algorithm for reclaim in the server code itself. Perhaps the engineers
> can jump in here with ideas they've kicked around.
>
> How about a new ADSM command: Consolidate filespace. This command would
> determine which tapes contain a filespace, mount those tapes and move the
> data to new tapes. This could be done on a weekly/monthtly basis for
> problem filespaces. Better yet, do it on a threshold. If a filespace
> begins to span more than x tapes, then consolidate. Once the data has been
> moved, it is expired on the original tapes and reclamation can do its
> thing.
>
> Simple matter of programming. Should take a week or two, don't you think?
>
> Kelly J. Lipp
> Storage Solutions Specialists, Inc.
> www.storsol.com
> lipp AT storsol DOT com
> (719)531-5926
>
> -----Original Message-----
> From: Dusedau, Stefan
> Sent: Wednesday, March 17, 1999 12:28 PM
> To: ADSM-L AT VM.MARIST DOT EDU
> Subject: Re: restore times
>
> Kelly,
>
> We do not have collocation turned on since we could not afford the number
> of
> tapes it would require. I just ran a simple SQL (select node_name, count(*)
> from volumeusage group by node_name). This produces a list of how many
> volumes contain data for each node on the server. The top hitter is a node
> that has been backing up for 26 months and uses 1036 volumes. Of course
> this
> includes drm copies, archives and inactive copies so the actual number for
> a
> restore will be less. I have spoken with IBM about this on several
> occasions
> and they have recommended exporting and importing the nodes. This would
> probably take a week to do for a node using 500 tapes. By the way we have
> our copy group setup as VERE=7 VERD=1 RETE=45 RETO=60.
>
> Also, I have found that the average tape mount on a 3494 is between 30 and
> 60 seconds. This is for 5 3494s connected to AIX and AS/400 environments so
> I believe that it is the 3494s not the operating system.
>
> I am interested in to hear what others have done to help in this area.
>
> Thank You,
>
> Stefan Dusedau
> infoWorks
> A Viacom technology service
> (212)258-6739
> stefan.dusedau AT viacom DOT com <mailto:stefan.dusedau AT viacom DOT com>
>
>
> -----Original Message-----
> From: Kelly J. Lipp [mailto:lipp AT storsol DOT com]
> Sent: Tuesday, March 16, 1999 6:09 PM
> To: 'Dusedau, Stefan'
> Subject: RE: restore times
>
> I don't think the reclamation process would spread data on
> that many tapes.
> I wonder what actual experience will show with a system
> that has been
> running for a year or more.
>
>
> Kelly J. Lipp
> Storage Solutions Specialists, Inc.
> www.storsol.com
> lipp AT storsol DOT com
> (719)531-5926
>
> -----Original Message-----
> From: Dusedau, Stefan
> Sent: Tuesday, March 16, 1999 1:08 PM
> To: 'lipp AT storsol DOT com'
> Subject: RE: restore times
>
> Kelly,
>
> I agree with most of what you said regarding tape mounts.
> The one
> difference
> is that 30 day retention does not mean 30 tapes. If you
> nave
> a node that
> has
> been backing up for a year and file "A" has not changed or
> been deleted in
> that time, it may be on any of the hundreds of tapes that
> have been used
> during reclamation. Now multiply this by the number of
> files
> that do not
> change often and you have spread your data across more
> tapes
> then the last
> 30 days have used.
>
> Thank You,
>
> Stefan Dusedau
> infoWorks
> A Viacom technology service
> (212)258-6739
> stefan.dusedau AT viacom DOT com
> <mailto:stefan.dusedau AT viacom DOT com>
>
>
> -----Original Message-----
> From: Kelly J. Lipp
> [mailto:lipp AT STORSOL DOT COM]
> Sent: Tuesday, March 16, 1999 11:30 AM
> To: ADSM-L AT VM.MARIST DOT EDU
> Subject: Re: restore times
>
> If one does a realistic examination of
> restore times, tape
> mounts, unless
> you have very slow tapes, usually won't
> come
> into play.
>
> Let's start the discussion with service
> level agreements.
> For instance,
> all of us should have in place agreements
> with our customers
> that sound
> something like this:
>
> 1. For single file restores, we offer a
> restore time of one
> hour or less.
> 2. For a directory tree restore up to 200
> MB, we offer a
> restore time of
> one hour or less.
> 3. For a full disk restore up to 4 GB, we
> offer a restore
> time of four
> hours or less.
> 4. For a full system restore up to 100 GB,
> we offer a
> restore time of 24
> hours.
>
> Wiggle these to meet your needs.
>
> Think about what many of us can offer now.
> Are the numbers
> above
> reasonable from a business standpoint?
> Most
> of us state
> we'll do it as
> fast as possible when we should be putting
> in place SLAs
> that more
> accurately reflect reality.
>
> Restore statistics:
>
> 99% of restores are for single files, or
> small numbers of
> files from
> yesterday. Not a problem with ADSM.
> 1% of the restores are bigger than this:
> complete
> directories, all the data
> for an application, a complete disk, a
> complete system.
> This flat does not
> happen very often, but one should be
> concerned (but not
> overly so) about
> it.
>
> Let's look at the larger restores. Pick a
> hard one like the
> full disk
> restore. Assume a 30 day data retention
> (verdeleted,
> verexists, retextra,
> retonly all set to 30). Worst case, 30
> tape
> mounts will be
> required during
> a full restore of the disk. Most of the
> data will be on the
> first tape
> with small amounts scattered on the rest of
> the 30. In
> fact, looking at
> typical usage, the same files tend to
> change
> so the same
> files will show up
> on many of the tapes. So during a restore,
> these files will
> only be
> restored once, and perhaps from the most
> recent tape. Tape
> mounts just
> went way down.
>
> But let's keep it at 30 mounts. Say each
> mount takes two
> minutes and the
> time to reach data is another minute for a
> total of three
> minutes per tape.
> Then we can do some real work and move
> data. For this
> case, we'll expend
> 90 minutes of our four hour window. Are we
> in trouble? I
> don't think so.
> We should meet our SLA for this restore.
> Will we actually
> need to mount
> 30 tapes? Probably not, since we've got
> reclamation going
> on as well.
>
> If tape mounts really become a problem, and
> I'm maintaining
> this is the
> least of your worries about large restores,
> one can always
> use collocation
> to reduce them.
>
> Think about your data. It's usage and its
> change patterns.
> Think about
> your other protection mechanisms: RAID,
> shadowing, etc.
> Think about your
> restore requirements and SLAs. For those
> of
> you still
> believing GFS
> (Grandfather, father, son) backups are
> still
> better, think
> about this: you
> have to mount all of the incrementals
> between a full even
> though most of
> same files change everyday. Using this
> method, more files
> are actually
> restored and then deleted than ADSM would
> actually restore!
>
> BTW, one should test the restore times
> before publishing
> SLAs. However, if
> one applies reason and logic, this just
> isn't all that hard.
>
> Thanks,
>
> Kelly J. Lipp
> Storage Solutions Specialists, Inc.
> www.storsol.com
> lipp AT storsol DOT com
> (719)531-5926
>
> -----Original Message-----
> From: McAllister Craig-WCM033
> Sent: Tuesday, March 16, 1999 9:00 AM
> To: ADSM-L AT VM.MARIST DOT EDU
> Subject: Re: restore times
>
> That certainly appealed to me at one point,
> however, this
> will "push" one
> version of your files out of the queue you
> have made from
> your management
> class. (Selective backup will keep two
> copies of exactly the
> same file, and
> may push one older, different version out
> into expiration)
> For this reason, perhaps if you want this
> type of
> functionality, you should
> be looking at file archival, rather than
> backup.
>
> It certainly works for me.
>
> -Craig.
>
> -----Original Message-----
> From: Fluker, Tom R
> [mailto:Tom.Fluker AT VIASYSTEMS DOT COM]
> Sent: 16 March 1999 15:34
> To: ADSM-L AT VM.MARIST DOT EDU
> Subject: restore times
>
>
> It appears to me that one of the problems
> with restores
> taking so long is
> that the incremental nature of ADSM will,
> over time, spread
> the required
> files over many volumes in a tape storage
> pool. Requiring
> mounts, and
> searches, of multiple tapes requires a lot
> of time.
>
> Has anybody tried, say once a month,
> performing selective
> backups for
> entire
> file systems in an effort to aggregate all
> the files
> required for a restore
> in reasonable proximity to each other?
> Incremental backups
> would then get
> scattered but it may limit the number of
> tapes that would be
> required.
> Would this help the restore time problem?
>
> I'm considering such an idea and was
> wondering if anybody
> has had any
> similar experiences that may help.
>
> Tom Fluker
> Tom.Fluker AT viasystems DOT com
>
|
