We had an actual disaster last spring. An 80 gigabyte Unix file system
that contained all of our email became corrupted. This node was being
backed up both by TSM (with collocation, as this is obviously a critical
node), and by old-fashioned full+incremental backups.
We had a restore race. TSM lost, but not by much. Close enough to call
it a tie, and we were very pleasantly surprised that it was this close.
3:45 for TSM, versus 3:15 for the old-fashioned restore. If we could
plan all our disasters for the day after the full backup, like this one
was, the full+incremental method would always win. But by day 6 of the
weekly cycle, TSM would become considerably faster.
With a full+differential scheme, restore time might be faster, but
differential backups are so much more costly to make. By the end of the
cycle, a differential backup could practically be a full backup.
Consider all those unnecessary copies of a file that changed only once,
on day 1 of the cycle.
In theory, the worst case without collocation is that a node's files
could be scattered over all the tapes in your library. In practice, that
is not the case. I'm not sure why, but I think that migration and
reclamation at least make an effort not to make the scattering any
worse. Larger disk storage pools definitely help reduce the scattering.
I completely agree with Don France about taking the time to accurately
separate your nodes by criticality. We do this, using exactly his
three-tiered scheme.
The key to TSM's progressive backup scheme is that it is totally
database-driven. Don't try to circumvent it by imposing periodic full
backups on it - TSM _ALWAYS_ has a full backup of each node! Every day,
not just on the day after your full backup. It's in the database. This,
however, takes a slight "leap of faith" for us who were brought up in
the full+incremental cycles of mainframe days. It's easier if you forget
about individual tape volumes, and consider your entire tape library and
its robot to be one huge tape.
Do away with those full backups, and your tape drive bandwidth and
database size problems should be significantly reduced. What is the
purpose of a full backup? To reduce restore time in a disaster. You're
better off achieving that goal with 1) collocation for the most critical
nodes, and 2) TSM Database disk tuning. That's how TSM earned a virtual
tie in the restore race in my actual disaster.
Roger Deschner University of Illinois at Chicago rogerd AT uic DOT edu
Academic Computing & Communications Center
On Thu, 6 Mar 2003, William Rosette wrote:
>I agree with A), B) I'm not sure about the "consolidating the tape mounts",
>could be my DIRMC for NT file servers are off. On the (suppose C)) I am
>trying to implement but I am having trouble with Month Ends. Currently we
>have 117 nodes and 31 maximum days in a month. I run 2 backupsets that use
>4 tape drives giving 6 left for daily and nightly operations. Each
>backupset takes about 24 hours and is getting longer because of
>nocollocation and reclamation spreading the information further out. What
>I want to do is reduce my storage pools to criticality (my own word haha)
>and also Do Monthly Absolute backups. Problem is disk space will double on
>the database side. Backupset minimum impact on DB where Monthly Absolute
>(same as full) increases DB considerable, especially 117 x 12 months. I am
>having a hard time convincing the upper powers that the criticality and
>Monthly Absolutes are the best way.
>
>Thank You,
>Bill Rosette
>Data Center/IS/Papa Johns International
>WWJD
>
>
>
> DFrance
> <DFrance-TSM@ATT. To: ADSM-L AT VM.MARIST
> DOT EDU
> NET> cc:
> Sent by: "ADSM: Subject: Re: progressive
> backup vs. full + incremental
> Dist Stor
> Manager"
> <[email protected]
> .EDU>
>
>
> 03/06/2003 03:09
> PM
> Please respond to
> "ADSM: Dist Stor
> Manager"
>
>
>
>
>
>
>A) The speed of new tape drives (eg, 9840 & 3590) with their mid-point
>load mostly mitigates the restore speed issue; even IBM's LTO or STK's
>9940 seem to be sufficiently fast, they're more like the speed of disk of
>just a few years ago;
>
>B) TSM further mitigates restore speed by consolidating the tape mounts for
>a given restore -- provided you properly implement DIRMC for your NT file
>servers!@!
>
>If you cannot tolerate collocation, consider smaller storage pools,
>organized by mission-critical vs. production vs. non-prod/desktops...
>personally, I've found that when a customer takes the time to properly
>identify his mission-critical servers (from a file-system recovery
>perspective), collocation is needed for less than 20% of the "farm"! And,
>for the business-critical servers (if that different than "mission-critical
>for file-system recovery), it's more appropriate to use HACMP solution
>(possibly, in concert with collocation).
>
>Hope this helps!
>
>
>Don France
>Technical Architect -- Tivoli Certified Consultant
>Tivoli Storage Manager, WinNT/2K, AIX/Unix, OS/390
>San Jose, Ca
>(408) 257-3037
>mailto:don_france AT ayett DOT net (change aye to a for replies)
>
>Professional Association of Contract Employees
>(P.A.C.E. -- www.pacepros.com)
>
>
>
>-----Original Message-----
>From: ADSM: Dist Stor Manager [mailto:ADSM-L AT VM.MARIST DOT EDU]On Behalf Of
>William Rosette
>Sent: Wednesday, March 05, 2003 7:22 AM
>To: ADSM-L AT VM.MARIST DOT EDU
>Subject: Re: progressive backup vs. full + incremental
>
>
>This progressive incremental confuses me. This is what I thought was going
>on:
>
>1. Differential is all changes from last FULL backup.
>2. Incremental is all changes from last ANY backup
>3. Full is all files not matter change on backup.
>
>We used to do Differential with Weekend Fulls. During a restore we would
>restore Full if file had not changed, and then the last differential for
>the other files that changed. We never restored more than necessary. It
>depended on the restore. Restore 1 file was the same on Differential as
>Incremental (most current or before corruption). The problem comes when
>you are restoring directories or a whole system as in DR. In the
>Differential Weekend Full world you would restore Full and lay on top the
>last Differential and your done. Always 2 restores was all and restores
>flew since data was all together.
>
>Now the TSM world has its own database with its own reclamation,
>expiration, migration, collocation, and the works. Come restoring 1 file it
>is the same as above. Come restoring directories or whole systems it will
>depend where all the data is. Our case, if the restore is not far back we
>have a quick restore, but the further back we go in date the slower the
>restore because of the no-collocation and the data is spread over 100's of
>tapes. This seems to be the same as an Incremental that the TSM database
>keeps track of every file from every tape.
>
>Thus, the reason we did not use Incrementals before was 1. Restores were
>long, 2. No database to keep track of all Incremental tapes, 3.
>Differential & Full used less tapes, and 4. money. I am still dealing with
>the progressive incremental that progressively eats resources/money. My
>suggestion would be for reclamations to reclaim to a collocate status or
>somehow keeping the data together as it gets older. Right now I am
>probably going to run FULLs just to keep my restores to a minimum since the
>tape issue will hurt us if we collocate
>
>If I am off, I would appreciate anyone that can straighten out my
>backups/restores.
>
>Thank You,
>Bill Rosette
>Data Center/IS/Papa Johns International
>WWJD
>
>
>
> Gianluca Mariani1
> <gianluca_mariani@ To:
> ADSM-L AT VM.MARIST DOT EDU
> IT.IBM.COM> cc:
> Sent by: "ADSM: Subject: Re: progressive
> backup vs. full + incremental
> Dist Stor Manager"
> <[email protected].
> EDU>
>
>
> 03/05/2003 09:47
> AM
> Please respond to
> "ADSM: Dist Stor
> Manager"
>
>
>
>
>
>
>Progressive incremental backs up only new or changed files. during the
>initial backup the client backs up all eligible files of course(full
>backup). Subsequently, files are backed up again only if they are new or
>have changed since the last backup. In TSMs case, a pointer to each version
>of every file for every client is kept in the database , so there is no
>need for another full backup.
>When you need to restore, you can choose the specific version of the file
>or point-in-time to restore, and TSM will restore only that particular
>file or files. The approach used for full + incremental backups (NetBackup)
>requires an initial full backup, followed by regular incremental or
>differential backups (usually once a day), with the complete cycle needing
>a full backup to be repeated on a regular (usually weekly) basis. This
>backup method results in redundant weekly full backups of files that have
>not changed, wasting both network and media resources. The multi-step
>restore process in this approach requires the software to restore the last
>full backup, then to restore incremental or differential backups on top
>of that in order to recover the latest version of a file or an entire
>system. This methodology not only involves restoring more data, it also
>means more tape mounts and tape positioning and consumes more network
>bandwidth all of which amounts to having longer restore times.
>
>
>
>Cordiali saluti
>Gianluca Mariani
>Tivoli TSM Global Response Team, Roma
>Via Sciangai 53, Roma
> phones : +39(0)659664598
> +393351270554 (mobile)
>gianluca_mariani AT it.ibm DOT com
>----------------------------------------------------------------------------------------------------
>
>
>
>The Hitch Hiker's Guide to the Galaxy says of the Sirius Cybernetics
>Corporation product that "it is very easy to be blinded to the essential
>uselessness of them by the sense of achievement you get from getting them
>to work at all. In other words ? and this is the rock solid principle on
>which the whole of the Corporation's Galaxy-wide success is founded
>-their fundamental design flaws are completely hidden by their
>superficial design flaws"...
>
>
>
> Joni Moyer
> <joni.moyer@HIG
> HMARK.COM> To
> Sent by: "ADSM: ADSM-L AT VM.MARIST DOT EDU
> Dist Stor cc
> Manager"
> <ADSM-L AT VM DOT MARI
> bcc
> ST.EDU>
> Subject
> progressive backup vs. full +
> 05/03/2003 incremental
> 15.21
>
>
> Please respond
> to "ADSM: Dist
> Stor Manager"
>
>
>
>
>
>
>Hello everyone!
>
>I was wondering why the full + incremental would result in a longer restore
>time than the progressive backup methodology? From several co-workers
>point of view they thought that it would be quicker on the full +
>incremental because you wouldn't have to go back to the beginning backups
>of the file and restore all of the incrementals, you would just go back to
>the most recent full backup and apply the incrementals after that point.
>When I went to explain the reasoning behind this, I had some problems
>understanding the concept myself, so I was hoping someone could explain
>both methods and why they differ in restore time and why progressive is
>better than the full + incremental. Thank you so much for any help you can
>lend on this matter!
>
>
>
>Joni Moyer
>Systems Programmer
>joni.moyer AT highmark DOT com
>(717)975-8338
>
|
