With that many files, I think Richard Sims is on target as to where your
problems will arise. Your database will grow quite a bit, but you can
calculate that from the Admin Guide formulas. Our DB is over 100GB and
it runs fine, so just throw some disk space at it and it will be fine.
One thing you might want to do to reduce the performance impact of this
client's backup on the rest of your TSM system, and on this client's own
workload, is to have it do "journal-based" backups Monday-Friday, and
then a regular "Incremental" backup on the weekend. This eliminates the
huge overhead of extracting this guy's list of files from the database
and downloading it - except on weekends when nobody cares about
performance. But you do want to do a regular incremental at least on
weekends, in case something got overlooked or out of sync. The
Jounal-Based backup method is described in the book "Backup-Archive
Clients Installation and User's Guide". You are lucky that client is
Windows, because that is the only client platform this type backup is
supported on.
Another faster backup method that isn't quite as complete as
journal-based, is "partial incremental", which compares the date/time of
the last backup to the date/time stamp of the file in the file system.
Here, too, you should do a regular full Incremental backup on weekends,
when performance isn't an issue, to get everything caught up and take
care of any loose ends.
Roger Deschner University of Illinois at Chicago rogerd AT uic DOT edu
======I have not lost my mind -- it is backed up on tape somewhere.=====
On Wed, 13 Oct 2004, Richard Sims wrote:
>>Up to now I kept my DB and Rlog sizes to a minimum (96MB and 50MB
>>respectively). Most of my backups are SQL dbases or large files so there was
>>no need to increase them.
>>However, I have a new client (windows) that requires to backup a complex
>>folder structure with about 30 million files! These are cheque images of
>>about 20KB each - the total space is 600GB.
>>I have no problem with my storage pool as I have enough empty tapes.
>>I'm worried about the DB and Rlog sizes that I have to increase - I just
>>don't know how much. A colleague suggested something between 1% and 5% of
>>the size of the data to be backed up.
>>Has anyone faced a similar situation, and if yes, any suggestions on the
>>sizes ?
>>Are there any catches as to how performance is affected by such a large
>>database ?
>
>Yiannakis - Your database and recovery log have thus far been tiny, but
> now it's time for them to grow to handle more challenging
>work. We've all been through this: it's just the demands of reality.
>
>I'd be more concerned about the client and its subsequent impact than the
>TSM database per se. It's not the first backup that's going to be
>painful - it's every incremental after that, as the whole inventory list
>of millions of Active files has to be pulled from the TSM db, sent to the
>client, and there be sorted and scanned. That will make for a long-
>running backup and will severely tax the memory on the client system
>(where other, "real" work will probably be trying to happen at the same
>time).
>
>The people running that client system application should be encouraged to
>pursue files consolidation, amassing files into a single composite using
>an appropriate utility. That will be much easier for them to manage, and
>will greatly reduce backup overhead.
>
> Richard Sims
>
|
