On Apr 5, 2006, at 5:26 PM, Bell, Charles (Chip) wrote:

I've posed the sob story on here before, but I have a client that
has 39
NTFS mounts with millions of files on each. When backing this up on an
incremental basis, it takes multiple days now. Also, the TSM database
grows at a rapid rate, and I have PMRs open on both issues. ...

Charles -

I would perform some test runs under client tracing to fully define
where the time is actually going in your backups...not just in terms
of categorical area (network, active files list acquisition, file
system traversal, retries, etc.) but also in specific area such as a
particularly bad directory.  This will give you some solid
perspective on what best can be done.

You can look at the "Backup performance" and "Many small files"
topics in ADSM QuickFacts for pointers.  Disk fragmentation is
probably a big hurt there.  A good disk analysis package may be in
order.  Old-technology disks may also be hampering throughput and
seek times.

This is where site data architecture comes in...  Keeping millions of
unused files on costly, spinning disks is nuts, and that's where
archiving and HSM would be of value.  Not just TSM HSM, but
architectural hierarchical data management, such as shifting aging
data to secondary storage.  After all, there's zero value and pure
waste in continually traversing data files to ascertain backup
candidacy when it's known to be unreferenced data.

Many opportunities here for analysis and cure.  Remember that, as
Mason Williams said, tradition is a wonderful thing, but sometimes it
stays around too long.  Don't look at a data store and think that it
has to be accommodated as is, but rather question why it should be as
it is.  Windows systems in particular are notorious for being
neglected, pack-rat accumulations of useless data whose
administrative costs only increase over time, due to organizational
inertia and lack of data governance.

    Richard Sims