Wanda,
Yuck.
The short answer is that, yes, moving from a 32-bit OS to a 64-bit OS and 
piling as much memory as you can afford will give you some relief - especially 
during full scan backups (when the journal gets invalidated).
Depending on the nature of the file structure (many directories vs kazillions 
of files in a single directory), with a powerful enough server and fast disk, 
you can play games such as creating multiple nodes within each server, each of 
which is responsible for backing up a portion of the file tree.
Having said that, I sure wouldn't want to be the one who has to administer the 
thing.
I know you said they cant change the app, but I have found it is surprising how 
that can change when higher mgmt finds out that the system has designed itself 
to be unrecoverable in the event of a disaster.
Good luck,

Steve Schaub
Systems Engineer II, Windows Backup/Recovery
BlueCross BlueShield of Tennessee

-----Original Message-----
From: ADSM: Dist Stor Manager [mailto:ADSM-L AT VM.MARIST DOT EDU] On Behalf Of 
Prather, Wanda
Sent: Friday, February 25, 2011 2:03 PM
To: ADSM-L AT VM.MARIST DOT EDU
Subject: [ADSM-L] Windows servers with a kazillion files and Win2K8...

I have a site with an application that generates kazillions of tiny files that 
are stored forever.
I've already yelled about it, but it's a purchased, customer-facing black-box 
app that they really can't change.
(Naturally, when it was bought umpty years ago, nobody thought about the 
problem reaching this size or what the ramifications would be.)  Every day the 
app creates more files.

They have multiple Win2K3 servers that already have multiple luns containing 
over 35M files each, one is over 75M files.

We are using journaling to back them up successfully (most days).
But it's a struggle just to expand the file tree with Windows explorer, and 
there are exposures on the days when the journal gets overrun (takes 72 hours 
for TSM to scan the filesystem and revalidate the journal).

Looking for anything that might help save our bacon.

Has anybody had experience with this issue and Win2K8?
Does Win2K8 do any better than Win2K3 at handling huge numbers of files in 1 
NTFS directory?
Upgrading the OS is something application-independent we might be able to do.

Thanks for any insight!
W


Wanda Prather  |  Senior Technical Specialist  | wprather AT icfi DOT 
com<mailto:wprather AT icfi DOT com>  |  www.jasi.com<www.jasi.com%20>
ICF Jacob & Sundstrom  | 401 E. Pratt St, Suite 2214, Baltimore, MD 21202 | 
410.539.1135
-----------------------------------------------------
Please see the following link for the BlueCross BlueShield of Tennessee E-mail 
disclaimer:  http://www.bcbst.com/email_disclaimer.shtm