They key issue here is that you have millions of files on one
filesystem...   You will find that the client kernel is consuming a lot
of cpu and memory in its efforts to walk down the filesystem tree.

So firstly, are you comfortable with the amount of CPU and memory
available?   If you can beef up the memory you might  see some
improvement, also having more than one cpu core should help.

I have also been told there are a few kernel adjustments you can make to
increase the heap size available, and some other buffer settings to
optimise filesystem priority... Of course it all depends on what
application needs to run on the system also.... Unfortunately I leave
that up to my windowsey colleagues and I don't know the settings needed.

Other things to consider:

Win2k3  - I believe has some filesystem performance improvements over
win2k.

Change Journal -   using the change journal feature will help with
incremental and differential backups because it obviates the need to
walk the directory trees. Otherwise I find that incrementals of complex
filesystems like this take almost as long as a full...

If you have the resources on your backup server to do consolidated
backups (3 drives are needed) and you use the change journal your full
backup will be a level 1 followed by tape cloning on the server.

Backup to adv_file...    The problem with a slow client is it's unable
to stream data fast enough to keep your tape device spinnning, so you
get a double penalty because you spend a lot of time waiting for the
tape drive to retrain - backing up to disk can help.

Another filesystem type.....   No idea what would happen here, but
veritas do have a version of VxFS for windows... Maybe someone out there
has some experience.

NAS:    put your data on a NAS, and use NDMP to back it up.   This makes
the filesystem traversal the problem of the vendor, some vendors are
better than others in this regard.



-----Original Message-----
From: Legato NetWorker discussion [mailto:NETWORKER AT listserv.temple DOT edu]
On Behalf Of Dewhirst, Rob
Sent: Wednesday, 7 June 2006 8:49 AM
To: NETWORKER AT listserv.temple DOT edu
Subject: [Networker] Slow Win2k backup with lots of files

I have a Win2k server with over 3.5 million tiny files on it comprising
about 200GB. (not by my choice or design)

I see from the archives that older versions of the client had problems
with large file systems structured like this.  I am using 7.1.x, and my
backup times are over 24 hours only for this client.  My Networker
server is a windows server.

Any suggestions on improving the performance here?  I have a lot of
options available, but I can't skip the millions of tiny files, nor
require that users tar/zip up these directories of files.  

To sign off this list, send email to listserv AT listserv.temple DOT edu and
type "signoff networker" in the body of the email. Please write to
networker-request AT listserv.temple DOT edu if you have any problems wit this
list. You can access the archives at
http://listserv.temple.edu/archives/networker.html or via RSS at
http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER



NOTICE
This e-mail and any attachments are confidential and may contain copyright 
material of Macquarie Bank or third parties. If you are not the intended 
recipient of this email you should not read, print, re-transmit, store or act 
in reliance on this e-mail or any attachments, and should destroy all copies of 
them. Macquarie Bank does not guarantee the integrity of any emails or any 
attached files. The views or opinions expressed are the author's own and may 
not reflect the views or opinions of Macquarie Bank.

To sign off this list, send email to listserv AT listserv.temple DOT edu and 
type "signoff networker" in the
body of the email. Please write to networker-request AT listserv.temple DOT edu 
if you have any problems
wit this list. You can access the archives at 
http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER