Veritas-bu

Re: [Veritas-bu] NBU 6 Tuning for Small Files Backup

2007-06-13 13:25:48
Subject: Re: [Veritas-bu] NBU 6 Tuning for Small Files Backup
From: "bob944" <bob944 AT attglobal DOT net>
To: <veritas-bu AT mailman.eng.auburn DOT edu>
Date: Wed, 13 Jun 2007 13:11:41 -0400
> The performance gain for FlashBackup depends on the overhead 
> of your file system operations.  For NTFS, the overhead is
> quite high (but our big file systems are in the millions of
> files).  Additionally, NTFS is not very efficient in walking
> the file system in terms of allocating resources.

Word.

Related comments:
o  Several years ago I had to back up some software distribution systems
with zillions of files and lots o' disk (Yet Another Misapplication of
Windows, NTFS and PCs).

It was ugly.  I could demonstrate the extent of ugly to PC people who
didn't think that a command line was some sort of Tool of the Devil or
Dirty Trick with

  time <nul & dir /s >nul & time <nul

showing anything from 42 minutes to a couple of hours--just to traverse
the directory structure from that point.  So (referring to an earlier
post) I don't think "it just takes forever for NBU to walk the
filesystem" speaks to the problem, which is that it takes _NTFS_ forever
to walk the filesystem--one needs to work on improving or bypassing NTFS
to make improvement.  BTW, re-running that line will show a small
fraction of the elapsed time--good directory caching in NTFS, I assume.
(3m58s for my laptop's C:/ drive the first time, 1m00s the second.)

o  This finding prompted the app vendor to look into it and they claimed
to discover a Microsoft bug that caused a directory-walking exercise as
above to pause for long periods when going over the ~500,000-files mark.
For all I know, that might have been buffer flushing or maybe it was a
real bug.  I don't have those boxes any more.

o  Backing up one problem Windows system (I'll skip the easy lay-up)
took 34 hours with Nortan AntiVirus enabled and 17 hours with it
disabled.


_______________________________________________
Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu