The bogus part of Steven Cashman's kind reply was

   "So NBU has to scan each file / folder and compare it against our 
include and exclude."

What it "has" to do is scan each file *if* an include might be applicable.

Unfortunately, this is a design flaw, not "bug", unless someone is good 
buddies with someone in high places ;-)

cheers, wayne

Mark.Donaldson AT cexp DOT com wrote, in part,  on 1/5/2006 12:13 PM:
> A copy for the list - seems like an issue that we'd all be interested in.  A
> reply from a Symantec employee is below, Ed & my replies are below that.
>
> -----Original Message-----
> From: Steven Cashman 
> To: Ed Wilts; Mark Donaldson
> Subject: RE: [Veritas-bu] Exclude then search or vice-versa?
>
>
> Ed and Mark
>
> Question - Does NB, to anybody's knowledge, build a list of all changed
> files then apply the exclusion rule or does it do the opposite.
>
> Answer - From my Exp its neither, it never builds a complete list of all
> changed files. But maybe little of both
>
> NBU uses a MS call (on windows) to get a starting file / folder. This
> structure is read and each file is compared against our exclude list. If
> the file / folder is set to be backed up (archive bit set) and the file
> is not in the exclude list it is backed up. We then move onto the next
> file / folder and repeat the process.
>
> So even when you exclude a large sub directory from backups NBU still
> has to scan each file under that dir to see if it should back it up. I
> would not consider this a bug because you can exclude /parent/sub1 yet
> put an include in the same policy for /parent/sub1/exec/ So NBU has to
> scan each file / folder and compare it against our include and exclude.
> If NBU was changed to not scan a directory when it meet a parent exclude
> this would not be possible.
>
> The only thing I can think of is trying to move the folders with many
> small files to another drive so they can be backed up that way.
>
> Hope that helps
>
>
> ....to which Ed replied:
> -----Original Message-----
>
> Thanks for the explanation Steve.  Here's my problem though, and I'm
> guessing that Mark's is similar.
>
> RWare, a commercial product, creates hundreds of thousands of small
> (mostly empty) files in /var/tmp on Solaris.  I need to backup /var
> which should take well under an hour.  Because of all of the files in
> /var/tmp, the backup takes well over 16 hours even though I'm excluding
> /var/tmp.  Obviously I'm taking a hit on the client doing the
> comparison and wasting resources.  How can I speed this backup up?
>
> Thanks again,
>         .../Ed
>
> ....and I countered....
> -----Original Message-----
>
> My product is Documentum but the concept is right on.  Our Docbase is huge,
> growing fast, and fits the millions of small files profile.
>
> I have no include list for this policy.  The contents of /parent are
> variable but I always want to exclude backing up /parent/sub1.  Backing up
> an open Docbase for Documentum is of no value - this size of docbase has to
> be done cold.  Both Master & Client are Unix (The same server, in fact).
>
> A workaround I can I can see is to specify, for my example, the /parent/sub3
> directly in the filelist for the policy and not sub1 & sub2.  The problem
> is, of course, that files located directly in /parent won't be backed up (A
> problem for Ed's "/var" situation) and now I've got a real maintenance
> problem whenever the contents of /parent change at all.
>
> Another work-around would be to not backup the entire filesystem, opting for
> a script generated list of items to be backed up, followed in the end by a
> user-backup.  Fulls & incrementals become manually defined.  This would be a
> major hassle, of course, and subverts the reason to buy any enterprise
> backup product.
>
> A partial "fix" to this would be to prevent the scan of excluded directories
> if there's no applicable include file for the backup.  In my experience, the
> use of an include file is pretty rare so this might be an effective
> optimazation.  In the event an include file exists, then do the subdir scan
> & compare as required.
>
> -M
>
> ...and that's all there is so far...
> _______________________________________________
> Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu
> http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
>