Re: [Networker] NetWorker terribly slow with large number of small files

Small files can cuase problems for any filesystem based backup, basically 
the time spent reading the information about the file becomes greater than 
the time taken to read the contents of the file.
While it is possible that that the way Networker interacts with the OS is 
not as effiecient as BackupExec ( and I don't know) it is also possible 
that things have just got worse ( more files, smaller average file size or 
more fragmented disc). It is also possible that they backed the files up 
to a locally attached drive and presumably you are using the network.
The best improvement is to avoid the filesystem altogether and backup the 
disk blocks directly, read the list or talk to EMC for these options.

On Mon, 3 Mar 2008 08:52:02 -0500, Francis Swasey <Frank.Swasey AT UVM DOT EDU> 
wrote:

>I am converting a Windows cluster that was being backed up with
>BackupExec to NetWorker (absorbing another departments backups into our
>central system).
>
>The problem is that there is this one resource cluster that has a single
>disk with 1,627,737 files taking up 25GB.  So, the average file size is
>16.1KB.  NetWorker is backing this up in six and a third hours at a rate
>that is barely 1.1 MB/second.  The customer is complaining bitterly
>because his last recorded BackupExec full of this system (with 1,219,074
>files and 23.36 GB) was completed in one and a half hours at a rate of
>4.43 MB/second.
>
>I know this comes up from time to time, but sadly, I seem to recall that
>the end of the conversation is typically that NetWorker is just slow
>with lots of small files.  Are there tactics that others are using to
>speed things up?  Anyone with BackupExec experience that can tell me if
>what the customer is telling me is reasonable?
>
>--
>Frank Swasey                    | http://www.uvm.edu/~fcs
>Sr Systems Administrator        | Always remember: You are UNIQUE,
>University of Vermont           |    just like everyone else.
>   "I am not young enough to know everything." - Oscar Wilde (1854-1900)
>
>To sign off this list, send email to listserv AT listserv.temple DOT edu and 
type "signoff networker" in the body of the email. Please write to 
networker-request AT listserv.temple DOT edu if you have any problems with this 
list. You can access the archives at 
http://listserv.temple.edu/archives/networker.html or
>via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER
>=========================================================================

To sign off this list, send email to listserv AT listserv.temple DOT edu and 
type "signoff networker" in the body of the email. Please write to 
networker-request AT listserv.temple DOT edu if you have any problems with this 
list. You can access the archives at 
http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER