Networker

Re: [Networker] slow or failed backup due to too many files

2003-07-02 14:33:05
Subject: Re: [Networker] slow or failed backup due to too many files
From: George Sinclair <George.Sinclair AT NOAA DOT GOV>
To: NETWORKER AT LISTMAIL.TEMPLE DOT EDU
Date: Wed, 2 Jul 2003 14:33:00 -0400
If there are a reasonable number of top-level parent directories then
you could simply attack all of them simultaneously by listing them as
separate savesets. For example, instead of specifying 'All', for the
given client, how about something like:

/filesystem/path1
/filesystem/path2
/filesystem/path3
/filesystem/path4

etc, etc. You could simply list the largest directories, and then
capture the rest by adding:

/filesystem

as the last entry and then create a /filesystem/.nsr file that skips the
other named paths. This way, if anyone adds anything, you'll capture it.
NetWorker can backup any file system significantly faster when you break
it up in pieces as it multi-plexes the streams, running each at the same
speed as it would have done the whole -- well, no less fast anyway --
and thus it can finish the whole thing wayyy faster. Also makes it nice
if there's a failure as you don't have to re-start the whole enchilada
from scratch. You might have to adjust your server and client
parallelism and the number of allowed sessions per device. I generally
do not set the device sessions above 5 myself.

George

Adam Jackson wrote:
>
> I have a lan running 100m full backing up a number of Sun boxes, all of the 
> boxes backup very quickly expect one (<100k/s at times).
> The server I am having problems with has a very large file system (in terms 
> of number of files).  It contains an old database that uses a flat file
> system.  The database consists of about 700,000 plus very small files layed 
> out on many individual directories, which I believe is the result of the
> slow backups which ultimately end up failing.  I have transferred this 
> filesystem to other machines and still have the same problems, so I am
> pretty sure it is not the machine, but the large number of small files on it.
>
> Is there any way to tweek networker to handle backing up a large number of 
> small files ??
>
> Thanks in advance for you help
>
> Adam
> adam.jackson AT bell DOT ca
>
> --
> Note: To sign off this list, send a "signoff networker" command via email
> to listserv AT listmail.temple DOT edu or visit the list's Web site at
> http://listmail.temple.edu/archives/networker.html where you can
> also view and post messages to the list.
> =*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=

--
Note: To sign off this list, send a "signoff networker" command via email
to listserv AT listmail.temple DOT edu or visit the list's Web site at
http://listmail.temple.edu/archives/networker.html where you can
also view and post messages to the list.
=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=