Networker

Re: [Networker] slow or failed backup due to too many files

2003-07-02 14:52:14
Subject: Re: [Networker] slow or failed backup due to too many files
From: Carlos Tijerina <ctijerina AT TRUSTNET.COM DOT MX>
To: NETWORKER AT LISTMAIL.TEMPLE DOT EDU
Date: Wed, 2 Jul 2003 13:40:51 -0500
Adam

If you have many small files running on Sun boxes you can use Networker
Snapimage to do the block level backup

Bye
Carlos Tijerina

-----Original Message-----
From: Legato NetWorker discussion
[mailto:NETWORKER AT LISTMAIL.TEMPLE DOT EDU]On Behalf Of George Sinclair
Sent: Miércoles, 02 de Julio de 2003 01:33 p.m.
To: NETWORKER AT LISTMAIL.TEMPLE DOT EDU
Subject: Re: [Networker] slow or failed backup due to too many files


If there are a reasonable number of top-level parent directories then
you could simply attack all of them simultaneously by listing them as
separate savesets. For example, instead of specifying 'All', for the
given client, how about something like:

/filesystem/path1
/filesystem/path2
/filesystem/path3
/filesystem/path4

etc, etc. You could simply list the largest directories, and then
capture the rest by adding:

/filesystem

as the last entry and then create a /filesystem/.nsr file that skips the
other named paths. This way, if anyone adds anything, you'll capture it.
NetWorker can backup any file system significantly faster when you break
it up in pieces as it multi-plexes the streams, running each at the same
speed as it would have done the whole -- well, no less fast anyway --
and thus it can finish the whole thing wayyy faster. Also makes it nice
if there's a failure as you don't have to re-start the whole enchilada
from scratch. You might have to adjust your server and client
parallelism and the number of allowed sessions per device. I generally
do not set the device sessions above 5 myself.

George

Adam Jackson wrote:
>
> I have a lan running 100m full backing up a number of Sun boxes, all of
the boxes backup very quickly expect one (<100k/s at times).
> The server I am having problems with has a very large file system (in
terms of number of files).  It contains an old database that uses a flat
file
> system.  The database consists of about 700,000 plus very small files
layed out on many individual directories, which I believe is the result of
the
> slow backups which ultimately end up failing.  I have transferred this
filesystem to other machines and still have the same problems, so I am
> pretty sure it is not the machine, but the large number of small files on
it.
>
> Is there any way to tweek networker to handle backing up a large number of
small files ??
>
> Thanks in advance for you help
>
> Adam
> adam.jackson AT bell DOT ca
>
> --
> Note: To sign off this list, send a "signoff networker" command via email
> to listserv AT listmail.temple DOT edu or visit the list's Web site at
> http://listmail.temple.edu/archives/networker.html where you can
> also view and post messages to the list.
> =*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=

--
Note: To sign off this list, send a "signoff networker" command via email
to listserv AT listmail.temple DOT edu or visit the list's Web site at
http://listmail.temple.edu/archives/networker.html where you can
also view and post messages to the list.
=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=

---
Incoming mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.476 / Virus Database: 273 - Release Date: 24/04/2003

---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.476 / Virus Database: 273 - Release Date: 24/04/2003

--
Note: To sign off this list, send a "signoff networker" command via email
to listserv AT listmail.temple DOT edu or visit the list's Web site at
http://listmail.temple.edu/archives/networker.html where you can
also view and post messages to the list.
=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=