Networker

Re: [Networker] recommendation to backup 1.5 million small files?

2008-01-29 12:38:41
Subject: Re: [Networker] recommendation to backup 1.5 million small files?
From: Tim Mooney <Tim.Mooney AT NDSU DOT EDU>
To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
Date: Tue, 29 Jan 2008 11:35:46 -0600
In regard to: Re: [Networker] recommendation to backup 1.5 million small...:

I'm having the same problem with 2 directories on UNIX servers (SGI and
Solaris), 1 is 3million and the other 15million files.

Wait -- you're not just talking about 15 million files on one filesystem,
you're talking about 15 million files in one directory?  What's the
filesystem type?  How long does

        time ls > /dev/null

in that directory take?

Very few filesystems handle large numbers of files in one directory very
well.  If standard commands take a long time to process the directory,
NetWorker will do no better.  You will have to bypass the filesystem layer
to get any kind of decent performance.

Tim
--
Tim Mooney                                        Tim.Mooney AT ndsu DOT edu
Information Technology Services                   (701) 231-1076 (Voice)
Room 242-J6, IACC Building                        (701) 231-8541 (Fax)
North Dakota State University, Fargo, ND 58105-5164

To sign off this list, send email to listserv AT listserv.temple DOT edu and type 
"signoff networker" in the body of the email. Please write to networker-request 
AT listserv.temple DOT edu if you have any problems with this list. You can access the 
archives at http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER