Networker

Re: [Networker] Large quantity of small text files

2004-05-17 11:02:56
Subject: Re: [Networker] Large quantity of small text files
From: Darren Dunham <ddunham AT TAOS DOT COM>
To: NETWORKER AT LISTMAIL.TEMPLE DOT EDU
Date: Mon, 17 May 2004 08:02:42 -0700
> We have many file systems which contain similar small files ( approx
> 20g and couple of million small files on Solaris 2.6 ) and now do raw
> file system backups/recoverys to aid speed of recovery. Luckily there is
> no requirement for restoring individual files. We got our disaster
> recovery times down from 4 days down to 4 hours..
> I'm hoping that the Solaris 9 jfs file systems will allow faster
> restores of this type.

What is "Solaris 9 jfs"?  The "logging" feature in UFS (which has been
there since Solaris 7) could provide some marginal improvement by
coalescing several metadata writes.  However the directory structure
still requires a linear search.  If you're not using "logging", you
could enable fastfs to delay metadata writes.  That can speed up
multi-file restores significantly, but the large directories will still
be slow for lookup times.

I would imagine if anything does, that the DNLC improvements in Solaris
8 and Solaris 9 would provide a larger performance benefit.

Since the linear directory search is also still present in the Veritas
VxFS and the Netapp WAFL filesystems, I don't think this is an easy
problem to solve.  File systems are not databases, and they do not
perform well in that capacity.

--
Darren Dunham                                           ddunham AT taos DOT com
Senior Technical Consultant         TAOS            http://www.taos.com/
Got some Dr Pepper?                           San Francisco, CA bay area
         < This line left intentionally blank to confuse you. >

--
Note: To sign off this list, send a "signoff networker" command via email
to listserv AT listmail.temple DOT edu or visit the list's Web site at
http://listmail.temple.edu/archives/networker.html where you can
also view and post messages to the list.
=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=

<Prev in Thread] Current Thread [Next in Thread>