Networker

[Networker] Large filesystem backup takes ages.

2002-08-27 01:35:05
Subject: [Networker] Large filesystem backup takes ages.
From: Stuart Lamble <Stuart.Lamble AT ITS.MONASH.EDU DOT AU>
To: NETWORKER AT LISTMAIL.TEMPLE DOT EDU
Date: Tue, 27 Aug 2002 15:37:06 +1000
Hey ho.

We have a Sun Ultra 10 hooked up to a fibre channel hardware RAID
controller, with a 522 GB RAID 5 partition. This partition is
currently holding 39.4 GB of data, in 2,410,991 files. (so about
17 kB per file on average.) UFS filesystem, with filesystem logging
turned on (we really, really, _really_ don't want to be fscking
over 500 GB of data when the filesystem is full and the system goes
down!)

For some reason, Networker is taking an exorbitantly long amount of
time to backup this filesystem. I've seen it sitting idle for
extended periods of time (on a full backup), and when it finally
does start doing things, it'll be at tens or hundreds of kilobytes
per second. This on a system which can quite happily transfer a
100 MB file over ssh in 30 seconds or so (so no, it isn't a duplex
issue. The network connection is a 100 Mbps full duplex Ethernet.)

Tarring up a 100 MB directory tree is also reasonably fast --
certainly faster than Networker seems to be in backing up that amount
of data. I have seen Networker manage peak transfer rates of around
3 MB/s, but that is rarely sustained for more than a few minutes, if
that.

There are things that I can do: I could, for example, reconfigure the
RAID set to have a 16 kB "chunk size" (instead of its current 128 kB),
but I'm not certain if that will have the desired effect, especially
given that tar seems to be reasonably fast. This backup is taking ages
as it is; once the filesystem grows in usage, I can only see things
getting worse.

Networker 6.1.2 on both the server and the client. The server's on an
ATM hookup, and other clients don't exhibit this problem in any event,
so it doesn't seem to be a network issue. Any hints, pointers, or
suggestions would be very gratefully received.

Thanks,

Stuart.

PS: I would have searched the archives, but they seem to be down at
the moment. Am I simply looking at the wrong place? --
http://listserv.temple.edu/archives/networker.html, but that goes to
May 2002 (from March 2000) and the files appear to be not there
("error 2" -- ENOENT -- is a pretty clear indication that the file
the cgi script is trying to open isn't there...) Cheers.

--
Note: To sign off this list, send a "signoff" command via email
to listserv AT listmail.temple DOT edu or visit the list's Web site at
http://listmail.temple.edu/archives/networker.html where you can
also view and post messages to the list.
=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=

<Prev in Thread] Current Thread [Next in Thread>