Hi, Joshua Baker-LePain,

on Mittwoch, 21. Juli 2004 at 02:28 you wrote to amanda-users:

JBL> On Wed, 21 Jul 2004 at 12:00am, Stefan G. Weichinger wrote

>> KV> sendsize[27747]: time 11114.784: Total bytes written: 429923983360 
>> (400GB, 37MB/s)
>> 
>> ok ...
>> 
>> KV> sendsize[27747]: time 18815.342: Total bytes written: 69237104640 (64GB, 
>> 8.6MB/s)
>> 
>> not ok ---

JBL> Actually, both of those are *very* slow.  Remember, those are estimates,
JBL> and they "write" to /dev/null.  When tar does that, it doesn't actually
JBL> read the bits off the spindles, it just stats the files.  On my big
JBL> (Linux) file servers, I get estimate rates on the order or GB/s -- up to
JBL> 80GB/s in some cases.  One difference, though, is that I'm using XFS.

Yes, I didn't think of the fact that this is estimating. My first "ok"
was meant as "Ok, but ..." ;-)

I think that one of the main reasons for the slow estimates here is
the fact that there seem to be loads of small files (think of the
CVS).

So tar has to stat all those many files and slows down ...

This in "cooperation" with filesystem and maybe kernel-related issues.

Converting to Reiser or XFS would help maybe, on the other hand you
would get more cpu-load with Reiser and such.

>> I would:
>> 
>> - split venus:/home into several DLEs (via exclude/include)

JBL> I don't know that this will help.

If there are high-level-subdirs for CVS and others that are not, Kris
could define specific dumptypes to speed things up for the DLEs, using
different exclude-patterns for different types of data.

>> This would spawn several sendsize-processes in parallel ...

JBL> Actually, if you look at sendsize*debug, the estimates occur sequentially,
JBL> not in parallel.  ICBW, but that's how it looks to me.

Right again. I should not post after 14 hours of work ... I just
thought of the output of amstatus and didn't do the lookup you did.

But I will be quiet for another 14 hours now ;-)

best regards,
Stefan