On Wed, 21 Jul 2004 at 12:17pm, Kris Vassallo wrote

> On Tue, 2004-07-20 at 17:28, Joshua Baker-LePain wrote:
> > 
> > Actually, both of those are *very* slow.  Remember, those are estimates, 
> > and they "write" to /dev/null.  When tar does that, it doesn't actually 
> > read the bits off the spindles, it just stats the files.  On my big 
> > (Linux) file servers, I get estimate rates on the order or GB/s -- up to 
> > 80GB/s in some cases.  One difference, though, is that I'm using XFS.
> 
> Maybe I am missing something here, but do I need to have the estimates?
> Does something depend on this? If I just ripped this out somehow (no
> idea how I would go about doing this) then waiting 7 hours for an
> estimate would no longer be a problem... right?

You need to have some form of the estimates.  Amanda uses them to decide 
what backups to run each night (whether to demote or promote certain DLEs, 
e.g.).  Some folks on the list have talked of replacing tar for estimates 
with some more efficient method.  But that involves a script that detects 
whether tar is being run to do estimates or the actual backups, and runs 
an appropriate command.  Not trivial -- the discussion should be in the 
archives.

> > Kris, I think that you need to some performance testing/optimizing of your 
> > system.  What controllers are you using? 
> 
> I am using a 3ware 12 port RAID card. Initially when I set the card up I
> did some benchmarking and the read/write speeds to the drives being
> backed up were excellent, what they were doesn't come to mind at this
> moment. I am trying not to take this machine down for any reason as it
> is a HUGE hassle, but it looks like I am going to have to. 

One of my 3ware based systems (single 7500-8, hardware RAID5) is much 
slower than the others doing the estimates.  I *think* it's due to large 
numbers of small files -- at least, it helped a lot when I had the user 
clean up.  But it was never as bad as your case -- it's a 1TB filesystem, 
and estimates were taking about 1.5 hours at worst.  Also, it's using XFS.

> >  Have you tested with bonnie++ 
> > and/or tiobench?  Are there mount parameters to ext3 you can play with 
> > (data=writeback comes to mind)? 
> 
> I tried using   noatime  for the run last night and that didn't seem to
> help worth squat. :( It seems as if i set data=writeback I might as well
> turn off journaling completely because there is no guarantee that old
> data from the journal wont end up getting written back to the disk in
> the event of the system going down. However, would this really speed
> things up? Dramatically? 

I don't know, as I don't use ext3 for this type of stuff.  My suspicion, 
actually, is no -- the problem isn't throughput, just the huge number of 
files.

> >  You may also want to spend some time on 
> > bugzilla and see if there's some other kernel-foo you can apply to RH's 
> > kernel (I assume you're using RH's kernel -- if not vanilla 2.4.20 is 
> > awfully old...) to speed up disk stuff.
> 
> I am using redhat's kernel 2.4.20-20.9smp 

Is it too disruptive to just reboot the system?  It'd be nice to try a 
couple of other kernels, like a vanilla 2.4.26.  Also, I'd ask on a redhat 
list and/or an ext2/3 list about any kernel tweaks to get better 
performance for lots of small files.

-- 
Joshua Baker-LePain
Department of Biomedical Engineering
Duke University