Amanda-Users

Re: Expected completion of an "add *" for a DLE with 378k files (2.4.4p3)?

2007-09-13 13:06:04
Subject: Re: Expected completion of an "add *" for a DLE with 378k files (2.4.4p3)?
From: Jean-Louis Martineau <martineau AT zmanda DOT com>
To: Francis Galiegue <fg AT one2team DOT com>
Date: Thu, 13 Sep 2007 13:07:39 -0400
2.4.? and early 2.5.? can become extremely slow because in some case, the index are not sorted correctly.

A 'add .' might be faster than a 'add *', but I'm not sure.

Jean-Louis

Francis Galiegue wrote:
Hello again!

I have a DLE with more than 378k files in it (that's not a joke), I'm trying to restore it using (again) amrecover. The DLE is found without a hitch, however, it's been now 30 minutes that I've issued the "add *" command, and I still don't have the prompt. OUCH.

On a previous DLE with 170k files, this command barely took more than 2 minutes, so there's something mysterious to me...

Out of curiosity, I ran an strace on the running amindexd process on the server side. I see that it reads from the uncompressed index file 4k at a time, but not very fast, as witnessed by this command (Linux RHEL 4.x, the amanda package comes from RH):

----
root@circe ~ # strace -r -p 4033
Process 4033 attached - interrupt to quit
     0.000000 read(3, "04-14/xml/TEST-com.one2team.sear"..., 4096) = 4096
     0.415061 read(3, "T-com.one2team.webkit.bean.lock."..., 4096) = 4096
     1.291776 read(3, "t\n/junit/2007-04-19/html/com/one"..., 4096) = 4096
     1.706153 read(3, "ing/editor/package-frame.html\n/j"..., 4096) = 4096
     1.628976 read(3, "tabase/bean/TestPackageEditor05."..., 4096) = 4096
     1.568277 read(3, "nit/2007-04-19/html/com/one2team"..., 4096) = 4096
----

[you'll have guessed that this share contains junit test results]

Is this expected? No, I haven't had a peek at the source yet, but I find surprising that it takes so long to complete. Is this another reason why I should go 2.5.x, at least on the server side?

Have fun,


<Prev in Thread] Current Thread [Next in Thread>