On Mon, 2 Apr 2007, Jean-Louis Martineau wrote:
Charles,
amdump can't write to your /var/db/amanda sub-directories?
That's the problem you must fix.
It should be able to, the perms look correct and the old install had no
issues with it (2.4.3).
One other thing to note is that before I upgraded the server to 2.5.1p3, I
had a handful of clients upgraded to 2.5.1p3 running with the 2.4.3 server
with no problems...
Did amcheck succeed?
Yes. Reports no problems found:
[devel2]/home/spork/tmp/amanda # su -m operator -c "amcheck DailySet1"
Amanda Tape Server Host Check
-----------------------------
Holding disk /spool2/amanda: 34715440 kB disk space available, using
29472560 kB
WARNING: skipping tape test because amdump or amflush seem to be running
WARNING: if they are not, you must run amcleanup
NOTE: info file /var/db/amanda/db/b02.foo.com/_usr_local/info does not
exist
NOTE: it will be created on the next run.
NOTE: info file /var/db/amanda/db/b02.foo.com/_var_qmail/info does not
exist
NOTE: it will be created on the next run.
NOTE: info dir /var/db/amanda/db/b02.foo.com/_var_db_pkg does not
exist
NOTE: it will be created on the next run.
NOTE: index dir /var/db/amanda/index/b02.foo.com/_var_db_pkg does not
exist
NOTE: it will be created on the next run.
NOTE: info dir /var/db/amanda/db/b02.foo.com/_ does not exist
NOTE: it will be created on the next run.
Server check took 0.157 seconds
Amanda Backup Client Hosts Check
--------------------------------
Client check: 21 hosts checked in 6.757 seconds, 0 problems found
(brought to you by Amanda 2.5.1p3)
How do you run amdump? which user run it?
Normally it's run out of cron as user "operator":
[devel2]/home/spork/tmp/amanda # grep amdump /etc/crontab
10 3 */2 * * operator /usr/local/sbin/amdump DailySet1
Could you run it on the command line? Do you have the same problem?
Yep, same issue. One thing I noticed that's different is that with the
old server we had issues with big estimates getting lost due to some issue
with UDP/MTU issues between the two sites. I could get the missed dumps
by simply running "amdump DailySet1 host /partition". The email report
I'd get back would only show the DLEs/hosts I specified. If I do a single
host manually with 2.5.1p3, the report contains errors (results missing)
on the hosts/DLEs that I did not specify.
/var/db/amanda is a local filesystem or nfs mounted?
Local.
ls -ald /var
drwxr-xr-x 24 root wheel 512 Dec 5 2002 /var
ls -ald /var/db
drwxr-xr-x 11 root wheel 512 Mar 16 19:49 /var/db
ls -ald /var/db/amanda
drwxr-xr-x 4 operator wheel 512 Feb 14 00:16 /var/db/amanda
du /var/db/amanda
That's a big one... Before pasting it in, I'll just note that the next
thing I'm going to do is to just test as extensively as possible on the
server host (ie: backing itself up) and then get all the logs of that
failed run together.
Now the du:
[devel2]/var/db/amanda # du
22 ./index/d02.foo.com/_home
28 ./index/d02.foo.com/_var_db_pkg
51 ./index/d02.foo.com/_
249 ./index/d02.foo.com/_usr_local
21 ./index/d02.foo.com/_spool
372 ./index/d02.foo.com
23 ./index/salsa1.foo.com/_var_qmail_alias
22 ./index/salsa1.foo.com/_var_qmail_control
22 ./index/salsa1.foo.com/_var_qmail_bin
31 ./index/salsa1.foo.com/_var_db_pkg
51 ./index/salsa1.foo.com/_
22 ./index/salsa1.foo.com/_spool
859 ./index/salsa1.foo.com/_usr_local
565 ./index/salsa1.foo.com/_home
1596 ./index/salsa1.foo.com
22 ./index/h02.foo.com/_var_qmail_control
23 ./index/h02.foo.com/_var_qmail_alias
22 ./index/h02.foo.com/_spool
23 ./index/h02.foo.com/_var_qmail_bin
38 ./index/h02.foo.com/_var_db
58 ./index/h02.foo.com/_
86 ./index/h02.foo.com/_home
617 ./index/h02.foo.com/_usr_local
890 ./index/h02.foo.com
23 ./index/h03.foo.com/_var_qmail_alias
23 ./index/h03.foo.com/_var_qmail_control
23 ./index/h03.foo.com/_var_qmail_bin
38 ./index/h03.foo.com/_var_db_pkg
62 ./index/h03.foo.com/_
2831 ./index/h03.foo.com/_home
1617 ./index/h03.foo.com/_usr_local
21 ./index/h03.foo.com/_spool
4639 ./index/h03.foo.com
22 ./index/h01.foo.com/_var_qmail_alias
22 ./index/h01.foo.com/_var_qmail_control
23 ./index/h01.foo.com/_var_qmail_bin
38 ./index/h01.foo.com/_var_db
51 ./index/h01.foo.com/_
7337 ./index/h01.foo.com/_var_backups
1331 ./index/h01.foo.com/_usr_local
5500 ./index/h01.foo.com/_home
14325 ./index/h01.foo.com
22 ./index/d01.foo.com/_var_qmail_control
23 ./index/d01.foo.com/_var_qmail_alias
22 ./index/d01.foo.com/_spool
23 ./index/d01.foo.com/_var_qmail_bin
29 ./index/d01.foo.com/_var_db_pkg
50 ./index/d01.foo.com/_
172 ./index/d01.foo.com/_usr_local
45 ./index/d01.foo.com/_home
387 ./index/d01.foo.com
20 ./index/toolbox.foo.com/_var_qmail_control
20 ./index/toolbox.foo.com/_var_qmail_alias
20 ./index/toolbox.foo.com/_var_qmail_bin
19 ./index/toolbox.foo.com/_spool
55 ./index/toolbox.foo.com/_
119 ./index/toolbox.foo.com/_home
1136 ./index/toolbox.foo.com/_usr_local
68 ./index/toolbox.foo.com/_var_db
1458 ./index/toolbox.foo.com
23 ./index/h12.foo.com/_var_qmail_control
23 ./index/h12.foo.com/_var_qmail_bin
29 ./index/h12.foo.com/_var_db_pkg
34 ./index/h12.foo.com/_home
57 ./index/h12.foo.com/_
197 ./index/h12.foo.com/_usr_local
364 ./index/h12.foo.com
4071 ./index/h10.foo.com/_spool
33 ./index/h10.foo.com/_home
41 ./index/h10.foo.com/_
139 ./index/h10.foo.com/_usr_local
404 ./index/h10.foo.com/_var_db
4689 ./index/h10.foo.com
48 ./index/devel2.foo.com/_
1775 ./index/devel2.foo.com/_usr_local
231 ./index/devel2.foo.com/_var_db
248 ./index/devel2.foo.com/_spool
2215 ./index/devel2.foo.com/_home
7697 ./index/devel2.foo.com/_spool2_jails
18 ./index/devel2.foo.com/_spool2_data_CQ_Logs
12233 ./index/devel2.foo.com
23 ./index/foo.com/_var_qmail_alias
22 ./index/foo.com/_var_qmail_control
23 ./index/foo.com/_var_qmail_bin
61 ./index/foo.com/_var_db_pkg
50 ./index/foo.com/_
703 ./index/foo.com/_usr_local
842 ./index/foo.com/_spool2
348 ./index/foo.com/_home2
1030 ./index/foo.com/_home
2986 ./index/foo.com/_spool
6089 ./index/foo.com
21 ./index/h14.foo.com/_var_qmail_control
21 ./index/h14.foo.com/_var_qmail_bin
35 ./index/h14.foo.com/_var_db_pkg
49 ./index/h14.foo.com/_
595 ./index/h14.foo.com/_usr_local
21 ./index/h14.foo.com/_spool
2165 ./index/h14.foo.com/_home
2908 ./index/h14.foo.com
22 ./index/h13.foo.com/_var_qmail_control
22 ./index/h13.foo.com/_var_qmail_bin
38 ./index/h13.foo.com/_var_db_pkg
50 ./index/h13.foo.com/_
22 ./index/h13.foo.com/_spool
668 ./index/h13.foo.com/_usr_local
403 ./index/h13.foo.com/_home
295 ./index/h13.foo.com/_home2_savingse
1521 ./index/h13.foo.com
22 ./index/h07.foo.com/_var_qmail_control
23 ./index/h07.foo.com/_var_qmail_bin
29 ./index/h07.foo.com/_var_db_pkg
35 ./index/h07.foo.com/_home
49 ./index/h07.foo.com/_
314 ./index/h07.foo.com/_usr_local
473 ./index/h07.foo.com
23 ./index/h15.foo.com/_var_qmail_control
23 ./index/h15.foo.com/_var_qmail_bin
29 ./index/h15.foo.com/_var_db_pkg
42 ./index/h15.foo.com/_home
50 ./index/h15.foo.com/_
340 ./index/h15.foo.com/_usr_local
508 ./index/h15.foo.com
23 ./index/h16.foo.com/_var_qmail_control
23 ./index/h16.foo.com/_var_qmail_bin
29 ./index/h16.foo.com/_var_db_pkg
49 ./index/h16.foo.com/_
70 ./index/h16.foo.com/_home
362 ./index/h16.foo.com/_usr_local
557 ./index/h16.foo.com
1 ./index/h18.foo.com
23 ./index/h20.foo.com/_var_qmail_control
23 ./index/h20.foo.com/_var_qmail_bin
29 ./index/h20.foo.com/_var_db_pkg
42 ./index/h20.foo.com/_home
50 ./index/h20.foo.com/_
369 ./index/h20.foo.com/_usr_local
537 ./index/h20.foo.com
45 ./index/h17.foo.com/_home
20 ./index/h17.foo.com/_var_qmail_control
21 ./index/h17.foo.com/_var_qmail_bin
39 ./index/h17.foo.com/_var_db_pkg
62 ./index/h17.foo.com/_
602 ./index/h17.foo.com/_usr_local
19 ./index/h17.foo.com/_spool
809 ./index/h17.foo.com
22 ./index/h19.foo.com/_var_qmail_control
22 ./index/h19.foo.com/_var_qmail_bin
27 ./index/h19.foo.com/_var_db_pkg
32 ./index/h19.foo.com/_home
49 ./index/h19.foo.com/_
232 ./index/h19.foo.com/_usr_local
385 ./index/h19.foo.com
32 ./index/newwww.foo.com/_
25 ./index/newwww.foo.com/_var_db_pkg
8 ./index/newwww.foo.com/_var_qmail_control
9 ./index/newwww.foo.com/_var_qmail_alias
1466 ./index/newwww.foo.com/_spool
9 ./index/newwww.foo.com/_var_qmail_bin
9 ./index/newwww.foo.com/_home
278 ./index/newwww.foo.com/_usr_local
1837 ./index/newwww.foo.com
1 ./index/b02.foo.com/_
59 ./index/b02.foo.com/_usr_local
3 ./index/b02.foo.com/_var_qmail
64 ./index/b02.foo.com
56643 ./index
2 ./db/salsa1.foo.com/_var_qmail_alias
2 ./db/salsa1.foo.com/_var_qmail_control
2 ./db/salsa1.foo.com/_var_qmail_bin
2 ./db/salsa1.foo.com/_var_db_pkg
2 ./db/salsa1.foo.com/_
2 ./db/salsa1.foo.com/_spool
2 ./db/salsa1.foo.com/_usr_local
2 ./db/salsa1.foo.com/_home
17 ./db/salsa1.foo.com
2 ./db/h02.foo.com/_var_qmail_control
2 ./db/h02.foo.com/_var_qmail_alias
2 ./db/h02.foo.com/_spool
2 ./db/h02.foo.com/_var_qmail_bin
2 ./db/h02.foo.com/_var_db
2 ./db/h02.foo.com/_
2 ./db/h02.foo.com/_home
2 ./db/h02.foo.com/_usr_local
17 ./db/h02.foo.com
2 ./db/d02.foo.com/_home
2 ./db/d02.foo.com/_var_db_pkg
2 ./db/d02.foo.com/_
2 ./db/d02.foo.com/_usr_local
2 ./db/d02.foo.com/_spool
11 ./db/d02.foo.com
2 ./db/toolbox.foo.com/_var_qmail_control
2 ./db/toolbox.foo.com/_var_qmail_alias
2 ./db/toolbox.foo.com/_var_qmail_bin
2 ./db/toolbox.foo.com/_spool
2 ./db/toolbox.foo.com/_
2 ./db/toolbox.foo.com/_home
2 ./db/toolbox.foo.com/_usr_local
2 ./db/toolbox.foo.com/_var_db
17 ./db/toolbox.foo.com
2 ./db/d01.foo.com/_var_qmail_control
2 ./db/d01.foo.com/_var_qmail_alias
2 ./db/d01.foo.com/_spool
2 ./db/d01.foo.com/_var_qmail_bin
2 ./db/d01.foo.com/_var_db_pkg
2 ./db/d01.foo.com/_
2 ./db/d01.foo.com/_usr_local
2 ./db/d01.foo.com/_home
17 ./db/d01.foo.com
2 ./db/h10.foo.com/_home
2 ./db/h10.foo.com/_
2 ./db/h10.foo.com/_usr_local
2 ./db/h10.foo.com/_var_db
2 ./db/h10.foo.com/_spool
11 ./db/h10.foo.com
2 ./db/h12.foo.com/_var_qmail_control
2 ./db/h12.foo.com/_var_qmail_bin
2 ./db/h12.foo.com/_var_db_pkg
2 ./db/h12.foo.com/_home
2 ./db/h12.foo.com/_
2 ./db/h12.foo.com/_usr_local
13 ./db/h12.foo.com
2 ./db/devel2.foo.com/_
2 ./db/devel2.foo.com/_usr_local
2 ./db/devel2.foo.com/_var_db
2 ./db/devel2.foo.com/_spool
2 ./db/devel2.foo.com/_home
2 ./db/devel2.foo.com/_spool2_jails
2 ./db/devel2.foo.com/_spool2_data_CQ_Logs
15 ./db/devel2.foo.com
2 ./db/h01.foo.com/_var_qmail_alias
2 ./db/h01.foo.com/_var_qmail_control
2 ./db/h01.foo.com/_var_qmail_bin
2 ./db/h01.foo.com/_var_db
2 ./db/h01.foo.com/_
2 ./db/h01.foo.com/_var_backups
2 ./db/h01.foo.com/_usr_local
2 ./db/h01.foo.com/_home
17 ./db/h01.foo.com
2 ./db/h03.foo.com/_var_qmail_alias
2 ./db/h03.foo.com/_var_qmail_control
2 ./db/h03.foo.com/_var_qmail_bin
2 ./db/h03.foo.com/_var_db_pkg
2 ./db/h03.foo.com/_
2 ./db/h03.foo.com/_home
2 ./db/h03.foo.com/_usr_local
2 ./db/h03.foo.com/_spool
17 ./db/h03.foo.com
2 ./db/foo.com/_var_qmail_alias
2 ./db/foo.com/_var_qmail_control
2 ./db/foo.com/_var_qmail_bin
2 ./db/foo.com/_var_db_pkg
2 ./db/foo.com/_
2 ./db/foo.com/_usr_local
2 ./db/foo.com/_spool2
2 ./db/foo.com/_home2
2 ./db/foo.com/_home
2 ./db/foo.com/_spool
21 ./db/foo.com
2 ./db/h14.foo.com/_var_qmail_control
2 ./db/h14.foo.com/_var_qmail_bin
2 ./db/h14.foo.com/_var_db_pkg
2 ./db/h14.foo.com/_
2 ./db/h14.foo.com/_usr_local
2 ./db/h14.foo.com/_spool
2 ./db/h14.foo.com/_home
15 ./db/h14.foo.com
2 ./db/h13.foo.com/_var_qmail_control
2 ./db/h13.foo.com/_var_qmail_bin
2 ./db/h13.foo.com/_var_db_pkg
2 ./db/h13.foo.com/_
2 ./db/h13.foo.com/_spool
2 ./db/h13.foo.com/_usr_local
2 ./db/h13.foo.com/_home
2 ./db/h13.foo.com/_home2_savingse
17 ./db/h13.foo.com
2 ./db/h07.foo.com/_var_qmail_control
2 ./db/h07.foo.com/_var_qmail_bin
2 ./db/h07.foo.com/_var_db_pkg
2 ./db/h07.foo.com/_home
2 ./db/h07.foo.com/_
2 ./db/h07.foo.com/_usr_local
13 ./db/h07.foo.com
2 ./db/h15.foo.com/_var_qmail_control
2 ./db/h15.foo.com/_var_qmail_bin
2 ./db/h15.foo.com/_var_db_pkg
2 ./db/h15.foo.com/_home
2 ./db/h15.foo.com/_
2 ./db/h15.foo.com/_usr_local
13 ./db/h15.foo.com
2 ./db/h16.foo.com/_var_qmail_control
2 ./db/h16.foo.com/_var_qmail_bin
2 ./db/h16.foo.com/_var_db_pkg
2 ./db/h16.foo.com/_
2 ./db/h16.foo.com/_home
2 ./db/h16.foo.com/_usr_local
13 ./db/h16.foo.com
1 ./db/h18.foo.com
2 ./db/h20.foo.com/_var_qmail_control
2 ./db/h20.foo.com/_var_qmail_bin
2 ./db/h20.foo.com/_var_db_pkg
2 ./db/h20.foo.com/_home
2 ./db/h20.foo.com/_
2 ./db/h20.foo.com/_usr_local
13 ./db/h20.foo.com
2 ./db/h17.foo.com/_var_qmail_control
2 ./db/h17.foo.com/_var_qmail_bin
2 ./db/h17.foo.com/_var_db_pkg
2 ./db/h17.foo.com/_
2 ./db/h17.foo.com/_usr_local
2 ./db/h17.foo.com/_spool
2 ./db/h17.foo.com/_home
15 ./db/h17.foo.com
2 ./db/h19.foo.com/_var_qmail_control
2 ./db/h19.foo.com/_var_qmail_bin
2 ./db/h19.foo.com/_var_db_pkg
2 ./db/h19.foo.com/_home
2 ./db/h19.foo.com/_
2 ./db/h19.foo.com/_usr_local
13 ./db/h19.foo.com
2 ./db/newwww.foo.com/_
2 ./db/newwww.foo.com/_var_db_pkg
2 ./db/newwww.foo.com/_var_qmail_control
2 ./db/newwww.foo.com/_var_qmail_alias
2 ./db/newwww.foo.com/_spool
2 ./db/newwww.foo.com/_var_qmail_bin
2 ./db/newwww.foo.com/_home
2 ./db/newwww.foo.com/_usr_local
17 ./db/newwww.foo.com
1 ./db/b02.foo.com/_usr_local
1 ./db/b02.foo.com/_var_qmail
3 ./db/b02.foo.com
307 ./db
56951 .
Thanks so much for your help... I very much appreciate it. I'm having
lots of trouble following all the processes that get spawned, correlating
all the client logs, etc.
Charles
Jean-Louis
Charles Sprickman wrote:
Hello all,
On Sat, 31 Mar 2007, Jean-Louis Martineau wrote:
Could you post the amdump.1 log files and the
curinfo/b02.foo.com/_var_db_pkg/info file.
Last night's run has gotten worse. No report was generated, and amcheck
thinks that a dump is still running (it's not). While before I had two
problem hosts, it seems to be growing. This run had a ton of "MISSING
RESULTS" in my manually-generated amreport. I've attached the last amdump
log - it's gzipped because it's around 1MB uncompressed.
| 