Amanda-Users

Re: Disaster recovery and amanda-2.6.0b1

2008-01-14 12:29:03
Subject: Re: Disaster recovery and amanda-2.6.0b1
From: Gene Heskett <gene.heskett AT verizon DOT net>
To: amanda-users AT amanda DOT org
Date: Mon, 14 Jan 2008 11:22:34 -0500
On Monday 14 January 2008, Dustin J. Mitchell wrote:
>On Jan 14, 2008 3:53 AM, Gene Heskett <gene.heskett AT verizon DOT net> wrote:
>> PS, from the amverify run I do after the backup run:
>
>amverify is deprecated -- please use amcheckdump.
>
>>  driver: FATAL event_register: Invalid file descriptor 4294967295
>
>That's probably not a maxfilesize problem.  That's the "unsigned"
>32-bit representation of -1, which suggests that someone is passing
>around an invalid file descriptor somewhere -- definitely a bug.  Can
>you sniff out and send the debug logs from that flush?

I just looked at the runtar-debug and it looks like we have a version mixing 
problem, they all say they were generated by 2.5.2p1!  Or did someone forget 
to update an internal version string?

Now, I'm thinking that my script which has some of the executables locations 
hard coded in a config file, might need that config file brought up to date.  
Once I get some caffiene injected, I need to goto work and won't get get back 
to this till this evening.

As far as testing, I can run the make installcheck without screwing things up 
as I can re-run the mkvtapes script and re-init the vtapes in a few seconds.
A 'make check' seemed to be happy.
A 'make installcheck' however bails out from a missing perl module:

/usr/bin/perl -I. -e 'use Test::Harness qw(&runtests); runtests(@ARGV);' 
Amanda_Logfile Amanda_Changer Amanda_Cmdline Amanda_Config Amanda_Types 
amcheckdump amdevcheck amgetconf
Amanda_Logfile...Can't locate Test/More.pm in @INC (@INC 
contains: /usr/lib/perl5/site_perl/5.8.8/i386-linux-thread-multi 
/usr/lib/perl5/site_perl/5.8.7/i386-linux-thread-multi 
/usr/lib/perl5/site_perl/5.8.6/i386-linux-thread-multi 
/usr/lib/perl5/site_perl/5.8.5/i386-linux-thread-multi 
/usr/lib/perl5/site_perl/5.8.8 /usr/lib/perl5/site_perl/5.8.7 
/usr/lib/perl5/site_perl/5.8.6 /usr/lib/perl5/site_perl/5.8.5 
/usr/lib/perl5/site_perl 
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi 
/usr/lib/perl5/vendor_perl/5.8.7/i386-linux-thread-multi 
/usr/lib/perl5/vendor_perl/5.8.6/i386-linux-thread-multi 
/usr/lib/perl5/vendor_perl/5.8.5/i386-linux-thread-multi 
/usr/lib/perl5/vendor_perl/5.8.8 /usr/lib/perl5/vendor_perl/5.8.7 
/usr/lib/perl5/vendor_perl/5.8.6 /usr/lib/perl5/vendor_perl/5.8.5 
/usr/lib/perl5/vendor_perl /usr/lib/perl5/5.8.8/i386-linux-thread-multi 
/usr/lib/perl5/5.8.8 .) 
at Amanda_Logfile line 19.
BEGIN failed--compilation aborted at Amanda_Logfile line 19.
Amanda_Logfile...dubious
        Test returned status 2 (wstat 512, 0x200)
Amanda_Changer...Can't locate Test/More.pm in @INC (@INC 
contains: /usr/lib/perl5/site_perl/5.8.8/i386-linux-thread-multi 
/usr/lib/perl5/site_perl/5.8.7/i386-linux-thread-multi 
/usr/lib/perl5/site_perl/5.8.6/i386-linux-thread-multi 
/usr/lib/perl5/site_perl/5.8.5/i386-linux-thread-multi 
/usr/lib/perl5/site_perl/5.8.8 /usr/lib/perl5/site_perl/5.8.7 
/usr/lib/perl5/site_perl/5.8.6 /usr/lib/perl5/site_perl/5.8.5 
/usr/lib/perl5/site_perl 
/usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi 
/usr/lib/perl5/vendor_perl/5.8.7/i386-linux-thread-multi 
/usr/lib/perl5/vendor_perl/5.8.6/i386-linux-thread-multi 
/usr/lib/perl5/vendor_perl/5.8.5/i386-linux-thread-multi 
/usr/lib/perl5/vendor_perl/5.8.8 /usr/lib/perl5/vendor_perl/5.8.7 
/usr/lib/perl5/vendor_perl/5.8.6 /usr/lib/perl5/vendor_perl/5.8.5 
/usr/lib/perl5/vendor_perl /usr/lib/perl5/5.8.8/i386-linux-thread-multi 
/usr/lib/perl5/5.8.8 .) 
at Amanda_Changer line 19.
BEGIN failed--compilation aborted at Amanda_Changer line 19.
Amanda_Changer...dubious

And repeats that stanza several dozen more times.  I'll see if I can get that 
installed.  Mmm, no, I have it but in /root!
[root@coyote amanda-2.6.0b1-20080111]# locate More.pm
/root/.cpan/build/CPAN-1.80/inc/Test/More.pm
/root/.cpan/build/PathTools-3.14/t/lib/Test/More.pm

Thinking that's old FC6 stuff, I fired up yumex, and I have the 
Perl-Test-Harness installed, but there is not a 'Perl-Test-More' package 
available.  Lemme see if I have the cpan shell..  No, not installed.  Will be 
shortly.  Tis now, but it can't find it, and recommends I do an Install 
Bundle::CPAN cuz the f8 version is precambrian.  Doing that now.  Then I'll 
do a 'reload cpan' followed by 'Install Bundle::TEST' and see what I get.  
I'm getting to hate fedora more and more because they stick us with old 
software for the 'around the edges shit'.  For FC6 the achilles heel was 
gimp-print, 5 year old code that was long ago replaced with gutenprint, so I 
had to build my own there too.  Jerks.

finally, after about ten minutes worth of putzing around taking the default 
answers everytime it pauses for input, I get this prompt:

Writing Makefile for CPAN
---- Unsatisfied dependencies detected during 
[A/AN/ANDK/CPAN-1.9205.tar.gz] -----
    Test::Harness
    Test::More
Shall I follow them and prepend them to the queue
of modules we are processing right now? [yes]

Followed in time by this confusing bit of output:
BEGIN failed--compilation aborted at t/compat/env.t line 11.
t/compat/env...................... Dubious, test returned 2 (wstat 512, 0x200)
 No subtests run
t/compat/failure..................ok

A failure is 'ok'?

Anyway, it keeps finding new dependencies, so I suppose I'll have most of the 
cpan archive installed here, eventually.  It just failed one test because 
bzip2 is not available, but yumex says its installed. ??

Anyway, a 'make installcheck' as root now ends with this:

make[1]: Entering directory 
`/home/amanda/amanda-2.6.0b1-20080111/installcheck'
/usr/bin/perl -I. -e 'use Test::Harness qw(&runtests); runtests(@ARGV);' 
Amanda_Logfile Amanda_Changer Amanda_Cmdline Amanda_Config Amanda_Types 
amcheckdump amdevcheck amgetconf
Amanda_Logfile...ok
Amanda_Changer...ok
Amanda_Cmdline...ok
Amanda_Config....ok
Amanda_Types.....ok
amcheckdump......ok
amdevcheck.......ok
amgetconf........ok
All tests successful.
Files=8, Tests=192,  3 wallclock secs ( 1.82 cusr +  0.61 csys =  2.43 CPU)
rm -rf "/usr/local/etc/amanda/TESTCONF"
make[1]: Leaving directory `/home/amanda/amanda-2.6.0b1-20080111/installcheck'
make[1]: Entering directory `/home/amanda/amanda-2.6.0b1-20080111'
make[1]: Nothing to be done for `installcheck-am'.
make[1]: Leaving directory `/home/amanda/amanda-2.6.0b1-20080111'

As the user amanda, it fails:

make[1]: Entering directory 
`/home/amanda/amanda-2.6.0b1-20080111/installcheck'
/usr/bin/perl -I. -e 'use Test::Harness qw(&runtests); runtests(@ARGV);' 
Amanda_Logfile Amanda_Changer Amanda_Cmdline Amanda_Config Amanda_Types 
amcheckdump amdevcheck amgetconf
Amanda_Logfile...ok 1/0Could not create temporary log file at Amanda_Logfile 
line 39.
# Looks like your test died just after 2.
Amanda_Logfile...dubious
        Test returned status 255 (wstat 65280, 0xff00)
        after all the subtests completed successfully
Amanda_Changer...ok
Amanda_Cmdline...ok
Amanda_Config....ok
Amanda_Types.....ok
amcheckdump......ok
amdevcheck.......ok
amgetconf........ok
Failed Test    Stat Wstat Total Fail  Failed  List of Failed
-------------------------------------------------------------------------------
Amanda_Logfile  255 65280     2    0   0.00%  ??
Failed 1/8 test scripts, 87.50% okay. 0/166 subtests failed, 100.00% okay.
make[1]: *** [installcheck-local] Error 255
make[1]: Leaving directory `/home/amanda/amanda-2.6.0b1-20080111/installcheck'
make: *** [installcheck-recursive] Error 1

I don't think that should happen.  The question is: where is this logfile it 
cannot write?  From Daily/amanda.conf, its /usr/local/var/amanda which was 
owned by amanda:amanda, so I did a chown -R (as root) amanda:disk 
on /usr/local/var/amanda, but the installcheck still fails with the above 
message. 

So, I'm going to make my vtapes again, and see what falls out this time.

>I think there are a number of problems piling onto one another here,
>which is going to make debugging difficult if not impossible.  Can you
>try to reduce the complexity a little bit -- maybe limit to one
>(smallish) DLE, and don't run a flush?

I won't flush this time, backup.sh is running again, s/b fresh start.  I 
didn't trim the disklist though.

>Also, it would be great if you (and others!) could run the
>installchecks on your system.  See
>  http://wiki.zmanda.com/index.php/Testing#Running_Installcheck_without_Root
>please send any failures along in a separate thread.  When we get to
>the bottom of the "Invalid file descriptor" problem, I'll try to
>invent a new check to replicate it.
>
>Dustin

We keep the naproxin sodium folks in Armani suits doing this you know...

-- 
Cheers, Gene
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
The world needs more people like us and fewer like them.