FYI - I finally tracked this down.  It was caused by cleaning tape that had
no more cleanings in it...

-steve j


On 5/6/08 11:36 PM, "Steve Jacobson" <steve AT cozi DOT com> wrote:

> Running 2.5.0.
> 
> The debug file is in-line below.  It's short.  Looking at it, it seems that it
> had a timeout waiting for the tape array to respond.  It says that it sent a
> mail message, but I haven't received any messages when amcheck has crashed.
> 
> Thanks!
> 
> amcheck: debug 1 pid 1257 ruid 33 euid 0: start at Tue May  6 15:24:46 2008
> amcheck-clients: time 0.012: bind_portrange2: trying port=921
> amcheck-clients: time 0.012: dgram_bind: socket bound to 0.0.0.0.921
> changer: got exit: 0 str: 7 8 1
> changer_query: changer return was 8 1
> changer_query: searchable = 0
> changer_find: looking for NULL changer is searchable = 0
> changer: got exit: 0 str: 7 /dev/nst0
> changer: got exit: 2 str: 8 Drive not ready after 120 seconds, rewind said
> "/dev/nst0: Input/output error"
> amcheck: spawning /usr/bin/Mail in pipeline
> amcheck: argument list: /usr/bin/Mail -s "BackupDailies AMANDA PROBLEM: FIX
> BEFORE RUN, IF POSSIBLE" steve AT cozi DOT com
> amcheck: pid 1257 finish time Tue May  6 15:31:30 2008
> 
> 
> 
> steve jacobson  *  cozi  *  IT Manager  *  steve AT cozi DOT com  *  m: 
> 206.310.7760
>  *  www.cozi.com
> 
> -----Original Message-----
> From: owner-amanda-users AT amanda DOT org [mailto:owner-amanda-users AT 
> amanda DOT org] On
> Behalf Of Jean-Louis Martineau
> Sent: Tuesday, May 06, 2008 6:31 PM
> To: Steve Jacobson
> Cc: amanda-users AT amanda DOT org
> Subject: Re: Amcheck crashing on CentOS 5.1 x86_64
> 
> Which amanda release?
> Post the amcheck.*.debug file
> 
> Jean-Louis
> 
> Steve Jacobson wrote:
>> All
>> 
>> A couple of weeks ago ­ after working for about two months ­ amcheck started
>> crashing on my system.  It dumps the following trace to stderr.
>> 
>> Any thoughts on where to begin looking to track this down would be greatly
>> appreciated.
>> 
>> Thanks!
>> 
>> -Steve J.
>> 
>> 
>> 
>> 
>> 
>> 
>> -------------Begin Trace------------------
>> *** glibc detected *** /usr/sbin/amcheck: double free or corruption
>> (fasttop): 0x000055556feb8e70 ***
>> ======= Backtrace: =========
>> /lib64/libc.so.6[0x2aaaabc6b4f4]
>> /lib64/libc.so.6(cfree+0x8c)[0x2aaaabc6eb1c]
>> /usr/sbin/amcheck[0x555555558ef0]
>> /usr/sbin/amcheck(main+0xa2f)[0x55555555a48f]
>> /lib64/libc.so.6(__libc_start_main+0xf4)[0x2aaaabc198a4]
>> /usr/sbin/amcheck[0x5555555565e9]
>> ======= Memory map: ========
>> 2aaaaaaab000-2aaaaaac5000 r-xp 00000000 08:02 2867273
>> /lib64/ld-2.5.so
>> 2aaaaaac5000-2aaaaaac8000 rw-p 2aaaaaac5000 00:00 0
>> 2aaaaacc4000-2aaaaacc5000 r--p 00019000 08:02 2867273
>> /lib64/ld-2.5.so
>> 2aaaaacc5000-2aaaaacc6000 rw-p 0001a000 08:02 2867273
>> /lib64/ld-2.5.so
>> 2aaaaacc6000-2aaaaace7000 r-xp 00000000 08:02 8753372
>> /usr/lib64/libamserver-2.5.0p2.so
>> 2aaaaace7000-2aaaaaee7000 ---p 00021000 08:02 8753372
>> /usr/lib64/libamserver-2.5.0p2.so
>> 2aaaaaee7000-2aaaaaee9000 rw-p 00021000 08:02 8753372
>> /usr/lib64/libamserver-2.5.0p2.so
>> 2aaaaaee9000-2aaaaaeec000 rw-p 2aaaaaee9000 00:00 0
>> 2aaaaaeec000-2aaaaaef6000 r-xp 00000000 08:02 8753371
>> /usr/lib64/libamtape-2.5.0p2.so
>> 2aaaaaef6000-2aaaab0f6000 ---p 0000a000 08:02 8753371
>> /usr/lib64/libamtape-2.5.0p2.so
>> 2aaaab0f6000-2aaaab0f7000 rw-p 0000a000 08:02 8753371
>> /usr/lib64/libamtape-2.5.0p2.so
>> 2aaaab0f7000-2aaaab119000 r-xp 00000000 08:02 8753373
>> /usr/lib64/libamanda-2.5.0p2.so
>> 2aaaab119000-2aaaab319000 ---p 00022000 08:02 8753373
>> /usr/lib64/libamanda-2.5.0p2.so
>> 2aaaab319000-2aaaab31b000 rw-p 00022000 08:02 8753373
>> /usr/lib64/libamanda-2.5.0p2.so
>> 2aaaab31b000-2aaaab33b000 rw-p 2aaaab31b000 00:00 0
>> 2aaaab347000-2aaaab3c9000 r-xp 00000000 08:02 2867307
>> /lib64/libm-2.5.so
>> 2aaaab3c9000-2aaaab5c8000 ---p 00082000 08:02 2867307
>> /lib64/libm-2.5.so
>> 2aaaab5c8000-2aaaab5c9000 r--p 00081000 08:02 2867307
>> /lib64/libm-2.5.so
>> 2aaaab5c9000-2aaaab5ca000 rw-p 00082000 08:02 2867307
>> /lib64/libm-2.5.so
>> 2aaaab5ca000-2aaaab5cb000 rw-p 2aaaab5ca000 00:00 0
>> 2aaaab5cb000-2aaaab5ce000 r-xp 00000000 08:02 2867332
>> /lib64/libtermcap.so.2.0.8
>> 2aaaab5ce000-2aaaab7cd000 ---p 00003000 08:02 2867332
>> /lib64/libtermcap.so.2.0.8
>> 2aaaab7cd000-2aaaab7ce000 rw-p 00002000 08:02 2867332
>> /lib64/libtermcap.so.2.0.8
>> 2aaaab7ce000-2aaaab7e3000 r-xp 00000000 08:02 2867308
>> /lib64/libnsl-2.5.so
>> 2aaaab7e3000-2aaaab9e2000 ---p 00015000 08:02 2867308
>> /lib64/libnsl-2.5.so
>> 2aaaab9e2000-2aaaab9e3000 r--p 00014000 08:02 2867308
>> /lib64/libnsl-2.5.so
>> 2aaaab9e3000-2aaaab9e4000 rw-p 00015000 08:02 2867308
>> /lib64/libnsl-2.5.so
>> 2aaaab9e4000-2aaaab9e6000 rw-p 2aaaab9e4000 00:00 0
>> 2aaaab9e6000-2aaaab9f7000 r-xp 00000000 08:02 2867325
>> /lib64/libresolv-2.5.so
>> 2aaaab9f7000-2aaaabbf7000 ---p 00011000 08:02 2867325
>> /lib64/libresolv-2.5.so
>> 2aaaabbf7000-2aaaabbf8000 r--p 00011000 08:02 2867325
>> /lib64/libresolv-2.5.so
>> 2aaaabbf8000-2aaaabbf9000 rw-p 00012000 08:02 2867325
>> /lib64/libresolv-2.5.so
>> 2aaaabbf9000-2aaaabbfc000 rw-p 2aaaabbf9000 00:00 0
>> 2aaaabbfc000-2aaaabd42000 r-xp 00000000 08:02 2867283
>> /lib64/libc-2.5.so
>> 2aaaabd42000-2aaaabf42000 ---p 00146000 08:02 2867283
>> /lib64/libc-2.5.so
>> 2aaaabf42000-2aaaabf46000 r--p 00146000 08:02 2867283
>> /lib64/libc-2.5.so
>> 2aaaabf46000-2aaaabf47000 rw-p 0014a000 08:02 2867283
>> /lib64/libc-2.5.so
>> 2aaaabf47000-2aaaabf4e000 rw-p 2aaaabf47000 00:00 0
>> 2aaaabf4e000-2aaaabf58000 r-xp 00000000 08:02 2867312
>> /lib64/libnss_files-2.5.so
>> 2aaaabf58000-2aaaac157000 ---p 0000a000 08:02 2867312
>> /lib64/libnss_files-2.5.so
>> 2aaaac157000-2aaaac158000 r--p 00009000 08:02 2867312
>> /lib64/libnss_files-2.5.so
>> 2aaaac158000-2aaaac159000 rw-p 0000a000 08:02 2867312
>> /lib64/libnss_files-2.5.so
>> 2aaaac159000-2aaaac166000 r-xp 00000000 08:02 2867300
>> /lib64/libgcc_s-4.1.2-20070626.so.1
>> 2aaaac166000-2aaaac366000 ---p 0000d000 08:02 2867300
>> /lib64/libgcc_s-4.1.2-20070626.so.1
>> 2aaaac366000-2aaaac367000 rw-p 0000d000 08:02 2867300
>> /lib64/libgcc_s-4.1.2-20070626.so.1
>> 2aaab0000000-2aaab0021000 rw-p 2aaab0000000 00:00 0
>> 2aaab0021000-2aaab4000000 ---p 2aaab0021000 00:00 0
>> 555555554000-55555555d000 r-xp 00000000 08:02 8851674
>> /usr/sbin/amcheck
>> 55555575c000-55555575d000 rw-p 00008000 08:02 8851674
>> /usr/sbin/amcheck
>> 55556feb4000-55556fed5000 rw-p 55556feb4000 00:00 0
>> 7fffa9c93000-7fffa9ca8000 rw-p 7fffa9c93000 00:00 0
>> [stack]
>> ffffffffff600000-ffffffffffe00000 ---p 00000000 00:00 0
>> [vdso]
>> 
>> ----------------End Trace--------------------
>> 
>> 
>> 
>> 
>>   
> 

-- 
   
 
Steve Jacobson  ? Cozi  ? IT Manager  ? m: 206.310.7760  ? www.cozi.com