Amanda-Users

Re: Timeout waiting for ack after adding dle

2009-02-28 22:07:43
Subject: Re: Timeout waiting for ack after adding dle
From: John Hein <jhein AT timing DOT com>
To: Toomas Aas <toomas.aas AT raad.tartu DOT ee>
Date: Sat, 28 Feb 2009 19:59:54 -0700
Toomas Aas wrote at 11:04 +0200 on Feb 28, 2009:
 > I have a single-machine (client==server) setup which has been working  
 > well for quite a long time. It's running Amanda 2.5.1p3 on FreeBSD 6.4.
 > 
 > Yesterday I added a new disk to the machine, mounted it under /db and  
 > added corresponding entry to the disklist. On tonights backup run,  
 > Amanda backed up  first two small DLEs but all the rest (including the  
 > newly added one) failed with:
 > 
 > host.domain.ee  /usr lev 1  FAILED [cannot read header: got 0 instead  
 > of 32768]
 > host.domain.ee  /usr lev 1  FAILED [cannot read header: got 0 instead  
 > of 32768]
 > host.domain.ee  /usr lev 1  FAILED [too many dumper retry: "[request  
 > failed: timeout waiting for ACK]"]
 > 
 > This shouldn't be a firewall problem, since the firewall on the  
 > machine is set to unconditionally pass all traffic on loopback  
 > interface and I couldn't find any relevant dropped packets in the  
 > firewall log. Also amcheck -c passes with no errors.
 > 
 > I looked at the amdump.1 file, and the first indication of any problem  
 > is on the 3rd DLE (which is the newly added one - coincidence?):
 > 
 > driver: result time 2761.656 from chunker0: FAILED 00-00005 "[cannot  
 > read header: got 0 instead of 32768]"
 > 
 > (2761 seconds is approximately 04:06 local time)
 > 
 > Couldn't see anything wrong before that. In the server's general error  
 > log there are just these messages tonight:
 > 
 > Feb 28 04:14:12 host sendbackup[11511]: index tee cannot write [Broken pipe]
 > Feb 28 04:15:02 host sendbackup[11632]: index tee cannot write [Broken pipe]

sendbackup is dying early - possible your timeouts are set too low
in amanda.conf.

Is this new DLE big?  Lots of files?

It's also possible you're hitting a udp datagram size limit.  This can
be improved with a sysctl tweak, or a source patch or using tcp
(sorry - don't recall if amanda 2.5.1 supports the latter).

The client debug files might tell more.  You didn't say you looked
at those.

<Prev in Thread] Current Thread [Next in Thread>