Amanda-Users

Re: Coredumps with 2.5.1p3

2007-03-22 19:37:23
Subject: Re: Coredumps with 2.5.1p3
From: rand AT meridian-enviro DOT com (Douglas K. Rand)
To: Gene Heskett <gene.heskett AT verizon DOT net>
Date: 22 Mar 2007 18:28:30 -0500
Doug> I'm running an Amanda client on a pair of FreeBSD 5.4 and after
Doug> upgrading from 2.4.5p1 to 2.5.1p3 I'm getting core dumps of amandad.

Doug> Anybody have any ideas before I simply downgrade back to 2.4.5?

Gene> When you did the upgrade was an ldconfig done as the last thing,
Gene> or very close to last in order to refresh the library linkages
Gene> correctly? 

Yup. I did the upgrade via portupgrade on FreeBSD, and at the very end
of the session:

  ===>   Compressing manual pages for amanda-client-2.5.1p3_1,1
  ===>   Running ldconfig
  /sbin/ldconfig -m /usr/local/lib
  ===>   Registering installation for amanda-client-2.5.1p3_1,1


Gene> It shouldn't hurt to do it again by hand, as root, just to
Gene> check.

Just did so and re-ran my test. Same result:

  Program terminated with signal 11, Segmentation fault.
  #0  0x0804ac8d in s_ackwait (as=0x805b000, action=A_RECVPKT, pkt=0x280bb000) 
at amandad.c:1068
  1068            security_stream_read(dh->netfd, process_writenetfd, dh);

Doug> These hosts are behind a firewall, but I checked that the
Doug> AMANDA_PORTRANGE=50001,50099 and AMANDA_UDPPORTRANGE=801,899 knobs
Doug> from /etc/make.conf are correctly finding their way into
Doug> config.status: '--with-udpportrange=801,899'
Doug> '--with-portrange=50001,50099'

Gene> And are these ports actually open to these protocols according
Gene> to the firewall rules? 

Yes, because the 2.4.5p1 version worked like a charm, and because my
packet filter config has:

  amanda="67.134.74.26"
  amandaports="50000:50100"
  [...]
  pass in log quick inet proto tcp  from $amanda    to $lan port $amandaports 
flags S/SARF keep state
  pass in log quick inet proto udp  from $amanda    to $lan port amanda         
           keep state

Damn it, the above is a lie. 

I just watched with tcpdump the actual traffic and clearly something
isn't happening right. From the packet filter log:

17:50:48.462364 rule 21/0(match): pass in on fxp0: IP 67.134.74.26.874 > 
207.109.234.250.10080: UDP, length: 125
17:50:48.591917 rule 21/0(match): pass in on fxp0: IP 67.134.74.26.877 > 
207.109.234.250.10080: UDP, length: 260
17:50:48.636197 rule 0/0(match): block in on fxp0: IP 67.134.74.26.1026 > 
207.109.234.250.53055: S 160641676:160641676(0) win 65535 <mss 1460,nop,wscale 
1,nop,nop,timestamp 611117423 0,sackOK,eol>
17:50:51.634698 rule 0/0(match): block in on fxp0: IP 67.134.74.26.1026 > 
207.109.234.250.53055: S 160641676:160641676(0) win 65535 <mss 1460,nop,wscale 
1,nop,nop,timestamp 611120423 0,sackOK,eol>
17:50:54.833986 rule 0/0(match): block in on fxp0: IP 67.134.74.26.1026 > 
207.109.234.250.53055: S 160641676:160641676(0) win 65535 <mss 1460,nop,wscale 
1,nop,nop,timestamp 611123623 0,sackOK,eol>
17:50:58.033361 rule 0/0(match): block in on fxp0: IP 67.134.74.26.1026 > 
207.109.234.250.53055: S 160641676:160641676(0) win 65535 <mss 1460,sackOK,eol>
17:51:01.232929 rule 0/0(match): block in on fxp0: IP 67.134.74.26.1026 > 
207.109.234.250.53055: S 160641676:160641676(0) win 65535 <mss 1460,sackOK,eol>
17:51:04.432183 rule 0/0(match): block in on fxp0: IP 67.134.74.26.1026 > 
207.109.234.250.53055: S 160641676:160641676(0) win 65535 <mss 1460,sackOK,eol>
17:51:10.630986 rule 0/0(match): block in on fxp0: IP 67.134.74.26.1026 > 
207.109.234.250.53055: S 160641676:160641676(0) win 65535 <mss 1460,sackOK,eol>
17:51:22.828770 rule 0/0(match): block in on fxp0: IP 67.134.74.26.1026 > 
207.109.234.250.53055: S 160641676:160641676(0) win 65535 <mss 1460,sackOK,eol>
17:51:47.024174 rule 0/0(match): block in on fxp0: IP 67.134.74.26.1026 > 
207.109.234.250.53055: S 160641676:160641676(0) win 65535 <mss 1460,sackOK,eol>
17:52:03.654909 rule 21/0(match): pass in on fxp0: IP 67.134.74.26.877 > 
207.109.234.250.10080: UDP, length: 257
17:52:03.735340 rule 0/0(match): block in on fxp0: IP 67.134.74.26.1026 > 
207.109.234.250.57850: S 2945510063:2945510063(0) win 65535 <mss 
1460,nop,wscale 1,nop,nop,timestamp 611192537 0,sackOK,eol>
17:52:06.734533 rule 0/0(match): block in on fxp0: IP 67.134.74.26.1026 > 
207.109.234.250.57850: S 2945510063:2945510063(0) win 65535 <mss 
1460,nop,wscale 1,nop,nop,timestamp 611195537 0,sackOK,eol>
17:52:09.933969 rule 0/0(match): block in on fxp0: IP 67.134.74.26.1026 > 
207.109.234.250.57850: S 2945510063:2945510063(0) win 65535 <mss 
1460,nop,wscale 1,nop,nop,timestamp 611198737 0,sackOK,eol>
17:52:13.133348 rule 0/0(match): block in on fxp0: IP 67.134.74.26.1026 > 
207.109.234.250.57850: S 2945510063:2945510063(0) win 65535 <mss 
1460,sackOK,eol>
17:52:16.332645 rule 0/0(match): block in on fxp0: IP 67.134.74.26.1026 > 
207.109.234.250.57850: S 2945510063:2945510063(0) win 65535 <mss 
1460,sackOK,eol>
17:52:19.532136 rule 0/0(match): block in on fxp0: IP 67.134.74.26.1026 > 
207.109.234.250.57850: S 2945510063:2945510063(0) win 65535 <mss 
1460,sackOK,eol>
17:52:25.730933 rule 0/0(match): block in on fxp0: IP 67.134.74.26.1026 > 
207.109.234.250.57850: S 2945510063:2945510063(0) win 65535 <mss 
1460,sackOK,eol>
17:52:37.928605 rule 0/0(match): block in on fxp0: IP 67.134.74.26.1026 > 
207.109.234.250.57850: S 2945510063:2945510063(0) win 65535 <mss 
1460,sackOK,eol>
17:53:02.124096 rule 0/0(match): block in on fxp0: IP 67.134.74.26.1026 > 
207.109.234.250.57850: S 2945510063:2945510063(0) win 65535 <mss 
1460,sackOK,eol>

So clearly the problem is that my new client isn't using the right
ports, which is further shown by this packet from the client to the
server:

  17:57:30.601244 IP 207.109.234.250.10080 > 67.134.74.26.891: UDP, length: 131
        0x0000:  4500 009f d4a8 0000 4011 5d9d cf6d eafa  E.......@.]..m..
        0x0010:  4386 4a1a 2760 037b 008b f2a2 416d 616e  C.J.'`.{....Aman
        0x0020:  6461 2032 2e35 2052 4550 2048 414e 444c  da.2.5.REP.HANDL
        0x0030:  4520 3030 302d 3030 3030 3030 3030 2053  E.000-00000000.S
        0x0040:  4551 2031 3137 3531 3634 3738 320a 434f  EQ.1175164782.CO
        0x0050:  4e4e 4543 5420 4441 5441 2035 3433 3230  NNECT.DATA.54320
        0x0060:  204d 4553 4720 3535 3433 3220 494e 4445  .MESG.55432.INDE
        0x0070:  5820 3537 3333 380a 4f50 5449 4f4e 5320  X.57338.OPTIONS.
        0x0080:  6665 6174 7572 6573 3d66 6666 6666 6566  features=fffffef
        0x0090:  6639 6666 6566 6666 6666 6637 663b 0a    f9ffeffffff7f;.

And then the server is unable to establish a connection to port 54320
on the client. From the config.status file here's the config line,
with some arbitrary white space added for readability:

  configured by ./configure, generated by GNU Autoconf 2.59,
  with options \"
      '--libexecdir=/usr/local/libexec/amanda'
      '--with-amandahosts'
      '--with-fqdn'
      '--with-dump-honor-nodump'
      '--with-buffered-dump'
      '--disable-libtool'
      '--with-user=operator'
      '--with-group=operator'
      '--with-index-server=amanda.meridian-enviro.com'
      '--with-tape-server=amanda.meridian-enviro.com'
      '--with-config=daily'
      '--with-udpportrange=801,899'
      '--with-portrange=50001,50099'
      '--with-gnutar-listdir=/var/amanda/gnutar-lists'
      '--with-gnutar=/usr/local/bin/gtar'
      '--without-server'
      '--prefix=/usr/local'
      '--build=i386-portbld-freebsd5.4'
      'CFLAGS=-g'
      'CXX=c++'
      'build_alias=i386-portbld-freebsd5.4'
      'CC=cc'
      'CXXFLAGS=-g'\"

And inspection of the configure script suggests that the ChangeLog is
incorrect:

  2001-08-14 John R. Jackson (jrj AT purdue DOT edu)
        [...]
        * configure.in: Added several more port range sanity checks.  Added
          --with-tcpportrange as an alias for --with-portrange.

It seems that rev 1.327 removed support for --with-portrange.

I just tweaked the Makefile for the misc/amanda-server port (which is
a slave of the amanda-client port) to use --with-tcpportrange instead
of --with-portrange and it works like a champ.

The interesting part is that I'm actually the guy that submitted the
patch to add the AMANDA_PORTRANGE and AMANDA_UDPPORTRANGE knobs to
/etc/make.conf for the FreeBSD port of amanda.  ;)

I'll submit a PR to fix this problem.



<Prev in Thread] Current Thread [Next in Thread>