Veritas-bu

[Veritas-bu] "40" errors backing up from Tru64 system

2004-05-19 13:06:05
Subject: [Veritas-bu] "40" errors backing up from Tru64 system
From: ckstehman AT pepco DOT com (ckstehman AT pepco DOT com)
Date: Wed, 19 May 2004 13:06:05 -0400
This is a multipart message in MIME format.
--=_alternative 005DF0EC85256E99_=
Content-Type: text/plain; charset="US-ASCII"

I am at a loss.  I have not been able to backup any data from a couple of 
Tru64 systems.
I have tried everything I can think of. Everything is set to 100FDX No 
autonegotiate.
We are backing up through a cisco switch to a PIX firewall and then to the 
media server.
The firewall is open in both directions.  The firewall logs do not show 
any connection being broken. 

Maybe once in 10 tries we will get a backup started, it runs at a good 
rate for a few
minutes then gradually slows to a stop, then we get a "40" error. 
Sometimes it just
hangs for 10 minutes and then aborts with a 40 error.  Veritas tries two 
times before
giving up.  Usually both tries behave the same.

We have two other Tru64 systems. Alpha 1000A running Tru64 version 40G 
that have no
problems backing up.

The ones that fail are AlphaES40 running Tru64 version 51B with the latest 
patches 
including the one below . 

Any help would be greatly appreciated..


Logs from the latest attempt (after installing the recommended batch - see 
below) follow:

5/19/2004 11:22:33 AM - started process bptm (672)
5/19/2004 11:22:34 AM - started process bpbrm (4140)
5/19/2004 11:22:34 AM - connecting
5/19/2004 11:22:34 AM - connected; connect time: 00:00:00
5/19/2004 11:22:36 AM - started process bptm (3764)
5/19/2004 11:22:36 AM - mounted
5/19/2004 11:22:36 AM - positioning L15279 to file 2
5/19/2004 11:22:38 AM - positioned; position time: 00:00:02
5/19/2004 11:22:38 AM - begin writing
5/19/2004 11:32:04 AM - end writing; write time: 00:09:26
network connection broken(40)
5/19/2004 11:32:05 AM - Error bpsched(pid=1480) suspending further backup 
attempts for client supportbackup, policy BSMEDIA4_EMS_SUPPORT_DATA, 
schedule EMS_Support_Weekly_full because it has exceeded the configured 
number of tries
5/19/2004 11:32:05 AM - Error bpsched(pid=1480) backup of client 
supportbackup exited with status 40 (network connection broken)

5/19/2004 11:13:06 AM - started process bptm (4140)
5/19/2004 11:13:51 AM - mounted; mount time: 00:00:43
5/19/2004 11:13:54 AM - positioning L15279 to file 2
5/19/2004 11:15:16 AM - positioned; position time: 00:01:22
5/19/2004 11:15:16 AM - begin writing
5/19/2004 11:22:25 AM - end writing; write time: 00:07:09
network connection broken(40)
5/19/2004 11:22:30 AM - Error bpsched(pid=1480) backup of client 
supportbackup exited with status 40 (network connection broken)

/19/2004 11:18:53 AM - mounting L15276
5/19/2004 11:19:39 AM - mounted; mount time: 00:00:46
5/19/2004 11:19:45 AM - positioning L15276 to file 1
5/19/2004 11:19:56 AM - positioned; position time: 00:00:11
5/19/2004 11:19:56 AM - begin writing
5/19/2004 11:32:02 AM - end writing; write time: 00:12:06
network connection broken(40)
5/19/2004 11:32:03 AM - Error bpsched(pid=1480) backup of client emsbackup 
exited with status 40 (network connection broken)

5/19/2004 11:32:09 AM - started process bptm (1704)
5/19/2004 11:32:12 AM - started process bpbrm (2616)
5/19/2004 11:32:12 AM - connecting
5/19/2004 11:32:12 AM - connected; connect time: 00:00:00
5/19/2004 11:32:12 AM - started process bptm (4112)
5/19/2004 11:32:15 AM - mounted
5/19/2004 11:32:15 AM - positioning L15279 to file 2
5/19/2004 11:32:16 AM - positioned; position time: 00:00:01
5/19/2004 11:32:16 AM - begin writing
5/19/2004 11:46:50 AM - Error bpbrm(pid=2616) socket read failed, An 
existing connection was forcibly closed by the remote host.  (10054)
5/19/2004 11:49:39 AM - end writing; write time: 00:17:23
file read failed(13)
5/19/2004 11:49:43 AM - Error bpsched(pid=1480) suspending further backup 
attempts for client emsbackup, policy BSMEDIA4_EMS_DATA, schedule 
EMS_Weekly_Full because it has exceeded the configured number of tries
5/19/2004 11:49:43 AM - Error bpsched(pid=1480) backup of client emsbackup 
exited with status 13 (file read failed)

=============================
Carl Stehman
PHI Services Company
202-331-6619
Pager 301-765-2703
ckstehman AT pepco DOT com



George M Menefee/CC/PEP
05/19/2004 11:01 AM

To
Carl K Stehman/EP/PEP
cc
Murali Vippu/EP/PEP@PEP, Stanley Williams/CC/PEP@PEP
Subject
Re: Tru 64 version  and patch





Carl,

I guess you didn't get my e-mail a couple of weeks ago.

I have applied the patch and am ready for you to try a backup.

Thanks,
George
Work:  301-469-5285
Pager: 301-765-1780
Email:  gmmenefee AT pepco DOT com



Carl K Stehman/EP/PEP 
05/19/2004 10:45 AM

To
George M Menefee/CC/PEP@PEP
cc
Murali Vippu/EP/PEP@PEP, Stanley Williams/CC/PEP@PEP
Subject
Tru 64 version  and patch





George,

I am planning to be at the CC tomorrow 5/20.
Have you had a chance to look into this yet? 
What versions of Tru64 are we running on the OMS system 
VS the EMS backup systems. 


> Hello,
> 
> The backup of a TRU64 V5.1 2650 aborts with the message :
> 
> backup of client zoug.vub.ac.be  exited with status 40 (network 
connection
> broken)
> media manager terminated during mount of media id NS0801, possible media
> mount timeout
> media manager terminated by parent process
> 
> The connections are OK  (ping, telnet, bpclntcmd...), I have tried with 
> another pool,
> but without success.
> 
> I have no problems with backups on  TRU64 V5.1 1885.
> The version of NBU is 4.5MP6.
> 
> Thanks for yours suggestions

Did you recently upgrade the OS to 5.1B?  We did and had this "status 40"
issue.  It is due to a change to inet.mod, Netbackup uses inetd to 
communicate
to clients (and media servers).  I've been fighting this for two weeks 
now.

HP has a patch, T64KIT0021340-V51BB24-E-20040120 to fix this.

Carl
=============================
Carl Stehman
PHI Services Company
202-331-6619
Pager 301-765-2703
ckstehman AT pepco DOT com


=============================
Carl Stehman
PHI Services Company
202-331-6619
Pager 301-765-2703
ckstehman AT pepco DOT com
--=_alternative 005DF0EC85256E99_=
Content-Type: text/html; charset="US-ASCII"


<br><font size=2 face="sans-serif">I am at a loss. &nbsp;I have not been
able to backup any data from a couple of Tru64 systems.</font>
<br><font size=2 face="sans-serif">I have tried everything I can think
of. Everything is set to 100FDX No autonegotiate.</font>
<br><font size=2 face="sans-serif">We are backing up through a cisco switch
to a PIX firewall and then to the media server.</font>
<br><font size=2 face="sans-serif">The firewall is open in both directions.
&nbsp;The firewall logs do not show any connection being broken. </font>
<br>
<br><font size=2 face="sans-serif">Maybe once in 10 tries we will get a
backup started, it runs at a good rate for a few</font>
<br><font size=2 face="sans-serif">minutes then gradually slows to a stop,
then we get a &quot;40&quot; error. &nbsp;Sometimes it just</font>
<br><font size=2 face="sans-serif">hangs for 10 minutes and then aborts
with a 40 error. &nbsp;Veritas tries two times before</font>
<br><font size=2 face="sans-serif">giving up. &nbsp;Usually both tries
behave the same.</font>
<br>
<br><font size=2 face="sans-serif">We have two other Tru64 systems. Alpha
1000A running Tru64 version 40G that have no</font>
<br><font size=2 face="sans-serif">problems backing up.</font>
<br>
<br><font size=2 face="sans-serif">The ones that fail are AlphaES40 running
Tru64 version 51B with the latest patches </font>
<br><font size=2 face="sans-serif">including the one below . &nbsp;</font>
<br>
<br><font size=2 face="sans-serif">Any help would be greatly 
appreciated..</font>
<br>
<br>
<br><font size=2 face="sans-serif">Logs from the latest attempt (after
installing the recommended batch - see below) follow:</font>
<br>
<br><font size=2 face="sans-serif">5/19/2004 11:22:33 AM - started process
bptm (672)</font>
<br><font size=2 face="sans-serif">5/19/2004 11:22:34 AM - started process
bpbrm (4140)</font>
<br><font size=2 face="sans-serif">5/19/2004 11:22:34 AM - connecting</font>
<br><font size=2 face="sans-serif">5/19/2004 11:22:34 AM - connected; connect
time: 00:00:00</font>
<br><font size=2 face="sans-serif">5/19/2004 11:22:36 AM - started process
bptm (3764)</font>
<br><font size=2 face="sans-serif">5/19/2004 11:22:36 AM - mounted</font>
<br><font size=2 face="sans-serif">5/19/2004 11:22:36 AM - positioning
L15279 to file 2</font>
<br><font size=2 face="sans-serif">5/19/2004 11:22:38 AM - positioned;
position time: 00:00:02</font>
<br><font size=2 face="sans-serif">5/19/2004 11:22:38 AM - begin writing</font>
<br><font size=2 face="sans-serif">5/19/2004 11:32:04 AM - end writing;
write time: 00:09:26</font>
<br><font size=2 face="sans-serif">network connection broken(40)</font>
<br><font size=2 face="sans-serif">5/19/2004 11:32:05 AM - Error 
bpsched(pid=1480)
suspending further backup attempts for client supportbackup, policy 
BSMEDIA4_EMS_SUPPORT_DATA,
schedule EMS_Support_Weekly_full because it has exceeded the configured
number of tries</font>
<br><font size=2 face="sans-serif">5/19/2004 11:32:05 AM - Error 
bpsched(pid=1480)
backup of client supportbackup exited with status 40 (network connection
broken)</font>
<br>
<br><font size=2 face="sans-serif">5/19/2004 11:13:06 AM - started process
bptm (4140)</font>
<br><font size=2 face="sans-serif">5/19/2004 11:13:51 AM - mounted; mount
time: 00:00:43</font>
<br><font size=2 face="sans-serif">5/19/2004 11:13:54 AM - positioning
L15279 to file 2</font>
<br><font size=2 face="sans-serif">5/19/2004 11:15:16 AM - positioned;
position time: 00:01:22</font>
<br><font size=2 face="sans-serif">5/19/2004 11:15:16 AM - begin writing</font>
<br><font size=2 face="sans-serif">5/19/2004 11:22:25 AM - end writing;
write time: 00:07:09</font>
<br><font size=2 face="sans-serif">network connection broken(40)</font>
<br><font size=2 face="sans-serif">5/19/2004 11:22:30 AM - Error 
bpsched(pid=1480)
backup of client supportbackup exited with status 40 (network connection
broken)</font>
<br>
<br><font size=2 face="sans-serif">/19/2004 11:18:53 AM - mounting L15276</font>
<br><font size=2 face="sans-serif">5/19/2004 11:19:39 AM - mounted; mount
time: 00:00:46</font>
<br><font size=2 face="sans-serif">5/19/2004 11:19:45 AM - positioning
L15276 to file 1</font>
<br><font size=2 face="sans-serif">5/19/2004 11:19:56 AM - positioned;
position time: 00:00:11</font>
<br><font size=2 face="sans-serif">5/19/2004 11:19:56 AM - begin writing</font>
<br><font size=2 face="sans-serif">5/19/2004 11:32:02 AM - end writing;
write time: 00:12:06</font>
<br><font size=2 face="sans-serif">network connection broken(40)</font>
<br><font size=2 face="sans-serif">5/19/2004 11:32:03 AM - Error 
bpsched(pid=1480)
backup of client emsbackup exited with status 40 (network connection 
broken)</font>
<br>
<br><font size=2 face="sans-serif">5/19/2004 11:32:09 AM - started process
bptm (1704)</font>
<br><font size=2 face="sans-serif">5/19/2004 11:32:12 AM - started process
bpbrm (2616)</font>
<br><font size=2 face="sans-serif">5/19/2004 11:32:12 AM - connecting</font>
<br><font size=2 face="sans-serif">5/19/2004 11:32:12 AM - connected; connect
time: 00:00:00</font>
<br><font size=2 face="sans-serif">5/19/2004 11:32:12 AM - started process
bptm (4112)</font>
<br><font size=2 face="sans-serif">5/19/2004 11:32:15 AM - mounted</font>
<br><font size=2 face="sans-serif">5/19/2004 11:32:15 AM - positioning
L15279 to file 2</font>
<br><font size=2 face="sans-serif">5/19/2004 11:32:16 AM - positioned;
position time: 00:00:01</font>
<br><font size=2 face="sans-serif">5/19/2004 11:32:16 AM - begin writing</font>
<br><font size=2 face="sans-serif">5/19/2004 11:46:50 AM - Error bpbrm(pid=2616)
socket read failed, An existing connection was forcibly closed by the remote
host. &nbsp;(10054)</font>
<br><font size=2 face="sans-serif">5/19/2004 11:49:39 AM - end writing;
write time: 00:17:23</font>
<br><font size=2 face="sans-serif">file read failed(13)</font>
<br><font size=2 face="sans-serif">5/19/2004 11:49:43 AM - Error 
bpsched(pid=1480)
suspending further backup attempts for client emsbackup, policy 
BSMEDIA4_EMS_DATA,
schedule EMS_Weekly_Full because it has exceeded the configured number
of tries</font>
<br><font size=2 face="sans-serif">5/19/2004 11:49:43 AM - Error 
bpsched(pid=1480)
backup of client emsbackup exited with status 13 (file read failed)</font>
<br>
<br><font size=2 face="sans-serif">=============================<br>
Carl Stehman<br>
PHI Services Company<br>
202-331-6619<br>
Pager 301-765-2703<br>
ckstehman AT pepco DOT com</font>
<br>
<br>
<br>
<table width=100%>
<tr valign=top>
<td width=40%><font size=1 face="sans-serif"><b>George M 
Menefee/CC/PEP</b></font>
<p><font size=1 face="sans-serif">05/19/2004 11:01 AM</font>
<td width=59%>
<table width=100%>
<tr>
<td>
<div align=right><font size=1 face="sans-serif">To</font></div>
<td valign=top><font size=1 face="sans-serif">Carl K Stehman/EP/PEP</font>
<tr>
<td>
<div align=right><font size=1 face="sans-serif">cc</font></div>
<td valign=top><font size=1 face="sans-serif">Murali Vippu/EP/PEP@PEP,
Stanley Williams/CC/PEP@PEP</font>
<tr>
<td>
<div align=right><font size=1 face="sans-serif">Subject</font></div>
<td valign=top><font size=1 face="sans-serif">Re: Tru 64 version &nbsp;and
patch</font><a 
href=Notes:///852567700055F156/38D46BF5E8F08834852564B500129B2C/701D5BD5B75D041F85256E990050B659>Link</a></table>
<br>
<table>
<tr valign=top>
<td>
<td></table>
<br></table>
<br>
<br><font size=2 face="sans-serif">Carl,</font>
<br>
<br><font size=2 face="sans-serif">I guess you didn't get my e-mail a couple
of weeks ago.</font>
<br>
<br><font size=2 face="sans-serif">I have applied the patch and am ready
for you to try a backup.</font>
<br>
<br><font size=2 face="sans-serif">Thanks,<br>
George<br>
Work: &nbsp;301-469-5285<br>
Pager: 301-765-1780<br>
Email: &nbsp;gmmenefee AT pepco DOT com</font>
<br>
<br>
<br>
<table width=100%>
<tr valign=top>
<td width=40%><font size=1 face="sans-serif"><b>Carl K Stehman/EP/PEP</b>
</font>
<p><font size=1 face="sans-serif">05/19/2004 10:45 AM</font>
<td width=59%>
<table width=100%>
<tr>
<td>
<div align=right><font size=1 face="sans-serif">To</font></div>
<td valign=top><font size=1 face="sans-serif">George M Menefee/CC/PEP@PEP</font>
<tr>
<td>
<div align=right><font size=1 face="sans-serif">cc</font></div>
<td valign=top><font size=1 face="sans-serif">Murali Vippu/EP/PEP@PEP,
Stanley Williams/CC/PEP@PEP</font>
<tr>
<td>
<div align=right><font size=1 face="sans-serif">Subject</font></div>
<td valign=top><font size=1 face="sans-serif">Tru 64 version &nbsp;and
patch</font></table>
<br>
<table>
<tr valign=top>
<td>
<td></table>
<br></table>
<br>
<br><font size=2 face="sans-serif">George,</font>
<br>
<br><font size=2 face="sans-serif">I am planning to be at the CC tomorrow
5/20.</font>
<br><font size=2 face="sans-serif">Have you had a chance to look into this
yet? </font>
<br><font size=2 face="Arial">What versions of Tru64 are we running on
the OMS system </font>
<br><font size=2 face="Arial">VS the EMS backup systems. &nbsp;</font>
<br><font size=2><tt><br>
<br>
&gt; Hello,<br>
&gt; <br>
&gt; The backup of a TRU64 V5.1 2650 aborts with the message :<br>
&gt; <br>
&gt; backup of client zoug.vub.ac.be &nbsp;exited with status 40 (network
connection<br>
&gt; broken)<br>
&gt; media manager terminated during mount of media id NS0801, possible
media<br>
&gt; mount timeout<br>
&gt; media manager terminated by parent process<br>
&gt; <br>
&gt; The connections are OK &nbsp;(ping, telnet, bpclntcmd...), I have
tried with <br>
&gt; another pool,<br>
&gt; but without success.<br>
&gt; <br>
&gt; I have no problems with backups on &nbsp;TRU64 V5.1 1885.<br>
&gt; The version of NBU is 4.5MP6.<br>
&gt; <br>
&gt; Thanks for yours suggestions<br>
<br>
Did you recently upgrade the OS to 5.1B? &nbsp;We did and had this &quot;status
40&quot;<br>
issue. &nbsp;It is due to a change to inet.mod, Netbackup uses inetd to
communicate<br>
to clients (and media servers). &nbsp;I've been fighting this for two weeks
now.<br>
<br>
HP has a patch, T64KIT0021340-V51BB24-E-20040120 to fix this.<br>
</tt></font>
<br><font size=2 face="sans-serif">Carl</font>
<br><font size=2 face="sans-serif">=============================<br>
Carl Stehman<br>
PHI Services Company<br>
202-331-6619<br>
Pager 301-765-2703<br>
ckstehman AT pepco DOT com</font>
<br>
<br>
<br><font size=2 face="sans-serif">=============================<br>
Carl Stehman<br>
PHI Services Company<br>
202-331-6619<br>
Pager 301-765-2703<br>
ckstehman AT pepco DOT com</font>
--=_alternative 005DF0EC85256E99_=--