Veritas-bu

[Veritas-bu] Network Timed Out (status 41). Cannot Backup! Ma jor problem!

2003-08-08 05:06:53
Subject: [Veritas-bu] Network Timed Out (status 41). Cannot Backup! Ma jor problem!
From: TsiamisP AT unisystems DOT gr (Tsiamis, Panagiotis)
Date: Fri, 8 Aug 2003 12:06:53 +0300
Good morning Katerina,
There is two interesting articles concerning OTM and multiple hanged
bpbkar32 processes, from the Veritas support site.
TechNote ID: 241029
http://seer.support.veritas.com/docs/241029.htm

TechNote ID: 248510
http://seer.support.veritas.com/docs/248510.htm

They mention about an "Event ID 3 : Driver or device is incorrectly
configured for." in the systems event log.

The first TechNote has an example of a bpbkar log that is exactly like the
one you get in this morning's test backup!
It identifies the problem in the "interaction between OTM and 'Upperfilter'
driver".
According to Veritas some types are:
1. Redundant or High Availability Fiber drivers.
2. Fibre Host Bus Adapter (HBA) drivers.
3. Mass storage disk array controller drivers.

... which (I can safely guess) some of those are present in your
environment.
In the case mentioned, Veritas suggested upgrading the appropriate driver,
and the problem was gone.

The second TechNote, which is somehow related, it mentions a similar
behaviour, and it concludes like this (the interesting part is in the last
line):

"... In systems in which the error is being seen at times other than reboots
(such as during backups) or is accompanied by hung bpbkar32.exe processes
(as observed by having numerous bpbkar32.exe processes in the Windows Task
Manager while no backups are running), please contact VERITAS Support.
This has been fixed in patch 341_5"

I hope that the above provide some clues or leads, as to what might be
wrong.
Do let us know how it all turns out.

Best Regards,

Panos


|  -----Original Message-----
| From:         Katerina Giannakeli [mailto:kgiannakeli AT probank DOT gr] 
| Sent: Friday, August 08, 2003 10:54 AM
| To:   veritas-bu AT mailman.eng.auburn DOT edu
| Subject:      RE: [Veritas-bu] Network Timed Out (status 41). 
| Cannot Backup! Major problem!
| 
| Dear everyone who replied asap
| 
| thanks for your posts, they really helped me.
| 
| It turns out that logging og bpbkar was not enabled on the 
| client, so I enabled it
| and noticed that  Logging mysteriously stops when OTM is mentioned.
| 
| Here's the log file for the curious ones:
| **************************************************************
| *********************************
| [2312]: 
| 08/08/03 9:42:47 p?: [2312]: INF - Starting log file: 
| C:\Program Files\VERITAS\NetBackup\logs\BPBKAR\080803.LOG
| 
| 08/08/03 9:42:47 p?: [2312]: INF - the log mutex: 120
| BPBKAR  NetBackup Backup/Archive  3.4GA  [Jun 27 2000]
| Copyright 2000 VERITAS Software Corporation
| All Rights Reserved.
| 
| 08/08/03 9:51:29 p?: [3272]: DAT - _pgmptr = 'C:\Program 
| Files\VERITAS\NetBackup\bin\BPBKAR32.exe'
| 08/08/03 9:51:29 p?: [3272]: DAT - lpCmdLine = '-r 1209600 
| -ru root -dt 0 -to 0 -clnt P800CLS001 -class Test -sched Full 
| -st FULL -bpstart_to 300 -bpend_to 300 -read_to 500 
| -stream_count 1 -stream_number 1 -use_otm -fso -Z -b 
| P800CLS001_1060325713 -kl 28 -ct 13 '
| 08/08/03 9:51:29 p?: [3272]: DAT - timezone: GTB Standard 
| Time, offset=-7200, dst: GTB Daylight Time
| 08/08/03 9:51:29 p?: [3272]: DAT - current time: 1060325489, 
| 8/8/2003 9:51:29 p?
| 08/08/03 9:51:29 p?: [3272]: DAT - 01/01/94 UCT:  757382400, 
| 1/1/1994 3:00:00 p?
| 08/08/03 9:51:29 p?: [3272]: DAT - 07/01/94 UCT:  773020800, 
| 1/7/1994 3:00:00 p?
| 08/08/03 9:51:29 p?: [3272]: DAT - standard input handle = 572
| 08/08/03 9:51:29 p?: [3272]: DAT - standard output handle = 108
| 08/08/03 9:51:29 p?: [3272]: DAT - standard error handle = 444
| 08/08/03 9:51:29 p?: [3272]: INF - 
| ======================================================================
| 08/08/03 9:51:29 p?: [3272]: INF - OTM Version Information
| 08/08/03 9:51:29 p?: [3272]: INF - 
| ----------------------------------------------------------------------
| 08/08/03 9:51:29 p?: [3272]: INF -    OSType: 00000024
| 08/08/03 9:51:29 p?: [3272]: INF - OSVersion: 08930005
| 08/08/03 9:51:29 p?: [3272]: INF -   Version: 00920111
| 08/08/03 9:51:29 p?: [3272]: INF - LoVersion: 00000110
| 08/08/03 9:51:29 p?: [3272]: INF - 
| ======================================================================
| 08/08/03 9:51:29 p?: [3272]: INF - OTM Initialize - able to register
| **************************************************************
| ***********************************************
| 
| Here's when logging stops and status 41 appears..
| Under normal circumstances, logging should have proceed as 
| follows (copied from a succesful backup)
| 
| **************************
| 06/08/03 9:49:12 p?: [2368]: INF - NetBackup Temp Directory: 
| 'C:\Program Files\VERITAS\NetBackup\Temp'
| 06/08/03 9:49:12 p?: [2368]: INF - dwJobData: 00000000
| 06/08/03 9:49:12 p?: [2368]: INF -     dwJob: 00000001
| 06/08/03 9:49:12 p?: [2368]: INF - backup privileges enabled, 
| previous = 46071820
| 06/08/03 9:49:12 p?: [2368]: INF - restore privileges 
| enabled, previous = 46071820
| 06/08/03 9:49:12 p?: [2368]: INF - security privileges 
| enabled, previous = 46071820
| 06/08/03 9:49:12 p?: [2368]: INF - tcb privileges enabled, 
| previous = 46071820
| 06/08/03 9:49:12 p?: [2368]: INF - create token privileges 
| enabled, previous = 46071820
| ....
| ....
| ....
| ***************************
| and so on..
| 
| So, I assume it has to do with OTM. What do u think?
| 
| 
| Katerina
| 
| 
| 
| -----Original Message-----
| From: Toelken, William [mailto:ToelkenW AT aetna DOT com]
| Sent: Thursday, August 07, 2003 7:14 PM
| To: Bob Grabbe; veritas-bu AT mailman.eng.auburn DOT edu; 
| kgiannakeli AT probank DOT gr
| Subject: RE: [Veritas-bu] Network Timed Out (status 41). 
| Cannot Backup! Major problem!
| 
| 
| 
| We have the same problem in a Windows 2000 environment. We use pskill
| from sysinternals and that takes care of MOST of the hung processes.
| Even with 4.5F_3 we have seen issues with ltid being unkillable. We
| currently have a sev 1 case open. Rebooting should NOT be the answer.
| Microsoft has indicated that there are NT 4 api calls being made and
| this is contributing to the issue.
| 
| -----Original Message-----
| From: Bob Grabbe [mailto:GRABBEB AT dominos DOT com] 
| Sent: Thursday, August 07, 2003 9:34 AM
| To: veritas-bu AT mailman.eng.auburn DOT edu; kgiannakeli AT probank DOT gr
| Subject: Re: [Veritas-bu] Network Timed Out (status 41). 
| Cannot Backup!
| Major problem!
| 
| 
| Yes, I had the same problem and support was no help to me either. The
| only way to get rid of the extra processes is to reboot the server.
| Fortunately (for me anyway) this problem is not recurring 
| since going to
| 4.5 mp3, but I never found any fix for it anywhere. If anyone else has
| any suggestions, I'd like to see them also. 
| You have my profoundest sympathy, but I'm afraid I can't 
| offer any hope
| other than upgrading your whole system. 
| Bob Grabbe
| Dominos Pizza LLC
| grabbeb AT dominos DOT com
| 
| 
| >>> "Katerina Giannakeli" <kgiannakeli AT probank DOT gr> 8/7/03 9:08:01 AM
| >>>
| Hi all,
| 
| the last few days the backup procedure fails, stating that "network
| timed out' with status 41. FYI, the software is NetBackup 3.41 (still)
| on Windows2000, which is backing up the 3 out of 4 parts of a cluster
| system.
| 
| Two nights ago, without any change made, the 1st cluster part(CLS001)
| didn't backup stating the above error. When I logged in to CLS001, I
| noticed in TaskManager that bpbkar32.exe was loaded many times (I did
| perform various test backups, but they had all ended). As the
| "wonderful" support person (who I managed to contact after calling him
| all day long, and haven't heard from ever since) told me to stop the
| NetBackup service on CLS001, kill all those hanging processes, 
| and
| start the NB Service again, and that should fix the problem.
| 
| Well, it didn't, and I have a major problem, as I can't 
| backup the main
| part of the system! I am now copying various important data to a disk,
| but that is not very reliable, not to mention that I use NetBackup to
| perform various operations and now I am writing scripts to 
| override the
| problem.
| 
| 
| Please, if someone has encountered anything like this , let me know.
| Support isn't much help and we have a big problem over here!
| 
| 
| Thank you all in advance,
| 
| Katerina
| 
| 
| **************************************************************
| **********
| **************************
| The contents of this email and any attachments are confidential. It is
| intended for the named recipient(s) only. If you have received this
| email in error please notify the system manager or  the 
| sender immediately and do not disclose the contents to any one or make
| copies.
| 
| ** eSafe scanned this email for viruses, vandals and malicious content
| **
| **************************************************************
| **********
| **************************
| _______________________________________________
| Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu
| http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
| 
|  
| This e-mail may contain confidential or privileged 
| information.  If you
| think you have received this e-mail in error, please advise 
| the sender by
| reply e-mail and then delete this e-mail immediately.  Thank 
| you.  Aetna
| **************************************************************
| ************************************
| The contents of this email and any attachments are confidential.
| It is intended for the named recipient(s) only.
| If you have received this email in error please notify the 
| system manager or  the 
| sender immediately and do not disclose the contents to any 
| one or make copies.
| 
| ** eSafe scanned this email for viruses, vandals and 
| malicious content **
| **************************************************************
| ************************************
| 

<Prev in Thread] Current Thread [Next in Thread>