Veritas-bu

Re: [Veritas-bu] Odd hangs on RHEL6 master

2011-04-15 11:34:46
Subject: Re: [Veritas-bu] Odd hangs on RHEL6 master
From: "Lightner, Jeff" <jlightner AT water DOT com>
To: <veritas-bu AT mailman.eng.auburn DOT edu>
Date: Fri, 15 Apr 2011 11:34:41 -0400

Spoke too soon.  

 

While it appeared to run OK through most of the day yesterday last night it started having issues again and was slowing down backups going to our Data Domain dedupe unit.

 

On killing it those backups resumed normal operation and completed.   When I restarted vaulting this morning I could see it was again slowing down backups.

 


From: veritas-bu-bounces AT mailman.eng.auburn DOT edu [mailto:veritas-bu-bounces AT mailman.eng.auburn DOT edu] On Behalf Of Lightner, Jeff
Sent: Thursday, April 14, 2011 3:03 PM
To: veritas-bu AT mailman.eng.auburn DOT edu
Subject: Re: [Veritas-bu] Odd hangs on RHEL6 master

 

On further research my co-worker noted there was a high number of procs_blocked being shown in /proc/stat output while the vaulting was running.   He also later found that if he ran scli in /opt/QLogic_Corporation/QConvergeConsoleCLI it causes a kernel dump every time when under the load for vaulting.  

 

Based on that he checked and  found a newer version of driver for Qlogic fibre HBAS (we were something like 8.3.01x) which was 8.3.04x.   He upgraded the driver to that and it appears that it has resolved the issue.

 


From: veritas-bu-bounces AT mailman.eng.auburn DOT edu [mailto:veritas-bu-bounces AT mailman.eng.auburn DOT edu] On Behalf Of Lightner, Jeff
Sent: Wednesday, April 13, 2011 4:42 PM
To: veritas-bu AT mailman.eng.auburn DOT edu
Subject: [Veritas-bu] Odd hangs on RHEL6 master

 

We are running a RedHat Enterprise Linux (RHEL) 6 master with 7.1.

 

We’ve been doing backups for a few days without much trouble.

 

Today I setup vaulting and kicked it off.   It is mainly doing duplication from Data Domain and Quantum DXi deduplication units to LTO5.  

 

Shortly after starting vaulting and duplication jobs I’m seeing odd things.   For example the “ps –ef” (or any flags to ps) command will hang.   The ps command by itself doesn’t.    Eventually (after many minutes) the command will actually clear up.    Many other commands (e.g. dg –h) work fine.   It also appears the duplications are actually running.

 

As a test I stopped vaulting and deactivated the policy then killed the running duplicates.   After doing that ps command with flags was working fine.

 

I then reactivated and restarted vaulting and a short while later saw the odd issue again.

 

Has anyone experienced something similar to this? 

 

 

Proud partner. Susan G. Komen for the Cure.

 

Please consider our environment before printing this e-mail or attachments.

----------------------------------
CONFIDENTIALITY NOTICE: This e-mail may contain privileged or confidential information and is for the sole use of the intended recipient(s). If you are not the intended recipient, any disclosure, copying, distribution, or use of the contents of this information is prohibited and may be unlawful. If you have received this electronic transmission in error, please reply immediately to the sender that you have received the message in error, and delete it. Thank you.
----------------------------------

_______________________________________________
Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
<Prev in Thread] Current Thread [Next in Thread>