Networker

Re: [Networker] RHEL6/XFS/NW 7.6.2

2011-11-01 12:43:28
Subject: Re: [Networker] RHEL6/XFS/NW 7.6.2
From: Charles Weber <chaweber AT GMAIL DOT COM>
To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
Date: Tue, 1 Nov 2011 12:43:01 -0400
On Nov 1, 2011, at 9:59 AM, Francis Swasey wrote:

> Good day, everyone.
> 
> I am here today because I have this year upgraded my NW server infrastructure 
> (server and
> storage nodes) to x86_64 RHEL6 on four year old IBM 3650's and purchased the 
> extra (Academic
> pricing) license to use XFS on the storage nodes with NexSan fibre arrays 
> (point-to-point
> attached, no fibre switch).  Twice since I have upgraded to 7.6.2 (once at NW 
> 7.6.2.3 the last
> weekend of August, and just this past last weekend of October at NW 7.6.2.5) 
> the one storage
> node that I have converted to using XFS has hung between midnight and 1am on 
> the Saturday
> morning. 
> 
> When it happened in August, I upgraded to the latest RedHat maintenance (new 
> kernel) because
> the dump that the system took pointed into the kernel and likely at the XFS 
> code.  This time,
> there was no dump, the system just hung.   The support person on call took a 
> screen capture to
> show what was on the unresponsive console, forced a reboot, and then dropped 
> and broke his
> laptop before giving me the screen image (OUCH!).  There's nothing in the 
> logs anywhere of
> course...
> 
> Before I ask EMC and RedHat to point fingers at each other, I thought I'd 
> ping you illustrious
> folk to see if any of you know of anything that would indicate I have either 
> royally screwed up
> by choosing XFS or if perhaps you have more experience with XFS and can 
> suggest something in
> the way of tuning to "make it stop" (right now, I'm using mostly default 
> options, with the
> single exception of adding the inode64 option to allow inodes to be placed 
> beyond the first 4TB
> of the 17TB disk).
> 
> Thanks for any pointers!
> 
> -- 
> Frank Swasey                    | http://www.uvm.edu/~fcs
> Sr Systems Administrator        | Always remember: You are UNIQUE,
> University of Vermont           |    just like everyone else.
>  "I am not young enough to know everything." - Oscar Wilde (1854-1900)
> 
> To sign off this list, send email to listserv AT listserv.temple DOT edu and 
> type "signoff networker" in the body of the email. Please write to 
> networker-request AT listserv.temple DOT edu if you have any problems with 
> this list. You can access the archives at 
> http://listserv.temple.edu/archives/networker.html or
> via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER


I've been using xfs on Fedora/CentOS for many years for our primary storage. I 
had something similar occur with a troublesome server/external disk chassis 
some years ago. I replaced various software/hardware parts and never fixed it. 
Seemingly identical server/storage never had the issue. I used SysRQ to reboot 
the thing as Tim suggested. It only went away after I replaced the parts 
entirely, I never really resolved it.  

We have not put Centos 6 in production yet, so currently I am using CO5 with HP 
EVA fiber channel or HP SAS external disk carriers. No issues since I dumped 
the old hardware. Hard to say if it was the disk carriers, the raid cards or 
what. This was first gen Proliant DL585, so a while ago.

I have some EXT4 in service, seems fine within it's limitations.
I generally don't set up FS over 5 TB since I have lots of groups with storage 
and want parallel backup streams whenever possible.
Depending on size/use of the FS I might create it with higher agcount for more 
parallel journalling.

Chuck

To sign off this list, send email to listserv AT listserv.temple DOT edu and 
type "signoff networker" in the body of the email. Please write to 
networker-request AT listserv.temple DOT edu if you have any problems with this 
list. You can access the archives at 
http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER

<Prev in Thread] Current Thread [Next in Thread>