Veritas-bu

Re: [Veritas-bu] Some info on my experiences with 10GbE

2008-01-07 20:46:38
Subject: Re: [Veritas-bu] Some info on my experiences with 10GbE
From: Marion Hakanson <hakansom AT ohsu DOT edu>
To: "Curtis Preston" <cpreston AT glasshouse DOT com>
Date: Mon, 07 Jan 2008 17:19:36 -0800
cpreston AT glasshouse DOT com said:
> This is another pro-T2000 report.  What makes them special? 

I'll chime in with my additional comments along the lines of the other
responder, who mentioned the large number of cores and threads, etc.
Some of the kernel tunables mentioned have to do with "interrupt fanout"
support for the Sun 10GbE NIC -- so the NIC's interrupts can be handled
by a number of CPU cores in parallel, which is pretty important when
each core isn't all that speedy (in the T2000, anyway).  The T2000's
onboard NIC's also take advantage of interrupt fanout, and it's making
its way into Sun's other hardware platforms (both SPARC and x86) as well.

To go along with the driver support, it's recommended that you use the
Solaris "psradm" and "pooladm" commands to arrange for device interrupts
to be limited to only 1 thread per physical core on the T2000.  This way
non-interrupt threads won't compete with the device-based threads.  I'm
not an expert with this, but I can report that following the recommendations
does make a marked difference in our overall T2000 system throughput even
with only a single GbE interface.  Here are a couple references:

http://www.solarisinternals.com/wiki/index.php/Networks#Tuning_Sun_Dual_10GbE_o
n_T1000.2FT2000
http://serversidetechnologies.blogspot.com/

As a clue to my vintage (:-), I'll say that the T2000 reminds me of the
Sequent Balance and Symmetry computer systems.  These had up to 30 little
CPU's in them (NS32032's, or Intel 80386's), as "fast" as 16 or 20MHz.
The systems were not real speedy on a per-Unix-process basis, but they had
a huge amount of bandwidth -- it took quite an effort to bog one down.  My
recollection of those days is a bit rusty, but I remember hearing that much
of Sequent's expertise with parallelizing the Unix kernel ended up getting
folded into Solaris/SVR4 back when AT&T and Sun made their grand SVR4/BSD
reunion deal.

Regards,

Marion


_______________________________________________
Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu