Regarding the poor performance you've seen, I'm curious how many data
streams you were using?  The number of simultaneous streams seems to
have a big impact on total throughput.

Also, if you haven't done so, you should check out the Solaris Internals
website for some recommended tunables with these cards:
http://www.solarisinternals.com/wiki/index.php/Networks

I sent out the settings I'm using in a previous email, so you might be
able to take those and see if they work for you...

-devon
 
------------------------------
Date: Thu, 18 Oct 2007 16:30:40 +0800
From: "Mellor, Adam A." <Adam.Mellor AT woodside.com DOT au>
Subject: Re: [Veritas-bu] Some info on my experiences with 10GbE
To: <VERITAS-BU AT mailman.eng.auburn DOT edu>
Message-ID:
        
<55A93A8FDF17844FBD201175FB138CC604D6F811 AT permls07.wde.woodside.com DOT au>
        
Content-Type: text/plain; charset="us-ascii"

Also, Very Interested,
 
My backup servers are 8core T2000's, we will be putting the SUN nxge
10Gbit cards into them.
 
I have only seen poor results from the card in back to back
configuration (the network is not at 10Gbit yet). so far i have been
misserable with results in line with Mr Preston's.
 
More that happy (Change controll permitting) to duplicate your setup
with the 8 core boxes.
 
Adam.

________________________________

From: veritas-bu-bounces AT mailman.eng.auburn DOT edu
[mailto:veritas-bu-bounces AT mailman.eng.auburn DOT edu] On Behalf Of Curtis
Preston
Sent: Thursday, 18 October 2007 4:07 PM
To: Peters, Devon C; VERITAS-BU AT mailman.eng.auburn DOT edu
Subject: Re: [Veritas-bu] Some info on my experiences with 10GbE



7500 MB/s!  That's the most impressive numbers I've ever seen by FAR.  I
may have to take back my "10 GbE is a Lie!" blog post, and I'd be happy
to do so.

 

Can you share things besides the T2000?  For example, 

 

what OS and patch levels are you running?

Any IP patches?

Any IP-specific patches?

What ndd settings are you using?

Is rss enabled?

 

"Input, I need Input!"

 

---

W. Curtis Preston

Backup Blog @ www.backupcentral.com

VP Data Protection, GlassHouse Technologies

________________________________

From: veritas-bu-bounces AT mailman.eng.auburn DOT edu
[mailto:veritas-bu-bounces AT mailman.eng.auburn DOT edu] On Behalf Of Peters,
Devon C
Sent: Wednesday, October 17, 2007 12:12 PM
To: VERITAS-BU AT mailman.eng.auburn DOT edu
Subject: [Veritas-bu] Some info on my experiences with 10GbE

 

Since I've seen a little bit of talk about 10GbE on here in the past I
figured I'd share some of my experiences... 

I've recently been testing some of Sun's dual-port 10GbE NICs on some
small T2000's (1Ghz, 4-core).  I'm only using a single port on each
card, and the servers are currently directly connected to each other
(waiting for my network team to get switches and fibre in place).

So far, I've been able to drive throughput between these two systems to
about 7500Mbit/sec using iperf.  When the throughput gets this high, all
the cores/threads on the receiving T2000 become saturated and TCP
retransmits start climbing, but both systems remain quite responsive.
Since these are only 4-core T2000's, I would guess that the 6 or 8-core
T2000's (especially with 1.2Ghz or 1.4Ghz processors) should be capable
of more throughput, possibly near line speed.

The down side achieving this high of throughput is that it requires lots
of data streams.  When transmitting with a single data stream, the most
throughput I've gotten is about 1500Mbit/sec.  I only got up to
7500Mbit/s when using 64 data streams...  Also, the biggest gains seem
to be in the jump from 1 to 8 data streams;  with 8 streams I was able
to get throughput up to 6500Mbit/sec.

Our goal for 10GbE, is to be able to restore data from tape at a speed
of at least 2400Mbit/sec (300MB/sec).  We have large daily backups
(3-4TB) that we would like to be able to restore (not backup) in a
reasonable amount of time.  These restores are used to refresh our test
and development environments with current data.  The actual backups are
done with array based snapshots (HDS ShadowCopy), which then get mounted
and backed up by a dedicated media server (6-core T2000).  We're
currently getting about 650MB/sec of throughput with the backups (9
streams on 3 LTO3 tape drives - MPX=3 and it's very compressible data).

Going off my iperf results, the restoring this data using 9 streams
should get us well over 2400Mbit/sec.  But - we haven't installed the
cards on our media servers yet, so I have yet to see what the actual
performanee of netbackup and LTO3 over 10GbE is.  I'm hopeful it'll be
close to the iperf results, but if it doesn't meet the goal then we'll
be looking at other options.

-- 
Devon Peters 


NOTICE: This email and any attachments are confidential. 
They may contain legally privileged information or 
copyright material. You must not read, copy, use or 
disclose them without authorisation. If you are not an 
intended recipient, please contact us at once by return 
email and then delete both messages and all attachments.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://mailman.eng.auburn.edu/pipermail/veritas-bu/attachments/20071018/
73c7b0a6/attachment.html


_______________________________________________
Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu