7500 MB/s! That’s
the most impressive numbers I’ve ever seen by FAR. I may have to take back
my “10 GbE is a Lie!” blog post, and I’d be happy to do
so.
Can you share things
besides the T2000? For example,
what OS and patch
levels are you running?
Any IP
patches?
Any IP-specific
patches?
What ndd settings are
you using?
Is rss
enabled?
“Input, I need
Input!”
From:
veritas-bu-bounces AT mailman.eng.auburn DOT edu
[mailto:veritas-bu-bounces AT mailman.eng.auburn DOT edu] On Behalf Of Peters, Devon C
Sent: Wednesday, October 17, 2007 12:12
PM
To:
VERITAS-BU AT mailman.eng.auburn DOT edu
Subject: [Veritas-bu] Some info on my
experiences with 10GbE
Since I've seen a little bit of talk
about 10GbE on here in the past I figured I'd share some of my
experiences...
I've recently been testing some of
Sun's dual-port 10GbE NICs on some small T2000's (1Ghz, 4-core). I'm only
using a single port on each card, and the servers are currently directly
connected to each other (waiting for my network team to get switches and fibre
in place).
So
far, I've been able to drive throughput between these two systems to about
7500Mbit/sec using iperf. When the throughput gets this high, all the
cores/threads on the receiving T2000 become saturated and TCP retransmits start
climbing, but both systems remain quite responsive. Since these are only
4-core T2000's, I would guess that the 6 or 8-core T2000's (especially with
1.2Ghz or 1.4Ghz processors) should be capable of more throughput, possibly near
line speed.
The
down side achieving this high of throughput is that it requires lots of data
streams. When transmitting with a single data stream, the most throughput
I've gotten is about 1500Mbit/sec. I only got up to 7500Mbit/s when using
64 data streams… Also, the biggest gains seem to be in the jump from 1 to
8 data streams; with 8 streams I was able to get throughput up to
6500Mbit/sec.
Our
goal for 10GbE, is to be able to restore data from tape at a speed of at least
2400Mbit/sec (300MB/sec). We have large daily backups (3-4TB) that we
would like to be able to restore (not backup) in a reasonable amount of
time. These restores are used to refresh our test and development
environments with current data. The actual backups are done with array
based snapshots (HDS ShadowCopy), which then get mounted and backed up by a
dedicated media server (6-core T2000). We're currently getting about
650MB/sec of throughput with the backups (9 streams on 3 LTO3 tape drives -
MPX=3 and it's very compressible data).
Going off my iperf results, the
restoring this data using 9 streams should get us well over 2400Mbit/sec.
But - we haven't installed the cards on our media servers yet, so I have yet to
see what the actual performanee of netbackup and LTO3 over 10GbE is. I'm
hopeful it'll be close to the iperf results, but if it doesn't meet the goal
then we'll be looking at other options.
--
Devon
Peters