Yep, I'm using jumbo frames.  The performance was around 50% lower
without it.  I'm not currently using any switches for 10GbE, the servers
are connected directly together.

Re 4Gb vs 2Gb tape drives - since the data is compressed at the drive,
we still need to be able to transfer the data to the drives as fast as
possible.  The highest throughput we've been able to get with a single
2Gb fibre HBA is about 190MB/s (using multiple 2Gb disk-subsystem ports
zoned to a single HBA port).  The highest throughput we've gotten with a
single 2Gb tape drive is 170MB/s.  Since this is near the peak we can
get with 2Gb, I assume that the 2Gb interface on the tape drive is
what's limiting our throughput.

Also, we get about 4x compression of this data on the tapes (~1600MB on
an LTO3 tape).  So, with 265MB/s at 4x compression, the physical write
speed of the drive is probably somewhere around 65MB/s (265/4).  Since
the tape compression ratio has remained the same with both 2Gb and 4Gb
drives, I'd guess that the physical drive speeds with the 2Gb drives
were probably closer to 40MB/s (170/4)...

-devon


-----Original Message-----
From: Nick Majeran [mailto:nmajeran AT gmail DOT com] 
Sent: Thursday, October 18, 2007 2:18 PM
To: veritas-bu AT mailman.eng.auburn DOT edu; Peters, Devon C
Subject: Re: [Veritas-bu] Some info on my experiences with 10GbE

Devon, just a few more questions:

So you *are* using jumbo frames?  I saw that it was enabled in ndd,
but you haven't mentioned it outright.

Also, what network switching equipment are you using for these tests?

Also, I'm curious, how is it that 4Gb/s LTO-3 drives can write
"faster" than 2 Gb/s with contrived data?  It seems like it shouldn't
make a difference, since the data stream is compressed at the drive.

thanks!

-- nick



We've been pretty happy with the T2000's.

The tape library is an IBM 3584, the tape drives are IBM's 4Gb FC LTO-3
drives, there's a dedicated 4Gb HBA for each drive, and everything is
connected to 4Gb McData switches.

We used to have IBM's 2Gb FC LTO-3 drives, and with those the peak
performance was around 165MB/s per drive.  These 4Gb drives peak at
around 265MB/s per drive, though with all 3 tape drives active, we see
throughput closer to 220MB/s per drive...I'm guessing we're bottlenecked
by the ports on our disk subsystem at the moment, but since performance
is more than acceptable we're not looking to tune this any further - at
least not until our LTO-4 drives are installed next month ;).

-devon

_______________________________________________
Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu