All,
A little help please. I have Sunfire 480 Sun server. This is a 4cpu,8GB
of memory server. I have 3 ce gigabit copper nic cards. I have 2 emulex
HBA connected to two IBM LTOII tapes drives each. I do not seem to be
able to get more than 60MBs a second inbound for all 3 cards. I should
be able to get 3 times as much or something better than what I am
seeing. I should see better. I am not able to stream any better than
80-90mbs total to my drives. With hardware compression I am writing the
amount of data I am getting through the front end. The thread below was
very interesting to me. We have the latest kernel and CE patches on our
Solaris server.
Any and all thoughs welcome on how I can improve performance. Maybe the
SF480 cannot handle 3 CE cards, two HBA, and 4 IBM LTOII drives.
Ed Coty
Open Systems Storage Engineering, LCNA
973-533-2098
-----Original Message-----
From: Legato NetWorker discussion [mailto:NETWORKER AT LISTSERV.TEMPLE DOT EDU]
On Behalf Of Robert Maiello
Sent: Thursday, December 08, 2005 5:55 PM
To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
Subject: Re: [Networker] ANALYSIS: Networker server price/performance
Ted,
Yeah, I was beating my head against the wall. I have 2 backup NICS on
the box (same LAN) . If I sent all my backups to the backupNIC1 by
setting each clients "server network interface" field I maxed out the
gigabit at 100MBytes/sec. The server is feeding 6 LTO2 drives.
Then if I set half the client's 'server network interface" to
backupNIC2, I saw traffic coming in on both NICs, BUT, each NIC was
running at about half the speed..total throughput at the end of the day
was exactly 1Gbps.
Working with SUN, they have me run ttcp with 5 streams to backup NIC1
while at the same time doing another ttcp of 5 streams to backupNIC2.
Sure enough each NIC's throughput was halfed. Bottom line is they
traced this to the stream queue on Solaris 8..it is the bottleneck.
There's no work around with Solaris8. They tested Solaris 9 and got
1400 Mbps with 2
NICs, Solaris10 was 1500+ Mbps.
So with the system upgraded to Solaris 9 and the below /etc/system
parameters from them with 2 NICs I'm getting 145+ MB/sec. I've
positioned the cards the best I can with V880 bus design. I did notice
with Solaris 9 the load is lower..I would reccomend it for your setup.
In /etc/system the stream queues are increased and the ce ring buffers
are
increased:
* Increase stream queues to get rid of nocanputs on ce NICs set
sq_max_size=30
* Added to help with rx_ov_flows on ce
set ce:ce_ring_size=1024
set ce:ce_comp_ring_size=4096
Robert Maiello
Pioneer Data Systems
On Thu, 8 Dec 2005 13:57:35 -0600, Reed, Ted G II [IT]
<Ted.Reed AT SPRINT DOT COM> wrote:
>Robert,
>Between this and your other thread, I have a (new) concern that I
>didn't have 15 minutes ago.....and maybe you have some insight?
>
>I am replacing my antiquated e4500 Storage Node (8x400Mhz/4G RAM/4x TOE
>[tcp offload engine] GigE/4x 1G HBA/1x100Mb admin NIC) with a e490
>(4x1.35Ghz dual core/8G RAM/4x TOE GigE + 1 onboard GigE/2x 2port 2G
>HBA/1xGigE admin)....both running solaris 8 outputting to 6x STK 9940B
>(30/60/90M write speed). I currently max out at 100-120 MB/sec
>aggregate throughput to the drives, but I also have 6 nsrmmd processes
>(1 per drive) that max out at 100% cpu (x6) when I hit that mark. So I
>have always worked under the assumption that my throughput limitations
>have been due to cpu constraints during max usage.
>
>I also assumed that updated servers could reach higher speeds through
>enhanced CPUs with corresponding nsrmmd processing power. However,
>reading your comments on 'streams queue' raises the question....Will we
>see the increased throughput from hardware upgrades alone? Or should
>we also be investigating an OS upgrade? Will Solaris 8 bite me as a
>solution? Unlike you, I've never been able to get any kind of good
>info from our SUN staff or support whenever I have raised the question
>of true TCP I/O performance tuning.....there's are reasons (other than
>the
>obvious) why our e4500 has all TOE cards.
>
>Thanks in advance for any thoughts/feelings/comments that you or any
>other listserv member has.
>--Ted
>
>
>-----Original Message-----
>From: Legato NetWorker discussion
>[mailto:NETWORKER AT LISTSERV.TEMPLE DOT EDU]
>On Behalf Of Robert Maiello
>Sent: Thursday, December 08, 2005 1:37 PM
>To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
>Subject: Re: [Networker] ANALYSIS: Networker server price/performance
>
>
>Jan,
>
>Actually its a little known fact the the controller on a V440 is
>actually a hardware raid controller. Booting off of it is probably not
>support nor do I know anyone that uses it this way..most SUN systems
>use Veritas or Sun Volume Manager (aka Disksuire) to mirror the boot
>disk etc.
>
>See my other reply in this thread. I found the V440 quite capable but
>apparrently as one tries to drive the gigabit NICs faster more CPU is
>needed.
>
>I'd love to know if any of SUN's new dual core CPUs help in this
regard.
>
>Robert Maiello
>Pioneer Data Systems
>
>On Wed, 7 Dec 2005 03:17:42 -0500, Jan Fredrik
>L=?ISO-8859-1?Q?=C3=B8vik?= <jan.lovik AT ROXAR DOT COM> wrote:
>
>>I am in the process of investing in a new Networker server and was
>wondering
>>what kind of disk system you are running on the V440? Since you are
>>able
>to
>>max out 4 procs., are you running sw raid?
>>I have been given a offer on a V240 and V440 as well as an AMD-based
>>X4200 server..
>>
>>To sign off this list, send email to listserv AT listserv.temple DOT edu and
>type "signoff networker" in the
>>body of the email. Please write to
>>networker-request AT listserv.temple DOT edu
>if you have any problems
>>wit this list. You can access the archives at
>http://listserv.temple.edu/archives/networker.html or
>>via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER
>>======================================================================
>>=
>>==
>
>To sign off this list, send email to listserv AT listserv.temple DOT edu and
>type "signoff networker" in the body of the email. Please write to
>networker-request AT listserv.temple DOT edu if you have any problems wit this
>list. You can access the archives at
>http://listserv.temple.edu/archives/networker.html or via RSS at
>http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER
>
>To sign off this list, send email to listserv AT listserv.temple DOT edu and
type "signoff networker" in the
>body of the email. Please write to
>networker-request AT listserv.temple DOT edu
if you have any problems
>wit this list. You can access the archives at
http://listserv.temple.edu/archives/networker.html or
>via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER
>=======================================================================
>==
To sign off this list, send email to listserv AT listserv.temple DOT edu and
type "signoff networker" in the body of the email. Please write to
networker-request AT listserv.temple DOT edu if you have any problems wit this
list. You can access the archives at
http://listserv.temple.edu/archives/networker.html or via RSS at
http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER
To sign off this list, send email to listserv AT listserv.temple DOT edu and
type "signoff networker" in the
body of the email. Please write to networker-request AT listserv.temple DOT edu
if you have any problems
wit this list. You can access the archives at
http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER
|
