Veritas-bu

[Veritas-bu] Gigabit performance

2005-02-11 04:24:43
Subject: [Veritas-bu] Gigabit performance
From: Gunter.Maier AT Sun DOT COM (Gunter Maier)
Date: Fri, 11 Feb 2005 10:24:43 +0100
This is a multi-part message in MIME format.

--Boundary_(ID_his7W0oQtT6oR+nWGpnOHQ)
Content-type: text/plain; charset=us-ascii
Content-transfer-encoding: 7BIT

Hi,

have you checked out, if your are working with "full-duplex".
Check-it (vi the "show.eri.sh" script to "show.ge.sh" ) out and run my
"ge0_tune" script to set it to full duplex (have a look into the script
before...)

Copy the script and do :# dos2unix ge0_tune.txt ge0_tune
                        # ./ge0_tune


Have you set the following NBU performance parameters on your master
server? See document:
http://support.veritas.com/docs/183702

If not, set it and adapt also /etc/system
p.e.:
# vi /etc/system
set shmsys:shminfo_shmmax = 1073741824
set shmsys:shminfo_shmmni = 100
set shmsys:shminfo_shmseg = 32
set semsys:seminfo_semmap = 30
set semsys:seminfo_semmni = 60
set semsys:seminfo_semmns = 200
set semsys:seminfo_semmnu = 90
set semsys:seminfo_semmsl = 60
set semsys:seminfo_semopm = 30
set semsys:seminfo_semume = 30
set semsys:seminfo_semvmx = 32767
set semsys:seminfo_semaem = 16384

# reboot -- -r

Play around; this will work !!

Cheers,




Johnny Oestergaard wrote:
> I don't know that much about Sun and Unix, but I have a Windows 2003 media 
> server running 5.1 where I have teamed two ethernet cards.
> The server has 3 STK9940B drives.
> The server is a Dell 2850 connected to an Extreme Networks Black Diamond 6808.
> And that server I have seen a little over 150 MB/s over the teamed ethernet 
> interfaces. And I every day see more then 120 MB/s
> 
> My biggest problems is to get the clients to deliver the data that the 
> media server can write.
> 
> I would try to use some different NIC's.
> 
> Have you tried to replace you network cables (I have had problems with 
> cables that where not CAT5e but only CAT5)
> 
> /johnny
> 
> At 16:44 10-02-2005 -0500, Mosseveld, Adriaan wrote:
> 
>>Our Netbackup infrastructure is based around an E450 with gigabit
>>connectivity.
>>
>>For some reason, it is impossible for us to get a throughput on the (ge)
>>Gigabit card of greater than 10Mb/sec. We have tried a direct connection
>>between the server and an E220R with a ce gigaswift card, with exactly the
>>same results (therefore ruling out the switch).
>>
>>We have tried using Netbackup, ftp, dd (with remote shell), but all showed a
>>similar throughput.
>>
>>The autonegotiation on the device has been set both on and off, with no
>>change, the messages file indicates that the link is up at 1000Mbs full
>>duplex and I am totally out of ideas.
>>
>>Can anyone suggest what else we can try.
>>
>>Thanks in advance for any replies.
>>
>>
>>Adriaan Mosseveld
>>Senior Analyst, Technology Services
>>CGI Canada
>>Tel:      416-429-3000 x4140
>>Fax:     416-429-6297
>>Email:  AMosseveld AT foresters DOT biz
>>
>>
>>CONFIDENTIALITY NOTICE - - - - - - - - - - 200345 (Can/US)(04/02)
>>This e-mail and any files transmitted with it are intended solely for the
>>use of the individual or entity to whom they are addressed. This
>>communication represents the originator's personal view and opinions, which
>>may not necessarily reflect those of Foresters.  It may also contain
>>confidential information and be protected by and subject to the
>>attorney-client privilege or be privileged work product or proprietary
>>information.  If you are not the original recipient or the person
>>responsible for delivering the e-mail to the intended recipient, be advised
>>that you have received this e-mail in error, and that any use,
>>dissemination, forwarding, printing or copying of this email is strictly
>>prohibited.  If you received this e-mail in error, please notify
>>postmaster AT foresters DOT biz immediately.
>>_______________________________________________
>>Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu
>>http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
> 
> 
> _______________________________________________
> Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu
> http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu

-- 
_________________________________________________________

 Mit freundlichen Gruessen    //    With Kind Regards

 Sun Microsystems (Schweiz) AG
     Gunter Maier
     Technical Specialist (Backup/Recovery / Storage)
     Sun Client Services
 Javastrasse 2 / Hegnau
 CH-8604 Volketswil
     Phone  +41  1 908 92 69
     Mobile +41 79 692 50 46
     Fax    +41  1 908 90 01
     E-Mail gunter.maier AT sun DOT com
     Web    www.sun.ch
_________________________________________________________

--Boundary_(ID_his7W0oQtT6oR+nWGpnOHQ)
Content-type: text/plain; name=ge0_tune.txt
Content-transfer-encoding: 7BIT
Content-disposition: inline; filename=ge0_tune.txt

#!/sbin/sh
# /etc/rc2.d/S68net-tune -> /etc/init.d/net-tune
# Copyright (c) 2000 by Sun Microsystems, Inc.
# All rights reserved.
#
#
# Edit the following example code for your needs.
# See Product Documentation and SunSolve Infodocs for proper use first.
# Most parameters must be set on both sides to have effect (check switch and IP 
partner settings).

PATH=/usr/bin:/usr/sbin

case "$1" in
     'start')
        echo "Implementing Solaris Network Tuning."

# hme-Interfaces
# hme0
#       ndd -set /dev/hme instance 0
#       ndd -set /dev/hme adv_100T4_cap 0
#       ndd -set /dev/hme adv_100fdx_cap 1
#       ndd -set /dev/hme adv_100hdx_cap 0
#       ndd -set /dev/hme adv_10fdx_cap 0
#       ndd -set /dev/hme adv_10hdx_cap 0
#       ndd -set /dev/hme adv_autoneg_cap 0
# hme1
#       ndd -set /dev/hme instance 1
#       ndd -set /dev/hme adv_100T4_cap 0
#       ndd -set /dev/hme adv_100fdx_cap 1
#       ndd -set /dev/hme adv_100hdx_cap 0
#       ndd -set /dev/hme adv_10fdx_cap 0
#       ndd -set /dev/hme adv_10hdx_cap 0
#       ndd -set /dev/hme adv_autoneg_cap 0
# hme2
#       ndd -set /dev/hme instance 2
#       ndd -set /dev/hme adv_100T4_cap 0
#       ndd -set /dev/hme adv_100fdx_cap 1
#        ndd -set /dev/hme adv_100hdx_cap 0
#        ndd -set /dev/hme adv_10fdx_cap 0
#        ndd -set /dev/hme adv_10hdx_cap 0
#        ndd -set /dev/hme adv_autoneg_cap 0

# eri-Interfaces
# eri0
#        ndd -set /dev/eri instance 0
#        ndd -set /dev/eri adv_100T4_cap 0
#        ndd -set /dev/eri adv_100fdx_cap 1
#        ndd -set /dev/eri adv_100hdx_cap 0
#        ndd -set /dev/eri adv_10fdx_cap 0
#        ndd -set /dev/eri adv_10hdx_cap 0
#        ndd -set /dev/eri adv_autoneg_cap 0

# qfe-Interfaces
# qfe0
        #ndd -set /dev/qfe instance 0
        #ndd -set /dev/qfe adv_100T4_cap 0
        #ndd -set /dev/qfe adv_100fdx_cap 1
        #ndd -set /dev/qfe adv_100hdx_cap 0
        #ndd -set /dev/qfe adv_10fdx_cap 0
        #ndd -set /dev/qfe adv_10hdx_cap 0
        #ndd -set /dev/qfe adv_autoneg_cap 0
# qfe1
        #ndd -set /dev/qfe instance 1
        #ndd -set /dev/qfe adv_100T4_cap 0
        #ndd -set /dev/qfe adv_100fdx_cap 1
        #ndd -set /dev/qfe adv_100hdx_cap 0
        #ndd -set /dev/qfe adv_10fdx_cap 0
        #ndd -set /dev/qfe adv_10hdx_cap 0
        #ndd -set /dev/qfe adv_autoneg_cap 0

# dmfe-Interfaces (Netra X1)
# dmfe0
        #ndd -set /dev/dmfe0 adv_100fdx_cap 1
        #ndd -set /dev/dmfe0 adv_100hdx_cap 0
        #ndd -set /dev/dmfe0 adv_10fdx_cap 0
        #ndd -set /dev/dmfe0 adv_10hdx_cap 0
        #ndd -set /dev/dmfe0 adv_autoneg_cap 0
# dmfe1
        #ndd -set /dev/dmfe1 adv_100fdx_cap 1
        #ndd -set /dev/dmfe1 adv_100hdx_cap 0
        #ndd -set /dev/dmfe1 adv_10fdx_cap 0
        #ndd -set /dev/dmfe1 adv_10hdx_cap 0
        #ndd -set /dev/dmfe1 adv_autoneg_cap 0

# vge-Interfaces (Gigabit 1.x)
# vge0
        #ndd -set /dev/vge instance 0
        #ndd -set /dev/vge fdr_filter 1
        #ndd -set /dev/vge link_negotiation 0
# vge1
        #ndd -set /dev/vge instance 1
        #ndd -set /dev/vge fdr_filter 1
        #ndd -set /dev/vge link_negotiation 0

# ge-Interfaces (Gigabit 2.x and 3.x)
# example: forced 1000 Mbit/s, 802.3x Flow Control send and receive
# check if your NIC is 802.3x capable (ndd parameter pause_cap, asm_dir_cap) 
# ge0
ndd -set /dev/ge instance 0
ndd -set /dev/ge adv_1000fdx_cap 1
ndd -set /dev/ge adv_1000hdx_cap 0
ndd -set /dev/ge adv_pauseTX 1
ndd -set /dev/ge adv_pauseRX 1
ndd -set /dev/ge adv_1000autoneg_cap 0
# ge1
ndd -set /dev/ge instance 1
ndd -set /dev/ge adv_1000fdx_cap 1
ndd -set /dev/ge adv_1000hdx_cap 0
ndd -set /dev/ge adv_pauseTX 1
ndd -set /dev/ge adv_pauseRX 1
ndd -set /dev/ge adv_1000autoneg_cap 0

# ce-Interfaces (GigaSwift 1.0 UTP)
# example: forced 1000 Mbit/s, 802.3x Flow Control send and receive
# ce0
        #ndd -set /dev/ce instance 0
        #ndd -set /dev/ce link_master 0 # one link partner must have 1, the 
other 0
        #ndd -set /dev/ce adv_1000fdx_cap 1
        #ndd -set /dev/ce adv_1000hdx_cap 0
        #ndd -set /dev/ce adv_100fdx_cap 0
        #ndd -set /dev/ce adv_100hdx_cap 0
        #ndd -set /dev/ce adv_10fdx_cap 0
        #ndd -set /dev/ce adv_10hdx_cap 0
        #ndd -set /dev/ce adv_asmpause_cap 0
        #ndd -set /dev/ce adv_pause_cap 1
        #ndd -set /dev/ce adv_autoneg_cap 0
# ce1
        #ndd -set /dev/ce instance 1
        #ndd -set /dev/ce link_master 0 # one link partner must have 1, the 
other 0
        #ndd -set /dev/ce adv_1000fdx_cap 1
        #ndd -set /dev/ce adv_1000hdx_cap 0
        #ndd -set /dev/ce adv_100fdx_cap 0
        #ndd -set /dev/ce adv_100hdx_cap 0
        #ndd -set /dev/ce adv_10fdx_cap 0
        #ndd -set /dev/ce adv_10hdx_cap 0
        #ndd -set /dev/ce adv_asmpause_cap 0
        #ndd -set /dev/ce adv_pause_cap 1
        #ndd -set /dev/ce adv_autoneg_cap 0

# switch MTU path discovery off
        #ndd -set /dev/ip ip_path_mtu_discovery 0
# TCP/UDP buffer tunings
        #ndd -set /dev/tcp tcp_xmit_hiwat 65535
        #ndd -set /dev/tcp tcp_recv_hiwat 65535
        #ndd -set /dev/udp udp_xmit_hiwat 65535
        #ndd -set /dev/udp udp_recv_hiwat 65535
# TCP congestion window size, for Solaris <= 2.5, only
        #ndd -set /dev/tcp tcp_cwnd_max 65535
# Change TCP maximum retransmission interval, for Solaris <= 7, only
        #ndd -set /dev/tcp tcp_rexmit_interval_max 60000
# Change number of TCP slow start initial packets to work arround
# slow start bug in BSD stack and Windows. For Solaris <= 7, only
        #ndd -set /dev/tcp tcp_slow_start_initial 2 

         ;;

     'stop')
         echo "No kernel parameters changed."
         ;;

        *)
         echo "Usage: $0 {start|stop}"
         ;;

  esac
  exit 0

--Boundary_(ID_his7W0oQtT6oR+nWGpnOHQ)
Content-type: text/plain; name=show_eri.sh
Content-transfer-encoding: 7BIT
Content-disposition: inline; filename=show_eri.sh

#!/bin/sh
#echo "show_eri"
# Display ERI/network parameters.
# Based on get_hme_paramters, Version 2 by Mark F 26/2/97
#
#
#       ******************************************************
#       *                                                    *
#       *                    DISCLAIMER                      *
#       *                                                    *
#       ******************************************************
#
#
# The contents of this file  are intended to  be read as an example.
# This  is not  a  supported product of Sun Microsystems  and  no hotline
# calls  will  be accepted which directly relate to this information.
#
# NO LIABILITY WILL BE  ACCEPTED BY SUN MICROSYSTEMS FOR ANY LOSS (DIRECT
# OR CONSEQUENTIAL) INCURRED IN ANY WAY BY ANY PARTY THROUGH THE USE OF
# THIS INFORMATION.
#
# NO WARRANTY  OF  ANY SORT  IS IMPLIED OR GIVEN FOR ANY CODE DERIVED
# FROM THIS INFORMATION.
#

# Did ya know on Solaris 2.x you can get simular results with:
# ndd /dev/eri `ndd /dev/eri \? | /bin/awk '$1 !~ /\?/ {print $1 "\n" }' | \
#     tee /tmp/eri1` > /tmp/eri2;pr -F -m -t /tmp/eri1 /tmp/eri2
# Stacey, 24/3/97

# get_ndd_info(): $NDD is set before this routine is called.
# argument 1 is the variable name we are interested in, and
# although not seen here it is used within the $NDD variable.
get_ndd_info(){
        echo "$1|$2|$3|`$NDD`" | \
        /bin/awk -F\| \
        'BEGIN {tab="                                                  "} \
        {
         if ( substr($4,length($4),1) == "0" ) { \
           ans=$2 } \
         else if ( substr($4,length($4),1) == "1" ) { ans=$3 } \
          else { ans="("$1" = "$4")" } \
        printf("%s %s(%s = %s)\n",ans, \
         substr(tab,length(ans),50),$1,$4)}'
}

# Mike, 16/Jun/00 : Use path_to_inst instead of prtconf command, because
#                   i.e. ge will also be reported as network
# 16/Feb/98 : 1.5:  INSTANCES should be space seperated....
# 11/Feb/98 : 1.4:  Ultra III platform reports onboard eri as 'network'!
# get_instances: find the instance numbers of eri interfaces installed.
get_instances(){
case $OS in
5.*)
        INSTANCES=`grep eri /etc/path_to_inst |grep network |/bin/nawk 
'{printf("%d ",$2)}'`
#       INSTANCES=`prtconf | /bin/nawk -F# '/network/ || /eri/ {printf("%d 
",$2)}'`
## For Solaris testing with no eri...
#        INSTANCES=1
        ;;
*)
        INSTANCES=`echo "erips/4X" | adb -k /vmunix /dev/mem |
                /bin/nawk 'BEGIN { FS = ":"; RS="" }
                { print \$NF }' | 
                /bin/nawk '{for (i = 1; i <= NF; i++) {
                if ($i != 0)
                printf "%s ", $i
                }}'`
        ;;
## For SunOS testing with no eri...
# *) INSTANCES=1
esac
}

set_instance(){
case $OS in
5.*)
## Comment out the following line if testing on Solaris without ERI
        /usr/sbin/ndd -set /dev/eri instance $1
        ;;
*)
        get_eri_info $1
        ;;
esac
}

get_eri_info() {
  parameters=`echo "$1+0x130/29D" | adb -k /vmunix /dev/mem | /bin/nawk 
'/physmem/ { next }
                               {print substr($0,index($0,":")+1,length($0))}'`
## For SunOS testing
# parameters='0 1 0 0 8 4 0 0 1 0 0 1 0 1 1 0 0 1 0 1 0 0 0 0 0 0 0 1 16'
 i=0
 for param in $parameters ; do
   i=`expr $i + 1`
   case $i in
     1) ERItransceiver_inuse=$param ;;
     2) ERIlink_status=$param ;;
     3) ERIlink_speed=$param ;;
     4) ERIlink_mode=$param ;;
     5) ERIipg1=$param ;;
     6) ERIipg2=$param ;;
     7) ERIuse_int_xcvr=$param ;;
     8) ERIpace_size=$param ;;
     9) ERIadv_autoneg_cap=$param ;;
    10) ERIadv_100T4_cap=$param ;;
    11) ERIadv_100fdx_cap=$param ;;
    12) ERIadv_100hdx_cap=$param ;;
    13) ERIadv_10fdx_cap=$param ;;
    14) ERIadv_10hdx_cap=$param ;;
    15) ERIautoneg_cap=$param ;;
    16) ERI100T4_cap=$param ;;
    17) ERI100fdx_cap=$param ;;
    18) ERI100hdx_cap=$param ;;
    19) ERI10fdx_cap=$param ;;
    20) ERI10hdx_cap=$param ;;
    21) ERIlp_autoneg_cap=$param ;;
    22) ERIlp_100T4_cap=$param ;;
    23) ERIlp_100fdx_cap=$param ;;
    24) ERIlp_100hdx_cap=$param ;;
    25) ERIlp_10fdx_cap=$param ;;
    26) ERIlp_10hdx_cap=$param ;;
    27) ERIinstance=$param ;;
    28) ERIlance_mode=$param ;;
    29) ERIipg0=$param ;;
    *) ;;
   esac
 done
}

check_root_id()
{
# Check that the users id is root 
 
   id | grep root >/dev/null 
   if [ $? -ne 0 ]
        then 
        echo "Only the super-user can execute this script " 
        abort 
   fi 
} 
 
get_version() {

# Mike, 16/Jun/00 : newest driver only called Ethernet Driver
#       VERSION=`strings /kernel/drv/hme |grep "FEPS Ethernet Driver"`
        VERSION=`strings /kernel/drv/sparcv9/eri |grep "Ethernet Driver"`

}

##
##  Start
##

check_root_id
get_version
OS=`uname -r`

case $OS in
  5.*)
        ECHO=/usr/bin/echo
        NDD='eval /usr/sbin/ndd /dev/eri $1'
        ## For Solaris testing with no ERI
        # NDD='eval $PWD/ndd /dev/eri $1'
        ;;
  *)
        ECHO=/usr/5bin/echo
        NDD='eval eval echo \$ERI$1'
        ;;
esac

$ECHO "\t\tConfigured Interfaces:\n"
ifconfig -a

$ECHO "\n\n\t\tERI $VERSION\n"
instance=${1}
get_instances

if [ "$INSTANCES" = "" ]
  then
    $ECHO "\nNo eri interfaces found on host `uname -n`\n"
    exit
fi
echo "Detailing Instances: ${INSTANCES}"

        $ECHO "NOTE Parameters MAY NOT be accurate if the card is not connected 
to a HUB/SWITCH"
        $ECHO "or if the interface is unused ( IE: Not plumbed in )."
        $ECHO "Try snooping the device to inititalize it."

for instance in $INSTANCES
        do
        $ECHO "\n\nConfiguration of eri interface # $instance.\n"

        $ECHO "\nLink Parameters.\n\n"
        set_instance $instance
        get_ndd_info link_status "The link is DOWN" "The link is UP"
        get_ndd_info link_speed "Speed = 10Mb/S" "Speed = 100Mb/S"
        get_ndd_info link_mode "Mode = Half Duplex" "Mode = Full Duplex"
        get_ndd_info transceiver_inuse "Using INTERNAL Transceiver" "Using 
EXTERNAL Transceiver"
        get_ndd_info use_int_xcvr "Use External Transceiver if present" "Only 
use Internal Transceiver"
        get_ndd_info lance_mode "LANCE mode is DISABLED" "LANCE mode is ENABLED"
        get_ndd_info adv_autoneg_cap "Will NOT auto-negotiate" "WILL 
auto-negotiate"
        get_ndd_info lp_autoneg_cap "Link partner (switch) DOES NOT 
auto-negotiate" "Link partner (switch) HAS auto-negotiate ability"

        $ECHO "\nMisc parameters.\n\n"

        get_ndd_info pace_size "Number of back to back packets" "Number of back 
to back packets"
        get_ndd_info ipg0 "ipg0" "ipg0"
        get_ndd_info ipg1 "ipg1" "ipg1"
        get_ndd_info ipg2 "ipg2" "ipg2"
        
        $ECHO "\nCard is set to advertise the following:\n"

        get_ndd_info adv_100T4_cap "100T4 = NO" "100T4 = YES"
        get_ndd_info adv_100fdx_cap "100FDX = NO" "100FDX = YES"
        get_ndd_info adv_100hdx_cap "100HDX = NO" "100HDX = YES"
        get_ndd_info adv_10fdx_cap "10FDX = NO" "10FDX = YES"
        get_ndd_info adv_10hdx_cap "10HDX = NO" "10HDX = YES"

        $ECHO "\nThe card supports the following:\n"

        get_ndd_info 100T4_cap "100T4 = NO" "100T4 = YES"
        get_ndd_info 100fdx_cap "100FDX = NO" "100FDX = YES"
        get_ndd_info 100hdx_cap  "100HDX = NO"  "100HDX = YES"
        get_ndd_info 10fdx_cap "10FDX = NO" "10FDX = YES"
        get_ndd_info 10hdx_cap "10HDX = NO" "10HDX = YES"
        get_ndd_info autoneg_cap "Auto Negotiation = NO" "Auto Negotiation = 
YES"

        $ECHO "\nThe HUB/SWITCH has advertised the following:\n"


        get_ndd_info lp_autoneg_cap "Link partner (switch) DOES NOT 
auto-negotiate"  "Link partner (switch) DOES HAVE auto-negotiate ability"
        $ECHO "\nThe following are ONLY valid if the Switch/HUB supports Auto 
Negotiation\n"

        get_ndd_info lp_100T4_cap "100T4 = NO" "100T4 = YES"
        get_ndd_info lp_100fdx_cap "100FDX = NO" "100FDX = YES"
        get_ndd_info lp_100hdx_cap "100HDX = NO" "100HDX = YES"
        get_ndd_info lp_10fdx_cap "10FDX = NO" "10FDX = YES"
        get_ndd_info lp_10hdx_cap "10HDX = NO" "10HDX = YES"

done

echo "\nNote1: if 'adv_autoneg_cap' is set to 0 (auto-negotiation disabled),"
echo "       the switch-side 'lp' link-partner, always displays 0."
echo "       In that case, you must telnet into the switch to inspect"
echo "       the port."

echo "\nNote2: If auto-negotiation is disabled (0) on one side, NIC or switch,"
echo "       it MUST BE disabled on both sides, or typically, one side"
echo "       may be locked down to 100-FDX and the other, negotiating"
echo "       side will ALWAYS negotiate to HDX, and result in mismatch"
echo "       failures, CRCs, RUNTS, FRAGS, LATE COLLISIONS, etc."

--Boundary_(ID_his7W0oQtT6oR+nWGpnOHQ)--