Matthew,
Do you know if there are any drawbacks to forcing window scaling as you
have set? Particularly, most of the clients to my Solaris 9 backup server
are Windows 2000. Will they behave well with the tcp settings you have ?
Robert Maiello
Pioneer Data Systems
On Sun, 30 Oct 2005 19:06:34 -0500, Matthew Huff <mhuff AT OX DOT COM> wrote:
>This is a pet peeve of mine. 100MB auto-negotiation isn't a "protocol"
>like PPP. In PPP, both sides negotiate capabilities. 100MB
>auto-negotiation isn't like that at all. What happens is when the link
>becomes active, both sides "listen in" to the traffic and make an
>educated guess as to what the other side is speaking. One of the
>problems with this is that Cisco and other switches have spanning-tree
>protocols that block outgoing traffic until the spanning-tree BDU fails
>to show up on another port.
>
>This blocking prevents the client card from seeing any traffic at all
>during the window and it really is making a "guess". Even disabling the
>spanning-tree blocking (set spantree portfast enable) isn't enough. You
>should disable etherchannel and trunk negotiation as well (catalyst have
>a macro - "set port host"). Even given all that, if a random broadcast
>packet such as an arp, etc... doesn't go over the wire during the short
>period during the initialization, both sides will have no choice but to
>make a blind guess. This is why sometimes it will work and fail other
>times. Later IOS and Catalyst version and newer NIC firmwares are better
>at the guess, but it will probably be someone who puts an old ethernet
>card into a new server that gets you.
>
>Why take the risk?
>
>
>Peter is right about Gigabit, they realized they had a problem and
>decided to add a real protocol (and needed to be able to negotiate
>hardware flowcontrol as well). BTW, everyone has made sure what there
>gigabit ethernet flow control is set on their Legato server, right? :)
>
>Network performance can be tricky and non-obvious. Sending terrabytes of
>data over the wire for backups can be tricky and sometimes even with
>gigabit ethernet require tuning. For example, we have a dedicated VLAN
>setup for our NDMP NAS backups with jumbo frames turned on, and tcp
>tweakings suchs as size of tcp buffers and window scaling forced on.
>Legato should be setting the window scaling when it opens the tcp
>sockets, but doesn't appear to be, however you can tune settings to make
>sure it is set for all sockets. This means bigger TCP flow control
>windows which is important in high-volume, high-speed data streams such
>as during backups.
>
>Here is my startup script on my Sun V490 running Solairs 9:
>
>#!/bin/sh
>
>PATH=/bin:/usr/sbin:/usr/sbin;export PATH
>
>ndd -set /dev/tcp tcp_max_buf 4194304
>ndd -set /dev/tcp tcp_cwnd_max 2097152
>ndd -set /dev/tcp tcp_xmit_hiwat 1048576
>ndd -set /dev/tcp tcp_recv_hiwat 1048576
>ndd -set /dev/tcp tcp_tstamp_always 1
>ndd -set /dev/tcp tcp_wscale_always 1
>
>
>I haven't verified for sure that the TCP timestamp (tcp_tstamp_always)
>is necessary; i.e, protecting from sequence number wrapping, but it's
>usually recommended for the same type of data streams.
>
>
>
>
>
>
>--
>Matthew Huff | One Manhattanville Rd
>Director of Operations | Purchase, NY 10577
>OTA LLC | Phone: 914-460-4039
>mailto:mhuff AT ox DOT com | Fax: 914-460-4139
>-----Original Message-----
>From: Legato NetWorker discussion [mailto:NETWORKER AT LISTSERV.TEMPLE DOT EDU]
>On Behalf Of Peter Viertel
>Sent: Sunday, October 30, 2005 6:45 PM
>To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
>Subject: Re: [Networker] Compare Netoworker to other backup products
>
>Hey, I don't think network performance is at all off-topic on this
>list..
>
>And for my $0.02 worth. I agree with Oscar entirely about the
>reliability of auto-neg...
>
>On my site there's about 200 people who deal with infrastructure, and
>there are many differing opinions on the subject, and every week I end
>up chasing down another lame server set up by someone who hasn't grasped
>the concept, or its been plugged into a switch port that has old config
>on it....
>
>Purely from the issue of confusion over auto-neg I view Fast Ethernet as
>a broken protocol, and only support gigabit on new installations... Of
>course, inevitably, some people have tried to 'lock-down' gigabit... But
>at least in these cases you can point at the IEEE standard which states
>clearly that autoneg is not optional.
>
>... And if anyone else wants a bit more light reading, here is Sun's
>position on the subject...
>http://www.sun.com/blueprints/0704/817-7526.pdf
>
>
>-----Original Message-----
>From: Legato NetWorker discussion [mailto:NETWORKER AT listserv.temple DOT edu]
>On Behalf Of Oscar Olsson
>Sent: Monday, 31 October 2005 9:50 AM
>To: NETWORKER AT listserv.temple DOT edu
>Subject: Re: [Networker] Compare Netoworker to other backup products
>
>On Sat, 29 Oct 2005, Dag Nygren wrote:
>
>DN> And here is a third one!
>DN> I installed 10 -20 bigger Networker systems last year and saw about
>DN> 50% of these failing to negotiate decently under auto-negostiate.
>DN> Usually they ended up with one end 100/half and the other 100/full.
>
>And both ends were configured to auto-negotiate?
>
>DN> So IMHO it is really stupid to trust the auto-stuff. And most of the
>
>DN> times you dont't even nitice the poor performance until installing
>Networker...
>DN>
>DN> And it is REALLY hard to convince the network-guys that Their
>DN> Network has a flaw in it ;-)
>
>True, since I'm the network guy as well, or at least one of them, at our
>site. :)
>
>Anyway, I feel like saying "You're probably all wrong, or you use crappy
>switches, or fail to correctly set both ends to auto-negotiate or
>possibly all of the above!". :) But I won't, since I'm such a nice guy.
>
>Instead I'll include an URL on Cisco's website, that deals somewhat with
>this topic:
>
>http://www.cisco.com/en/US/tech/tk389/tk214/technologies_tech_note09186a
>0080094781.shtml
>
>One interesting part is:
>
>"When to Use Ethernet 10/100Mb Auto-Negotiation
>
>Auto-negotiation is an optional function of the IEEE 802.3u Fast
>Ethernet standard that enables devices to automatically exchange
>information over a link about speed and duplex abilities.
>
>Auto-negotiation is targeted at ports which are allocated to areas where
>transient users or devices connect to a network. For example, many
>companies provide shared offices or cubes for Account Managers and
>System Engineers to use when they are in the office rather than on the
>road. Each office or cube will have an Ethernet port permanently
>connected to the office's network. Because it may not be possible to
>ensure that every user has either a 10Mb, a 100Mb Ethernet, or a
>10/100Mb card in their laptop, the switch ports that handle these
>connections must be able to negotiate their speed and duplex mode. The
>alternative would be to provide both a 10Mb and a 100Mb port in each
>office or cube and label them accordingly.
>
>One of the most common causes of performance issues on 10/100Mb Ethernet
>links is when one port on the link is operating at half-duplex while the
>other port is operating at full-duplex. This occasionally happens when
>one or both ports on a link are reset and the auto-negotiation process
>doesn't result in both link partners having the same configuration. It
>also happens when users reconfigure one side of a link and forget to
>reconfigure the other side. Both sides of a link should have
>auto-negotiation on, or both sides should have it off. Our current
>recommendation is to leave auto-negotiation on for those devices
>compliant with 802.3u.
>
>Many performance-related support calls will be avoided by correctly
>configuring auto-negotiation. Many Catalyst Ethernet switching modules
>support 10/100Mb and half- or full-duplex. Exceptions include the
>Ethernet Group switch modules. The show port capabilities {mod_num} |
>{mod_num/port_num} command will show you if the module you are working
>on supports 10/100Mb and half- or full-duplex. This document uses two
>WS-X5530 Supervisor Engine IIIs, each with two optional uplink
>10/100BaseTX Ethernet ports installed."
>
>I would also like to recommend this document:
>http://www.cisco.com/en/US/products/hw/switches/ps700/products_tech_note
>09186a00800a7af0.shtml
>
>
>And yes, I have seen several cases of where other network administrators
>have claimed what has already been claimed about the suspected negative
>effects of enabling auto negotiation. All of those cases ended up to be
>related to the following errors:
>
>* One end was configured for auto negotiation, or the two ends of the
>link didn't match the speed/duplex setting.
>* Some 3com cards were per default set, in windows, to use "hardware
>default" instead of auto negotiate, which for some reason wasn't the
>"hardware default" setting.
>
>However, considering the number of reports, its also probable that some
>low-end switches and/or NIC's haren't 100% compliant with the 802.3u
>standard. But if one runs backups, I guess one wouldn't use that range
>of equipment anyway? :) I can't think of an adapter based on Intel, 3com
>or broadcom chipsets that has had any problem with this in at least five
>years.
>
>I understand that this thread has become somewhat offtopic, and
>furthermore, I don't think I have much more to say about this issue, so
>this will probably be my last post in this thread. But it would be
>interesting for me to know as a network administrator what specific NICs
>and switches are having problems with auto negotiation. Who knows, I
>might bump into those combinations one day. Email me off-list.
>
>//Oscar
>
>To sign off this list, send email to listserv AT listserv.temple DOT edu and
>type "signoff networker" in the body of the email. Please write to
>networker-request AT listserv.temple DOT edu if you have any problems wit this
>list. You can access the archives at
>http://listserv.temple.edu/archives/networker.html or via RSS at
>http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER
>
>
>
>NOTICE
>This e-mail and any attachments are confidential and may contain
>copyright material of Macquarie Bank or third parties. If you are not
>the intended recipient of this email you should not read, print,
>re-transmit, store or act in reliance on this e-mail or any attachments,
>and should destroy all copies of them. Macquarie Bank does not guarantee
>the integrity of any emails or any attached files. The views or opinions
>expressed are the author's own and may not reflect the views or opinions
>of Macquarie Bank.
>
>To sign off this list, send email to listserv AT listserv.temple DOT edu and
>type "signoff networker" in the body of the email. Please write to
>networker-request AT listserv.temple DOT edu if you have any problems wit this
>list. You can access the archives at
>http://listserv.temple.edu/archives/networker.html or via RSS at
>http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER
>
>To sign off this list, send email to listserv AT listserv.temple DOT edu and
type "signoff networker" in the
>body of the email. Please write to networker-request AT listserv.temple DOT
>edu
if you have any problems
>wit this list. You can access the archives at
http://listserv.temple.edu/archives/networker.html or
>via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER
>=========================================================================
To sign off this list, send email to listserv AT listserv.temple DOT edu and
type "signoff networker" in the
body of the email. Please write to networker-request AT listserv.temple DOT edu
if you have any problems
wit this list. You can access the archives at
http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER
|
