Gary,

Silly question first:  are you sure the NFS server has 10 Gb connections?

In addition to trying the offload settings that Skylar mentioned, you could
look at using jumbo frames.  You'll need it supported and enabled on the NFS
server/array as well, and also on any intervening switches or routers.

If you weren't using jumbo frames with the 1 Gb NIC either, that wouldn't
account for the better performance before.  It definitely won't make up the
difference between your TSM performance and your iPerf3 test results, but you
could still expect a noticeable improvement.

Also check for firmware and driver updates for your 10 Gb NIC, and look at
available driver options or flags that might affect performance.  Depending
on the NIC, you may also be better off using the NIC OEM's driver instead of
the one Red Hat provides (if they do provide one).

Good luck, and let us know if (and how!) you solve it.

=Dave


On 02/03/2016 07:15 AM, Lee, Gary wrote:
> To Mike:
>
> Thanks for the link, hadn't seen that.
>
> Also, the nfs server is a dedicated SAN head end, so I have little to no 
> control of its nfs parms.
>
> To
>
> -----Original Message-----
> From: ADSM: Dist Stor Manager [mailto:ADSM-L AT VM.MARIST DOT EDU] On Behalf 
> Of Skylar Thompson
> Sent: Tuesday, February 02, 2016 4:43 PM
> To: ADSM-L AT VM.MARIST DOT EDU
> Subject: Re: [ADSM-L] getting performance from nfs storage over 10 gb link
>
> Hi Gary,
>
> To Skylar:
>
> I will check into those options.
>
> To all:
>
> I can say that tsm performance was better when the same storage was mounted 
> with the same mount options using a 1 gb adapter.
> Strange, I know, but that's the riddle I'm dealing with.
>
> Thanks for everything so far. Will keep you posted on results.
>
> We don't use NFS for our TSM servers, but we have been struggling with NFS
> over 10GbE in other areas. While not a universal solution, we've gotten
> significant performance improvements by disabling the following NIC offload
> options:
>
> gro
> lro
> rxvlan
> txvlan
> rxhash
>
> For instance, you can disable gro with
>
> ethtool --offload eth0 gro off
>
> (assuming eth0 is your NIC)
>
> There's a bunch more we haven't had a chance to play with, but hopefully
> that's a starting point.
>
> On Tue, Feb 02, 2016 at 05:54:46PM +0000, Lee, Gary wrote:
>> Tsm server 6.4.3,
>> RHEL 6.7
>> 4 Dual-Core AMD Opteron(tm) Processor 8220 SE CPUs
>> 128 gB memory
>>
>> I recently installed an intel 10 gB Ethernet card.
>> Iperf3 test shown below gives around 7 gb throughput.
>> However, when running multiple migration and reclamation processes, and 
>> watching network traffic through an independent tool, I cannot get over the 
>> 900 mb/s threshold.
>>
>> The tsm storage pools are on a file system mounted as follows:
>>
>> Nxback-Pool02.servers.bsu.edu:/volumes/Pool02/TSMBackup/tsm02 
>> /tsminst1/storage nfs 
>> rw,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys
>>  0 0
>>
>> I am running out of options.
>>
>> I don't expect to see the full throughput, as disk speeds will have a good 
>> deal of impact.
>>
>> Any ideas would be helpful.
>
> --
> -- Skylar Thompson (skylar2 AT u.washington DOT edu)
> -- Genome Sciences Department, System Administrator
> -- Foege Building S046, (206)-685-7354
> -- University of Washington School of Medicine
>

--
Hello World.                                David Bronder - Systems Architect
Segmentation Fault                                      ITS-EI, Univ. of Iowa
Core dumped, disk trashed, quota filled, soda warm.   david-bronder AT uiowa 
DOT edu