Networker

Re: [Networker] Performace isssues with LTO 2 Ultrium and unable to write to all tapes at same time

2004-01-11 17:17:24
Subject: Re: [Networker] Performace isssues with LTO 2 Ultrium and unable to write to all tapes at same time
From: "Mark Bradshaw (BTOpenWorld)" <notthehoople AT BTOPENWORLD DOT COM>
To: NETWORKER AT LISTMAIL.TEMPLE DOT EDU
Date: Sun, 11 Jan 2004 22:20:44 +0000
Alex,

A few things to check:

- Are you using CDI under NetWorker 7.1? If so, turn it off - this seems to
introduce some performance problems with LTO drives.

- Could be wrong here but isn't the X6541A an HVD SCSI controller? Are your
drives HVD? I wasn't aware that HP LTO2 drives came in HVD! If everything
checks out and *is* the same type of SCSI and you've gone for HVD (rather
than LVD) then this will be a major bottleneck. HVD usually sustains around
20MB/s in total - just under half of what you can expect from a single LTO2
drive. You need to move your drives so they are each on a single SCSI
channel to get the best throughput in your config.

- What disk units are you using with your V880 and E4500s? Are these units
capable of the data rates you are trying to extract from them? If you are
just using normal, internal disks then you are not going to achieve the
throughput you want.

- Are you using tape media compatible with your tape drives? Check you are
using LTO2 tapes *and* that they are certified for use with HP drives. Using
the wrong tapes can cause slow performance and poor reliability.

- What number and type of CPUs have you installed in the V880? If you are
trunking your gigabit connections then you will lose a significant amount of
CPU which will be used just talking to the network. Monitor the CPU load on
the V880 during backup and see if that is causing a bottleneck. Also monitor
the CPU load on the E4500 to see what effect the trunking is having on this
server.

- Can you make the drives run at a good speed using bigasm? This removes
disk traffic and network traffic from the equation and shows you
(effectively) what can be written down to the tape via CPU. A good start
when using new tape drives!

After checking all of this I'd start with a single drive and load it until
you get a good throughput. If you can't produce enough data to achieve good
throughput on 1 drive you'll have no chance when using lots of drives! With
the data rates you have seen so far you really should be able to beat your
total data rate seen using just a single drive!

A final note - I don't believe the IO architecture of the V880 will manage
to run all 6 LTO2 drives at full tilt. I haven't done all the sums for this
box but would expect, based on previous tests done using LTO1, that the
server will max out at between 3-4 LTO2 drives.

Good luck

Mark

> I tried local backups and I get the same story 3-6 mb/s.
>
> The L700 is connected SCSI configuration is like so:
>
> - 2 drives per scsi bus (X6541  Dual Differential Ultra/Wide SCSI - pci) thus
> 3 controllers
>
> I'll look into the whole i/o contention theory with the nsr db by using iostat
> on the box while backing it up.
>
> Thanks,
>
> Alex
>
> On Sun, 11 Jan 2004 12:45:57 -0500
> Paul Galjan <pgaljan AT YAHOO DOT COM> wrote:
>
>> I don't know if you're going to be able to get the advertised 180 MB/s  for
>> all six drives.  But the place to start would be to see if you can get
>> anywhere close to that rate backing stuff up without going over the network.
>> See if you can write two data streams at 30 MB/s each, step it up from
>> there.  See when it breaks.
>>
>> Another area to check would be to make sure that you are effectively load
>> balancing over both the NICs.
>>
>> Yet another suspect would be the Networker database.  You've got a partition
>> with 3 MB files, so each drive should be backing up 10 records per second.
>> That 60 inodes per second, each updating the database.  I'm no Networker
>> database expert, so I can't say for sure if the software can handle that
>> (perhaps someone on this list can), but you can definitely tell if it's a
>> hardware bottleneck.
>>
>> How is the SCSI bus configured (how many channels are the drives spread
>> over?).
>>
>> --paul
>>
>>
>> ----- Original Message -----
>> From: "Alex Theodore" <alex AT FUZZYCHEESE DOT COM>
>> To: <NETWORKER AT LISTMAIL.TEMPLE DOT EDU>
>> Sent: Sunday, January 11, 2004 11:19 AM
>> Subject: [Networker] Performace isssues with LTO 2 Ultrium and unable to
>> write to all tapes at same time
>>
>>
>>> I am only getting about 3-6 megabytes/s when backing up to LTO 2 tape
>> drives.  I would expect that I should get somewhere near the advertised
>> value, but I am not.  Here are the details.
>>>
>>> - Legato Network 6.7.1
>>> - Solaris 8 (latest patches)
>>> - Sun L700 with 6 HP LTO 2 tape drives
>>> - Backup Server is a Sun Fire V880
>>> - Backup Client is Sun Enterprise 4500
>>> - Network pipe for backups is a dedicated, directly connected (no switch),
>> dual gigabit ethernet, which is trunked (glued together into 1 virtual pipe)
>>>
>>>
>>> The data on the client which is being backed up are images (photos) which
>> are a few megabytes in size.  I decided to split up the save sets into 6
>> different client configurations with the parallelism set to 6 so that all 6
>> tape drives would be used at the same time during backups but this isn't
>> happening either.. I can only seem to write to a few tape drives at once..
>>>
>>> Any ideas would be greatly appreciated.
>>>
>>> TIA,
>>>
>>>   Alex
>>>
>>> --
>>> Alex Theodore
>>> alex AT fuzzycheese DOT com
>>> Boca Raton, FL  USA
>>>
>>> --
>>> Note: To sign off this list, send a "signoff networker" command via email
>>> to listserv AT listmail.temple DOT edu or visit the list's Web site at
>>> http://listmail.temple.edu/archives/networker.html where you can
>>> also view and post messages to the list.
>>> =*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=
>>
>> --
>> Note: To sign off this list, send a "signoff networker" command via email
>> to listserv AT listmail.temple DOT edu or visit the list's Web site at
>> http://listmail.temple.edu/archives/networker.html where you can
>> also view and post messages to the list.
>> =*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=
>>
>
>
> --
> Alex Theodore
> alex AT fuzzycheese DOT com
> Boca Raton, FL  USA
>
> --
> Note: To sign off this list, send a "signoff networker" command via email
> to listserv AT listmail.temple DOT edu or visit the list's Web site at
> http://listmail.temple.edu/archives/networker.html where you can
> also view and post messages to the list.
> =*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=

--
Note: To sign off this list, send a "signoff networker" command via email
to listserv AT listmail.temple DOT edu or visit the list's Web site at
http://listmail.temple.edu/archives/networker.html where you can
also view and post messages to the list.
=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=