Networker

Re: [Networker] Performance issues with LTO 2 Ultrium and unable to write to all tapes at same time

2004-01-12 11:38:32
Subject: Re: [Networker] Performance issues with LTO 2 Ultrium and unable to write to all tapes at same time
From: Paul Langford <plangfor AT AB.BLUECROSS DOT CA>
To: NETWORKER AT LISTMAIL.TEMPLE DOT EDU
Date: Mon, 12 Jan 2004 09:41:21 -0700
We run an HP-UX server with a Storagetek L80 holding 4 HP LTO-2 drives and
had the same issue.

To get the higher speeds you must have either Turbopak or Power Edition of
Networker on the backup server.  These stop NetWorker from using the local
loopback.  Also if there is the following file, /nsr/debug/noimmediate, it
disables the higher speeds if Turbo or Power Edition are installed.

We can get over 25-30MB/s per drive, on all drives. If we load the streams
properly.

Without power edition or turbopak we maxed out at a max speed of 10-15Mbs
total for the server.


Paul Langford


-----Original Message-----
From: Alex Theodore [mailto:alex AT FUZZYCHEESE DOT COM]
Sent: Sunday, January 11, 2004 3:38 PM
To: NETWORKER AT LISTMAIL.TEMPLE DOT EDU
Subject: Re: [Networker] Performace isssues with LTO 2 Ultrium and
unable to write to all tapes at same time


I was mistakened.. the SCSI layout is like so..

-   robot is on its own single ended scsi bus
-   there are two drives per controller, of which the controller is a
dual-channel with each drive on its own channel

-   The trunking software doesn't seem to make a difference as far as the
througput goes.  We are considering changing it out for a Fibre-Channel
cards instead (perhaps two 2gb Fibre Channel Qlogic cards)  At that point it
may be worth while to configure the client as a storage node and do san
backups.

-   I'll have to check out the other stuff ..

Thanks !



On Sun, 11 Jan 2004 22:20:44 +0000
"Mark Bradshaw (BTOpenWorld)" <notthehoople AT BTOPENWORLD DOT COM> wrote:

> Alex,
>
> A few things to check:
>
> - Are you using CDI under NetWorker 7.1? If so, turn it off - this seems
to
> introduce some performance problems with LTO drives.
>
> - Could be wrong here but isn't the X6541A an HVD SCSI controller? Are
your
> drives HVD? I wasn't aware that HP LTO2 drives came in HVD! If everything
> checks out and *is* the same type of SCSI and you've gone for HVD (rather
> than LVD) then this will be a major bottleneck. HVD usually sustains
around
> 20MB/s in total - just under half of what you can expect from a single
LTO2
> drive. You need to move your drives so they are each on a single SCSI
> channel to get the best throughput in your config.
>
> - What disk units are you using with your V880 and E4500s? Are these units
> capable of the data rates you are trying to extract from them? If you are
> just using normal, internal disks then you are not going to achieve the
> throughput you want.
>
> - Are you using tape media compatible with your tape drives? Check you are
> using LTO2 tapes *and* that they are certified for use with HP drives.
Using
> the wrong tapes can cause slow performance and poor reliability.
>
> - What number and type of CPUs have you installed in the V880? If you are
> trunking your gigabit connections then you will lose a significant amount
of
> CPU which will be used just talking to the network. Monitor the CPU load
on
> the V880 during backup and see if that is causing a bottleneck. Also
monitor
> the CPU load on the E4500 to see what effect the trunking is having on
this
> server.
>
> - Can you make the drives run at a good speed using bigasm? This removes
> disk traffic and network traffic from the equation and shows you
> (effectively) what can be written down to the tape via CPU. A good start
> when using new tape drives!
>
> After checking all of this I'd start with a single drive and load it until
> you get a good throughput. If you can't produce enough data to achieve
good
> throughput on 1 drive you'll have no chance when using lots of drives!
With
> the data rates you have seen so far you really should be able to beat your
> total data rate seen using just a single drive!
>
> A final note - I don't believe the IO architecture of the V880 will manage
> to run all 6 LTO2 drives at full tilt. I haven't done all the sums for
this
> box but would expect, based on previous tests done using LTO1, that the
> server will max out at between 3-4 LTO2 drives.
>
> Good luck
>
> Mark
>
> > I tried local backups and I get the same story 3-6 mb/s.
> >
> > The L700 is connected SCSI configuration is like so:
> >
> > - 2 drives per scsi bus (X6541  Dual Differential Ultra/Wide SCSI - pci)
thus
> > 3 controllers
> >
> > I'll look into the whole i/o contention theory with the nsr db by using
iostat
> > on the box while backing it up.
> >
> > Thanks,
> >
> > Alex
> >
> > On Sun, 11 Jan 2004 12:45:57 -0500
> > Paul Galjan <pgaljan AT YAHOO DOT COM> wrote:
> >
> >> I don't know if you're going to be able to get the advertised 180 MB/s
for
> >> all six drives.  But the place to start would be to see if you can get
> >> anywhere close to that rate backing stuff up without going over the
network.
> >> See if you can write two data streams at 30 MB/s each, step it up from
> >> there.  See when it breaks.
> >>
> >> Another area to check would be to make sure that you are effectively
load
> >> balancing over both the NICs.
> >>
> >> Yet another suspect would be the Networker database.  You've got a
partition
> >> with 3 MB files, so each drive should be backing up 10 records per
second.
> >> That 60 inodes per second, each updating the database.  I'm no
Networker
> >> database expert, so I can't say for sure if the software can handle
that
> >> (perhaps someone on this list can), but you can definitely tell if it's
a
> >> hardware bottleneck.
> >>
> >> How is the SCSI bus configured (how many channels are the drives spread
> >> over?).
> >>
> >> --paul
> >>
> >>
> >> ----- Original Message -----
> >> From: "Alex Theodore" <alex AT FUZZYCHEESE DOT COM>
> >> To: <NETWORKER AT LISTMAIL.TEMPLE DOT EDU>
> >> Sent: Sunday, January 11, 2004 11:19 AM
> >> Subject: [Networker] Performace isssues with LTO 2 Ultrium and unable
to
> >> write to all tapes at same time
> >>
> >>
> >>> I am only getting about 3-6 megabytes/s when backing up to LTO 2 tape
> >> drives.  I would expect that I should get somewhere near the advertised
> >> value, but I am not.  Here are the details.
> >>>
> >>> - Legato Network 6.7.1
> >>> - Solaris 8 (latest patches)
> >>> - Sun L700 with 6 HP LTO 2 tape drives
> >>> - Backup Server is a Sun Fire V880
> >>> - Backup Client is Sun Enterprise 4500
> >>> - Network pipe for backups is a dedicated, directly connected (no
switch),
> >> dual gigabit ethernet, which is trunked (glued together into 1 virtual
pipe)
> >>>
> >>>
> >>> The data on the client which is being backed up are images (photos)
which
> >> are a few megabytes in size.  I decided to split up the save sets into
6
> >> different client configurations with the parallelism set to 6 so that
all 6
> >> tape drives would be used at the same time during backups but this
isn't
> >> happening either.. I can only seem to write to a few tape drives at
once..
> >>>
> >>> Any ideas would be greatly appreciated.
> >>>
> >>> TIA,
> >>>
> >>>   Alex
> >>>
> >>> --
> >>> Alex Theodore
> >>> alex AT fuzzycheese DOT com
> >>> Boca Raton, FL  USA
> >>>
> >>> --
> >>> Note: To sign off this list, send a "signoff networker" command via
email
> >>> to listserv AT listmail.temple DOT edu or visit the list's Web site at
> >>> http://listmail.temple.edu/archives/networker.html where you can
> >>> also view and post messages to the list.
> >>> =*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=
> >>
> >> --
> >> Note: To sign off this list, send a "signoff networker" command via
email
> >> to listserv AT listmail.temple DOT edu or visit the list's Web site at
> >> http://listmail.temple.edu/archives/networker.html where you can
> >> also view and post messages to the list.
> >> =*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=
> >>
> >
> >
> > --
> > Alex Theodore
> > alex AT fuzzycheese DOT com
> > Boca Raton, FL  USA
> >
> > --
> > Note: To sign off this list, send a "signoff networker" command via
email
> > to listserv AT listmail.temple DOT edu or visit the list's Web site at
> > http://listmail.temple.edu/archives/networker.html where you can
> > also view and post messages to the list.
> > =*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=
>
> --
> Note: To sign off this list, send a "signoff networker" command via email
> to listserv AT listmail.temple DOT edu or visit the list's Web site at
> http://listmail.temple.edu/archives/networker.html where you can
> also view and post messages to the list.
> =*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=
>


--
Alex Theodore
alex AT fuzzycheese DOT com
Boca Raton, FL  USA

--
Note: To sign off this list, send a "signoff networker" command via email
to listserv AT listmail.temple DOT edu or visit the list's Web site at
http://listmail.temple.edu/archives/networker.html where you can
also view and post messages to the list.
=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=
PLEASE NOTE:     This communication, including any attached documentation,
is intended only for the person or entity to which it is addressed, and may
contain confidential, personal, and/or privileged information.   Any
unauthorized disclosure, copying, or taking action on the contents is
strictly prohibited.  If you have received this message in error, please
contact us immediately so we may correct our records.   Please then delete
or destroy the original transmission and any subsequent reply.   Thank you.

--
Note: To sign off this list, send a "signoff networker" command via email
to listserv AT listmail.temple DOT edu or visit the list's Web site at
http://listmail.temple.edu/archives/networker.html where you can
also view and post messages to the list.
=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=

<Prev in Thread] Current Thread [Next in Thread>