Networker

Re: [Networker] How to improve performance

2004-10-22 13:02:01
Subject: Re: [Networker] How to improve performance
From: "Fields, David" <David.Fields AT ACS-INC DOT COM>
To: NETWORKER AT LISTMAIL.TEMPLE DOT EDU
Date: Fri, 22 Oct 2004 13:00:46 -0400
If I remember right, Veritas basically does a serial backup, meaning that if
you tell it to backup drive C: and Drive D:, it will backup drive C: first,
then Drive D:.  As I mentioned in my previous message, if these drives are
actually contained on the same PHYSICAL disk (either by subdividing a RAID
container, SAN configuration, or just plain disk partitioning of multiple
drive letters on one physical drive), you can cause a lot of disk thrashing.
Remember, drive letters don't necessarily mean that there is are separate
physical disks for each drive letter.  For Veritas, when it's backing up
Drive C: for example, the disk is not having to also seek for data on Drive
D:.

Legato, on the other hand, can backup drive C: and D: at the same time.
This means that if the drive letters are actually on the same physical
disks, then the drives are doing a lot of head seeking in order to satisfy
the requests for data for both drive letters.  Again, remember that
different drive letters can reside on the same physical disks.

The only way to force Legato to backup one drive letter at a time is to
reduce the parallelism for either backup group, or for the particular
client.  You have to be careful how you do this, cause you can seriously
degrade your overall backup performance here.  Reducing the target sessions
for each tape drive only changes how many sessions that the drive will
accept before it starts doing round-robin assignment of sessions between
drives.  For example, if you have two drives, and the target sessions for
each drive is set to two (2), then if you start backing up server A which
has 5 savesets (say drive C, D, E, F, G, and this also assumes a parallelism
of at least 5 for the client and group), then Legato would assign the backup
streams of drive C and D to the first tape drive, sessions E and F to the
second tape drive, and then it has to start over, so session G goes to tape
drive 1.  In general, if you have LTO-2 drives, you want to make sure that
they are fed enough data to keep them busy, which usually means increasing
the target sessions.

Also, when you say that you see Legato go from 40MB/sec to 256kB/sec, that
is just a snap shot at that particular time, specially if you're looking at
it from the Networker Administrator program.  I haven't had a chance to
implement them yet, but there were some scripts written and published in the
Networker Journal which show how to get more accurate data of what is being
written to each drive.  One of these days I'm going to implement those
scripts so that I can do some better tuning of our backups.

Tuning Legato can be a horrendously complex job because of all the different
ways you can configure things.  When using Legato, the goal is to keep the
drive busy writing so that it doesn't have to write some, wait for data,
rewind, then start writing again.  I really like the power, flexibility, and
potential that Legato Networker has over Veritas and Arcserve, but it can be
tricky to get the most out of it.  I've heard that Veritas can do some of
what Legato can, but you have to buy the Netbackup version, and that is
extremely pricey compared to Legato the last time I checked.

And yes, Legato is not very good at handling lots of small files.  I would
really like for them start using some kind of buffering mechanism when
sending saveset streams so that they can maintain maximum throughput over
the network and tape drive.  With modern servers and networks, this should
be that difficult.  One of these days I'm going to play around with some of
the old Unix tools for doing stuff like this, and see if I can make them
work with Legato in some way.

Dave
-----Original Message-----
From: Legato NetWorker discussion [mailto:NETWORKER AT LISTMAIL.TEMPLE DOT EDU] 
On
Behalf Of Rahul Parasnis
Sent: Friday, October 22, 2004 11:41 AM
To: NETWORKER AT LISTMAIL.TEMPLE DOT EDU
Subject: Re: [Networker] How to improve performance

I can defnitely lookup with this Drive allocation . But the I am having only
one drive with 260 and one drive with 80 GB , that legato just starts one
seesion for it , which means legato start one session for one drive
regardles of it's size .

Moreover the same server was backup by Veritas backup exec much faster then
I am begining to believe that there sever design problem .

Backup speed goes to 40 mb /s to 256 kb/s .
regards,
- Rahul

>From: "Fields, David" <David.Fields AT ACS-INC DOT COM>
>Reply-To: Legato NetWorker discussion <NETWORKER AT LISTMAIL.TEMPLE DOT EDU>,
>         "Fields, David" <David.Fields AT ACS-INC DOT COM>
>To: NETWORKER AT LISTMAIL.TEMPLE DOT EDU
>Subject: Re: [Networker] How to improve performance
>Date: Fri, 22 Oct 2004 10:26:39 -0400
>
>As an experiment, try running a test backup by picking just one of your big
>filesystems, and then compare the time that it takes to run that backup
>against your normal backups.
>
>The reason I say do this is that you have to remember that unlike other
>backup software, Legato starts up multiple streams so that it can make the
>best use out of the tape drive that it can.  However, you have to keep some
>stuff in mind.  For instance, if you have a server, with 3 drives, defined
>into one Raid-5 container, and then you create multiple "drives" within
>that
>container, all three drive are using the same spindles (disks).  If these
>drives are big, and if you tell Legato to back up all 3 at the same time,
>you can actually slow down your backup because the disks are being
>overworked (disk thrashing).  I know you're using a SAN for this, but you
>still might want to look to see how the particular drives you are backing
>up
>are actually configured within the SAN, as you could very easily have
>multiple filesystems using the same physical drives.
>
>If you were doing this over a Gig Ethernet network, then I would say you're
>possibly running into some other bottlenecks, such as TCP window size or
>some other TCP setting, which can have a big effect on performance.
>Windows
>uses some very conservative network settings for Gig-E networks, and it can
>be very tricky trying to figure out the best configuration (I'm still
>working on it actually..if anyone has any info on this, I would appreciate
>your input).  For instance, if you try to ftp from a Windows box over a Gig
>network, I rarely get over 20MB (Mega Bytes)/sec out of the network, using
>default settings.  However, if I change the TCP window size, I can increase
>that to almost 40 or 50MB/sec.  At that point, the PCI bus is likely to
>start being a bottleneck.
>
>Dave
>
>-----Original Message-----
>From: Legato NetWorker discussion [mailto:NETWORKER AT LISTMAIL.TEMPLE DOT 
>EDU] On
>Behalf Of Rahul Parasnis
>Sent: Friday, October 22, 2004 10:09 AM
>To: NETWORKER AT LISTMAIL.TEMPLE DOT EDU
>Subject: Re: [Networker] How to improve performance
>
>I am using Networker 7.1.2 .
>
>
> >From: "Kuin, CNM" <CKuin AT WLZ DOT NL>
> >Reply-To: Legato NetWorker discussion <NETWORKER AT LISTMAIL.TEMPLE DOT EDU>,
> >         "Kuin, CNM" <CKuin AT WLZ DOT NL>
> >To: NETWORKER AT LISTMAIL.TEMPLE DOT EDU
> >Subject: Re: [Networker] How to improve performance
> >Date: Fri, 22 Oct 2004 14:48:54 +0200
> >
> > > -----Original Message-----
> > > I have recently installd new Networker Server on Solaris 9
> > > with L180 of 8
> > > Drives  .
> > > Enviromment .
> > >
> > > Server Solaris 9 , V440  2 CPU 4 GB
> > > Brocade Switch .
> > > L180 and Lto-2 drives
> > > Lp9000 HBA card on Solaris
> > > LP10000 card on Windows 2000 and windows 2003 cluster DSN's  .
> >Which version of Networker are you using?
> >7.x has some improvements regarding backing up a high number of files.
> >
> >With regards,
> >Cor.
> >
> >--
> >Note: To sign off this list, send a "signoff networker" command via email
> >to listserv AT listmail.temple DOT edu or visit the list's Web site at
> >http://listmail.temple.edu/archives/networker.html where you can
> >also view and post messages to the list. Questions regarding this list
> >should be sent to stan AT temple DOT edu
> >=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=
>
>_________________________________________________________________
>FREE pop-up blocking with the new MSN Toolbar - get it now!
>http://toolbar.msn.com/
>
>--
>Note: To sign off this list, send a "signoff networker" command via email
>to listserv AT listmail.temple DOT edu or visit the list's Web site at
>http://listmail.temple.edu/archives/networker.html where you can
>also view and post messages to the list. Questions regarding this list
>should be sent to stan AT temple DOT edu
>=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=
>
>--
>Note: To sign off this list, send a "signoff networker" command via email
>to listserv AT listmail.temple DOT edu or visit the list's Web site at
>http://listmail.temple.edu/archives/networker.html where you can
>also view and post messages to the list. Questions regarding this list
>should be sent to stan AT temple DOT edu
>=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=

_________________________________________________________________
Express yourself instantly with MSN Messenger! Download today it's FREE!
http://messenger.msn.com/

--
Note: To sign off this list, send a "signoff networker" command via email
to listserv AT listmail.temple DOT edu or visit the list's Web site at
http://listmail.temple.edu/archives/networker.html where you can
also view and post messages to the list. Questions regarding this list
should be sent to stan AT temple DOT edu
=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=

--
Note: To sign off this list, send a "signoff networker" command via email
to listserv AT listmail.temple DOT edu or visit the list's Web site at
http://listmail.temple.edu/archives/networker.html where you can
also view and post messages to the list. Questions regarding this list
should be sent to stan AT temple DOT edu
=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=