Bill,
I am told that when performance drops like that. That we can blame the
file make-up of the data. A bunch of small files do not transfer as well
as one large file.
Semper Fidelis, /ALE
//SIGNED//
Eddie Albert, Contract Network Engineer
Air Force Real Property Agency
Chief of Information Office
Division of Information Technology
OFC (703) 696 - 5509
CEL (703) 517 - 3855
> -----Original Message-----
> From: Legato NetWorker discussion
> [mailto:NETWORKER AT LISTSERV.TEMPLE DOT EDU] On Behalf Of William M. Fennell
> Sent: Friday, October 28, 2005 4:23 PM
> To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
> Subject: Re: [Networker] Best usage of tape drives
>
> Yes, I should be getting better performance out of this
> system but each tape drive is backing up at no more than
> 5,000k per second.
> So tonight I am checking all of the cables to make sure I do
> not have an SE cable slipped in there somewhere.
> What is interesting is that I did a test backup of my legato
> server and it was going strong at 10+ MS/S then the bottom
> fell out and it is back to 1,000-2000 K/S.
>
>
> --------------------
> William M. Fennell
> UNIX Administrator
> Channing Laboratory
>
>
>
> Ernst Bokkelkamp wrote:
>
> >Hallo Bill,
> >
> >Only you can answer the question. This is (a part of) the art of
> >administrating a networker backup application.
> >But just looking at the bare details, you should be able to
> backup all
> >data using a single stream on a single LTO-1 tape drive
> within 6 hours easely.
> >About 12,5 Mbyte/sec gives about 50GByte/Hour (if the
> clients can deliver).
> >This very much depends on your hardware and software.
> >300GB isn't really a lot with the equipment you have
> available, you are
> >a lot better off compared to the configuration I used to
> maintain with
> >DLT7 drives backing up 1.5TB / weekend using a 100TX network with a
> >large number of lame clients such as 133Mhz Pentiums servers. With
> >todays equipment it is not unusual to see backup performance of
> >200Gbyte/hour from a single client using multiple drives. A
> few years
> >(2?) ago I did some testing using a NetApp F960 with 4 LTO drives
> >attached and this system was able to push 240GByte/hour
> using 4 simultaneous NDMP backups.
> >
> >Bye
> >Ernie
> >
> >
> >-----Original Message-----
> >From: Legato NetWorker discussion
> >[mailto:NETWORKER AT LISTSERV.TEMPLE DOT EDU] On Behalf Of William
> M. Fennell
> >Sent: Thursday, October 27, 2005 10:46 PM
> >To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
> >Subject: Re: [Networker] Best usage of tape drives
> >
> >
> >clients: Local Solaris and Linux hosts
> >data: mostly sas datasets, MS Office docs, email and Oracle
> databases.
> >available bandwidth: network bandwidth is gigabit. your
> server:Sun Fire
> >V120
> >
> >the scsi bus: 80-MB/sec Ultra2 SCSI SE/LVD your tape drives: 3 Sony
> >AIT-3 drives 700 SDX
> > on the server and two more on storage node.
> >
> >Backup load per night is about 300GB.
> >
> >Bill
> >
> >--------------------
> >William M. Fennell
> >UNIX Administrator
> >Channing Laboratory
> >
> >
> >
> >Ernst Bokkelkamp wrote:
> >
> >
> >
> >>It depends, on a lot of factors.
> >>
> >>You have to find the optimum setting that will work for you.
> >>Unfortunately this depends on your clients, the type of data, the
> >>available bandwidth, your server, the scsi bus, your tape
> drives, and
> >>the current moon phase. There is no clear answer to your question.
> >>
> >>Bye
> >>Ernie
> >>
> >>-----Original Message-----
> >>From: Legato NetWorker discussion
> >>[mailto:NETWORKER AT LISTSERV.TEMPLE DOT EDU] On Behalf Of William
> M. Fennell
> >>Sent: Thursday, October 27, 2005 8:12 PM
> >>To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
> >>Subject: Re: [Networker] Best usage of tape drives
> >>
> >>
> >>Reducing the target sessions and server parallelism will
> decrease the
> >>amount of time to back up the data?
> >>
> >>
> >>--------------------
> >>William M. Fennell
> >>UNIX Administrator
> >>Channing Laboratory
> >>
> >>
> >>
> >>Ernst Bokkelkamp wrote:
> >>
> >>
> >>
> >>
> >>
> >>>Your message needs a bit of rewording:
> >>>
> >>>Our SERVER parallelism is at 40.
> >>>Three tape devices are configured for 4 TARGET sessions
> and the other
> >>>two are configured for two TARGET sessions each. If we are at
> >>>generating 16 streams using all of our tape devices,
> parallelism will
> >>>exceed the configured target session maximum and keep
> going up until
> >>>16 sessions are running.
> >>>
> >>>Server parallelism: Max number of sessions a server has been
> >>>configured to be active at the same time. Please note that
> the real
> >>>limit depends on your license and the number of storage nodes that
> >>>have been configured (see the manual!). Target setting:
> This is the
> >>>number of sessions per tape drive before the next drive should be
> >>>selected.
> >>>However this is not a limit!
> >>>
> >>>Btw: You should target the minumum number of sessions to keep your
> >>>drives spinning. It is bad practise to overload the drives with
> >>>sessions, altough it doesn't make a difference it the only
> thing you
> >>>do is backup and never intend to restore!. The reason for
> targetting
> >>>the minimum number of sessions is easely explained: Just
> assume that
> >>>you are backing up 16 clients with one saveset of 200
> Gbyte each to 4
> >>>drives, each with target sessions set to 4 and server
> parallelism set
> >>>to 64. Running all sessions at the same time will cause
> each drive
> >>>to hit the target level. Now if each client can generate
> sufficient
> >>>bandwith to produce half the bandwith of a tape drive then
> you will
> >>>need double the time for each client to backup all the
> data. But if
> >>>you reduce the server parallelism (8) and target sessions
> (2) so that
> >>>only half the clients can be active then each client will only use
> >>>half the time and you can do all the backups in the same
> time. Does
> >>>this make sense ?
> >>>
> >>>However, looking at restore, the target sessions can have very
> >>>detrimental effect on the time needed. If you assume that
> the client
> >>>is capable of taking up everything the tape drive can be delivered
> >>>then the time to recover will depend on how fast the tape
> drive can
> >>>move the tape (inches/second). Lets assume that the drive can move
> >>>the tape, while reading, from beginning to end in 1 hour, then it
> >>>will take one hour to read all the data on the tape. Now
> if one tape
> >>>cartridge has a capacity to backup one client, then you
> will need n
> >>>cartridges if n clients are backed up to the same tape drive with
> >>>target sessions set to n and will run for n hours. This means that
> >>>you will have to read n cartridges, or n x the tape
> length, using n
> >>>hours to restore all the data on the client. This means that the
> >>>restore of 1 (ONE) client will take n hours. This means the larger
> >>>number of sessions to the drive then the longer it will
> take to read
> >>>the tape, meaning the longer it will take to restore. For
> this reason
> >>>it is wise to go for the minimum number of sessions to keep the
> >>>drives spinning and to obtain an acceptable restore
> performance. This
> >>>also means that if you don't care about restore then you
> can set the
> >>>maximum number of sessions as you like.
> >>>
> >>>I hope this helps a bit to understand what can drive a networker
> >>>backup administrator into insanity.
> >>>
> >>>Bye
> >>>Ernie
> >>>
> >>>
> >>>-----Original Message-----
> >>>From: Legato NetWorker discussion
> >>>[mailto:NETWORKER AT LISTSERV.TEMPLE DOT EDU] On Behalf Of William M.
> >>>Fennell
> >>>Sent: Wednesday, October 26, 2005 10:04 PM
> >>>To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
> >>>Subject: Re: [Networker] Best usage of tape drives
> >>>
> >>>
> >>>Our parallelism is at 40.
> >>>Three tape devices can take 4 target sessions and the
> other two can
> >>>handle two each. Even if we are at 16 streams taking up all of our
> >>>tape devices, parallelism will trump the target session limit and
> >>>keep going up until 40 is reached. So I should expect to see 40
> >>>streams saving right?
> >>>
> >>>--------------------
> >>>William M. Fennell
> >>>UNIX Administrator
> >>>Channing Laboratory
> >>>
> >>>
> >>>
> >>>Albert Eddie Contractor AFRPA/ESS wrote:
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>>Archive Performance is kind of an itch to me that I am
> always trying
> >>>>to scratch.
> >>>>
> >>>>Speaking to Legato last week we discussed a strategy
> where I would
> >>>>list various specific directory trees to backup. Then you turn up
> >>>>Parallelism allowing more data streams and voila this results in
> >>>>multiple data streams per drive.
> >>>>
> >>>>Of course weighing in Backup data drive bandwidth ability,
> >>>>controller bandwidth, LAN bandwidth and then the
> bandwidth ability
> >>>>of your NetWorker components as well.
> >>>>
> >>>>(Does anyone else feel like they are tweaking VWs to race in the
> >>>>sand?)
> >>>>
> >>>>The parallelism setup for the server (default 4 in NW7.1?) means
> >>>>that if you don't have 5 data streams coming in,
> Networker is only
> >>>>going to use ONE drive and leave the other drive for tape
> operations.
> >>>>
> >>>>If you lower your parallelism to 2 you should see the
> second drive
> >>>>join in when the third data stream is identified. I hope
> this helps.
> >>>>
> >>>>Semper Fidelis, /ALE
> >>>>
> >>>>Eddie Albert, Network Serf
> >>>>Air Force Real Property Agency
> >>>>Executive Services/Computer Systems
> >>>>Desk (703) 696 - 5509 - Hip (703) 517-3855
> >>>>Eddie.Albert AT afrpa.Pentagon.af DOT mil
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>>-----Original Message-----
> >>>>>From: Legato NetWorker discussion
> >>>>>[mailto:NETWORKER AT LISTSERV.TEMPLE DOT EDU] On Behalf Of William M.
> >>>>>Fennell
> >>>>>Sent: Tuesday, October 25, 2005 4:41 PM
> >>>>>To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
> >>>>>Subject: [Networker] Best usage of tape drives
> >>>>>
> >>>>>Hi folks,
> >>>>>
> >>>>>I have a Server with three tape drives hooked up to it.
> >>>>>I have a dedicated storage node with two tape drives
> hooked up to
> >>>>>it. All drives are in a qualstar TLS-4000 and the tape
> drives are
> >>>>>Sony 700CDX AIT-3. We're running Solaris 8 on the
> Networker server
> >>>>>and 9 on the storage node. We're at Networker 7.1.
> >>>>>
> >>>>>My storage node can of course only write local data to tape. My
> >>>>>issue is that it only uses one drive! So it is trying to write
> >>>>>200+GB nightly from three file systems on this host to one tape.
> >>>>>How can I make it use both tape drives? Currently it is taking
> >>>>>about 8 hours to backup about 200GB nightly to one drive.
> >>>>>I would like to improve on that performance.
> >>>>>
> >>>>>My two ideas are:
> >>>>>1: reduce the number of target streams on each drive to two and
> >>>>>then have one drive take two streams and the other take
> one stream.
> >>>>>2: Maybe the drives need to be shared with each other?
> >>>>>
> >>>>>Many thanks,
> >>>>>
> >>>>>Bill
> >>>>>
> >>>>>To sign off this list, send email to
> listserv AT listserv.temple DOT edu
> >>>>>and type "signoff networker" in the body of the email.
> Please write
> >>>>>to networker-request AT listserv.temple DOT edu if you have any
> problems
> >>>>>wit this list. You can access the archives at
> >>>>>http://listserv.temple.edu/archives/networker.html or via RSS at
> >>>>>http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>To sign off this list, send email to
> listserv AT listserv.temple DOT edu and
> >>>type "signoff networker" in the body of the email. Please write to
> >>>networker-request AT listserv.temple DOT edu if you have any problems wit
> >>>this list. You can access the archives at
> >>>http://listserv.temple.edu/archives/networker.html or via RSS at
> >>>http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER
> >>>
> >>>To sign off this list, send email to
> listserv AT listserv.temple DOT edu and
> >>>type "signoff networker" in the body of the email. Please write to
> >>>networker-request AT listserv.temple DOT edu if you have any problems wit
> >>>this list. You can access the archives at
> >>>http://listserv.temple.edu/archives/networker.html or via RSS at
> >>>
> >>>
> >>>
> >>>
> >>http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER
> >>
> >>
> >>
> >>
> >>>
> >>>
> >>>
> >>>
> >>To sign off this list, send email to
> listserv AT listserv.temple DOT edu and
> >>type "signoff networker" in the body of the email. Please write to
> >>networker-request AT listserv.temple DOT edu if you have any problems wit
> >>this list. You can access the archives at
> >>http://listserv.temple.edu/archives/networker.html or via RSS at
> >>http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER
> >>
> >>
> >>
> >>
> >>
> >>
> >
> >To sign off this list, send email to
> listserv AT listserv.temple DOT edu and
> >type "signoff networker" in the body of the email. Please write to
> >networker-request AT listserv.temple DOT edu if you have any
> problems wit this
> >list. You can access the archives at
> >http://listserv.temple.edu/archives/networker.html or via RSS at
> >http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER
> >
> >
> >
> >
> >
>
> To sign off this list, send email to
> listserv AT listserv.temple DOT edu and type "signoff networker" in
> the body of the email. Please write to
> networker-request AT listserv.temple DOT edu if you have any
> problems wit this list. You can access the archives at
> http://listserv.temple.edu/archives/networker.html or via RSS
> at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER
>
To sign off this list, send email to listserv AT listserv.temple DOT edu and
type "signoff networker" in the
body of the email. Please write to networker-request AT listserv.temple DOT edu
if you have any problems
wit this list. You can access the archives at
http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER
|
