Networker

Re: [Networker] Best usage of tape drives

2005-10-27 16:53:02
Subject: Re: [Networker] Best usage of tape drives
From: "William M. Fennell" <William.Fennell AT CHANNING.HARVARD DOT EDU>
To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
Date: Thu, 27 Oct 2005 16:46:03 -0400
clients: Local Solaris and Linux hosts
data: mostly sas datasets, MS Office docs, email and Oracle databases.
available bandwidth: network bandwidth is gigabit.
your server:Sun Fire V120 the scsi bus: 80-MB/sec Ultra2 SCSI SE/LVD your tape drives: 3 Sony AIT-3 drives 700 SDX on the server and two more on storage node.

Backup load per night is about 300GB.

Bill

--------------------
William M. Fennell
UNIX Administrator
Channing Laboratory



Ernst Bokkelkamp wrote:

It depends, on a lot of factors.

You have to find the optimum setting that will work for you. Unfortunately
this depends on your clients, the type of data, the available bandwidth,
your server, the scsi bus, your tape drives, and the current moon phase.
There is no clear answer to your question.

Bye
Ernie

-----Original Message-----
From: Legato NetWorker discussion [mailto:NETWORKER AT LISTSERV.TEMPLE DOT EDU] 
On
Behalf Of William M. Fennell
Sent: Thursday, October 27, 2005 8:12 PM
To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
Subject: Re: [Networker] Best usage of tape drives


Reducing the target sessions and server parallelism will decrease the amount of time to
back up the data?


--------------------
William M. Fennell
UNIX Administrator
Channing Laboratory



Ernst Bokkelkamp wrote:

Your message needs a bit of rewording:

Our SERVER parallelism is at 40.
Three tape devices are configured for 4 TARGET sessions and the other two are configured for two TARGET sessions each. If we are at generating 16 streams using all of our tape devices, parallelism will
exceed the configured target session maximum and keep going up until 16
sessions are running. Server parallelism: Max number of sessions a server has been configured to be active at the same time. Please note that the real limit depends on your license and the number of storage nodes that have been configured (see the manual!). Target setting: This is the number of sessions per tape drive before the next drive should be selected. However this is not a limit!

Btw: You should target the minumum number of sessions to keep your drives spinning. It is bad practise to overload the drives with sessions, altough it doesn't make a difference it the only thing you do is backup and never intend to restore!. The reason for targetting the minimum number of sessions is easely explained: Just assume that you are backing up 16 clients with one saveset of 200 Gbyte each to 4 drives, each with target sessions set to 4 and server parallelism set to 64. Running all sessions at the same time will cause each drive to hit the target level. Now if each client can generate sufficient bandwith to produce half the bandwith of a tape drive then you will need double the time for each client to backup all the data. But if you reduce the server parallelism (8) and target sessions (2) so that only half the clients can be active then each client will only use half the time and you can do all the backups in the same time. Does this make sense ?

However, looking at restore, the target sessions can have very detrimental effect on the time needed. If you assume that the client is capable of taking up everything the tape drive can be delivered then the time to recover will depend on how fast the tape drive can move the tape (inches/second). Lets assume that the drive can move the tape, while reading, from beginning to end in 1 hour, then it will take one hour to read all the data on the tape. Now if one tape cartridge has a capacity to backup one client, then you will need n cartridges if n clients are backed up to the same tape drive with target sessions set to n and will run for n hours. This means that you will have to read n cartridges, or n x the tape length, using n hours to restore all the data on the client. This means that the restore of 1 (ONE) client will take n hours. This means the larger number of sessions to the drive then the longer it will take to read the tape, meaning the longer it will take to restore. For this reason it is wise to go for the minimum number of sessions to keep the drives spinning and to obtain an acceptable restore performance. This also means that if you don't care about restore then you can set the maximum number of sessions as you like.

I hope this helps a bit to understand what can drive a networker backup administrator into insanity.

Bye
Ernie


-----Original Message-----
From: Legato NetWorker discussion [mailto:NETWORKER AT LISTSERV.TEMPLE DOT EDU] On Behalf Of William M. Fennell
Sent: Wednesday, October 26, 2005 10:04 PM
To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
Subject: Re: [Networker] Best usage of tape drives


Our parallelism is at 40.
Three tape devices can take 4 target sessions and the other two can
handle two each.
Even if we are at 16 streams taking up all of our tape devices, parallelism will
trump the target session limit and keep going up until 40 is reached. So I
should expect to see 40 streams saving right?
--------------------
William M. Fennell
UNIX Administrator
Channing Laboratory



Albert Eddie Contractor AFRPA/ESS wrote:



Archive Performance is kind of an itch to me that I am always trying to
scratch.

Speaking to Legato last week we discussed a strategy where I would list various specific directory trees to backup. Then you turn up Parallelism allowing more data streams and voila this results in multiple data streams per drive.

Of course weighing in Backup data drive bandwidth ability, controller
bandwidth, LAN bandwidth and then the bandwidth ability of your NetWorker components as well.

(Does anyone else feel like they are tweaking VWs to race in the sand?)

The parallelism setup for the server (default 4 in NW7.1?) means that
if you don't have 5 data streams coming in, Networker is only going to use ONE drive and leave the other drive for tape operations.

If you lower your parallelism to 2 you should see the second drive join
in when the third data stream is identified. I hope this helps.

Semper Fidelis, /ALE

Eddie Albert, Network Serf
Air Force Real Property Agency
Executive Services/Computer Systems
Desk (703) 696 - 5509 - Hip (703) 517-3855
Eddie.Albert AT afrpa.Pentagon.af DOT mil



-----Original Message-----
From: Legato NetWorker discussion [mailto:NETWORKER AT LISTSERV.TEMPLE DOT EDU] On Behalf Of William M. Fennell
Sent: Tuesday, October 25, 2005 4:41 PM
To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
Subject: [Networker] Best usage of tape drives

Hi folks,

I have a Server with three tape drives hooked up to it.
I have a dedicated storage node with two tape drives hooked up to it.
All drives are in a qualstar TLS-4000 and the tape drives are Sony 700CDX AIT-3. We're running Solaris 8 on the Networker server and 9 on the storage node.
We're at Networker 7.1.

My storage node can of course only write local data to tape. My issue is that it only uses one drive! So it is trying to write 200+GB nightly from three file systems on this host to one tape. How can I make it use both tape drives? Currently it is taking about 8 hours to backup about 200GB nightly to one drive.
I would like to improve on that performance.

My two ideas are:
1: reduce the number of target streams on each drive to two and then have one drive take two streams and the other take one stream.
2: Maybe the drives need to be shared with each other?

Many thanks,

Bill

To sign off this list, send email to listserv AT listserv.temple DOT edu and type "signoff networker" in the body of the email. Please write to networker-request AT listserv.temple DOT edu if you have any problems wit this list. You can access the archives at http://listserv.temple.edu/archives/networker.html or via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER

To sign off this list, send email to listserv AT listserv.temple DOT edu and type "signoff networker" in the body of the email. Please write to networker-request AT listserv.temple DOT edu if you have any problems wit this list. You can access the archives at http://listserv.temple.edu/archives/networker.html or via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER

To sign off this list, send email to listserv AT listserv.temple DOT edu and type "signoff networker" in the body of the email. Please write to networker-request AT listserv.temple DOT edu if you have any problems wit this list. You can access the archives at http://listserv.temple.edu/archives/networker.html or via RSS at
http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER



To sign off this list, send email to listserv AT listserv.temple DOT edu and 
type
"signoff networker" in the body of the email. Please write to
networker-request AT listserv.temple DOT edu if you have any problems wit this
list. You can access the archives at
http://listserv.temple.edu/archives/networker.html or via RSS at
http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER



To sign off this list, send email to listserv AT listserv.temple DOT edu and type 
"signoff networker" in the
body of the email. Please write to networker-request AT listserv.temple DOT edu 
if you have any problems
wit this list. You can access the archives at 
http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER

<Prev in Thread] Current Thread [Next in Thread>