Re: [Networker] Best usage of tape drives

Hallo Bill,

Only you can answer the question. This is (a part of) the art of
administrating a networker backup application.
But just looking at the bare details, you should be able to backup all data
using a single stream on a single LTO-1 tape drive within 6 hours easely.
About 12,5 Mbyte/sec gives about 50GByte/Hour (if the clients can deliver).
This very much depends on your hardware and software.
300GB isn't really a lot with the equipment you have available, you are a
lot better off compared to the configuration I used to maintain with DLT7
drives backing up 1.5TB / weekend using a 100TX network with a large number
of lame clients such as 133Mhz Pentiums servers. With todays equipment it is
not unusual to see backup performance of 200Gbyte/hour from a single client
using multiple drives. A few years (2?) ago I did some testing using a
NetApp F960 with 4 LTO drives attached and this system was able to push
240GByte/hour using 4 simultaneous NDMP backups.

Bye
Ernie


-----Original Message-----
From: Legato NetWorker discussion [mailto:NETWORKER AT LISTSERV.TEMPLE DOT EDU] 
On
Behalf Of William M. Fennell
Sent: Thursday, October 27, 2005 10:46 PM
To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
Subject: Re: [Networker] Best usage of tape drives


clients: Local Solaris and Linux hosts
data: mostly sas datasets, MS Office docs, email and Oracle databases.
available bandwidth: network bandwidth is gigabit. your server:Sun Fire V120

the scsi bus: 80-MB/sec Ultra2 SCSI SE/LVD
your tape drives: 3 Sony AIT-3 drives 700 SDX 
                on the server and two more on storage node.

Backup load per night is about 300GB.

Bill

--------------------
William M. Fennell
UNIX Administrator
Channing Laboratory



Ernst Bokkelkamp wrote:

>It depends, on a lot of factors.
>
>You have to find the optimum setting that will work for you. 
>Unfortunately this depends on your clients, the type of data, the 
>available bandwidth, your server, the scsi bus, your tape drives, and 
>the current moon phase. There is no clear answer to your question.
>
>Bye
>Ernie
>
>-----Original Message-----
>From: Legato NetWorker discussion 
>[mailto:NETWORKER AT LISTSERV.TEMPLE DOT EDU] On Behalf Of William M. Fennell
>Sent: Thursday, October 27, 2005 8:12 PM
>To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
>Subject: Re: [Networker] Best usage of tape drives
>
>
>Reducing the target sessions and server parallelism will decrease the
>amount of time to
>back up the data?
>
>
>--------------------
>William M. Fennell
>UNIX Administrator
>Channing Laboratory
>
>
>
>Ernst Bokkelkamp wrote:
>
>  
>
>>Your message needs a bit of rewording:
>>
>>Our SERVER parallelism is at 40.
>>Three tape devices are configured for 4 TARGET sessions and the other
>>two are configured for two TARGET sessions each. If we are at 
>>generating 16 streams using all of our tape devices, parallelism will
>>exceed the configured target session maximum and keep going up until 16
>>sessions are running. 
>>
>>Server parallelism: Max number of sessions a server has been 
>>configured
>>to be active at the same time. Please note that the real limit depends 
>>on your license and the number of storage nodes that have been 
>>configured (see the manual!). Target setting: This is the number of 
>>sessions per tape drive before the next drive should be selected. 
>>However this is not a limit!
>>
>>Btw: You should target the minumum number of sessions to keep your
>>drives spinning. It is bad practise to overload the drives with 
>>sessions, altough it doesn't make a difference it the only thing you do 
>>is backup and never intend to restore!. The reason for targetting the 
>>minimum number of sessions is easely explained: Just assume that you 
>>are backing up 16 clients with one saveset of 200 Gbyte each to 4 
>>drives, each with target sessions set to 4 and server parallelism set 
>>to 64.  Running all sessions at the same time will cause each drive to 
>>hit the target level. Now if each client can generate sufficient 
>>bandwith to produce half the bandwith of a tape drive then you will 
>>need double the time for each client to backup all the data. But if you 
>>reduce the server parallelism (8) and target sessions (2) so that only 
>>half the clients can be active then each client will only use half the 
>>time and you can do all the backups in the same time. Does this make 
>>sense ?
>>
>>However, looking at restore, the target sessions can have very
>>detrimental effect on the time needed. If you assume that the client is 
>>capable of taking up everything the tape drive can be delivered then 
>>the time to recover will depend on how fast the tape drive can move the 
>>tape (inches/second). Lets assume that the drive can move the tape, 
>>while reading, from beginning to end in 1 hour, then it will take one 
>>hour to read all the data on the tape. Now if one tape cartridge has a 
>>capacity to backup one client, then you will need n cartridges if n 
>>clients are backed up to the same tape drive with target sessions set 
>>to n and will run for n hours. This means that you will have to read n 
>>cartridges, or n x the tape length, using n hours to restore all the 
>>data on the client. This means that the restore of 1 (ONE) client will 
>>take n hours. This means the larger number of sessions to the drive 
>>then the longer it will take to read the tape, meaning the longer it 
>>will take to restore. For this reason it is wise to go for the minimum 
>>number of sessions to keep the drives spinning and to obtain an 
>>acceptable restore performance. This also means that if you don't care 
>>about restore then you can set the maximum number of sessions as you 
>>like.
>>
>>I hope this helps a bit to understand what can drive a networker 
>>backup
>>administrator into insanity.
>>
>>Bye
>>Ernie
>>
>>
>>-----Original Message-----
>>From: Legato NetWorker discussion
>>[mailto:NETWORKER AT LISTSERV.TEMPLE DOT EDU] On Behalf Of William M. Fennell
>>Sent: Wednesday, October 26, 2005 10:04 PM
>>To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
>>Subject: Re: [Networker] Best usage of tape drives
>>
>>
>>Our parallelism is at 40.
>>Three tape devices can take 4 target sessions and the other two can 
>>handle two each. Even if we are at 16 streams taking up all of our 
>>tape devices, parallelism will
>>trump the target session limit and keep going up until 40 is reached. So I
>>should expect to see 40 streams saving right? 
>>
>>--------------------
>>William M. Fennell
>>UNIX Administrator
>>Channing Laboratory
>>
>>
>>
>>Albert Eddie Contractor AFRPA/ESS wrote:
>>
>> 
>>
>>    
>>
>>>Archive Performance is kind of an itch to me that I am always trying
>>>to
>>>scratch.
>>>
>>>Speaking to Legato last week we discussed a strategy where I would
>>>list
>>>various specific directory trees to backup. Then you turn up 
>>>Parallelism allowing more data streams and voila this results in 
>>>multiple data streams per drive.
>>>
>>>Of course weighing in Backup data drive bandwidth ability, controller 
>>>bandwidth, LAN bandwidth and then the bandwidth ability of your 
>>>NetWorker components as well.
>>>
>>>(Does anyone else feel like they are tweaking VWs to race in the
>>>sand?)
>>>
>>>The parallelism setup for the server (default 4 in NW7.1?) means that 
>>>if you don't have 5 data streams coming in, Networker is only going 
>>>to use ONE drive and leave the other drive for tape operations.
>>>
>>>If you lower your parallelism to 2 you should see the second drive
>>>join
>>>in when the third data stream is identified. I hope this helps.
>>>
>>>Semper Fidelis, /ALE
>>>
>>>Eddie Albert, Network Serf
>>>Air Force Real Property Agency
>>>Executive Services/Computer Systems
>>>Desk (703) 696 - 5509 - Hip (703) 517-3855 
>>>Eddie.Albert AT afrpa.Pentagon.af DOT mil
>>>
>>>
>>>
>>>   
>>>
>>>      
>>>
>>>>-----Original Message-----
>>>>From: Legato NetWorker discussion
>>>>[mailto:NETWORKER AT LISTSERV.TEMPLE DOT EDU] On Behalf Of William M. 
>>>>Fennell
>>>>Sent: Tuesday, October 25, 2005 4:41 PM
>>>>To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
>>>>Subject: [Networker] Best usage of tape drives
>>>>
>>>>Hi folks,
>>>>
>>>>I have a Server with three tape drives hooked up to it.
>>>>I have a dedicated storage node with two tape drives hooked up to 
>>>>it. All drives are in a qualstar TLS-4000 and the tape drives are 
>>>>Sony 700CDX AIT-3. We're running Solaris 8 on the Networker server 
>>>>and 9 on the storage node. We're at Networker 7.1.
>>>>
>>>>My storage node can of course only write local data to tape. My 
>>>>issue
>>>>is that it only uses one drive! So it is trying to write 200+GB 
>>>>nightly from three file systems on this host to one tape. How can I 
>>>>make it use both tape drives? Currently it is taking about 8 hours to 
>>>>backup about 200GB nightly to one drive.
>>>>I would like to improve on that performance.
>>>>
>>>>My two ideas are:
>>>>1: reduce the number of target streams on each drive to two and then
>>>>have one drive take two streams and the other take one stream.
>>>>2: Maybe the drives need to be shared with each other?
>>>>
>>>>Many thanks,
>>>>
>>>>Bill
>>>>
>>>>To sign off this list, send email to listserv AT listserv.temple DOT edu 
>>>>and
>>>>type "signoff networker" in the body of the email. Please write to
>>>>networker-request AT listserv.temple DOT edu if you have any 
>>>>problems wit this list. You can access the archives at 
>>>>http://listserv.temple.edu/archives/networker.html or via RSS 
>>>>at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER
>>>>
>>>>  
>>>>
>>>>     
>>>>
>>>>        
>>>>
>>>   
>>>
>>>      
>>>
>>To sign off this list, send email to listserv AT listserv.temple DOT edu and
>>type "signoff networker" in the body of the email. Please write to 
>>networker-request AT listserv.temple DOT edu if you have any problems wit 
>>this 
>>list. You can access the archives at 
>>http://listserv.temple.edu/archives/networker.html or via RSS at 
>>http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER
>>
>>To sign off this list, send email to listserv AT listserv.temple DOT edu and
>>type "signoff networker" in the body of the email. Please write to 
>>networker-request AT listserv.temple DOT edu if you have any problems wit 
>>this 
>>list. You can access the archives at 
>>http://listserv.temple.edu/archives/networker.html or via RSS at
>>    
>>
>http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER
>  
>
>> 
>>
>>    
>>
>
>To sign off this list, send email to listserv AT listserv.temple DOT edu and 
>type "signoff networker" in the body of the email. Please write to 
>networker-request AT listserv.temple DOT edu if you have any problems wit this 
>list. You can access the archives at 
>http://listserv.temple.edu/archives/networker.html or via RSS at 
>http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER
>
>
>  
>

To sign off this list, send email to listserv AT listserv.temple DOT edu and 
type
"signoff networker" in the body of the email. Please write to
networker-request AT listserv.temple DOT edu if you have any problems wit this
list. You can access the archives at
http://listserv.temple.edu/archives/networker.html or via RSS at
http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER

To sign off this list, send email to listserv AT listserv.temple DOT edu and 
type "signoff networker" in the
body of the email. Please write to networker-request AT listserv.temple DOT edu 
if you have any problems
wit this list. You can access the archives at 
http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER