Veritas-bu

[Veritas-bu] Not fully utilizing tapes

2004-06-08 05:01:47
Subject: [Veritas-bu] Not fully utilizing tapes
From: william.d.brown AT gsk DOT com (william.d.brown AT gsk DOT com)
Date: Tue, 8 Jun 2004 10:01:47 +0100
Don't forget almost all tape writing software is writing some kind of 
files to the tape.  Between each file there are gaps, markers etc.  So 
there is always a % overhead.   If you write ANSI tapes 80-block tapes 
(i.e. 1 file on disk = 1 file on tape) like with a 'copy' command the 
overhead is very large, especially if the files are small.   You get a gap 
between blocks, and another between files.

Most backup software writes some kind of 'save-set' which is a wrapper for 
the actual files.  I think for NetBackup the storage unit maximum fragment 
size is the number to watch that sets the size of the files it writes. 
This is typically set to 2GB (2000MB in the GUI), as many o/s cannot 
handle disk files bigger than this, and if you wish to use backup-to-disk 
that is a limit.   There is also the tape buffer size (SIZE_DATA_BUFFERS), 
which determines the block size written to tape.    For DLT that is often 
64K, for LTO often 256K.

You can see that for a 100G tape (ignoring compression by the tape 
hardware) you get 50 files @2GB, and 1.6M blocks @64k.  So there can be an 
overhead, 10-15% may not be bad.  I don't know how big the gaps are.   The 
tape capacity will be calculated as if there was one continuous bit stream 
on the tape.

So - use 2000MB fragments in the storage unit, and as big a 
SIZE_DATA_BUFFERS as your hardware will support - but be careful with the 
latter as it affects memory usage, and you are most likely to tune that 
for performance, not tape capacity.

William D L Brown


veritas-bu-admin AT mailman.eng.auburn DOT edu wrote on 07/06/2004 17:22:26:

> You got 98G on a DLT tape.  That sounds very full to me.  They're 
> only 40G Native.  That's better than 2:1.  What makes you think it's
> not "fully utilized"?
> 
> 
> -M
> -----Original Message-----
> From: Hubler, Daniel [mailto:daniel.hubler AT cobalt-corp DOT com]
> Sent: Monday, June 07, 2004 10:17 AM
> To: Donaldson, Mark - Broomfield, CO; veritas-bu AT mailman.eng.auburn DOT edu
> Subject: RE: [Veritas-bu] Not fully utilizing tapes

> Here is some output from available_media (I deleted some stuff).
> The tapes in question are:  BWU010  &  BWU011 (about half-way down the 
list):
> 
> ====================================
> /home/xxxxxx$ available_media
> media   media   robot   robot   robot   side/   ret    size     status
>  ID     type    type      #     slot    face    level  KBytes
> 
----------------------------------------------------------------------------
> test-STARS-backup-1 pool
> 
> BWU004  DLT      NONE     -       -      -       1     15461204 
ACTIVE/MPX
> BWU047  DLT      NONE     -       -      -       1     60600121 
ACTIVE/MPX
> BWU048  DLT      NONE     -       -      -       1     42365962 
ACTIVE/MPX
> BWU051  DLT      NONE     -       -      -       1     409520 ACTIVE/MPX
> BWU036  DLT      NONE     -       -      -       -     - AVAILABLE
> BWU037  DLT      NONE     -       -      -       -     - AVAILABLE
> BWU038  DLT      NONE     -       -      -       -     - AVAILABLE
> BWU039  DLT      NONE     -       -      -       -     - AVAILABLE
> BWU040  DLT      NONE     -       -      -       -     - AVAILABLE
> BWU052  DLT      TLD      0       15     -       -     - AVAILABLE
> BWU002  DLT      NONE     -       -      -       1     102795955 
FULL/MPX
> BWU003  DLT      NONE     -       -      -       1     102507391 
FULL/MPX
> BWU008  DLT      NONE     -       -      -       1     100847104 
FULL/MPX
> BWU009  DLT      NONE     -       -      -       1     113999174 
FULL/MPX
> BWU010  DLT      NONE     -       -      -       1     98881611 FULL/MPX
> BWU011  DLT      NONE     -       -      -       1     94186203 FULL/MPX
> BWU012  DLT      NONE     -       -      -       1     125489547 
FULL/MPX
> BWU013  DLT      NONE     -       -      -       1     128797560 
FULL/MPX
> BWU021  DLT      NONE     -       -      -       1     126117420 
FULL/MPX
> BWU022  DLT      NONE     -       -      -       1     123570554 
FULL/MPX
> BWU033  DLT      NONE     -       -      -       1     121749425 
FULL/MPX
> BWU041  DLT      NONE     -       -      -       1     125116945 
FULL/MPX
> BWU042  DLT      NONE     -       -      -       1     121046181 
FULL/MPX
> BWU043  DLT      NONE     -       -      -       1     110269596 
FULL/MPX
> BWU044  DLT      NONE     -       -      -       1     96771748 FULL/MPX
> BWU045  DLT      NONE     -       -      -       1     115321358 
FULL/MPX
> BWU046  DLT      NONE     -       -      -       1     105210453 
FULL/MPX
> BWU053  DLT      NONE     -       -      -       1     126886727 
FULL/MPX
> BWU054  DLT      NONE     -       -      -       1     122242787 
FULL/MPX
> BWU055  DLT      NONE     -       -      -       1     116116379 
FULL/MPX
> BWU056  DLT      NONE     -       -      -       1     107045233 
FULL/MPX
> 
> =================================================================
> =================================================================
> -----Original Message-----
> From: Mark.Donaldson AT cexp DOT com [mailto:Mark.Donaldson AT cexp DOT com] 
> Sent: Monday, June 07, 2004 10:57 AM
> To: daniel.hubler AT cobalt-corp DOT com; veritas-bu AT mailman.eng.auburn 
> DOT edu
> Subject: RE: [Veritas-bu] Not fully utilizing tapes

> How are you determining this?  Can you snip some output here?
> -M
> -----Original Message-----
> From: veritas-bu-admin AT mailman.eng.auburn DOT edu [mailto:veritas-bu-
> admin AT mailman.eng.auburn DOT edu]On Behalf Of Hubler, Daniel
> Sent: Monday, June 07, 2004 9:50 AM
> To: veritas-bu AT mailman.eng.auburn DOT edu
> Subject: [Veritas-bu] Not fully utilizing tapes

> NBU  v4.5  on Solaris 8. 
> What would cause NBU to stop writing to a tape before the tape is full? 
> One of our large backup jobs consumes about 12 tapes. 
> 2 that participated in the middle of the job flow were left 10-25% 
empty. 
> 

> Daniel Hubler 
> Database Administrator 
> TrustSolutions, LLC 
> Milwaukee,  WI 
> daniel.hubler AT cobalt-corp DOT com 
> phone:  414-226-5391 
> pager:  877-580-0731 
> pager email:  8775800731 AT my2way DOT com 
> page via internet:  www.my2way.com 
> 
> The information contained in this electronic message may be 
> confidential, is intended only for the use of the recipient(s) name 
> above, and may be legally privileged.  If you are not the intended 
> recipient of this communication, you are hereby notified that any 
> use, distribution, or copying of this message is strictly 
> prohibited.  If you have received this communication in error, 
> please immediately return it to the sender and delete the original 
> message and any copy of it from your computer system.  If you have 
> any questions concerning this message, please contact the sender.  Thank 
you.


<Prev in Thread] Current Thread [Next in Thread>