Re: [Veritas-bu] Script to label expired tapes in a VTL

The problem is not de-dupe; the problem is thin provisioning and 
oversubscription.  There are a lot of VTLs that allow both (with or w/o 
de-dupe), and if you define more tapes than you actually have disk, you will 
have this problem.  I'll concede that oversubscription is a natural state in a 
de-dupe VTL, as you define probably 20 times more tapes than you have "real" 
storage for.

The problem I see you're describing is this:
1. You define more tapes than you really have capacity for (again, this is 
normal in the de-dupe world)
2. You have a bunch that are partially full
3. You have a bunch that are in scratch, but have not been relabeled.
4. You may even have some that are brand new tapes that haven't been used at 
all.
5. You're out of real free space
6. Since NBU prefers to append rather than use a new tape, it will grab the #2 
tapes before the #3 tapes.  Had it grabbed the #3 tapes, it would have cleared 
the space they're taking up and you wouldn't have the problem, but that's not 
how NBU (or any other backup s/w) works.

Here's a thought.  Isn't there a way to tell NBU to mark tapes 
full/frozen/something after a backup?  If you did that, additional backups 
wouldn't append to those tapes, and you wouldn't have to worry about labeling.  
Would that work?

Of course, this whole labeling thing really isn't that big of a deal.  A simple 
shell script could handle it easy enough.

---
W. Curtis Preston
Backup Blog @ www.backupcentral.com
VP Data Protection, GlassHouse Technologies 

-----Original Message-----
From: Paul Keating [mailto:pkeating AT bank-banque-canada DOT ca] 
Sent: Monday, September 24, 2007 9:34 AM
To: Curtis Preston; Clem Kruger; VERITAS-BU AT mailman.eng.auburn DOT edu
Subject: RE: [Veritas-bu] Script to label expired tapes in a VTL

Not entirely true, Curtis.

When your virutal tape expires, the VTL has no way of knowing this
untill the tape is written to again.
Depending on the VTL, this may be too late.

I've got about 2TB of "free" space on my VTL, and about 1000 scratch VTs
(2800 total).

After a couple of weeks of Netbackup using and expiring VTs, The VTs are
going back to Scratch, but untill they're re-written, the pointers in
the repository still exist, therefore the VTL thinks the space is still
in use....so if you're using a de-duping VTL (or DSU to a de-duping FS)
and you're only doing "real time" freeing of space  (ie, re-labelling a
VT only when you re-write it, or letting NBU delete images from the DSU
only as the DSU approaches a high disk utilization threshold), then your
target will need to do some high-perf defragging in order to provide you
with sufficient "free" space to write more images.

Non-de-dup TLs such as Quantums, which "hard" allocate a fixed chunk of
disk for each cart will not have this problem, nor will a DSU on a
standard FS, but with my current VTL, it's definitely a requirement to
occasionally kick off a script to "bplabel" recently expired tapes, so
that the defrag process can run at a higher priority (when IO to the TL
is lower.)

Paul

-- 


> -----Original Message-----
> From: veritas-bu-bounces AT mailman.eng.auburn DOT edu 
> [mailto:veritas-bu-bounces AT mailman.eng.auburn DOT edu] On Behalf 
> Of Curtis Preston
> Sent: September 22, 2007 5:15 AM
> To: Clem Kruger; VERITAS-BU AT mailman.eng.auburn DOT edu
> Subject: Re: [Veritas-bu] Script to label expired tapes in a VTL
> 
> Oversubscription aside, once his tapes are expired, the space taken up
> by those tapes is immediately available for reuse.  The next time the
> tape gets written to, it will delete all pointers to the 
> space taken up
> by that tape.
> 
> As to the VTL vs disk debate, I still think you should bring 
> in all disk
> devices and let them duke it out before excluding an entire 
> category of
> them.  You're going to exclude a lot of really good products 
> if you just
> "no VTLs."  
> 
> Remember that saying "I don't want a VTL but I do want de-dupe" means
> that you're going to use NAS.  While that will meet a whole 
> lot of needs
> for a whole lot of people, there's also some really big backups that
> need a lot more than you can push over IP.  For those backups, you're
> going to want a block transfer protocol (i.e. SCSI), and for that,
> you're currently going to be buying a VTL.  (Unless you're 
> just going to
> buy a non-deduped disk in which case I'd say you're REALLY 
> wasting your
> money.)
> 
> ---
> W. Curtis Preston
> Backup Blog @ www.backupcentral.com
> VP Data Protection, GlassHouse Technologies 
> -----Original Message-----
> From: veritas-bu-bounces AT mailman.eng.auburn DOT edu
> [mailto:veritas-bu-bounces AT mailman.eng.auburn DOT edu] On Behalf Of Clem
> Kruger
> Sent: Saturday, September 22, 2007 4:24 AM
> To: VERITAS-BU AT mailman.eng.auburn DOT edu
> Subject: Re: [Veritas-bu] Script to label expired tapes in a VTL
> 
> Hi Steve,
> 
> This is the downer on VTL's. You do not get your "tape" space back
> automatically. It is for these reasons I recommend that one never go
> VTL's. NetBackup 6.0 and 6.5 allow disk to disk backups; the 
> images are
> easily replicated to an offsite facility.
> 
> The time for all "tape" has come and gone. The de-duplication facility
> in 6.5 makes life even easier. Why VTL's (which does SCSI emulation)
> when you and use disk which is faster and has more protection?
> 
> Clem.
> 
> -----Original Message-----
> From: veritas-bu-bounces AT mailman.eng.auburn DOT edu
> [mailto:veritas-bu-bounces AT mailman.eng.auburn DOT edu] On Behalf 
> Of swaltner
> Sent: 21 September 2007 17:32 PM
> To: VERITAS-BU AT mailman.eng.auburn DOT edu
> Subject: [Veritas-bu] Script to label expired tapes in a VTL
> 
> 
> We deployed a VTL last month, which has been working very nicely. This
> is in a NetBackup 5.1 environment with the VTL attached to our Solaris
> based master server as well as to our NAS server for local 
> NDMP backups.
> One thing I'd like to do is over-subscribe on the back-end 
> storage, but
> before I do that I'd like to automate the process of freeing 
> up the disk
> space used in the VTL when a NetBackup tape is expired. Just 
> curious if
> anyone has already written such a beast and would like to 
> share with me
> as a starting point.
> 
> If not, I suspect I'll use the following logic:
> 
> - Every day (at noon??), query the robots defined in the VTL 
> and keep a
> record of tapes that are allocated.
> - When a tape goes from allocated to non-allocated from one day to the
> next, use a command like the following to erase the tape's contents:
> bplabel -erase -o -d dlt -m VTL123
> 
> This would write a small label at the beginning of the virtual tape,
> causing the VTL to drop all the other data that had been stored on the
> tape.
> 
> Any reason this wouldn't work? Any gotchas with writing this 
> script that
> I should look out for?
> 
> Steve
> 
> +-------------------------------------------------------------
> ---------
> |This was sent by steve.waltner AT lsi DOT com via Backup Central.
> |Forward SPAM to abuse AT backupcentral DOT com.
> +-------------------------------------------------------------
> ---------
> 
> 
> _______________________________________________
> Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu
> http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
> 
> _______________________________________________
> Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu
> http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
> 
> _______________________________________________
> Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu
> http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu
> 
====================================================================================

La version française suit le texte anglais.

------------------------------------------------------------------------------------

This email may contain privileged and/or confidential information, and the Bank 
of
Canada does not waive any related rights. Any distribution, use, or copying of 
this
email or the information it contains by other than the intended recipient is
unauthorized. If you received this email in error please delete it immediately 
from
your system and notify the sender promptly by email that you have done so. 

------------------------------------------------------------------------------------

Le présent courriel peut contenir de l'information privilégiée ou 
confidentielle.
La Banque du Canada ne renonce pas aux droits qui s'y rapportent. Toute 
diffusion,
utilisation ou copie de ce courriel ou des renseignements qu'il contient par une
personne autre que le ou les destinataires désignés est interdite. Si vous 
recevez
ce courriel par erreur, veuillez le supprimer immédiatement et envoyer sans 
délai à
l'expéditeur un message électronique pour l'aviser que vous avez éliminé de 
votre
ordinateur toute copie du courriel reçu.

_______________________________________________
Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu