Veritas-bu

Re: [Veritas-bu] bulk import of images

2008-06-13 15:35:02
Subject: Re: [Veritas-bu] bulk import of images
From: "Martin, Jonathan" <JMARTI05 AT intersil DOT com>
To: "Justin Piszcz" <jpiszcz AT lucidpixels DOT com>, "Esson, Paul" <Paul.Esson AT Redstor DOT com>
Date: Fri, 13 Jun 2008 15:16:31 -0400
Step 1 - Create MediaIds.txt with the media to import

Step 2 - Assign the media to something that is NOT scratch using the
following command.

for /f %%c in (C:\temp\impt5\MediaIds.txt) do vmchange -h <master> -p 12
-m %%c

The -p 12 is the volume pool number of where you want the media to go.

Step 3 - Stage I Import

for /f %%c in (C:\temp\impt5\MediaIds.txt) do bpimport -create_db_info
-id %%c -L C:\temp\impt5\%%c.txt

Step 4 - Run this script to populate .csv file with media, image and
image info

#Open CSV File for Output
open(DAT1, ">Catalog_media_rpt1.csv");
#Find all the <mediaid>.txt files
open(FIZZLE,"dir /B *.txt |");
#Parse the directory for Media IDs
while (<FIZZLE>) {
if (m/(.+)\.txt/) { $mediaid = $1;
#Open <mediaid>.txt file
open(DAT2, "$_");
#Parse <mediaid>.txt file for image names
while (<DAT2>) {
if ((/^Backup-ID/) || (/^Copy Frag/) || (/^\s/) || (/^---/)) {}
else {
if (/(\w+)\s/) {
$image = $1;
#Run bpimage list to get image information

open(FIZZLE1,"bpimagelist -U -backupid $image |");
while (<FIZZLE1>) {
if (/[0-9]/) {print "$mediaid,$image,$_\n";}
}}}}}}
close(DAT1);

Sorry for the difficult to read perl, I threw this together quickly.

>>From this point I put the .csv into excel and created a pivot table with
backups of which servers when.  Then I manually imported whatever images
were required.  If you wanted to simply pahse II import everything you
could simply run another bpimport command in a for loop.

Good luck!

-Jonathan


-----Original Message-----
From: Justin Piszcz [mailto:jpiszcz AT lucidpixels DOT com] 
Sent: Friday, June 13, 2008 4:36 AM
To: Esson, Paul
Cc: Martin, Jonathan; veritas-bu AT mailman.eng.auburn DOT edu
Subject: Re: [Veritas-bu] bulk import of images

I would be interested in that as well.

On Fri, 13 Jun 2008, Esson, Paul wrote:

> Jonathan,
>
>
>
> Thanks for quick the response.  A copy of the script would be welcome 
> to get me started.
>
>
>
> Regards,
>
>
>
> Paul Esson
>
>
>
> ________________________________
>
> From: Martin, Jonathan [mailto:JMARTI05 AT intersil DOT com]
> Sent: 13 June 2008 01:42
> To: Esson, Paul; veritas-bu AT mailman.eng.auburn DOT edu
> Subject: RE: [Veritas-bu] bulk import of images
>
>
>
> I scripted the import of several hundred SDLT Media running a simple 
> script.  I output the entries of the Phase I import to a text file 
> called <mediaID>.txt and  wrote a perl script to read that in and 
> create an excel file with the media id, date of the backup and what 
> server was backed up.  I then used a pivot chart to combine what 
> servers had backups available for what days / months / years.  As and 
> when someone makes a request I run a phase II import only on the 
> images required, then restore the data.
>
>
>
> My suggestions are as follows:
>
> 1) Watch the tape drives; I had the occasional media get "stuck."  I 
> think, due to bad media.  The issue would cause the whole process to 
> hang.
>
> 2) I had a huge library with over 300 slots and a media server 
> dedicated to this effort.  I'm sure a smaller library would have been 
> a major pain.
>
> 3) Write a script or look for phase I logs that are very small - these

> are usually errors (in my case anyway)
>
>
>
> I can send you my scripts but they are just simple for loops, looking 
> at a list of media that I loaded.  On SDLT (110GB / 220GB Compressed) 
> the Phase I's took a little better than an hour per (we multiplexed 4x
> /sigh) and the phase IIs take 4-8 hours depending.  I'm sure LTOs 
> would take longer, so good luck!
>
>
>
> -Jonathan
>
>
>
> ________________________________
>
> From: veritas-bu-bounces AT mailman.eng.auburn DOT edu
> [mailto:veritas-bu-bounces AT mailman.eng.auburn DOT edu] On Behalf Of Esson,

> Paul
> Sent: Thursday, June 12, 2008 8:22 PM
> To: veritas-bu AT mailman.eng.auburn DOT edu
> Subject: [Veritas-bu] bulk import of images
>
>
>
> Folks,
>
>
>
> Anybody out there successfully automated a large volume import of tape

> media into NetBackup?  I am curious as to how you controlled the 
> processing, given the potential conflict in using available tapes 
> drives for example and how you validated the success of the phases of 
> the import.  We have several hundred tapes to process (800+). I am 
> also looking for indications as to how long a phase 1 and phase 2 
> import will take on LTO-1 media with close to 2:1 compression (190+ 
> GB).  We have multiplexed media and 2 GB fragment sizes which I 
> imagine will slow things down too.
>
>
>
> Regards,
>
>
>
> Paul Esson
> Redstor Limited
>
> Direct:               +44 (0) 1224 595381
> Mobile:          +44 (0) 7766 906514
> E-Mail:          paul.esson AT redstor DOT com
> Web:            www.redstor.com
>
> REDSTOR LIMITED
> Torridon House
> 73-75 Regent Quay
> Aberdeen
> UK
> AB11 5AR
>
> Disclaimer:
> The information included in this e-mail is of a confidential nature 
> and is intended only for the addressee.  If you are not the intended 
> addressee, any disclosure, copying or distribution by you is 
> prohibited and may be unlawful.  Disclosure to any party other than 
> the addressee, whether inadvertent or otherwise is not intended to 
> waive privilege or confidentiality.
>
>
>
>
> ______________________________________________________________________
> This email has been scanned by the MessageLabs Email Security System.
> For more information please visit http://www.messagelabs.com/email 
> ______________________________________________________________________
>
>

_______________________________________________
Veritas-bu maillist  -  Veritas-bu AT mailman.eng.auburn DOT edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu

<Prev in Thread] Current Thread [Next in Thread>