Networker

[Networker] Cloning parallelism

2008-01-23 14:28:10
Subject: [Networker] Cloning parallelism
From: brerrabbit <networker-forum AT BACKUPCENTRAL DOT COM>
To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
Date: Wed, 23 Jan 2008 11:18:35 -0800
Landwehr, Jerome wrote:
> 
> 
> but every nsrclone process gets one stream, so to multiplex, you use
> more than one tape
> 
> 


snip


> 
> something like this:
> 
> mminfo -a -q "pool=VTL" -q "copies=1" -r
> "ssid,ssflags,client,sscreate,ssbrowse,ssretent,totalsize(14),level,name
> " | sort -ub +2 +0 | grep -v " vr" | grep -v " ssid " | awk -f
> ~/bin/sizeclone.awk
> # split by hand into files at /var/tmp/clone/setN
> cd /var/tmp/clone
> foreach SET (set?)
> batch <<!
> echo working /var/tmp/clone/$SET
> nsrclone -v -b "Period Clone" -S -f /var/tmp/clone/$SET |& tee
> /var/tmp/log.$SET 
> !
> end
> 
> 
> where sizeclone.awk splits them into ~2TB chunks:
> 
> # cat bin/sizeclone.awk       
> BEGIN { { a = 1 } 
> { tot = 0 }
> { sets = 0 }
>       { totsets = 1 } }
> { if ( a == 1 ) {
> print "cloneset ", a++ ," starts after ", $1 , $3 , $9 } }
> { s = s + $7 }
> { print $1 }
> { sets ++ }
> { if ( s >= 2048779069440 ) {
>       print "      ( ", sets ," savesets, data size ", s/1099511627776
> , "TB )"
>       print "cloneset ", a++ ," starts after ", $1 , $3 , $9 
>       tot = tot + s
>       totsets = sets + totsets
>       sets=0
>       s=$7 } }
> END { 
>       print "      ( ", sets ," savesets, data size ", s/1099511627776
> , "TB )"
>       tot = tot + s
>       totsets = sets + totsets - 1
>       print "\nTotals: savesets", totsets, ", clonesets", a-1, ", data
> size", tot/1099511627776 , "TB" }
> 
> 
> 
> HTH
> Jerry
> 


It's funny, I just started working on something very similar for our Windows 
Networker server myself.  It's not done yet, but I hope to further organize the 
savesets so that the VTL tapes to be cloned from are loaded a minimal number of 
times, a la "clone everything off this tape, then clone everything off that 
one" etc etc.  I'd be happy to post the working copy (when it's done) if anyone 
is interested.

-brerrabbit

+----------------------------------------------------------------------
|This was sent by drhulme AT tarrantcounty DOT com via Backup Central.
|Forward SPAM to abuse AT backupcentral DOT com.
+----------------------------------------------------------------------

To sign off this list, send email to listserv AT listserv.temple DOT edu and 
type "signoff networker" in the body of the email. Please write to 
networker-request AT listserv.temple DOT edu if you have any problems with this 
list. You can access the archives at 
http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER