Networker

Re: [Networker] Cloning parallelism

2008-01-23 09:32:39
Subject: Re: [Networker] Cloning parallelism
From: "Landwehr, Jerome" <jlandweh AT HARRIS DOT COM>
To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
Date: Wed, 23 Jan 2008 09:28:36 -0500
if you are doing (automatic) savegroup cloning, that's all you get

in fact, cloning can't duplex, it actually un-muxes.  only one stream is
ever sent to a clone tape at a time

but every nsrclone process gets one stream, so to multiplex, you use
more than one tape

due to this obvious defeciency, we stopped using automatic cloning,
instead scripting something to determine the savesets to clone and
splitting them N ways and running nsrclone on each group  

something like this:

mminfo -a -q "pool=VTL" -q "copies=1" -r
"ssid,ssflags,client,sscreate,ssbrowse,ssretent,totalsize(14),level,name
" | sort -ub +2 +0 | grep -v " vr" | grep -v " ssid " | awk -f
~/bin/sizeclone.awk
# split by hand into files at /var/tmp/clone/setN
cd /var/tmp/clone
foreach SET (set?)
batch <<!
echo working /var/tmp/clone/$SET
nsrclone -v -b "Period Clone" -S -f /var/tmp/clone/$SET |& tee
/var/tmp/log.$SET 
!
end


where sizeclone.awk splits them into ~2TB chunks:

# cat bin/sizeclone.awk 
BEGIN { { a = 1 } 
        { tot = 0 }
        { sets = 0 }
        { totsets = 1 } }
{ if ( a == 1 ) {
        print "cloneset ", a++ ," starts after ", $1 , $3 , $9 } }
{ s = s + $7 }
{ print $1 }
{ sets ++ }
{ if ( s >= 2048779069440 ) {
        print "      ( ", sets ," savesets, data size ", s/1099511627776
, "TB )"
        print "cloneset ", a++ ," starts after ", $1 , $3 , $9 
        tot = tot + s
        totsets = sets + totsets
        sets=0
        s=$7 } }
END { 
        print "      ( ", sets ," savesets, data size ", s/1099511627776
, "TB )"
        tot = tot + s
        totsets = sets + totsets - 1
        print "\nTotals: savesets", totsets, ", clonesets", a-1, ", data
size", tot/1099511627776 , "TB" }
 


HTH
Jerry


-----Original Message-----
From: EMC NetWorker discussion [mailto:NETWORKER AT LISTSERV.TEMPLE DOT EDU] On
Behalf Of Tim Verbois
Sent: Wednesday, January 23, 2008 2:44 AM
To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
Subject: [Networker] Cloning parallelism


Hello,

Environment:

VTL
L700 (LTO3 + LTO2 drives)
Legato 7.3.2 from SUN
Server + storage node

is it possible to have the parallelism on a cloning set to 2?  I changed

the setting in the java interface, but it doesn't change a thing.

Why do I want to do that?  We clone from VTL to L700 LTO3 tape drives.  
The speed of an LTO3 drive is 70 MB/s native and can reach 100 MB when 
compressed.  Our VTL can supply data at a speed of 70 MB/s.  A client 
can do a restore at max 50 MB/s.  So while we clone at a speed of 70 
MB/s compressable data, we lose at least 20 MB/s average speed for 
cloning.  For restores, our tape is to fast, we would be satisfied with 
an everage restore speed of 40MB/s. 

The problem we have is that we need the top speed for cloning because of

the small time window for cloning (backupping every day and the full 
weekend, needs cloning at top speeds during all the free time). 

In a few weeks we start using LTO4, this is going to make the loss of 
speed even bigger, much bigger.  Can this be solved?

-- 
Tim Verbois
Unix Team
EDS-Telindus
02/553.71.73

To sign off this list, send email to listserv AT listserv.temple DOT edu and
type "signoff networker" in the body of the email. Please write to
networker-request AT listserv.temple DOT edu if you have any problems with this
list. You can access the archives at
http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER

To sign off this list, send email to listserv AT listserv.temple DOT edu and 
type "signoff networker" in the body of the email. Please write to 
networker-request AT listserv.temple DOT edu if you have any problems with this 
list. You can access the archives at 
http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER