Networker

Re: [Networker] Clone from EDL to library

2009-03-21 17:39:16
Subject: Re: [Networker] Clone from EDL to library
From: Mitko Stoyanov <mstoyan AT MEGADOT.COM DOT AU>
To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
Date: Sun, 22 Mar 2009 08:31:48 +1100
From your posting it is not clear whether this is a one-off job, or you need to set up this on a permanent basis...if it is a one-of job, then a few ad-hoc commands like these are OK.

We do similar stuff, cloning from AFTD to tape, but our requirements are a bit more complex, as some pools are cloned once, other are cloned twice, depending how important the backups are. In short, we ended with a few thousand lines of shell scripting doing all this :) For our hardware setup and data volumes, it works very well.

Here are a few things to consider:

* "nsrclone" clones the savesets sequentially one by one. That could be a bit of a problem with multi-terabyte backups (time to complete), or fast clone devices (a bit of LTO4 shoe-shining if your savesets are small). We compress all backups going to AFTD to maximize the data stream going from the AFTDs to the clone drives later.

* From your script below, if you run the second clone operation and your cloning fails and needs to be rerun, your selection criteria may pick up savesets which are already cloned twice. We use "copies<X" as one of the selections, to ensure that we don't end up with savesets cloned more than twice. The "copies" selection would harvest as well any savesets which failed to clone outside your "savetime" window for various reasons.

* We run a daily "alert" script to warn us if there is anything left for cloning when all cloning is finished, as we don't want to have a saveset not cloned to a tape.

* nsrclone won't clone the same ssid twice to a single volume (e.g the ssid is cloned already and the ops forget to change the volumes next day), so we check whether we have that ssid already in the target volumes.

* We order the mminfo selection output by "savetime,client,name" (or something similar, I don't have access to the scripts at the moment) so that they are ordered for optimal recovery from the tapes.

The main problem we have with the cloning so far is with the Networker GSS authentication. The clone jobs sometimes would fail with "Authentication type 0 non adequate" We have an open CR with EMC about this, which is not sorted out yet.

Gordon wrote:
What is the best way to do this?

I have all backup pools run in a VTL and I need  2 clones on real tape every 
day, the first copy stay in the same building in a fire proof safe-deposit box, 
the second copy is transported daily to another safe location.
I need to make today a clon of all yesterday savesets twice, each ones on a 
diferent clone pool. To do that I make this mminfo query, I like to know your 
opinion about this. Thanks!

mminfo -q 'savetime>=yesterday 00:00:00, savetime<=23:59:59 yesterday , pool=filesystemW, 
pool=filesystemP, pool=Default, pool=filesystemL, !incomplete, !suspect' -r "volume, barcode, volid, 
ssid, cloneid, name, client, copies"|awk '{print $4 "/" $5}'|tail +2>ssid.cloneid


nsrclone -S -f ssid.cloneid -b Copy1

+----------------------------------------------------------------------
|This was sent by tuamigobender AT gmail DOT com via Backup Central.
|Forward SPAM to abuse AT backupcentral DOT com.
+----------------------------------------------------------------------

To sign off this list, send email to listserv AT listserv.temple DOT edu and type 
"signoff networker" in the body of the email. Please write to networker-request 
AT listserv.temple DOT edu if you have any problems with this list. You can access the 
archives at http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER


To sign off this list, send email to listserv AT listserv.temple DOT edu and type 
"signoff networker" in the body of the email. Please write to networker-request 
AT listserv.temple DOT edu if you have any problems with this list. You can access the 
archives at http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER