Re: [Networker] Limit Staging Operations

Some notes

Use wait for the 2 background staging process to finish befor you use the 5 
cleaning processes

As in 

p1 &
p2 &
wait
p3

You could use the -xc, option to separate output from mminfo with , instead of 
space
It takes care of surprises when output fields from mminfo are empty

As I think there should be no data older than 5 days on your disk devices I 
would use 
Save time in the query already like -q' .....,savetime > 4 days ago,....' 


Maarten

-----Original Message-----
From: EMC NetWorker discussion [mailto:NETWORKER AT LISTSERV.TEMPLE DOT EDU] On 
Behalf Of Ronny Egner
Sent: Thursday, November 09, 2006 11:09 AM
To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
Subject: Re: [Networker] Limit Staging Operations

Hi List,

i wrote a special script regarding my problem with staging:

My problem was that a needed to control the staging process a lot better, i.e. 
the networker should stage the "old" savesets (= written to staging area 4 days 
before but not 3,2,1 days) in the morning so staging process is finished in the 
evening when the dailly backups starting.

The script fetches all savesets on the staging pool (in our case:
"Default Stage") and writes the SSID to a temp file. Afterwards it splits the 
temp file into two files (odd and even) and starts the "nsrstage" process with 
these two files (= parallelizing the staging progress over two tape drives).

The global networker staging directive is set to 10 days: If my script doesnt 
get a saveset (it shouldnt :-) the networker will stage it after 10 days. The 
second (and most important thing) is that networker controls the usage of the 
volume and stages automatically if the staging device is nearly full (high 
water mark -> low water mark).


The script (remember - it is still experimental):

#
# Script for staging all savesets which was written # $DAYS before to the 
staging area # # 09.11.2006 SIV.AG (Ronny Egner): initial Version # # GLOBAL 
VARs
export DIR=/nsr/scripts         #working dir
export TAGE='4'                 #the amount of days
export TAGE_T=`date --date="$TAGE days ago" '+%m/%d/%y'` # date for grep export 
TAGE_S=`date --date="$TAGE days ago" '+%m.%d.%y'` # date for filename # # # 
determine which savesets needed to stage # # create temp stage file for 
nsrstage /usr/sbin/mminfo -a -r 'ssid,savetime,client,pool' -q 
'!ssrecycle,!incomplete,pool="Default Stage"' | grep $TAGE_T | cut -d ' 
' -f 1 > nsrstage-$TAGE_S.cmd
#
# create verbose stage file for user control /usr/sbin/mminfo -a -r 
'savetime,client,pool,ssid' -q '!ssrecycle,!incomplete,pool="Default Stage"' | 
grep  $TAGE_T > nsrstage-$TAGE_S.txt # # # create two files to parallize 
staging cat nsrstage-$TAGE_S.cmd | sed -ne 'w nsrstage-'"$TAGE_S-1.cmd"'' -e 
'n;w nsrstage-'"$TAGE_S-2.cmd"''
#
#
# perform the actual staging operation with two drives # nsrstage -b 'Default' 
-m -s yellow-eyed.siv.de -v -S -f $DIR/nsrstage-$TAGE_S-1.cmd > 
$DIR/nsrstage-$TAGE_S-1.out 2>&1 & # nsrstage -b 'Default' -m -s 
yellow-eyed.siv.de -v -S -f $DIR/nsrstage-$TAGE_S-2.cmd > 
$DIR/nsrstage-$TAGE_S-2.out 2>&1 & # # perform cleanig nsrstage -C -V Stage1 
nsrstage -C -V Stage2 nsrstage -C -V Stage3 nsrstage -C -V Stage4 nsrstage -C 
-V Stage5



 > Maarten Boot schrieb:
> You can script,
> 
> Either extra runs of the automatic processes Nsradmin would be 
> relevant here "type: NSR Stage"
> 
> Or manually all savesets older than a certain date
> 
> Mminfo 
>       Query: !ssrecycle,!incomplete, pool = ..., volume .... , savetime 
>       Report: ssid,cloneid
> Then nsrstage with -S
> 
> Hope this helps a bit
> 
> Maarten
> 
> Remember scripting can be fun !
>  
> 
> -----Original Message-----
> From: EMC NetWorker discussion [mailto:NETWORKER AT LISTSERV.TEMPLE DOT EDU] 
> On Behalf Of Ronny Egner
> Sent: Tuesday, November 07, 2006 10:23 AM
> To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
> Subject: [Networker] Limit Staging Operations
> 
> Hi,
> 
> we are running Networker 7.2.2 and having a staging area of 9 TB and 4 tape 
> drives.
> 
> Due to operation system limitations which limits the file system to 2TB each 
> we have 5 file systems for staging.
> 
> During the nichtly backups most of the data goes to the staging area.
> However some database backups goes directly to tape. In the evening the data 
> which was backed up 4 days ago to the staging area gets staged to tape. 
> Because we have 5 staging devices the staging uses all 4 available tape 
> drives for staging leaving no tape drive for the "normal"
> backup to tape.
> 
> Is there a possibility to limit the time "normal"  (i.e. the volume is not at 
> the high water mark and the data gets staged from disk to tape according to 
> the policy) staging operations to the daytime and to avoid staging during the 
> nightly backup time ?
> 
> 
> Thanks!
> 
> 


-- 



Mit freundlichen Grüßen

Ronny Egner
Diplom-Ingenieur (BA)

SIV.AG
Konrad-Zuse-Straße 1
18184 Roggentin

Telefon: +49 (0)3 81 / 25 24 422
Telefax: +49 (0)3 81 / 25 24 399

mailto:ronny.egner AT siv DOT de
http://www.siv.de

**********************************************************************
This email and any files transmitted with it are confidential and intended 
solely for the use of the individual or entity to whom they are addressed. The 
views expressed in this e-mail are those of the individual author and not 
necessarily those of SIV.AG.

This footnote also confirms that this email message has been swept by serval 
anti-virus tools for the presence of computer viruses.
**********************************************************************

To sign off this list, send email to listserv AT listserv.temple DOT edu and 
type "signoff networker" in the body of the email. Please write to 
networker-request AT listserv.temple DOT edu if you have any problems wit this 
list. You can access the archives at 
http://listserv.temple.edu/archives/networker.html or via RSS at 
http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER

To sign off this list, send email to listserv AT listserv.temple DOT edu and 
type "signoff networker" in the
body of the email. Please write to networker-request AT listserv.temple DOT edu 
if you have any problems
wit this list. You can access the archives at 
http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER