Networker

Re: [Networker] Limit Staging Operations

2006-11-09 18:15:24
Subject: Re: [Networker] Limit Staging Operations
From: Ronny Egner <Ronny.Egner AT SIV DOT DE>
To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
Date: Thu, 9 Nov 2006 11:08:36 +0100
Hi List,

i wrote a special script regarding my problem with staging:

My problem was that a needed to control the staging process
a lot better, i.e. the networker should stage the "old" savesets
(= written to staging area 4 days before but not 3,2,1 days) in
the morning so staging process is finished in the evening when
the dailly backups starting.

The script fetches all savesets on the staging pool (in our case:
"Default Stage") and writes the SSID to a temp file. Afterwards it
splits the temp file into two files (odd and even) and starts the
"nsrstage" process with these two files (= parallelizing the staging
progress over two tape drives).

The global networker staging directive is set to 10 days: If my
script doesnt get a saveset (it shouldnt :-) the networker will
stage it after 10 days. The second (and most important thing) is
that networker controls the usage of the volume and stages automatically
if the staging device is nearly full (high water mark -> low water
mark).


The script (remember - it is still experimental):

#
# Script for staging all savesets which was written
# $DAYS before to the staging area
#
# 09.11.2006 SIV.AG (Ronny Egner): initial Version
#
# GLOBAL VARs
export DIR=/nsr/scripts         #working dir
export TAGE='4'                 #the amount of days
export TAGE_T=`date --date="$TAGE days ago" '+%m/%d/%y'` # date for grep
export TAGE_S=`date --date="$TAGE days ago" '+%m.%d.%y'` # date for filename
#
#
# determine which savesets needed to stage
#
# create temp stage file for nsrstage
/usr/sbin/mminfo -a -r 'ssid,savetime,client,pool' -q '!ssrecycle,!incomplete,pool="Default Stage"' | grep $TAGE_T | cut -d ' ' -f 1 > nsrstage-$TAGE_S.cmd
#
# create verbose stage file for user control
/usr/sbin/mminfo -a -r 'savetime,client,pool,ssid' -q '!ssrecycle,!incomplete,pool="Default Stage"' | grep $TAGE_T > nsrstage-$TAGE_S.txt
#
#
# create two files to parallize staging
cat nsrstage-$TAGE_S.cmd | sed -ne 'w nsrstage-'"$TAGE_S-1.cmd"'' -e 'n;w nsrstage-'"$TAGE_S-2.cmd"''
#
#
# perform the actual staging operation with two drives
#
nsrstage -b 'Default' -m -s yellow-eyed.siv.de -v -S -f $DIR/nsrstage-$TAGE_S-1.cmd > $DIR/nsrstage-$TAGE_S-1.out 2>&1 &
#
nsrstage -b 'Default' -m -s yellow-eyed.siv.de -v -S -f $DIR/nsrstage-$TAGE_S-2.cmd > $DIR/nsrstage-$TAGE_S-2.out 2>&1 &
#
# perform cleanig
nsrstage -C -V Stage1
nsrstage -C -V Stage2
nsrstage -C -V Stage3
nsrstage -C -V Stage4
nsrstage -C -V Stage5



> Maarten Boot schrieb:
You can script, Either extra runs of the automatic processes Nsradmin would be relevant here "type: NSR Stage"

Or manually all savesets older than a certain date

Mminfo Query: !ssrecycle,!incomplete, pool = ..., volume .... , savetime Report: ssid,cloneid
Then nsrstage with -S

Hope this helps a bit

Maarten

Remember scripting can be fun !
-----Original Message-----
From: EMC NetWorker discussion [mailto:NETWORKER AT LISTSERV.TEMPLE DOT EDU] On 
Behalf Of Ronny Egner
Sent: Tuesday, November 07, 2006 10:23 AM
To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
Subject: [Networker] Limit Staging Operations

Hi,

we are running Networker 7.2.2 and having a staging area of 9 TB and 4 tape 
drives.

Due to operation system limitations which limits the file system to 2TB each we 
have 5 file systems for staging.

During the nichtly backups most of the data goes to the staging area.
However some database backups goes directly to tape. In the evening the data which was 
backed up 4 days ago to the staging area gets staged to tape. Because we have 5 staging 
devices the staging uses all 4 available tape drives for staging leaving no tape drive 
for the "normal"
backup to tape.

Is there a possibility to limit the time "normal"  (i.e. the volume is not at 
the high water mark and the data gets staged from disk to tape according to the policy) 
staging operations to the daytime and to avoid staging during the nightly backup time ?


Thanks!




--



Mit freundlichen Grüßen

Ronny Egner
Diplom-Ingenieur (BA)

SIV.AG
Konrad-Zuse-Straße 1
18184 Roggentin

Telefon: +49 (0)3 81 / 25 24 422
Telefax: +49 (0)3 81 / 25 24 399

mailto:ronny.egner AT siv DOT de
http://www.siv.de

**********************************************************************
This email and any files transmitted with it are confidential
and intended solely for the use of the individual or entity
to whom they are addressed. The views expressed in this
e-mail are those of the individual author and not necessarily
those of SIV.AG.

This footnote also confirms that this email message has
been swept by serval anti-virus tools for the presence
of computer viruses.
**********************************************************************

To sign off this list, send email to listserv AT listserv.temple DOT edu and type 
"signoff networker" in the
body of the email. Please write to networker-request AT listserv.temple DOT edu 
if you have any problems
wit this list. You can access the archives at 
http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER

<Prev in Thread] Current Thread [Next in Thread>