Networker

[Networker] hard drive staging in Networker:(Version 2)

2003-01-06 16:04:30
Subject: [Networker] hard drive staging in Networker:(Version 2)
From: "Thomas, Calvin" <calvin.thomas AT NACALOGISTICS DOT COM>
To: NETWORKER AT LISTMAIL.TEMPLE DOT EDU
Date: Mon, 6 Jan 2003 13:01:25 -0800
After the earlier "staging in networker" thread, I have re-worked my staging
system, and wanted to invite comments.

Networker configuration NOW:
1 large hard drive partition, using ADVFS on Tru64 to make a single 180GB
partition.
1 File device called DailyHardDrive_1
All backups in 2 backup groups (one with clone, and one without) backup
simultaneously to this partition.  It backs up our 50GB of compressed data
(200GB uncompressed) in about 2 hours, at 12MBPS.
While the clone is running, I run the following series of scripts.

(awk.cmd is adapted from a script by Maarten Boot.  Thanks Maarten)


1. Newstage
makelist
awk -f awk.cmd backlist | sh

2. Makelist
mminfo -q'savetime < 48 hours ago' -q'volume=DailyHardDrive_1'
-r'ssid,cloneid' > backlist

3. awk.cmd
NR == 1 { next }
{
        printf "/usr/opt/networker/bin/nsrstage -v -b 'Daily Jukebox' -m -S
%s/%s\n", $1,$2
}


Forget about automatic stageing in networker.  I have given it up as a lost
cause.

The above scripts works on Tru64 Alpha.  You will need to modify them for
your system.  I have removed all paths.  I couldn't get it to run as a
single script because my backup pool "Daily Jukebox" required a third level
of quotes, and I couldn't resolve that issue.  Note the 'savetime < 48 hours
ago' in the makelist script seems to have the arrow going the wrong way.  It
isn't.

This method resolves several problems with hard drive staging.

1. It stages one save set at a time.  After each save set is transfered, it
is promptly deleted from the hard drive, freeing up the space.

2. If a backup starts while this script is running, it allows the backup to
start running as soon as the current save set is finished.  However if the
current save set takes too long, the backup will time out.

3. This staging command will politely wait for the clone to finish, and then
immediately begin staging to the jukebox.

4. It resolves the issue of the file device being marked "full" by never
allowing the file device to fill up.  You  have to adjust the scripts, but
this version keeps the last 2 days of backup on hard drive, and anything
older is transfered to jukebox.

5. NSRstage command has a problem.  If you have an aborted save set, the
NSRstage command will not work. Then you need to find and delete the problem
save set.  I use the aformentioned script by Maarten to do this.


Any improvements, or suggestions?

Calvin Thomas
UNIX System Administrator
NACA Logistics

--
Note: To sign off this list, send a "signoff networker" command via email
to listserv AT listmail.temple DOT edu or visit the list's Web site at
http://listmail.temple.edu/archives/networker.html where you can
also view and post messages to the list.
=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=

<Prev in Thread] Current Thread [Next in Thread>
  • [Networker] hard drive staging in Networker:(Version 2), Thomas, Calvin <=