Networker

Re: [Networker] Initial Report on Functionality improvements in NW7 using Adv_Fil e devices

2003-05-22 06:42:38
Subject: Re: [Networker] Initial Report on Functionality improvements in NW7 using Adv_Fil e devices
From: "Wood, R A (Bob)" <WoodR AT CHEVRONTEXACO DOT COM>
To: NETWORKER AT LISTMAIL.TEMPLE DOT EDU
Date: Thu, 22 May 2003 11:42:20 +0100
Very interesting comments, it is good to see some reactions to NW7
starting to appear.

As for your backup, stage, clone routine. My instinct would be to backup
to file, clone for offsite, then stage from disk to tape. I haven't done
this yet, just pushing round some ideas for the best use of NW7
features.

Here's, ideally, what I would like to be able to do;

- Backup to disk - the disk should be big enough to take several days
worth of data (restores for data that has recently been backed up should
run fast)
- A set of clone tapes for offsite should be produced as soon as
possible (hopefully, using the adv-file device I can clone savesets as
they finish backing up saving loads of time and making the offsite tapes
available earlier than at present)
- Stage from disk to tape (not sure whether this should be by high water
mark or by time - but that's just a detail, I can try both and then
decide). Also, not sure if this would impact the following nights backup
(savegroups should have finished, even with cloning enabled). May just
need to ensure that there is sufficient disk to cope.

I think I'd rather clone from disk (should be a performance gain over
cloning from tape?). 
You mention that the cloning fails because the file device is being used
by staging. Would a second file device help? Maybe alternate the file
devices daily (gives more than 24 hours for the cloning/staging to
complete without impacting following nights backup)

I'd like to hear other views on preferred data handling. Maybe something
like - backup to disk, clone to offsite disk device, stage to tape
(hmmm, eliminates offsite tape handling?)

There's certainly more than one way to skin this particular cat....

Regards
Bob

-----Original Message-----
From: Thomas, Calvin [mailto:calvin.thomas AT NACALOGISTICS DOT COM] 
Sent: 21 May 2003 20:10
To: NETWORKER AT LISTMAIL.TEMPLE DOT EDU
Subject: [Networker] Initial Report on Functionality improvements in NW7
using Adv_Fil e devices


My reasons for posting this are to stimulate some comments from other's
about NW7.0 and it's new features.  If you aren't interested in NW7,
stop here.

I adopted NW7 for Tru64 last month, and after the initial glow has worn
off, here is what I find.

First a few stats:
I use Adv_File staging on the server.  My current staging drive is
300GB. My jukebox is 140GB and it only copys at 1MB/sec.  My daily
backups are about 50GB.

One problem I had with staging previously is fixed.  If the system is
staging to my jukebox, my backups still run.  Successfully.  Big
Improvement here.

Now to the down side.

1. NW7 can now stage, and backup at the same time, but it can not stage
and clone at the same time. When my backups finish, the automatic
cloneing waits for the staging to finish.  Since this may take up to a
day on my jukebox, the clone doesn't start until the stageing is done.
When the clone isn't done in a timely manner, it SEVERLY limits the
functionality of the backups
+ staging improvement.

2.  Automatic staging creates a list of files to stage, and then stages
them ALL at once.  After staging ALL the files to tape, it then deletes
ALL the files at the same time.  Sounds logical right?  True, but not
very functional.  I was using a script previously (on 6.1.1) that staged
files ONE by ONE to tape, and then deleted the files ONE by ONE from the
file device.  This was much superior to the ALL at once way.  What I
find is if my jukebox is full (like over a long weekend,) and the file
device gets full, when I put in new tapes after the weekend, the staging
will stage ALL the files at once.  In my case that can be 200GB, and it
takes 36 hours to stage all the files.  This causes the automatic
cloneing to fail since the next backup started before the file device
was freed up.  It also caused concern on my part because I only had 20GB
of space left for the nightly 50GB backup.  Even though 175GB had been
staged to tape, I could not free up the space for the next nightly
backup.  I was forced to kill the staging process, and manually start
the "recover space" process to get enough room to do the nightly backup.

The Wish List:
It should be a relatively simple programing change for legato to change
the staging process from an ALL (save sets) at once process to a ONE
(save set) at a time process.  I was able to accomplish this feat with a
simple script on NW6.1.1.

My first script ran just like NW7.0's does. It created a giant file of
SSID/CLONEID's, and passed this to a single nsrstage command with the
same problems I have with NW7.0 now.

My second script was much better.  It simple used AWK to run the
nsrstage command once with a single ssid/cloneid number as input.  Once
the nsrstage command finished the single save set, it deleted the save
set file from the file device, and then AWK would run the next nsrstage
command. This allowed the gradual reduction in size of the files stored
on the file devices, and it allowed backups, and clones to run (though
it had to wait until after the current nsrstage command finished).  It
was kludgy, and temperamental, but it worked.


What do you think?  Comments everyone?

--
Note: To sign off this list, send a "signoff networker" command via
email to listserv AT listmail.temple DOT edu or visit the list's Web site at
http://listmail.temple.edu/archives/networker.html where you can also
view and post messages to the list.
=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=

--
Note: To sign off this list, send a "signoff networker" command via email
to listserv AT listmail.temple DOT edu or visit the list's Web site at
http://listmail.temple.edu/archives/networker.html where you can
also view and post messages to the list.
=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=