Terry,
I think I've got the jist of what you're saying.
What I've got set up in my environment is:
Daily:
By admin sched a differential from day 1 of each node and subsequent
differentials daily to stg=naspool.
versions data exists - unlimited
versions data deleted - 2 days
retain extra versions - 90 days
retain only version - 90 days
Monthly:
by admin sched a full each month on the last saturday to stg=nasmonthly
versions data exists - 12 months
versions data deleted - 12 months
retain extra versions - 12 months
retain only version - 12 months
I've set up a cmd script to move the media in the nasmonthly pool to
ovflo=firesafe which will run the first day of each month.
although I'm now aware I need to change the 90 days to 180 days, I was hoping I
was right in thinking that this would (once I've increased from 90 to 180)
allow me to go back 90 days and restore data from the naspool.
and allow me to go back as far as 12 months and restore data via the full
monthly backups from the firesafe. i may reduce that to 6 months depending on
the amount of tapes I end up needing.
Am I totally on the wrong track ?
Iain
p.s. I really appreciate your help with this.
-----Original Message-----
From: Terry McColgan [mailto:tmccolgan AT oct DOT ca]
Sent: Fri 04/03/2005 21:49
To: 'joni.moyer AT highmark DOT com'; Iain Barnetson
Cc: 'ADSM: Dist Stor Manager'
Subject: RE: NetApp NAS NDMP Data
Versions Data Deleted and Retain Only Version are not supported with NDMP.
The operations on pools Tape_NAS and Copy_NAS will not interfere with each
other.
You should realize that because there is no Backup/Archive client for NetApp
other than the WEB based one, the Display active/inactive files switch is
not available. That's one of the reasons why we retain so many versions. The
other reason is how NDMP backups decay. NDMP does not understand versioning.
Bear with me here, this takes a bit.
Lets pretend that we want a 5 day retention and we only work on weekdays
(insert laughter here).
Week 1
Day 1. Full backup
Day 2. Differential
Day 3. Differential
Day 4. Differential
Day 5. Differential
Week 2
Day 1. Full Backup
Day 2. Differential
Day 3. Differential
Day 4. Differential
Day 5. Differential
Week 3
Day 1. Full Backup
Day 2. Differential
Day 3. Differential
Day 4. Differential
Day 5. Differential
The Full Backups will not decay until the related Differential's do. That
means that on Week 2-Day 1, the Week 1 - Day 1 Full will not decay. On Week
2-Day 2, Week 1 - Day 2 will disappear. Week 2 - Day 3, Week 1 - Day 3 will
go away and so on ... On Week 3 - Day 1, Week 1 - Day 1's Full will go away.
If our retention is set to 5 and, on Week 2 - Day 2, the end user deletes a
file and the only other copy is from Week 1 - Day 2, its gone. That's why we
set the policy to twice what we want. With Versions Data Exist=Retain Extra
Versions=10, Week 1 - Day 2 will not decay until Week 3 - Day2. That way we
can go back to the previous backup cycle and be within our policy time.
The other issue is versions retained. Here we go... On Week 1 - Day 2, the
end user creates a document. On Week 1 - Day4, the end user modifies the
document. On Week 2 - Day 4, the end user wants the original back. If we are
on 5 day retention (actual, not policy), we're in trouble. If we are on 10
day retention, we're okay. In our organization, we promise (there's a word)
to keep 14 versions. We also promised to retain the last deleted version for
30 days. Remember NDMP does not understand versioning or Active/Inactive
files. Because of this and because the Diff's decay before the related
Full's, we set it to 60 days. Hence our policy is set to 60 days.
our NDMP policy
Policy Domain Name NETAPP_ON_POL
Policy Set Name ACTIVE
Mgmt Class Name NETAPP_ON_MC
Copy Group Name STANDARD
Versions Data Exists 60
Versions Data Deleted 0
Retain Extra Versions 60
Retain Only Version 0
Copy Mode MODIFIED
Copy Serialization SHRSTATIC
Copy Frequency 0
Copy Destination NPOOL_ON
Table of Contents (TOC) Destination TOC_DSKPOOL
Last Update Date/Time
Last Update by (administrator) ADMIN
Managing profile -
SCRIPTS:
(note: Library is an IBM 3583)
To reclaim tapes from the NPOOL_OFF storage pool and return them to scratch
status, cut and paste the following into the Tivoli command line:
mov med * stg= NPOOL_OFF wherestate=mountablenotinlib wherestatus=empty
rem=n checkl=n
To reclaim tapes from the NPOOL_ON storage pool and return them to scratch
status, cut and paste the following into the Tivoli command line:
mov med * stg=npool_on wherestate=mountablenotinlib wherestatus=empty rem=n
checkl=n
To create space in the library by removing stg=NPOOL_ON tapes (if necessary
for when the full's run):
mov med * stg=npool_on wherestate=mountableinlib wherestatus=full rem=b
checkl=n ovflo=YOURLOCATION
Another way is to explicitly name the vol you want removed:
mov med <vol_name>stg=npool_on wherestate=mountableinlib
wherestatus=full,filling rem=b checkl=n
I also use MS Excel to talk to the Tivoli server via ODBC to select
VOLUME_NAME, LAST_WRITE_DATE, PCT_UTILIZED,LOCATION sorted by last write
date on my onsite and offsite pools. When the PCT_UTILIZED returns to zero I
know to bring them back. VERY IMPORTANT: I run the Excel query before I run
the above commands as they remove the vols from the pools and return them to
scratch
Got a headache yet? I hope this helps more than it confuses. I'm hope I'm
not repeating myself.
Another red letter day in IT.
- t
-----Original Message-----
From: joni.moyer AT highmark DOT com [mailto:joni.moyer AT highmark DOT com]
Sent: Friday, March 04, 2005 7:48 AM
To: Terry McColgan
Subject: Re: NetApp NAS NDMP Data
Hi Terry,
Thank you for clarifying about full, differential and incremental backups.
I believe I was completely misunderstanding that concept. I hope I now
understand what needs to be done in my environment. Please let me know if
this makes sense.
Onsite Tape Pool: Tape_NAS
1. Full backup of all data.
2. Differential backups on a daily basis.
MC for onsite data: If I want to keep 60 days as you do in your environment,
what would the MC look like? I am a little confused as to what the versions
data deleted, retain extra versions and retain only versions parameter
should be.
Mgmt Class Name NASMGMTCLASS
Copy Group Name STANDARD
Versions Data Exists 10
Versions Data Deleted 1
Retain Extra Versions 30
Retain Only Version 60
Copy Mode MODIFIED
Copy Serialization SHRSTATIC
Copy Frequency 0
Copy Destination TAPE_NDMP
Table of Contents (TOC) Destination NAS_TOC
Offsite Tape Pool: Copy_NAS
1. Full backup on Sunday that will be sent offsite with a 14 day
retention period. Will this conflict with the original full + differential
backups that I take on a daily basis?
If you would be willing to share some of your scripts that you use to
automate this, I would appreciate it. Thanks!
********************************
Joni Moyer
Highmark
Storage Systems
Work:(717)302-6603
Fax:(717)302-5974
joni.moyer AT highmark DOT com
********************************
"Terry McColgan"
<tmccolgan AT OCT DOT CA
> To
Sent by: "ADSM: ADSM-L AT VM.MARIST DOT EDU
Dist Stor cc
Manager"
<[email protected] Subject
.EDU> Re: NetApp NAS NDMP Data
03/03/2005 03:56
PM
Please respond to
"ADSM: Dist Stor
Manager"
<[email protected]
.EDU>
Not to niggle here but NDMP only supports FULL and DIFFERENTIAL backup, not
INCREMENTAL. Also, make sure that your NDMP versions are the same on both
Tivoli and your NetApp device. Tivoli supports version 3. The NetApp command
is ndmpd version 3.
Joni: Does your "full backup of our NAS on Sunday" run in its own storage
pool. If it does then it should not interfere with anything else.
We have two storage pools (one for onsite, one for offsite) that are set to
retain 60 versions (approx. 2 months). We do a full backup monthly on a
weekend day and then run differentials on weekdays. This allows us to have
30 days of backups from any given point. After 60 days everything goes away
on its own. I have rudimentary scripts to automate some of this. Email me if
you wish.
Terry McColgan
Help Desk Support Specialist
Tivoli Storage Manager Specialist
Ontario College of Teachers
121 Bloor Street East, Floor 6
Toronto, Ontario
M4W 3M5
tmccolgan AT oct.on DOT ca
416-961-8800 ext. 270
"Go ahead... backup."
-----Original Message-----
From: Joni Moyer [mailto:joni.moyer AT HIGHMARK DOT COM]
Sent: Thursday, March 03, 2005 3:15 PM
To: ADSM-L AT VM.MARIST DOT EDU
Subject: Re: NetApp NAS NDMP Data
Hi Terry!
That is exactly what I wanted to do! I wanted to do an initial full backup
and then do incrementals and keep them all onsite. I then wanted to do a
full backup of our NAS on Sunday, but how do you do yours? I thought that
if I did a full, selective, backup on Sunday that would interfere with the
other backups I would want to keep onsite for regular restore purposes? Any
thoughts are appeciated! Thanks!
********************************
Joni Moyer
Highmark
Storage Systems
Work:(717)302-6603
Fax:(717)302-5974
joni.moyer AT highmark DOT com
********************************
"Terry McColgan"
<tmccolgan AT OCT DOT CA
> To
Sent by: "ADSM: ADSM-L AT VM.MARIST DOT EDU
Dist Stor cc
Manager"
<[email protected] Subject
.EDU> Re: NetApp NAS NDMP Data
03/03/2005 02:21
PM
Please respond to
"ADSM: Dist Stor
Manager"
<[email protected]
.EDU>
Hello; I have some experience with NDMP on a NetApp to Tivoli system. To
the
best of my knowledge, NDMP does not support archiving. It supports FULL and
DIFFERENTIAL backups only. Using the TOC I am able to do file level
restores
without a problem. Given the limitations on NDMP (no DRM ...), we scripted
a
system that has ONSITE and OFFSITE pools where the ONSITE pools stay in
the
library and the OFFSITE pools are ejected from the library (IBM 3583)
automatically to be stored offsite. Then, by running some SELECT statements
in MS excel we are able to determine when to bring the empties back. Given
the horror stories I've been reading about YSM 5.3 I think we'll just stick
to 5.2.x.x, thanks. - t
-----Original Message-----
From: Ben Bullock [mailto:bbullock AT MICRON DOT COM]
Sent: Thursday, March 03, 2005 1:16 PM
To: ADSM-L AT VM.MARIST DOT EDU
Subject: Re: NetApp NAS NDMP Data
Hmm, but can that be done through the NDMP methodology of backup? I don't
think so. You only really have a "backup" method, not an "archive" with
NDMP. I think....
We are just dipping our toes in NDMP, so I'm not an expert, but it looks
like for that data what we want to keep longer than just DR purposes, we
will need to archive it though an NFS mount, especially where there are a
few files in a whole qtree that you want kept forever but the rest only for
30 days.
NDMP just lacks that granularity, especially at the file level.
There is a 'virtualfsmapping' method in TSM 5.3 that can get you to the
directory level, so that may work if directory level is granular enough for
you.
Ben
-----Original Message-----
From: ADSM: Dist Stor Manager [mailto:ADSM-L AT VM.MARIST DOT EDU] On Behalf
Of
Stapleton, Mark
Sent: Thursday, March 03, 2005 11:04 AM
To: ADSM-L AT VM.MARIST DOT EDU
Subject: Re: NetApp NAS NDMP Data
From: ADSM: Dist Stor Manager [mailto:ADSM-L AT VM.MARIST DOT EDU] On Behalf
Of
Iain Barnetson
>I was wondering if anyone had any suggestions as to what I could do to
>get a series of Monthly full backups of the NAS nodes?
Easy. Use archives, not backups, for your monthlies. Set your archive
expiration to whatever timeframe is appropriate for your business needs.
--
Mark Stapleton (stapleton AT berbee DOT com)
Office 262.521.5627
|
