ADSM-L

Once a week selectives Vs Copypool

2015-10-04 17:31:35
Subject: Once a week selectives Vs Copypool
From: ADSM: Dist Stor Manager [mailto:ADSM-L AT VM.MARIST DOT EDU]On Behalf Of
To: ADSM-L AT VM.MARIST DOT EDU


Gerrit,

Unless you are  backing up a huge amount of "temporary" data, wouldn't you
be better off to still use a copypool but only run the backup stg command
on a weekly basis?

With your method you are backing up stuff that you already have, and will
need a sizeable backup window.  The copypool approach will only copy
changed data and will not impact the client system.

I'm sure you have your reasons for doing it your way - but I'd really like
to know what they are.

Steve.

ADSM etc Guy
The Wesley Hospital, Brisbane Australia






Gerrit van Zyl <gvanzyl AT FARITEC.CO DOT ZA>@VM.MARIST.EDU> on 10/04/2000
22:29:12

Please respond to "ADSM: Dist Stor Manager" <ADSM-L AT VM.MARIST DOT EDU>

Sent by:  "ADSM: Dist Stor Manager" <ADSM-L AT VM.MARIST DOT EDU>


To:   ADSM-L AT VM.MARIST DOT EDU
cc:
Fax to:
Subject:  Re: HSM not migrating


Reinhard,

Because of the huge amount of backups we are doing, we cannot use copy stg
pools to create offsite copies.  So we develop out own scripts to determine
which "live" tapes to take offsite and which ones to return to the robot.
To guarentee we always have a copy of any file offsite, we need to once a
week do a full selective backup.

In any case here is the selective backup of a file : istat iostat.25nov


File istat before selective backup :

02n09sw /bgs/best1 # istat iostat.25nov
Inode 2060 on device 39/1       File
Protection: rw-r--r--
Owner: 500(best1)               Group: 1(staff)
Link count:   1         Length 71731 bytes

Last updated:   Thu Nov 25 10:34:54 1999
Last modified:  Thu Nov 25 10:34:54 1999
Last accessed:  Wed Apr 05 12:12:53 2000

Doing selective backup  with dsmnode -t=p:

 /bgs/best1 # dsmmode -t=p dsmc sel iostat.25nov
ADSTAR Distributed Storage Manager
space management Interface - Version 3, Release 1, Level 0.7
(C) Copyright IBM Corporation, 1990, 1999. All Rights Reserved.
ADSTAR Distributed Storage Manager
Command Line Backup Client Interface - Version 3, Release 1, Level 0.7
(C) Copyright IBM Corporation, 1990, 1999, All Rights Reserved.

Selective Backup function invoked.

Node Name: F02N09SW-N25
Session established with server ADSM: AIX-RS/6000
  Server Version 3, Release 1, Level 2.55
  Server date/time: 04/10/00   14:15:47  Last access: 04/10/00   10:59:57

Normal File-->            71,731 /bgs/best1/iostat.25nov [Sent]
Selective Backup processing of 'iostat.25nov' finished without failure.

Total number of objects inspected:        1
Total number of objects backed up:        1
Total number of objects updated:          0
Total number of objects rebound:          0
Total number of objects deleted:          0
Total number of objects failed:           0
Total number of bytes transferred:    70.07 KB
Data transfer time:                    0.00 sec
Network data transfer rate:        10,909.58 KB/sec
Aggregate data transfer rate:         47.64 KB/sec
Objects compressed by:                    0%
Elapsed processing time:           00:00:01

File istat after selective backup :

 /bgs/best1 # istat iostat.25nov
Inode 2060 on device 39/1       File
Protection: rw-r--r--
Owner: 500(best1)               Group: 1(staff)
Link count:   1         Length 71731 bytes

Last updated:   Thu Nov 25 10:34:54 1999
Last modified:  Thu Nov 25 10:34:54 1999
Last accessed:  Mon Apr 10 14:15:55 2000

As you can see, the access time has change.

Regards
Gerrit


Reinhard Mersch wrote: Gerrit,

I can not reproduce that. In my environments, a "dsmmode -t p dsmc sel
file"
leaves the file's atime unchanged.

But even if the backup changes the atime: A file that has not been accessed
for 30 days should not have been backed up for 30 days because there was no
need to. So the atime should nevertheless be old. It looks quite strange to
me. What are you doing there?

Gerrit van Zyl writes:
 > Reinhard,
 >
 > Even using the dsmmode -t=p option when doing a selective backup,
changes the atime of the file.
 > I've just tested it.  Using "istat filename", one can see the access
time for that file.  The dsmmode
 > -t=p option does not preserve the access time as they claim.  So this is
a huge problem for me.  We can
 > only migrate files on our /R3_data filesystem if the files have not been
accessed for at least 30 days.
 > Any other ideas as on how to solve this problem?
 >
 > Regards
 > Gerrit
 >
 >
 > Reinhard Mersch wrote:
 >
 > > Gerrit van Zyl writes:
 > >  > Reinhard,
 > >  >
 > >  > I had a look:
 > >  >
 > >  > 1.  No errors in the AIX errorlog.
 > >  > 2.  No, I do not catch HSM error messages with the ERRORPROG
option.  How do you use this?
 > >
 > > What I am doing is: In dsm.sys:
 > >    ERRORPROG          /var/adsm/errorprog
 > > /var/adsm/errorprog is a shell script essentially containing one line:
 > >    mail -s "Mail from HSM" backup
 > > This sends the HSM error message to "backup" (which is me). Don't
forget
 > > to make that script executable.
 > >
 > >  > 3.  Dsmmonitord is running.
 > >  > 4.  This is a huge filesystem, and no, it is impossible that all of
the files have been accessed.
 > >
 > > You are backing them up, don't you? (See below.)
 > >
 > >  > 5.  Dsmreconcile gives the following:
 > >  > f03n13sw / # dsmreconcile -c /R3_data
 > >  > ADSTAR Distributed Storage Manager
 > >  > space management Interface - Version 3, Release 1, Level 0.7
 > >  > (C) Copyright IBM Corporation, 1990, 1999. All Rights Reserved.
 > >  >
 > >  >
 > >  > Reconciling '/R3_data' file system:
 > >  >   Querying the ADSM server for a list of backed up files...
 > >  >          Received 705752 entries
 > >
 > > Boy, this IS huge. There seem to be a lot of small files within the
 > > filesystem. Don't forget that small files (4KB or less) will not be
 > > migrated! Does this explain your problem?
 > >
 > >  >   Now traversing the file system...
 > >  >   Writing the new migration candidates list...
 > >  >          Note: unable to find any candidates in the file system.
 > >             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 > > Obviously, this is the problem. I can only guess about the reason. All
 > > possible candidates smaller than 4KB? To repeat myself: I would set
 > > AUTOMIGNOUSE to 0. APAR IX89650 might apply to you (text appended
below).
 > >
 > >  > ANS9250I File system '/R3_data' reconciliation completed.
 > >  >
 > >  > 6.  I am able to manually migrate files.
 > >  > 7.  Set the quota (e.g. "dsmmigfs update /R3_data -Quota=1024G"
 > >  >      you do allow 1 TB from this filesystem to be migrated, don't
you?)
 > >  >      What do you mean with this?
 > >
 > > The quota specifies the amount of data that may be migrated. You did
not
 > > specify a quota, I would definitely do so. (The default is quite
small.)
 > >
 > > APAR IX89650:
 > >
 > > APAR NUMBER: IX89650           RESOLVED AS: DOCUMENTATION ERROR
 > >
 > >  ABSTRACT:
 > >  IX89650: UNABLE TO SPACE MANAGE FILES BASED ON AUTOMIGNONUSE=1 AS
ADSM
 > >  INCREMENTAL AND SELECTIVE BACKUPS ALTER THE FILES ATIME VALUE.
 > >
 > >  ORIGINATING DETAILS:
 > >  The ADSM backup/archive client alters files atime value during
 > >  selective or incremental backups.  This results in HSM being
 > >  unable to properly migrate files based on the automignonuse
 > >  value set in the mgmtclass. This was tested on AIX 4.2.1 with
 > >  the 3.1.0.6 client code.
 > >
 > >  LOCAL FIX AS REPORTED BY ORIGINATOR:
 > >  Run the selective or incremental backup as a command parameter
 > >  of the dsmmode -t=p command. Example:
 > >  'dsmmode -t=p dsmc i'
 > >
 > >  RESPONDER SUMMARY:
 > >  The BA client in ADSM v2 preserved the atime
 > >  of files for backup and restore. Preserving the atime
 > >  of files for reading has been disabled 11/97. Reason:
 > >  when preserving the atime (using syscall utime(), the ctime
 > >  of the file was indirectly updated. HSM uses ctime as part of
 > >  the key for the premigration database. The ba client uses the
 > >  ctime (besides other file attributes) to compare the state
 > >  of files. Unfortunately, this modification has never been
 > >  reflected in the documentation,  since it limits the use of the
 > >  'non-usage' option of the ADSM HSM client.
 > >
 > >  RESPONDER CONCLUSION:
 > >  Trying to preserve the time stamp of a file,
 > >  using syscall utime(), indirectly changes the ctime time stamp,
 > >  which leads to undesired side affects.
 > >
 > > --
 > > Reinhard Mersch                        Westfaelische
Wilhelms-Universitaet
 > > Zentrum fuer Informationsverarbeitung - ehemals
Universitaetsrechenzentrum
 > > Roentgenstrasse 9-13, D-48149 Muenster, Germany      Tel:
+49(251)83-31583
 > > E-Mail: mersch AT uni-muenster DOT de                       Fax:
+49(251)83-31653
 > <!doctype html public "-//w3c//dtd html 4.0 transitional//en">
 > <html>
 > Reinhard,
 > <p>Even using the dsmmode -t=p option when doing a selective backup,
changes
 > the atime of the file.
 > <br>I've just tested it.&nbsp; Using "istat <i>filename", </i>one can
see
 > the access time for that file.&nbsp; The dsmmode -t=p option does not
preserve
 > the access time as they claim.&nbsp; So this is a huge problem for me.
&nbsp;
 > We can only migrate files on our /R3_data filesystem if the files have
 > not been accessed for at least 30 days.&nbsp; Any other ideas as on how
 > to solve this problem?
 > <p>Regards
 > <br>Gerrit
 > <br>&nbsp;
 > <p>Reinhard Mersch wrote:
 > <blockquote TYPE=CITE>Gerrit van Zyl writes:
 > <br>&nbsp;> Reinhard,
 > <br>&nbsp;>
 > <br>&nbsp;> I had a look:
 > <br>&nbsp;>
 > <br>&nbsp;> 1.&nbsp; No errors in the AIX errorlog.
 > <br>&nbsp;> 2.&nbsp; No, I do not catch HSM error messages with the
ERRORPROG
 > option.&nbsp; How do you use this?
 > <p>What I am doing is: In dsm.sys:
 > <br>&nbsp;&nbsp; ERRORPROG&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
&nbsp;&nbsp;
 > /var/adsm/errorprog
 > <br>/var/adsm/errorprog is a shell script essentially containing one
line:
 > <br>&nbsp;&nbsp; mail -s "Mail from HSM" backup
 > <br>This sends the HSM error message to "backup" (which is me). Don't
forget
 > <br>to make that script executable.
 > <p>&nbsp;> 3.&nbsp; Dsmmonitord is running.
 > <br>&nbsp;> 4.&nbsp; This is a huge filesystem, and no, it is impossible
 > that all of the files have been accessed.
 > <p>You are backing them up, don't you? (See below.)
 > <p>&nbsp;> 5.&nbsp; Dsmreconcile gives the following:
 > <br>&nbsp;> f03n13sw / # dsmreconcile -c /R3_data
 > <br>&nbsp;> ADSTAR Distributed Storage Manager
 > <br>&nbsp;> space management Interface - Version 3, Release 1, Level 0.7
 > <br>&nbsp;> (C) Copyright IBM Corporation, 1990, 1999. All Rights
Reserved.
 > <br>&nbsp;>
 > <br>&nbsp;>
 > <br>&nbsp;> Reconciling '/R3_data' file system:
 > <br>&nbsp;>&nbsp;&nbsp; Querying the ADSM server for a list of backed up
 > files...
 > <br>&nbsp;>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
Received
 > 705752 entries
 > <p>Boy, this IS huge. There seem to be a lot of small files within the
 > <br>filesystem. Don't forget that small files (4KB or less) will not be
 > <br>migrated! Does this explain your problem?
 > <p>&nbsp;>&nbsp;&nbsp; Now traversing the file system...
 > <br>&nbsp;>&nbsp;&nbsp; Writing the new migration candidates list...
 > <br>&nbsp;>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; Note:
 > unable to find any candidates in the file system.
 > <br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
 > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 > <br>Obviously, this is the problem. I can only guess about the reason.
 > All
 > <br>possible candidates smaller than 4KB? To repeat myself: I would set
 > <br>AUTOMIGNOUSE to 0. APAR IX89650 might apply to you (text appended
below).
 > <p>&nbsp;> ANS9250I File system '/R3_data' reconciliation completed.
 > <br>&nbsp;>
 > <br>&nbsp;> 6.&nbsp; I am able to manually migrate files.
 > <br>&nbsp;> 7.&nbsp; Set the quota (e.g. "dsmmigfs update /R3_data
-Quota=1024G"
 > <br>&nbsp;>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; you do allow 1 TB from this
 > <br>&nbsp;>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; you do allow 1 TB from this
filesystem
 > to be migrated, don't you?)
 > <br>&nbsp;>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; What do you mean with this?
 > <p>The quota specifies the amount of data that may be migrated. You did
 > not
 > <br>specify a quota, I would definitely do so. (The default is quite
small.)
 > <p>APAR IX89650:
 > <p>APAR NUMBER: IX89650&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
&nbsp;&nbsp;
 > RESOLVED AS: DOCUMENTATION ERROR
 > <p>&nbsp;ABSTRACT:
 > <br>&nbsp;IX89650: UNABLE TO SPACE MANAGE FILES BASED ON
AUTOMIGNONUSE=1
 > AS ADSM
 > <br>&nbsp;INCREMENTAL AND SELECTIVE BACKUPS ALTER THE FILES ATIME VALUE.
 > <p>&nbsp;ORIGINATING DETAILS:
 > <br>&nbsp;The ADSM backup/archive client alters files atime value during
 > <br>&nbsp;selective or incremental backups.&nbsp; This results in HSM
being
 > <br>&nbsp;unable to properly migrate files based on the automignonuse
 > <br>&nbsp;value set in the mgmtclass. This was tested on AIX 4.2.1 with
 > <br>&nbsp;the 3.1.0.6 client code.
 > <p>&nbsp;LOCAL FIX AS REPORTED BY ORIGINATOR:
 > <br>&nbsp;Run the selective or incremental backup as a command parameter
 > <br>&nbsp;of the dsmmode -t=p command. Example:
 > <br>&nbsp;'dsmmode -t=p dsmc i'
 > <p>&nbsp;RESPONDER SUMMARY:
 > <br>&nbsp;The BA client in ADSM v2 preserved the atime
 > <br>&nbsp;of files for backup and restore. Preserving the atime
 > <br>&nbsp;of files for reading has been disabled 11/97. Reason:
 > <br>&nbsp;when preserving the atime (using syscall utime(), the ctime
 > <br>&nbsp;of the file was indirectly updated. HSM uses ctime as part of
 > <br>&nbsp;the key for the premigration database. The ba client uses the
 > <br>&nbsp;ctime (besides other file attributes) to compare the state
 > <br>&nbsp;of files. Unfortunately, this modification has never been
 > <br>&nbsp;reflected in the documentation,&nbsp; since it limits the use
 > of the
 > <br>&nbsp;'non-usage' option of the ADSM HSM client.
 > <p>&nbsp;RESPONDER CONCLUSION:
 > <br>&nbsp;Trying to preserve the time stamp of a file,
 > <br>&nbsp;using syscall utime(), indirectly changes the ctime time
stamp,
 > <br>&nbsp;which leads to undesired side affects.
 > <p>--
 > <br>Reinhard Mersch&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
&nbsp;&nbsp;&nbsp;
 > Westfaelische Wilhelms-Universitaet
 > <br>Zentrum fuer Informationsverarbeitung - ehemals
Universitaetsrechenzentrum
 > <br>Roentgenstrasse 9-13, D-48149 Muenster, Germany&nbsp;&nbsp;&nbsp;
&nbsp;&nbsp;
 > Tel: +49(251)83-31583
 > <br>E-Mail: mersch AT uni-muenster DOT de&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
&nbsp;&nbsp;&nbsp;&nbsp;
 > Fax: +49(251)83-31653</blockquote>
 > </html>
 > begin:vcard
 > n:;
 > x-mozilla-html:FALSE
 > adr:;;;;;;
 > version:2.1
 > note;quoted-printable:                     ///=0D=0A
(o o)=0D=0A+----------oOO--(_)
------------+=0D=0A|                                      |=0D=0A| Gerrit
van Zyl                |=0D=0A| Snr Consultant              |=0D=0A|
van Zyl                |=0D=0A| Snr Consultant              |=0D=0A|
Faritec (Pty) Ltd            |=0D=0A| gvanzyl AT faritec.co DOT za  |=0D=0A|
0825704266                 |=0D=0A|
|=0D=0A+---------------------oOO-------+=0D=0A
|_|_|=0D=0A                    /| |\=0D=0A______
_ooO_Ooo___________=0D=0A=0D=0A
 > end:vcard

--
Reinhard Mersch                        Westfaelische Wilhelms-Universitaet
Reinhard Mersch                        Westfaelische Wilhelms-Universitaet
Zentrum fuer Informationsverarbeitung - ehemals Universitaetsrechenzentrum
Roentgenstrasse 9-13, D-48149 Muenster, Germany      Tel: +49(251)83-31583
E-Mail: mersch AT uni-muenster DOT de                       Fax: 
+49(251)83-31653

===========================================
This email message has been swept by
MIMESweeper at The Wesley Hospital - Australia
===========================================

(See attached file: gvanzyl.vcf)
<Prev in Thread] Current Thread [Next in Thread>