ADSM-L

Re: Archive retention for directories

1999-03-18 11:35:20
Subject: Re: Archive retention for directories
From: Paul Zarnowski <vkm AT CORNELLC.CIT.CORNELL DOT EDU>
Date: Thu, 18 Mar 1999 11:35:20 -0500
At 12:44 PM 3/18/99 +0100, Grendelman M. (Martijn) wrote:
>Hi,
>
>I am sorry for asking this question on the list, coz it has been covered
>before. All I could find in the archives however, was the same question and
>a lot of me-too's and I had some trouble going back in the archives any
>further than a month. Could someone post a definate answer to this one...
>
>The problem concerns the retention period of archived directories. I know
>that archived directories get bound to the management class with the longest
>retention period in the active policy set. But that's not what I want. I
>have two management classes, one with a longer retention period than the
>other, and I would like all objects in a certain archive (including
>directories) to be bound to the same management class, being the one I
>specify with "archmc=...". Is this at all possible or will I have to live
>with it (or delete the obsolete archives manually)?
>
>Thanx,
>Martijn.

No need to apologize, Martijn.  Your analysis is correct.  This has indeed
been brought up on the list before, but without any resolution or response
from IBM.  I am still struggling with the new archive logic that was
introduced in version 3, and have a couple of PMRs open on the topic.

First, the short answer to your question is, I believe, that you will have
to live with the current behaviour.  I do not believe there is a good
workaround available for the situation you describe.

Here is what I have found so far.  Others who make use of archiving may
be interested in this.

1. In version 3, when a package of files (or a single file) is archived,
the directory containers for those files are also archived.  This is new
behavior in version 3, and was done so that the directory structure for
files can be recreated when doing a retrieve.  This enhancement was done
to satisfy requirements from the ADSM user community.  Unfortunately, the
design change had some undesireable side effects (from my point of view).

2. If you archive N files from the same directory (or N
different files from the same directory), in N separate archive sessions,
ADSM will end up archiving N separate copies of the directory container.
The following example illustrates this, with N=2:

  dsmc> archive /tmp/foo                     (archive file first time)
  Archive function invoked.

  Directory-->               6,656 /tmp/ [Sent]
  Normal File-->                 4 /tmp/foo [Sent]
  Archive processing of '/tmp/foo' finished without failure.


  Total number of objects inspected:       27
  Total number of objects archived:         2
  Total number of objects updated:          0
  Total number of objects rebound:          0
  Total number of objects deleted:          0
  Total number of objects failed:           0
  Total number of bytes transferred:       27
  Data transfer time:                    0.00 sec
  Network data transfer rate:           73.85 KB/sec
  Aggregate data transfer rate:          0.01 KB/sec
  Objects compressed by:                    0%
  Elapsed processing time:           00:00:02
  dsmc> archive /tmp/foo -archmc=1year              (archive file 2nd time)
  Archive function invoked.

  Directory-->               6,656 /tmp/ [Sent]
  Normal File-->                 4 /tmp/foo [Sent]
  Archive processing of '/tmp/foo' finished without failure.


  Total number of objects inspected:       27
  Total number of objects archived:         2
  Total number of objects updated:          0
  Total number of objects rebound:          0
  Total number of objects deleted:          0
  Total number of objects failed:           0
  Total number of bytes transferred:       27
  Data transfer time:                    0.00 sec
  Network data transfer rate:          148.13 KB/sec
  Aggregate data transfer rate:          0.00 KB/sec
  Objects compressed by:                    0%
  Elapsed processing time:           00:00:02

  dsmc> q archive /tmp/* -su=yes
        Size  Archive Date - Time    File - Expires on - Description
        ----  -------------------    -------------------------------
       6,656  03/18/99   10:49:09    /tmp/ Never Archive Date: 03/18/99
                                       Time: 10:49:08
       6,656  03/18/99   10:49:19    /tmp/ Never Archive Date: 03/18/99
                                       Time: 10:49:17
           4  03/18/99   10:49:09    /tmp/foo Never Archive Date: 03/18/99
                                       Time: 10:49:08
           4  03/18/99   10:49:19    /tmp/foo 03/17/00   Archive Date: 03/
                                       18/99    Time: 10:49:17

Note that both directory containers are archived with the management class
having an unlimited retention period, while only 1 of the files are.  When
the file having a 1 year retention period expires, the directory container
that was archived with it will remain in ADSM, forever.  This is one of the
behaviors which I am attempting to APAR.  Given that each archive package
gets its own set of directory containers archived with it, I can see no
reason to not simply associate the directory containers with the same
management class as the files.

3. If you do a DELETE ARCHIVE of a file (or all of the files in a package),
the associated directory containers do NOT get deleted.  They will remain.
Here is another example that illustrates this:

  dsmc> del archive foo

  Archived files will be deleted.  Do you wish to proceed? (Yes/No) yes
  Archive Delete->               4 /tmp/foo [Sent]
  Archive Delete->               4 /tmp/foo [Sent]

  Archive Delete processing finished.
  dsmc> q archive /tmp/* -su=yes
          Size  Archive Date - Time    File - Expires on - Description
          ----  -------------------    -------------------------------
         6,656  03/18/99   10:49:09    /tmp/ Never Archive Date: 03/18/99
                                        Time: 10:49:08
         6,656  03/18/99   10:49:19    /tmp/ Never Archive Date: 03/18/99
                                        Time: 10:49:17

Note that the two files were deleted, but the directory containers remain
archived.  For anyone used to using ADSM version 2 to archive files, this
will present a problem.  Users now have to be aware of this new behavior,
or -more likely- just ignore it since it does not directly effect them, in
which case these directory turds get left behind in your ADSM database.
The impact of this will depend on how much archiving you do and whether
you relied on the version 2 behavior or not.

I would argue that after the last file of a package is deleted, the
related directory containers should be automatically deleted by ADSM.

IBM has suggested a workaround of using DEL ARCHIVE -SU=YES as a way of
getting rid of the directory containers.  This, however, only works if
you want to delete ALL of the files in the directory and not just a
subset of them.  Another suggestion made was to use the description field
as a way of identifying the files and directories to be deleted.  While
this would work, the overhead for our users in doing this is significanly
higher than just doing a simple DEL ARCHIVE <filename>.  First they would
have to do a query to determine the appropriate description, then issue
the DEL ARCHIVE with the correct syntax.  And, understand all the related
issues, such as what if they only want to delete 1 file out of a package,
and not all of them.

I am sure that the new archive behaviour has satisfied some user
requirements, but for us the new behaviour creates a bunch of problems.
IBM is telling me that my requests to "fix" this new behaviour are really
requests for a design change.  My point of view is that they changed the
product in a way that broke it for our usage - they made a non-upward-
compatible change - and I am looking for them to fix what they broke.
I would be interested in hearing how others on this list feel about this.
Thanks.

..Paul

PS:  If you look in the manual and find -DIRSONLY and -FILESONLY, and think
they may be useful in addressing this issue, think again.  IBM tells me
that they are removing these options from ARCHIVE.  If you actually try
using the option, you will see that it does not work.  To illustrate:

  dsmc> archive foo -filesonly
  Archive function invoked.

  Session established with server ADSM1: AIX-RS/6000
    Server Version 3, Release 1, Level 2.13
    Data compression forced on by the server
    Server date/time: 03/18/99   11:34:03  Last access: 03/18/99   10:40:41

  Directory-->               6,656 /tmp/ [Sent]   <-- directory is sent anyway
  Normal File-->                 4 /tmp/foo [Sent]
  Archive processing of 'foo' finished without failure.


  Total number of objects inspected:        2
  Total number of objects archived:         2
  Total number of objects updated:          0
  Total number of objects rebound:          0
  Total number of objects deleted:          0
  Total number of objects failed:           0
  Total number of bytes transferred:       27
  Data transfer time:                    0.00 sec
  Network data transfer rate:           88.18 KB/sec
  Aggregate data transfer rate:          0.00 KB/sec
  Objects compressed by:                    0%
  Elapsed processing time:           00:00:27
<Prev in Thread] Current Thread [Next in Thread>