ADSM-L

Problems in Using ADSM for UNIX Backups and Restores

1994-11-29 09:45:34
Subject: Problems in Using ADSM for UNIX Backups and Restores
From: Melinda Varian <[email protected]>
Date: Tue, 29 Nov 1994 09:45:34 EST
On Tue, 29 Nov 1994 at 10:50:08 +0100, Reinhard Mersch said:

> I just retrieved some files and discovered that symbolic links are not
> retrieved as such; the files they point to are retrieved instead. I
> consider this to be a bug. Any opinions?

Here at Princeton, we have just completed a rather extensive test of
using ADSM to back up and restore SUNs.  We encountered four problems:

1. As you mention, symbolic links are not restored.

2. The last access date/time of normal files is not restored.

3. The last modification date/time and last access date/time of
   directories are not restored.

4. Special files (character and block) are not restored.

Here's what we have done about these problems so far:

1. We opened an incident on the symbolic link problem.  After much
   discussion, we have closed this as a suggestion for future improve-
   ment.  The basic problem is that ADSM cannot "see" below a mount
   point using normal operating system interfaces.  My colleagues had
   several suggestions, however, on ways in which ADSM could be improved
   in this area (see their notes which I will append below).  At the very
   least, the existence of the mount point could be noted and a warning
   message issued during both backup and restore.

2. We have an incident open with regard to the file access dates.  We
   consider this to be a very serious problem.

3. The same incident also addresses the directory access and modification
   dates.  We consider this to be such a serious problem that we will not
   be using ADSM for UNIX systems unless it is fixed.

4. The problem of not restoring special files is documented as an ADSM
   restriction, but we believe that it would be trivial to fix (see
   below for a discussion from one of my colleagues).  I intend to
   try to get a "SUGG" APAR on this one, too.

I'll also append some other notes from colleagues discussing the details
of our ADSM UNIX client tests.

Melinda Varian,
Princeton University

<>  <>  <>  <>  <>  <>  <>  <>  <>  <>  <>  <>  <>  <>  <>  <>  <>  <>

Date: Mon, 7 Nov 94 14:20:03 -0500
From: "Serge J. Goldstein" <serge AT dadofsam.Princeton DOT EDU>
To: maint AT pucc.Princeton DOT EDU
Subject: What a Unix "restore" tries to do

Here is what "restore" attempts to restore for various flavors of Unix
files:

For character and block special devices:
        name of file
        type (character or block special device)
        owner (uid)
        group (gid)
        mode (16 permission/mode/type bits)
        last acces time
        last modification time
        major device number
        minor device number

All of the above is done with a "mknod" call, which sets the name of the
file and its type (character or block special device), followed by a
chmod and chown call (to set the modes and owner and group), followed by
a utimes call (to set the last access time and last modification time).

For regular files:
        name of file
        contents of file
        owner (uid)
        group (gid)
        mode (16 permission/mode/type bits)
        last access time
        last modification time

All of the above is done with a "creat" call (to make the file and set
its name), followed by writes to set the contents, followed by chown,
chmod and utimes calls to set the owner/group, modes and permissions.

For directories:
        name of file
        contents of file
        owner (uid)
        group (gid)
        mode (16 permission/mode/type bits)
        last access time
        last modification time

Same comments as for regular files, except that an interactive restore
in Unix actually asks the user if he/she wants to reset directory
times/owner/group/modes ... a batch restore just does it.

For symbolic links:
        name of file

Symlinks are just rebuilt with a symlink system call ... no attempt is
made to reset owner/group, times, modes etc (modes on a symlink file are
always 777).

Serge

<>  <>  <>  <>  <>  <>  <>  <>  <>  <>  <>  <>  <>  <>  <>  <>  <>  <>

Date: Wed, 2 Nov 94 16:34:45 -0500
From: "Serge J. Goldstein" <serge AT dadofsam.Princeton DOT EDU>
To: maint AT pucc.Princeton DOT EDU
Subject: glitch with restoring Unix file systems

Let me know if this is clear:

Let's say we've got a Unix machine which has two (2) filesystems, one on
a disk called /dev/sd0a, another on a disk called /dev/sd1a.  Now device
/dev/sd0a comprises the root filesystem, and is mounted as "/".  Now we
go to mount our second disk.  To do that, we have to create a "dummy"
directory that will serve as a mount point.  So we do soemthing like:

mkdir /usr

We've now created a directory called "usr" in the "/" filesystem.  We now
issue the mount command to mount /dev/sd1a on top of the "usr" directory.

mount /dev/sd1a /usr

Now, when we refer to "/usr", we are actually referring to the second
filesystem (the one on /dev/sd1a).  So far so good.

Now the problem.  When we ask adsm to dump all of the "/" filesystem, it
correctly discerns that "/usr" is on a different filesystem than "/" and
does not dump it.  However, the problem is that adsm fails to dump the
original, dummy directory called "usr" on which we mounted /dev/sd1a.  If
we loose all of /dev/sd0a, and we restore it from adsm, adsm will NOT
restore that dummy "usr" directory, so that later, our attempt to mount
/dev/sd1a on "/usr" will fail, because there is no dummy "/usr" directory
on which to mount.  Most flavors of Unix require that a directory exist
before you mount on top of it (NextStep will actually create the dummy
directory for you as part of the mount command, but that is an
exception).

The problem is complicated by the fact that Unix let's you mount a file
system on top of ANY directory, even one that is not empty ... thus one
cannot solve the above problem by simply creating a dummy directory in
the adsm backup ... you have to actually look UNDER the mount point and
dump the original directory (and any files and sub-directories that might
be in it).  I don't know if standard Unix file semantics will let you do
that.  The Unix "dump" command does exactly that ... it ignores the mount
points and dumps the directories/files under the mount points.  It is
quite possible to have an entire, extensive directory stucture underlying
a mount point, and it is important that this structure be dumped when the
whole filesystem is dumped.

Serge

<>  <>  <>  <>  <>  <>  <>  <>  <>  <>  <>  <>  <>  <>  <>  <>  <>  <>

Date: Mon, 21 Nov 1994 11:58:21 -0500 (EST)
From: clk AT pseudo.Princeton DOT EDU (Christopher Kranz)
To: MAINT AT pucc.Princeton DOT EDU (Melinda Varian)
Subject: Re: ADSM SUN Client and mount points

[From the keyboard of Melinda Varian . . . . ]

| I need to respond to IBM on our incident concerning the failure of
| ADSM to back up mount points.  I believe that we are all agreed that
| using normal operating system interfaces they can't see the directory
| (and any files it contains) "behind" the mount point.  However, they
| presumably can detect that there is a mount point.
|
| Should we close the incident with a suggestion for a future enhance-
| ment?  If so, what should our suggestion be?
|
| Melinda

Melinda,  They can determine the name of the mount point a couple of
different ways.

One way would be to run /etc/mount and parse the output.  This would
probably be the most accurate way of determining the name of the mount
point.  Another way would be to parse /etc/fstab looking for mount
points.

Figuring out what what id owns the mount point and its group id as well
as its permissions is not easily done.  Defaulting to root (user id zero)
ownership and setting the group as daemon (group id one) with permissions
set to 755 (drwxr-xr-x) would be a good first cut.

If there is a configuration file for ADSM maybe mount point ownership,
group id, and permissions could be added to that.  They might also want
to consider adding comment directives to /etc/fstab that they could then
parse.

For example, this could be a typical set of /etc/fstab entries:

#########################################################################
# Local disk partitions                                                 #
#########################################################################

/dev/sd0a       /       4.2     rw,noquota      1 1
#ADSM dirname=/usr owner=0 group=1 perms=0755
/dev/sd0g       /usr    4.2     rw,noquota      1 2
#ADSM dirname=/var owner=0 group=1 perms=0755
/dev/sd0d       /var    4.2     rw,noquota      1 3

Anyway, that's my $.02.

        Chris

<>  <>  <>  <>  <>  <>  <>  <>  <>  <>  <>  <>  <>  <>  <>  <>  <>  <>

Date: Thu, 17 Nov 94 10:47:42 -0500
From: "Serge J. Goldstein" <serge AT dadofsam.Princeton DOT EDU>
To: Melinda Varian <MAINT AT pucc.Princeton DOT EDU>
Subject: Re: ADSM SUN Client and mount points

I think it needs to be documented as a restriction.  Further, they should
dump the mount point itself as a "dummy" directory, with the same owner/
group and permissions and dates as the real mount point.  That way a
restore would at least re-create the mount point directory (although it
would not recover any data in/under that directory).

Serge

<>  <>  <>  <>  <>  <>  <>  <>  <>  <>  <>  <>  <>  <>  <>  <>  <>  <>

Date: Wed, 2 Nov 94 17:10:11 -0500
From: "Serge J. Goldstein" <serge AT dadofsam.Princeton DOT EDU>
To: melinda AT pucc.Princeton DOT EDU
Subject: adsm

Melinda,

I can't see any reason why adsm would not dump the character and block
special devices.  These are truly "dummy" files which have just 6 items
of information associated with them ... their user and group ownership
(two numbers), their permission (one long number), their type (chracter
or block), and their major and minor device numbers (two numbers).  If
you save those items of information, you can "restore" the special device
file simply by issuing the "mknod" command (to create the special device
file, set its type, and set its major and minor device numbers), followed
by a "chown" (to set user and group ownership) and "chmod" (to set
permissions).  This is essentially all that the MAKEDEV script does on
any Unix system.  In other words, special device files involve less
information and effort to save/restore than any other kind of Unix file.

Serge

<>  <>  <>  <>  <>  <>  <>  <>  <>  <>  <>  <>  <>  <>  <>  <>  <>  <>

Date: Wed, 23 Nov 94 11:35:00 -0500
From: "Serge J. Goldstein" <serge AT dadofsam.Princeton DOT EDU>
To: maint AT pucc.Princeton DOT EDU, lrr AT daytona.Princeton DOT EDU,
        lvarian AT pucc.Princeton DOT EDU
Subject: adsm dumps versus budtool (dump/restore)

We've finished our test of adsm on sun3x (sun).  Here's what we found:

First, the good news:

1) All regular files are correctly restored in content, group and user
   ownership.  All permissions are correctly restored. The last
   modification time is restored.

2) All directories EXCEPT directories used as mount points are rebuilt.

3) All symbolic links are rebuilt (their ownership/dates are not
   restored, but the unix dump/restore doesn't restore these either).

Now, the bad news:

1) Special files (character and block) are not restored.  This is a
   documented restriction of adsm, but one that makes little sense, since
   this is the easiest of all files to save/restore.

2) Regular files have their last modification date restored.  However,
   their last access date is NOT restored ... it is reset to the restore
   date/time.  The Unix dump/restore facility restores both the last
   modification and last access times.  The same system call is used to
   restore both last modification and last access time (utimes), so it is
   puzzling to find adsm restoring only one of these ...  it would have
   to be a deliberate decision, rather than an oversight.  I vaguely
   recall seeing something in the adsm Unix documentation which stated
   that last modification date/time was restored, but said nothing about
   last access time.  Neither adsm nor dump/restore restore the last
   inode change time.

3) Directories do not have either their last modification or last access
   times restored --- this is an error (the same utimes system call can
   be used to reset times for both directories and regular files).  It
   seems that this is an oversight (the ownership and permissions ARE
   restored, so all that would be needed would be an additional utimes
   call to restore the dates).

I'd say that 2) and 3) would need to be fixed before we can use adsm
for full dumps of Unix systems (I think the last access time is pretty
critical, and directory times have to be restored).  1) isn't critical at
all, as we can usually rebuild the device files via MAKEDEV, but it is
not clear why adsm doesn't back these up.

Serge
<Prev in Thread] Current Thread [Next in Thread>