Networker

Re: [Networker] Windows 2000 not resetting the archive bit!!

2003-01-22 12:23:28
Subject: Re: [Networker] Windows 2000 not resetting the archive bit!!
From: "Maiello, Robert" <Robert.Maiello AT MEDEC DOT COM>
To: NETWORKER AT LISTMAIL.TEMPLE DOT EDU
Date: Wed, 22 Jan 2003 12:23:14 -0500
James,

Thanks, yes... we have some reservations about just ignoring the archive bit
with
the NSR_AVOID_ARCHIVE variable ... We may want to catch files that have been
moved
or such..  As you said, most backup products after they back the file up
turn off
the archive bit.   I know Legato presented me with one case where one
wouldn't
want this but it still seems that turning it off after the incremental would
work for
us.    Your idea of manually turning off the bits would work for me but
others at my
company would not think it elegant or right.

So far my choices seem to be:

a.) Do more full backups to clear the archive bits..  (eats tapes)
b.) Use the NSR_AVOID_ARCHIVE variable.. ( may work but others at my company
wonder what we'll miss in the backups).
c.) Use the NTFS change journal..  We had this on for a while. Because of
the arhive bit we
still got a huge incremental..but then the following incrementals were
small.  (eats less tape)
d.) Manually turn off the archive bit after incrementals or on an regular
interval.  (behaves like other backup products).

Having backed up unix for some time with Networker I must say its Windows
side is much more complicated.   So far I'm
leaning toward the change journal...

Robert Maiello


 -----Original Message-----
From:   Rohrich, James [mailto:James.Rohrich AT uop DOT com]
Sent:   Wednesday, January 22, 2003 11:30 AM
To:     'Legato NetWorker discussion'; 'Robert Maiello'
Subject:        RE: [Networker] Windows 2000 not resetting the archive bit!!

Some cheese have holes such as cheese as does some logic. It would seem that
tow types of backups can ignore dates as long as archive bits are turned on
and off appropriately. A full does not need to consider dates since
everything is backed up. An incremental using the archive bit only needs to
consider that and turn it off. The other types of backups are date
comparisons, what was backed up in a previous full or a partial such as a 1,
2, full, or incremental though it can use the archive bit as a safety net to
capture those from a different server. So it can compare the last
modification date to the last backup as specified by the type of backup, if
it has been modified since the last specified backup as determined by 1, 2,
etc. it should be backed up; however if the previous backup is not older
than the modification date, as a safety net the archive bit can be checked
and if it is on back it up and turn off the archive bit. Other logic can be
used to do the same thing, but once a backup has occurred the archive bit
should be turned off.

Having said that in the real world since I did not write the code, I have
attempted to have data migrated just before a full, but where necessary, I
have manually turned off all archive bits after a backup has completed to
stop the contradictory "incremental fulls".

-----Original Message-----
From: Robert Maiello [mailto:robert.maiello AT MEDEC DOT COM]
Sent: Wednesday, January 22, 2003 9:18 AM
To: NETWORKER AT LISTMAIL.TEMPLE DOT EDU
Subject: Re: [Networker] Windows 2000 not resetting the archive bit!!


Thanks Chuck and Koen,

Legato did get back to me about their "logic".  Quoted here it is:
--------
"Yes, if Networker finds a file with its archive bit turned on
yet, last modifed date did not change since the last backup
it will back it up at an incremental (incr,level1...level9) level
yet, it will not turn the archive bit off.

This works as designed as I said before. I have investigated
the issue more and found that there were good reasons
for this functionality:

- When files from some other systems are transferred to Windows NT, the
archive
  bit is turned on, yet the last modified date is not changed. If the
   backup date is later than the last modified date, those files
  would not be backed up. That is why the files with
  archive bit on are backed up even if they are not  modified.

- The reason Networker does not turn off the archive bit is if a low
  level back up follows a higher level incremental backup that turned
  off the archive bit, the file will not be backed up in the lower level
back up.
  Networker must ensure that all files which are supposed to be backed
  up at any level, be backed up even if this means more backup
  time or tapes until the next full backup.

  For example, if there is a level 5
  backup (suppose the files have an older modified date than that level 5
  yet, they are introduced to Win NT from another system, after level 5
date)
  and if incr backup turns the bit off, the next level 5 backup will not
  back up those files, thus defeating the purpose of incremental level
backups.

- The reason the full backup turns the archive bit off, is that no
incremental
  level (incr, level 1... level 9) can go beyond the full level. When an
  incremental backup (incr, 1 or 2 ... 9) is done, it will always be
compared
  to the last full backup.

This way Networker ensures that all files at all levels, regardless of their
origin (from any other platform) on Windows are backed up
as the backup level requires.

As I stated earlier, you can turn off archive bit behaviour by setting
NSR_AVOID_ARCHIVE=yes. This way the flexibility is provided to the
customer if the possibilities above are not a concern for the customer.
Yet, by default (NSR_AVOID_ARCHIVE=no) Networker must cover
the possibilities like those above."
-------

I must say I still don't agree with this logic.In effect
the incremental (or any other level besides a full) is meaningless to files
of this type..the whole backup level system is broken or bypassed...all to
catch files that missed a higer level backup (5) but made a lower level (0).
..from another machine.

In effect,  setting the archive bit on an already backed up file guarantees
that file will be backed up each and every night (until a full backup
is done...in our case, all month long). These files, in effect, become
differential backups (all files since the full backup) despite a different
level being specified...

Both NT backup (backup exec) and ArcServe clear the archive bit on
incrementals.   Anyone know how Veritas Netbackup handles this?

Robert Maiello
Thomson Healthcare


On Tue, 21 Jan 2003 19:16:07 -0500, Chuck Davis <networker AT DMSS.ARMY DOT MIL>
wrote:

>I went around on this one for a while myself.  I finally threw in the towel
>and decided to set the NSR_AVOID_ARCHIVE parameter on the client.  The huge
>incrementals stopped.  I have been doing this for about 1 year now with no
>problem.  This makes sense to me.  Their logic does not.
>
>This is using NT client 6.0.1, HP-UX server 6.1.2.
>
>YMMV.
>
>Chuck Davis
>Army Medical Surveillance Activity
>Walter Reed Army Medical Center
>Washington DC
>
>On Tue, 21 Jan 2003 09:52:01 -0500, Robert Maiello
><robert.maiello AT MEDEC DOT COM> wrote:
>
>>We've been getting giant incrementals on a Windows 2000 6.1.2 client;
>server
>>in Solaris 8, 6.1.2.
>>
>>Apparently some large permission changes were being made. The archive bit
>of
>>the files were changed by this.   The weekday incremental backups then
back
>>up these files but does not reset/clear the archive bit.  Each day then
the
>>same data gets backed up.   Only a full backup will clear these archive
>bits.
>>
>>I received the reply below from Legato...the infamous "working as
>designed".
>>I was wondering if anyone else backing up W2K is having this problem and
>>how you dealed with?  Did it always behave this way?
>>
>>I must say, for the life me, I don't understand Legato's logic.  The
>>definition of an incremental backup is everthing that's changed since the
>>last incremental...not backup the files forever as their archive bit is
set
>>(regardless of how it was set).
>>
>>I'm concerned also, that if we ignore the arhive bit then we'll miss files
>in
>>our backups?   Again, how is everyone handling this Networker "design".
>>
>>---
>>During incremental backups Networker
>>does not turn the archive bit off, if it is already on.
>>This is normal behaviour as designed. Please see below.
>>Therefore, we should make sure that the archive
>>bit was not turned on by somebody else or another
>>program, before the backup. If that is the case
>>then it is different from the expected behaviour.
>>
>>You may have to consider to set the environment variable
>>to yes.
>>
>>Here is the behaviour as designed:
>>
>>Handling archive bits depends on the value of the environment variable:
>>NSR_AVOID_ARCHIVE:
>>
>>1) NSR_AVOID_ARCHIVE=no *or* not present (default)
>>This is the new default behavior of scheduled backups on NT by
>>using the Archive file attribute as another factor in determining
>>if a file needs to be backed up incrementally in addition to the
>>Last Written, Creation time of a file. The Archive file attribute
>>is turned off based on the criteria listed below after a schedule
>>backup.
>>
>>   A) full scheduled backups
>>       archive-bit(Attribute file attribute) turned off if on
>>
>>   B) Last Written time *or* Creation time >= last scheduled backup time
>>       file is backed up, archive-bit turned off if on
>>
>>   C) Last Written time *and* Creation time < last scheduled backup time
>>        a) archive-bit is on
>>            file is backed up, archive-bit will not get turned off until
>>            the file is "captured" by a later future scheduled backup
>>            (typically case A above, but possibly case B)
>>        b) archive-bit is off
>>            file is not backed up
>>
>>2) NSR_AVOID_ARCHIVE=yes (any value other than case insensitive "no")
>>    This is the older behavior of scheduled backup without using the
>>    archive bit, except for older filesystem which used archive-bit as in
>(1).
>>
>>     A) full scheduled backups
>>
>>     B) Last Written time *or* Creation time >= last scheduled backup time
>>          file is backed up
>>
>>     C) Last Written time *and* Creation time < last scheduled backup time
>>          file is not backed up
>>
>>By specifying NSR_AVOID_ARCHIVE environment variable, one can turn
>>off the new default archive-bit backup behavior. After changing this
>>environment variable, the NetWorker/NT client must be rebooted to get the
>>proper effect.
>>
>>
>>Robert Maiello
>>Thomson Healthcare
>>
>>--
>>Note: To sign off this list, send a "signoff networker" command via email
>>to listserv AT listmail.temple DOT edu or visit the list's Web site at
>>http://listmail.temple.edu/archives/networker.html where you can
>>also view and post messages to the list.
>>=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=
>
>--
>Note: To sign off this list, send a "signoff networker" command via email
>to listserv AT listmail.temple DOT edu or visit the list's Web site at
>http://listmail.temple.edu/archives/networker.html where you can
>also view and post messages to the list.
>=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=

--
Note: To sign off this list, send a "signoff networker" command via email
to listserv AT listmail.temple DOT edu or visit the list's Web site at
http://listmail.temple.edu/archives/networker.html where you can
also view and post messages to the list.
=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=

________________________________________________________________________
This email has been scanned for all viruses by the MessageLabs SkyScan
service.
________________________________________________________________________


________________________________________________________________________

--
Note: To sign off this list, send a "signoff networker" command via email
to listserv AT listmail.temple DOT edu or visit the list's Web site at
http://listmail.temple.edu/archives/networker.html where you can
also view and post messages to the list.
=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=
The information contained in this communication may be confidential or legally 
privileged and is intended only for the recipient named above. If the reader of 
this message is not the intended recipient, you are hereby notified that any 
dissemination, distribution or copying of this communication or its contents is 
strictly prohibited. If you have received this communication in error, please 
immediately advise the sender and delete the original and any copies from your 
computer system.