Networker

Re: [Networker] Should core directories/files be skipped?

2007-09-12 13:48:28
Subject: Re: [Networker] Should core directories/files be skipped?
From: Darren Dunham <ddunham AT TAOS DOT COM>
To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
Date: Wed, 12 Sep 2007 09:34:46 -0700
> Hi,
> 
> By default, the 'Unix standard directives' skips all core files as:
> 
> << / >>
>         skip: tmp_mnt
>         +skip: core
> 
> This is the directive I usually use for most clients, but I've found 
> that this causes any file or directory with the name 'core' to get 
> skipped (not surprising, of course), even if it lives under some other 
> file system other than / (that seems kinda surprising).

Yes.  The string in << >> specifies a directory, not a filesystem.  And
+ specifies continuation to children, not limited by filesystems.

> I would have 
> thought that in order for a directory named 'core' to get skipped under, 
> say, '/export/disk2/core', assuming /disk2 is a separate file system, 
> that there would need to be another entry for this in the directive but 
> apparently not.

No.  See the man page for directives, nsr(5).
[...]
     The following algorithm  is  used  to  match  files  to  the
     appropriate  ASM  specification.  First the .nsr file in the
     current directory (if any) is scanned from top to bottom for
     an  ASM  specification  without  a leading "+" whose pattern
     matches the file name.  If no match is found, then the  .nsr
     in the current directory is re-scanned for an ASM specifica-
     tion with a leading "+" whose pattern matches the file  name
     (for  clarity,  we  recommend  placing all propagating ("+")
     directives after all the  non-propagating  directives  in  a
     .nsr  file).  If no match is found, then the .nsr file found
     in the ".." directory (if any) is scanned from top to bottom
     looking  for  a  match  with an ASM specification that has a
     leading +.  This process continues until the  .nsr  file  in
     the "/" directory (if any) is scanned.  If no match is found
     (or a match is found with an ASM specification whose name is
     the  same  as the currently running ASM), then the currently
     running ASM will handle the save of the file.

While this talks about '.nsr' files, it applies equally to directives,
but consider a directory specification (like << / >>) to be equivalent
to the contents being in the .nsr file in that directory.

> Anyway, maybe skipping core files is not a bad idea 
> (some can be quite large), but maybe we should still remove the line 
> that does this so in case a user, or some software package, creates a 
> directory named 'core' it will get backed up? For example, consider the 
> following directory:
> 
> /var/apache/tomcat/webapps/tomcat-docs/catalina/docs/api/org/apache/catalina/core
> 
> This does contain data files. Also, I've seen some core directories, 
> under various OS patch install directories, for Java, too, and there are 
> a lot of directories on Linux clients named 'core' that are all buried 
> down umpteen levels like 
> /usr/src/kernels/2.6.9-42.0.10.EL-i686/net/core, etc.  Hmm ...

Yup.

> Any consensus on this? Any way to skip 'core' files but not directories? 
> Maybe just remove the line from the directive and get these previously 
> skipped areas backed up on the next fulls?

Education works in some cases (telling folks that files named 'core' and
'tmp' are not backed up.  Also, you could scan your filesystems
periodically to look for files name 'core' that are not recognized by
'file' as a corefile.

In some cases it might be easier to disable cores (or change the
settings so that cores are only placed in limited locations), and then
not have to worry about excluding them.

Partially I think this is an environmental thing.  NetBackup and
Networker have origins in a day when most UNIX boxes were also used for
development, and most users were very familiar with core and temporary
editor files.  So it was natural for backup systems to exclude them by
default.  The expectations of most system users are probably different
than they were in those days, so maybe the default directives don't make
sense for those environments any longer.  


-- 
Darren Dunham                                           ddunham AT taos DOT com
Senior Technical Consultant         TAOS            http://www.taos.com/
Got some Dr Pepper?                           San Francisco, CA bay area
         < This line left intentionally blank to confuse you. >

To sign off this list, send email to listserv AT listserv.temple DOT edu and 
type "signoff networker" in the body of the email. Please write to 
networker-request AT listserv.temple DOT edu if you have any problems with this 
list. You can access the archives at 
http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER