Bacula-users

Re: [Bacula-users] [Bacula-devel] Can bacula change the ctime on files during a backup?

2008-08-06 18:08:21
Subject: Re: [Bacula-users] [Bacula-devel] Can bacula change the ctime on files during a backup?
From: Hemant Shah <hjrrs AT yahoo DOT com>
To: bacula-devel AT lists.sourceforge DOT net, Kern Sibbald <kern AT sibbald DOT com>
Date: Wed, 6 Aug 2008 15:08:09 -0700 (PDT)

--- On Wed, 8/6/08, Kern Sibbald <kern AT sibbald DOT com> wrote:

> From: Kern Sibbald <kern AT sibbald DOT com>
> Subject: Re: [Bacula-devel] Can bacula change the ctime on files during a 
> backup?
> To: bacula-devel AT lists.sourceforge DOT net, hjrrs AT yahoo DOT com
> Cc: "baculausers" <bacula-users AT lists.sourceforge DOT net>
> Date: Wednesday, August 6, 2008, 12:30 PM
> On Wednesday 06 August 2008 18:53:10 Hemant Shah wrote:
> > --- On Tue, 7/22/08, Hemant Shah
> <hjrrs AT yahoo DOT com> wrote:
> > > From: Hemant Shah <hjrrs AT yahoo DOT com>
> > > Subject: Re: [Bacula-devel] Can bacula change the
> ctime on files during a
> > > backup? To: bacula-devel AT lists.sourceforge DOT net
> > > Date: Tuesday, July 22, 2008, 3:27 PM
> > > --- On Tue, 7/22/08, Kern Sibbald
> <kern AT sibbald DOT com>
> > >
> > > wrote:
> > > > From: Kern Sibbald <kern AT sibbald DOT com>
> > > > Subject: Re: [Bacula-devel] Can bacula
> change the
> > >
> > > ctime on files during a backup?
> > >
> > > > To: bacula-devel AT lists.sourceforge DOT net
> > > > Date: Tuesday, July 22, 2008, 1:25 PM
> > > >
> > > > On Tuesday 22 July 2008 18:37:14 John
> Drescher wrote:
> > > > > There is a long thread in the users
> list that a
> > >
> > > user
> > >
> > > > is getting nearly
> > > >
> > > > > full backups for his incrementals with
> > >
> > > bacula-2.4.X
> > >
> > > > and it was
> > > >
> > > > > discovered that the reason was that the
> ctime on
> > >
> > > files
> > >
> > > > was changing
> > > >
> > > > > between backups. He believes that
> bacula is doing
> > >
> > > this
> > >
> > > > but I am not so
> > > >
> > > > > sure.
> > > >
> > > > Almost anything is possible, but there have
> been no
> > >
> > > changes
> > >
> > > > in the way Bacula
> > > > does backups for quite a few versions, so I
> doubt that
> > > > Bacula is doing it.
> > > > Typically it is virus protection software.
> > > >
> > > > If he can provide proof positive, i.e.
> disable all
> > >
> > > software
> > >
> > > > other than Bacula
> > > > do a backup and show that file times change,
> then that
> > > > might be worth looking
> > > > at.  Also, if it is one user only one user
> having the
> > > > problem, it points the
> > > > finger elsewhere.  Finally, I am running
> 2.4.1 here
> > >
> > > and
> > >
> > > > have been since
> > > > before it was released, and I am seeing
> normal
> > >
> > > Incremental
> > >
> > > > backups.
> > > >
> > > > Thanks for mentioning this ...  I'll
> take a look
> > >
> > > at the
> > >
> > > > thread.
> > > >
> > > > Best regards,
> > > >
> > > > Kern
> > >
> > >
> -------------------------------------------------------------------------
> > >
> > > > This SF.Net email is sponsored by the Moblin
> Your Move
> > > > Developer's challenge
> > > > Build the coolest Linux based applications
> with Moblin
> > >
> > > SDK
> > >
> > > > & win great prizes
> > > > Grand prize is a trip for two to an Open
> Source event
> > > > anywhere in the world
> > >
> > >
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> > >
> > > >
> _______________________________________________
> > > > Bacula-devel mailing list
> > > > Bacula-devel AT lists.sourceforge DOT net
> > >
> > >
> https://lists.sourceforge.net/lists/listinfo/bacula-devel
> > >
> > >
> > >   I upgraded the server to 2.4.1. I will run
> another full
> > > cycle starting today. I have also written a
> cronjob that
> > > will run every hour and stat few files, some on
> local disk
> > > and some on NFS filesystems. I am having problem
> on bacula
> > > server so I will be running the script on bacula
> server
> > > machine which only runs bacula, and postgresql.
> > >
> > >
> > > Will see if I get problem with 2.4.1.
> > >
> > > Hemant Shah
> > > E-mail: hjrrs AT yahoo DOT com
> > >
> > >
> > >
> > >
> > >
> > >
> -------------------------------------------------------------------------
> > > This SF.Net email is sponsored by the Moblin Your
> Move
> > > Developer's challenge
> > > Build the coolest Linux based applications with
> Moblin SDK
> > > & win great prizes
> > > Grand prize is a trip for two to an Open Source
> event
> > > anywhere in the world
> > >
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> > > _______________________________________________
> > > Bacula-devel mailing list
> > > Bacula-devel AT lists.sourceforge DOT net
> > >
> https://lists.sourceforge.net/lists/listinfo/bacula-devel
> >
> > Folks,
> >
> >   It did it again. Last weekend was another start of
> the month full backup,
> > then it was time to to the incremental backup, bacula
> did a full or almost
> > full backup again, only on some clients, and few were
> not same as last
> > time.
> >
> > Here are the clients that had problem:
> >
> > -rw-r----- 1 root root 1.9G 2008-08-02 16:07
> Full-lidp5-2008-08-02-15:46:51
> > -rw-r----- 1 root root 1.8G 2008-08-03 22:23
> Incr-lidp5-2008-08-03-22:03:06
> > -rw-r----- 1 root root 3.7M 2008-08-04 22:02
> Incr-lidp5-2008-08-04-22:02:33
> > -rw-r----- 1 root root 3.0M 2008-08-05 22:03
> Incr-lidp5-2008-08-05-22:03:05
> >
> >
> > -rw-r----- 1 root root 19G 2008-08-02 17:39
> Full-lidp8-2008-08-02-15:48:08
> > -rw-r----- 1 root root 19G 2008-08-04 00:05
> Incr-lidp8-2008-08-03-22:05:13
> > -rw-r----- 1 root root 45M 2008-08-04 22:19
> Incr-lidp8-2008-08-04-22:02:58
> > -rw-r----- 1 root root 15M 2008-08-05 22:20
> Incr-lidp8-2008-08-05-22:03:29
> >
> >
> > -rw-r----- 1 root root 5.5G 2008-08-02 18:59
> > Full-lidp10-2008-08-02-17:24:03 -rw-r----- 1 root root
> 3.7G 2008-08-03
> > 23:28 Incr-lidp10-2008-08-03-22:05:34 -rw-r----- 1
> root root  17M
> > 2008-08-04 22:05 Incr-lidp10-2008-08-04-22:03:34
> -rw-r----- 1 root root 
> > 17M 2008-08-05 22:06 Incr-lidp10-2008-08-05-22:04:36
> >
> > -rw-r----- 1 root root  58G 2008-08-02 22:36
> > Full-lidp11-2008-08-02-17:50:02 -rw-r----- 1 root root
>  56G 2008-08-04
> > 02:52 Incr-lidp11-2008-08-03-22:12:39 -rw-r----- 1
> root root 3.0G
> > 2008-08-04 22:42 Incr-lidp11-2008-08-04-22:05:15
> -rw-r----- 1 root root
> > 3.8G 2008-08-05 22:52 Incr-lidp11-2008-08-05-22:05:37
> >
> >
> > -rw-r----- 1 root root 6.4G 2008-08-02 21:22
> > Full-lidp20-2008-08-02-19:30:22 -rw-r----- 1 root root
> 5.8G 2008-08-03
> > 23:45 Incr-lidp20-2008-08-03-22:13:43 -rw-r----- 1
> root root  30M
> > 2008-08-04 22:15 Incr-lidp20-2008-08-04-22:07:51
> -rw-r----- 1 root root 
> > 30M 2008-08-05 22:16 Incr-lidp20-2008-08-05-22:08:27
> >
> > -rw-r----- 1 root root 6.7G 2008-08-02 21:56
> > Full-lidp22-2008-08-02-21:09:01 -rw-r----- 1 root root
> 4.6G 2008-08-03
> > 23:28 Incr-lidp22-2008-08-03-22:47:34 -rw-r----- 1
> root root  19M
> > 2008-08-04 22:18 Incr-lidp22-2008-08-04-22:15:49
> -rw-r----- 1 root root 
> > 19M 2008-08-05 22:19 Incr-lidp22-2008-08-05-22:16:27
> >
> >
> > I am backing up 26 clients (AIX, HP-UX, Linux, Win2k
> server), problem only
> > occurs on some clients, and they are not same clients
> every time. The
> > problem has never occured on a AIX or windows client.
> >
> > I am also running veritas netbackup in parallel (my
> old backup system). I
> > am  reluctantly attributing the problem to the
> netbackup, because if
> > netbackup changes ctime why doesn't it change it
> on all the systems and why
> > not same systems every time. In many cases bacula does
> almost full backup,
> > if netbackup is changing ctime it would change it on
> all files and bacula
> > would backup all files too.
> 
> Perhaps there is some configuration parameter that is not
> set correctly, and 
> when it references the files, it does not always reset the
> time/date 
> correctly on some particular backup level ...
> 
> I would recommend that you talk to Veritas about the
> problem without 
> mentioning Bacula.  Unless you configure Bacula
> differently, it checks both 
> mtime and ctime, so if netbackup modified either one of
> them, Bacula would 
> treat the file as having changed.  If you configure Bacula
> differently, you 
> can make it ignore ctime changes.

  I did more research and found that Netbackup has 
DO_NOT_RESET_FILE_ACCESS_TIME option which is not enabled on any of the 
systems, so it is likely that netbackup is changing ctime but does not do it 
consistently.

  How do I configure bacula to only use mtime for incremental backup?

  What are the side effects of looking only at mtime?


> 
> One thing you could do (perhaps some bacula-users list guys
> can help if you 
> need it) is to run a Bacula verify InitCatalog on one or
> more of the affected 
> systems just before running netbackup then after you run
> netbackup run a 
> verify catalog to see what files have changed.  You can set
> the parameters of 
> the verify to compare only mtime and ctime.  That way, you
> can definitively 
> determine whether or not netbackup is modifying anything on
> your system.
> 
> >
> > I will have to wait for few more months before I
> shutdown netbackup system
> > and run only bacula. If the problem goes away after
> that then I will know
> > it was netbackup.
> 
> I'll be *very* surprised if it turns out to be related
> to Bacula itself as 
> this is such a fundamental problem that it surely would
> have shown up on a 
> good number of other user's systems, which is not the
> case.
> 
> >
> > I am running bacula 2.4.2 on server (fc9 64-bit) and
> all the clients are at
> > 2.2.8.
> 
> The 2.2.8 clients should be 100% compatible with 2.4.2, but
> I always recommend 
> that you upgrade as soon as possible ...
> 
> Best regards,
> 
> Kern



Hemant Shah
E-mail: hjrrs AT yahoo DOT com



      

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users