Bacula-users

Re: [Bacula-users] confused about differentials

2015-05-18 10:27:00
Subject: Re: [Bacula-users] confused about differentials
From: Luc Van der Veken <lucvdv AT wimionline DOT com>
To: bacula-users <bacula-users AT lists.sourceforge DOT net>
Date: Mon, 18 May 2015 14:21:37 +0000

Hi Ana Emilia, thanks for clearing that up.

 

But then I think I may have a problem :(

 

I am using the Dir and FD from standard Ubuntu 12.04 repositories, 5.2.5.

 

I only have a few Windows clients. For those I got the Windows FD offered by Bacula Systems two years ago, which was at that time 6.0.6. That was the only one offered, as far as I remember. Today it would be 7.0.5.

 

Backups made with that combination run without errors or warnings.

I never had (or tried) to restore a full system, only some individual files, but that always worked too, so far.

 

I am also using community FD 5.2.10 on one machine (my own workstation), that looks OK too.

 

This is some logging output, from a job picked at random, from 6.0.6 FD with director/SD 5.2.5.

 

  Build OS:               x86_64-pc-linux-gnu ubuntu 12.04

  JobId:                  29291

  Job:                    qvsrv.2015-05-15_20.05.01_50

  Backup Level:           Differential, since=2015-04-24 21:37:40

  Client:                 "qvsrv-fd" 6.0.6 (30Sep12) Microsoft Windows Server 2008 R2 Standard Edition Service Pack 1 (build 7601), 64-bit,Cross-compile,Win64

  FileSet:                "qvsrvSet" 2013-07-30 20:05:01

  Pool:                   "File" (From Job DiffPool override)

  Catalog:                "MyCatalog" (From Client resource)

  Storage:                "File" (From Pool resource)

  Scheduled time:         15-May-2015 20:05:01

  Start time:             16-May-2015 00:53:13

  End time:               16-May-2015 01:37:31

  Elapsed time:           44 mins 18 secs

  Priority:               10

  FD Files Written:       4,044

  SD Files Written:       4,044

  FD Bytes Written:       8,663,741,214 (8.663 GB)

  SD Bytes Written:       8,664,580,063 (8.664 GB)

  Rate:                   3259.5 KB/s

  Software Compression:   65.5 %

  VSS:                    yes

  Encryption:             no

  Accurate:               no

  Volume name(s):         FileStorage0133|FileStorage0134|FileStorage0135

  Volume Session Id:      284

  Volume Session Time:    1430987205

  Last Volume Bytes:      4,089,208,805 (4.089 GB)

  Non-fatal FD errors:    0

  SD Errors:              0

  FD termination status:  OK

  SD termination status:  OK

  Termination:            Backup OK

 

I don’t see anything abnormal in it. The backup size may appear small even for a differential, but it’s an intranet reporting server with only little activity.

 

 

From: Ana Emília M. Arruda [mailto:emiliaarruda AT gmail DOT com]
Sent: 18 May 2015 15:14
To: Luc Van der Veken
Cc: bacula-users
Subject: Re: [Bacula-users] confused about differentials

 

Hello Luc,

 

I think there is a misunderstooding here.

 

On Mon, May 18, 2015 at 3:28 AM, Luc Van der Veken <lucvdv AT wimionline DOT com> wrote:

Hi,Kern,

 

Do you mean that exactly as you say it,

​​

“FD that is more recent that the Dir and the SD […] is a big problem”?

I thought it was the other way around, that a FD _older_ than dir/SD would cause problems.

 

“FD that is more recent that the Dir and the SD […] is a big problem”

This is exactly what Kern said. Older versions of FD working with newer Dir and SD vesions are not a problem. Instead, "FD more recent than Dir and SD" are.

 

 

The way you say it here, the FD Bacula Systems provides for community users would force everyone who wants to add Windows clients to an existing configuration, to upgrade his dir and SD to 7.0.5.

 

​FD versions older than Dir and SD are not a problem. I have a 5.2.10 Windows client working with a 7.0.5 Dir and SD.​ If you have 5.2.10 windows clients (the most recent community version), you should have Dir and SD running at least version 5.2.10.

 

Regards,

Ana

 

 

 

From: Kern Sibbald [mailto:kern AT sibbald DOT com]
Sent: 13 May 2015 18:13
To: Ian Young; Radosław Korzeniewski
Cc: bacula-users
Subject: Re: [Bacula-users] confused about differentials

 


We have never had any version of Bacula in which Differential backups were incomplete.  Running with a FD that is more recent that the Dir and the SD or a Dir and SD that are not identical is a big problem.  Until that is fixed it doesn't make much sense to speculate about any odd behavior.  Also, if you are doing something very unusual in your FileSet or using multiple FileSets in your backup, there could be a configuration problem or perhaps even an undiscovered bug, but the first thing to do is correct any possible version problems you have.

Best regards,
Kern

On 13.05.2015 15:18, Ian Young wrote:

 

On 13 May 2015, at 07:20, Radosław Korzeniewski <radoslaw AT korzeniewski DOT net> wrote:

 

I appear to be seeing the same problem with CentOS 6 / CentOS 6 combinations:

 

 

OS versions doesn't matter. What is important: Bacula Dir/SD vs. File Daemon versions. The supported configuration require Dir/SD in the same version every time and FD not newer.

 

I should have been clearer. I meant that I was apparently seeing the same problem on at least one setup where the FD, DIR and SD versions were all the same.

 

I take your point, though, that the Director/SD should not be older than the clients, so I need to fix that. Fortunately the (virtual) machine running the Director and Storage daemons is dedicated to that task, so it should be relatively easy to build a new CentOS 7 machine to get 5.2.13.

 

Recommended version in May 2015 is Bacula 7.0.5, not 5.2.13.

 

I understand that, but deploying 7.0.5 would be significantly harder in my environment than moving to 5.2.13 so it would be something of a last resort.

 

I don't think you're saying that I need to move to the latest version to get reliable backups, are you?

 

If anyone knew of a bug in 5.2.x (or for that matter in 5.0.x) that caused differentials to be incomplete, I'd obviously feel differently (but then, I imagine Red Hat would too, as 5.2.13 is what they are shipping in their most recent release).

 

I don't think I actually have a version mismatch problem (as I'm seeing the same issue with matched versions), but there are all sorts of reasons this might make my problem go away: there may be a bug in the version of 5.0 shipped with RHEL/CentOS, or I may have a configuration problem. Either way, starting from scratch and transitioning clients over may help.

 

First of all. Did you ever test that it is not working?

 

As stated, I have seen significant data loss when attempting to restore a production system. This is not theoretical, although perhaps the subject line led you astray.

 

Differential backup does not backup a files which were deleted in the mean time. So in real system it is very unlikely (it must meet a specific conditions) you get the same number of files backed up in Incremental and Differential levels.

 

I don't believe this is the problem I'm seeing. For example, from my original mail:

 

| 9,929 | srv-c701-backup | 2015-05-01 23:05:04 | B    | I     |   91,030 | 3,450,906,888 | T
| 9,938 | srv-c701-backup | 2015-05-02 23:05:03 | B    | D     |      112 |     2,184,302 | T

 

The 91,000 files in job 9929 were NOT all deleted before job 9938 was run, but do not appear in that job.

 

    -- Ian

 

 




------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud 
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y



_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users

 


------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users

 

------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud 
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users