Bacula-users
[Top] [All Lists]
<prev [Date] next>
[Advanced]
 <prev [Thread] next>

[Bacula-users] Lower elvel backups saving way too many data

2010-06-14 03:03:34
Subject: [Bacula-users] Lower elvel backups saving way too many data
From: Stan Meier <stan.meier AT billigmail DOT org>
To: bacula-users AT lists.sourceforge DOT net
Date: Mon, 14 Jun 2010 08:40:51 +0200 
Hello world,

our Problem at the moment is this: We have configured monthly full,
weekly differential and daily incremental backups. Unfortunately,
the differential and incremental backups seem to lose all information
about the higher level jobs.

Symptoms: The differential level backup, one week after the full
backup, backs up the entire client. Every time a differential backup
runs, the day after, the incremental backup does save the whole
client, too. For example, this is the log of our differential backup
on Saturday:

12-Jun 10:00 maelstrom-dir JobId 203: Start Backup JobId 203, 
Job=meltdown2-std-job.2010-06-12_10.00.00_53
12-Jun 10:00 maelstrom-dir JobId 203: Using Device "diffp-filedev-1"
12-Jun 12:21 maelstrom-sd JobId 203: User defined maximum volume capacity 
268,435,456,000 exceeded on device "diffp-filedev-1" (/filepools/differential).
12-Jun 12:21 maelstrom-sd JobId 203: End of medium on Volume "DIFFP-0016" 
Bytes=268,435,440,558 Blocks=4,161,025 at 12-Jun-2010 12:21.
12-Jun 12:21 maelstrom-dir JobId 203: There are no more Jobs associated with 
Volume "DIFFP-0017". Marking it purged.
12-Jun 12:21 maelstrom-dir JobId 203: All records pruned from Volume 
"DIFFP-0017"; marking it "Purged"
12-Jun 12:21 maelstrom-dir JobId 203: Recycled volume "DIFFP-0017"
12-Jun 12:31 maelstrom-sd JobId 203: Recycled volume "DIFFP-0017" on device 
"diffp-filedev-1" (/filepools/differential), all previous data lost.
12-Jun 12:31 maelstrom-sd JobId 203: New volume "DIFFP-0017" mounted on device 
"diffp-filedev-1" (/filepools/differential) at 12-Jun-2010 12:31.
12-Jun 16:07 maelstrom-sd JobId 203: User defined maximum volume capacity 
268,435,456,000 exceeded on device "diffp-filedev-1" (/filepools/differential).
12-Jun 16:07 maelstrom-sd JobId 203: End of medium on Volume "DIFFP-0017" 
Bytes=268,435,394,088 Blocks=4,161,015 at 12-Jun-2010 16:07.
12-Jun 16:07 maelstrom-dir JobId 203: There are no more Jobs associated with 
Volume "DIFFP-0018". Marking it purged.
12-Jun 16:07 maelstrom-dir JobId 203: All records pruned from Volume 
"DIFFP-0018"; marking it "Purged"
12-Jun 16:07 maelstrom-dir JobId 203: Recycled volume "DIFFP-0018"
12-Jun 16:18 maelstrom-sd JobId 203: Recycled volume "DIFFP-0018" on device 
"diffp-filedev-1" (/filepools/differential), all previous data lost.
12-Jun 16:18 maelstrom-sd JobId 203: New volume "DIFFP-0018" mounted on device 
"diffp-filedev-1" (/filepools/differential) at 12-Jun-2010 16:18.
12-Jun 20:36 maelstrom-sd JobId 203: User defined maximum volume capacity 
268,435,456,000 exceeded on device "diffp-filedev-1" (/filepools/differential).
12-Jun 20:36 maelstrom-sd JobId 203: End of medium on Volume "DIFFP-0018" 
Bytes=268,435,394,683 Blocks=4,161,015 at 12-Jun-2010 20:36.
12-Jun 20:40 maelstrom-dir JobId 203: There are no more Jobs associated with 
Volume "DIFFP-0009". Marking it purged.
12-Jun 20:40 maelstrom-dir JobId 203: All records pruned from Volume 
"DIFFP-0009"; marking it "Purged"
12-Jun 20:40 maelstrom-dir JobId 203: Recycled volume "DIFFP-0009"
12-Jun 20:51 maelstrom-sd JobId 203: Recycled volume "DIFFP-0009" on device 
"diffp-filedev-1" (/filepools/differential), all previous data lost.
12-Jun 20:51 maelstrom-sd JobId 203: New volume "DIFFP-0009" mounted on device 
"diffp-filedev-1" (/filepools/differential) at 12-Jun-2010 20:51.
13-Jun 00:02 maelstrom-sd JobId 203: User defined maximum volume capacity 
268,435,456,000 exceeded on device "diffp-filedev-1" (/filepools/differential).
13-Jun 00:02 maelstrom-sd JobId 203: End of medium on Volume "DIFFP-0009" 
Bytes=268,435,394,664 Blocks=4,161,015 at 13-Jun-2010 00:02.
13-Jun 00:02 maelstrom-dir JobId 203: There are no more Jobs associated with 
Volume "DIFFP-0003". Marking it purged.
13-Jun 00:02 maelstrom-dir JobId 203: All records pruned from Volume 
"DIFFP-0003"; marking it "Purged"
13-Jun 00:02 maelstrom-dir JobId 203: Recycled volume "DIFFP-0003"
13-Jun 00:14 maelstrom-sd JobId 203: Recycled volume "DIFFP-0003" on device 
"diffp-filedev-1" (/filepools/differential), all previous data lost.
13-Jun 00:14 maelstrom-sd JobId 203: New volume "DIFFP-0003" mounted on device 
"diffp-filedev-1" (/filepools/differential) at 13-Jun-2010 00:14.
13-Jun 03:01 maelstrom-sd JobId 203: User defined maximum volume capacity 
268,435,456,000 exceeded on device "diffp-filedev-1" (/filepools/differential).
13-Jun 03:01 maelstrom-sd JobId 203: End of medium on Volume "DIFFP-0003" 
Bytes=268,435,394,966 Blocks=4,161,015 at 13-Jun-2010 03:01.
13-Jun 03:01 maelstrom-dir JobId 203: There are no more Jobs associated with 
Volume "DIFFP-0002". Marking it purged.
13-Jun 03:01 maelstrom-dir JobId 203: All records pruned from Volume 
"DIFFP-0002"; marking it "Purged"
13-Jun 03:01 maelstrom-dir JobId 203: Recycled volume "DIFFP-0002"
13-Jun 03:12 maelstrom-sd JobId 203: Recycled volume "DIFFP-0002" on device 
"diffp-filedev-1" (/filepools/differential), all previous data lost.
13-Jun 03:12 maelstrom-sd JobId 203: New volume "DIFFP-0002" mounted on device 
"diffp-filedev-1" (/filepools/differential) at 13-Jun-2010 03:12.
13-Jun 03:12 maelstrom-sd JobId 203: Job write elapsed time = 16:13:35, 
Transfer rate = 22.30 M Bytes/second
13-Jun 03:22 maelstrom-dir JobId 203: Bacula maelstrom-dir 5.0.1 (24Feb10): 
13-Jun-2010 03:22:00
  Build OS:               x86_64-redhat-linux-gnu redhat Enterprise release
  JobId:                  203
  Job:                    meltdown2-std-job.2010-06-12_10.00.00_53
  Backup Level:           Differential, since=2010-06-05 10:00:41
  Client:                 "meltdown2-fd" 5.0.1 (24Feb10) 
i686-pc-linux-gnu,redhat,
  FileSet:                "boshare-std-fileset" 2010-05-03 15:27:52
  Pool:                   "DIFF_POOL" (From Run pool override)
  Catalog:                "MyCatalog" (From Client resource)
  Storage:                "diffp-storage" (From Pool resource)
  Scheduled time:         12-Jun-2010 10:00:00
  Start time:             12-Jun-2010 10:00:02
  End time:               13-Jun-2010 03:22:00
  Elapsed time:           17 hours 21 mins 58 secs
  Priority:               10
  FD Files Written:       2,444,185
  SD Files Written:       2,444,185
  FD Bytes Written:       1,302,559,178,660 (1.302 TB)
  SD Bytes Written:       1,302,988,821,917 (1.302 TB)
  Rate:                   20834.9 KB/s
  Software Compression:   None
  VSS:                    no
  Encryption:             no
  Accurate:               no
  Volume name(s):         
DIFFP-0016|DIFFP-0017|DIFFP-0018|DIFFP-0009|DIFFP-0003|DIFFP-0002
  Volume Session Id:      101
  Volume Session Time:    1275392230
  Last Volume Bytes:      1,117,821,240 (1.117 GB)
  Non-fatal FD errors:    0
  SD Errors:              0
  FD termination status:  OK
  SD termination status:  OK
  Termination:            Backup OK

13-Jun 03:22 maelstrom-dir JobId 203: Begin pruning Jobs older than 6 months .
13-Jun 03:22 maelstrom-dir JobId 203: No Jobs found to prune.
13-Jun 03:22 maelstrom-dir JobId 203: Begin pruning Jobs.
13-Jun 03:22 maelstrom-dir JobId 203: No Files found to prune.
13-Jun 03:22 maelstrom-dir JobId 203: End auto prune.


On Sunday at 7pm, the incremental job runs and backs up the whole
client:

13-Jun 19:00 maelstrom-dir JobId 211: Start Backup JobId 211, 
Job=meltdown2-std-job.2010-06-13_19.00.01_01
13-Jun 19:00 maelstrom-dir JobId 211: Using Device "diffp-filedev-1"
13-Jun 21:20 maelstrom-sd JobId 211: User defined maximum volume capacity 
268,435,456,000 exceeded on device "diffp-filedev-1" (/filepools/differential).
13-Jun 21:20 maelstrom-sd JobId 211: End of medium on Volume "DIFFP-0002" 
Bytes=268,435,443,465 Blocks=4,161,021 at 13-Jun-2010 21:20.
13-Jun 21:20 maelstrom-dir JobId 211: There are no more Jobs associated with 
Volume "DIFFP-0010". Marking it purged.
13-Jun 21:20 maelstrom-dir JobId 211: All records pruned from Volume 
"DIFFP-0010"; marking it "Purged"
13-Jun 21:20 maelstrom-dir JobId 211: Recycled volume "DIFFP-0010"
13-Jun 21:31 maelstrom-sd JobId 211: Recycled volume "DIFFP-0010" on device 
"diffp-filedev-1" (/filepools/differential), all previous data lost.
13-Jun 21:31 maelstrom-sd JobId 211: New volume "DIFFP-0010" mounted on device 
"diffp-filedev-1" (/filepools/differential) at 13-Jun-2010 21:31.
13-Jun 23:46 maelstrom-sd JobId 211: User defined maximum volume capacity 
268,435,456,000 exceeded on device "diffp-filedev-1" (/filepools/differential).
13-Jun 23:46 maelstrom-sd JobId 211: End of medium on Volume "DIFFP-0010" 
Bytes=268,435,394,727 Blocks=4,161,015 at 13-Jun-2010 23:46.
13-Jun 23:46 maelstrom-dir JobId 211: There are no more Jobs associated with 
Volume "DIFFP-0011". Marking it purged.
13-Jun 23:46 maelstrom-dir JobId 211: All records pruned from Volume 
"DIFFP-0011"; marking it "Purged"
13-Jun 23:46 maelstrom-dir JobId 211: Recycled volume "DIFFP-0011"
13-Jun 23:57 maelstrom-sd JobId 211: Recycled volume "DIFFP-0011" on device 
"diffp-filedev-1" (/filepools/differential), all previous data lost.
13-Jun 23:57 maelstrom-sd JobId 211: New volume "DIFFP-0011" mounted on device 
"diffp-filedev-1" (/filepools/differential) at 13-Jun-2010 23:57.
14-Jun 02:05 maelstrom-sd JobId 211: User defined maximum volume capacity 
268,435,456,000 exceeded on device "diffp-filedev-1" (/filepools/differential).
14-Jun 02:05 maelstrom-sd JobId 211: End of medium on Volume "DIFFP-0011" 
Bytes=268,435,394,397 Blocks=4,161,015 at 14-Jun-2010 02:05.
14-Jun 02:06 maelstrom-dir JobId 211: There are no more Jobs associated with 
Volume "DIFFP-0012". Marking it purged.
14-Jun 02:06 maelstrom-dir JobId 211: All records pruned from Volume 
"DIFFP-0012"; marking it "Purged"
14-Jun 02:06 maelstrom-dir JobId 211: Recycled volume "DIFFP-0012"
14-Jun 02:18 maelstrom-sd JobId 211: Recycled volume "DIFFP-0012" on device 
"diffp-filedev-1" (/filepools/differential), all previous data lost.
14-Jun 02:18 maelstrom-sd JobId 211: New volume "DIFFP-0012" mounted on device 
"diffp-filedev-1" (/filepools/differential) at 14-Jun-2010 02:18.
14-Jun 04:08 maelstrom-sd JobId 211: User defined maximum volume capacity 
268,435,456,000 exceeded on device "diffp-filedev-1" (/filepools/differential).
14-Jun 04:08 maelstrom-sd JobId 211: End of medium on Volume "DIFFP-0012" 
Bytes=268,435,394,855 Blocks=4,161,015 at 14-Jun-2010 04:08.
14-Jun 04:08 maelstrom-dir JobId 211: Created new Volume "DIFFP-0023" in 
catalog.
14-Jun 04:08 maelstrom-sd JobId 211: Labeled new Volume "DIFFP-0023" on device 
"diffp-filedev-1" (/filepools/differential).
14-Jun 04:08 maelstrom-sd JobId 211: Wrote label to prelabeled Volume 
"DIFFP-0023" on device "diffp-filedev-1" (/filepools/differential)
14-Jun 04:08 maelstrom-sd JobId 211: New volume "DIFFP-0023" mounted on device 
"diffp-filedev-1" (/filepools/differential) at 14-Jun-2010 04:08.
14-Jun 06:01 maelstrom-sd JobId 211: Job write elapsed time = 10:26:32, 
Transfer rate = 34.66 M Bytes/second
14-Jun 06:10 maelstrom-dir JobId 211: Bacula maelstrom-dir 5.0.1 (24Feb10): 
14-Jun-2010 06:10:01
  Build OS:               x86_64-redhat-linux-gnu redhat Enterprise release
  JobId:                  211
  Job:                    meltdown2-std-job.2010-06-13_19.00.01_01
  Backup Level:           Incremental, since=2010-06-12 10:00:02
  Client:                 "meltdown2-fd" 5.0.1 (24Feb10) 
i686-pc-linux-gnu,redhat,
  FileSet:                "boshare-std-fileset" 2010-05-03 15:27:52
  Pool:                   "DIFF_POOL" (From Run pool override)
  Catalog:                "MyCatalog" (From Client resource)
  Storage:                "diffp-storage" (From Pool resource)
  Scheduled time:         13-Jun-2010 19:00:01
  Start time:             13-Jun-2010 19:00:01
  End time:               14-Jun-2010 06:10:01
  Elapsed time:           11 hours 10 mins 
  Priority:               10
  FD Files Written:       2,445,116
  SD Files Written:       2,445,116
  FD Bytes Written:       1,302,640,905,745 (1.302 TB)
  SD Bytes Written:       1,303,070,734,443 (1.303 TB)
  Rate:                   32404.0 KB/s
  Software Compression:   None
  VSS:                    no
  Encryption:             no
  Accurate:               no
  Volume name(s):         DIFFP-0002|DIFFP-0010|DIFFP-0011|DIFFP-0012|DIFFP-0023
  Volume Session Id:      108
  Volume Session Time:    1275392230
  Last Volume Bytes:      236,187,350,003 (236.1 GB)
  Non-fatal FD errors:    0
  SD Errors:              0
  FD termination status:  OK
  SD termination status:  OK
  Termination:            Backup OK

14-Jun 06:10 maelstrom-dir JobId 211: Begin pruning Jobs older than 6 months .
14-Jun 06:10 maelstrom-dir JobId 211: No Jobs found to prune.
14-Jun 06:10 maelstrom-dir JobId 211: Begin pruning Jobs.
14-Jun 06:10 maelstrom-dir JobId 211: No Files found to prune.
14-Jun 06:10 maelstrom-dir JobId 211: End auto prune.


How de we fix this? Below is a dump of our configuration:

Pool {
  Name = FULL_POOL
  Pool Type = Backup
  Storage = fullp-storage
  Recycle = yes
  AutoPrune = yes
  Volume Retention = 4w
  Maximum Volume Bytes = 250G
  Maximum Volumes = 55
  Label Format = "FULLP-"
  Next Pool = FULL_POOL
  Action On Purge = Truncate
}
Pool {
  Name = DIFF_POOL
  Next Pool = DIFF_POOL
  Pool Type = Backup
  Storage = diffp-storage
  Recycle = yes
  AutoPrune = yes
  Volume Retention = 1w
  Maximum Volume Bytes = 250G
  Maximum Volumes = 22
  Label Format = "DIFFP-"
  Action On Purge = Truncate
}

Schedule {
  Name = "StandardLinuxCycle"
  Run = Full Pool=FULL_POOL 1st sat at 10:00
  Run = Differential Pool=DIFF_POOL 2nd-5th sat at 10:00
  Run = Incremental Pool=DIFF_POOL sun-fri at 19:00
}

FileSet {
  Name = boshare-std-fileset
  Include {
    Options {
      signature = MD5
    }
        File = /root
        File = /opt
        File = /etc
        File = /var/spool/cron
        File = /var/log
        File = /data
  }
    Exclude {
        File = /opt/bacula/current/server/var/bacula/working
        File = /opt/bacula/current/client/var/bacula/working
  }
}
JobDefs {
  Name = "boshare-std-jobdef"
  Type = Backup
  FileSet = "boshare-std-fileset"
  Schedule = "StandardLinuxCycle"
  Messages = Standard
  Pool = FULL_POOL
}
Client {
  Name = meltdown2-fd
  Address = meltdown2.example-websrvdmz.de
  Catalog = MyCatalog
  Password = "secret"
  Heartbeat Interval = 15 seconds
}
Job {
  Name = meltdown2-std-job
  Client = meltdown2-fd
  JobDefs = boshare-std-jobdef
}


Cheers
Stan

------------------------------------------------------------------------------
ThinkGeek and WIRED's GeekDad team up for the Ultimate 
GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the 
lucky parental unit.  See the prize list and enter to win: 
http://p.sf.net/sfu/thinkgeek-promo
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users
[More with this subject...]
<Prev in Thread] Current Thread [Next in Thread>
  • [Bacula-users] Lower elvel backups saving way too many data, Stan Meier <=
Previous by Date:  Re: [Bacula-users] Using HP Storageworks 1/8 G2 with Bacula, Thomas Mueller
Next by Date:  Re: [Bacula-users] Bacula fails to backup jobs to Tape, Martin Simmons
Previous by Thread:  [Bacula-users] Copy Jobs, bwellsnc
Next by Thread:  [Bacula-users] Storage daemon didn't accept Device, Michael Patzer
Indexes:  [Date] [Thread] [Top] [All Lists]