Bacula-users

Re: [Bacula-users] Job pre-allocating 20 volumes from a pool un-necessary

2010-11-28 22:11:28
Subject: Re: [Bacula-users] Job pre-allocating 20 volumes from a pool un-necessary
From: "Francois Duchatelet" <bacula-f AT duchatelet DOT org>
To: "bacula-users" <bacula-users AT lists.sourceforge DOT net>
Date: Mon, 29 Nov 2010 04:07:22 +0100
No it isn’t.
 
The normal way of working is:
- fill a volume,
- THEN look for an available (Appendable,Purged,...) one,
- THEN ONLY if there are no available volumes in the Pool, create a new one (if the pool allows it).
 
 
The issue is: SOMETIMES, the locutus job PRE-creates 20 volumes, although it doesn’t need them.
In this case, the job was only 400Mb and normally completed on ONE volume.
 
The “Sleep 60” is necessary because otherwise the 2 jobs, starting at the same, would compete for the same volume.
With this ‘sleep 60’, the sulu job has time to mark it’s first volume as ‘Used’ (by virtue of ‘Maximum Volume Jobs = 1‘) before the locutus job does its reservation.
At least this was an issue before I upgraded to 5.0.2, while the two jobs ran concurrently.
 
As I need to upgrade to 5.0.3 anyway, I’ll see if the problem persist.
Does someone have 5.0.3 packages for debian/squeeze ? Otherwize I’ll try to build them.
 
Cheers,
François
 
 
 
Sent: Sunday, November 28, 2010 9:45 PM
Subject: Re: [Bacula-users] Job pre-allocating 20 volumes from a poolun-necessary
 
It seems correct!
How many GB you have on spock host?
You difined maximum volume size with 1GB and only one use for each volume, then if the backup of spock host exceed 1GB other volumes will be created. If you want keep the job on single volume you should remove the maximum volume size definition.

Why you need this "sleep 60"?

Kleber

2010/11/28 Francois D. <bacula-f AT duchatelet DOT org>
Hello,

I've an issue preventing the correct recycling of my Pool's volume.

Given the definitions at the end of the message, I have the job for locutus
pre-create 20 volumes in the Default pool right after the 60 seconds sleep.

What happens:
- at 02:05, two jobs (Sulu and Locutus) starts.
- to avoid them compelling for the same Volume, the Locutus job does a
'sleep 60'.
- due, I think, to the known bug in 5.0.2 (bug-1578), the locutus job waits
for the sulu completion before going on.
- sometimes (I was not able to deduce a pattern yet), while it is waiting,
the locutus job creates new volumes in the target pool, even before it needs
it.

When the pool is empty, like today (I've cleaned and rebuild the pool
yesterday), this is not really an issue, but when there are 100+ volumes,
with about 50 of them purged, and ready to be re-used, this is annoying, as
this makes the pool growing.

Any ideas of what I did wrong ?

Cheers,
François


=============================================

Here is an extract of the Sulu job log:
--------------------
28-Nov 02:05 spock-dir JobId 1446: Start Backup JobId 1446,
Job=sulu.2010-11-28_02.05.00_44
28-Nov 02:05 spock-dir JobId 1446: Created new Volume "BDSK-002" in catalog.
28-Nov 02:05 spock-dir JobId 1446: Using Device "FileStorage"
28-Nov 02:05 spock-sd JobId 1446: Labeled new Volume "BDSK-002" on device
"FileStorage" (/backups).
28-Nov 02:05 spock-sd JobId 1446: Wrote label to prelabeled Volume
"BDSK-002" on device "FileStorage" (/backups)
28-Nov 02:05 spock-dir JobId 1446: Max Volume jobs exceeded. Marking Volume
"BDSK-002" as Used.
28-Nov 02:05 sulu-fd JobId 1446: Generate VSS snapshots. Driver="VSS Vista",
Drive(s)="C"
28-Nov 02:08 spock-sd JobId 1446: User defined maximum volume capacity
1,073,741,824 exceeded on device "FileStorage" (/backups).
28-Nov 02:08 spock-sd JobId 1446: End of medium on Volume "BDSK-002"
Bytes=1,073,737,912 Blocks=16,644 at 28-Nov-2010 02:08.
28-Nov 02:08 spock-sd JobId 1446: Labeled new Volume "BDSK-003" on device
"FileStorage" (/backups).
28-Nov 02:08 spock-sd JobId 1446: Wrote label to prelabeled Volume
"BDSK-003" on device "FileStorage" (/backups)
28-Nov 02:08 spock-dir JobId 1446: Max Volume jobs exceeded. Marking Volume
"BDSK-003" as Used.
28-Nov 02:08 spock-sd JobId 1446: New volume "BDSK-003" mounted on device
"FileStorage" (/backups) at 28-Nov-2010 02:08.
28-Nov 02:24 spock-sd JobId 1446: User defined maximum volume capacity
1,073,741,824 exceeded on device "FileStorage" (/backups).
28-Nov 02:24 spock-sd JobId 1446: End of medium on Volume "BDSK-003"
Bytes=1,073,737,675 Blocks=16,644 at 28-Nov-2010 02:24.
28-Nov 02:24 spock-sd JobId 1446: Labeled new Volume "BDSK-004" on device
"FileStorage" (/backups).
28-Nov 02:24 spock-sd JobId 1446: Wrote label to prelabeled Volume
"BDSK-004" on device "FileStorage" (/backups)
28-Nov 02:24 spock-dir JobId 1446: Max Volume jobs exceeded. Marking Volume
"BDSK-004" as Used.
28-Nov 02:24 spock-sd JobId 1446: New volume "BDSK-004" mounted on device
"FileStorage" (/backups) at 28-Nov-2010 02:24.

And here is an extract of the locutus job log:
--------------------
28-Nov 02:05 spock-dir JobId 1447: shell command: run BeforeJob "/bin/sleep
60"
28-Nov 02:06 spock-dir JobId 1447: Start Backup JobId 1447,
Job=locutus.2010-11-28_02.05.01_45
28-Nov 02:06 spock-dir JobId 1447: Created new Volume "BDSK-003" in catalog.
28-Nov 02:06 spock-dir JobId 1447: Created new Volume "BDSK-004" in catalog.
28-Nov 02:06 spock-dir JobId 1447: Created new Volume "BDSK-005" in catalog.
28-Nov 02:06 spock-dir JobId 1447: Created new Volume "BDSK-006" in catalog.
28-Nov 02:06 spock-dir JobId 1447: Created new Volume "BDSK-007" in catalog.
28-Nov 02:06 spock-dir JobId 1447: Created new Volume "BDSK-008" in catalog.
28-Nov 02:06 spock-dir JobId 1447: Created new Volume "BDSK-009" in catalog.
28-Nov 02:06 spock-dir JobId 1447: Created new Volume "BDSK-010" in catalog.
28-Nov 02:06 spock-dir JobId 1447: Created new Volume "BDSK-011" in catalog.
28-Nov 02:06 spock-dir JobId 1447: Created new Volume "BDSK-012" in catalog.
28-Nov 02:06 spock-dir JobId 1447: Created new Volume "BDSK-013" in catalog.
28-Nov 02:06 spock-dir JobId 1447: Created new Volume "BDSK-014" in catalog.
28-Nov 02:06 spock-dir JobId 1447: Created new Volume "BDSK-015" in catalog.
28-Nov 02:06 spock-dir JobId 1447: Created new Volume "BDSK-016" in catalog.
28-Nov 02:06 spock-dir JobId 1447: Created new Volume "BDSK-017" in catalog.
28-Nov 02:06 spock-dir JobId 1447: Created new Volume "BDSK-018" in catalog.
28-Nov 02:06 spock-dir JobId 1447: Created new Volume "BDSK-019" in catalog.
28-Nov 02:06 spock-dir JobId 1447: Created new Volume "BDSK-020" in catalog.
28-Nov 02:06 spock-dir JobId 1447: Created new Volume "BDSK-021" in catalog.
28-Nov 02:08 spock-dir JobId 1447: Created new Volume "BDSK-022" in catalog.
28-Nov 02:24 spock-dir JobId 1447: Created new Volume "BDSK-023" in catalog.
28-Nov 02:29 spock-dir JobId 1447: Using Device "FileStorage"
28-Nov 02:29 spock-sd JobId 1447: Labeled new Volume "BDSK-005" on device
"FileStorage" (/backups).
28-Nov 02:29 spock-sd JobId 1447: Wrote label to prelabeled Volume
"BDSK-005" on device "FileStorage" (/backups)

Extract of configuration:
-------------------------
# Default pool definition, targeting the disk storage
Pool {
Name = Default
Pool Type = Backup
Recycle = yes                 # Bacula can automatically recycle Volumes
AutoPrune = yes               # Prune expired volumes
Volume Retention = 7 days
Maximum Volume Jobs = 1       # On disk, I want a new volume for each job
Maximum Volume Bytes = 1G     # Limit the size of volumes to something
reasonnable for disk files
LabelFormat = "BDSK-${BDSK+:p/3/0/r}"
Storage = File
Next Pool = LTO1
Migration Time = 1 hour
}

# Those are generic defaults for all my backups
JobDefs {
Name = "DefaultJob"
Type = Backup
Level = Incremental
Schedule = "WeeklyCycle"
Messages = Standard
Pool = Default
Priority = 10
}

Job {
Name = "sulu"
Client = sulu-fd
FileSet = "Sulu Set"
JobDefs = "DefaultJob"
# Force it to be started ahead of others, so the schedule is kept"
Priority = 9
}

Job {
Name = "locutus"
Client = locutus-fd
FileSet = "Locutus Set"
JobDefs = "DefaultJob"
# Force it to be started ahead of others, so the schedule is kept"
Priority = 9

RunScript {
   RunsWhen = Before
   RunsOnClient  = No
   FailJobOnError = No
   Command = "/bin/sleep 60"
}
}




------------------------------------------------------------------------------
Increase Visibility of Your 3D Game App & Earn a Chance To Win $500!
Tap into the largest installed PC base & get more eyes on your game by
optimizing for Intel(R) Graphics Technology. Get started today with the
Intel(R) Software Partner Program. Five $500 cash prizes are up for grabs.
http://p.sf.net/sfu/intelisp-dev2dev
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users


------------------------------------------------------------------------------
Increase Visibility of Your 3D Game App & Earn a Chance To Win $500!
Tap into the largest installed PC base & get more eyes on your game by
optimizing for Intel(R) Graphics Technology. Get started today with the
Intel(R) Software Partner Program. Five $500 cash prizes are up for grabs.
http://p.sf.net/sfu/intelisp-dev2dev


_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users
------------------------------------------------------------------------------
Increase Visibility of Your 3D Game App & Earn a Chance To Win $500!
Tap into the largest installed PC base & get more eyes on your game by
optimizing for Intel(R) Graphics Technology. Get started today with the
Intel(R) Software Partner Program. Five $500 cash prizes are up for grabs.
http://p.sf.net/sfu/intelisp-dev2dev
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users
<Prev in Thread] Current Thread [Next in Thread>