Bacula-users

Re: [Bacula-users] Problem of "is waiting on max storage jobs" and "is waiting for an appendable volume"

2014-11-19 08:54:53
Subject: Re: [Bacula-users] Problem of "is waiting on max storage jobs" and "is waiting for an appendable volume"
From: Steven Hammond <bcf1689 AT gmail DOT com>
To: bacula-users AT lists.sourceforge DOT net
Date: Wed, 19 Nov 2014 07:48:34 -0600
I've found this thread very helpful.  I have just a couple of questions.

1. I have jobs that backup to disk.  It has data spooling turned on and MAXIMUM CONCURRENT JOBS = 5.  They all have the same priority and write to the same pool.  It appears to be working, is this the proper way to handle concurrent jobs?
2. I have a COPY JOB that kicks later that night that copies from DISK to TAPE (LTO-5).  It too has the same configuration above but it on runs one job at a time.  I sort of expected it to run concurrent and use the data spooling to make sure everything was in order before writing to the tape but is always seems to block on storage.
3. Sometimes the individual responsible forgets to put in the tape the night before.  I have a critical job (Priority = 1) that kicks off the next morning.  However it never runs because the tape job is in the queue waiting for the tape (Priority = 10).  Is there a way to make sure the higher priority job gets run?

Thanks again.

Steve Hammond

On 11/19/2014 7:23 AM, heitor AT bacula.com DOT br wrote:
*Just to give a personal contribution about the subject*.

1. It's redundant to have date variables on the volume names, since you have the Label Date information on Bacula Catalog, that can be accessed with the "llist" command.

2. For disk backups I like to use something like: Label Format = "${Pool}-${NumVols}", or maybe: "${Pool}-${NumVols}-${Storage}"; NumVols is a incremental counter of volumes in pool, and this way you will have always a unique name for each volume.

Regards,
=========================================================================
Heitor Medrado de Faria  - LPIC-III | ITIL-F
Faltam poucos dias - Treinamento Telepresencial Bacula: http://www.bacula.com.br/?p=2174
==========================================================================


De: "Ana Emília M. Arruda" <emiliaarruda AT gmail DOT com>
Para: "Sieu Truc" <sieutruc AT gmail DOT com>
Cc: "Martin Simmons" <martin AT lispworks DOT com>, "Bacula-users AT lists.sourceforge DOT net" <bacula-users AT lists.sourceforge DOT net>
Enviadas: Quarta-feira, 19 de novembro de 2014 10:18:40
Assunto: Re: [Bacula-users] Problem of "is waiting on max storage jobs" and "is waiting for an appendable volume"

You´re welcome Sieu :-)

About the label format: when Bacula recycles a volume, it doesn´t change the label. So if you have a thursday-2014-11-6 volume, with 13 days retention period and , then it could be recycled and used by Bacula at 2014-11-27, but your volume name will stay the first label thursday-2014-11-6. So, in the case of reusing volumes, it is better to not have dates on volumes names. Instead, maybe you can have  thursday-1,  thursday-2, etc.

Best regards,
Ana

On Wed, Nov 19, 2014 at 6:40 AM, Sieu Truc <sieutruc AT gmail DOT com> wrote:
Thanks to Ana,

It works without any problem now.

But what do you mean "Label Format clauses are strange". Are you sure whether the same volume is reused ?

Does bacula detect Maximum Volumes greater than 1 so il will automatically add the number after ?

For ex: monday-2014-11-17001 and monday-2014-11-1700.

Truc

On Tue, Nov 18, 2014 at 7:57 PM, Martin Simmons <martin AT lispworks DOT com> wrote:
Also, I think those Label Format clauses are strange when used with Recycle =
yes.  The volumes will be labelled forever with the creation date
(e.g. monday-2014-11-17), not the recycling date.

__Martin


>>>>> On Tue, 18 Nov 2014 13:51:57 -0300, Ana Emília M. Arruda said:
>
> Yes, you are correct. That was exactly what ocurred. If you want a 13 days
> retention period for your incremental jobs, you will have to configure
> a Maximum
> Volumes of 2. Also, you have different retention periods for incremental
> and full backups (me too). So I would recommend you to have different pools
> for your full and incremental backups.
>
> Best regards,
> Ana
>
> On Tue, Nov 18, 2014 at 1:35 PM, Sieu Truc <sieutruc AT gmail DOT com> wrote:
>
> > Yes, the actual value is 13, and it caused the problem.
> >
> > In my opinion,  the first week , bacula created successfully the monday
> > incremental backup, and its status was "USED" but it had to be wait 13 days.
> > During that time, in the second week, bacula tried to write again to the
> > same volume that is not pruned so it caused the problem. And maybe it hangs
> > the remain jobs.
> >
> > Is it right ?
> >
> > Or do you have any suggestion ?
> >
> > Truc
> >
> > On Tue, Nov 18, 2014 at 5:27 PM, Ana Emília M. Arruda <
> > emiliaarruda AT gmail DOT com> wrote:
> >
> >> Hi Sieu,
> >>
> >> If you have changed the "volume retention" for your volumes in the pool
> >> configuration, this change only is applied to new volumes created. For the
> >> volumes that were already in the catalog, you should run an "update volume"
> >> through bconsole and change this value there too.
> >>
> >> Since you said that you have changed from 6 to 13 days your retention
> >> periods, I´m supposing maybe this is your problem with  "is waiting on
> >> max storage jobs".
> >>
> >> Best regards,
> >> Ana
> >>
> >> On Tue, Nov 18, 2014 at 12:39 PM, Sieu Truc <sieutruc AT gmail DOT com> wrote:
> >>
> >>> sorry for my mistake, in the pools.conf, the value of parameter "Volume
> >>> Retention" is all 13 days, not 6 days as above.
> >>>
> >>> Truc
> >>>
> >>> On Tue, Nov 18, 2014 at 4:34 PM, Sieu Truc <sieutruc AT gmail DOT com> wrote:
> >>>
> >>>> Hello,
> >>>>
> >>>> I have a schedules.conf like
> >>>>
> >>>> #
> >>>> # Planification des sauvegardes nfs.
> >>>> #
> >>>> Schedule {
> >>>> Name = "schedule_intra1"
> >>>> Run = Level=Incremental Pool=lundi Monday at 22:00
> >>>> Run = Level=Incremental Pool=mardi Tuesday at 22:00
> >>>> Run = Level=Incremental Pool=mercredi Wednesday at 22:00
> >>>> Run = Level=Incremental Pool=jeudi Thursday at 22:00
> >>>> Run = Level=Incremental Pool=vendredi Friday at 22:00
> >>>> Run = Level=Incremental Pool=samedi Saturday at 22:00
> >>>>         Run = Level=Full Pool=dimanche_pair w02 w04 w06 w08 w10 w12 w14
> >>>> w16 w18 w20 w22 w24 w26 w28 w30 w32 w34 w36 w38 w40 w42 w44 w46 w48 w50 w52
> >>>> Sunday at 22:00
> >>>>         Run = Level=Full Pool=dimanche_impair w01 w03 w05 w07 w09 w11
> >>>> w13 w15 w17 w19 w21 w23 w25 w27 w29 w31 w33 w35 w37 w39 w41 w43 w45 w47 w49
> >>>> w51 w53 Sunday at 22:00
> >>>> }
> >>>>
> >>>> Schedule {
> >>>>         Name = "schedule_intra2"
> >>>>         Run = Level=Incremental Pool=monday Monday at 22:10
> >>>>         Run = Level=Incremental Pool=tuesday Tuesday at 22:10
> >>>>         Run = Level=Incremental Pool=wednesdayi Wednesday at 22:10
> >>>>         Run = Level=Incremental Pool=thursday Thursday at 22:10
> >>>>         Run = Level=Incremental Pool=friday Friday at 22:10
> >>>>         Run = Level=Incremental Pool=saturday Saturday at 22:10
> >>>>         Run = Level=Full Pool=sunday_even w02 w04 w06 w08 w10 w12 w14
> >>>> w16 w18 w20 w22 w24 w26 w28 w30 w32 w34 w36 w38 w40 w42 w44 w46 w48 w50 w52
> >>>> Sunday at 22:10
> >>>>         Run = Level=Full Pool=sunday_odd w01 w03 w05 w07 w09 w11 w13
> >>>> w15 w17 w19 w21 w23 w25 w27 w29 w31 w33 w35 w37 w39 w41 w43 w45 w47 w49 w51
> >>>> w53 Sunday at 22:10
> >>>> }
> >>>>
> >>>>
> >>>> And here the definitions of the pools are:
> >>>>
> >>>> Pool {
> >>>> Name = monday
> >>>> Pool Type = Backup
> >>>> AutoPrune = yes
> >>>> Volume Retention = 6 days
> >>>> Volume Use Duration = 1 days
> >>>> Recycle = yes
> >>>>         #recyclepool= lundi
> >>>>         Maximum Volumes = 1
> >>>>         Recycle Current Volume = yes
> >>>> Label Format = "monday-${Year}-${Month:p/2/0/r}-${Day:p/2/0/r}"
> >>>> }
> >>>>
> >>>> Pool {
> >>>>         Name = tuesday
> >>>>         Pool Type = Backup
> >>>> AutoPrune = yes
> >>>>         Volume Retention = 6 days
> >>>> Volume Use Duration = 1 days
> >>>> Recycle = yes
> >>>>         #recyclepool= mardi
> >>>>         Maximum Volumes = 1
> >>>>         Recycle Current Volume = yes
> >>>>         Label Format = "tuesday-${Year}-${Month:p/2/0/r}-${Day:p/2/0/r}"
> >>>> }
> >>>> Pool {
> >>>>         Name = wednesday
> >>>>         Pool Type = Backup
> >>>> AutoPrune = yes
> >>>>         Volume Retention = 6 days
> >>>> Volume Use Duration = 1 days
> >>>> Recycle = yes
> >>>>         #recyclepool= mercredi
> >>>>         Maximum Volumes = 1
> >>>>         Recycle Current Volume = yes
> >>>>         Label Format =
> >>>> "wednesday-${Year}-${Month:p/2/0/r}-${Day:p/2/0/r}"
> >>>> }
> >>>> Pool {
> >>>>         Name = thursday
> >>>>         Pool Type = Backup
> >>>> AutoPrune = yes
> >>>>         Volume Retention = 6 days
> >>>> Volume Use Duration = 1 days
> >>>> Recycle = yes
> >>>>         #recyclepool= jeudi
> >>>>         Maximum Volumes = 1
> >>>>         Recycle Current Volume = yes
> >>>>         Label Format =
> >>>> "thursday-${Year}-${Month:p/2/0/r}-${Day:p/2/0/r}"
> >>>> }
> >>>> Pool {
> >>>>         Name =friday
> >>>>         Pool Type = Backup
> >>>> AutoPrune = yes
> >>>>         Volume Retention = 6 days
> >>>> Volume Use Duration = 1 days
> >>>> Recycle = yes
> >>>>         #recyclepool= vendredi
> >>>>         Maximum Volumes = 1
> >>>>         Recycle Current Volume = yes
> >>>>         Label Format = "friday-${Year}-${Month:p/2/0/r}-${Day:p/2/0/r}"
> >>>> }
> >>>> Pool {
> >>>>         Name = saturday
> >>>>         Pool Type = Backup
> >>>> AutoPrune = yes
> >>>>         Volume Retention = 6 days
> >>>> Volume Use Duration = 1 days
> >>>> Recycle = yes
> >>>>         #recyclepool= samedi
> >>>>         Maximum Volumes = 1
> >>>>         Recycle Current Volume = yes
> >>>>         Label Format =
> >>>> "saturday-${Year}-${Month:p/2/0/r}-${Day:p/2/0/r}"
> >>>> }
> >>>>
> >>>> Pool {
> >>>>         Name = sunday_even
> >>>>         Pool Type = Backup
> >>>>         AutoPrune = yes
> >>>>         Volume Retention = 13 days
> >>>>         Volume Use Duration = 1 days
> >>>>         Recycle = yes
> >>>>         recyclepool= dimanche_pair
> >>>>         Maximum Volumes = 1
> >>>>         Recycle Current Volume = yes
> >>>>         Label Format =
> >>>> "sunday_even-${Year}-${Month:p/2/0/r}-${Day:p/2/0/r}"
> >>>> }
> >>>>
> >>>> Pool {
> >>>>         Name = sunday_odd
> >>>>         Pool Type = Backup
> >>>>         AutoPrune = yes
> >>>>         Volume Retention = 13 days
> >>>>         Volume Use Duration = 1 days
> >>>>         Recycle = yes
> >>>>         recyclepool= dimanche_impair
> >>>>         Maximum Volumes = 1
> >>>>         Recycle Current Volume = yes
> >>>>         Label Format =
> >>>> "sunday_odd-${Year}-${Month:p/2/0/r}-${Day:p/2/0/r}"
> >>>> }
> >>>>
> >>>> storages.conf
> >>>>      Storage {
> >>>> Name = FileStorage
> >>>> Address = "192.168.56.10"
> >>>> SDPORT = 9103
> >>>> Password = "test"
> >>>> Device = FileStorage
> >>>> Media Type = File
> >>>> }
> >>>>
> >>>> There wasn't any problem in the first week, but since the second week,
> >>>> i got a lot of messages of "is waiting on max storage jobs" from almost the
> >>>> jobs.
> >>>>
> >>>> Some of them got "is waiting for an appendable volume"
> >>>>
> >>>> Can you suggest me what to solve it ?
> >>>>
> >>>> Truc
> >>>>
> >>>
> >>>
> >>>
> >>> ------------------------------------------------------------------------------
> >>> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
> >>> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
> >>> with Interactivity, Sharing, Native Excel Exports, App Integration & more
> >>> Get technology previously reserved for billion-dollar corporations, FREE
> >>>
> >>> http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
> >>> _______________________________________________
> >>> Bacula-users mailing list
> >>> Bacula-users AT lists.sourceforge DOT net
> >>> https://lists.sourceforge.net/lists/listinfo/bacula-users
> >>>
> >>>
> >>
> >
>



------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users



------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk


_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users

------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users