Bacula-users

Re: [Bacula-users] seeking advice re. splitting up large backups -- dynamic filesets to prevent duplicate jobs and reduce backup time

2011-10-12 20:57:07
Subject: Re: [Bacula-users] seeking advice re. splitting up large backups -- dynamic filesets to prevent duplicate jobs and reduce backup time
From: "James Harper" <james.harper AT bendigoit.com DOT au>
To: <mark.bergman AT uphs.upenn DOT edu>, <bacula-users AT lists.sourceforge DOT net>
Date: Thu, 13 Oct 2011 11:54:47 +1100
> 
> In an effort to work around the fact that bacula kills long-running
jobs, I'm
> about to partition my backups into smaller sets. For example, instead
of
> backing up:
> 
>       /home
> 
> I would like to backup the content of /home as separate jobs. For
example:
> 
>       /home/[0-9]*
>       /home/[A-G]*
>       /home/[H-M]*
>       /home/[N-Q]*
>       /home/[R-U]*
>       /home/[V-Z]*
>       /home/[a-g]*
>       /home/[h-m]*
>       /home/[n-q]*
>       /home/[r-u]*
>       /home/[v-z]*
> 
> I'm looking for advice for how prevent multiple jobs of different
names, that
> access the same client, from running simultaneously. For example, to
> prevent an incremental of job "home0-9" running at the same time as a
full
> of job "homeA-G".
> 
> The only method I can think of is to use a dynamic fileset in the
director to
> generate the different filesets, so that there's only a single named
job that
> backs up a different set of files on each full backup. This way the
"Allow
> Duplicate Jobs" setting can be effective.
> 

Does Bacula really kill long running jobs? Or are you seeing the effect
of something at layer 3 or below (eg TCP connections timing out in
firewalls)?

I think your dynamic fileset idea would break Bacula's 'Accurate Backup'
code. If you are not using Accurate then it might work but it still
seems like a lot of trouble to go to to solve this problem.

If you limited the maximum jobs on the FD it would only run one at once,
but if the link was broken it might fail all the jobs.

Another option would be a "Run After" to start the next job. Only the
first job would be scheduled, and it would run the next job in turn.
Then they would all just run in series. You could even take it a step
further and have the "Run After" script to retry the same job if it
failed due to a connection problem, and to give up after so many
retries. Maybe it could even start pinging the FD to see if it was
reachable (if backing up over an unreliable link is the problem you are
trying to solve).

James

------------------------------------------------------------------------------
All the data continuously generated in your IT infrastructure contains a
definitive record of customers, application performance, security
threats, fraudulent activity and more. Splunk takes this data and makes
sense of it. Business sense. IT sense. Common sense.
http://p.sf.net/sfu/splunk-d2d-oct
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users