Bacula-users

Re: [Bacula-users] seeking advice re. splitting up large backups --dynamic filesets to prevent duplicate jobs and reduce backup time

2011-10-13 04:23:07
Subject: Re: [Bacula-users] seeking advice re. splitting up large backups --dynamic filesets to prevent duplicate jobs and reduce backup time
From: "Robert.Mortimer" <Robert.Mortimer AT lifestylegroup.co DOT uk>
To: <bacula-users AT lists.sourceforge DOT net>
Date: Thu, 13 Oct 2011 08:50:59 +0100
From: James Harper [mailto:james.harper AT bendigoit.com DOT au] 
> 
> In an effort to work around the fact that bacula kills long-running
jobs, I'm
> about to partition my backups into smaller sets. For example, instead
of
> backing up:
> 
>       /home
> 
> I would like to backup the content of /home as separate jobs. For
example:
> 

Have you thought of a script that creates links to a backup directory
split into the groups you want?
Run the linking script to link /home/[0-9]* to /backup/0-9/,
/home/[A-G]* to /backup/A-G, etc.

>       /home/[0-9]*
>       /home/[A-G]*
>       /home/[H-M]*
>       /home/[N-Q]*
>       /home/[R-U]*
>       /home/[V-Z]*
>       /home/[a-g]*
>       /home/[h-m]*
>       /home/[n-q]*
>       /home/[r-u]*
>       /home/[v-z]*
> 
> I'm looking for advice for how prevent multiple jobs of different
names, that
> access the same client, from running simultaneously. For example, to
> prevent an incremental of job "home0-9" running at the same time as a
full
> of job "homeA-G".
> 
> The only method I can think of is to use a dynamic fileset in the
director to
> generate the different filesets, so that there's only a single named
job that
> backs up a different set of files on each full backup. This way the
"Allow
> Duplicate Jobs" setting can be effective.
> 

Does Bacula really kill long running jobs? Or are you seeing the effect
of something at layer 3 or below (eg TCP connections timing out in
firewalls)?

I think your dynamic fileset idea would break Bacula's 'Accurate Backup'
code. If you are not using Accurate then it might work but it still
seems like a lot of trouble to go to to solve this problem.

If you limited the maximum jobs on the FD it would only run one at once,
but if the link was broken it might fail all the jobs.

Another option would be a "Run After" to start the next job. Only the
first job would be scheduled, and it would run the next job in turn.
Then they would all just run in series. You could even take it a step
further and have the "Run After" script to retry the same job if it
failed due to a connection problem, and to give up after so many
retries. Maybe it could even start pinging the FD to see if it was
reachable (if backing up over an unreliable link is the problem you are
trying to solve).

James

------------------------------------------------------------------------
------
All the data continuously generated in your IT infrastructure contains a
definitive record of customers, application performance, security
threats, fraudulent activity and more. Splunk takes this data and makes
sense of it. Business sense. IT sense. Common sense.
http://p.sf.net/sfu/splunk-d2d-oct

 
http://www.lifestylegroup.co.uk/
                               http://www.linkedin.com/company/39206 ;
 
This e-mail is confidential and only intended for the named recipient. If you 
are not the intended recipient please notify the sender immediately by phone or 
email. Please then delete it from your system. Any opinions are those of the 
author and do not necessarily represent those of Lifestyle Services Group Ltd. 
Please note that this e-mail and any attachments have not been encrypted. They 
may therefore be liable to be compromised. We do not accept any liability for 
any virus infection, external compromise of security or confidentiality in 
relation to e-mails.
 
Lifestyle Services Group Limited (LSG) is a company registered in England and 
Wales No. 5114385 and is authorised and regulated by the Financial Services 
Authority in respect of insurance mediation activities. Registered Office: 
Osprey House, Ore Close, Lymedale Business Park, Newcastle-under-Lyme, 
Staffordshire ST5 9QD.
 
_______________________________________________

Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users

------------------------------------------------------------------------------
All the data continuously generated in your IT infrastructure contains a
definitive record of customers, application performance, security
threats, fraudulent activity and more. Splunk takes this data and makes
sense of it. Business sense. IT sense. Common sense.
http://p.sf.net/sfu/splunk-d2d-oct
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users