Re: [Bacula-users] Bacula backup Clusters

On 05/26/2014 02:58 PM, Radosław Korzeniewski wrote:

Hello,

2014-05-24 18:15 GMT+02:00 Dmitri Maziuk <dmaziuk AT bmrb.wisc DOT edu>:

On 5/24/2014 1:58 AM, Radosław Korzeniewski wrote:
...

> It is the different problem. Yes, bacula could be a more robust
> regarding network problems.

I wonder how much state you'll to have keep to successfully reconnect.
E.g. re-establishing database connection at the start of a backup job
should be trivial, however, what if you get cut off from bacula-sd in
the middle of a write. There are downsides to restarting the job from
scratch.

There is a something like "Incomplete Job", but it is available in BEE only (I think, could someone correct me please).

Correct.

It is based on Accurete Job mechanism and allow to restart a backup job from the last successfully written file.

Correct.

To successfully implement a reconnection FD->SD (and others) you need to design and develop a reconnection protocol which will authenticate connection and switch it to the valid thread, which should swap a current connection handle with a new one. Additionally this protocol should handle a backup stream restart points, i.e. repeat the current/last file backup (you need to know what data SD received and wrote and what data you need to resend). Reconnecting DIR->FD connection should be simpler, because it does not exchange any interesting data during backup (not to count commands/status and accurate job data). For DIR->SD you need to implement something similar to dir->fd with additional metadata information transfer (logical sd->dir). In my opinion it is a lot of work and it was simpler for me to implement variable block level global deduplication for Bacula then implement above workflow. :) IMVHO.

The problem is not restarting the comm line, which is relatively easy. The real problem lies in figuring out exactly what was written and where to start writing. Currently this is done when an Incomplete job is restarted (currently only in BEE). It could be done dynamically, but that is a feature that is not as important as many others so it is not on the top of our development list at the moment.

Kern

best regards

--
Radosław Korzeniewski
radoslaw AT korzeniewski DOT net

------------------------------------------------------------------------------
The best possible search technologies are now affordable for all companies.
Download your FREE open source Enterprise Search Engine today!
Our experts will assist you in its installation for $59/mo, no commitment.
Test it for FREE on our Cloud platform anytime!
http://pubads.g.doubleclick.net/gampad/clk?id=145328191&iu=/4140/ostg.clktrk

_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users

------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and their 
applications. Written by three acclaimed leaders in the field, 
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/NeoTech

_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users