Hi,
30.10.2008 04:47, Dan Langille wrote:
> SZÉKELYI Szabolcs wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> Hi,
>>
>> the ability to use FIFOs to produce the data to be backed up is a nice
>> feature of Bacula, however, I don't really understand how this can work
>> reliably.
>>
>> I understand the concept, but I have problems with timing. I did a small
>> research on how FIFOs work. Consider the following script:
>>
>> ==
>> #!/bin/bash
>>
>> exec > /dev/null
>>
>> mkfifo /tmp/tmpfifo
>>
>> (
>> for n in `seq 20`; do
>> echo $n >/tmp/tmpfifo
>> sleep 1
>> done
>> ) &
>> ==
>>
>> Let's use this script to generate the data to be backed up, so it is run
>> from RunBeforeJob. Creates a FIFO, writes the numbers from 1 to 20 into
>> it, with a 1 second delay between them.
You need to be aware that your loop does the following:
1. open the FIFO for writing
2. write into it
3. close the FIFO
4. sleep
5. loop
The important thing is that the FIFO is opened and closed for each write.
If, on the other hand, you opened the FIFO in the surrounding code,
redirected your output to it (for testing, do it similarly as you do
with 'exec > /dev/null') and closed the FIFO once the loop is done
you'll see your cat reads all the numbers in one go.
That's the difference.
> The loop runs in the background,
>> so the well-known deadlock with Bacula waiting for RunBeforeJob waiting
>> for Bacula to read the FIFO is avoided.
>>
>> When I run this script from a shell, it returns almost immediately as
>> expected. Next, running a `cat /tmp/tmpfifo` returns "1". Next time it
>> returns "2", up to 20. So it looks like cat reads an EOF after each line.
It does at least something similar - I'm unsure. But what I'm sure
about is that a read from a FIFO usually blocks when no writer has it
open.
By the way: This information is taken from 'man mkfifo', 'man 3
mkfifo' and 'man 7 fifo' on a linux system.
> I do not see how running it from a shell is comparable to a backup job.
At least the example is quite different...
>> What I can't understand is if Bacula reads a FIFO, how does it know if
>> the stream is finished or there's only a small pause in the input (which
>> is not unlikely, eg. for database dumps)? Furthermore, if the script
>> writing to the FIFO is unable to produce the data at least at the rate
>> Bacula can consume it, only half of your data gets backed up, since
>> sooner or later Bacula will catch up with the data producer and read an
>> EOF from the FIFO (like cat does above).
>
> Have you let such a job run to completion?
>
> How do you know this example loop is representative of what a FIFO would do?
>
>> Can you confirm that this is a real threat or am I overlooking something?
>
> I think people much smarter than you or I have created and tested this.
One very good reason to trust FIFO backups in Bacula :-)
> People have been using FIFO in production for some time. I also think
> your question is not Bacula specific but FIFO generic.
>
> My conclusion: it works. It is not a real 'threat'.
>
> I say this completely without knowing much about FIFO but with the
> knowledge of the experience level of the people who implemented it. :)
I agree, even if I did look at this myself just a bit.
Arno
>
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Bacula-users mailing list
> Bacula-users AT lists.sourceforge DOT net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
--
Arno Lehmann
IT-Service Lehmann
Sandstr. 6, 49080 Osnabrück
www.its-lehmann.de
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users
|
