On 8/3/2010 7:09 PM, Rory Campbell-Lange wrote:

> Actually, this is what I don't get. Postgresql is a highly scalable,
> robust database system and it is being used as a data dump rather than a
> working tool for creating a transaction-based working catalogue.

Changes to one part of Bacula has to be compatible with all other parts 
of Bacula.  For example, given that we support SQLite, PostgreSQL, and 
MySQL, we have to keep each in mind.  Yes, it's a compromise.

> Yes, a batch insert is faster than a specfic insert, but the latter
> should be done at the "written-to-tape" transaction time, and could be
> done asynchronously, but in a transaction.

So... don't use batch-insert.  Go the single insert approach.  I dare 
you.  ;)

> Its pretty crazy for a>7TB
> tape backup to fail because of a temporary file/table problem at the END
> of the backup process rather than during it.

The running out of space is a PostgreSQL issue.

> Also the copy writes to a temporary table and then some rather curious
> inserts are done into the Bacula tables. E.g:
>
>       INSERT INTO
>          Path (Path)
>          SELECT
>              a.Path
>          FROM (
>              SELECT DISTINCT Path FROM batch
>          ) AS a
>          WHERE NOT EXISTS
>              (SELECT Path FROM Path WHERE Path = a.Path)
>
> This is a cludge (with an inefficient correlated subquery!) that could
> easily miss paths which exist from previous, unrelated backups. A
> continuous insert process against a job and mediaid simply wouldn't need
> to do this.

If you want to patch it, we'll certainly look at it.

> More native support for postgres would also allow, for instance, faster
> and more powerful searching of catalogues for retrieves, rather than the
> strange restore procedure required through bconsole.

Sure, it would.  We're always looking for more people to take on the 
heavy lifting.

> I'm delighted to be using Bacula (particularly after our tribulations
> with Amanda) but it seems to me that Bacula could lean much more heavily
> on Postgresql.

Yep.  I was this close to deploying Amanda when I found Bacula.  I 
switched immediately.  When the PostgreSQL module was written, it was 
based exactly upon the MySQL module.  Why?  Ease.  Walk first.  Run 
later.  As time has gone on, more PostgreSQL specific code has been 
added.  Now that we have extensive regression testing, such changes are 
easier to evaluate.

-- 
Dan Langille - http://langille.org/

------------------------------------------------------------------------------
The Palm PDK Hot Apps Program offers developers who use the
Plug-In Development Kit to bring their C/C++ apps to Palm for a share
of $1 Million in cash or HP Products. Visit us here for more details:
http://p.sf.net/sfu/dev2dev-palm
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users