Bacula-users

Re: [Bacula-users] Migrating form mysql to postgresql: Loading the database takes very long

2011-06-25 18:22:09
Subject: Re: [Bacula-users] Migrating form mysql to postgresql: Loading the database takes very long
From: Gavin McCullagh <gavin.mccullagh AT gcd DOT ie>
To: bacula-users AT lists.sourceforge DOT net
Date: Sat, 25 Jun 2011 23:18:24 +0100
On Tue, 21 Jun 2011, Marcus Mülbüsch wrote:

> I want to migrate my bacula installation from mysql to postgresql, 
> following the guide in 
> http://www.bacula.org/5.0.x-manuals/en/main/main/Installing_Configuring_Post.html#SECTION004450000000000000000
> 
> After dumping and converting the database the sql-file now has 9GiB.

One note I'd like to make on the page above is that the mysqldump command
includes the --skip-opt command.

This is not reliable to use if you have a reasonably large "File" table as
it includes an option to buffer query result in RAM before outputting to
file.  We have 65 million entries and our 6GB RAM + 2GB swap is not enough
to hold the buffer, so mysqldump ends up getting killed every time before
it finishes.

It seems you need to drop --quick which is implied in --skip-opt.  The
resulting command that I'm working with at the moment is:

mysqldump -t -n -c --compatible=postgresql --skip-quote-names --quick \
        --lock-tables --add-drop-table --add-locks --create-options 
--disable-keys \
        --extended-insert --set-charset  -u bacula -ppassword bacula \
        | grep -v "INSERT INTO Status"    \
        | sed -e 's/0000-00-00 00:00:00/1970-01-01 00:00:00/g' \
        | sed -e 's/\\0//' > bacula-backup.sql`

That being said, this is untested so far -- I haven't actually done the
migration -- but this is the plan thus far :-)

Feedback/corrections welcome...

Gavin


-- 
Gavin McCullagh
Senior System Administrator
IT Services
Griffith College 
South Circular Road
Dublin 8
Ireland
Tel: +353 1 4163365
http://www.gcd.ie
http://www.gcd.ie/brochure.pdf
http://www.gcd.ie/opendays
http://www.gcd.ie/ebrochure

This E-mail is from Griffith College.
The E-mail and any files transmitted with it are confidential and may be
privileged and are intended solely for the use of the individual or entity
to whom they are addressed. If you are not the addressee you are prohibited
from disclosing its content, copying it or distributing it otherwise than to
the addressee. If you have received this e-mail in error, please immediately
notify the sender by replying to this e-mail and delete the e-mail from your
computer.

Bellerophon Ltd, trades as Griffith College (registered in Ireland No.
60469) with its registered address as Griffith College Campus, South
Circular Road, Dublin 8, Ireland.


------------------------------------------------------------------------------
All the data continuously generated in your IT infrastructure contains a 
definitive record of customers, application performance, security 
threats, fraudulent activity and more. Splunk takes this data and makes 
sense of it. Business sense. IT sense. Common sense.. 
http://p.sf.net/sfu/splunk-d2d-c1
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users