Hi!
On Wed, Dec 3, 2008 at 10:46 AM, Tobias Bartel <tbartel AT eteleon DOT de>
wrote:
> Hello,
>
> sorry for my late response.
>
> I just finished migrating the catalog to a Postgre 8.1 Server and
> enabled spooling. Spooling space is 100GB, the horror directory only
> 70GB. I hope that both changes will give us a speed bump, well the
> Postgre should increase the performance and the spool dir should at
> least decrease the tapewear.
1. Did you optimized the PostgreSQL database to actually use the
server? (if you stick with defaults, you will see a bad performance
with many entries).
2. Please measure the speed of the spool space: if it is under 30MB/s,
it is not good for your LTO-3 drive.
I use these commands to give an approx of disk speed:
dd if=/dev/zero of=cosa.img bs=10M count=100 conv=fsync (this gives
write performance)
dd if=cosa.img of=/dev/null bs=10M iflag=direct (read performance).
>
> I just started another test, lets see how long it will take ;)
If everything is ok, it should take around 4 to 20 hours (max).
I hope this helps,
Ildefonso Camargo
>
> cya tobi
>
>
> Am Donnerstag, den 27.11.2008, 12:25 -0500 schrieb Ryan Novosielski:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> Tobias Bartel wrote:
>> > Hello,
>> >
>> >> Even with 800,000 files, that sounds very slow. How much data is
>> >> involved, how is it stored and how fast is your database server?
>> >
>> > It's about 70GB of data, stored on a Raid5 (3Ware controller).
>> >
>> > The database is a SQLite one, on the same machine but on a Software
>> > Raid 1.
>> >
>> > The backup device is an LTO3 connected via SCSI
>> >
>> > OS is a Debian stable.
>> >
>> >
>> > I already thought about moving the Database to MySQL but there is
>> > already a MySQL Server on the same box, it is a slave for our MySQL
>> > master and used for hourly Backups of our database (Stop the
>> > replication, do the backups and start the replication again).
>> > I don't really like the idea of adding a DB to the Slave that isn't on
>> > the master, nor do i like the idea of hacking up some custom MySQL
>> > install that runs parallel caus that will cost me with every future
>> > update.
>> >
>> > To be honest, i didn't expect that SQLite could be the bottle neck, it
>> > just can't be that slow. What made me think that its the number files,
>> > is that when i do an ls in that directory it takes ~15min before I see
>> > any output.
>>
>> My understanding is that you cannot expect decent performance out of
>> SQLite for Bacula for any production level backup. I could be wrong
>> here, but I say forget about SQLite for anything other than a trial, and
>> definitely not using it for a backup that is extra demanding.
>>
>> You could use PostgreSQL if you wanted to avoid messing with the slave
>> server (though something tells me that's not a major worry, but I am not
>> sure about it), or just run MySQL on a different port which I don't
>> think is all that hard (or, actually, use it in socket-only mode, which
>> is even easier and I think would suffice).
>>
>> - --
>> ---- _ _ _ _ ___ _ _ _
>> |Y#| | | |\/| | \ |\ | | |Ryan Novosielski - Systems Programmer II
>> |$&| |__| | | |__/ | \| _| |novosirj AT umdnj DOT edu - 973/972.0922
>> (2-0922)
>> \__/ Univ. of Med. and Dent.|IST/AST - NJMS Medical Science Bldg - C630
>> -----BEGIN PGP SIGNATURE-----
>> Version: GnuPG v1.4.6 (GNU/Linux)
>> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
>>
>> iD8DBQFJLtgHmb+gadEcsb4RApbKAJ4gTg9fF8susc4iS6e44D9s7uWTxwCg2T/n
>> hd0IuSIG6mg6J4FPrL/aRz8=
>> =M8R0
>> -----END PGP SIGNATURE-----
>> -------------------------------------------------------------------------
>> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
>> Build the coolest Linux based applications with Moblin SDK & win great prizes
>> Grand prize is a trip for two to an Open Source event anywhere in the world
>> http://moblin-contest.org/redirect.php?banner_id=100&url=/
>> _______________________________________________ Bacula-users mailing list
>> Bacula-users AT lists.sourceforge DOT net
>> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Bacula-users mailing list
> Bacula-users AT lists.sourceforge DOT net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users
|
