Bacula-users

Re: [Bacula-users] Slow Attribute spooling

2015-06-09 03:52:04
Subject: Re: [Bacula-users] Slow Attribute spooling
From: Denis Witt <denis.witt AT concepts-and-training DOT de>
To: Alan Brown <ajb2 AT mssl.ucl.ac DOT uk>
Date: Tue, 9 Jun 2015 09:49:21 +0200
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Mon, 08 Jun 2015 19:13:29 +0100
Alan Brown <ajb2 AT mssl.ucl.ac DOT uk> wrote:

> The problem with adding more concurrent jobs (up from one) is that
> the increased seek load will badly affect despooling time.

Well, at least at the moment this isn't a problem. If I understand
correctly there will be no data spooling during attribute despooling.
At least bacula keeps telling me no other job is running.

So during the attribute despooling there is no concurrent disk access.
And because there are only attributes from one job it should be a
rather sequential read operation.

> 0: Spooling areas _must_ be on seperate spindles to everythng else.

[x] checked

BTW: Data despooling is quite fast, only attribute despooling is
extremely slow.

> 1: If you have spindles, you will see an exponential-"plus a bit" 
> dropoff in overall throughput for each extra task added to the spool 
> array from what's seen with a single job - output divided amongst 
> jobs-in-progress, plus you've added effectively random seek latency
> of 8-12ms on top.

My goal was to have one job writing to spool while despooled data is
written on tape, as I have some machines with a large amount of very
small files. When I write them directly to tape I'm only able to send
about 10MB/s to the library. Despooled data is up to 10 times faster
(still not perfect but obviously better).

[several points i'll keep in mind, thanks]
 
> 7: If you can afford it, tossing 100GB of ramdrive at the task is

Well, no. ;)

> 8: It doesn't hurt to put the database on a dedicated raid10 SSD set.

I'll take a look if I got the attribute despooling working. Without
spooling my backups take about 40 hours (full-backup). 48 would be
fine, but as data will grow bigger over times I wanted to be prepared.

Thanks for sharing your experiences, I appreciate.

Regards.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)

iQIcBAEBAgAGBQJVdpqBAAoJELB26T3daKAJ9fUP/RmSfNyTH0ZUmvdtib/OUmTd
oDAmNdUe8uhuEZGAn8B+wrJMZ3nC6AuaMFP1iuQAu1qEOOpHQAh+2k5DaKI2t7mW
t1UYF/MvDUcZJuzkHkuVquh4vMVU7A7VhhEY+RvpGl99gl2jBlIGhbKG/rkeQ0Ue
nw8CL8yQclvyg0p8f6APE3RsvzHapn8t7wjZfJ4zK/Uo2lkeK6FiuDqppFGwvdpV
H0ziMIQHc6/WJI5J4l3hZNbrjYAaoaI1wchiYtA2QuWs9gLKn/M8qndPE77gNl/G
5VbMS+Kod7JCltt3p2fNA+TXqa6O2p71VE4fE4+RTO57mpL+oPQ47D5iYXjeJ6TH
mdWSqCzBxI7W27EqaS0pjYY5wYBD79nAZAGrjleaZLtmCOLLUU4wbJalZBogCI6g
lAFySMA7n/C6jz30Ayc+rImdfXXfJ74mNFvxMpyGjfxKP2E+TjeSAA5czAyupru9
6L+2tNTfTj3d2Kfkbrp+8wJhPUnMeUBp4Ak+pPNvcEncZ5aSv4sEl6XYK2RUs64M
YzIv73Rakqgv9QsE8uVnXfX97wvXWIij2W6McR1QJtc1N5jcbdgPsR9LA4WgFIo4
2D9ZjMudE96VoDWgrntF25dK6IxsPk+D5R8W4Tx0a1OE63eR1gJUYkYm4FnWt76B
gQ2FDM9aZx7H4OQSBI0G
=CR7r
-----END PGP SIGNATURE-----
------------------------------------------------------------------------------
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users