Bacula-users

[Bacula-users] Bacula performance so slow ???

2009-10-13 08:41:11
Subject: [Bacula-users] Bacula performance so slow ???
From: Klaus Troeger <klaus AT linuxstar DOT de>
To: bacula-users AT lists.sourceforge DOT net
Date: Fri, 02 Oct 2009 06:12:02 +0200
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

i did a clean setup of bacula on a

Intel(R) Xeon(TM) CPU 3.60GHz, 3 GB Memory, Intel raid controller forming
5 internal 72 GB-320/10k SCSI LVD drives to a raid 5 array, where
everything is on.
My Quantum M1500 LTO-3 loader is connected via SCSI 160 LVD to the
Adaptec 39160
card.

OS is (but same results with Ubuntu 9.04 server):

[root@denbvsbcks1 ~]# uname -a
Linux denbvsbcks1.int.linuxstar.de 2.6.30.8-64.fc11.x86_64 #1 SMP Fri
Sep 25 04:43:32 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux
[root@denbvsbcks1 ~]# cat /etc/redhat-release
Fedora release 11 (Leonidas)

Bacula versions installed:

[root@denbvsbcks1 disk1]# rpm -qa | grep bacula
bacula-docs-2.4.4-3.fc11.x86_64
bacula-storage-common-2.4.4-3.fc11.x86_64
bacula-console-bat-2.4.4-3.fc11.x86_64
bacula-client-2.4.4-3.fc11.x86_64
bacula-sysconfdir-2.4.4-3.fc11.x86_64
bacula-console-2.4.4-3.fc11.x86_64
bacula-director-mysql-2.4.4-3.fc11.x86_64
bacula-console-gnome-2.4.4-3.fc11.x86_64
bacula-director-common-2.4.4-3.fc11.x86_64
bacula-console-wxwidgets-2.4.4-3.fc11.x86_64
bacula-common-2.4.4-3.fc11.x86_64
bacula-traymonitor-2.4.4-3.fc11.x86_64
bacula-storage-mysql-2.4.4-3.fc11.x86_64

Main configuration storage-related:

#
Autochanger {
  Name = M1500
  Device = LTO-3-0
  Changer Command = "/usr/libexec/bacula/mtx-changer %c %o %S %a %d"
  Changer Device = /dev/sg3
}

Device {
  Name = LTO-3-0
  Drive Index = 0
  Media Type = LTO-3
  Archive Device = /dev/nst0
  AutomaticMount = yes;               # when device opened, read it
  AlwaysOpen = yes;
  RemovableMedia = yes;
  RandomAccess = no;
  AutoChanger = yes
  LabelMedia = yes
  Alert Command = "sh -c 'tapeinfo -f %c |grep TapeAlert|cat'"
}

Bacua worked out of the box like a charm, except the performance.

I get between 3 -5 MB/sec, depending on type of backup (local backup
server itself is near 5, via GigaBit copper networt it's 3), too slow
for a
LTO-3 drive (equipped with "only" LTO-2 tapes, but ....)

Physical drive performance is 28sec for 1 Gigabyte, so ~35MB/sec

[root@denbvsbcks1 disk1]# dd if=/dev/zero of=swapfile bs=1024
count=1000000
1000000+0 records in
1000000+0 records out
1024000000 bytes (1.0 GB) copied, 79.907 s, 12.8 MB/s
[root@denbvsbcks1 disk1]# date;tar cvf /dev/nst0 swapfile ;date
Fri Oct  2 06:00:22 CEST 2009
swapfile
Fri Oct  2 06:00:50 CEST 2009

So, it's not the physical hardware, it has to be Bacula itself.

btape tests are o.k.
[root@denbvsbcks1 ~]# btape /dev/nst0
Tape block granularity is 1024 bytes.
btape: butil.c:285 Using device: "/dev/nst0" for writing.
02-Oct 05:38 btape JobId 0: 3301 Issuing autochanger "loaded? drive 0"
command.
02-Oct 05:38 btape JobId 0: 3302 Autochanger "loaded? drive 0", result
is Slot 1.
btape: btape.c:372 open device "LTO-3-0" (/dev/nst0): OK
*test

=== Write, rewind, and re-read test ===

I'm going to write 1000 records and an EOF
then write 1000 records and an EOF, then rewind,
and re-read the data to verify that it is correct.

This is an *essential* feature ...

btape: btape.c:831 Wrote 1000 blocks of 64412 bytes.
btape: btape.c:505 Wrote 1 EOF to "LTO-3-0" (/dev/nst0)
btape: btape.c:847 Wrote 1000 blocks of 64412 bytes.
btape: btape.c:505 Wrote 1 EOF to "LTO-3-0" (/dev/nst0)
btape: btape.c:856 Rewind OK.
1000 blocks re-read correctly.
Got EOF on tape.
1000 blocks re-read correctly.
=== Test Succeeded. End Write, rewind, and re-read test ===


=== Write, rewind, and position test ===

I'm going to write 1000 records and an EOF
then write 1000 records and an EOF, then rewind,
and position to a few blocks and verify that it is correct.

This is an *essential* feature ...

btape: btape.c:943 Wrote 1000 blocks of 64412 bytes.
btape: btape.c:505 Wrote 1 EOF to "LTO-3-0" (/dev/nst0)
btape: btape.c:959 Wrote 1000 blocks of 64412 bytes.
btape: btape.c:505 Wrote 1 EOF to "LTO-3-0" (/dev/nst0)
btape: btape.c:968 Rewind OK.
Reposition to file:block 0:4
Block 5 re-read correctly.
Reposition to file:block 0:200
Block 201 re-read correctly.
Reposition to file:block 0:999
Block 1000 re-read correctly.
Reposition to file:block 1:0
Block 1001 re-read correctly.
Reposition to file:block 1:600
Block 1601 re-read correctly.
Reposition to file:block 1:999
Block 2000 re-read correctly.
=== Test Succeeded. End Write, rewind, and re-read test ===



=== Append files test ===

This test is essential to Bacula.

I'm going to write one record  in file 0,
                   two records in file 1,
             and three records in file 2

btape: btape.c:475 Rewound "LTO-3-0" (/dev/nst0)
btape: btape.c:1577 Wrote one record of 64412 bytes.
btape: btape.c:1579 Wrote block to device.
btape: btape.c:505 Wrote 1 EOF to "LTO-3-0" (/dev/nst0)
btape: btape.c:1577 Wrote one record of 64412 bytes.
btape: btape.c:1579 Wrote block to device.
btape: btape.c:1577 Wrote one record of 64412 bytes.
btape: btape.c:1579 Wrote block to device.
btape: btape.c:505 Wrote 1 EOF to "LTO-3-0" (/dev/nst0)
btape: btape.c:1577 Wrote one record of 64412 bytes.
btape: btape.c:1579 Wrote block to device.
btape: btape.c:1577 Wrote one record of 64412 bytes.
btape: btape.c:1579 Wrote block to device.
btape: btape.c:1577 Wrote one record of 64412 bytes.
btape: btape.c:1579 Wrote block to device.
btape: btape.c:505 Wrote 1 EOF to "LTO-3-0" (/dev/nst0)
btape: btape.c:372 open device "LTO-3-0" (/dev/nst0): OK
btape: btape.c:475 Rewound "LTO-3-0" (/dev/nst0)
btape: btape.c:1097 Now moving to end of medium.
btape: btape.c:526 Moved to end of medium.
We should be in file 3. I am at file 3. This is correct!

Now the important part, I am going to attempt to append to the tape.

btape: btape.c:1577 Wrote one record of 64412 bytes.
btape: btape.c:1579 Wrote block to device.
btape: btape.c:505 Wrote 1 EOF to "LTO-3-0" (/dev/nst0)
btape: btape.c:475 Rewound "LTO-3-0" (/dev/nst0)
Done appending, there should be no I/O errors

Doing Bacula scan of blocks:
1 block of 64448 bytes in file 1
End of File mark.
2 blocks of 64448 bytes in file 2
End of File mark.
3 blocks of 64448 bytes in file 3
End of File mark.
1 block of 64448 bytes in file 4
End of File mark.
Total files=4, blocks=7, bytes = 451,136
End scanning the tape.
We should be in file 4. I am at file 4. This is correct!

The above Bacula scan should have output identical to what follows.
Please double check it ...
=== Sample correct output ===
1 block of 64448 bytes in file 1
End of File mark.
2 blocks of 64448 bytes in file 2
End of File mark.
3 blocks of 64448 bytes in file 3
End of File mark.
1 block of 64448 bytes in file 4
End of File mark.
Total files=4, blocks=7, bytes = 451,136
=== End sample correct output ===

If the above scan output is not identical to the
sample output, you MUST correct the problem
or Bacula will not be able to write multiple Jobs to
the tape.


=== Write, backup, and re-read test ===

I'm going to write three records and an EOF
then backup over the EOF and re-read the last record.
Bacula does this after writing the last block on the
tape to verify that the block was written correctly.

This is not an *essential* feature ...

btape: btape.c:475 Rewound "LTO-3-0" (/dev/nst0)
btape: btape.c:710 Wrote first record of 64412 bytes.
btape: btape.c:721 Wrote second record of 64412 bytes.
btape: btape.c:732 Wrote third record of 64412 bytes.
btape: btape.c:505 Wrote 1 EOF to "LTO-3-0" (/dev/nst0)
btape: btape.c:748 Backspaced over EOF OK.
btape: btape.c:753 Backspace record OK.
btape: btape.c:771
Block re-read correct. Test succeeded!
=== End Write, backup, and re-read test ===



=== Forward space files test ===

This test is essential to Bacula.

I'm going to write five files then test forward spacing

btape: btape.c:475 Rewound "LTO-3-0" (/dev/nst0)
btape: btape.c:1577 Wrote one record of 64412 bytes.
btape: btape.c:1579 Wrote block to device.
btape: btape.c:505 Wrote 1 EOF to "LTO-3-0" (/dev/nst0)
btape: btape.c:1577 Wrote one record of 64412 bytes.
btape: btape.c:1579 Wrote block to device.
btape: btape.c:1577 Wrote one record of 64412 bytes.
btape: btape.c:1579 Wrote block to device.
btape: btape.c:505 Wrote 1 EOF to "LTO-3-0" (/dev/nst0)
btape: btape.c:1577 Wrote one record of 64412 bytes.
btape: btape.c:1579 Wrote block to device.
btape: btape.c:1577 Wrote one record of 64412 bytes.
btape: btape.c:1579 Wrote block to device.
btape: btape.c:1577 Wrote one record of 64412 bytes.
btape: btape.c:1579 Wrote block to device.
btape: btape.c:505 Wrote 1 EOF to "LTO-3-0" (/dev/nst0)
btape: btape.c:1577 Wrote one record of 64412 bytes.
btape: btape.c:1579 Wrote block to device.
btape: btape.c:1577 Wrote one record of 64412 bytes.
btape: btape.c:1579 Wrote block to device.
btape: btape.c:505 Wrote 1 EOF to "LTO-3-0" (/dev/nst0)
btape: btape.c:1577 Wrote one record of 64412 bytes.
btape: btape.c:1579 Wrote block to device.
btape: btape.c:505 Wrote 1 EOF to "LTO-3-0" (/dev/nst0)
btape: btape.c:475 Rewound "LTO-3-0" (/dev/nst0)
btape: btape.c:1311 Now forward spacing 1 file.
We should be in file 1. I am at file 1. This is correct!
btape: btape.c:1323 Now forward spacing 2 files.
We should be in file 3. I am at file 3. This is correct!
btape: btape.c:475 Rewound "LTO-3-0" (/dev/nst0)
btape: btape.c:1336 Now forward spacing 4 files.
We should be in file 4. I am at file 4. This is correct!

btape: btape.c:1354 Now forward spacing 1 more file.
We should be in file 5. I am at file 5. This is correct!

=== End Forward space files test ===


Ah, I see you have an autochanger configured.
To test the autochanger you must have a blank tape
 that I can write on in Slot 1.

Do you wish to continue with the Autochanger test? (y/n): y


=== Autochanger test ===

3301 Issuing autochanger "loaded" command.
Slot 1 loaded. I am going to unload it.
3302 Issuing autochanger "unload 1 0" command.
unload status=OK 0
3303 Issuing autochanger "load 1 0" command.
3303 Autochanger "load 1 0" status is OK.
btape: btape.c:372 open device "LTO-3-0" (/dev/nst0): OK
btape: btape.c:1241 Rewound "LTO-3-0" (/dev/nst0)
btape: btape.c:1248 Wrote EOF to "LTO-3-0" (/dev/nst0)

The test autochanger worked!!

*q

Any clues, what the cause of this bad performance could be ???

Any help very apreciated

Thanks

Klaus
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkrFfZEACgkQzEBPDKWHbs9DYACePhy82VMbTNlSGZxI3PdTPQTH
zP8An0VY5xAm5vRJFqT8yVYzoUvDZbEb
=uFD1
-----END PGP SIGNATURE-----


------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay 
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users