Hi
I am trying to make Bacula work with my DVD+RW and DVD-RAM media, as raw
devices (similar to tape, but with random r/w support, which speeds up
appending and lseek64()ing to restore files). I have tested the access
method to properly work on Linux 2.6.x and NetBSD, and have been using
this method for other data archival purposes for a while. Other OSes
should support this seamlessly, as even Windows supports writing DVD-RAM
without the need of specialized applications or drivers.
I have included some test runs with 'dd' utility, as it is easily
understood and versatile enough to provide the actual tests, proving
that such access is viable. I have included only the NetBSD version of
the test, but should you require it, I can provide the Linux version as
well.
Motivation:
- I feel that accessing the device directly, just like Bacula does to
tape devices, is a better way to handle backups.
- There are some restrictions with appending a new ISO9660 session for
each append to the disk, one of them being able to correctly append data
to the SAME volume (no need to create ANOTHER file on the disk to hold
extra data)
- Even if negligible, creating a new session wastes some disk space
- Support for multiple sessions varies in implementation quality across
platforms (I recently had trouble recovering data from a multi-session
volume on a Debian system, and such data was available when mounted on
NetBSD, Windows and Slackware), but I expect "reading raw data from the
disk" to be portable across different OSes.
- I read on the lecture slides by *Kern Sibbald for *UKUUG that one
of the goals of the project is "assure data compatibility for 30 years
(providing you have the appropriate hardware)". It seems logical to me
that the less software layers we need to ensure that, the better, so raw
device is _my_ logical choice. (Even though DVD+RW might be low-quality
media, DVD-RAM is higher quality and is supposed to have much higher
shelf time. There are some manufacturers supposedly claiming 50 year
archival lifetime for DVD+RW already, so it might be worth considering.)
- Specially on Linux, I have big performance improvements whenever I use
raw devices instead of files within filesystems, for whatever reason.
This applies to Databases, Virtual Machine images, and other data as well.
- Avoid Data Spooling
- Avoid the overhead of flowing all the backup data through a
preprocessor (mkisofs, for instance), saving sometimes precious CPU
cycles in these days of virtualized servers.
The following scenarios list my test experiences with Bacula 2.2.4 on
NetBSD (I can, again, provide results with a more recent build if necessary)
Case 1:
Device Type = DVD
Archive Device = /dev/cd0d
Random Access = Yes;
RemovableMedia = yes;
RequiresMount = no;
# MountPoint =
# MountCommand =
# UnmountCommand =
# SpoolDirectory =
# WritePartCommand =
# FreeSpaceCommand =
result:
goldstein-sd ERROR TERMINATION
dev.c:195 Mount and unmount commands must defined for a device which
requires mount.
Case 2:
Device Type = Tape
Archive Device = /dev/cd0d
Random Access = Yes;
RemovableMedia = yes;
RequiresMount = no;
# MountPoint =
# MountCommand =
# UnmountCommand =
# SpoolDirectory =
# WritePartCommand =
# FreeSpaceCommand =
result:
goldstein# btape -v /dev/cd0d
Tape block granularity is 1024 bytes.
btape: butil.c:285 Using device: "/dev/cd0d" for writing.
19-Nov 23:05 btape: Fatal Error at device.c:296 because:
dev open failed: dev.c:422 Unable to open device "DVD Writer"
(/dev/cd0d): ERR=Bad file descriptor
19-Nov 23:05 btape: btape Fatal error: butil.c:194 Cannot open "DVD
Writer" (/dev/cd0d)
Case 3:
Device Type = File
Archive Device = /dev/cd0d
Random Access = Yes;
RemovableMedia = yes;
RequiresMount = no;
# MountPoint =
# MountCommand =
# UnmountCommand =
# SpoolDirectory =
# WritePartCommand =
# FreeSpaceCommand =
result:
Sending label command for Volume "TESTE" Slot 0 ...
3910 Unable to open device "DVD Writer" (/dev/cd0d): ERR=dev.c:480 Could
not open: /dev/cd0d/TESTE, ERR=Not a directory
For the following tests, the disk was forced to SPIN DOWN before the
beginning of each run and block sizes were chosen at random, so
real-world speeds should be way greater.
Small test to demonstrate the RW nature of the device, and the ability
to work with offsets (expected from DVD+RW and DVD-RAM media, unexpected
otherwise)
goldstein# dd if=/dev/urandom of=1mfile bs=2k count=512
512+0 records in
512+0 records out
goldstein# dd if=./1mfile of=/dev/cd0d bs=2k seek=2
512+0 records in
512+0 records out
1048576 bytes transferred in 0.774 secs (1354749 bytes/sec)
goldstein# dd of=./1mfile2 if=/dev/cd0d bs=2k skip=2 count=512
512+0 records in
512+0 records out
1048576 bytes transferred in 0.919 secs (1140996 bytes/sec)
goldstein# cmp ./1mfile ./1mfile2 || echo differ
goldstein#
Test to demonstrate the random access of the device, as expected from
DVD media.
goldstein# dd if=/dev/cd0d bs=1m seek=500 count=1 of=/dev/null
1+0 records in
1+0 records out
1048576 bytes transferred in 1.247 secs (840878 bytes/sec)
goldstein# dd if=/dev/cd0d bs=1m seek=700 count=1 of=/dev/null
1+0 records in
1+0 records out
1048576 bytes transferred in 1.518 secs (690761 bytes/sec)
Test to demonstrate the random WRITE access of the media (expected on
DVD+RW and DVD-RAM, unexpected otherwise)
goldstein# dd if=/dev/urandom bs=1k count=1 of=/dev/cd0d seek=100000
1+0 records in
1+0 records out
1024 bytes transferred in 3.046 secs (336 bytes/sec)
goldstein# dd if=/dev/cd0d bs=1k skip=100000 count=1 | hexdump -C
00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
|................|
*
00000400
1+0 records in
1+0 records out
1024 bytes transferred in 0.001 secs (1024000 bytes/sec)
Thanks for considering the idea
Evaldo Gardenali
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users
|
