Kern Sibbald wrote:
> Hello,
> 
> I haven't seen the original messages, so I am not sure if I understand the 
> full concept here so my remarks may not be pertinent.  

Personally I wasn't suggesting a format change - I'm pretty happy with
the fake-tape volumes for on-disk storage (though I wish bextract and
bls didn't need a configured sd to read them).

My own suggestion (which started this thread) boils down to "getting
decent concurrent writes to disk with a bacula-sd currently needs (IMO)
too much configuration of -sd and -dir and too much fiddling around.
Should a disk-aware sd provide a mechanism to open the same device
multiple times with DIFFERENT mounted volumes, auto-create disk devices
on demand, or otherwise provide an easier config-free way to
concurrently write to multiple volumes?".

If I wanted to propose another on-disk format, which I don't, I'd
propose zip64 with all metadata, checksum info etc not supported
directly by that format embedded in a custom-format .bacula file inside
the archive. That way backups could be extracted by something other than
bacula in a disaster recovery situation.

As it is, though, I'm pretty happy with the format, it's the way
bacula-sd acts as a gateway to disk storage that I think needs improvement.

BTW, When I suggested that greater write concurrency would be desirable
and should be easier, Phil Stracchino raised some concerns about
concurrent writes to a file system increasing fragmentation and hurting
overall performance. Rather than just wave my hands about that I've put
together a testing tool for concurrently writing big files from multiple
threads that optionally uses posix_fallocate to pre-allocate ahead in
chunks up to total file size. It tracks per-thread and total runtime and
time spent in write() and posix_fallocate() calls. I'm doing some tests
with it now and will post it to the bacula-devel list too. I'm planning
on testing the real effects of various levels of concurrent writes on
common file systems with it using a controlling shell script and will
report back when that eventually finishes.

--
Craig Ringer

------------------------------------------------------------------------------
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users