Bacula-users

Re: [Bacula-users] Concurrent spooling and despooling

2010-02-16 15:55:29
Subject: Re: [Bacula-users] Concurrent spooling and despooling
From: "Edward M. Markowski" <emm AT edmarkowski DOT com>
To: Arno Lehmann <al AT its-lehmann DOT de>
Date: Tue, 16 Feb 2010 14:37:08 -0600 (CST)

I thought about this reciently, after I moved my spool dir to a 1.5 GB ramdisk, I had been using 100GB of a mirrored pair of 500GB SATA drives my throughtput to tape went up be a little over 10Mps.

A few thoughts, see inline......

On Tue, 16 Feb 2010, Arno Lehmann wrote:

Hi,

16.02.2010 11:50, Daniel Kamm wrote:
That's maybe a stupid question, but I really wonder...

Using Disk Spooling prior to write data to tape, heads in a sequential
write order:
a) write data from backup client to disk spool directory
b) write data from spool directory to tape

Why are those tasks done sequentially?

I know those three reasons:
- simultaneously spooling and despooling increases the requirements of
your spool space storage system even more

agree, but space is cheep, even boatloads of RAM  :-)


- it's never been implemented
- it's probably harder to implement than it looks at first.

One way to implements this is to have several smallish spool partitions with the producer and consumer processes never using the same one at the same time. For this to be most efficient the partitions would have to be on different disks/busses/controllers.

A better way in my opinion is to used a spool sized ring buffer in memory rather then a disk based spool. The consumer would only start after the producer had put a large set amount in it and continued until drained the buffer.



Let's say if the spool is reaching a waterlevel mark, the storage daemon
will start to write the spool data to tape _and_ at the same time the
backup client still sends data to the spool directory. Isn't that possible?

It's not possible now, but IIRC this has been discussed quite often,
but, while many users thought it might be a nice feature, nobody
started actually working on it.

There's actually a feature request regarding that, though it isn't on
the web site yet. It is, though still unnumbered, in the projects file
of the git tree.

Cheers,

Arno

Cheerz,
  - Dan

------------------------------------------------------------------------------
SOLARIS 10 is the OS for Data Centers - provides features such as DTrace,
Predictive Self Healing and Award Winning ZFS. Get Solaris 10 NOW
http://p.sf.net/sfu/solaris-dev2dev
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users


--
Arno Lehmann
IT-Service Lehmann
Sandstr. 6, 49080 Osnabrück
www.its-lehmann.de

------------------------------------------------------------------------------
SOLARIS 10 is the OS for Data Centers - provides features such as DTrace,
Predictive Self Healing and Award Winning ZFS. Get Solaris 10 NOW
http://p.sf.net/sfu/solaris-dev2dev
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users






!DSPAM:4b7b01f411881650419593!
------------------------------------------------------------------------------
SOLARIS 10 is the OS for Data Centers - provides features such as DTrace,
Predictive Self Healing and Award Winning ZFS. Get Solaris 10 NOW
http://p.sf.net/sfu/solaris-dev2dev
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users