Re: [Bacula-users] Questions about spooling

In the message dated: Thu, 01 Sep 2011 14:09:36 +0200,
The pithy ruminations from Marcello Romani on 
<Re: [Bacula-users] Questions about spooling> were:
=> Il 01/09/2011 13:38, frank_sg ha scritto:
=> > Thanks fpr replying.
=> >
=> > @Alexandre: Yes, exactly, the 10MB/s (average) come from bacula job
=> > report.
=> >
=> > @Marcello: No - very time the spool fs is full (or the maximum spool
=> > size per job etc.) is reached the spool fs is despooled to tape. And
=> > that is where I hope to get the advantage from: despooling with full

There is an advantage from despooling from a fast filesystem to tape--but
that "advantage" is more in terms of saving wear & tear on the tape
drive and media than in speed.

In my tests (and others [1][2]), because bacula stops collecting files
from clients as soon as it begins despooling to tape, the overall
throughput will never be as great as the filesystem is capable of
providing.

        [1] http://copilotco.com/mail-archives/bacula-devel.2007/msg02642.html
        [2] 
http://www.bacula.org/git/cgit.cgi/bacula/plain/bacula/projects?h=Branch-5.1

=> > speed to tape. You are right, the fs has to be fast enough - which
=> > SSDs would be obviously. That the reason for the idea about them. A
=> > RAID0 with 12 disks direct attached with SAS should also be fast
=> > enough - but which option is the one to choose?

Either option is much more than fast enough.


=> >
=> > Regards, Frank
=> >
=> 

        [SNIP!]

=> 
=> About job speed: notice how the final speed reported for the whole job 
=> is very low compared to the despooling speed.
=> That figure can be misleading. I think one has to consider transfer rate 
=> and despooling speed separately.

Maybe. From the point of view of selecting components and tuning, yes,
it does make sense to consider the transfer rates (client to spool file
and spool file to tape) separately. However, the most important value
for any business is the cumulative time--how long it takes to backup a
client, not the individual parts that make up that time.

Here are some numbers from our environment:

[A]     Throughput w/o spooling: ~22MB/s
                this represents the aggregate of the speed to read data
                from disk and write to tape, with shoe-shining, network
                congestion, disk contention, etc.

[B]     Throughput to spool file: ~55MB/s
                this represents the aggregate of the speed to read data
                from disk (a 9TB logical volume made up from multiple
                RAID5 and RAID6 LUNs) via 4Mb/s fibre and write to the
                RAID-10 spool partition on 10K RPM SAS disks. This
                includes any SAN congestion, disk contention, etc.

[C]     Throuput from disk spool file to LTO-4 tape: ~108MB/s
                This is the raw despooling-speed from 10K SAS disks
                to the tape drive over 4Mb/s fibre.

[D]     End-to-end throughput with spooling: ~27MB/s
                This is very disappointing...this is the overall
                throughput of [B] + [C] above. While eliminating
                shoe-shining is much better for the tape media and tape
                drive, the overall performance is almost identical to
                [A], while it should be close to [B]. The reason for
                the decrease in performance is that bacula stops all
                spooling as soon as it starts de-spooling.


In this case, the imporant value is in [D]...that determines the total time
for a backup. It really doesn't matter that the throughput from the spool disk
to the tape drive is 4x greater than the aggregate throughput, because
bacula's design prohibits better performance.

In an ideal configuration, there could be multiple spool directories defined,
and bacula would open a new spool file in the next directory as soon as it
begins despooling.

=> 
=> -- 
=> Marcello Romani
=> 

Thanks,

Mark

---advertisements below this line were added without my consent or endorsement--

------------------------------------------------------------------------------
Special Offer -- Download ArcSight Logger for FREE!
Finally, a world-class log management solution at an even better 
price-free! And you'll get a free "Love Thy Logs" t-shirt when you
download Logger. Secure your free ArcSight Logger TODAY!
http://p.sf.net/sfu/arcsisghtdev2dev
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users