Bacula-users

Re: [Bacula-users] Bacula + Postgres : copy batch problem

2010-08-09 20:20:28
Subject: Re: [Bacula-users] Bacula + Postgres : copy batch problem
From: Dan Langille <dan AT langille DOT org>
To: Rory Campbell-Lange <rory AT campbell-lange DOT net>
Date: Mon, 09 Aug 2010 20:16:41 -0400
On 8/9/2010 4:23 PM, Rory Campbell-Lange wrote:
> On 04/08/10, Rory Campbell-Lange (rory AT campbell-lange DOT net) wrote:
>> On 03/08/10, Dan Langille (dan AT langille DOT org) wrote:
>>> On 8/3/2010 7:09 PM, Rory Campbell-Lange wrote:
>>
>>>> Yes, a batch insert is faster than a specfic insert, but the latter
>>>> should be done at the "written-to-tape" transaction time, and could be
>>>> done asynchronously, but in a transaction.
>>>
>>> So... don't use batch-insert.  Go the single insert approach.  I dare
>>> you.  ;)
>>
>> Yes, I'm trying! I'm trying to do it properly by recompiling debian
>> stable backports and I'm running into library dependancy problems.
>
> Done. I've recompiled with batch inserts set to off.
>
> I tested spooling to see how this would work although it isn't strictly
> necessary for my situation (a single server with AoE and internal
> storage and a locally attached tape drive). The backup started running
> off disk at around 100MB/s and then spooling at around 100MB/s. The disk
> copy slowed down dramatically over the weekend due to contention due to
> some external MD5 audits and rsync processes.

Well, spooling data to disk first makes sense if your network cannot 
keep up with your tape drive.  You want to avoid start/stop on the tape.

Spooling attributes is different.  You may want to try that on and then 
off to see how things go.  Off is what we wanted to avoid I think.

>
> I'm now going to play with non-spooled backups and some options such as
> noatime.
>
>    Build OS:               x86_64-pc-linux-gnu debian 5.0.5
>    JobId:                  1
>    Job:                    HAbkp.2010-08-06_15.46.23_03
>    Backup Level:           Full
>    Client:                 "clwbackup-fd" 5.0.2 (28Apr10) 
> x86_64-pc-linux-gnu,debian,5.0.5
>    FileSet:                "HAfileset" 2010-08-06 15:46:23
>    Pool:                   "HAPool" (From Job resource)
>    Catalog:                "MyCatalog" (From Client resource)
>    Storage:                "CLW_LTO4" (From Job resource)
>    Scheduled time:         06-Aug-2010 15:46:07
>    Start time:             06-Aug-2010 15:46:25
>    End time:               09-Aug-2010 18:23:16
>    Elapsed time:           3 days 2 hours 36 mins 51 secs
>    Priority:               10
>    FD Files Written:       7,706,717
>    SD Files Written:       7,706,717
>    FD Bytes Written:       7,337,839,018,613 (7.337 TB)
>    SD Bytes Written:       7,339,475,824,330 (7.339 TB)
>    Rate:                   27317.7 KB/s
>    Software Compression:   None
>    VSS:                    no
>    Encryption:             no
>    Accurate:               no
>    Volume name(s):         HA-01|HA-02|HA-03|HA-04|HA-05|HA-06|HA-07
>    Volume Session Id:      1
>    Volume Session Time:    1281090914
>    Last Volume Bytes:      896,114,386,944 (896.1 GB)
>    Non-fatal FD errors:    0
>    SD Errors:              0
>    FD termination status:  OK
>    SD termination status:  OK
>    Termination:            Backup OK
>
>


-- 
Dan Langille - http://langille.org/

------------------------------------------------------------------------------
This SF.net email is sponsored by 

Make an app they can't live without
Enter the BlackBerry Developer Challenge
http://p.sf.net/sfu/RIM-dev2dev 
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users