Bacula-users

Re: [Bacula-users] Faster reading of Volumes from disk

2016-02-17 17:19:00
Subject: Re: [Bacula-users] Faster reading of Volumes from disk
From: Kern Sibbald <kern AT sibbald DOT com>
To: Dan Langille <dan AT langille DOT org>, bacula-users <bacula-users AT lists.sourceforge DOT net>
Date: Thu, 18 Feb 2016 08:13:25 +1000
scp can move binary copies of blocks very efficiently.  A copy (or 
migrate) job reads one record (generally smaller than blocks) one at a 
time, and repacks them into a new block.  The block is then checksummed 
(takes significant time) before it is written.  Look at your CPU usage 
that should largely explain the speed difference.


Best regards,
Kern

On 02/17/2016 06:21 AM, Dan Langille wrote:
> I can scp a 5GB Volume from one system to another in about 90 seconds.
>
> Why does that take the SD 8-10 minutes to do the same?
>
> The two systems are on the same 1Gb/s network.  We are copying from HDD
> local to SSD remote.
>
> I'm running a copy job, from disk on one SD to tape on another SD.  Data
> spooling is enabled.
>
> The full job can be seen here
> https://gist.github.com/dlangille/210fd8a8c3418f9004f6 :
>
> 15-Feb 13:32 crey-sd JobId 231187: Forward spacing Volume
> "FullAuto-3531" to file:block 0:216.
> 15-Feb 13:41 crey-sd JobId 231187: End of Volume at file 1 on device
> "vDrive-0" (/usr/local/bacula/volumes), Volume "FullAuto-3531"
> 15-Feb 13:41 crey-sd JobId 231187: Ready to read from volume
> "FullAuto-3534" on file device "vDrive-0" (/usr/local/bacula/volumes).
> 15-Feb 13:41 crey-sd JobId 231187: Forward spacing Volume
> "FullAuto-3534" to file:block 0:216.
> 15-Feb 13:51 crey-sd JobId 231187: End of Volume at file 1 on device
> "vDrive-0" (/usr/local/bacula/volumes), Volume "FullAuto-3534"
> 15-Feb 13:51 crey-sd JobId 231187: Ready to read from volume
> "FullAuto-3582" on file device "vDrive-0" (/usr/local/bacula/volumes).
> 15-Feb 13:51 crey-sd JobId 231187: Forward spacing Volume
> "FullAuto-3582" to file:block 0:216.
> 15-Feb 13:59 crey-sd JobId 231187: End of Volume at file 1 on device
> "vDrive-0" (/usr/local/bacula/volumes), Volume "FullAuto-3582"
> 15-Feb 13:59 crey-sd JobId 231187: Ready to read from volume
> "FullAuto-3594" on file device "vDrive-0" (/usr/local/bacula/volumes).
> 15-Feb 13:59 crey-sd JobId 231187: Forward spacing Volume
> "FullAuto-3594" to file:block 0:218.
>
> Each volume is 5G and it's taking the SD nearly three times longer than
> scp to transfer.
>
> $ ls -l FullAuto-3531 FullAuto-3534 FullAuto-3582  FullAuto-3594
> -rw-r-----  1 bacula  bacula  5368688762 Feb  7 03:26 FullAuto-3531
> -rw-r-----  1 bacula  bacula  5368688716 Feb  7 03:29 FullAuto-3534
> -rw-r-----  1 bacula  bacula  5368688096 Feb  7 04:29 FullAuto-3582
> -rw-r-----  1 bacula  bacula  5368688834 Feb  7 04:31 FullAuto-3594
>
> This is the device in use:
>
> Device {
>    Name           = vDrive-0
>    Media Type     = File
>    Archive Device = /usr/local/bacula/volumes
>    LabelMedia     = yes
>    Random Access  = yes
>    AutomaticMount = yes
>    RemovableMedia = no
>    AlwaysOpen     = no
>
>    Autochanger    = yes
>    Drive Index    = 0
>
>    Maximum Concurrent Jobs = 1
>    Volume Poll Interval    = 15
> }
>
> --
> Dan Langille - BSDCan / PGCon
> dan AT langille DOT org <mailto:dan AT langille DOT org>
>
>
>
>
>
>
> ------------------------------------------------------------------------------
> Site24x7 APM Insight: Get Deep Visibility into Application Performance
> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
> Monitor end-to-end web transactions and take corrective actions now
> Troubleshoot faster and improve end-user experience. Signup Now!
> http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
>
>
>
> _______________________________________________
> Bacula-users mailing list
> Bacula-users AT lists.sourceforge DOT net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>

------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users