Re: [Bacula-users] multiple concurrent tape jobs

On Feb 8, 2016, at 5:42 PM, Dan Langille <dan AT langille DOT org> wrote:

Hello,

I am working with an LTO-4 tape library. It has two drives but I plan to write to only one for backups.

I will backup to disk first, on another SD. Later, I will copy the jobs to the tape library on this new SD
which is on another server. The copy jobs will be spooled to local SSD before being written to tape.

re http://bacula-users.narkive.com/QRkTVfEz/typical-tape-write-performance

Now what I'm thinking of is streaming multiple concurrent jobs to a single drive.

Sure, downside on restore is interleaving of blocks....

I don't see any downsides to going down this path. I have yet to run any copy jobs to the new library,
but it may be ready this week.

Comments?

There have been good suggestions, but I'm posting below my original email to maintain context.

I was just reading: http://www.bacula.org/7.4.x-manuals/en/main/Data_Spooling.html

Main points:

With concurrent tape jobs, only one job will write to tape at a time:

• When Bacula begins despooling data spooled to disk, it takes exclusive use of the tape. This has the major advantage that in running multiple simultaneous jobs at the same time, the blocks of several jobs will not be intermingled.

Multiple jobs can spool concurrently:

• If you are running multiple simultaneous jobs, Bacula will continue spooling other jobs while one is despooling to tape, provided there is sufficient spool file space.

I ran my first copy job this morning.

* 18 minutes to copy 7.075 GB from one SD to the other, spooling onto SSD.

* 7 minutes to spool 577,884,981 bytes back to the Director.

14-Feb 16:30 bacula-dir JobId 231106: Warning: FileSet MD5 digest not found.
14-Feb 16:30 bacula-dir JobId 231106: The following 1 JobId was chosen to be copied: 225267
14-Feb 16:30 bacula-dir JobId 231106: Copying using JobId=225267 Job=supernews_FP_msgs.2015-12-06_03.05.35_25
14-Feb 16:30 bacula-dir JobId 231106: Bootstrap records written to /usr/local/bacula/working/bacula-dir.restore.26.bsr
14-Feb 16:30 bacula-dir JobId 231106: Start Copying JobId 231106, Job=CopyToTape-Full-Just-One-tape-01.2016-02-14_16.30.41_38
14-Feb 16:30 bacula-dir JobId 231106: Using Device "vDrive-0" to read.
14-Feb 16:30 bacula-dir JobId 231107: Using Device "LTO_0" to write.
14-Feb 16:30 crey-sd JobId 231106: Ready to read from volume "FullAuto-3337" on file device "vDrive-0" (/usr/local/bacula/volumes).
14-Feb 16:30 crey-sd JobId 231106: Forward spacing Volume "FullAuto-3337" to file:block 0:1957810277.
14-Feb 16:31 crey-sd JobId 231106: End of Volume at file 1 on device "vDrive-0" (/usr/local/bacula/volumes), Volume "FullAuto-3337"
14-Feb 16:31 crey-sd JobId 231106: Ready to read from volume "FullAuto-3340" on file device "vDrive-0" (/usr/local/bacula/volumes).
14-Feb 16:31 crey-sd JobId 231106: Forward spacing Volume "FullAuto-3340" to file:block 0:64728.
14-Feb 16:31 crey-sd JobId 231106: End of Volume at file 1 on device "vDrive-0" (/usr/local/bacula/volumes), Volume "FullAuto-3340"
14-Feb 16:31 crey-sd JobId 231106: Ready to read from volume "FullAuto-3341" on file device "vDrive-0" (/usr/local/bacula/volumes).
14-Feb 16:31 crey-sd JobId 231106: Forward spacing Volume "FullAuto-3341" to file:block 0:216.
14-Feb 16:33 crey-sd JobId 231106: End of Volume at file 1 on device "vDrive-0" (/usr/local/bacula/volumes), Volume "FullAuto-3341"
14-Feb 16:33 crey-sd JobId 231106: Ready to read from volume "FullAuto-3351" on file device "vDrive-0" (/usr/local/bacula/volumes).
14-Feb 16:33 crey-sd JobId 231106: Forward spacing Volume "FullAuto-3351" to file:block 0:64728.
14-Feb 16:34 crey-sd JobId 231106: End of Volume at file 1 on device "vDrive-0" (/usr/local/bacula/volumes), Volume "FullAuto-3351"
14-Feb 16:34 crey-sd JobId 231106: Ready to read from volume "FullAuto-3357" on file device "vDrive-0" (/usr/local/bacula/volumes).
14-Feb 16:34 crey-sd JobId 231106: Forward spacing Volume "FullAuto-3357" to file:block 0:216.
14-Feb 16:36 crey-sd JobId 231106: End of Volume at file 1 on device "vDrive-0" (/usr/local/bacula/volumes), Volume "FullAuto-3357"
14-Feb 16:36 crey-sd JobId 231106: Ready to read from volume "FullAuto-3370" on file device "vDrive-0" (/usr/local/bacula/volumes).
14-Feb 16:36 crey-sd JobId 231106: Forward spacing Volume "FullAuto-3370" to file:block 0:64728.
14-Feb 16:37 crey-sd JobId 231106: End of Volume at file 1 on device "vDrive-0" (/usr/local/bacula/volumes), Volume "FullAuto-3370"
14-Feb 16:37 crey-sd JobId 231106: Ready to read from volume "FullAuto-3373" on file device "vDrive-0" (/usr/local/bacula/volumes).
14-Feb 16:37 crey-sd JobId 231106: Forward spacing Volume "FullAuto-3373" to file:block 0:64728.
14-Feb 16:38 crey-sd JobId 231106: End of Volume at file 1 on device "vDrive-0" (/usr/local/bacula/volumes), Volume "FullAuto-3373"
14-Feb 16:38 crey-sd JobId 231106: Ready to read from volume "FullAuto-3382" on file device "vDrive-0" (/usr/local/bacula/volumes).
14-Feb 16:38 crey-sd JobId 231106: Forward spacing Volume "FullAuto-3382" to file:block 0:216.
14-Feb 16:40 crey-sd JobId 231106: End of Volume at file 1 on device "vDrive-0" (/usr/local/bacula/volumes), Volume "FullAuto-3382"
14-Feb 16:40 crey-sd JobId 231106: Ready to read from volume "FullAuto-3391" on file device "vDrive-0" (/usr/local/bacula/volumes).
14-Feb 16:40 crey-sd JobId 231106: Forward spacing Volume "FullAuto-3391" to file:block 0:64728.
14-Feb 16:42 crey-sd JobId 231106: End of Volume at file 1 on device "vDrive-0" (/usr/local/bacula/volumes), Volume "FullAuto-3391"
14-Feb 16:42 crey-sd JobId 231106: Ready to read from volume "FullAuto-3393" on file device "vDrive-0" (/usr/local/bacula/volumes).
14-Feb 16:42 crey-sd JobId 231106: Forward spacing Volume "FullAuto-3393" to file:block 0:64728.
14-Feb 16:43 crey-sd JobId 231106: End of Volume at file 1 on device "vDrive-0" (/usr/local/bacula/volumes), Volume "FullAuto-3393"
14-Feb 16:43 crey-sd JobId 231106: Ready to read from volume "FullAuto-3397" on file device "vDrive-0" (/usr/local/bacula/volumes).
14-Feb 16:43 crey-sd JobId 231106: Forward spacing Volume "FullAuto-3397" to file:block 0:64728.
14-Feb 16:45 crey-sd JobId 231106: End of Volume at file 1 on device "vDrive-0" (/usr/local/bacula/volumes), Volume "FullAuto-3397"
14-Feb 16:45 crey-sd JobId 231106: Ready to read from volume "FullAuto-3398" on file device "vDrive-0" (/usr/local/bacula/volumes).
14-Feb 16:45 crey-sd JobId 231106: Forward spacing Volume "FullAuto-3398" to file:block 0:64728.
14-Feb 16:46 crey-sd JobId 231106: End of Volume at file 1 on device "vDrive-0" (/usr/local/bacula/volumes), Volume "FullAuto-3398"
14-Feb 16:46 crey-sd JobId 231106: Ready to read from volume "FullAuto-3401" on file device "vDrive-0" (/usr/local/bacula/volumes).
14-Feb 16:46 crey-sd JobId 231106: Forward spacing Volume "FullAuto-3401" to file:block 0:64728.
14-Feb 16:48 crey-sd JobId 231106: End of Volume at file 1 on device "vDrive-0" (/usr/local/bacula/volumes), Volume "FullAuto-3401"
14-Feb 16:48 crey-sd JobId 231106: End of all volumes.
14-Feb 16:48 crey-sd JobId 231106: Elapsed time=00:17:25, Transfer rate=6.770 M Bytes/second
14-Feb 16:48 tape01-sd JobId 231107: Elapsed time=00:17:26, Transfer rate=6.763 M Bytes/second
14-Feb 16:48 tape01-sd JobId 231107: Sending spooled attrs to the Director. Despooling 577,884,981 bytes ...
14-Feb 16:55 bacula-dir JobId 231106: Bacula bacula-dir 7.4.0 (16Jan16):
Build OS: amd64-portbld-freebsd10.2 freebsd 10.2-RELEASE-p8
Prev Backup JobId: 225267
Prev Backup Job: supernews_FP_msgs.2015-12-06_03.05.35_25
New Backup JobId: 231107
Current JobId: 231106
Current Job: CopyToTape-Full-Just-One-tape-01.2016-02-14_16.30.41_38
Backup Level: Full
Client: tape01-fd
FileSet: "EmptyCopyToTape" 2011-02-20 20:53:31
Read Pool: "FullFile" (From Job resource)
Read Storage: "CreyFile" (From Pool resource)
Write Pool: "FullsLTO4" (From Job resource)
Write Storage: "tape01" (From Job resource)
Catalog: "MyCatalog" (From Client resource)
Start time: 14-Feb-2016 16:30:44
End time: 14-Feb-2016 16:55:13
Elapsed time: 24 mins 29 secs
Priority: 410
SD Files Written: 1,567,729
SD Bytes Written: 7,075,031,080 (7.075 GB)
Rate: 4816.2 KB/s
Volume name(s): 000003L4
Volume Session Id: 1168
Volume Session Time: 1454189071
Last Volume Bytes: 8,648,930,304 (8.648 GB)
SD Errors: 0
SD termination status: OK
Termination: Copying OK

14-Feb 16:55 bacula-dir JobId 231106: Begin pruning Jobs older than 3 years .
14-Feb 16:55 bacula-dir JobId 231106: No Jobs found to prune.
14-Feb 16:55 bacula-dir JobId 231106: Begin pruning Files.
14-Feb 16:55 bacula-dir JobId 231106: No Files found to prune.
14-Feb 16:55 bacula-dir JobId 231106: End auto prune.

Dan Langille - BSDCan / PGCon

dan AT langille DOT org

signature.asc
Description: Message signed with OpenPGP using GPGMail

------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140

_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users