Bacula-users
[Top] [All Lists]
<prev [Date] next>
[Advanced]
 <prev [Thread] next>

Re: [Bacula-users] tuning lto-4

2011-12-15 13:11:36
Subject: Re: [Bacula-users] tuning lto-4
From: gary artim <gartim AT gmail DOT com>
To: bacula-users AT lists.sourceforge DOT net
Date: Thu, 15 Dec 2011 10:09:15 -0800 
using this bacula-sd.conf, the best I get is about 2.4GB a minute. I'm
not working with network backups, this is just a straight raid 5 to
lto-4. I'm now thinking that my db (mysql) or raid is the
drag/slowdown since I can get over 180MBs with btape. Any suggestions
welcomes, I feel I've exhausted trying every blocksize, file size,
much appreciated:

Device {
 Name = LTO-4
 Media Type = LTO-4
 Archive Device = /dev/nst0
 AutomaticMount = yes;               # when device opened, read it
 AlwaysOpen = yes;
 RemovableMedia = yes;
 RandomAccess = no;
 Maximum Block Size = 2M   # ( about 2.4GB/minute/with 12GB Max File Size  )
 Maximum File Size = 12GB
 Autochanger = yes
 Alert Command = "sh -c 'tapeinfo -f %c |grep TapeAlert|cat'"
 Alert Command = "sh -c 'smartctl -H -l error %c'"
}


On Fri, Dec 2, 2011 at 2:12 PM, gary artim <gartim AT gmail DOT com> wrote:
> 180 MBs, 256MB min/max blocksize.
>
> [root@genepi1 bacula]# tapeinfo -f /dev/nst0
> Product Type: Tape Drive
> Vendor ID: 'HP      '
> Product ID: 'Ultrium 4-SCSI  '
> Revision: 'B12H'
> Attached Changer API: No
> SerialNumber: 'HU17450M8L'
> MinBlock: 1
> MaxBlock: 16777215
> SCSI ID: 1
> SCSI LUN: 0
> Ready: yes
> BufferedMode: yes
> Medium Type: Not Loaded
> Density Code: 0x46
> BlockSize: 0
> DataCompEnabled: yes
> DataCompCapable: yes
> DataDeCompEnabled: yes
> CompType: 0x1
> DeCompType: 0x1
> Block Position: 471909
> Partition 0 Remaining Kbytes: 799204
> Partition 0 Size in Kbytes: 799204
> ActivePartition: 0
> EarlyWarningSize: 0
> NumPartitions: 0
> MaxPartitions: 0
> [root@genepi1 bacula]#  btape -c /etc/bacula/bacula-sd.conf /dev/nst0
> Tape block granularity is 1024 bytes.
> btape: butil.c:284 Using device: "/dev/nst0" for writing.
> 02-Dec 13:07 btape JobId 0: 3301 Issuing autochanger "loaded? drive 0" 
> command.
> 02-Dec 13:07 btape JobId 0: 3302 Autochanger "loaded? drive 0", result
> is Slot 12.
> btape: btape.c:476 open device "LTO-4" (/dev/nst0): OK
> *speed file_size=20 nb_file=10 skip_raw
> btape: btape.c:1082 Test with zero data and bacula block structure.
> btape: btape.c:960 Begin writing 10 files of 21.47 GB with blocks of
> 524288 bytes.
> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> btape: btape.c:608 Wrote 1 EOF to "LTO-4" (/dev/nst0)
> btape: btape.c:410 Volume bytes=21.47 GB. Write rate = 177.4 MB/s
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> btape: btape.c:608 Wrote 1 EOF to "LTO-4" (/dev/nst0)
> btape: btape.c:410 Volume bytes=21.47 GB. Write rate = 180.4 MB/s
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> btape: btape.c:608 Wrote 1 EOF to "LTO-4" (/dev/nst0)
> btape: btape.c:410 Volume bytes=21.47 GB. Write rate = 178.9 MB/s
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> btape: btape.c:608 Wrote 1 EOF to "LTO-4" (/dev/nst0)
> btape: btape.c:410 Volume bytes=21.47 GB. Write rate = 180.4 MB/s
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> btape: btape.c:608 Wrote 1 EOF to "LTO-4" (/dev/nst0)
> btape: btape.c:410 Volume bytes=21.47 GB. Write rate = 178.9 MB/s
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> btape: btape.c:608 Wrote 1 EOF to "LTO-4" (/dev/nst0)
> btape: btape.c:410 Volume bytes=21.47 GB. Write rate = 180.4 MB/s
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> btape: btape.c:608 Wrote 1 EOF to "LTO-4" (/dev/nst0)
> btape: btape.c:410 Volume bytes=21.47 GB. Write rate = 180.4 MB/s
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> btape: btape.c:608 Wrote 1 EOF to "LTO-4" (/dev/nst0)
> btape: btape.c:410 Volume bytes=21.47 GB. Write rate = 178.9 MB/s
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> btape: btape.c:608 Wrote 1 EOF to "LTO-4" (/dev/nst0)
> btape: btape.c:410 Volume bytes=21.47 GB. Write rate = 180.4 MB/s
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> btape: btape.c:608 Wrote 1 EOF to "LTO-4" (/dev/nst0)
> btape: btape.c:410 Volume bytes=21.47 GB. Write rate = 178.9 MB/s
> btape: btape.c:384 Total Volume bytes=214.7 GB. Total Write rate = 179.5 MB/s
>
> btape: btape.c:1094 Test with random data, should give the minimum throughput.
> btape: btape.c:960 Begin writing 10 files of 21.47 GB with blocks of
> 524288 bytes.
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> btape: btape.c:608 Wrote 1 EOF to "LTO-4" (/dev/nst0)
> btape: btape.c:410 Volume bytes=21.47 GB. Write rate = 26.47 MB/s
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> btape: btape.c:608 Wrote 1 EOF to "LTO-4" (/dev/nst0)
> btape: btape.c:410 Volume bytes=21.47 GB. Write rate = 26.51 MB/s
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
>
> On Wed, Nov 30, 2011 at 8:02 AM, gary artim <gartim AT gmail DOT com> wrote:
>> Hi --
>>
>> Getting about 41.6/MBs and hoping for closer to the max (120MB). I
>> tried maximum file sizes of 5, 8, 12GB -- 12GB the best the others
>> where about 35/MBs. Any advise welcomed...should I look at max/min
>> block sizes?
>> most of the data is big, genetics data -- filesizes avg in the 500/MB
>> to 3-4/GB -- looking at a growth from 4TB to 15TB in the next 2 years.
>>
>> run results and bacula-sd.conf and bacula-dir.conf below...
>>
>> thanks
>> -- gary
>>
>> Run:
>> ===
>>
>>  Build OS:               x86_64-redhat-linux-gnu redhat
>>  JobId:                  5
>>  Job:                    Prodbackup.2011-11-29_19.32.42_05
>>  Backup Level:           Full
>>  Client:                 "bacula-fd" 5.0.3 (04Aug10)
>> x86_64-redhat-linux-gnu,red
>>
>>
>> hat,
>>  FileSet:                "FileSetProd" 2011-11-29 19:32:42
>>  Pool:                   "FullProd" (From Job FullPool override)
>>  Catalog:                "MyCatalog" (From Client resource)
>>  Storage:                "LTO-4" (From Job resource)
>>  Scheduled time:         29-Nov-2011 19:32:26
>>  Start time:             29-Nov-2011 19:32:45
>>  End time:               29-Nov-2011 21:15:53
>>  Elapsed time:           1 hour 43 mins 8 secs
>>  Priority:               10
>>  FD Files Written:       35,588
>>  SD Files Written:       35,588
>>  FD Bytes Written:       257,543,090,368 (257.5 GB)
>>  SD Bytes Written:       257,548,502,159 (257.5 GB)
>>  Rate:                   41619.8 KB/s
>>  Software Compression:   None
>>  VSS:                    no
>>  Encryption:             no
>>  Accurate:               no
>>  Volume name(s):         f03
>>  Volume Session Id:      1
>>  Volume Session Time:    1322622337
>>  Last Volume Bytes:      257,740,342,272 (257.7 GB)
>>  Non-fatal FD errors:    0
>>  SD Errors:              0
>>  FD termination status:  OK
>>  SD termination status:  OK
>>  Termination:            Backup OK
>>
>> bacula-sd.conf:
>> ==========
>>
>> Autochanger {
>>  Name = Autochanger
>>  Device = LTO-4
>>  Changer Command = "/usr/libexec/bacula/mtx-changer %c %o %S %a %d"
>>  Changer Device = /dev/changer
>> }
>> Device {
>>  Name = LTO-4
>>  Media Type = LTO-4
>>  Archive Device = /dev/nst0
>>  AutomaticMount = yes;               # when device opened, read it
>>  AlwaysOpen = yes;
>>  RemovableMedia = yes;
>>  RandomAccess = no;
>>  Maximum File Size = 12GB
>>  Autochanger = yes
>>  Alert Command = "sh -c 'tapeinfo -f %c |grep TapeAlert|cat'"
>>  Alert Command = "sh -c 'smartctl -H -l error %c'"
>> }
>> bacula-dir.conf:
>> ==========
>>
>> Job {
>>        Name = "Prodbackup"
>>        Client = bacula-fd
>>        FileSet = "FileSetProd"
>>        Schedule = "ScheduleProd"
>>        Write Bootstrap = "/var/spool/bacula/%c.bsr"
>>        Full Backup Pool = FullProd
>>        Incremental Backup Pool = IncrProd
>>        Differential Backup Pool = DiffProd
>>        Storage =  LTO-4
>>        Type = Backup
>>        Level = Incremental
>>        Pool = IncrProd
>>        Priority = 10
>>        Messages = Standard
>> }
>> FileSet {
>>        Name = "FileSetProd"
>>        Include {
>>                Options {
>>                        WildFile = "*.OLD"
>>                        WildFile = "*.o"
>>                        WildFile = "*.bak"
>>                        exclude = yes
>>                }
>>                File = /my/home/xxxxxxx
>>        }
>>        Exclude {
>>                File = /my/home/tmp
>>        }
>> }
>> Schedule {
>>        Name = "ScheduleProd"
>>        Run = Full 1st sun at 16:05
>>        Run = Pool {
>>        Name = FullProd
>>        Label Format = "FullProd"
>>        Pool Type = Backup
>>        Recycle = yes           # Bacula can automatically recycle Volumes
>>        AutoPrune = yes         # Prune expired volumes
>>        Volume Retention = 10 years
>>        Maximum Volume Jobs = 1
>> }
>>
>> Pool {
>>        Name = DiffProd
>>        Label Format = "DiffProd"
>>        Pool Type = Backup
>>        Recycle = yes
>>        AutoPrune = yes
>>        Volume Retention =  40 days
>>        Maximum Volume Jobs = 1
>> }
>>
>> Pool {
>>        Name = IncrProd
>>        Label Format = "IncrProd"
>>        Pool Type = Backup
>>        Recycle = yes
>>        AutoPrune = yes
>>        Volume Retention =  10 days
>>        Maximum Volume Jobs = 1
>> }
>> Differential 2nd-5th sun at 16:05
>>        Run = Incremental mon-sat at 16:05
>> }

------------------------------------------------------------------------------
10 Tips for Better Server Consolidation
Server virtualization is being driven by many needs.  
But none more important than the need to reduce IT complexity 
while improving strategic productivity.  Learn More! 
http://www.accelacomm.com/jaw/sdnl/114/51507609/
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users
[More with this subject...]
<Prev in Thread] Current Thread [Next in Thread>
Previous by Date:  Re: [Bacula-users] Fedora 16 - RHEL 5/6 Bacula RPM repository, Joseph L. Casale
Next by Date:  Re: [Bacula-users] tuning lto-4, John Drescher
Previous by Thread:  Re: [Bacula-users] tuning lto-4, gary artim
Next by Thread:  Re: [Bacula-users] tuning lto-4, John Drescher
Indexes:  [Date] [Thread] [Top] [All Lists]