Hi,
Am Do 08.01.2009 22:22 schrieb Ulrich Leodolter
<ulrich.leodolter AT obvsg DOT at>:
> Hello,
>
> On Thu, 2009-01-08 at 12:21 -0500, J-P wrote:
> > Hi everyone,
> >
> > I wrote an e-mail regarding this issue before Christmas and had some
> > good
> > replies - Thanks to all. However, we were still unable to solve the
> > issue.
> > We decided to upgrade from 2.0.3 to 2.4.2 (using Ubuntu packages
> > from
> > Hardy-Backports) to see if we could get it solved.
> >
> > Unfortunately, it didn't solve the slow migration to tape issue. We
> > had a
> > suggestion from Ulrich to move spooling to another RAID array, which
> > can't
> > be done at the moment. So I'm looking at other solutions (if
> > possible).
> >
> > Here is a recap of our situation:
> >
> > - Backups are taken from network clients to the Director/SD hard
> > disk
> > (RAID0, with MySQL on the same array) and then later migrated from
> > disk to
> > Tape (LTO3 with hardware compression)
> > - When migrating, it takes hours to write some MB to the tape (i.e.
> > 1h45mins
> > for 5MB, with a rate of 0.9KB/s)
> > - I tested the Tape drive with dd and have good throughput, around
> > 135MB/s.
> >
> >
> > Related to that, I have 2 questions:
> >
> >
> > Question 1:
> >
> > When I do a migration to tape, while the first job is migrated, I'm
> > executing "status storage=Tape" and I see something like the
> > following
> > output:
> >
> > --snip--
> > Device status:
> > Autochanger "Autochanger" with devices:
> > "IBM-ULTRIUM-TD3" (/dev/nst0)
> > Device "FileStorage" (/backup/bacula) is mounted with:
> > Volume: WeeklyFileVolume-0002
> > Pool: *unknown*
> > Media type: File
> > Total Bytes Read=159,528,829,516 Blocks Read=1,360,087
> > Bytes/block=64,512
> > Positioned at File=57 Block=2,323,636,495
> > Device "IBM-ULTRIUM-TD3" (/dev/nst0) is mounted with:
> > Volume: 000014L3
> > Pool: Weekly Tape Pool
> > Media type: LTO-3
> > Slot 4 is loaded in drive 0.
> > Total Bytes=87,741,932,544 Blocks=0 Bytes/block=64,512
> > Positioned at File=0 Block=1
> > --snip--
> >
> > >From "FileStorage", the "Total Bytes Read" always go over the
> > >"Total Bytes"
> > in the "IBM-ULTRIUM-TD3" section. Why is that? Isn't supposed to be
> > the same
> > total in both section? It seems like the following is happening:
> >
> > 1. Read and write to tape (I've done the math, and write speed is
> > around 3
> > GB/min there?!)
> > 2. When job writing is done, the FileStorage read continues (I think
> > this is
> > where we have a problem)
>
> Bacula reads till the end of your FileStorage Volumes because BSR
> matching doesn't work.
>
> Without spooling relative small jobs are split into many smaller
> blocks
> which are spread over your FileStorage volume (maybe uniformly
> distributed, depends on client speed, number of backup jobs and
> Maximum Concurrent Jobs). You can check this in the JobMedia sql
> table.
>
> During migration Bacula seeks to the first block and then reads to the
> end of the volume (because BSR matching doesnt work).
> Bacula does NOT SEEK between consecutive blocks, it READS ALL BLOCKS
> TILL THE END of the volume. This is done for each migration job.
> (This is not 100% true, but almost)
>
Ahhh! That's the reason! I found this behaviour, too, but didn't know
the reason. We daily migraten several TB from Disk to Tape and it takes
a very long time to do this.
> > 3. When reading is done, "Migration OK" e-mail report is sent,
> > showing poor
> > stats about duration and speed rate.
> >
> > Can someone explain that to me?
> >
> >
> > Question 2:
> >
> > Would spooling help in our situation? Or it is just a waste of Disk
> > I/O
> > since it is a migration from local hard disk to Tape Drive and not
> > backup
> > from a network client directly to Tape?
>
> Spooling would definitely help, this was my experience.
> But you have to limit the size of your FileStorage Volumes
> to relative small size (depends on your average incr/full job size,
> i used Maximum Volume Bytes = 4G).
We use 65GB files an it takes up to 10Minutes per Job - depending on the
position of the first block of the job.
> This problem is fixed in the latest beta 2.5.28-b1,
> but not in 2.4.x AFAIK.
Did you mean the not working BSR matching? This would be the next
important thing to do an upgrade to version 3.x, if migration is working
faster then in version 2.4.x
Greetings
Sebastian
Ulrich
>
>
> PS: check
>
> Device {
> Name = "IBM-ULTRIUM-TD3"
> ...
> AlwaysOpen = yes
> }
>
> >
> >
> > Thanks for you help, very appreciated!
> >
> >
> > J-P
> >
> >
> >
> >
> >
> > ------------------------------------------------------------------------------
> > Check out the new SourceForge.net Marketplace.
> > It is the best place to buy or sell services for
> > just about anything Open Source.
> > http://p.sf.net/sfu/Xq1LFB
> > _______________________________________________
> > Bacula-users mailing list
> > Bacula-users AT lists.sourceforge DOT net
> > https://lists.sourceforge.net/lists/listinfo/bacula-users
> >
>
>
>
> ------------------------------------------------------------------------------
> Check out the new SourceForge.net Marketplace.
> It is the best place to buy or sell services for
> just about anything Open Source.
> http://p.sf.net/sfu/Xq1LFB
> _______________________________________________
> Bacula-users mailing list
> Bacula-users AT lists.sourceforge DOT net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
>
>
------------------------------------------------------------------------------
This SF.net email is sponsored by:
SourcForge Community
SourceForge wants to tell your story.
http://p.sf.net/sfu/sf-spreadtheword
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users
|
