Re: [Bacula-users] Bacula 5.0.2 restore files from tape very slow

>>>>> On Thu, 22 Jul 2010 14:37:22 -0600, May, John said:
> 
> The point I was making with my original post was that the file restore of
> the 6.2TB of data is taking many times longer than it did to back it up.
> For instance, I started the backup last Friday 7/22 and it still isn't
> finished today, only about half-finished.  If the original backup time was
> 37 hours, shouldn't the restore time be about the same or around the same
> time, not 10 times longer?

Yes, if the hardware is working optimally.  Restore doesn't use spooling
though, so it can trigger shoe-shining more easily that backup.

__Martin


> -John
> 
> -----Original Message-----
> From: Martin Simmons [mailto:martin AT lispworks DOT com] 
> Sent: Thursday, July 22, 2010 1:51 PM
> To: bacula-users AT lists.sourceforge DOT net
> Subject: Re: [Bacula-users] Bacula 5.0.2 restore files from tape very slow
> 
>>>>> On Thu, 22 Jul 2010 08:51:58 -0600, May, John said:
> > 
> > Bacula says the durantion for the job was 36:49:03 to backup 6.2TB.
> > 
> > So, the time it takes to backup the job affects the time it takes to 
> > restore?
> 
> No, but both times are affected by the size of the job.  But isn't that
> obvious?  Maybe I misunderstood your question.
> 
> __Martin
> 
>  
> > -John
> > 
> > -----Original Message-----
> > From: Martin Simmons [mailto:martin AT lispworks DOT com] 
> > Sent: Thursday, July 22, 2010 5:50 AM
> > To: bacula-users AT lists.sourceforge DOT net
> > Subject: Re: [Bacula-users] Bacula 5.0.2 restore files from tape very slow
> > 
>>>>> On Wed, 21 Jul 2010 13:59:34 -0600, May, John said:
> > > 
> > > As far as I know it was running at about 70MB/s.  It could have been
> > > shoe-shining a bit though, not really sure.
> > 
> > 600GB at 70MB/s would only take 2.5 hours so it can't have been getting that
> > at a sustained rate.  Shoe-shining sounds like a real possibility.
> > 
> > 
> > > -John
> > > 
> > > -----Original Message-----
> > > From: Martin Simmons [mailto:martin AT lispworks DOT com] 
> > > Sent: Wednesday, July 21, 2010 1:26 PM
> > > To: bacula-users AT lists.sourceforge DOT net
> > > Subject: Re: [Bacula-users] Bacula 5.0.2 restore files from tape very slow
> > > 
>>>>> On Mon, 19 Jul 2010 07:50:41 -0600, May, John said:
> > > > 
> > > > Monday morning, 3 days later and it is still restoring.  It's finished 
> > > > about
> > > > 600GB, which is less than half of what is on the two tapes.  I'm pretty 
> > > > sure
> > > > that only one job was written to the tape at the time of backup.  Also, 
> > > > I
> > > > thought 'bextract' would be faster than a normal restore, since I'm 
> > > > telling
> > > > it restore everything on tape, and not just individual jobs.  Is there
> > > > anyway I can speed this up?  Next up is an 8TB restore, and I dread to 
> > > > think
> > > > how long that will take.
> > > 
> > > Do you know how the tape drive was functioning during the restore?  Was it
> > > continually repositioning the tape (shoe-shining)?  That would happen if
> > > Bacula (or the target filesystem) was unable to keep up with the speed at
> > > which the drive was sending data.
> > > 
> > > __Martin
> > > 
> > > 
> > > > 
> > > > John
> > > > 
> > > > -----Original Message-----
> > > > From: May, John [mailto:john.may AT fugrohorizons DOT com] 
> > > > Sent: Friday, July 16, 2010 7:52 AM
> > > > To: bacula-users AT lists.sourceforge DOT net
> > > > Subject: Re: [Bacula-users] Bacula 5.0.2 restore files from tape very 
> > > > slow
> > > > 
> > > > >Hi John... Is it possible that the backup job you are trying to 
> > > > >restore from
> > > > >was running at the same time as other backup jobs were running (eg: 
> > > > >concurrent
> > > > >jobs enabled)?
> > > > >
> > > > >If yes, then what might be going on here is that several jobs from 
> > > > >several
> > > > >clients were written to the tape at the same time, and the data from 
> > > > >each job
> > > > >was "interleaved",  so the write to tape during the backup was fast, 
> > > > >but now
> > > > >the trade-off is that restores will be slow since the tape drive needs 
> > > > >to read
> > > > >parts of the files it wants to restore, skip the files from other
> > > > >servers/jobs, lather, rinse, repeat.
> > > > >
> > > > >It could also explain the initial fast restore of some data as it is 
> > > > >possible
> > > > >that this particular backup job you are trying to restore was "already 
> > > > >in
> > > > >progress" when the other started so it had full access to the storage 
> > > > >and it
> > > > >was the only one writing to the tape.
> > > > >
> > > > >Something to consider.
> > > > 
> > > > Well, this was an archive job and I ran it during the day when no other 
> > > > backups were running.  Also, I have Bacula configured to only write a 
> > > > single job to each tape for the archive pool (Maximum Volume Jobs = 1).
> > > > 
> > > > John
> > > > 
> > > > -----Original Message-----
> > > > From: Bill Arlofski [mailto:waa-bacula AT revpol DOT com] 
> > > > Sent: Thursday, July 15, 2010 7:06 PM
> > > > To: bacula-users AT lists.sourceforge DOT net
> > > > Subject: Re: [Bacula-users] Bacula 5.0.2 restore files from tape very 
> > > > slow
> > > > 
> > > > On 07/15/10 14:07, May, John wrote:
> > > > > I am using Bacula 5.0.2 using MySQL on Centos 5.4 x64.   I'm trying 
> > > > > to restore some files from a backup I made a couple months ago.  I 
> > > > > the restore is about 1.5 TB in size and spans two LTO4 tapes.  I can 
> > > > > start the restore just fine and the first 5GB flies by in a minute or 
> > > > > so, then the restore sort of stalls and starts crawling.  It slows 
> > > > > down to about 1MB/s down from about 70MB/s.  There are only about 
> > > > > 15,000 files on the tapes and I am restoring to a local Raid 0 array.
> > > > > 
> > > > > I tried to bextract the files with and without a bootstrap file and 
> > > > > the speed is still very slow.  I also verified I have the correct the 
> > > > > indexes in Mysql and I compacted the database.
> > > > > 
> > > > > At this rate, I don't think the restore will finish, because it is 
> > > > > going so slow.
> > > > > 
> > > > > Any ideas on what's going on?
> > > > > 
> > > > > -- John
> > > > 
> > > > 
> > > > Hi John... Is it possible that the backup job you are trying to restore 
> > > > from
> > > > was running at the same time as other backup jobs were running (eg: 
> > > > concurrent
> > > > jobs enabled)?
> > > > 
> > > > If yes, then what might be going on here is that several jobs from 
> > > > several
> > > > clients were written to the tape at the same time, and the data from 
> > > > each job
> > > > was "interleaved",  so the write to tape during the backup was fast, 
> > > > but now
> > > > the trade-off is that restores will be slow since the tape drive needs 
> > > > to read
> > > > parts of the files it wants to restore, skip the files from other
> > > > servers/jobs, lather, rinse, repeat.
> > > > 
> > > > It could also explain the initial fast restore of some data as it is 
> > > > possible
> > > > that this particular backup job you are trying to restore was "already 
> > > > in
> > > > progress" when the other started so it had full access to the storage 
> > > > and it
> > > > was the only one writing to the tape.
> > > > 
> > > > Something to consider.
> > > > 
> > > > --
> > > > Bill Arlofski
> > > > Reverse Polarity, LLC
> > > > 
> > > > ------------------------------------------------------------------------------
> > > > This SF.net email is sponsored by Sprint
> > > > What will you do first with EVO, the first 4G phone?
> > > > Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
> > > > _______________________________________________
> > > > Bacula-users mailing list
> > > > Bacula-users AT lists.sourceforge DOT net
> > > > https://lists.sourceforge.net/lists/listinfo/bacula-users
> > > > 
> > > > ------------------------------------------------------------------------------
> > > > This SF.net email is sponsored by Sprint
> > > > What will you do first with EVO, the first 4G phone?
> > > > Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
> > > > _______________________________________________
> > > > Bacula-users mailing list
> > > > Bacula-users AT lists.sourceforge DOT net
> > > > https://lists.sourceforge.net/lists/listinfo/bacula-users
> > > > 
> > > > ------------------------------------------------------------------------------
> > > > This SF.net email is sponsored by Sprint
> > > > What will you do first with EVO, the first 4G phone?
> > > > Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
> > > > _______________________________________________
> > > > Bacula-users mailing list
> > > > Bacula-users AT lists.sourceforge DOT net
> > > > https://lists.sourceforge.net/lists/listinfo/bacula-users
> > > > 
> > > 
> > > ------------------------------------------------------------------------------
> > > This SF.net email is sponsored by Sprint
> > > What will you do first with EVO, the first 4G phone?
> > > Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
> > > _______________________________________________
> > > Bacula-users mailing list
> > > Bacula-users AT lists.sourceforge DOT net
> > > https://lists.sourceforge.net/lists/listinfo/bacula-users
> > > 
> > > ------------------------------------------------------------------------------
> > > This SF.net email is sponsored by Sprint
> > > What will you do first with EVO, the first 4G phone?
> > > Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
> > > _______________________________________________
> > > Bacula-users mailing list
> > > Bacula-users AT lists.sourceforge DOT net
> > > https://lists.sourceforge.net/lists/listinfo/bacula-users
> > > 
> > 
> > ------------------------------------------------------------------------------
> > This SF.net email is sponsored by Sprint
> > What will you do first with EVO, the first 4G phone?
> > Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
> > _______________________________________________
> > Bacula-users mailing list
> > Bacula-users AT lists.sourceforge DOT net
> > https://lists.sourceforge.net/lists/listinfo/bacula-users
> > 
> > ------------------------------------------------------------------------------
> > This SF.net email is sponsored by Sprint
> > What will you do first with EVO, the first 4G phone?
> > Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
> > _______________________________________________
> > Bacula-users mailing list
> > Bacula-users AT lists.sourceforge DOT net
> > https://lists.sourceforge.net/lists/listinfo/bacula-users
> > 
> 
> ------------------------------------------------------------------------------
> This SF.net email is sponsored by Sprint
> What will you do first with EVO, the first 4G phone?
> Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
> _______________________________________________
> Bacula-users mailing list
> Bacula-users AT lists.sourceforge DOT net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
> 
> ------------------------------------------------------------------------------
> This SF.net email is sponsored by Sprint
> What will you do first with EVO, the first 4G phone?
> Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
> _______________________________________________
> Bacula-users mailing list
> Bacula-users AT lists.sourceforge DOT net
> https://lists.sourceforge.net/lists/listinfo/bacula-users
> 

------------------------------------------------------------------------------
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users