Bacula-users

Re: [Bacula-users] concurrent backup and restore

2008-09-12 07:15:11
Subject: Re: [Bacula-users] concurrent backup and restore
From: Christoph Litauer <litauer AT uni-koblenz DOT de>
To: Ronald Buder <rbuder AT proficom-ag DOT de>
Date: Fri, 12 Sep 2008 13:14:55 +0200
Ronald Buder schrieb:
> Christoph Litauer wrote:
>> Sebastian Lehmann schrieb:
>>   
>>> Hi,
>>>
>>> Am Mo 08.09.2008 15:54 schrieb Silver Salonen <silver AT ultrasoft DOT ee>:
>>>
>>>     
>>>> On Monday 08 September 2008 16:47:19 Christoph Litauer wrote:
>>>>       
>>>>> Hi,
>>>>>
>>>>> is it possible to backup and restore data in parallel when using
>>>>> File
>>>>> devices?
>>>>>
>>>>> My problems is, that some of my backups take big amounts of time (3
>>>>> days
>>>>> because of a slow network connection). Starting a restore in
>>>>> parallel
>>>>> leads to "RestoreFiles.2008-09-08_15.38.44 is waiting on Storage
>>>>> Files".
>>>>> My setup is to allow concurrent backups to File device which works
>>>>> like
>>>>> a charm during backup.
>>>>>
>>>>> Output of status storage is:
>>>>>
>>>>> --------------
>>>>> Running Jobs:
>>>>> Writing: Full Backup job sun2 JobId=20148 Volume="File-0571"
>>>>>      pool="Default" device="FileStorage" (/storage)
>>>>>      spooling=1 despooling=0 despool_wait=0
>>>>>      Files=73,342 Bytes=1,239,082,954 Bytes/sec=1,832,962
>>>>>      FDReadSeqNo=639,961 in_msg=451631 out_msg=5 fd=6
>>>>> Writing: Full Restore job RestoreFiles JobId=20150 Volume=""
>>>>>      pool="Default" device="FileStorage" (/storage)
>>>>>      spooling=0 despooling=0 despool_wait=0
>>>>>      Files=0 Bytes=0 Bytes/sec=0
>>>>>      FDSocket closed
>>>>> ====
>>>>>
>>>>> Jobs waiting to reserve a drive:
>>>>> 3602 JobId=20150 device "FileStorage" (/storage) is busy (already
>>>>> reading/writing).
>>>>> --------------
>>>>>
>>>>>
>>>>> It seems as the RestoreFiles job is waiting for _writing_ instead of
>>>>> _reading_?
>>>>>
>>>>> Did I miss anything? Thanks a lot in advance!
>>>>>         
>>>> To my mind the limitation was that devices cannot have concurrent
>>>> connections.
>>>> So yeah.. backup- and restore jobs have to share that 1 possible
>>>> connection.
>>>>
>>>>       
>>> We made it with a trick. We use a file auto changer, (virtual
>>> autochanger). We then have two devices and several virtual tapes. Now it
>>> is possible to backup and restore at the same time from a file-device.
>>> Only one restriction is, that you can not use the same virtual tape for
>>> backup and restore at the same time.
>>>     
>> Thanks a lot, this would help! Would you please send your 
>> bacula-sd.conf? Or parts of it?
>> Thanks in advance!
>>
>>   
> Sorry, kind of busy lately, Sebastian asked me to email the sd-config 
> for the file auto changer. Here goes:
> 
> First the virtual drives:
> 
> Device {
>   Name = FileStorage
>   Media Type = File
>   Archive Device = /stage0/drive0
>   Label Media = yes
>   Random Access = Yes
>   Automatic Mount = yes
>   Removable Media = no
>   Always Open = no
>   Autochanger = yes
>   Drive Index = 0
>   Device Type = File
>   Autoselect = yes
> }
> 
> Device {
>   Name = FileStorage1
>   Media Type = File
>   Archive Device = /stage0/drive1
>   Label Media = yes
>   Random Access = yes
>   Automatic Mount = yes
>   Removable Media = no
>   Always Open = no
>   Autochanger = yes
>   Drive Index = 1
>   Device Type = File
>   Autoselect = yes
> }
> 
> And then the virtual changer:
> 
> Autochanger {
>   Name = FileAutoChanger
>   Device = FileStorage
>   Device = FileStorage1
>   Changer Command = "/etc/bacula/scripts/disk-changer %c %o %S %a %d"
>   Changer Device = "/etc/bacula/scripts/fileautochanger.conf"
> }
> 
> 
> Here is a working disk pool configuration that goes with the above 
> mentioned autochanger:
> 
> Pool {
>   Name = Disk
>   Pool Type = Backup
>   Recycle = yes
>   Auto Prune = yes
>   Volume Retention = 2d
>   Volume Use Duration = 23h
> #  Label Format =
>   Storage = File
>   Maximum Volume Bytes = 51200000000    # max. 50GB for each "tape"
>   Recycle Pool = Disk
> 
> # Migration options
>  Migration High Bytes = 429496729600    # 400GB
> # Migration High Bytes = 322122547200   # 300GB
> # Migration High Bytes = 154145200537   # 150GB
> 
>  Migration Low Bytes  = 214748364800    # 200GB
> # Migration Low Bytes = 53687091200    # 50GB
> 
> # Migration Time = 388800   # 4 days
> #  Migration Time = 259200   # 3 days
> #  Migration Time = 216000   # 2.5 days
> #   Migration Time = 172800  # 2 days
> #  Migration Time = 129600  # 1.5 days
> #  Migration Time = 86400   # 1 days
> #  Migration Time = 43200    # 0.5 days
> 
>    Next Pool = MONTH-TAPE
> }
> 
> 
> Hope this helps. Let me (us) know if there is anything more you might need.

Thanks to you all, I've got my disk-changer working.

-- 
Regards
Christoph
________________________________________________________________________
Christoph Litauer                  litauer AT uni-koblenz DOT de
Uni Koblenz, Computing Center,     http://www.uni-koblenz.de/~litauer
Postfach 201602, 56016 Koblenz     Fon: +49 261 287-1311, Fax: -100 1311
PGP-Fingerprint: F39C E314 2650 650D 8092 9514 3A56 FBD8 79E3 27B2


-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users