Bacula-users

Re: [Bacula-users] concurrent backup and restore

2008-09-10 04:50:56
Subject: Re: [Bacula-users] concurrent backup and restore
From: Ronald Buder <rbuder AT proficom-ag DOT de>
To: Christoph Litauer <litauer AT uni-koblenz DOT de>
Date: Wed, 10 Sep 2008 10:50:26 +0200
Christoph Litauer wrote:
> Sebastian Lehmann schrieb:
>   
>> Hi,
>>
>> Am Mo 08.09.2008 15:54 schrieb Silver Salonen <silver AT ultrasoft DOT ee>:
>>
>>     
>>> On Monday 08 September 2008 16:47:19 Christoph Litauer wrote:
>>>       
>>>> Hi,
>>>>
>>>> is it possible to backup and restore data in parallel when using
>>>> File
>>>> devices?
>>>>
>>>> My problems is, that some of my backups take big amounts of time (3
>>>> days
>>>> because of a slow network connection). Starting a restore in
>>>> parallel
>>>> leads to "RestoreFiles.2008-09-08_15.38.44 is waiting on Storage
>>>> Files".
>>>> My setup is to allow concurrent backups to File device which works
>>>> like
>>>> a charm during backup.
>>>>
>>>> Output of status storage is:
>>>>
>>>> --------------
>>>> Running Jobs:
>>>> Writing: Full Backup job sun2 JobId=20148 Volume="File-0571"
>>>>      pool="Default" device="FileStorage" (/storage)
>>>>      spooling=1 despooling=0 despool_wait=0
>>>>      Files=73,342 Bytes=1,239,082,954 Bytes/sec=1,832,962
>>>>      FDReadSeqNo=639,961 in_msg=451631 out_msg=5 fd=6
>>>> Writing: Full Restore job RestoreFiles JobId=20150 Volume=""
>>>>      pool="Default" device="FileStorage" (/storage)
>>>>      spooling=0 despooling=0 despool_wait=0
>>>>      Files=0 Bytes=0 Bytes/sec=0
>>>>      FDSocket closed
>>>> ====
>>>>
>>>> Jobs waiting to reserve a drive:
>>>> 3602 JobId=20150 device "FileStorage" (/storage) is busy (already
>>>> reading/writing).
>>>> --------------
>>>>
>>>>
>>>> It seems as the RestoreFiles job is waiting for _writing_ instead of
>>>> _reading_?
>>>>
>>>> Did I miss anything? Thanks a lot in advance!
>>>>         
>>> To my mind the limitation was that devices cannot have concurrent
>>> connections.
>>> So yeah.. backup- and restore jobs have to share that 1 possible
>>> connection.
>>>
>>>       
>> We made it with a trick. We use a file auto changer, (virtual
>> autochanger). We then have two devices and several virtual tapes. Now it
>> is possible to backup and restore at the same time from a file-device.
>> Only one restriction is, that you can not use the same virtual tape for
>> backup and restore at the same time.
>>     
>
> Thanks a lot, this would help! Would you please send your 
> bacula-sd.conf? Or parts of it?
> Thanks in advance!
>
>   
Sorry, kind of busy lately, Sebastian asked me to email the sd-config 
for the file auto changer. Here goes:

First the virtual drives:

Device {
  Name = FileStorage
  Media Type = File
  Archive Device = /stage0/drive0
  Label Media = yes
  Random Access = Yes
  Automatic Mount = yes
  Removable Media = no
  Always Open = no
  Autochanger = yes
  Drive Index = 0
  Device Type = File
  Autoselect = yes
}

Device {
  Name = FileStorage1
  Media Type = File
  Archive Device = /stage0/drive1
  Label Media = yes
  Random Access = yes
  Automatic Mount = yes
  Removable Media = no
  Always Open = no
  Autochanger = yes
  Drive Index = 1
  Device Type = File
  Autoselect = yes
}

And then the virtual changer:

Autochanger {
  Name = FileAutoChanger
  Device = FileStorage
  Device = FileStorage1
  Changer Command = "/etc/bacula/scripts/disk-changer %c %o %S %a %d"
  Changer Device = "/etc/bacula/scripts/fileautochanger.conf"
}


Here is a working disk pool configuration that goes with the above 
mentioned autochanger:

Pool {
  Name = Disk
  Pool Type = Backup
  Recycle = yes
  Auto Prune = yes
  Volume Retention = 2d
  Volume Use Duration = 23h
#  Label Format =
  Storage = File
  Maximum Volume Bytes = 51200000000    # max. 50GB for each "tape"
  Recycle Pool = Disk

# Migration options
 Migration High Bytes = 429496729600    # 400GB
# Migration High Bytes = 322122547200   # 300GB
# Migration High Bytes = 154145200537   # 150GB

 Migration Low Bytes  = 214748364800    # 200GB
# Migration Low Bytes = 53687091200    # 50GB

# Migration Time = 388800   # 4 days
#  Migration Time = 259200   # 3 days
#  Migration Time = 216000   # 2.5 days
#   Migration Time = 172800  # 2 days
#  Migration Time = 129600  # 1.5 days
#  Migration Time = 86400   # 1 days
#  Migration Time = 43200    # 0.5 days

   Next Pool = MONTH-TAPE
}


Hope this helps. Let me (us) know if there is anything more you might need.

Best regards, Ronald

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users