Bacula-users

Re: [Bacula-users] General questions after 1 year use

2010-11-12 18:10:02
Subject: Re: [Bacula-users] General questions after 1 year use
From: Ryan Novosielski <novosirj AT umdnj DOT edu>
To: bacula-users AT lists.sourceforge DOT net
Date: Fri, 12 Nov 2010 18:07:09 -0500
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

You clearly missed Timo's response where he answered question #1. Take a
look at the list.

On 11/12/2010 05:24 PM, Dermot Beirne wrote:
> Hello,
> I got no response to my last mail below, so I'll try and summarise my
> questions a little:
> (Bacula 3.0.2 on Ubuntu 9.04)
> 
> 1. If I copy a backup job, is the retention period of the original
> backup job or the copy backup job used by bacula.  I ask because my
> originals (disk) have a 2 day retention, but their copies (tape) have
> up to 12 months, but are still not kept in the catalogue and I need to
> use bscan to restore them.  What do I need to check/change?
> 
> 2. Is there a way Bacula can recycle tape volumes back into Scratch
> before the pool is empty.  Without purging volumes, the Scratch pool
> runs out.
> 
> 3. I currently have Daily, Weekly and Monthly disk volumes, which are
> copied to tape within a few hours of backup, and have a retention of
> just 2 days to keep space from running out on disk.  The Daily disk
> backup now needs more space, and can't recycle the weekly or monthly
> disk volumes as they are in different pools in different directories.
> Hence over 2 TB of disk is inaccessible to the Daily jobs, even though
> those volumes have expired.  Is it possible to have all 3 backup types
> share and recycle the same disk volumes, in one directory with the
> same naming scheme, but for the copy job to still know which is a
> daily, weekly, monthly disk volume so it can apply the required
> retention to the tape volume?  This would allow much better use of
> disk space.
> 
> 4. The writing from disk to tape appears very slow.  Is there a
> document/thread somewhere that will help me troubleshoot or benchmark
> the autoloader throughput (Dell P124T)
> 
> Thank you for any info.
> Dermot.
> 
> On 9 November 2010 10:07, Dermot Beirne <dermot.beirne AT dpd DOT ie> wrote:
>> Thanks for replying.
>>
>> On my first point below, where you suggest the volume retention is too
>> low, the volume retention for the monthly tape backups to 12 months,
>> yet if I attempt to restore a file from a jobid of, say, last March
>> monthly backup, it is no longer in the catalogue, and I need to bscan
>> that monthly tape volume to repopulate the catalogue, so I can restore
>> a file.   Is the fact that these are "copy" jobs overriding the
>> retention on the tape pools with the shorter disk pools retentions, or
>> can you expand on your suggestion that the retention is the problem.
>>
>> On the second point, I understand that purging is the last resort, but
>> could not find an alternative.  We have a tape autoloader with 8
>> slots.  Every day we remove the tapes used that day and replace with
>> Scratch tapes.  The problem is that Bacula never moves tapes back into
>> Scratch until it has none left.  If it is writing the backups to tape,
>> and uses the last Scratch tape, it will then recycle an old tape
>> volume (presumably) and ask for manual intervention for someone to
>> insert this tape, hence the backups stop.  Is there a way it can
>> recycle the volumes back into Scratch before the pool is empty, so the
>> admin has enough to refill the autoloader each day?
>>
>> On the third point, I do use limits on the disk volumes, which is
>> forcing the recycling as you say, and this works well to stop bacula
>> causing the backup server to run out of space before it tries to
>> recycle, however, how do I do this for Tape volumes?  If I set such a
>> limit on the tape pools, its hard to predict which ones it will
>> recycle, and hence it will stop and wait for manual intervention to
>> load the tape it has just decided to recycle.  I think it needs to do
>> recycling earlier in the process, so that the Scratch pool is self
>> replenishing.
>>
>> Appreciate your advice.
>> Dermot.
>>
>>
>> On 6 November 2010 23:30, Dan Langille <dan AT langille DOT org> wrote:
>>> On 11/5/2010 12:00 PM, Dermot Beirne wrote:
>>>>
>>>> Hello,
>>>> I have been using bacula now for 1 year this month, and it's been a
>>>> fairly smooth ride, but I have a number of questions that i'd be
>>>> grateful for some direction on:
>>>>
>>>> Bacula version 3.0.2 on Ubuntu 9.04
>>>>
>>>> 1. I am concerned about my retention policy.  I have 4 disk pools and
>>>> 4 corresponding tape pools.
>>>>
>>>> All the disk pools (Daily, Weekly, Monthly, Yearly) have a volume
>>>> retention of 2 days and Volume Use Duration also 2 days
>>>>
>>>> The Tape Pools are as follows:
>>>> Daily - 12 days
>>>> Weekly - 32 days
>>>> Monthly - 12 months
>>>> Yearly - never recycle
>>>>
>>>> I am finding that when needing to restore from an old job, I have to
>>>> run bscan on the tape as Bacula says it has no records for that job in
>>>> it's catalog.
>>>> This is even for monthly backups.  Why can this be?
>>>
>>> Your Job Retention is too low.
>>>
>>>> 2. Use of scratch pools.
>>>> Every few weeks we have to print a list of the volumes (from the daily
>>>> and sometimes weekly tape pools), and manually purge (purge jobs
>>>> volume) anything older than the above retention times in order to
>>>> force them back into the Scratch pool, as the Scratch pool runs out of
>>>> volumes because Bacula is not automatically purging once the retention
>>>> policy expires.
>>>> Is there a fix for this?  Could this be causing/affecting point 1
>>>> above (i.e. doing manual purges)?
>>>
>>> Purging is the last resort. If the scratch pool contains Volumes, they will
>>> be used.
>>>
>>>> 3. Disk pool structure.
>>>> I have 4 directories on my backup server for the Disk pools, Daily,
>>>> Weekly, Monthly and Yearly.  The Daily, Weekly and Monthly directories
>>>> are all in the region of 1.4Tb in size.  The problem is that the
>>>> server is near it's capacity of disk space, and I can't add any more
>>>> clients until I address this.  I feel the current structure is not
>>>> making the best use of the space.  The retention policy of 2 days on
>>>> the disk volumes is to allow a tape copy job to run the morning after
>>>> a nightly backup, and these tapes are then moved to a safe and
>>>> replaced with Scratch tapes in the autoloader.  It also allows us to
>>>> do a quick restore of a file that may have been deleted the previous
>>>> day, which is the most common restore request.
>>>
>>> It sound like you need to set a limit on the Pool so Bacula starts to
>>> recycle Volumes.  Bacula will not recycle Volumes if it is permitted to
>>> create new Volumes.
>>>
>>>>
>>>> The space taken up by the weekly and monthly volumes is wasted, as
>>>> they have expired 2 days after use, but the space is still not
>>>> available to the Daily volumes.  I suspect I'd get a lot more clients
>>>> covered if I could change this.
>>>>
>>>> Is it possible to have a single directory shared by all pools, and all
>>>> pools share and recycle the same volumes.  What are the pros and cons
>>>> of doing this, or is there a better way.
>>>>
>>>> Is there any benefit in upgrading bacula to address any of these items.
>>>>
>>>> I'm sorry for the long mail, and appreciate anyone taking the time to read
>>>> it.
>>>> Advice welcome on how to proceed with Bacula from here.
>>>
>>>
>>> --
>>> Dan Langille - http://langille.org/
>>>
>>>
>>
> 
> ------------------------------------------------------------------------------
> Centralized Desktop Delivery: Dell and VMware Reference Architecture
> Simplifying enterprise desktop deployment and management using
> Dell EqualLogic storage and VMware View: A highly scalable, end-to-end
> client virtualization framework. Read more!
> http://p.sf.net/sfu/dell-eql-dev2dev
> _______________________________________________
> Bacula-users mailing list
> Bacula-users AT lists.sourceforge DOT net
> https://lists.sourceforge.net/lists/listinfo/bacula-users


- -- 
- ---- _  _ _  _ ___  _  _  _
|Y#| |  | |\/| |  \ |\ |  | |Ryan Novosielski - Sr. Systems Programmer
|$&| |__| |  | |__/ | \| _| |novosirj AT umdnj DOT edu - 973/972.0922 (2-0922)
\__/ Univ. of Med. and Dent.|IST/CST-Academic Svcs. - ADMC 450, Newark
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkzdyJwACgkQmb+gadEcsb5xSwCglAqqNPLbpabHDBqsaZhW3mKe
hPgAn3BcyuaZA1BfPwMuQyuXJ8em4Wie
=qE56
-----END PGP SIGNATURE-----

Attachment: novosirj.vcf
Description: Vcard

------------------------------------------------------------------------------
Centralized Desktop Delivery: Dell and VMware Reference Architecture
Simplifying enterprise desktop deployment and management using
Dell EqualLogic storage and VMware View: A highly scalable, end-to-end
client virtualization framework. Read more!
http://p.sf.net/sfu/dell-eql-dev2dev
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users