Bacula-users

Re: [Bacula-users] [SPAM] Re: [SPAM] Bacula "TimeMachine" type SOHOconfig?

2009-12-06 15:07:44
Subject: Re: [Bacula-users] [SPAM] Re: [SPAM] Bacula "TimeMachine" type SOHOconfig?
From: "Timo Neuvonen" <timo-news AT tee-en DOT net>
To: bacula-users AT lists.sourceforge DOT net
Date: Sun, 6 Dec 2009 22:02:39 +0200
"Simon J Mudd" <sjmudd AT pobox DOT com> kirjoitti viestissä 
news:m3aaxw15sz.fsf AT mad06.wl0 DOT org...
> timo-news AT tee-en DOT net ("Timo Neuvonen") writes:
>
>> "Simon J Mudd" <sjmudd AT pobox DOT com> kirjoitti viestissä
>> news:20091206082523.GA612 AT mad06.wl0 DOT org...
>> > Hello,
>> >
>> > I've been using Bacula for some time for home use, and trying to get
>> > a working "TimeMachine" type configuration working.
>> >
>> > That is I'd like to configure bacula to store to an external hard disk
>> > using a number of fixed sizes files, occupying up to a certain amount
>> > of disk space. In my case 100 x 2GB files. I'd like to auto label
>> > new files and purge old ones automatically to make space if needed.
>> >
>> > This sounds like a simple recipe which is appropriate for a large 
>> > number
>> > of SOHO type situations. I know Bacula can do more, but to minimise
>> > intervention this looks nice.
>> >
>> > However, I don't quite get this to work. I've had issues with getting
>> > the auto-label to always work, and also the auto-expire. I wonder
>> > if anyone can look at my configuration or offer an alternative to do
>> > this?
>> >
>>
>> I guess "Time Machine" is something Apple-like, and I don't know nothing
>> about it but the name. But if what you told is the essential, that is the
>> disk usage strategy, it shouldn't be a problem though minor differences
>> might exist.
>
> Yes, sorry. It's basically a very simple application which comes with the 
> Mac OS X.
> You plug in an external disk and it will backup as much as possible 
> filling up the
> disk.  The implementation is quite different but for most people that's 
> all
> they want: to "designate a location/size for backups", and then to keep as 
> many
> backups as possible in that location. When the disk space fills up the 
> oldest
> "backups" are automatically thrown away.
>
>> I didn't notice "Volume Retention" specified in your Pool config. I 
>> controls
>> how soon after the last write the volumes can be recycled. Since you make
>> full backup once a month, this should be set to the minimum of more than 
>> one
>> month (eg. 40 days) to make sure you'll always have at least one (I'd
>> seriously recommend more, at least two) full backup(s) available. After 
>> this
>> period, if necessary, Bacula _can_ recycle the existing volumes. However,
>> actual recycle won't happen until really needed to free the previously 
>> used
>> volumes, but it can't happen before this time limit has expired.
>
> Yes, but that's what I'm trying to avoid. I realise that I MUST have 
> sufficient
> space really for at least 2 full backups plus some extra for incrementals
> but I don't want to worry about the details. Therefore I want to configure


You said you don't want to worry about the details. However, one such very 
strong detail is the schedule you already have specified, it says to run a 
full backup once a month. Required retention time is closely related to 
this, and needs to be specified too.

Since now you haven't specified the volume retention, Bacula uses its 
internal default which is one year, 365 days. You have to specify a shorter 
volume retention time if you want to be able to recycle the volumes sooner. 
I don't see why specifying this would be a thing to avoid. At least it would 
be a step getting closer to functionality what you want, unless I have 
seriously misunderstood something. Now it expects that you specifically want 
to forbid recycling volumes under one year of age, and therefore keeps 
asking fore more volumes. Just try setting it to 40 days, for example, 
update (using console update command) the existing volumes to obey this 
value, and give it a try. I guess then you would be very close to what you 
want: the oldest volumes will get recycled one by one when new storage space 
is needed, as long that volume is at least 40 days (or whatever you specify) 
old. If you can not use that long time (40 days), you could change your 
schedule to run full backup eg. weekly, and set volume retention time to 2 
weeks, for example. This way you might need less space for the non-full 
backup storage.

When playing with the volume retention, be careful not to specify it too 
close to the full-backup-cycle period. Just as an example, running a full 
weekly, and specifying volume retention to 8 days might sound like a working 
solution. But if something goes wrong with the full backup, there has to be 
enough time to solve the problem before the previous full 
exipires -otherwise it could get recycled before you have re-run the failed 
one, and in this case a system crash with no full backup at all might be a 
nightmare. Currently, there is no way to specify the retention time in terms 
of succesfull full backups, but I recall having seen this kind of feature 
request.

Btw, you can use "list media" command to see the status of the existing 
volumes. There you can see the current volume retention times, and verify 
they'll get changed (after update command) according to the conf change. The 
times shown in the list are always in seconds, independently from the units 
you'll use to specify them (eg. days).


> the "pool" to auto purge if it fills up. New full or incremental backups
> will create new volumes as needed, and the older ones will get purged.
>

Actually, Bacula will recycle the existing volumes, that is, discard the old 
data in the volume, and use the same "recycled" volume again. So the volume 
name won't change (unless this is possible due to some very new Bacula 
feature). Though Bacula can (if asked to do so) create new volumes as many 
as there is disk space available for, it will never free up disk space for 
new volumes by deleting the old volumes, but it can recycle the old ones. 
This should cause no problem to you, but it may be necessary to know this to 
understand what is happening.


> Doing this means I don't get to the point where bacula stops and asks me
> to provide new volumes, or runs out of space as older ones have not been
> purged.
>

Within reasonable limits (reasonable amount of disk space available), this 
should be possible with Bacula.


>> Also remember that modifying the pool parameters in the conf does not
>> automatically apply to the existing volumes, only to the new ones created
>> thereafter. To make the existing volumes to obey the new values, you'll 
>> need
>> to use the update pool / update volumes from pool commands from the 
>> bacula
>> console.
>
> Perhaps that is what I've been missing. I've adjusted parameters and 
> haven't
> noticed if things were working as expected.
>
> As I say that's why I'm looking for a simple "recipe" to do this which
> will help me but probably will be EASY to configure for a large group of
> SOHO type users who don't want to worry about the more complex issues
> of having separate pools for FULL or incremental backups etc.
>

Basically, one pool is enough. More pools could be used to fine-tune 
something, but you'll be fine with just one pool. No problem with that.

> Simon
>

--
Timo 



------------------------------------------------------------------------------
Join us December 9, 2009 for the Red Hat Virtual Experience,
a free event focused on virtualization and cloud computing. 
Attend in-depth sessions from your desk. Your couch. Anywhere.
http://p.sf.net/sfu/redhat-sfdev2dev
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users