Bacula-users

Re: [Bacula-users] Getting back to the basics; Volumes, Pools, Reusability

2011-08-05 14:20:32
Subject: Re: [Bacula-users] Getting back to the basics; Volumes, Pools, Reusability
From: Jeff Cleverley <jeff.cleverley AT avagotech DOT com>
To: Dan Trainor <dan.trainor AT gmail DOT com>
Date: Fri, 5 Aug 2011 12:17:59 -0600
Dan,

I'll throw in some general explanations of my own.  Perhaps a
different concept of things would help.

Consider a garage to be a storage pool.  Inside the garage are 100 1
gallon buckets.  The buckets are your volumes.  You data is water.
Outside the garage you have a 5 gallon bucket of water (your data).
The job puts the 5 gallons of water into the 1 gallon buckets.  Your
files are 1 teaspoon each.  Since all the data gets mixed in the
buckets, you cannot empty (recycle) a bucket until every teaspoon in
them is ready to be dumped out.  The backup software will put your
water in any bucket that has room in that garage.  It generally will
fill each bucket in turn and not partially fill multiple buckets.

You would generally have a different garage for each retention period.
 You can move empty buckets (volumes) between garages if you need more
in one or the other.  You may also set up a new garage when bigger
buckets (larger tape capacity) become available and they require
different handling.

>
> One more thing that I'm not 100% on is how volume retention and
> recycling works.  I believe I understand that of all three retentions
> (File, Job, Volume), Volume always has the last word.  Is this
> correct?  Is it normal (allowed?) to add additional volumes to my

A volume (tape or disk) will keep the longest retention of anything on
that volume.  If you back up one file and tell it to keep it for a
year, then back up 1 million files to the same volume and say to keep
them for 1 week, the volume will not recycle until the 1 year is up.
>>From that stand point the file actually has the last word.

> configuration, or is it more common to add another Pool?  I think I'd
> be wanting to add volumes to each pool, as it's been pointed out that
> I should reserve Pool creation to their specific role (daily, weekly,
> monthly, archive).  I know that's a loaded question as well and it's
> highly site-specific, but I wanted to discuss it for a moment if you
> wouldn't mind.

Most people set up different pools due to the retention period of
data.  In the example above may people would have a Full pool they
keep for a year, and an Incremental pool they may keep for 30 days.
Otherwise your Incremental tapes would not recycle for up to a year if
any Full backup data was on them..  Because tape really cannot spot
overwrite data like disk can, you will use a lot more tapes if you let
everything write to one pool of tapes.  Most people will add/delete
volumes from the pools to adjust for usage.

Once you have a functional pool/volume/backup configuration, you would
only add or delete pools if you have new backups that have new needs
that cannot be accommodated by an existing pool.  You might set up a
pool to do one time archives of data to send off-site and keep them
for 10 years.  You don't want other data on these tapes so a new pool
and volumes would be best for these.
>
> Does only a Volume get recycled?  How about pools?  If pools as well,
> why?  I know that volumes (assumed to be tape) are recycled because
> they're always appended (if not always, then most-always since it's
> good practice to prevent overwriting existing data).

Pools do not recycle since they don't actually contain data.  They
only contain volumes that have the data.
>
> What can I do to ensure that a volume that, say, contains a years'
> worth of data doesn't get recycled too soon and potentially
> overwritten?  What's my stop-gap to ensure that doesn't happen?  I'd
> much rather be bothered with some administration tasks because Bacula
> complained about not having any other place to store backups, rather
> than having Bacula overwrite my long lost archives.  Are retention
> periods (on volumes) hard rules that will guarantee that this does not
> happen?

As long as your catalog/database are good, the retention rules are
hard.  If your catalog loses track of a tape it won't know anything
about it and could overwrite it.  I'm just learning Bacula also but
there are generally rules on when software is allowed to overwrite and
format new media.  If you use physical tapes, you could remove things
from the library or just flip the write-protect tab.
>
> Is there any direct correlation between a File and Volume retention
> period, and if so, between the two, which takes precedence (can I
> think of it that way?)?  Of the two, which should I be more concerned
> or interested with/in?

As mentioned above, the volume will not expire until all files on the
volume are expired.

>
> I hope I'm not wasting anyone's time by asking these kinds of
> questions.  Having them answered is certainly helping me out, and I
> really hope it helps a lot of other people get over their case of cold
> feet, as well.

I don't see it as a waste of time.

Jeff

-- 
Jeff Cleverley
Unix Systems Administrator
4380 Ziegler Road
Fort Collins, Colorado 80525
970-288-4611

------------------------------------------------------------------------------
BlackBerry&reg; DevCon Americas, Oct. 18-20, San Francisco, CA
The must-attend event for mobile developers. Connect with experts. 
Get tools for creating Super Apps. See the latest technologies.
Sessions, hands-on labs, demos & much more. Register early & save!
http://p.sf.net/sfu/rim-blackberry-1
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users