Bacula-users

Re: [Bacula-users] How to organize File-Bases Volumes...

2011-01-27 06:15:18
Subject: Re: [Bacula-users] How to organize File-Bases Volumes...
From: Graham Keeling <graham AT equiinet DOT com>
To: bacula-users AT lists.sourceforge DOT net
Date: Thu, 27 Jan 2011 11:12:37 +0000
On Thu, Jan 27, 2011 at 05:42:10AM -0500, Phil Stracchino wrote:
> On 01/27/11 04:29, ml ml wrote:
> > Hello List,
> > 
> > I have this Pool definition with autolabeling.
> > 
> > Pool {
> >   Name = File
> >   Pool Type = Backup
> >   Recycle = yes                       # Bacula can automatically recycle 
> > Volumes
> >   AutoPrune = yes                     # Prune expired volumes
> >   Volume Retention = 21 days         # one year
> >   Maximum Volume Bytes = 1M          # Limit Volume size to something 
> > reasonable
> >   Maximum Volumes = 500               # Limit number of Volumes in Pool
> >   Label Format = "Volume-"
> > }
> 
> Um, 1Mb volume size limit is "reasonable" on which planet?  Perhaps if
> you're running a computer that boots from floppy disks.
> 
> > The big disadvantage is that i will run out of Volumes soon since each
> > Backup Job will create a Volume, even if its only 1MB big.
> > Can i set "Maximum Volumes = 64000". I guess not since autolabeling
> > starts with -0001.
> > 
> > I also failed to "reuse" the old Volumes since they are on one Pool
> > but have diffrent "Archive Device" paths.
> > 
> > 
> > Any idea how to solve this?
> 
> Well, I'd start by setting a more realistic Volume size limit.  At one
> megabyte, you're going to need more than one volume to hold many single
> files.  For instance, a typical Linux kernel will occupy two to four
> volumes.  That's like backing up to 3.5" floppies.  Purely apart from
> the insane number of Volumes it'll create, the overhead of constantly
> creating, opening and closing volumes will kill your throughput.
> 
> With your settings, setting volume size to something saner like, say,
> 10GB or more would solve most of your problems.  But frankly, setting
> size limits is far from the best way to handle disk volumes in the first
> place.  You're much better off to specify a Maximum Volume Jobs
> sufficient to hold a day's or a weeks' worth of backups, or setting a
> Volume Use Duration just short of 24 hours or 7 days (or however long
> you want to use a single Volume before moving on to the next one).

I was thinking about something related to this the other day.

If you wanted to manage the space on your disk, you might decide to
limit the number of equal-sized volumes that bacula can have.

But how big do you make your equal-sized volumes?
Assuming one job per volumes...
If you make them too big (like 10GB or more each), you will inevitably
waste disk space because if your backup was small (an incremental might well
be expected to be a few MB), a whole 10GB will be used up.
If you make them too small (like 1MB each, which makes about 1 million volumes
on a terabyte disk), it seems quite likely that you will start hitting some
sort of disk access overhead problem, or that the database will be
unmanageable, or that bacula will slow down massively on particular
database queries.
So, I don't know what a reasonable size would be.

Perhaps 10GB becomes more reasonable if you allow more than one job per volume,
but then you still have the problem of disk space being wasted because you
cannot purge the volume until the last small 1MB job passes its retention time.

I think this last problem is what Phil is trying to solve by setting either
Maximum Volume Jobs or Volume Use Duration. But these solutions seem
unsatisfactory for disks (I can't comment on tapes because I don't know enough
about them).
You are wasting space if the volume is not full up by the time Volume Use
Duration expires.
And you are wasting space if the volume is not full up with Maximum Volume
Jobs.


------------------------------------------------------------------------------
Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)!
Finally, a world-class log management solution at an even better price-free!
Download using promo code Free_Logger_4_Dev2Dev. Offer expires 
February 28th, so secure your free ArcSight Logger TODAY! 
http://p.sf.net/sfu/arcsight-sfd2d
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users