Bacula-users

Re: [Bacula-users] Catastrophic overflow block problems

2013-01-17 16:06:31
Subject: Re: [Bacula-users] Catastrophic overflow block problems
From: Josh Fisher <jfisher AT pvct DOT com>
To: Ruth Ivimey-Cook <ruth AT ivimey DOT org>
Date: Thu, 17 Jan 2013 16:03:20 -0500
On 1/17/2013 2:10 PM, Ruth Ivimey-Cook wrote:
Josh Fisher wrote:
You don't.
I find it very strange that returning "device full" from a volume write can reasonably be interpreted as "device not quite full".
The trick is to define a maximum volume size and number of volumes on the drive so that it is impossible to reach 100% of the physical drive's capacity. This will prevent the i/o error, and Bacula will instead hit end of volume and seek another volume. Of course, if no existing volumes can be recycled yet, then there simply isn't enough space on the drive. In that case, it is easy to add another drive to an existing autochanger, since vchanger allows for multiple simultaneous "magazine" drives.
I don't understand how to do this then without defining the number of volumes so low that I waste huge amounts of space on the drives as a matter of course.

One way is to partition the drives. Keeping volumes of the same size on the same partition allows specifying the exact number of volumes. Each partition is a magazine, and any number of partitions can be used simultaneously. For example, break a 1 TB drive into two partitions, one 200 GB partition holding 10 volumes in a pool with a max volume size of ~20 GB for incremental jobs, and an 800 GB partition holding 8 volumes in a pool with max volume size of 100 GB for full jobs. Etc.


A little more detail about what I'm doing:
  • Some backups are assigned longer retention times than others - e.g. some full backups live for a year, some incrs live for just 3 months.
  • I have various max volume sizes from 20GB to 400GB, assigned to each file pool depending on the likely size of a backup (e.g. incrs are likely smaller than full) so that a volume will expire in a reasonable time - I don't want 100GB of backups to be kept alive (and using space) because they are in the same volume as more recent backups that haven't expired yet.
  • I have set up 24 volumes per disk so that, should the volumes be shorter 90GB ones, I don't (on average) run out of volumes too quickly.
  • The result is that most disks are reasonably full most of the time, which is good.
To be honest, I wish Bacula had a "disk mode" in which the concept of volumes was mostly eliminated: devices had backup pools and backups within them and it would be backups that were recycled. It would make much more sense for a random-access medium.

True, but Bacula must also work with tape drives, and that would be a very extensive rewrite.


Would an alternative solution be to adapt the vchanger program so that it monitored disk space and returned device full "early"?

No, because vchanger only runs very briefly when Bacula requests a volume be "loaded" or "unloaded". It basically points Bacula to the particular volume file it is to use and then exits. Bacula reads/writes the file directly, so there is no interaction between vchanger and Bacula when the data is actually being written


Ruth

-- 
Software Manager & Engineer
Tel: 01223 414180
Blog: http://www.ivimey.org/blog
LinkedIn: http://uk.linkedin.com/in/ruthivimeycook/ 

------------------------------------------------------------------------------
Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS,
MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current
with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft
MVPs and experts. ON SALE this month only -- learn more at:
http://p.sf.net/sfu/learnmore_122712
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users
<Prev in Thread] Current Thread [Next in Thread>