Bacula-users

[Bacula-users] Recreate boostrap files for jobs from the output of `bls -j`

2017-03-22 05:09:03
Subject: [Bacula-users] Recreate boostrap files for jobs from the output of `bls -j`
From: Konstantin Khomoutov <flatworm AT users.sourceforge DOT net>
To: bacula-users AT lists.sourceforge DOT net
Date: Wed, 22 Mar 2017 12:07:55 +0300
Hi!

I have a somewhat unusual Bacula setup as we're backing call records
produced by a call-center software -- so the files are tiny in size,
there is very large number of them produced daily, they are all unique
and we have to keep them for several years as required by the local
law.

Due to the uniqueness + tiny size + large quantities property of these
data, we have decided to keep only the information on the last few
backup session in the database to keep its size manageable, and resort
to "painful" restoration of old files -- for which job records are
expired by the volumes are still there -- using `bextract`.

To do restores using `bextract` we have lists of files backed during
each backup session saved in flat files, and we also have Bacula
writing bootstrap file for each backup session.  So the restoration
of old files basically rolls like this:

1) Know the date of the files to be restored.
2) Find what backup session backed them up.
3) Grab the matching bootstrap file.
4) Use `bextract -b boostrap.bsr -V ... /output/dir`.

The problem is that as it turned out, Bacula silently wasn't generating
bootstrap files for our backup jobs despite having

  Write bootstrap = "/path/to/bootstrap/files/bsr-%j"

directive.  Okay, that's for another question.

So when we need to restore an old file, we know (1) and (2) but fail at
(3).  I learned that `bls -j` is able to dump the list of jobs recorded
on a given volume.  But what I'd really like to know is how could I
recreate bootstrap files from the output of `bls -j` run on old volumes?

Programming the solution is not the problem but I'd like to know how do
I properly form the contents for such bootstrap files given the records
I'd parse from the output of `bls -j`.
The problem I have with interpreting them is that `bls -j` gives
me out two pairs of file+block numbers identifying each backup session.
That appears to be neatly covering the range of data occupied by a
single backup job run but how do I properly put VolFile and VolBlock
ranges if a job's data crosses the boundary of a file?

For instance, here's a fragment from the real output for one of my
volumes (elided a bit for terseness):

Begin Job Session Record: File:blk=338:1 SessId=142 \
   SessTime=1427996423 JobId=5007
  Job=call-recs.2015-05-12_14.00.00_41 ...
End Job Session Record: File:blk=339:10868 SessId=142 \
   SessTime=1427996423 JobId=5007
  Date=12-May-2015 14:44:22 Level=F Type=B ...

Here, the job starts at 338:1 and ends at 339:10868, how do I write out
the bootstrap file's data to cover that whole range in it?
>>From [1], it appears that I could use

  VolFile=338-339

to cover the files but the VolBlock spec appears to pertain to a single
VolFile.  I thought of using multiple VolFile + VolBlock pairs but
that would require me to use an open range for the VolBlock of the
first pair, something like

  VolFile=338
  VolBlock=1-
  VolFile=339
  VolBlock=1-10868

(and by the way, are blocks numbered from 0 or 1?)

Is there a solution for my case?
Or should I maybe look for some other data from the output of `bls -j`?

FWIW, for these backups, there's always a single client backing up to
that tape at any given time, so the jobs do not have their data
intermixed on the tape.

1. http://www.bacula.org/5.1.x-manuals/de/main/main/Bootstrap_File.html

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Bacula-users mailing list
Bacula-users AT lists.sourceforge DOT net
https://lists.sourceforge.net/lists/listinfo/bacula-users