Re: Virtual tape to Real tape cycles

As I read through the man pages I am finding hints that it has beenthought of. In the amdd man page "This may be used for debugging or toconvert from one format to another."

I began looking at the contents of various directories and files basedon comments found in Gene's Amanda Helper scripts. The index is just alist of files contained in the images contained on the the tape. Itwouldn't change at all as the images aren't being modified. I couldn'tfind anything that maps an image file to a particular location on atape. It appears that the contents of a virtual tape need no furtherprocessing before moving them to a real tape. Copying those imagesonto a tape and moving the index, curinfo and tapelist date to anothercycle seems like it might be straightforward. Since all the imagesizes are known in advance they might even be reordered more optimallyto fit on tape. I didn't like Gene's suggestion that the virtual tapesize and the real tape size should be kept the same. We might evenchoose to put different file systems on different tapes as they may goto different offsite locations for security reasons.

Restores appear to collect the images associated with a particularfile system and give them back to the appropriate program to extractthe file(s) requested. That process wouldn't change just because themedia changed.


I'll take a look at the Developer Documentation to continue my exploration.

Quoting "Dustin J. Mitchell" <dustin AT zmanda DOT com>:

On Fri, Jun 15, 2007 at 02:14:43PM -0400, Jon LaBadie wrote:

On Tue, Jun 12, 2007 at 04:27:32PM -0400, Charlie Reitsma wrote:
> Has there been any work on amanda being aware of backups going to
> virtual tape and then the virtual tape being archived to real tape?
> I'd love to be able to keep as much as possible on virtual tape but at
> some point I need to archive the data to an offsite location. But in
> order to do a restore I would need the index of the virtual tape
> associated with the real tape. Very likely, multiple real tapes would
> be required to hold a single virtual tape so the index might need to
> be rewritten to reflect the media change. I see the move from virtual
> tape to real tape more of a tape duplication  - almost as if the
> virtual disk were looked at as the holding disk being flushed to tape.


To Charlie's initial question -- this is a *hard* problem, and one for
which a complete answer is a long, long way off.  However, we are
looking in that direction.  In fact, the Device API is a step along the
way, as it gives us a consistent way to talk about different devices
(e.g., vtape and tape) and moving data between them.

Related to potential projects such as Charlie suggests:

Does an documentation exist that might be called "Amanda Internals"?
Things that describe the backup process, authentication procedures,
network communications and particularly for this project, amanda
data location, layout, and parsing info?

Obviously several amanda utilities can access these data.  For example,
amoverview can lookup the dates and levels for each active DLE backup
and amrecover can determine on which tapes they are located.  But
aside from reading the sources for these utilities is the location
and procedure for accessing these data documented somewhere?


We created the "Developer Documentation" section on wiki.zmanda.com a
few weeks ago.  It's far from comprehensive, but it contains links to
all of the developer-targetted documentation we could find, which
includes descriptions of many of the APIs, authentication methods, and
data storage formats.

The (small) parts of that section that I wrote myself are based on
things I learned while exploring new areas of the codebase.  If someone
wanted to take on a significant project in Amanda, I think it would make
sense to leverage the exploration that would take place to generate some
additional developer documentation.

Dustin

--
        Dustin J. Mitchell
        Storage Software Engineer, Zmanda, Inc.
        http://www.zmanda.com/




Charlie Reitsma
x6642