Amanda-Users

Changer API (WAS: Re: BSD Users: Barcode Readers w/ chio(1)/mtx(1))

2005-08-25 17:51:16
Subject: Changer API (WAS: Re: BSD Users: Barcode Readers w/ chio(1)/mtx(1))
From: "Brian A. Seklecki" <lavalamp AT spiritual-machines DOT org>
To: amanda <amanda-users AT amanda DOT org>
Date: Thu, 25 Aug 2005 17:38:59 -0400 (EDT)

All:

The reason I inquired about the output and in general was because of a plan to re-write part of chg-chio in order to add more functionality that we require at our location. I know it sucks big-time, but unfortunately during my research on the topic, I stumbled onto another OSS/FS network tape backup project that already has these features (whose name we won't mention here) and don't forsee time to buget towards coding.

Either way, I think Amanda has some promise and since I've already allocated significant resources to deploying it here, I'll propose these questions anyway:

Given the loosely defined API between amdump(8)->amtape(8)->chg-{?}, and the amdump(8) algorithms on available media selection such as:
http://www.amanda.org/docs/tapechangers.html#id2538096
...I was wondering what would be required to create a more-platform independent hardware API? I.e., the features that each changer-script can provide vary greatly depending on the platform and the drive. Greater communication abour features is required.

The changer scripts accept: -reset, -clean, -eject, -slot, -info, -search, -label

amtape(8) accepts (from amdump(8) or amcheck(8): 'eject', 'reset', 'show', 'label' [var], 'taper', 'device', 'current', 'update', 'slot' [#,current,prev,next,first,last, and advance]

However, given that pretty reasonable level of abstraction, with a very simple interface, there is still only one tape selection algorithm. For example:

"With a full-access tape changer, amdump searches the entire rack for the preferred tape label. This tape will usually be found at the current or next position, but might be located anywhere. If the tape is found, it is used. If it is not found, the first tape encountered that matches the labelstr and is not active is picked."

*) Couldn't additional levels of administrative control be added to the tape / changer / hardware level?

- I.E., AFAIK, the "preferred tape" is the tape at the bottom-most line in the 'tapelist' file. Current and Next seem to be relative to the presence of the labeled tape in the drive.

- The 'taper' algorithm uses the least efficient possible method for searching for the preferred tape. It also puts undue usage stress on the hardware in question, working the picker and drive loader mechanism unnecessarily and shortening the life there of. It does so because there is no existing way of telling amtape(8) about a changer's capabilities (bar code reader, gravity, etc.) and as a result, because Amanda provides no changer-script independent mechanism for caching data on the contents of a changer's magazine. (What tapes are in the changer), a "scan forward" algorithm is required.

- Per the remark in server-src/changer.c, if the changer is chg-scsi or so, then then that bar code reader data may be able to flow back to amtape(8) but only if the labels on the physical tape enclosure match the Amanda internal labels identically.

--

*) I.E., If amtape(8) knew more about a changer's capabilities, it could make a better tape changing decision. If the changer has barcode reader support and the operator chooses to utilize it:

- Maintain a changer-script independent format database (even if just a Berkeley DBD) of which tapes are in the rotation tape cycle are in which slot in the magazine, when they were last used, active status (offline, online, in storage, etc.), size, general history, the current contents of the drive, the current position of the picker, etc.

- A next-tape decision can be made by some operator chosen mechanism / algorithm, possibly by manual election by the operator.

- If the barcode reader is present AND the tape's physical barcode labels match Amanda's regexp, use a hardware/platform independent method (read raw SCSI commands, parse the output of chio(1)/mtx(1), etc.) to locate and load the tape by it's bar code label.

- But a more likely scenario: If the bar code reader IS NOT PRESENT or is present the physical labels DIFFER FROM THE AMANDA INTERNAL LABELS in the regexp, use the aforementioned database to cache the result of an "amtape update" operation to cache the relation between tape slot contents, amanda labels, and/or physical labels.

- The only time a full load/read/eject "scan" cycle on the magazine would need to be performed is when the physical meda changes in the magazine (by operator intervention). In which case, a CLI or administrative interface to the database could be used to manually update the contents, for a full scan could occur

The "taper" operation will then instantly know the position of the next tape.

- In either case, the "taper" operation will then instantly know the position of the next desired tape.


*) Regarding a CLI or API to a tape database, without actually converting the primary logs/status database stores in ${configdir}/[index,curinfo], there should definitely be additional algorithms and an enhanced interface for selecting the next tape from the cycle. I.e.,

- interface for manual selection/promotion of the next tape for events like media failure and manual runs (amflush/amcleanup), permanent or temporary - interface for helping plan every day operations like off site storage
        - also for planning retention times
- statistics collection for monitoring growth and predicting expansion needs, etc.
        - cleaning tapes usage

*) This would also function as a stepping stone for ambitious goals such as multi-drive changers, on-tape encryption, etc. >:}


        --
~ TIA,

Brian A. Seklecki
Collaborative Fusion, Inc.
bseklecki AT collaborativefusion DOT com
412-422-3463 x 4018
1710 Murray Avenue, Suite 320
Pittsburgh, PA 15217


On Thu, 25 Aug 2005, Douglas K. Rand wrote:

** On Fri, 19 Aug 2005 19:19:51 -0400 (EDT), "Brian A. Seklecki" <lavalamp AT 
spiritual-machines DOT org> said:

Brian> Is anyone using a barcode reader enabled tape changer on FreeBSD?

Yup. We are.

We have a Sony labled AIT changer, a AIT Library 15^2 E. Its a
re-branded Spectra Logic 215:

ch1 at ahc1 bus 0 target 13 lun 0
ch1: <SPECTRA 215 S019> Removable Changer SCSI-2 device
ch1: 3.300MB/s transfers
ch1: 15 slots, 1 drive, 1 picker, 0 portals

Brian> I can't seem to find any good posts in the archive with example
Brian> output from "chio status -v" or "mtx status" on [Free]BSD w/ a
Brian> working barcode reader.

# chio -f /dev/ch1 status -v
picker 0:  voltag: <:0>
slot 0: <ACCESS,FULL> voltag: <DAL013:0>
slot 1: <ACCESS,FULL> voltag: <DAL014:0>
slot 2: <ACCESS,FULL> voltag: <DAL015:0>
slot 3: <ACCESS,FULL> voltag: <DAL016:0>
slot 4: <ACCESS,FULL> voltag: <DAL017:0>
slot 5: <ACCESS,FULL> voltag: <DAL018:0>
slot 6: <ACCESS,FULL> voltag: <DAL019:0>
slot 7: <ACCESS,FULL> voltag: <DAL020:0>
slot 8: <ACCESS,FULL> voltag: <DAL021:0>
slot 9: <ACCESS,FULL> voltag: <DAL022:0>
slot 10: <ACCESS,FULL> voltag: <DAL006:0>
slot 11: <ACCESS,FULL> voltag: <OFF007:0>
slot 12: <ACCESS,FULL> voltag: <OFF008:0>
slot 13: <ACCESS,FULL> voltag: <OFF009:0>
slot 14: <ACCESS,FULL> voltag: <CLN002:0>
drive 0: <ACCESS> voltag: <:0>

# mtx -f /dev/pass4 status
 Storage Changer /dev/pass4:1 Drives, 15 Slots ( 0 Import/Export )
Data Transfer Element 0:Empty
     Storage Element 1:Full :VolumeTag=DAL013
     Storage Element 2:Full :VolumeTag=DAL014
     Storage Element 3:Full :VolumeTag=DAL015
     Storage Element 4:Full :VolumeTag=DAL016
     Storage Element 5:Full :VolumeTag=DAL017
     Storage Element 6:Full :VolumeTag=DAL018
     Storage Element 7:Full :VolumeTag=DAL019
     Storage Element 8:Full :VolumeTag=DAL020
     Storage Element 9:Full :VolumeTag=DAL021
     Storage Element 10:Full :VolumeTag=DAL022
     Storage Element 11:Full :VolumeTag=DAL006
     Storage Element 12:Full :VolumeTag=OFF007
     Storage Element 13:Full :VolumeTag=OFF008
     Storage Element 14:Full :VolumeTag=OFF009
     Storage Element 15:Full :VolumeTag=CLN002

Brian> Some web sites offer printing of custom labels.

We ordered ours from ISSI for our AIT tapes. It was only one order,
and a while ago, and I don't remember any good or bad things about the
experience.

Brian> There's also a separte remark FreeBSD chio man page regarding a
Brian> 'voltag' command:

Brian> I'm curious how this relates to the physical bar code placed on
Brian> the tape? Is this some sort of proprietary feature only
Brian> supported by a select few tape drives?  Such as a small
Brian> meta-label written to the first block the tape similiar to the
Brian> Amanda lablel?

The volume tag is what is printed on the bar code. It has nothing at
all to do with any contents on the tape media itself. It is only the
bar code.

Brian> Supposedly, chio(1) can execute a command:

Brian>      # chio move voltag VOLUME01 drive 0

Yup.  Although I use Amanda's amtape command instead.




l8*
        -lava

x.25 - minix - bitnet - plan9 - 110 bps - ASR 33 - base8

<Prev in Thread] Current Thread [Next in Thread>