Networker

Re: [Networker] Need help with stinit.def for LTO-3

2007-01-30 19:46:36
Subject: Re: [Networker] Need help with stinit.def for LTO-3
From: George Sinclair <George.Sinclair AT NOAA DOT GOV>
To: NETWORKER AT LISTSERV.TEMPLE DOT EDU
Date: Tue, 30 Jan 2007 19:36:38 -0500
Thanks, Tim! I've provided my responses below, along with a few more questions
and one particularly puzzling discrepancy on the density codes.

Tim Mooney wrote:

In regard to: Re: [Networker] Need help with stinit.def for LTO-3, George...:

Here's ours, for HP drives (which is what StorageTek strongly recommended
to us).  They couldn't give us any suggestions for the mode/density


That's interesting because they shipped us IBM drives. We never requested any specific manufacturer. I find it surprising that they would use IBM drives, though, if HP was recommended?


That could just be a difference between our field service engineers and
yours.  Our FSEs and pre-sales support said we could use any of the
available vendors, but they recommended HP because they had seen much
higher times between failures with those.

I honestly wouldn't worry about it. I'm sure the IBM drives will be fine.


I think so, too. I feel better now, though. And others have said that the IBMs work
well, but just needed to be configured properly.

# LTO Generation 3
#
# TVM: here's what Sun/STK recommended:
#
# manufacturer=HP model="Ultrium 3-SCSI" {
# can-bsr scsi2logical drive-buffering auto-lock
# timeout=800
# long-timeout=14400
# mode=1
# blocksize=0
# density=0x00
# fast-eom=1
# }

manufacturer="HP" model = "Ultrium 3-SCSI" {
        scsi2logical=1
        can-bsr=1
        auto-lock=0
        two-fms=0
        drive-buffering=1
        buffer-writes
        read-ahead=1
        async-writes=1
        can-partitions=0
        fast-eom=1


1. I see that you have scsi2logical set to 1. I realize that this is for HP LTO-3, but Quantum's sample stinit.def file (in their PDF document) has this value remarked out as:
#scsi2logical=1.


No matter which vendor you choose, the SCSI protocol support and drive
behavior should be extremely similar, so an stinit.def for one vendor's
drive should be a good starting point when working with a different
vendor's LTO drive -- I would just start by changing the manufacturer
and model strings.

As far as scsi2logical=1 (use logical block addresses rather than
device-specific addresses), I'm not sure.  When I asked Legato about
stinit.def entries for our SuperDLT1 drives years ago, their support
pointed me at this

  http://www.nclug.org/pipermail/nclug/2002-August/004162.html

which has scsi2logical=1 set for all the DLT type drives.  I've also seen
scsi2logical=1 set for other types of drives.  That doesn't mean that LTO
should have it set to 1, but I've definitely seen more stinit.def entries
that have it set than those that don't.  Since the example I received
from STK/Sun had scsi2logical listed (which means scsi2logical=1), it
matched my previous experience with most tape drives.


From what I read in the stinit man page, the default is to use the device-specific addresses. So, if we set this to 1, then we will be using Logical block addresses instead. If we change this, should we still be able to recover data that was written to tape when the value was
previously remarked out (i.e.  set to use device-specific addresses.)?


Their stinit.def file includes definitions for all the SDLT and various
LTO devices. Nowhere else do they have scsi2logical set. Does this sound surprising?

2. I see you have 'buffer-writes', which I assume is equivalent to 'buffer-writes=1',


It is (assuming buffer-writes is even valid!).

but I
don't see a 'buffer-writes' keyword in the man page for stinit, but I do see
one called 'buffering', and I see "MT_ST_BUFFER_WRITES  (Default: true)"
under the man page for st. Maybe that's the same thing? The Quantum
stinit.def file has:
buffering=0


It probably is the same, since stinit uses the shortest distinct substring
to determine what the actual keyword is.  You could probably use

    buff-the-floor=1

and still get buffering.  I haven't actually tried that.

Note that I'm surprised that Quantum suggests buffering=0.  I would
think that would hurt performance.


It could very well be hurting our performance for all I know, but I've
not seen anything flagrant yet.

4. You have 'async-writes=1'. Again, they have: 'async-writes=0'. Does
that make sense? Would it make sense that maybe for HP LTO it might be
set to 1, but maybe, say for Quantum SDLT and LTO, it's set to 0?


I'm operating under the assumption that LTO drives from different
vendors should be very very similar WRT these settings.  I think Quantum
is setting buffering=0 because it's the more conservative setting.  Ditto
for async-writes=0.

The drivers/scsi/README.st file in the Linux kernel source tree says:

"Asynchronous writing. Writing the buffer contents to the tape is
started and the write call returns immediately. The status is checked
at the next tape operation. Applies only to variable block mode.

Buffered writes and asynchronous writes may in some rare cases cause
problems in multivolume operations if there is not enough space on the
tape after the early-warning mark to flush the driver buffer."

The man page for st also says this:

             MT_ST_BUFFER_WRITES  (Default: true)
Buffer all write operations in fixed block mode. If this option is false and the drive uses a fixed block size, then all write operations must be for a multiple of the block size. This option must be set false to
                    write reliable multi-volume archives.
             MT_ST_ASYNC_WRITES  (Default: true)
When this options is true write operations return imme- diately without waiting for the data to be transferred to the drive if the data fits into the driver's buffer. The write threshold determines how full the buffer must be before a new SCSI write command is issued. Any errors reported by the drive will be held until the next operation. This option must be set false to write reli-
                    able multi-volume archives.

Since NetWorker does span tapes, is that what they mean by multi-volume archives? So does this sound like we need to set both buffer_writes=0
and async-writes=0?  As far as I know, I've not seen any problems
spanning volumes on our SDLT600 drives, and we have 'buffering=0' and
'async-writes=0' but no keyword for 'buffer-writes'.


Those are more conservative settings than what we're using.  I'm guessing
that you're taking a performance hit for writes, but it might be pretty
small.

NetWorker goes to great lengths to correctly handle tape transitions,
and AFAIK using the more "dangerous" settings has never bitten us.  We've
been using NetWorker for approximately 11 years with about the last five
of those being on a Linux server, so I would think we would have seen
problems by now if the config settings we were using were problematic.


Good point. Maybe I'll risk changing it. Again, though, I wonder how changing this might effect our ability to recover data from tapes that were written when these values were previously
set to 0?

The other keywords and values that you have are all the same as those in Quantum's stinit.def file,
but they also have these:

no-wait=0
buffering=0
noblklimits=0


buffering is probably the same as buffer-writes, the other two look like
reasonable settings.

Not sure what the default is for keywords not overtly defined in stinit.def?


That probably depends on the st module.

# CERTANCE ULTRIUM
manufacturer=CERTANCE model="ULTRIUM 3" {
timeout=800
long-timeout=14400
can-partitions=0
mode 1 blocksize=0 density=0x44 compression=1 # ULTRIUM 3 density, compression on mode 2 blocksize=0 density=0x44 compression=0 # ULTRIUM 3 density, compression off mode 3 blocksize=0 density=0x42 compression=1 # ULTRIUM 2 density, compression on mode 4 blocksize=0 density=0x40 compression=0 # ULTRIUM density, compression


Very similar to yours, but they don't use a compression 'off' line for
LTO-2 like you have, and they include a line for LTO-1. We, too, have
never used LTO-2, only LTO-1, but I thought that even though we can't
write to the LTO-1 media in LTO-3 drives that we would still have to
define a mode for it to be able to read properly?


Perhaps, but I always thought that those modes only mattered for
writing, and since LTO3 cannot write in LTO1 format, there's no point
defining a mode for it.


It's odd, though, because the current documentation that Quantum has, and what you use for your HPs, and one other source I found all indicate that LTO3, 2 and 1 should use 0x44, 0x42 and 0x40 for the density codes respectively. However, the older documentation that Legato has that someone else posted earlier (ftp://ftp.legato.com/pub/HW_Support/compatguide/stinit.def) indicates density=0x00 for both LTO-1 and LTO-2 for IBM and HP. This seems like an odd discrepancy, doesn't it? The link you mentioned ( http://www.nclug.org/pipermail/nclug/2002-August/004162.html) doesn't include definitions for LTO. The document does list another link, but
that link comes up empty but looked to be dlt only.

I should note that we have another Linux storage node that actually runs the STK L80 with the LTO-1 Seagate drives that we'll be upgrading to IBM LTO-3. We also have an older SDLT-1 Quantum library attached as well. That stinit.def file uses the 0x00 code for the LTO drives, but I'm not sure where we got that stinit.def file from. We've been using it for the past several years, though. I think I pulled it off the internet way back when, but it's basically the same
as what Legato has. Here's the entry we have for the LTO-1:

manufacturer=SEAGATE model = "ULTRIUM06242-XXX" {
can-bsr scsi2logical drive-buffering
timeout=800
long-timeout=14400
mode1 blocksize=0 density=0x00  # as close as possible to st.conf entry
}

Maybe this was never correct to begin with, but this has worked fine for us. Now, I'm wondering, though, what will happen when I try to read those tapes on the new LTO-3 drives with the following
modes???:

mode 1 blocksize=0 density=0x44 compression=1 # ULTRIUM 3 density, compression on mode 2 blocksize=0 density=0x44 compression=0 # ULTRIUM 3 density, compression off mode 3 blocksize=0 density=0x42 compression=1 # ULTRIUM 2 density, compression on mode 4 blocksize=0 density=0x40 compression=0 # ULTRIUM density, compression

Our other storage node we have is also a Linux box but is running a newer Quantum library with SDLT-600 drives, so it's stinit.def file matches the current information that
Quantum has for the SDLT600 device since that's where I got that one from.

You have the uncommonly good luck of being in a position to test it
and report back, though, since you have LTO1!  ;-) All you have to do
is take one of your LTO1 tapes, load it in one of your LTO3
drives, and then read it using one of the device files that is not
configured for LTO1.  If you can read it, you've proved that the modes
don't matter for the read case.

You can either try running scanner on an LTO1 tape (use the -n -i options,
to scan the tape but not modify your indexes) or you can just scarf the
label off the tape:

    dd if=/dev/nst0 of=/tmp/tape-label bs=128k

You will need to match the setting for blocksize (bs) to whatever you were
using to write to the LTO1 tapes with your older drives.  That should
give you a file in /tmp with the contents of the NetWorker tape label in
it, you can get a view of the format using

    od -t c /tmp/tape-label | less

If that works, you'll have proved that you don't need special settings
for reading a tape written in an older (but compatible) format.


According to the NetWorker GUI for the devices, it shows Volume block size: 64 KB for the LTO-1 devices. The SDLT-600 devices and the SDLT-1 devices show 128 KB. I will certainly try what you suggest and report back once the new drives have been
installed.


Maybe I can track down a similar book for the IBM drives, but my guess
is that the density codes will be the same.


I'm sure they will be.

Tim



--
George Sinclair - NOAA/NESDIS/National Oceanographic Data Center
SSMC3 4th Floor Rm 4145       | Voice: (301) 713-3284 x210
1315 East West Highway        | Fax:   (301) 713-3301
Silver Spring, MD 20910-3282  | Web Site:  http://www.nodc.noaa.gov/
- Any opinions expressed in this message are NOT those of the US Govt. -
To sign off this list, send email to listserv AT listserv.temple DOT edu and type 
"signoff networker" in the body of the email. Please write to networker-request 
AT listserv.temple DOT edu if you have any problems with this list. You can access the 
archives at http://listserv.temple.edu/archives/networker.html or
via RSS at http://listserv.temple.edu/cgi-bin/wa?RSS&L=NETWORKER