Networker

Re: [Networker] setting block size

2003-08-12 23:05:03
Subject: Re: [Networker] setting block size
From: Matt Temple <mht AT RESEARCH.DFCI.HARVARD DOT EDU>
To: NETWORKER AT LISTMAIL.TEMPLE DOT EDU
Date: Tue, 12 Aug 2003 23:04:19 -0400
Since I've spent a good bit of time trying to deal with some
of these issues, I'd appreciate some reduction in the sparring.
Also, I think it's nice to have Legato employees reading and responding
to posts, it would also be REALLY nice if they didn't respond as though
in a sparring contest when they are challenged.   Also, it's a real
cheat to claim knowledge from looking at source if release notes say
elsewise.   (I would think that if there were some disparity, then the
correct Legato Employee response would be to say something like "I'll
be sure they issue a correction to the statements in the release notes
as soon as possible.)

As a long term Linux user and a VAX / Unix system manager since 1981,
I wouldn't say that any of the release notes about stinit.def has been
helpful to me until I stumbled across one with settings for AIT-2 and
AIT-3 by accident.   Legato hasn't been what I'd call an excellent
reference for this.

It sure seems like this conversation would benefit from group input and
<definitive> input from Legato.   (i.e., it sure looks to me that
my Linux machine was doing 512-byte blocks on my AIT-2 drives, not just
my AIT-3 drives.

Anyway, I don't like to be pummeled by in-house expertise.   Jose, if
you
have inside knowledge, just give us the source and quit the showing off,
please.

                                                                                
               Matt Temple

On Tuesday, Aug 12, 2003, at 14:41 US/Eastern, Jose Quinteiro wrote:

Davina Treiber wrote:
Oh dear, so much misinformation in one thread. This subject has been
covered so many times before, yet people are still posting stuff that
is
plain wrong. Let's try to set the record straight.

Jose Quinteiro <jquintei AT LEGATO DOT COM> from Legato, who should know
better
and should have read the release notes and been on all the training
posted:


Please forgive me.  I thought looking at the source was sufficient.


Block size is set automagically by Networker based on device type.
As
long as the devices are configured to be the same type, the block
size
should be the same regardless of platform.


The environment variables are used to override Networker's defaults.
The defaults should be fine.


This is wrong. The default block sizes do vary between platforms for
some
device types. This is because of the limitations of certain HBAs and
their
drivers.


The only such limitation I am aware of is the one I mentioned on
Windows.   Please enlighten me as to the other exceptions.

In any case, Networker is trying to set the same blocksize it would use
in other platforms, and the driver just can't handle it.  It is, of
course, possible to mis-configure your OS (to use fixed-size blocks,
for
example) to ignore what Networker wants. But Networker wants the same
device-specific block size, regardless of platform.

The same Legato employee then posted with apparent authority:


(snip)
FWIW, the "Not enough space" messages typically mean the tape is
blank,
but you read a label sucessfully.  Any errors in your system log?


He of course should have known that "not enough space" frequently
means
that there is a block size mismatch. He also should know his
employer's
stance on employees posting to this list with a Legato email address -
especially when what they are posting is garbage.


That's why I said "typically".  That means "not every time."

Yura Pismerov then wrote:

Isn't it recommended to keep the default block size 0 (variable
length)
in Linux ?

Yes it is, but this is because setting the block size to 0 allows
NetWorker
to use the values configured. If your don't do this, Linux will only
allow
you to used a fixed block size, and if this is only 512 bytes it will
seriously restrict the system's performance.


Linux's st driver does not default to fixed-size blocks for most tape
drives I've used. AIT-3 is a notable exception.

Kit Cunningham then suggested:

My experince is windows2k networker uses the variable setting for
writing
only and the registry change for writing/reading. I do not know why.

This is wrong. The block size is set by setting the relevant
environment
variable. However this won't work if the scatter/gather buffer
settings are
insufficient, so it will just set it to a lower value - giving the
effects
described. YOU NEED TO DO BOTH!


The block size is set internally by Networker.  The environment
variables override the internal defaults. You absolutely DO NOT need to
mess with these environment variables! Please don't make things more
complicated than they need be.

and he continued:

I suggest ( if not already suggested by someone) you use st.conf
entries
to set your blocksize.

This is a pointless blind alley, because st.conf doesn't set the block
size. The application (NetWorker) does this, provided that you have
set
other factors in your operating system to allow this (e.g.
scatter/gather
buffers, stinit.def and so on).


An improperly configured st.conf can cause all sorts of blocksize
troubles.  If you don't set the ST_NO_RECSIZE_LIMIT flag the biggest
block Solaris will write will 64K - 2 bytes.  Furthermore,  it silently
breaks up bigger writes into these smaller blocks.  So if you try to
write a 128K block you'll get three blocks; 64K - 2 bytes,  64K - 2
bytes, and 4 bytes.  These tapes will have serious performance
problems
recovering in NW 6.1.3 and later, and may be unrecoverable with  some
combinations of Solaris/HBA in earlier versions of Networker.

The moral of the story is "don't mess with st.conf."  Use Sun's st
patches.  That's what Sun wants you to do, and they have good reasons.

I will send you my manager's contact information off-list.  Please
direct any further complaints about my postings to the list to him.


Saludos,
Jose.

--
Note: To sign off this list, send a "signoff networker" command via
email
to listserv AT listmail.temple DOT edu or visit the list's Web site at
http://listmail.temple.edu/archives/networker.html where you can
also view and post messages to the list.
=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=

========================================================
Matthew Temple                Tel:    617/632-2597
Director, Research Computing  Fax:    617/632-4012
Dana-Farber Cancer Inst       mht AT research.dfci.harvard DOT edu
44 Binney Street,  JF 314     http://research.dfci.harvard.edu
Boston, MA 02115              Choice is the Choice!

--
Note: To sign off this list, send a "signoff networker" command via email
to listserv AT listmail.temple DOT edu or visit the list's Web site at
http://listmail.temple.edu/archives/networker.html where you can
also view and post messages to the list.
=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=

<Prev in Thread] Current Thread [Next in Thread>