Networker

Re: [Networker] Big endian versus little endian?

2003-12-11 22:25:57
Subject: Re: [Networker] Big endian versus little endian?
From: Byron Servies <bservies AT PACANG DOT COM>
To: NETWORKER AT LISTMAIL.TEMPLE DOT EDU
Date: Thu, 11 Dec 2003 19:25:32 -0800
Hi there!

I am about to suggest something that will probably not work. It is not
supported by Legato in any way.  You should use the NetWorker migration
guide (where ever they keep that these days) instead.  I have also not
tried it, which makes this an even worse suggestion.

However, if it works, it will work dandy.  If not, it shouldn't
function at all.

On Dec 11, 2003, at 12:54 PM, George Sinclair wrote:

Let's say I want to move the primary server from Solaris to Linux. I'm
not sure which is Big endian or which is little endian.

Solaris and Linux are neigther big nor little endian.  Endian-ness
refers to the order in which a particular architecture stores data in
memory.  For reference:
<http://www.webopedia.com/TERM/b/big_endian.html>

Sparc is big endian and x86 is little endian.  MIPS can actually be
either big or little endian.  PowerPC also has this option and MacOS
uses it in big endian mode normally, but VirtualPC switches it to
little endian mode to emulate x86 instructions.

So, how does this effect NetWorker?

Guess the bytes are read in a different order, though, or some such
thing.

All data NetWorker exchanges between the client and the server is
independent of endian-ness.  The data from clients is converted to
eXternal Data Representation (XDR) before it is sent to the server and
all data written to tapes is in this platform-agnostic format.  This is
why you can scan a tape created on a little-endian server (e.g. windows
on x86) into a big endian server (e.g. solaris on sparc) and it will
work.  However, the server stores it's bookeeping data, indexes,
differently.

Can't recall my Jonathan Swift details on that, but I do recall the
main
problem spoken of was the transferring of the client indexes from one
endian to a different endian. Was this in fact the real only concern
when moving from one platform to another?

Before NetWorker 6.x both the media and client (aka "file") indexes
used the WISS database to store information.  WISS is similar in
concept to the Berkeley DB (www.sleepycat.com).  These files are
endian-specific because they store the binary representation of
information directly without converting it to a platform-agnostic
format first.

For version NetWorker 6.x, the client indexes were changed to use the
XDR format for their information, which is why server migration of
client indexes is now somewhat more straight forward.  It also provided
massive performance increases. The media database, however, has not
been converted yet.

Why does Legato not make an index convertor? Would that be too hard?

In fact, it does.  Upgrades to the index data format are done using
ASM's.

WARNING:  this is where I depart from what I know as a former NetWorker
engineer, and what I have actually tried to do.  If you try this, USE A
TEST SERVER!  Indeed, we had several discussions about this and whether
it would work or not, but I don't know anybody who actually tried.

NetWorker has the application-specific modules (ASM) architecture which
makes adapting to different clients and applications (like databases)
relatively simple.  The indexes are like these databases.  When save
runs, actual backups are done by ASM's that are complied in to save, or
executed by save. One of the tasks of an ASM is to read the data in and
convert it to the platform-agnostic XDR format.  This does NOT,
however, mean that the application can necessarily cope with data in a
different endian representation at restore time.

This is why restoring a binary file from one machine to another may not
necessarily work.  The bits are restored as they were backed up, but
the destination machine's application may still not be able to read it.

One way NetWorker can be used that is generally disregarded is as a
replacement for tar.  The uasm command is just like save, but without
the connection to the NetWorker server.  It creates the
platform-agnostic data stream and sends it to stdout.  So, in theory,
you should be able to say:

       # nsr_shutdown
       # uasm -s /nsr/index /nsr/mm > /tmp/nsr.uasm

Note that I am so unsure of this I don't even remember completely the
names of the index directories.  Caveat emptor.  Have I said this is a
bad idea yet?  Don't yell at me if you use this on a production server
and it all goes ahoo.

copy the nsr.uasm file to the new server and run something like:

       # mv /nsr/mm /nsr/mm.save
       # mv /nsr/index /nsr/index.save
       # uasm -r < /tmp/nsr.uasm

uasm will take care of using the appropriate ASMs (nsrindexasm and
nsrmmdbasm) and you should get a working /nsr/mm and /nsr/index
configuration that is just like your old one. I don't know if it will
blow away your old indexes or not (it shouldn't based on my
recollection).

Remember, the migration guide is the correct way.  This might work, and
it might not.  Good luck to those who will try.

Byron

--
Note: To sign off this list, send a "signoff networker" command via email
to listserv AT listmail.temple DOT edu or visit the list's Web site at
http://listmail.temple.edu/archives/networker.html where you can
also view and post messages to the list.
=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=