Networker

Re: [Networker] Multiplexing and demultiplexing on clones?

2003-05-01 19:36:08
Subject: Re: [Networker] Multiplexing and demultiplexing on clones?
From: George Sinclair <George.Sinclair AT NOAA DOT GOV>
To: NETWORKER AT LISTMAIL.TEMPLE DOT EDU
Date: Thu, 1 May 2003 19:36:06 -0400
Okay, Terry, please be patient. I think I'm close to understanding this.
Now, just to clarify, you can't separate the SSID values with a comma. I
tried that. The nsrclone command will not accept that. The correct and
only acceptable syntax is a space, or I suppose one or more spaces.

I follow what you're saying if you're cloning everything from say one
source tape, but if you're specifying different savesets on the same
tape or different ones from different tapes then how could NetWorker
preserve the interleaved nature of the data? It seems to me that the
only way it could would be if it included the other stuff that that
saveset is wrapped together with -- namely, the other savestreams that
were multiplexed with it when it was originally backed up. Now, since
the clone doesn't end up with those other savesets, how could the data
still be interleaved or multiplexed on the tape the way it was on the
source? I mean, if you said:

nsrclone ssid1 ssid2 ssid3 ssid4 ssid5

and ssid1-3 were multiplexed together on the source tape then yeah, I
can see that ssid1-3 will now be multiplexed to the clone, but if ssid4
and ssid5 are on separate places on the source tape and were never saved
to the source tape at the same time (i.e., they were never multiplexed
together when their savestreams were written) then NetWorker could
hardly interleave these with what ever they were interleaved with on the
source tape since maybe whatever ssids they were originally interleaved
with might not have been specified on the nsrclone command. In my
example, I'm going to assume that ssid4 cam from the same tape but ssid5
did not. Maybe ssid4 was interleaved with ssid50-55 and ssid5 was
interleaved with ssid 14-20.

Perhaps you are suggesting that the savesets that do get cloned are
themselves interleaved or multiplexed onto the clone volume, and this
has nothing to do with the nature of the way those savesets were
originally laid out on the source tape or what they were interleaved
with on the source tape?

Perhaps I am confusing the term "interleaved" with the term
"multiplexed"?

I know you guys pursued this to a much deeper level than I did, with
things like record numbers and starting and ending file number, etc. so
I'm not disagreeing but rather I just need some more explanation here. I
must be looking at this all wrong.

Do we agree, though, that NetWorker is cloning the savesets one at a
time regardless of how you do it? I see the devices window always shows
1 of total, 2 of total, ... total of total.

Thanks.

George


> Terry Clayton wrote:
>
> Hi George,
>
> I sent this to the mail list a week or so ago. I was hoping it
> clarified the issue - Please tell me where it is unclear and I will
> try and get an answer for you.
>
> "nsrclone SSID1, SSID2, SSID3.....SSIDX  will move data from the
> source to the target without any change to the data and no tape
> thrashing.....
>
> If a tape has 4 savesets interleaved, the data chunks are moved
> in order and laid down exactly without any de-deinterleaving - and
> then  re-interleaving the data.
>
> If you want to de-interleave the data then:
>
>          nsrclone SSID1
>          nsrclone SSID2
>          nsrclone SSID3
>          nsrclone SSID4
>
> In this case the data is read and written one save set at a time
> and the source  tape is repositioned accordingly, while the target
> tape device has the savesets sequentially written. and of course it
> takes 4 times as long to complete than the previous case."
>
> I think you may have made an incorrect assumption ..
>
> you stated "...since NetWorker has to read it -- just like it would if
> you were recovering it -- before it can write it..."
>
> writing the savesets one at a time, if MUST be de-multiplexing it,
> right?"
>
> Wrong. When there is a list of savesets on the nsrclone command it
> reads the tape segment and asks 'Is this part of a Saveset I need to
> clone?' If the answer is 'Yes' it writes it to the clone tape. If the
> source tape has the savesets interleaved so does the clone tape.
> Otherwise Networker would have to read and find all the separate
> segments of a saveset, then rewind the tape and do the same for the
> next saveset.
>
> OK?
>
> > -----Original Message-----
> > From: George Sinclair [mailto:George.Sinclair AT noaa DOT gov]
> > Sent: Thursday, May 01, 2003 3:13 PM
> > To: NETWORKER AT LISTMAIL.TEMPLE DOT EDU
> > Subject: [Networker] Multiplexing and demultiplexing on clones?
> >
> >
> > Okay, gang, since I was away, I see that my original posting titled:
>
> > "Merits of cloning versus dual backups?" has since created a
> firestorm
> > of questions in this area of multi-plexing versus de-multi-plexing
> of
> > data when cloning is involved. I read all of the replies -- can't
> say
> > that I completely understand all of the nitty gritty -- and I must
> say
> > that I am impressed with the level of detail that this discussion
> has
> > gone into, but I'm not clear that there was ever a final word or
> > agreement on this issue. I did some testing, and I thought I'd
> present
> > this and see if there's a final word here, or if I'm just in the
> dark.
> >
> > I backed up 4 clients to one tape (scratch.001) for a total of 15
> > savesets. Only one drive was used with a parallelism of 5.
> > The sessions
> > was never exceeded. 5 was the max I ever saw writing at one time.
> > Clearly, some, if not all, of the savesets are multi-plexed or
> > interleaved onto the target volume (scratch.001) if I can use
> > that term.
> > I then created 4 clone volumes (scratch_c.001 - 004) as follows:
> >
> > 1. nsrclone -s server -b 'Scratch Clone' -S ssid1 ssid2 ssid3 ....
> > ssid15
> >
> > 2. nsrclone -s server -b 'Scratch Clone' -S -f file
> >
> > where file contained a list of all 15 ssids in the same order
> > as listed
> > in option 1.
> >
> > 3. nsrclone -s server -b 'Scratch Clone' -V scratch.001
> >
> > 4. I looped through the file used in option 2 and then
> > directed this to
> > an nsrclone command to clone just that ssid. So, in essence, I ran a
>
> > separate clone process for each of the 15 savesets.
> >
> > I wanted to keep the test as fair to each option so I mounted
> > all of the
> > clone volumes before running any of the tests in order to
> > avoid any hang
> > time from ejects, mount operations, etc. All options produced the
> same
> > end results, but option 4 was very slow (no surprise, I'm sure) due
> to
> > the obvious fact that various rewinds and fast forwards were
> required
> > during each invocation of the nsrclone command. I don't think I even
>
> > bothered to time this. Options 2 and 3 were nearly identical in
> speed.
> > Option 1 was about 40 odd seconds slower than options 2 and 3.
> >
> > From what I can tell, NetWorker would have to de-multiplex the data
> > before it clones it because it clones the savesets one at a time
> > regardless of the method. This is obvious from the devices
> > window. I can
> > see the devices window listing 1 of 15, 2 of 15, 3 of 15 ... 15 of
> 15.
> > The data was clearly multi-plexed when it was written to the source
> > tape: scratch.001, but since NetWorker has to read it -- just like
> it
> > would if you were recovering it -- before it can write it, and it's
> > writing the savesets one at a time, if MUST be de-multiplexing it,
> > right? Now, perhaps this changes when you're cloning an
> > entire volume as
> > opposed to just specifying individual savesets, but it's been
> > mentioned
> > by several people that cloning a volume does not imply making an
> exact
> > copy of it but rather a clone of each saveset -- a different
> > concept all
> > together. Thus said, this is the same as cloning each one
> > individually,
> > although a lot faster.
> >
> > When you're writing one saveset at a time, you're not multi-plexing.
> I
> > asked Legato about this business with clones, and they agreed
> > that it is
> > not re-multiplexing the data when it clones, and is in fact
> > de-multiplexing since it has to read the saveset before it
> > can write it
> > to the clone copy, and it can only read one at a time. So, if the
> > saveset that is being read was multiplexed onto the source tape then
>
> > it's being de-multiplexed to read it and thus it will remain
> > that way on
> > the clone since it never writes more than one at a time to the
> clone.
> > So, recovers from clones should be indeed be faster since there's
> > nothing to de-multiplex.
> >
> > Does anyone have any comments. I'd like to hear them, please?
> > I suspect
> > I may have overlooked something in my thinking and could be drawing
> > false conclusions.
> >
> > Thanks.
> >
> > George
> >
> > --
> > Note: To sign off this list, send a "signoff networker"
> > command via email
> > to listserv AT listmail.temple DOT edu or visit the list's Web site at
> > http://listmail.temple.edu/archives/networker.html where you can
> > also view and post messages to the list.
> > =*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=
> >

--
Note: To sign off this list, send a "signoff networker" command via email
to listserv AT listmail.temple DOT edu or visit the list's Web site at
http://listmail.temple.edu/archives/networker.html where you can
also view and post messages to the list.
=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=