That makes perfect sense! I think if the savesets you were cloning were
not anywhere near in proximity on the same tape, or if each was
completely located on a separate tape and was not a continuation then I
think the multiplexing would be minimal or not at all, but it does make
sense that if two or more of the ssids that you're cloning are together
on the source tape then the interleaved nature would be preserved in
some manner, and clearly, some multiplexing must be occuring when the
ssids are written to the clone as you remarked. One thing I noticed, is
that the devices window at one point jumped from: "cloning 2 of 15" to
"cloning 4 of 15", with no mention of "3 of 15".  I saw this also at
like "12 of 15" with no "13" of 15", but the end results all show 15 of
15. Everything is there. Hmm...

Probably the reason why no conclusive information has been obtained from
Legato is because there are probably very few people there who truly
understand the code and how it works on a lower level. I think you'd
really need to speak to an engineer who knows the software on that
level, and they do exist, but you're unlikely to talk to such a person
because customer support normally always routes you through an outer
shell that moves inward only as need be. Getting someone who really
knows this stuff is not easy because most of them may not work in
customer support, so the information has to be relayed through various
people. The horse's voice sounds different after it's been translated by
five farmers.

George

Steve Barber wrote:
>
> I'm not posting to the list because I don't know the answer either
> but since I've done some coding and have some thoughts about how
> I would implement it if it were me, hopefully this is slightly
> better than just a guess...
>
> For the life of me I don't understand why the Legato people on the
> list won't speak up and clear the air about this issue once and
> for all; it's been an open question for years.
>
> On Thu, May 01, 2003 at 07:36:06PM -0400, George Sinclair wrote:
> > ...
> > I follow what you're saying if you're cloning everything from say one
> > source tape, but if you're specifying different savesets on the same
> > tape or different ones from different tapes then how could NetWorker
> > preserve the interleaved nature of the data? It seems to me that the
> > only way it could would be if it included the other stuff that that
> > saveset is wrapped together with -- namely, the other savestreams that
> > were multiplexed with it when it was originally backed up. Now, since
> > the clone doesn't end up with those other savesets, how could the data
> > still be interleaved or multiplexed on the tape the way it was on the
> > source? I mean, if you said:
> >
> > nsrclone ssid1 ssid2 ssid3 ssid4 ssid5
> >
> > and ssid1-3 were multiplexed together on the source tape then yeah, I
> > can see that ssid1-3 will now be multiplexed to the clone, but if ssid4
> > and ssid5 are on separate places on the source tape and were never saved
> > to the source tape at the same time (i.e., they were never multiplexed
> > together when their savestreams were written) then NetWorker could
> > hardly interleave these with what ever they were interleaved with on the
> > source tape since maybe whatever ssids they were originally interleaved
> > with might not have been specified on the nsrclone command. In my
> > example, I'm going to assume that ssid4 cam from the same tape but ssid5
> > did not. Maybe ssid4 was interleaved with ssid50-55 and ssid5 was
> > interleaved with ssid 14-20.
> >
> > Perhaps you are suggesting that the savesets that do get cloned are
> > themselves interleaved or multiplexed onto the clone volume, and this
> > has nothing to do with the nature of the way those savesets were
> > originally laid out on the source tape or what they were interleaved
> > with on the source tape?
> >
> > Perhaps I am confusing the term "interleaved" with the term
> > "multiplexed"?
>
> I think you're making it more complicated than it really is.  Here's how
> I think of it:
>
> - data is written to tape one chunk at a time.  Each chunk is part of one
>   savestream, i.e. from a single client.
> - when multiplexing is occurring it takes a chunk from each active
>   savestream in turn and writes them to tape.
>
> When you're cloning, it reads the tape serially.  It has to read each
> chunk off of the source tape and check to see if it's a saveset it's
> supposed to be cloning.  If it is, it simply copies it to the destination
> tape.
>
> So I assume it would simply start with the lowest numbered SSID that it
> wants to clone and start copying those chunks to the destination tape.
> However, if in the course of copying those chunks it runs into chunks of
> other SSIDs that it wants to copy, it probably just starts copying
> those also, thus some degree of multiplexing would be maintained.
> Chunks that are not to be cloned would be left out, of course.
>
> This may be a simplistic view; for instance it can probably figure out
> ahead of time which ones are multiplexed, and it may only pick up
> new SSIDs if it's starting at the beginning, i.e. it doesn't start
> seeing a new SSID in the middle of the stream.  (Like if the 1st part
> of the stream had been written to a different tape.)
>
> It probably tracks what SSIDs it's currently cloning, and as it reads
> through them that list will grow and shrink as it finds new SSIDs and
> finishes them and/or finds new ones.  It probably just runs until that
> list size reaches zero.  Then it removes those SSIDs that it finished
> from its master list and starts again with the next lowest SSID.
>
> I think the key is realizing that it's a serial operation.
>
> > I know you guys pursued this to a much deeper level than I did, with
> > things like record numbers and starting and ending file number, etc. so
> > I'm not disagreeing but rather I just need some more explanation here. I
> > must be looking at this all wrong.
>
> The evidence that was presented seemed pretty conclusive to me that
> multiplexing was maintained (at least partially) during cloning, and
> if it really did demultiplex during cloning, you would get horrible
> cloning throughput.  (And buffering the SSIDs you aren't actively
> copying would be prohibitive, since most of us don't have 100GB+
> of swap or even necessarily free disk space to buffer that data onto!)
>
> > Do we agree, though, that NetWorker is cloning the savesets one at a
> > time regardless of how you do it? I see the devices window always shows
> > 1 of total, 2 of total, ... total of total.
>
> No, I don't think this is a safe assumption.  It could be that the
> number is just how many it has started or finished.  Also, I've seen
> the total number reset several times during the course of a large
> cloning operation so it just isn't clear to me what's being counted
> or how.
>
> Hope that helps!
>
> Steve

--
Note: To sign off this list, send a "signoff networker" command via email
to listserv AT listmail.temple DOT edu or visit the list's Web site at
http://listmail.temple.edu/archives/networker.html where you can
also view and post messages to the list.
=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=