There are definitely other processes, on either Unix or WinXX, that
will run and make use of the other CPUs on the system. However, all actions
taken by Networker are ran through NSRD in what I call the Mother-May-I (MMI)
syndrome. So if I go to 'media waiting' on a storage node, the storage node
MMIs through nsrmmd to get the tape loaded. However, the load request is
processed through a queue that includes all index elements and other MMI
functions (almost any process run on master or storage node involves some level
of MMI). If the CPU in question running NSRD is already running at critical
mass (100% utilization of single cpu), then the requests get queued deeper and
deeper regardless if you have spare cpu-cycles due to multiple processors.
Some (SOME) prioritizing is in place dependant on the networker
version. For example, I used to have regular and consistent media critical
errors, sometimes for hours on end as the tape mount requests queued and
waited...and waited...and waited. That was on 6.1.1. Now that I'm on 6.1.3,
the mount requests have been reprioritized and I only see media criticals at
peak times in the backup window. However, I still can't increase my per-second
throughput of data because I still have a bottleneck at the MMI portion of the
NSRD process. Faking a 'single as multi thread' process through some form of
wrapper or virtual processor, even with fairly wicked overhead, would still
result in a net gain for my master processing power and a shorter MMI response
time.
--Ted G. Reed II, Engineering Storage Services
-----Original Message-----
From: Narkinsky, Brian [mailto:Brian.Narkinsky AT DEP.STATE.FL DOT US]
Sent: Friday, June 13, 2003 3:19 PM
To: NETWORKER AT LISTMAIL.TEMPLE DOT EDU
Subject: Re: [Networker] Random technical question - Single threaded
process on multi-processor server
I haven't really looked at in depth. I am curious as I was considering
upgrading the server to a 4 proc machine. Have to do some more research.
I think there are processes (save.exe ???) spawned for each backup which I
would assume these would use any processor.
I'll have to experiment when I am at the server.
Brian
-----Original Message-----
From: Bokkelkamp Ernst [mailto:ernst.bokkelkamp AT siemens DOT com]
Sent: Fri 6/13/2003 3:58 PM
To: Legato NetWorker discussion; Narkinsky, Brian
Cc:
Subject: RE: [Networker] Random technical question - Single threaded
process on multi-processor server
Hallo Brian,
it may not be obvious, but NSRD is always multithreaded on a W2K
system, but
that does not mean that it the work done by nsrd is distributed over
multiple CPUs. nsrd,nsrexecd and nsrpm are services, that means that
there
will be threads just to enable them to run as service, like at least
one
thread to monitor the communication with the service control manager.
Other
threads could be used to implement ipc or process coordination. It is
also
interesting to see that a "non-productive" nsrpm service has just as
many
threads as nsrd (5) but nsrmmd only 3 (not a service). It would be
more
interesting if the applications would use multiple threads for
productive
work because then SMP-systems will be able to spread the workload.
You should also remember that multiple CPU's does not mean that the
application must be multithreaded, the system (device drivers, etc)
must
also be services while the networker processes are active. During
backup
there will be many processes that need time slices and it wouldn't
help if
nsrd takes it all.
Bye
Ernie
-----Original Message-----
From: Narkinsky, Brian [mailto:Brian.Narkinsky AT DEP.STATE.FL DOT US]
Sent: Friday, June 13, 2003 9:34 PM
To: NETWORKER AT LISTMAIL.TEMPLE DOT EDU
Subject: Re: [Networker] Random technical question - Single threaded
process on multi-processor server
Seems a bit weird that NSRD is single threaded. I am not at my
server but,
it seems to me that I have seen multiple threads on a W2K box. At
any rate
it loads up my 2 processor W2K machine just fine.
Brian
-----Original Message-----
From: Reed, Ted G II [ITS] [mailto:ted.reed AT MAIL.SPRINT DOT COM]
Sent: Fri 6/13/2003 1:24 PM
To: NETWORKER AT LISTMAIL.TEMPLE DOT EDU
Cc:
Subject: [Networker] Random technical question - Single threaded
process on
multi-processor server
Legato 6.1.3 on Solaris 8 (master + 2 SN):
I've been working on various and sundry means to make
our
hardware architecture stretch to meet our backup and recovery
needs.
One of the primary issues we have is a weak brain. In other
words,
we
do enough work at any given time slice that the master NSRD
process
is
maxing out an entire single 400MHz processor on a 4x400 SUN
e450.
Meanwhile, the other 3 cpus are doing practically nothing.
Our
storage
nodes still seem to have room to grow as I/O-data movers, but
each
extra
bit sent to a node increases the overhead on the master and
results
in
further queuing of jobs waiting to be processed by the maxed
out
NSRD
job.
We are investigating moving to a new Sun server
master
(v280/480) with 2x950MHz, but if I can take advantage of the
3
underutilized cpus, then we may not need to spend the
dollars. So
in
that vein, does anyone know of any wrappers/kernel mods/etc
that
allow
you to present a single "virtual" cpu to the process that is
back-ended
by multi thread cpu process?
My thought are: if I have a wrapper (MyWrap) that
accepts
cpu
calls and translates them into a multi-threaded call to all 4
CPUs,
then
I could up my overall cpu load towards the 100% mark. Even
if I
lost
50% to overhead, I'd still have an effective doubling of
process
capabilities for the single process. FYI, I'm thinking the
preference
would be towards a "nohup" style of wrapper. I.E. execution
would
be
"MyWrap -$switches -PID $PID" or "MyWrap -$switches nsrd &"
There's no reason I could think of that you couldn't
make it
into some form of kernel mod or something (to build that
'single'
cpu
box on multiple cpus to provide single-function server).
It's not
something that everybody would want (why waste a perfectly
good
multi-proc box like that?) but I'd take sharing 4-as-1 CPU
with the
whole system versus where I am today maxing out 1 and leaving
the
other
3 out in the cold.
Thank you all for your time and attention to my
flight of
fancy.
If anyone has any information concerning an actual product
that
meets
these basic requirements, I would LOVE to hear about it.
Thank you
again.
--Ted G. Reed II, Engineering Storage
Services
Pager: (888)510-4934 Phone:
(913)794-7826
--
Note: To sign off this list, send a "signoff networker"
command via
email
to listserv AT listmail.temple DOT edu or visit the list's Web
site
at
http://listmail.temple.edu/archives/networker.html where you
can
also view and post messages to the list.
=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=
|
