Networker

Re: [Networker] 7.0 Experience

2003-07-07 02:22:04
Subject: Re: [Networker] 7.0 Experience
From: George Scott <George.Scott AT ITS.MONASH.EDU DOT AU>
To: NETWORKER AT LISTMAIL.TEMPLE DOT EDU
Date: Mon, 7 Jul 2003 16:13:27 +1000
Ted,

>> The end result is that the old NetWorker will not scale well.  It is
>> fine as a departmental backup system, but when you try to push it for
>> use as an enterprise backup system it will hit a performance wall.  The
>> obvious solution to this would be to use faster CPU's in your server.
>> Unfortunately this is a limited solution; there is a limit to how far
>> you can overclock a 3 GHz CPU.
>> In the new implementation the resource database is preserved on disk as
>> a large number of very small files, one per resource.  Every time an
>> attribute is changed only the one resource file is written out.  Since
>> these files are so small there is a lot less work involved in updating
>> them.  This effectively solves this problem for quite some time.
>
> How many clients were you running that a 3GHz processor had issues?  I have 
> 500 on a 4x400MHz Sun master with two e4500 storage nodes.  In that 
> environment, one cpu is pegged about 22 hours a day and nsrd is VERY much our 
> performance bottleneck.  Do you have any thoughts about how much (percentage) 
> the load dropped after moving to 7.x and the more efficient DB style?

My setup is relatively small (lets say "large department") so I don't
often run into CPU limits with nsrd.

I was trying to point out that you can't scale a NetWorker server just
by buying bigger iron.  That works fine up to a point, but then you
discover that bigger machines are just not available no matter what
your budget.  There are deficiencies in the current implementation that
can only be fixed by redesigning parts of NetWorker.  NSR7 is a step
along that path.

Make sure your server has enough memory (ie, file system cache).  The
OS will probably chew CPU to compensate.  Realistically it is quite
unlikely that you are short on memory - you would be seeing other
problems too.  Your nsr.res and nsrjb.res are probably somewhere around
1 MB which should fit easily.

>> For those that are curious: in NSR7 the resources are still stored in
>> text form, just like in the old nsr.res, but one resource per file.
>> When you first start it the resource database is automatically
>> converted to the new format.  There are no tools provided to convert
>> back.  This is why this upgrade is stated to be one-way only.  [text removed]
>
> I remember the 5.x to 6.x conversion being painfully long, particularly in 
> the client data updates on the master (nsr.res and indexes).  What was your 
> feel for how long it took to convert X number of clients?

My NSR7 data zone is a new one, so I didn't have a conversion step.

I would expect the conversion to be very quick and painless.  You end
up with one small file per resource.  A reasonable server should be
able to convert hundreds of resources per second, so something like
(X+150)/100 elapsed seconds.

George.
--
George Scott           George.Scott AT its.monash DOT edu
Systems Programmer, IT Services, Monash University

--
Note: To sign off this list, send a "signoff networker" command via email
to listserv AT listmail.temple DOT edu or visit the list's Web site at
http://listmail.temple.edu/archives/networker.html where you can
also view and post messages to the list.
=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=

<Prev in Thread] Current Thread [Next in Thread>