Amanda-Users

Re: system load

2007-07-31 14:28:19
Subject: Re: system load
From: Brian Cuttler <brian AT wadsworth DOT org>
To: Chris Hoogendyk <hoogendyk AT bio.umass DOT edu>
Date: Tue, 31 Jul 2007 14:13:09 -0400
Chris,

On Tue, Jul 31, 2007 at 01:36:49PM -0400, Chris Hoogendyk wrote:
> So, you have sendmail and lotus notes running on it. Does the other 
> lotus notes server load balance with this one? Or do they serve 
> different groups? Does it spawn a mess of processes the way sendmail 
> does? The load that is reported by uptime is the average number of jobs 
> in the run queue over the last minute, five minutes and ten minutes. I 
> think that means jobs that are just twiddling their thumbs waiting to 
> get the cpu.

Good questions, let me try to answer each in turn.

The system is a LNotes server, the sendmail is incidental, low
activity/background - this is not the smtp mailhub, that is another
system completely.

We have completed dumps, though we have not yet completed taping. Current
load average is under 10.

  1:46pm  up 23:40,  2 users,  load average: 9.61, 9.25, 9.14

No, I was wrong, there is still one dump in progress, big change
from three though.

I'm uncertain how the load average is calculated, is it all runnable
tasks, what counts as runable Solaris 9 (yes, that is the version),
does it include blocked wait, sleeping, what ?

Because these are now largely tar dumps, we left indexing on, while
that itself is a low overhead, and I'm certain the compressesion of
the index is not a drain (since there is relatively little data in
an index file) the uptime numbers are likely not indicative of throughput
for CPU bound tasks, and tar will be largely I/O bound rather tha CPU
bound since we also switched to HW compression for the actually data
going to tape.

> Amanda ought not to be cpu intensive. But I suppose if you are doing 
> software compression, running the local backup, it could chew a bit.

Amanda itself ? No, largely a scheduler. TAR though... even without
compression must use its share of CPU.

> Are you Solaris 9? The older Solaris versions are notably slower than 
> Solaris 9. I haven't jumped to 10 yet because there is so much change. 
> Going to 9 was easy and a significant payoff in speed.
> 
> How is the hardware configured on your E250? They have a pretty high 
> capacity for throughput, but if you're running multiple dumpers on 
> different partitions of the same internal drive and then putting holding 
> space on another partition and going to an internal tape drive, I can 
> see how you might tie things up so that other processes can't get a 
> transfer in edgewise. On my E250, I've configured a couple of totally 
> separate 10Krpm Seagate Cheetahs for holding disks. I've also added a 
> PCI LV320 SCSI card to connect my tape drive, so that i/o is actually on 
> a separate bus (it's in the 66 pci slot, not any of the 33's -- all the 
> 33's share one bus). I also have a 4-way 10/100 PCI ethernet card, but I 
> haven't configured that yet. It won't do me any good until we have 
> gigabit running between the switch rooms. Then the E250 could do 
> trunking and blend 4x100 into one pipe.

Only a couple of drives are local, boot disk, amanda work area, the
larger drives are now fiber connected to a RAID via a switch (preventing
accidental cross mounting of the RAID partitions with other servers).
The tape is also local, but on its own bus.

I don't recall how much memory and dmesg scrolled it off.

swapfile             dev  swaplo blocks   free
/dev/md/dsk/d20     85,20     16 1845744 1845744

I'm not completely certain that I have a "problem" beyond the fact
that sendmail, transmitting local mail, does not like to run at
high load average/process count, and I can 'fix' that fairly easily
by setting a variable in the /etc/mail/sendmail.cf

Sendmail is just what brought the system load to my attention, its
not critical at this point and almost no mail is handled by sendmail

However the idea of reducing the number of dumpers, after a night's
run, to some lower number, prior to start-of-business, might have some
value, but I don't know that I'm having a critical issue. Just 
brain storming.

> Memory is another issue. Dual CPUs (what speed are yours?), with 2G 
> memory. Swap space? I put 2G on each of the drives that is at or over 
> 10Krpm.
> 
> Have you tried running top? You can get it from sunfreeware. It can put 
> a little extra demand on the server, but I sometimes do it for just a 
> minute or so to give me a frame of reference for what the top processes 
> are in cpu usage. On my department server, the few incidents where smtp 
> refused connections, I found all the top processes were mimedefang. That 
> told me pretty definitively what was going on. From there I could look 
> at /var/adm/messages, mail.log, and uw-imap.log to confirm the level of 
> activity and the sources. Firewalled a couple of IP subnets on the pac 
> rim and all was happy again.
> 
> Also, after the backups have run and you have the reports, you could 
> look them over carefully and see if there were significant slowups on 
> the DLE's running after some particular time in the morning.

That is an idea, will do so when this completes, though I don't know
if there is a simple way to tell what order DLEs where actually 
dumped in. I also expect to see some difference between the local
root drive and the large partition remote RAID drives, though I'm not
sure what I would expect that difference to be.

                                                thank you,

                                                Brian


> ---------------
> 
> Chris Hoogendyk
> 
> -
>   O__  ---- Systems Administrator
>  c/ /'_ --- Biology & Geology Departments
> (*) \(*) -- 140 Morrill Science Center
> ~~~~~~~~~~ - University of Massachusetts, Amherst 
> 
> <hoogendyk AT bio.umass DOT edu>
> 
> --------------- 
> 
> Erd�s 4
> 
> 
---
   Brian R Cuttler                 brian.cuttler AT wadsworth DOT org
   Computer Systems Support        (v) 518 486-1697
   Wadsworth Center                (f) 518 473-6384
   NYS Department of Health        Help Desk 518 473-0773



IMPORTANT NOTICE: This e-mail and any attachments may contain
confidential or sensitive information which is, or may be, legally
privileged or otherwise protected by law from further disclosure.  It
is intended only for the addressee.  If you received this in error or
from someone who was not authorized to send it to you, please do not
distribute, copy or use it or any attachments.  Please notify the
sender immediately by reply e-mail and delete this from your
system. Thank you for your cooperation.



<Prev in Thread] Current Thread [Next in Thread>