ADSM-L

Re: Monitoring ADSM for errors.

1997-06-04 21:56:51
Subject: Re: Monitoring ADSM for errors.
From: Paul Zarnowski <vkm AT CORNELLC.CIT.CORNELL DOT EDU>
Date: Wed, 4 Jun 1997 21:56:51 -0400
A few folks have requested copies of the Rexx scripts that I wrote to filter
activity log messages and send only the interesting items via email to the
ADSM administrator(s).  I have now made this available on our FTP server,
as well
as updated versions of other utilities that I had previously made
available.  The
activity log filter is at

ftp://ftp.cit.cornell.edu/pub/special/adsm/server/daily.actlog.* (3 files).

A brief description of this utility, as well as the others, is included below:

---
Abstract:
Abstract:

   These utilities have been developed to help automate the ADSM Server
   support function at Cornell.  I hope you will find them to be of some
   use.  There is currently not very good documentation on how to install
   and use these utilities.  I am hoping to correct this situation soon.  If
   you are at all familiar with the Rexx language, you should be able to use
   most of these utilities after looking at the execs to see how they work.
   Each utility generally has documentation at the beginning of it, followed
   by some parameters that you may want to set.

Contents:
   The rest of this document contains:
      Utilities         - list of utilities described
      Rexx functions    - useful Rexx functions
      Sample Output     - sample output from each utility
      Author Info       - Where you can reach me
      Legal Information - Terms & Conditions of use

Utilities:
   daily.actlog          - strip "noise" out of ADSM activity log; email to
you
   daily.backup          - notify ADSM users if they don't get backed up
   daily.tape            - check for tape related problems
   daily.usage.summary   - summarize ADSM usage for day (or month)
   daily.errpt           - send AIX error log for past 24 hour period via
email.
   accntdisp             - display ADSM accounting log in human readable
format.
   driveuse              - display usage of each tape drive.

Utilities (somewhat esoteric):
   generate.node.admins  - display Node administrators (contact field info)
   generate.node.info    - display node information
   generate.node.stats   - display node statistics
   monthly.archive.actlog - archive old activity logs (from daily.actlog)
   monthly.archive.backlog - archive old backlogs (from daily.backup)
   driveuse.email        - send email when tape drive needs cleaning.
   nodeadmin             - ADSM HelpDesk administration tool (samply only!)

Rexx functions used by above utilities:
   adsm.file             - send ADSM command output to a file.
   adsmadm               - issue ADSM admin commands.  You must set id & pswd.
   cronrexx              - calls rexx exec from cron
   daily.actlog.filter   - called by daily.actlog to filter out noise
   daily.actlog.get      - called by daily.actlog
   edate                 - date manipulation function
   download              - shell script to download file via ftp.
   monitor.tapedrives    - monitor tape drive usage (c.f., driveuse, et al)
   edate                 - Enhanced date manipulation function.
   daily.account         - Collect usage data (for accounting & reporting)
   generate.usage.data   - display usage data summary from ADSM accounting
log.
   generate.usedata.excel- build Excel file; data collected by daily.account
   newnode.email         - used by nodeadmin

Rexx function packages (can be used to build on):
   GetDomains.rxx        - Parse output from 'QUERY DOMAIN'    into Rexx stem.
   GetFilespaceInfo.rxx  - Parse output from 'QUERY FILESPACE' into Rexx stem.
   GetNodeInfo.rxx       - Parse output from 'QUERY NODE'      into Rexx stem.
   GetSchedules.rxx      - Parse output from 'QUERY SCHEDULE'  into Rexx stem.
   PutNCInfoV2.rxx       - Update node contact fields.
   asn1.rxx              - ASN.1 routines for parsing node contact fields.
   osd.rxx               - Miscellaneous operating-system dependent routines.

Sample Output:   daily.backup (e-mail to user):
--- start of sample ---
Output from ADSM Backup Monitor:
Output from ADSM Backup Monitor:

System 'XYZ.EZBACKUP' has NOT been backed up by the ADSM Backup Server
within the last 1 day.  You are listed as the administrator for this system,
and our records indicate that you would like to receive this notification
whenever this system is not backed up regularly.  If you want to change the
notification policy for this system, please send E-mail to
adsm-maint AT cornell DOT edu.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
-

System Name:  PEACH.EZBACKUP   (Mac)
Notification: Always notify.
The only volume for this system has NOT been backed up within the last 1 day:

      Volume name                                Last Backup Done     Days Ago
      -----------------------------------------  -------------------  --------
      Hard Drive                                 10/06/1995 01:33:54       3.7

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
-
--- end of sample ---
Sample Output:   daily.backup (e-mail to server administrator):
Sample Output:   daily.backup (e-mail to server administrator):
--- start of sample ---
Output from ADSM Backup Monitor:
Output from ADSM Backup Monitor:

Summary Information:

Nodes backed up and not notified: 139
Nodes backed up and notified:      25
Nodes not backed up recently:      15
Nodes not wanting notification:     2
Nodes locked:                       0
Nodes never contacted server:       6

= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =
= =
Short version of mail sent to John Doe (4-1234):
4 of the following computer systems have been successfully backed up recently,
and 5 of the following computer systems have NOT been backed up recently, by
the ADSM Backup Server.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
-
System Name:  CC.ARAP.EZBACKUP   (Mac)
Notification: Always notify.
The only volume for this system has NOT been backed up within the last 1 day:

      Volume name                                Last Backup Done     Days Ago
      -----------------------------------------  -------------------  --------
      ARAP-RP hd                                 05/10/1995 00:16:18     152.4
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
-
(etc...)
--- end of sample ---
Sample Output:   daily.tape (e-mail to server administrator)
Sample Output:   daily.tape (e-mail to server administrator)
--- start of sample ---
Output from ADSM maintenance script: "daily.tape".
Output from ADSM maintenance script: "daily.tape".

WARNING: The following 8MM Tape Volumes are in ERROR STATE:
         (AB0082)

WARNING: The following 8MM Tape Volumes have been mounted > 300 times:
         AB0024 - 349 mounts
         AB0026 - 302 mounts
         AB0035 - 301 mounts

WARNING: The following 8mm Tape Volumes have > 0 Read Errors:
         AB0058 - 1 Read Error(s)

WARNING: Only 5 good volumes are left in Storage Pool BKP.TAP.
         (AB0028, AB0069, AB0079, AB0085, AB0087)
         The target for this Storage Pool is 6 good volumes.
         There are 29 other volumes in this Storage Pool.
         (AB0003, AB0008, AB0012, AB0024, AB0026, AB0029, AB0031, AB0033)
         (AB0034, AB0035, AB0036, AB0039, AB0040, AB0046, AB0047, AB0057)
         (AB0058, AB0059, AB0061, AB0062, AB0064, AB0065, AB0067, AB0068)
         (AB0073, AB0080, AB0081, AB0082, AB0084)
         (these are full, r/o, failing or old).
--- end of sample ---
Sample Output:   daily.usage.summary (e-mail to server administrator)
Sample Output:   daily.usage.summary (e-mail to server administrator)
--- start of sample ---
ADSM Activity Report for 10/07/1995
ADSM Activity Report for 10/07/1995

Activity     # Sess  # Obj # Bytes       Time MediaW  CommW  IdleW    Xfer
Rate
------------ ------ ------ ------- ---------- ------ ------ ------
------------
------------
Backup         160  25475    1.3GB   77:57:58 0.099%  30.2%  25.6%    6.24
Backup         160  25475    1.3GB   77:57:58 0.099%  30.2%  25.6%    6.24
KB/s
Restore          0      0      0 B    0:00:00

Archive        262    267   79.4MB    0:13:25  22.7%  30.4%  15.5%  159.75
KB/s
Retrieve         5     44    8.2MB    0:33:18  41.2%   5.7%  19.7%   10.48
KB/s
Other           18      0  63000 B    0:48:41     0%     0%  31.9%    0.03
KB/s
Total          445  25786    1.6GB   79:33:22   0.4%  29.7%  25.6%    7.74
KB/s

Notes:
  - Time      = total wall-clock time
  - MediaW    = % of wall-clock time spent waiting for Media
  - CommW     = % of wall-clock time spent waiting for Network
  - IdleW     = % of wall-clock time spent idle (waiting for user)
  - Xfer Rate = Transfer Rate (Bytes / (Time * (1 - (MediaW + IdleW))))

# Bytes  % Tot Top 10 nodes for 10/07/1995:
------- ------ ----------------------------------------
223.2MB  13.6% ORANGE
223.2MB  13.6% ORANGE
195.6MB  11.9% BANANA.CISER.CORNELL.EDU
131.0MB   8.0% PLUM.CISER.CORNELL.EDU
120.4MB   7.3% CHERRY.CFE.CORNELL.EDU
100.7MB   6.1% GRAPE
 82.5MB   5.0% PEAR
 38.2MB   2.3% APPLE.CFE.CORNELL.EDU
 35.1MB   2.1% PEACH.OS2.EZBACKUP
 34.7MB   2.1% RASPBERRY
 27.4MB   1.7% QUINCE.SERVER4.EZBACKUP
 10.7MB Average, for 153 nodes.
--- end of sample ---
Sample Output:   accntdisp
Sample Output:   accntdisp
--- start of sample ---
$ tail dsmaccnt.log | ./accntdisp.dist
$ tail dsmaccnt.log | ./accntdisp.dist
03/25/1997 21:42:24 CL.BBTH0RP.E BK    54 (910.0KB)   1.6MB Tot  0:03:02 ET
03/25/1997 21:43:20 CL.OB3KE3T.E BK     2 (    0 B) 25000 B Tot  0:00:15 ET
03/25/1997 21:44:40 HFJSERVER.EZ BK     7 ( 22.9MB)  23.0MB Tot  0:18:15 ET
03/25/1997 21:47:22 CL.8NQWBJC.E BK    29 (119.0KB) 851.0KB Tot  0:10:46 ET
03/25/1997 21:47:56 TELGRAM.CIT. BK     6 (75000 B)   3.2MB Tot  0:03:36 ET
03/25/1997 21:48:15 TRIXY.CIT.CO BK    25 (421.0KB)   3.1MB Tot  0:03:58 ET
03/25/1997 21:48:41 GAEA.C(jk15)                     4000 B Tot  0:00:01 ET
03/25/1997 21:48:43 CL.WFHNBHP.E BK    44 (100.0KB) 225.0KB Tot  0:01:37 ET
03/25/1997 21:49:45 GAEA.C(jk15) AR     2 (  8.3MB)   8.3MB Tot  0:00:36 ET
03/25/1997 21:49:47 CL.YDJXHDM.E BK    43 (  1.8MB)   3.2MB Tot  0:02:08 ET
--- end of sample ---
Sample Output:   accntdisp
Sample Output:   accntdisp
--- start of sample ---
$ driveuse.dist
$ driveuse.dist
 Library    Drive      Type  Device      Usage     Status
 ---------- ---------- ----- ---------- ---------- ----------
 BREECE1    DRIVE2     DLT   /dev/mt2   521:20     IDLE
 BREECE1    DRIVE3     DLT   /dev/mt3   500:20     IN USE
 BREECE1    DRIVE4     DLT   /dev/mt4   837:50     IN USE
 LAGO1      DRIVE0     8MM   /dev/mt0     6:20     Not In Use
 LAGO1      DRIVE1     8MM   /dev/mt1     7:50     Not In Use
--- end of sample ---
Author Information:
Author Information:

   My time is limited, but I welcome feedback and will try to answer your
   questions as my time permits.  I can be reached at:

     Paul Zarnowski
     315 CCC
     Cornell University
     Ithaca, NY  14853-2601
     e-mail: psz1 AT cornell DOT edu
     FAX:    607-255-6523

Legal Information:

   Although copyrighted, this software is being licensed to you for your
   use free of charge.  However, ownership of and interest in this software
   shall remain with the author.  Use and distribution of this software is
   governed by the following terms.

   This software is owned by the author and contains valuable and
   proprietary information of the author.  If you violate any part of this
   agreement, your right to use this software terminates automatically.
   In the event of termination of this agreement, you must destroy all copies
   of this software and derivatives of this software in your possession and
   cease distributing the same.

   This software is being licensed to you as provided by the terms of this
   agreement.

   You may:
   1.  Use this software on as many computers as you want at any given time.
   2.  Make as many backup copies of this software as you want.
   3.  Alter the software in any manner you see fit FOR YOUR OWN PERSONAL
       USE.  Such altered versions should not be distributed.  The creation
       of such derivatives shall not diminish the author's title to this
       software.
   4.  Terminate this agreement at any time by destroying all copies of this
       software and derivatives of this software and cease distributing the
       same.

   You may not:
   1.  Create any derivative works from this software for distribution.
   2.  Re-distribute this software for commercial (for-profit) purposes.

   Contact the author at the address within this document if you wish to
   distribute this software for commercial usage, or if you have any
   questions about its redistribution.

   Disclaimer of warranty:
   In using this software, you understand and agree that this software is
   provided "as is" without warranty of any kind.  The entire risk as to the
   results of and performance of using this software lies entirely with you,
   the user.  The author does not make any warranties, either expressed or
   implied, including but not limited to implied warranties of merchantability
   and fitness for a particular purpose, with respect to this software.

   In no event shall the author be liable for any consequential, incidental,
   or special damages whatsoever arising out of the use or inability to use
   this software.
<Prev in Thread] Current Thread [Next in Thread>
  • Re: Monitoring ADSM for errors., Paul Zarnowski <=