ADSM-L

Re[4]: Netware 4.11 backup taking extremely long

1997-04-21 10:05:00
Subject: Re[4]: Netware 4.11 backup taking extremely long
From: Bill Colwell <bcolwell AT DRAPER DOT COM>
Date: Mon, 21 Apr 1997 10:05:00 -0400
The values range from 4 to 256, default=16, for txnmaxgroup.  The
max value for txnbytelimit is 25600.  This is set in the client.

If you have a test server, put txnmaxgroup in the options and start
it.  If it complains about an invalid parameter then it isn't
supported.  I am attaching an apar from IBMLink which documents the
parameters.

Bill Colwell
The Charles Stark Draper Laboratory
Cambridge Ma.

- - apar pn67795 - - -

  APAR Identifier ...... PN67795      Last Changed ........ 95/03/14
  NEW SERVER OPTIONS FILE PARAMETER -TXNGROUPMAX- CONTROLS NUMBER
  OF FILES PER CLIENT/SERVER TRANSACTION

  Symptom ...... NF NEWFUNCTION       Status ........... CLOSED  DOC
  Severity ................... 4      Date Closed ......... 95/03/14
  Component .......... 564802001      Duplicate of ........
  Reported Release ......... 110      Fixed Release ............
  Component Name ADSM MVS SERVER      Special Notice
  Current Target Date ..95/05/08      Flags
  SCP ...................
  Platform ............

  Status Detail: ASSIGNMENT - APAR has been assigned to a
                              programmer.

  PE PTF List:

  PTF List:


  Parent APAR:
  Child APAR list:


  ERROR DESCRIPTION:
  The TXNGROUPMAX parameter in the server options file is document
  documented in this APAR. This is a performance improvement item.
  The parameter specifies how many files can be sent to the server
  in one transaction.


  LOCAL FIX:


  PROBLEM SUMMARY:
  ****************************************************************
  * USERS AFFECTED:                                              *
  All ADSM Rel1 and Rel2 users.
  ****************************************************************
  * PROBLEM DESCRIPTION:                                         *
  This is to doucument the TXNGROUPMAX parm in the server options
  file. Performance can be improved with the use of this parm.
  The parameter specifies how many files can be sent to the server
  in one transaction.
  ****************************************************************
  * RECOMMENDATION:                                              *
  none
  ****************************************************************
  To document the TXNGROUPMAX parameter in the server options
  file.


  PROBLEM CONCLUSION:
  The following will be added to the following manual:
  Installing the Server and Administrative Client

   TXNGROUPMAX - A NEW ADSM SERVER OPTIONS FILE PARAMETER

   TXNGROUPMAX is a parameter that can be specified
   in the server options file to control the number
   of files that will be transferred as a group
   between a client and a server between transaction
   commit points.  You may be able to improve the
   performance of these commands by using a large
   value for TXNGROUPMAX.

   TXNGROUPMAX applies to BACKUP, ARCHIVE, RESTORE,
   and RETRIEVE commands.  The files transferred are
   for these commands are either actual files or
   directories or both. Each actual file and each
   directory is counted as one file.

      Syntax:   TXNGroupmax <numfiles>

                where,

                <numfiles> specifies the maximum number
                           of files per transaction.

                The default value is 16 files.
                The minimum value is 4 files.
                The maximum value is 256 files.

   NOTE: To obtain a performance improvement on
   RESTORE and RETRIEVE commands, a client PTF is
   also needed.  The client PTF numbers are listed
   below.

   The reader is also referred to an associated
   parameter, TXNBYTELIMIT, in the CLIENT options
   file.  TXNBYTELIMIT controls the number of bytes,
   as opposed to files, that will be transferred in a
   group of files between transaction commit points.
   At the completion of transferring a file, the
   client commits the transaction if the number of
   bytes transferred during the transaction reaches
   or exceeds the value of TXNBYTELIMIT, regardless
   of the number of files transferred.

   The TXNBYTELIMIT parameter is implemented in the
   following client PTF's:

      IP20258   AIX/6000
      IP20261   DOS
      IP20262   OS/2 (2.0)
      IP20263   OS/2 (1.3)
      IP20264   Windows


      IP20265   Novell
      IP20266   HP
      IP20267   DEC
      IP20268   SCO
      IP20269   SUN (disk)
      IP20270   SUN (tape)

   ---- Diagnosis, Modification or Tuning Information ----

   When files are backed up or archived, a group of
   files is transferred as a single transaction, con-
   trolled by the value of the TXNGROUPMAX parameter.
   The client internally issues a command to the
   server (similar to the COMMIT command) that causes
   the database to be permanently updated with all
   changes that result from the successful completion
   of the transaction. Refer to the COMMIT command
   for controlling committing of commands in a macro
   in the Administrator's Reference for more dis-
   cussion.

   The transaction commit causes all buffered file
   data to be flushed to storage pool media.  There-
   fore, performance improvements should be greatest
   when files are backed up or archived to storage
   pools with a device class for 8mm tape drives
   (DEVTYPE = 8MM).  Significant performance improve-
   ments are also likely when files are backed up or
   archived to storage pools with a device class for
   IBM 3480 and IBM 3490 tape drives (DEVTYPE = CAR-
   TRIDGE).

   Be aware of the the following considerations when
   specifying a large value for TXNGROUPMAX:

   1.  Utilization of the recovery log will normally
       increase because more database updates need to
       be logged for each transaction.  The amount of
       recovery log space needed is primarily a func-
       tion of the value of TXNGROUPMAX and the
       number of client nodes that operate at the
       same time. After increasing TXNGROUPMAX,
       monitor the utilization of the recovery log to
       determine if the log should be extended.

   2.  If the transmission of a group of files fails,
       ADSM can rollback and retry. The time required
       to rollback and retry the transaction is gen-
       erally longer when the value of TXNGROUPMAX is
       large.

   3.  When using client compression it is possible
       for certain types of files (such as previously
       compressed files) to actually grow in size. In
       this case the transaction will be aborted and


       retried but without compressing the file.

   -- End of Diagnosis, Modification or Tuning Information --

_________________________Reply Header_________________________
Author: ADSM-L AT vm.marist DOT edu
Subject: Re: Re[2]: Netware 4.11 backup taking extremely long
04-21-1997 11:24 AM

I did a search through the ANR1110 member and it doesn't mention
anything about TXNGROUP or TXNBYTE.

Dianne Sharp
Operations Analyst
ISS CUSTOMER SUPPORT
Ext 4208

>----------
>From:  Bill Colwell[SMTP:bcolwell AT DRAPER DOT COM]
>Sent:  Friday, 18 April, 1997 1:32AM
>To:    ADSM-L AT VM.MARIST DOT EDU
>Subject:       Re[2]: Netware 4.11 backup taking extremely long
>
>I believe they are implemented in the recent ptfs for version 1.  Check the
>samplib member ANR1110 for documentation.
>
>Bill Colwell
>The Charles Stark Draper Laboratory
>Cambridge Ma.
>
>_________________________Reply Header_________________________
>Author: ADSM-L AT vm.marist DOT edu
>Subject: Re: Netware 4.11 backup taking extremely long
>04-17-1997 09:52 AM
>
>Are the TXNGROUPMAX and TXNBYTELIMIT options valid for ADSM Server for
>MVS V1?  I can't find anything about them in the manuals.  If valid, how
>do you know what value to choose?
>
>Dianne Sharp
>Operations Analyst
>ISS CUSTOMER SUPPORT
>Ext 4208
>
>>----------
>>From:  Elvin Peterson[SMTP:EPeterso AT SCJ DOT COM]
>>Sent:  Thursday, 17 April, 1997 7:27AM
>>To:    ADSM-L AT VM.MARIST DOT EDU
>>Subject:       Re: Netware 4.11 backup taking extremely long
>>
>>On the server, take a look at you Cache Hit Pct, if its below 97%, I
>>would increase the BUFPOOLSIZE.  Also investigage the TXNGROUPMAX
>>option.  Also, are you going to disk or directly to tape?  Are there
>>any other ADSM processes running, like migration or reclamation?
>>
>>On the client side, determine if you have ADSM compression on and
>>evaluate whether it's buying you anything (If you have Netware
>>compression turned on, you probably don't need ADSM compression).  The
>>TXNBYTELIMT option (which goes along with the TXNGROUPMAX option on the
>>server) may help your throughput also. Especially if you have a lot of
>>small files.
>>
>>
>>----------
>>From: ADSM-L(a)VM.MARIST.EDU; Naveed(u)Mustafa(a)MCKINSEY.COM
>>To: ADSM-L(a)VM.MARIST.EDU
>>Subject: Netware 4.11 backup taking extremely long
>>Date: Wednesday, April 16, 1997 1:51PM
>>
>>Hi there,
>>
>>We started to backup our Pentium 100MHz with 192MB memory Netware 4.11
>>server to ADSM last night at 8:00 P.M.  The estimated data on the server is
>>about 40GB and so far only 15GB of data has been backed up in 19 hours.
>>The ADSM server is at version 2.1.0.12 and is running on an AIX 3.2.5
>>machine.
>>
>>Any suggestion as to what parameters need to be fixed on either the server
>>or netware's dsm.opt file?
>>
>>Thanks for any suggestion and comments,
>>Naveed.
>>
>