*********************************************************************** ***** ADSM performance tuning in heterogeneous environments ******** ***** HOMA 09/97 ADSM Support Mainz ******** *********************************************************************** * last change: * HOMA/nov97 -- NetWare added * HOMA/jan98 -- SLOWINCREMENTAL/MEMORYEFFICIENTBACKUP added ADSM V3 added * HOMA 01/21/98 -- formal changes * HOMA 01/28/98 -- tcp/ip tuning ADSM OS/2 client added/changed * HOMA 03/11/98 -- tcpnodelay changed to NO; LARGECOMmbuffers added * HOMA 03/26/98 -- logpoolsize increase * HOMA 04/06/98 -- IX67978 added * HOMA 04/17/98 -- changed description of server's uselargebuffers option * HOMA 06/08/98 -- added IX73502, IX76643, IX73915 * HOMA 06/09/98 -- added "General Notes on AIX": Striping, JFS/RAW devices * and physical partition size * HOMA 06/15/98 -- changed recommendation MIRRORWrite LOG/DB to SEQUENTIAL * HOMA 06/25/98 -- added the Win 32Bit Client * HOMA 06/26/98 -- added ADSM Server V3 TCPBufsize * HOMA 07/01/98 -- corrected typo in TXNGROUPMAX * HOMA 08/17/98 -- added AIX vmtune SPECIAL NOTICES Information in this document is correct to the best of our knowledge at the time of this writing. Please use this information with care. IBM will not be responsible for damages of any kind resulting from its use. The use of this information is the sole responsibility of the customer and depends on the customer's ability to eval- uate and integrate this information into the customer's operational environment. ABOUT THIS DOCUMENT This document contains tips for performance tuning in heterogeneous ADSM environments. ADSM AIX Server 2.1.x.15 / 3.1.0.1 or newer service levels: ----------------------------------------------------------- BUFPoolsize 8192 /* !!! ATTENTION !!! */ /* <=2048 for 2.1.5.12 servers only */ /* see APAR IX65944 */ /* Setting this option to 8192 will */ /* reduce the total amount of available*/ /* RAM in the system by 8MB. Be shure */ /* that this will not increase system */ /* over all performance. */ /* Increase this value if QUERY DB F=D */ /* shows a cache hit percentage below */ /* 99% */ LOGPoolsize 2048 /* or less if you have less memory */ EXPINterval 24 /* or 0 if scheduled by command */ /* see EXPIRE INVENTORY command */ TCPWINDOWSIZE 64 /* needs to be discussed */ /* Value in KB! */ TCPBufsize 32 /* ADSM V3 only */ TCPNODELAY NO /* Network load? YES for FDDI, ATM, */ /* High Performance Switch or other */ /* fast networks. */ TXNGROUPMAX 256 /* 40 if only disk STG volumes */ MOVESIZETHRESH 500 MOVEBatchsize 1000 /* !!! ATTENTION !!! */ /* <= 100 if running an ADSM */ /* AIX server < PTF L10 */ /* see APAR IX60263/IC14461 */ /* Documentation incorrectly states */ /* the max value is 256. */ USELARGebuffers YES /* !!! ATTENTION !!! See client and */ /* server readme files for */ /* limitations / restrictions. */ MIRRORWrite DB Parallel /* !!! WARNING !!! */ MIRRORWrite LOG Parallel /* Setting ADSM Server write */ /* operations to parallel improves */ /* performance but may cause data */ /* integrity problems in case of */ /* a system outage during DB / LOG */ /* write operations. */ /* As a result it's possible that */ /* the ADSM server cannot be */ /* restarted. */ /* We recommend setting both */ /* MIRRORWrite options to SEQUENTIAL! */ /* Discuss this item with your IBM */ /* ADSM service representitive */ /* before setting the ADSM server */ /* write operations to parallel. */ AIX 'no' settings (client and server): -------------------------------------- rfc1323 = 1 /* if TCPWINDOWSIZE > 64KB */ sb_max = 1310720 /* min. (2 * TCPWINDOWSIZE) in byte */ thewall = 16834 tcp_mssdflt = 1448 /* or more if allowed by gateway... */ subnetsarelocal = 1 AIX adapter transmit queue size (client and server): ---------------------------------------------------- At the device driver layer, both the transmit and receive queues are configurable. It is possible to overrun these queues. To determine this use the 'netstat -v' command (example AIX 4.2.1): Transmit Statistics: -------------------- Max Packets on S/W Transmit Queue: 28 S/W Transmit Queue Overflow: 0 Current S/W+H/W Transmit Queue Length: 0 If the value for S/W Transmit Queue Overflow is greater than 0 the Transmit Queue size needs to be increase. Recommended values: AIX 4.x: Transmit queue size = 80 /* or more */ AIX 3.x: Adapter transmit queue = 80 /* or more */ Adapter receive queue = 80 /* or more */ ADSM AIX Client: ---------------- TXNBYTE 25600 TCPBUFFSIZE 32 TCPWINDOWSIZE 64 TCPNODELAY NO /* see server */ USELARGebuffers YES /* ADSM V2 */ LARGECOMmbuffers YES /* ADSM V3 */ /* !!! WARNING !!! ADSM V2 ONLY */ /* Set this option to NO for ADSM V2 */ /* Clients at PTF L6 and L7. */ /* !!! ATTENTION !!! See client and */ /* server readme files for */ /* limitations / restrictions. */ /* Known problems caused by setting */ /* USELARGEBUFFERS YES were fixed in */ /* the ADSM V2 L8 Client PTFs and do */ /* not affect ADSM V3 clients. */ ADSM Windows/NT Client (32Bit): ------------------------------- TXNBYTE 25600 TCPBUFFSIZE 32 TCPWINDOWSIZE 64 TCPNODELAY NO /* see server */ ADSM OS/2 Client: ----------------- Same as the AIX Client settings plus LAPS/MPTS configuration of TCPBUFFSIZE 31 /* ADSM V3 client only!!! We recommend */ /* setting 31 because of a problem we */ /* figured out during system test but */ /* don't know the cause of the problem */ /* yet. */ PROTOCOL.INI: /* mostly c:\ibmcom\protocol.ini */ maxtransmits 30 xmitbufs 6 xmitbufsize 1536 /* for Ethernet only; see MTU Size! */ 4440 /* for Token Ring only; see MTU Size! */ receivebufs 6-12 receivebufsize /* needs to be calculated */ --> 1. Get the following vaules: Shared RAM value for the adapter (TOTRAM) Used RAM without receives (USEDRAM) Transmit Buffer Size (xmitbufsize) Number of transmit buffers (xmitbufs) Number of receive buffers (receivebufs) Maximum value allowed for the receive buffer size parameter (MAXBUFSIZE) Note: The variable names in lower case are real parameter names which can be found elsewhere in this section. 2. Add the total size of all the transmit buffers to the used RAM value: USEDRAM = USEDRAM + (xmitbufsize * xmitbufs) 3. Calculate the receive RAM (RECVRAM) available: RECVRAM = TOTRAM - USEDRAM 4. Determine the maximum receive buffer size (RECVBUFSIZE) a. Divide receive RAM by the number of receive buffers (RECVRAM / receivebufs) b. RECVBUFSIZE = the largest integer divisible by 8 that is less than the result of step a. above. 5. Determine the receive buffer size (receivebufsize): If (RECVBUFSIZE <= MAXBUFSIZE) then (receivebufsize = RECVBUFSIZE) If (RECVBUFSIZE > MAXBUFSIZE) then (receivebufsize = MAXBUFSIZE) 6. Determine whether there is enough receive RAM (RECVRAM): If (receivebufsize * RECVNUM) >= (2 * xmitbufsize) then go to step 7. If (xmitbufs = 1), reduce xmitbufsize and go to step 2. If (xmitbufs > 1), reduce xmitbufs by 1 and go to step 2. 7. Done: receive buffer size = receivebufsize SETUP.CMD: /* mostly c:\mptn\bin\setup.cmd */ /* TCP/IP setup command file */ ifconfig netmask mtu /* mtusize = 1492 for Ethernet */ /* mtusize = 4400 for Token Ring */ ADSM NetWare Client: -------------------- TXNBYTE 25600 TCPBUFFSIZE 32 TCPWINDOWSIZE 64 /* min. 32 */ TCPNODELAY NO /* see server */ PROCESSORUTILIZATION 1 /* or higher depending on what else is */ /* running on the same NetWare Server. */ SLOWINCREMENTAL YES /* (ADSM V2) set to NO if... */ MEMORYEFFICIENTBACKUP YES /* (ADSM V3) set to NO if... */ /* NetWare 3.x: */ /* 25MB < 150 bytes * # files in system */ /* NetWare 4.x: */ /* RAM < 150 bytes * # files in system */ /* RAM = total available RAM */ STARTUP.NCF or AUTOEXEC.NCF: set minimum packet receive buffers 100 set maximum physical receive packet size 4202 sys:\etc\tcpip.cfg: TcpMSSinternetlimit off /* if going through a router or gateway */ Note: The ADSM NetWare Client is only running on Processor ONE of the NetWare Server if NetWare is running on a multi processor maschine. T/R Adapter MTU Size (client and server, all platforms): -------------------------------------------------------- If the receive buffer size in the adapter can be configured to 4040 then the 4000 MTU size is recommended. However, if the receive buffer size in the adapter cannot be made larger than 2040 bytes, then a packet size of 2000 bytes should be used to avoid the overhead of fragmentation by the adapter on receipt of incoming packets. In AIX the MTU size can be queried by 'netstat -i' and changed by 'ifconfig'. General Notes on AIX ==================== Striping -------- Our performance lab found that striping has not much effect on ADSM Our performance lab found that striping has not much effect on ADSM I/O performance because the ADSM Server I/Os are random and the read / writes occur in small increments (4-6KB I/Os). JFS / Raw Devices ----------------- We recommend using JFS for ADSM Server DB, LOG and STG volumes although We recommend using JFS for ADSM Server DB, LOG and STG volumes although RAW devices may give you some better performance. JFS provides it's own file caching in addition to ADSM Server DB / LOG caching better data integrity (e.g. in case of electrical power loss...) better data security more comfortable file handling (e.g. in case of ADSM Server disaster recovery) Physical Partition Size ----------------------- We tested with different physical partition sizes up to 64MB. We tested with different physical partition sizes up to 64MB. ADSM performance increased with increasing the physical partition size but this performance increasement is not worth reorganizing the ADSM Server's DB, LOG and STG volumes. Tune disk read ahead using 'vmtune' ----------------------------------- The 'vmtune' executable can be found in following locations: The 'vmtune' executable can be found in following locations: AIX 4.x - /usr/samples/kernel/vmtune AIX 3.x - /usr/lpp/bos/samples/vmtune Server: vmtune -R 64 -F 184 -c 1 Client: vmtune -R 256 -F 376 -c 1 Server and client running on same machine: use client values. VMTUNE settings are valid until next reboot of AIX, so put these into /etc/inittab. Performance APAR(s) =================== Please enshure that following APAR(S) are applied to your system. AIX 4.1.x --------- IX73502 - OPTIMIZE COPYIN/OUT FOR BIG-DATA THREADED PROGRAMS IX73502 - OPTIMIZE COPYIN/OUT FOR BIG-DATA THREADED PROGRAMS AIX 4.2.1 --------- IX67978 - O_SYNC WRITES ARE SLOW IN AIX 4.2.1 IX67978 - O_SYNC WRITES ARE SLOW IN AIX 4.2.1 IX76643 - LOCKING OF BIG DATA SEGMENTS IN THREADED APPS (includes IX73502) AIX 4.3.0 --------- Development dropped service. No fixes available. Development dropped service. No fixes available. AIX 4.3.1 --------- IX73915 - LOCKING OF BIG DATA SEGMENTS IN THREADED APPS (includes IX73915 - LOCKING OF BIG DATA SEGMENTS IN THREADED APPS (includes IX73502) ============================= End of FAX ====================================  ------_=_NextPart_000_01BEBE68.49990496-- =======================================================================