Healthcheck 07 application
Download as ODT, PDF0 likes493 views
The document requests various documentation and data from an organization running WebSphere Application Server 6.1, including collector tool output, verbose GC traces, service level agreements, application design documents, logs, configurations, concerns, and a questionnaire. The organization notes problems with hangs in WAS 6.1 response times and irregular hangs associated with the AIO included in WAS 6.1.
Healthcheck 07 application
- 1. WebSphere Application Server Repeat per Application Deployed to Cluster. Most documentation is available at http://www-306.ibm.com/software/webservers/appserv/was/library Documentation / Data requested 1. WebSphere Collector Tool Output 2. Verbose GC Trace 24 hours 3. Service Level Agreement and expected application request arrival rate. (If SLA is not defined pls describe the acceptable quality of service expectations) 4. Application design document 5. http access logs 6. http config 7. http was plug-in config 8. If applicable: Framework design document 9. Naming Conventions, Port Allocation Table 10. If applicable: was there ever executed an sizing by IBM, if yes please provide the in/output, if no please fill out W3 11. If applicable: did you run workload injection/simulation? Pls provide the reports 12. If available: Application profile of JINSIGHT (IBM internal) trace 13. If applicable: WAS capacity planning 14. z/OS only: WLM policy 15. z/OS only: RMF WLMGL and LPAR reports Concerns Please describe any specific concerns with your WebSphere installation. Problems with 6.1. In addition there seems to be some problems with the AIO included in the WAS 6.1. Currently the installation shows unregulary hangs in the response times. Questionnaire Please answer accordingly. W1: WebSphere Application Server General Q# 1. 2. Question WAS Version & Service Level SDK level, build 3. Local Subsystem level WAS interacts with 4. Interaction with other J2EE servers: J2EE certified? Protocols used for interaction? WAS configuration 5. 6. 7. # of application clones Heap Sizes Answer 6.1.0.2 Java 1.5 IBM J9 VM build 2.3 S390x-64 JIT enabled DB2: Version 8.2 UDB CICS: --IMS: --MQ: --DB2 JDBC Driver Type 4 on the same Linux image. WebSphere ND – Deployment Manager – Node Agent – 1 Application Server 1 Recommendation / Comment
- 2. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 512 MB Reload enabled for Energy Exchange, interval 10 1. http://www1.ibm.com/support/docview.wss? uid=swg1PQ92267 8. DM Controller Appl. Server # of WASs(cells) within one OS image Number of applications per WAS Are any traces on? Tracing Level? Is PMI in normal mode? PMI enabled? z/OS: is SMF 120 enabled? How? Do you think that your responsible staff understand WAS concepts & admin? Do you know the artifacts that are the biggest resource consumers? Do you have WAS monitors and alert triggers in place? protocol_iiop_daemon _listenIPAddress in Daemon WAS.ENV hostname written in Servlet Reload disabled? Sure??? Reload enabled for Supersnoop, interval 10 2. 512 MB 1 1 No Yes No Good question No ??? File not available Till WAS6, needs to be in Upper Case 3. At the administrative console, click on Applications > Enterprise Applications. For each application installed, select Reload Enabled, and set the Reload Enabled value to 0. Save and synchronize the changes. Alternatively, manually edit the ibm-webext.xmi file; set reloadInterval="0" and reloadingEnabled="true". z/OS Apply APAR PQ92267 on PTF UQ93769 to resolve this problem 19. Alarm Manager in quiet mode? http://www1.ibm.com/support/docview.wss? uid=swg21192890 20. 21. Do you have uninstalled all applications that are not needed? Remote GC disabled? com.ibm.ejs.am. mode.workbased=____ (true/false) Noch aktuell??? Partly, the „default application“ is still installed. All applications that are not needed should be uninstalled. Yes The verbose garbace collection should only be enabled for debugging or testing purposes. Though the overhead is relatively small this option in not needed for normal production systems. Not checked with PMI but should be 10k per session. The Servlet Sessions PMI module will provide the average, maximum and minimum HTTP session sizes. These sizes are calculated by HTTP Sessions 22. HTTP Session Size
- 3. serializing the session object and hence all the referenced objects must be serializable. However, since you are doing memory-tomemory replication they should be serializable in your case. Turn on the PMI and check for the sizes using the Tivoli performance viewer (if you are using WAS 6.0, it is bundled within the application console). 23. 24. 25. 26. 27. 28. 29. 30. Is DRS enabled for HTTP sessions? Are HTTP sessions stored in a DB? If stored in DB, is DB connected using a local TYPE2 driver? If HTTP sessions in DB, is this DB the same DB than used by the application? If HTTP sessions in DB, how often are sessions written to the DB Replication/ Persistance mode: EOS, TimeBaseWrite Session Affinity in place and tested? HTTP session complexity 31. Deployment Options 32. ND install? If yes: 33. ND what is your sync setting? No No --- --- No DB persistence better than replication, EOS is very expensive No automatic synchronization. (-> Node Agents -> File Synchronization) XD installation W2: WebSphere Application Server: Application Server Configuration A lot of the configuration can be found in the “server.xml” of the AppSrv. Q# 1. Question Thread Pools DCS Answer (min/max) min=5 max=20 2. 3. 4. RAS Logging enabled? Recommendation / Comment Yes W3: WebSphere Application Server: Sizing / workload injection Q# 1. 2. Question Have you ever done workload injections? What infrastructure Answer Yes Testing has been done by the Recommendation / Comment
- 4. was used for the workload injection? Describe the tested usecases. Usecase 1: How were the usecases created? Did IBM ever execute a sizing with you? Do you plan your capacity for the WAS workload? Do you know how much system resources you need for a typical real live TX? 3. 4. 5. 6. 7. help of real users. Collegues did a login via the Internet interface and emulated users. Every user tested the different functions. There have been no formal usecases. Yes, see for more information. Yes No W4: WebSphere Application Server: Application Sizing / Resource Consumption If no sizing, injection or real life data is available answer the following Questions per Application Q# Question Answer Recommendation / Comment 1. Java Application complexity (1-10, lowhigh) 2. HTTP SSL used? 3. WebPage file serving Complexity (low, med, hi) 4. WebServices used? 5. WebServices SSL? 6. Program 2 Program calle used (no, http, iiop) 7. XML parsing (lo, med, hi) 8. JDBC DB2 processing (1-10, triv – complx) 9. What is the peak transaction rate 10. Is Logging in place? What package? Intensive? Can it be switched of for production? 11. ___________________________________________________________________________ IBM internal comments for doc/data required & questionaire 1. collector tool http://publib.boulder.ibm.com/infocenter/wasinfo/index.jsp? topic=/com.ibm.websphere.base.doc/info/aes/ae/ctrb_ct.html 2. jinsight is available at http://w3.research.ibm.com/jinsight/
- 5. 3. Sizing information needs to be fed into zPSG, 4. remote GC is controlled by a. 5. -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 6. ___________________________________________________________________________ W5: WebSphere Application Server: Security Q# Question Answer Recommendation / Comment
- 6. 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. Is GLOBAL security enabled? Does clients SSL traffic end in WAS? Security Cache enabled? Crypto HW enabled & verified? SWAM, LTPA, ICSF? Any TAI used? Describe. Optimized for performance? Registry Local, CUR, LDAP? If not Local: What Registry Server is used Remote Registry used for authentication and/or authorization? JACC used? What local RYO security solutions are in place? What 3rd security party solutions are in place? Yes No ??? No, crypto HW is not used. 13. W6: WebSphere Application Server: Deployment Application Specific Server Settings Q# Question Answer 1. IHS in place on what platform 2. For session affinity? 3. As SSL terminator? 4. As request distributor? 5. Other IP distributors in place? Explain 6. Recommendation / Comment W7: WebSphere Application Server: HA & WLM Q# 1. 2. 3. 4. 5. Question z/OS: # of used WLMWAS service classes z/OS: # of used WLMWAS reporting classes z/OS: min/max srv Number of threads per servant z/OS WLM even Answer Recommendation / Comment
- 7. 6. 7. 8. 9. distribution on? Workload distribution by IHS-WAS plug-in? IHS-WAS plug-in redundant? Frontend IP sprayer before IHS-WAS? IP Sprayer redundant and hot standby? 10. W8: WebSphere Application Server: Backend connectivity Q# 1. 2. 3. 4. Question What backend connection do you have? What is your transactional model? TX logging on fast device? Do you have a reliable NW connectivity to JDBC targets? Answer Recommendation / Comment JDBC Type 4 Driver No network used as the DB is on the same Linux image. 5. W9: WebSphere Application Server: Managability Q# 1. 2. 3. 4. 5. 6. 7. Question Are you always able to restore a failed application? Are you always able to restore WAS configurations? How do you save WAS configurations & when? Do you deploy and configure WAS using GUI or scripts? How many persons have access to WSADMIN (Administrator role)? Are your developers allowed to administer WAS? Answer Not tested Not tested TAR BackupConfig.sh ???TIME GUI 4 No Recommendation / Comment
- 8. W10: WebSphere Application Server: Monitoring ___________________ Q# 1. 2. 3. 4. Question z/OS: only SMF120 settings Answer Recommendation / Comment No z/OS used. Usage of SMF Records in Production SMF interval should be maximal 300s, not shorter than 120s There are certain sub types, which shouldn’t be turned on in production. PMI settings minimal Intervals: WSAM : is trace=yes ? Basic Coding wsam_intent_trace = YES in the WSAM datacollector.env can be used to able to switch from WSAM monitoring level 1 to level 3 without needing to restart the application server. This disables the just in time compiler even one running at monitoring level 1. W11: WebSphere Application Server: Application ___________________ Q# 1. 2. 3. 4. 5. 6. 7. 8. 9. 1 0. 1 1. 1 Question Describe type of application. ISV, RYO application? Soruce available? Does the app support the targeted runtime environment officially? Is there an no IBM Framework involved? Streamlined for performance? Is dynamic fragment caching enabled and optimized? Do you offload static content? Was your application profiled recently? Findings fixed? Are EJBs called locally (pass by reference) Is you application doing async calls in RMI or HTTP? Is heavy parsing part of the application. XSLT used? What is the typical max HTTP session size? How many HTTP Answer Recommendation / Comment If RMI is used check for Remote GC. For all async calls check for thread consumption
- 9. session objects per request? 1 Does the application 3. use a ryo authentication / authorization module 1 Is you application 4. creating own threads? Are those pooled? ___________________________________________________________________________ Tuning hotlist, merge into list and/or check during healthcheck • Tune hardware and capacity settings as described in Tuning hardware capacity and settings. • Set the minimum and maximum Java virtual machine heap sizes as described in Java virtual machine settings. • Tune your applications. Refer to Tuning applications. • Tune data sources connection pool and prepared statement cache as explained in Tuning application servers. • Set the Solaris operating system TCP_TIME_WAIT_INTERVAL as described in Tuning operating systems. • Enable Pass by reference as described in Object Request Broker service settings in administrative console. • Disable IBM HTTP Server access logs as described in Web server tuning parameters. • Set the KeepAliveEnabled custom property as explained in HTTP transport custom properties. • Set the transaction log directory as described in Transaction service settings. • Adjust the com.ibm.CORBA.FragmentSize as explained in Object Request Broker service settings. ________________________________________________________________________
- 10. session objects per request? 1 Does the application 3. use a ryo authentication / authorization module 1 Is you application 4. creating own threads? Are those pooled? ___________________________________________________________________________ Tuning hotlist, merge into list and/or check during healthcheck • Tune hardware and capacity settings as described in Tuning hardware capacity and settings. • Set the minimum and maximum Java virtual machine heap sizes as described in Java virtual machine settings. • Tune your applications. Refer to Tuning applications. • Tune data sources connection pool and prepared statement cache as explained in Tuning application servers. • Set the Solaris operating system TCP_TIME_WAIT_INTERVAL as described in Tuning operating systems. • Enable Pass by reference as described in Object Request Broker service settings in administrative console. • Disable IBM HTTP Server access logs as described in Web server tuning parameters. • Set the KeepAliveEnabled custom property as explained in HTTP transport custom properties. • Set the transaction log directory as described in Transaction service settings. • Adjust the com.ibm.CORBA.FragmentSize as explained in Object Request Broker service settings. ________________________________________________________________________