performancetestinganoverview-110206071921-phpapp02.pdf
- 1. Performance Testing Highlights Sharad Jain Date – 07th Feb 2011
- 2. What is Performance Testing • Performance testing – refers to test activities on checking system performance The major objectives of performance testing: • To confirm and validate the specified system performance requirements. • To check the current product capacity to answer the questions from customers and marketing people. • To identify performance issues and performance degradation in a given system
- 3. Differences b/w Performance, Load and Stress Test • Performance Test determine the run time “behavior” of the application and its supporting infrastructure, under certain conditions. • Performance testing is used to measure several system characteristics, such as processing speed, response time, resource consumption, throughput and efficiency. • Load Test determine the applications “behavior under load”, up to and including its limits (not just as its limits). • Load tests specifically refers to the load size (number of concurrent users) and related values. • Stress Test determines the application ability to handle large amount of data • Stress testing can be much more successful with the full load applied to the server.
- 4. Why Performance Testing? • Scalability – Will the application handle the expected load and beyond? • Stability – Is the application stable under expected and unexpected user loads? • Availability – Is the application available to the end user without any interruption? • Serviceability – Can the system quickly recover from a Failure? • Speed - Does the application respond quickly. • Confidence – Are you sure that clients will have a positive experience on go-live day?
- 5. Performance Requirements • Following are some representative performance requirements which system would be expected to meet • The system should support peak load of x active users and y transactions/sec. (for web based application it could be requests/sec) • At peak load, system response to the X% of users should be Y Sec. • System should be able to support 24 X 7 operation (reliability) • System should be scalable to meet growth in demand. • The response time requirement could vary based on • Geographical location of users • Connection bandwidth (especially true for Internet users) • Transaction complexity.
- 6. Performance parameters to consider • Throughput: The number of requests processed per unit time (per second) • Latency: The time taken in establishing the network handshake. • Efficiency: Throughput - Latency • Degradation: The Throughput of the application when the concurrent requests are gradually increased – Test to see if the performance of the application degrades when the number of concurrent requests is increased. • Longevity : Execute tests for a pre-determined time interval with a pre-determined load. Monitor the health of the application throughout the test execution and check for memory leaks and analyze the GC dump
- 7. What Performance Problem may arise ? • Memory-related problems: • Application uses more memory than it should. • Memory leaks. • Excessive garbage allocation, i.e. application creates a lot of temporary objects. • Code-related problems: • Application algorithms are not optimal, and there are performance bottlenecks. • Hardware/Software related Problem: • Hardware you select for your database is not in vendor Hardware Compatibility List
- 8. Scalability • This section talks about the impact of scaling up the Hardware resources from the suggested minimum Hardware requirements • Processor impact • Database impact • I/O impact • Network bandwidth Impact
- 9. Processor Impact • Single Processor v/s Multi Processor The response time for searching or archiving a document decreased by 50 - 80% on adding an additional processor in a multi- threaded scenario (info based on assumptions)
- 10. Data base Impact • Single Processor v/s Multi Processor • As the number of concurrent users using application services are increased, it is observed that Oracle performs better than MS SQL Server with a 5 - 8% improvement in the performance (info based on assumptions)
- 11. I/O Impact • I/O has considerable impact on the performance of the Application with 15% to 18% reduction in the response time especially when there are huge reports to be generated on the system.
- 12. Network bandwidth Impact • A good Network connectivity between the Client, Application Server and database improves the performance considerably. There was 40% (approx) improvement in the throughput of the application when the Network Connectivity between Database system and Application Server system was increased from 100 mbps to 1Gbps (info based on assumptions).
- 13. Identify the Performance issues Application Database Application Code Application Design. JVM settings Application server configuration and usage Web Server Configuration and usage System OS System OS Client Side Database Schema, design, configurations, resource usage, SQLs, Indexes etc OS Resources – CPU, Network, Disk, Memory, OS kernel, Storage etc Browser settings, Client system, Page size, amount of data displayed
- 14. Performance Bottleneck Sources Network 8% Web Server 12% App Server 35% DB Server 45%
- 15. Database Server Causes Other 5% Slow specific query response 22% Slow overall query response 16% Slow overall DB response 13% Overuse of row-at-a-time processing 11% Missing indexes 8% Poor multi-user response 7% Query plan cache too small 7% Data cache too small 5% Deadlocks 3% Old optimizer statistics 3%
- 16. App Server Causes Memory leak 15% Inefficent garbage collection 12% Expired sessions still active 12% Poorly configured App Server 12% Insufficient hardware resources 10% Poorly configured DB connection pool 9% Inefficiently coded transaction 11% Inefficient DB access architecture 4% Inefficient object access method 5% Other 10%
- 17. Network Causes Bad network architecture 20% Other 20% Poorly configured/insufficient NICs 10% Security too tight 15% Insufficent overall bandwidth 13% Load balancing ineffective 22%
- 18. Software and Hardware Requirements • Hardware Requirement • Details of system to be tested • Workstation required for load simulation • Workstation required for running monitoring utilities • Network Equipment (e.g. Router, Firewall, Modems etc.) • Software Requirement • System Software • Load Testing Tools • Performance Monitoring utilities • Miscellaneous utilities like graph generation etc.
- 19. Performance Testing Process • Understand system and identify performance requirements • Identify performance test objectives. • Define performance test strategy: • Identify the needs of performance test tools and define performance test environment • Write performance test plan • Set up the target system and performance test beds • Design performance test cases and test suite • Performance test execution and data collection • Performance analysis and reporting
- 20. Performance Testing Approach • Performance testing: (during production) • Measure and analyze the system performance based on performance test data and results • Performance simulation: (pre-production) • Study and estimate system performance using a simulation approach • Performance measurement at the customer site: (post-production) • Measure and evaluate system performance during system operations
- 21. Performance Modeling Project Overview In order to analyze performance behavior, improve product performance, and serve as the foundation for capacity planning • The overall goal was to design and develop a model that would be: • realistic and general enough to be widely applicable across a variety of customer configuration and application scenarios in Altair target market. • simple enough to be easily usable by the field sales and support organizations to assist customers make system configuration decisions • possible to validate so that it could be applied with confidence.
- 22. What is the resultant deliverable The resultant of this performance testing exercise is a “Performance and Scalability document” with results of the tests performed and hardware recommendations. The document would not be a sizing guide per se but will have: • Throughput and scalability numbers for the end users to consume • Test metrics - numbers and graphs related to the tests executed • And an internally consumable document containing performance recommendations for the developers to work on. Also, note that the resultant document is not a Sizing guide. But this document will definitely be a good input to the performance team to come up with a sizing guide for the future releases.
- 23. Thank You