Webinar: Automated Load Testing Results Analysis
- 1. LOAD IMPACT Automated Results Analysis
- 2. LOAD IMPACT Robin Gustafsson CTO, Load Impact Mark Meier Team Lead, Client Success Agenda ● Getting started methodology ● Challenges with understanding load test results ● Basics of performance testing result analysis ● Introduction to Performance Alerts and result analysis in Insights ● Continuous testing using Checks and Thresholds
- 3. LOAD IMPACT Methodology ● Planning, test configuration and validation - START SMALL! ● Baseline testing ● Scale-up + complex tests ● Ongoing/Recurring testing ○ Path to Continuous Testing https://support.loadimpact.com/4.0/testing-methodologies/application-testing-methodology/
- 4. LOAD IMPACT Results analysis challenges ● New to Load Testing ○ What to look for ○ Scripting and configuration issues ● Large amount of data
- 5. LOAD IMPACT High-level results analysis ● Are load generation resources saturated? ● Is there a high rate of error responses? ● Are response times deviating from a flat trend? ● Is the request throughput not following the pattern of active Virtual Users?
- 6. LOAD IMPACT High-level result analysis ✅ Good result
- 7. LOAD IMPACT High-level result analysis Load generator resource saturation ❌ Bad result
- 8. LOAD IMPACT High-level result analysis High response failure rate ❌ Bad result
- 9. LOAD IMPACT High-level result analysis Flattening throughput and Increasing response time ❌ Bad result
- 10. LOAD IMPACT Performance Alerts ● What are they? ● Smart Result Performance Alerts - System Under Test performance issue ● Test Health Performance Alerts - Test script issue
- 11. LOAD IMPACT Performance Alerts ● Smart Result Performance Alerts - Automate the checklist: ○ No load generation resources saturated? ✅ ○ Not a high rate of error responses? ✅ ○ Response times are trending flat? ✅ ○ Request throughput is following the pattern of active Virtual Users? ✅ ● Test Health Performance Alerts - Provide alerts for issues arising from test scripting mistakes
- 12. LOAD IMPACT Performance Alerts > SUT performance issues ● Throughput limit ● High HTTP failure rate ● Not enough training data (informational) ○ Performance issue alerts require at least 100 script iterations of data for “training” + 15 seconds of additional data for detection
- 14. LOAD IMPACT Performance Alerts High HTTP failure rate
- 15. LOAD IMPACT Performance Alerts > Test health/scripting issues ● Load generator resource saturation: ○ High load generator CPU usage ○ High load generator memory usage ● Scripting issue: ○ Third-party content ○ Too many URLs ○ Too many groups ○ Too many metrics
- 16. LOAD IMPACT Performance Alerts Test Health Alerts - Load generator resource saturation
- 17. LOAD IMPACT Performance Alerts Test Health Alerts - Third party content
- 18. LOAD IMPACT Performance Alerts Test Health Alerts - Scripting issues
- 19. LOAD IMPACT Ongoing testing - Automation ● Performance regression testing ● Manual or scheduled ● CI tool integration
- 20. LOAD IMPACT Ongoing testing > Checks and Thresholds ● Thresholds are limits on metrics that are used by k6 and Load Impact 4.0 to pass/fail tests ● Checks are like asserts, allows you to assert functional aspects of your tests
- 21. LOAD IMPACT Continuous testing > Checks and Thresholds DEMO
- 22. LOAD IMPACT Learn More k6 documentation: https://docs.k6.io (join our slack team! https://k6.io/slack/) GitHub repo: https://github.com/loadimpact/k6 Insights: https://loadimpact.com/insights/ Load Impact Docs: https://support.loadimpact.com/4.0/ Load Impact Support - support@loadimpact.com Try Load Impact 4.0 with free 30-day trial
- 23. LOAD IMPACT Automated results analysis Thank you! Q & A time