Application Performance Testing: A Simplified Universal Approach
- 1. MD AM Tutorial 9/30/2013 8:30:00 AM "Application Performance Testing: A Simplified Universal Approach" Presented by: Scott Barber PerfTestPlus, Inc. Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073 888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com
- 2. Scott Barber PerfTestPlus, Inc. CTO of PerfTestPlus and recognized worldwide as a thought-leader in software system performance and testing, Scott Barber describes himself as a “tester, author, speaker, disrupter, and dad.” Scott is the author of Web Load Testing for Dummies and co-/contributing author of Performance Testing Guidance for Web Applications, Beautiful Testing, and How to Reduce the Cost of Testing.
- 3. 8/27/2013 Managing Application Performance: A Simplified Universal Approach 1 Managing Application Performance: A Simplified Universal Approach Created for: By: Scott Barber Chief Performance Evangelist SmartBear Software, Inc. 1
- 4. 8/27/2013 Chief Performance Evangelist SmartBear Software, Inc. scott.barber@smartbear.com @sbarber Co-Founder: Workshop On Performance and Reliability www.performance-workshop.org Author: Co-Author: Contributing Author: Books: www.perftestplus.com/pubs About me: about.me/scott.barber What is Performance? System or application characteristics related to: Speed: - responsiveness - user experience Scalability: - capacity - load - volume Stability: - consistency - reliability - stress 2
- 5. 8/27/2013 What is a Performant System? A system optimized to balance: - Cost - Time to Market - Capacity …while remaining focused on: Quality of Service to System Users What is Performance Testing? Testing designed to: Inform software system optimization by balancing: - Cost - Time to Market - Capacity while remaining focused on the quality of service to system users. 3
- 6. 8/27/2013 Performance vs. Load Testing? The Performance Lifecycle is: Conception to Headstone Not Cradle to Grave 4
- 7. 8/27/2013 Prevent Poor Performance… … don’t just react when it happens. © PerfTestPlus, Inc. Who is Responsible? 5
- 8. 8/27/2013 © PerfTestPlus, Inc. Where does it fit in the SDLC? 6
- 9. 8/27/2013 Delivering Performant Systems; Commonly, Y2K -> Today SPE (Software Performance Engineering) Math-centric Architect/Design method + Profiling Context-free code optimization + Load Testing Demonstrate (too late) poor performance characteristics + Capacity Planning Math-centric guessing to determine hardware needs + APM (Application Performance Management) Proactively react to production issues => Inefficient & Reactionary Delivery & Maintenance Delivering Performant Systems; Moving Forward Minimize Need for Complex Mathematics Few teams have mathematicians w/ advanced degrees Disconnect Accuracy from Empirical Production Data Because we can’t depend on what we don’t have yet Remove Socio-Political Barriers Collaboration improves information sharing & efficiency Keep/Adapt most Valuable Practices Many good concepts have been poorly implemented Distribute Valuable Practices Throughout Life Cycle Many valuable practices have suffered from stove-piping I’m testing a model for the entire ‘Performance Journey’ 7
- 10. 8/27/2013 Performant Application Delivery © PerfTestPlus, Inc. Key Points • • • • Entire Lifecycle Entire Team Focused Individual, Role, & Team Accountability & Responsibility Continuous & Collaborative “DevOps” • • • • • • • Code Profiling Requirements Revision Design Validation Budgets Units & Components Monitoring Tuning Perf Team • • • • • Load/Stress Testing Environment Config Capacity Planning Concurrency Checks Situation Analysis Individual • • • • Snapshots Comparisons Niche Concerns Mini-Field Studies (UAT) • “What-if…” © PerfTestPlus, Inc. 8
- 11. 8/27/2013 Preventing Poor Performance with a little work… every day… from every one. © PerfTestPlus, Inc. Proactive Micro & Macro Establish Goals Update Targets Dev & Prod Times Resources Sizes Frequencies Dashboard! Units Stories Tiers Resources Goals Assess Compare Investigate Accept Answer © PerfTestPlus, Inc. 9
- 12. 8/27/2013 © PerfTestPlus, Inc. Unit-Level Testing Tools (‘cause folks always ask) FireBenchmarks; Performance testing addin for NUnit JUnitPerf; a collection of JUnit test decorators for performance Firefox Performance Tester's Pack HTTPerf 10
- 13. 8/27/2013 © PerfTestPlus, Inc. An Ounce of Prevention… 11
- 14. 8/27/2013 © PerfTestPlus, Inc. © PerfTestPlus, Inc. 12
- 15. 8/27/2013 What is it? Load Tests Stress Tests Endurance Tests Infrastructure/Configuration Verification Load Variability Analysis Background Load for Acceptance Tests… Or any other deliberate approximation of Production Conditions © PerfTestPlus, Inc. 13
- 16. 8/27/2013 RPT is: Inspired by Rapid Software Testing Consistent with Rapid Software Testing themes Sanctioned by James Bach, Michael Bolton & the RST instructors to as a specific implementation of the Rapid Testing Methodology For more information about RST, visit: http://www.satisfice.com/info_rst.shtml Evolved from: “What have we got? What do we want? How do we get there…?” --Bob Barber (Scott’s dad) … as quickly, simply, and cheaply as possible? 14
- 17. 8/27/2013 …a common man’s way of expressing the problem solving approach that classical engineers employ. • Given: “What have we got?” • Find: “What do we want?” • Solve: “How do we get there?” What is it? An approach to respond to a specific performance-related question after 4 or fewer hours of team effort with 1 or more of: A) The answer B) A partial answer • • To determine the value of additional effort The level of effort to provide the answer C) Better questions to address the underlying concern 15
- 18. 8/27/2013 Conceptual Approach 1. Receive Question • • 2. 3. 4. 5. 6. Clarify the question Understand the driver(s) behind the question Generate Test Coverage Outline (TCO) (~20 min) • Simplest path to (partial) answer(s) • Comprehensive path to (partial) answer(s) Transform TCO into Rapid Strategy (~20 min) • Only tasks that fit in time box • Stick to tasks requiring available resources Execute Strategy (~2.5 hrs) • Snapshots are your friends • Anecdotal is sufficient Consolidate/Analyze Data (~30 min) • Identify patterns • Confirm patterns (time permitting) Report Results (~20 min) • Answer(s) / Time or Effort to get answer(s) • Follow-on questions of interest ~4 hr © PerfTestPlus, Inc. © PerfTestPlus, Inc. 16
- 19. 8/27/2013 © PerfTestPlus, Inc. The Bottom Line © PerfTestPlus, Inc. 17
- 21. 8/27/2013 Contact Info about.me/scott.barber Chief Performance Evangelist SmartBear Softwear, Inc. E-mail: Scott.barber@smartbear.com Web Site: SmartBear.com Blog: scott-barber.blogspot.com Twitter: @sbarber 19