Managing Application Performance: A Simplified Universal Approach
- 2. hief performance evangelist for SmartBear Scott Barber is a respected leader in the r as director of e Scott Barber SmartBear C advancement of software testing practices, an industry activist, and load testing celebrity of sorts. Scott authored several books―Performance Testing Guidance fo Web Applications, Beautiful Testing, How to Reduce the Cost of Testing, and Web Load Testing for Dummies―and more than 100 articles and blog posts. Founder/president of PerfTestPlus, Scott co-founded the WOPR, served the AST and CMG, and is a founding member of ISST. His industry writing, speaking, and activism focus on improving the effectiveness and business alignment of softwar development practices. Learn more about Scott Barber.
- 3. 2/19/2014 1 Managing Application Performance: A Simplified Universal Approach Revised for: Managing Application Performance: A Simplified Universal Approach By: Scott Barber Product Owner – Load Testing Tools SmartBear Software, Inc.
- 4. 2/19/2014 2 Product Owner – Load Testing Tools SmartBear Software, Inc. scott barber@smartbear comscott.barber@smartbear.com @sbarber Co‐Founder: Workshop On Performance and Reliability www.performance‐workshop.org Co‐Author:Author: Contributing Author: Books: www.perftestplus.com/pubs About me: about.me/scott.barber System or application characteristics related to: Speed: responsiveness What is Performance? ‐ responsiveness ‐ user experience Scalability: ‐ capacity ‐ load ‐ volume Stability: ‐ consistency ‐ reliability ‐ stress
- 5. 2/19/2014 3 What is a Performant System? A system optimized to balance: ‐ Cost‐ Cost ‐ Time to Market ‐ Capacity …while remaining focused on: Quality of Service to System Users What is Performance Testing? Testing designed to: Inform software system optimization byInform software system optimization by balancing: ‐ Cost ‐ Time to Market Capacity‐ Capacity while remaining focused on the quality of service to system users.
- 9. 2/19/2014 7 Delivering Performant Systems; Commonly, Y2K ‐> Today SPE (Software Performance Engineering) Math‐centric Architect/Design method + Profiling Context‐free code optimization + Load Testing Demonstrate (too late) poor performance characteristics + Capacity Planning+ Capacity Planning Math‐centric guessing to determine hardware needs + APM (Application Performance Management) Proactively react to production issues => Inefficient & Reactionary Delivery & Maintenance Delivering Performant Systems; Moving Forward Minimize Need for Complex Mathematics Few teams have mathematicians w/ advanced degreesFew teams have mathematicians w/ advanced degrees Disconnect Accuracy from Actual Production Data Because we can’t depend on what we don’t have yet Remove Socio‐Political Barriers Collaboration improves information sharing & efficiency Keep/Adapt most Valuable PracticesKeep/Adapt most Valuable Practices Many good concepts have been poorly implemented Distribute Valuable Practices Throughout LifeCycle Many valuable practices have suffered from stove‐piping We need one model for the whole ‘Performance Journey’
- 10. 2/19/2014 8 © PerfTestPlus, Inc. Key Points • Entire Lifecycle • Entire Team Focused • Individual, Role, & Team Accountability & Responsibility • Continuous & Collaborative “DevOps” Perf Team Individual • Code Profiling • Requirements Revision • Load/Stress Testing • Environment Config • Snapshots • Comparisonsq • Design Validation • Budgets • Units & Components • Monitoring • Tuning g • Capacity Planning • Concurrency Checks • Situation Analysis p • Niche Concerns • Mini‐Field Studies (UAT) • “What‐if…” © PerfTestPlus, Inc.
- 11. 2/19/2014 9 Preventing Poor Performance with © PerfTestPlus, Inc. Preventing Poor Performance with a little work… every day… from every one. Units Stories Tiers Resources Goals Proactive Micro & Macro Establish Goals Update Targets Dev & Prod © PerfTestPlus, Inc. Assess Compare Investigate Accept Answer Dev & Prod Times Resources Sizes Frequencies Dashboard!
- 13. 2/19/2014 11 © PerfTestPlus, Inc. Unit‐Level Testing Tools (‘cause folks always ask) FireBenchmarks; Performance testing addin for NUnit JUnitPerf; a collection of JUnit test decorators for performance Firefox Performance Tester's Pack HTTPerf
- 16. 2/19/2014 14 Load Tests Stress Tests d What is it? Endurance Tests Infrastructure/Configuration Verification Load Variability Analysis Background Load for Acceptance Tests… Or any other deliberate approximation of Production Conditions © PerfTestPlus, Inc.
- 17. 2/19/2014 15 RPT is: Inspired by Rapid Software TestingInspired by Rapid Software Testing Consistent with Rapid Software Testing themes Sanctioned by James Bach, Michael Bolton & the RST instructors to as a specific implementation of the Rapid Testing MethodologyRapid Testing Methodology For more information about RST, visit: http://www.satisfice.com/info_rst.shtml “What have we got? What do we want? Evolved from: What do we want? How do we get there…?” ‐‐Bob Barber (Scott’s dad) … as quickly, simply, and … as qu c y, s p y, a d cheaply as possible?
- 18. 2/19/2014 16 …a common man’s way of expressing the problem solving h h l i l iapproach that classical engineers employ. • Given: “What have we got?” • Find: “What do we want?” • Solve: “How do we get there?” An approach to respond to a specific performance‐related question after 4 or fewer What is it? hours of team effort with 1 or more of: A) The answer B) A partial answer • To determine the value of additional effort Th l l f ff t t id th• The level of effort to provide the answer C) Better questions to address the underlying concern
- 19. 2/19/2014 17 1. Receive Question • Clarify the question • Understand the driver(s) behind the question 2. Generate Test Coverage Outline (TCO) (~20 min) • Simplest path to (partial) answer(s) Conceptual Approach • Simplest path to (partial) answer(s) • Comprehensive path to (partial) answer(s) 3. Transform TCO into Rapid Strategy (~20 min) • Only tasks that fit in time box • Stick to tasks requiring available resources 4. Execute Strategy (~2.5 hrs) • Snapshots are your friends • Anecdotal is sufficient ~4 hr 5. Consolidate/Analyze Data (~30 min) • Identify patterns • Confirm patterns (time permitting) 6. Report Results (~20 min) • Answer(s) / Time or Effort to get answer(s) • Follow‐on questions of interest © PerfTestPlus, Inc. © PerfTestPlus, Inc.
- 22. 2/19/2014 20 Contact Info about me/scott barberabout.me/scott.barber Product Owner – Load Testing Tools SmartBear Softwear, Inc. E-mail: Scott barber@smartbear com Web Site: SmartBear.comScott.barber@smartbear.com Blog: scott-barber.blogspot.com SmartBear.com Twitter: @sbarber