Managing Application Performance: A Simplified Universal Approach
- 1. MD AM Tutorial 4/29/13 8:30AM Managing Application Performance: A Simplified Universal Approach Presented by: Scott Barber PerfTestPlus, Inc. Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073 888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com
- 2. Scott Barber CTO of PerfTestPlus and recognized worldwide as a thought-leader in software system performance and testing, Scott Barber describes himself as a “tester, author, speaker, disrupter, and dad.†Scott is the author of Web Load Testing for Dummies and co-/contributing author of Performance Testing Guidance for Web Applications, Beautiful Testing, and How to Reduce the Cost of Testing. He has written more than 100 articles and delivered keynotes on five continents. Currently, Scott’s focus is on applying, enhancing, and teaching Rapid Performance Testing and T4APM. When he’s not being a geek, Scott enjoys spending time with his partner Dawn and his sons Nick and Taylor. Learn more at about.me/scott.barber.
- 3. 4/11/2013 Managing Application Performance: A Simplified Universal Approach Created for: By: Scott Barber Chief Technologist PerfTestPlus, Inc. Managing Application Performance © 2013 PerfTestPlus, Inc. All rights reserved. Chief Technologist, PerfTestPlus, Inc. sbarber@perftestplus.com www.perftestplus.com @sbarber Co-Founder: Workshop On Performance and Reliability www.performance-workshop.org Author: Co-Author: Books: www.perftestplus.com/pubs Managing Application Performance Contributing Author: About me: about.me/scott.barber © 2013 PerfTestPlus, Inc. All rights reserved. 1
- 4. 4/11/2013 What is Performance? System or application characteristics related to: Speed: - responsiveness - user experience Scalability: - capacity - load - volume Stability: - consistency - reliability - stress Managing Application Performance © 2013 PerfTestPlus, Inc. All rights reserved. What is a Performant System? A system optimized to balance: • Cost • Time to Market • Capacity while remaining focused on: • Quality of Service to System Users Managing Application Performance © 2013 PerfTestPlus, Inc. All rights reserved. 2
- 5. 4/11/2013 Performance vs. Load Testing? Managing Application Performance © 2013 PerfTestPlus, Inc. All rights reserved. The Performance Lifecycle is: Conception to Headstone Not Cradle to Grave Managing Application Performance © 2013 PerfTestPlus, Inc. All rights reserved. 3
- 6. 4/11/2013 Prevent Poor Performance… … don’t just react when it happens. Who is Responsible? Managing Application Performance © 2013 PerfTestPlus, Inc. All rights reserved. 4
- 7. 4/11/2013 … But where does it fit in the SDLC? 5
- 8. 4/11/2013 …Or maybe like this? Managing Application Performance © 2013 PerfTestPlus, Inc. All rights reserved. …Or this? • • • • Current Capacity Projections Scalability Plan Reliability DevOps & Architects Prod Stage • Prepare for Prod • Tune/Optimize • Predict • Early Warning Identification Managing Application Performance Load Team & DevOps Architects Analysts & Scrum • Single User Responsiveness • Resource Management • Component-Level Concurrency Dev Test Scrum & Load Team • Multi-User Responsiveness • Resource Trending • Limit Identification © 2013 PerfTestPlus, Inc. All rights reserved. 6
- 9. 4/11/2013 …Or maybe even? Managing Application Performance © 2013 PerfTestPlus, Inc. All rights reserved. Performant Application Delivery Managing Application Performance © 2013 PerfTestPlus, Inc. All rights reserved. 7
- 10. 4/11/2013 Delivering Performant Systems; Commonly, Y2K -> Today SPE (Software Performance Engineering) Math-centric Architect/Design method + Profiling Context-free code optimization + Load Testing Demonstrate (too late) poor performance characteristics + Capacity Planning Math-centric guessing to determine hardware needs + APM (Application Performance Management) Proactively react to production issues => Inefficient & Reactionary Delivery & Maintenance Managing Application Performance © 2013 PerfTestPlus, Inc. All rights reserved. Delivering Performant Systems; Moving Forward Minimize Need for Complex Mathematics Few teams have mathematicians w/ advanced degrees Disconnect Accuracy from Empirical Production Data Because we can’t depend on what we don’t have yet Remove Socio-Political Barriers Collaboration improves information sharing & efficiency Keep/Adapt most Valuable Practices Many good concepts have been poorly implemented Distribute Valuable Practices Throughout Life Cycle Many valuable practices have suffered from stove-piping I’m testing a model for the entire ‘Performance Journey’ Managing Application Performance © 2013 PerfTestPlus, Inc. All rights reserved. 8
- 11. 4/11/2013 Performant Application Delivery Managing Application Performance © 2013 PerfTestPlus, Inc. All rights reserved. Key Points • Entire Lifecycle • Entire Team Focused • Individual, Role, & Team Accountability & Responsibility • Continuous & Collaborative “DevOps” • • • • • • • Code Profiling Requirements Revision Design Validation Budgets Units & Components Monitoring Tuning Managing Application Performance Perf Team • • • • • Load/Stress Testing Environment Config Capacity Planning Concurrency Checks Situation Analysis Individual • • • • Snapshots Comparisons Niche Concerns Mini-Field Studies (UAT) • “What-if ” © 2013 PerfTestPlus, Inc. All rights reserved. 9
- 12. 4/11/2013 Preventing Poor Performance with a little work… every day… from every one. Managing Application Performance Proactive Micro & Macro Establish Goals Update Targets Dev & Prod Times Resources Sizes Frequencies Dashboard! © 2013 PerfTestPlus, Inc. All rights reserved. Units Stories Tiers Resources Goals Assess Compare Investigate Accept Answer 10
- 13. 4/11/2013 Unit-Level Testing Tools (‘cause folks always ask) FireBenchmarks; Performance testing addin for NUnit JUnitPerf; a collection of JUnit test decorators for performance Firefox Performance Tester's Pack HTTPerf Managing Application Performance © 2013 PerfTestPlus, Inc. All rights reserved. 11
- 14. 4/11/2013 Managing Application Performance © 2013 PerfTestPlus, Inc. All rights reserved. An Ounce of Prevention… 12
- 15. 4/11/2013 Managing Application Performance © 2013 PerfTestPlus, Inc. All rights reserved. Managing Application Performance © 2013 PerfTestPlus, Inc. All rights reserved. 13
- 16. 4/11/2013 What is it? • Load Tests • Stress Tests • Endurance Tests • Infrastructure/Configuration Verification • Load Variability Analysis • Background Load for Acceptance Tests Or any other deliberate approximation of Production Conditions Managing Application Performance © 2013 PerfTestPlus, Inc. All rights reserved. Managing Application Performance © 2013 PerfTestPlus, Inc. All rights reserved. 14
- 17. 4/11/2013 Managing Application Performance © 2013 PerfTestPlus, Inc. All rights reserved. Managing Application Performance © 2013 PerfTestPlus, Inc. All rights reserved. 15
- 18. 4/11/2013 Managing Application Performance © 2013 PerfTestPlus, Inc. All rights reserved. Managing Application Performance © 2013 PerfTestPlus, Inc. All rights reserved. 16
- 19. 4/11/2013 Managing Application Performance • • Clarify the question Understand the driver(s) behind the question • • Simplest path to (partial) answer(s) Comprehensive path to (partial) answer(s) • • Only tasks that fit in time box Stick to tasks requiring available resources • • Snapshots are your friends Anecdotal is sufficient • • Identify patterns Confirm patterns (time permitting) • • • © 2013 PerfTestPlus, Inc. All rights reserved. Answer(s) Time/Effort to answer(s) Follow-on questions of interest Managing Application Performance © 2013 PerfTestPlus, Inc. All rights reserved. 17
- 20. 4/11/2013 Managing Application Performance © 2013 PerfTestPlus, Inc. All rights reserved. 18
- 21. 4/11/2013 The Bottom Line Managing Application Performance © 2013 PerfTestPlus, Inc. All rights reserved. Addendum Managing Application Performance © 2013 PerfTestPlus, Inc. All rights reserved. 19
- 22. 4/11/2013 Questions? Managing Application Performance © 2013 PerfTestPlus, Inc. All rights reserved. Contact Info about.me/scott.barber Chief Technologist PerfTestPlus, Inc. E-mail: sbarber@perftestplus.com Web Site: www.PerfTestPlus.com Blog: scott-barber.blogspot.com Twitter: @sbarber Managing Application Performance © 2013 PerfTestPlus, Inc. All rights reserved. 20