Stating the obvious - All Day DevOps 2017
Download as PPTX, PDF1 like154 views
This document discusses the importance of performance testing and integrating it into continuous integration/continuous deployment pipelines. It provides an overview of common performance testing tools and scenarios. Key points include that performance is a valuable business feature, performance tests can be easily integrated into CI/CD, and reading performance test results requires skill. The presenter provides their contact information and a bibliography of references on performance and load testing.
Stating the obvious - All Day DevOps 2017
- 1. Stating the obvious Giulio Vian DevOps Lead Glass Lewis Europe Ltd @giulio_vian
- 2. Agenda Why obvious? Integration with Continuous Integration / Continuous Deployment Performance / load/ stress testing basics Closing
- 4. 1.Performance is valuable 2.It’s easy to integrate Perf/load test in CI/CD 3.Reading the results is art Three Takeaways
- 6. Common scenarios Desktop / mobile / library Server / web Database server Web serverClient N1 N4 A1 A3 A2N3 N2
- 7. Common scenarios Desktop / mobile / library Server / web Database server Web serverClient N1 N4 A1 A3 A2N3 N2
- 9. Performance is a Feature «there are two kinds of websites: the quick and the dead» https://blog.codinghorror.com/performance-is-a-feature/
- 10. Response time 0.1 second is about the limit for having the user feel that the system is reacting instantaneously, meaning that no special feedback is necessary except to display the result. 1.0 second is about the limit for the user's flow of thought to stay uninterrupted, even though the user will notice the delay. Normally, no special feedback is necessary during delays of more than 0.1 but less than 1.0 second, but the user does lose the feeling of operating directly on the data. 10 seconds is about the limit for keeping the user's attention focused on the dialogue. For longer delays, users will want to perform other tasks while waiting for the computer to finish, so they should be given feedback indicating when the computer expects to be done. Feedback during the delay is especially important if the response time is likely to be highly variable, since users will then not know what to expect. Miller, R. B. (1968). Response time in man- computer conversational transactions. Proc. AFIPS Fall Joint Computer Conference Vol. 33, 267-277.
- 11. Hard facts Google Half a second delay caused a 20% drop in traffic. Amazon.com Even very small delays (100 ms) would result in substantial and costly drops in revenue.
- 12. Why lubricant in an engine? Do you like the motor to seize?
- 13. Always necessary? Gartner Hype Cycle, Source: Wikipedia
- 14. My perf test of the year 2M users 40,000 RPS 2Gbps Our tests around 1,000 There’s Nothing Like Production © 2016 IMG Universe, LLC. All Rights Reserved
- 15. II. CI / CD Integration
- 16. Tooling Scripts & Runners Load generators Monitoring
- 17. Tools: Script & Runners JMeter Visual Studio (.webtest / .loadtest) Selenium ×Unit family Gatling *Custom & More…
- 18. Tools: Load VSTS Cloud-based load testing CA BlazeMeter SOASTA (Akamai) CloudTest HPE LoadRunner *Custom & More…
- 19. Tools: Monitoring Loading tool Basic Performance Counters* Application logs Cloud & More…
- 20. Analyzing Build an aggregated view Find interesting events
- 21. High level architecture You Load Generator SUT Monitoring CI / CD pipeline
- 22. Web Load Testing Using Visual Studio, Visual Studio Team Services and Azure
- 23. Considerations for CI/CD Trunk or branch of pipeline Resources KPIs
- 24. Time factor Short-running Tests Long-running Tests Test Coverage Frequency of execution Value from test
- 25. Hidden Gremlins Cloud infrastructure warm up e.g. AWS ELB Default configuration nginx worker processes Client resources
- 26. III. Perf / load/ stress testing basics
- 27. (Technical) Goals Benchmarking Force defects to emerge code, configuration, architecture Capacity planning Breaking point Security leakage
- 28. Emerging Problems Connection pool exhaustion File locks Database locks (Thread) deadlocks Memory exahustion Domino Effect Lack of scalability
- 30. Compound graph +Error rates +Tools errors Relative Load Latency Required threshold Max N seconds 100%60% Throughput Usage
- 31. How you read this? Relative Load Latency Required threshold Max N seconds 100%60% Throughput Usage
- 32. How you read this? Resource Exhaustion! Relative Load Latency Required threshold Max N seconds 100%60% Throughput Usage
- 34. What to monitor CPU RAM Disk I/O Network I/O Threads / processes
- 35. IV. Closing
- 37. Bibliografy & References http://www.slideshare.net/giuliov/presentations https://github.com/giuliov/Stating-the-obvious/ https://docs.microsoft.com/en-us/vsts/build- release/tasks/test/cloud-based-load-test AWS Well-Architected Framework - Performance Efficiency Pillar https://www.amazon.com/dp/B01MSSLHBX Performance Testing Guidance for Web Applications https://msdn.microsoft.com/en-us/library/bb924375.aspx http://www.brendangregg.com/linuxperf.html
- 38. Bibliografy (2) Writing High-Performance .NET Code — Ben Watson (Ben Watson) https://www.amazon.it/Writing-High- Performance-NET-Code- Watson/dp/0990583430/ Time Is Money: The Business Value of Web Performance — Tammy Everts (O'Reilly Media) https://www.amazon.com/Time-Money- Business-Value- Performance/dp/1491928743
- 39. Bibliografy (3) Software Performance and Scalability: A Quantitative Approach — Henry H. Liu (Wiley) https://www.amazon.com/Software- Performance-Scalability-Quantitative- Approach/dp/0470462531 Continuous Delivery with Windows and .NET — Matthew Skelton and Chris O'Dell (O'Reilly) http://www.oreilly.com/webops- perf/free/continuous-delivery-with-windows- and-net.csp
- 40. Bibliografy (4) Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation — J.Humble, D.Farley (Addison-Wesley) https://www.amazon.com/Continuous- Delivery/dp/0321601912/ The DevOps Handbook — G.Kim, P.Debois, J.Willis, J.Humble (IT Revolution Press) https://www.amazon.com/DevOps-Handbook- World-Class-Reliability- Organizations/dp/1942788002/
- 41. 1.Performance is valuable for business 2.It’s easy to integrate Perf/load test in CI/CD 3.Reading the results is art Three Takeaways
- 42. Session Title Your Name Your Title Your Company Your @TwitterHandle
- 43. Session Title Your Name Your Title Your Company Your @TwitterHandle
- 44. Session Title Your Name Your Title Your Company Your @TwitterHandle
- 45. bit.ly/addo-slack Find me on slack, right now!
Editor's Notes
- #5: This should be at the beginning and end of your presentation: What are the three talking points your audience should be able to answer after this presentation?
- #10: https://blog.codinghorror.com/performance-is-a-feature/
- #23: https://docs.microsoft.com/en-us/vsts/release-notes/2017/sep-15-team-services#run-webtests-using-the-vstest-task
- #37: Performance is one of those areas that you could spend an infinite amount of time on, so failing to recognise it as a business interest could mean either chronic under-investment in it – or massive amounts of invisible over-spending if the technical team have placed too much emphasis on it. – Steve Fenton https://www.stevefenton.co.uk/2016/06/performance-is-a-feature/
- #42: This should be at the beginning and end of your presentation: What are the three talking points your audience should be able to answer after this presentation?
- #45: Thank you to our sponsors for their continuing support of the DevOps community -- Sonatype -- Puppet -- XebiaLabs -- Read Hat -- Cloudbees -- Sauce Labs -- Micro Focus -- VictorOps -- Carnegie Mellon University -- Dzone -- DevOps.com -- SD Times -- Tech Beacon