Build Your Open Source Performance Testing Platform in the Cloud
- 1. T9 Session 10/27/2016 1:30:00 PM Build Your Open Source Performance Testing Platform in the Cloud Presented by: Gopal Brugalette Brought to you by: 350 Corporate Way, Suite 400, Orange Park, FL 32073 888-‐268-‐8770 ·∙ 904-‐278-‐0524 - info@techwell.com - http://www.starcanada.techwell.com/
- 2. Gopal Brugalette Gopal Brugalette is a senior applied architect in performance engineering at upscale fashion retailer Nordstrom where he works on a variety of e-commerce omni-channel initiatives. Gopal's responsibilities include preparing the Nordstrom.com site for peak events, developing engineering frameworks, and expanding performance engineering activities into the development cycle and production. He has presented at numerous industry events and been featured in online magazines. Previously Gopal was at IBM and Washington Mutual Bank. Before he got into IT, he was a researcher at the Center for Experimental Nuclear Physics. In his spare time Gopal enjoys developing his permaculture farm and woodworking.
- 7. 10/17/2016 5 B k d Who will build it •Background •Skills •WhyWhy •Team Dynamics 9 Leadership Enterprise Direction Success MeasurementsSuccess Measurements Value Budget 10
- 9. 10/17/2016 7 Designing the framework Hand it off Re‐usable Structured inputs and outputs Modularity More than sum of parts 13 Deliver Value and Iterate R1 • Basic load and graphs • Pipeline integration R2 • Dynamic load • Deeper analytics R3 • Alerting • Reporting R4 • Auto‐ scripting • More alerting 14
- 10. 10/17/2016 8 D i T l S l ti Development Principles Design •Architecture •12 factor app Tool Selection •Integration •Capabilities •Containers •Cloud, on‐Prem •Community Support 15 16
- 11. 10/17/2016 9 Working in AWS Networking & Concepts • Automate It! • SLA’s • Cost & AWS Services • EC2, Lambda • Dynamo, S3 • Kinesis SQS Networking & Security • ELB • VPC • Security• Cost & Capacity • Kinesis, SQS • Cloudwatch, Elastic Search • Security Groups • IAM Roles 17 Scripting & Execution Engine Tool Consideration l ll•Jmeter,Gatling,Artillery... Scripting Integration 18 •Applications under Test •Language & capabilities •Code based or UI •Automation support •Results Output •AWS Load Generation 18
- 12. 10/17/2016 10 Load generation Resources dynamic or static •EC2 •Lambda Considerations •Capacity and Spin Up •Network Access •On‐prem •Cost •Management Effort •Test Types 19 API’s Hide Details Standardize Integrate Expand Tool Features Extensible Lambda in AWS 20
- 13. 10/17/2016 11 Orchestration Workflow/Pipeline Tools Test Assets Spin upp • Jenkins • Go • Rundeck API’s • Lambda SMS DynamoDB p p Run Test Analyze Results 21 • Lambda ‐ SMS, DynamoDB • Server‐based logic • Gateway Custom Scripting & Logic 21 Report Spin down Data Store A li i & Options • Elastic search • Influx • Splunk • APM Tools • Dynatrace Data Collection • Agent/forwarders • Log Stash/Beats • Telegraf • Direct writes • API's Extracting Data • API • Query Language • Raw or Analyzed • Size & Capacity Limits Application & Server Metrics • CloudWatch • APM • Logs and Plugins • topbeats Dynatrace • New Relic API s • Post • Streaming • Visualization Support 22
- 14. 10/17/2016 12 Analytics Performance Data • Response Time • Errors Application & Server • Stack traces • Resource Utilization Information • Trending • SLA, Pass/Fail • Throughput • Processing Time • Anomalies • Distribution Custom Built‐in Plug‐Ins & Packages Implementation • R • Python • Scala • Julia • Time Series • ELK, InfluxDB, Cloudwatch • APM • Dynatrace, New Relic g g • Watcher, Elastalert • Kapacitor • TimeLion • R, Python Packages 23 Reporting Visualization CI Integration AlertsVisualization • ES/Kibana • Grafana • Chronograf • Tableau CI Integration • Pass/fail • Stop the build Alerts • Email • Pager • Slack • Watcher• Tableau • Shiny • APM tools • Watcher • Elastalert • Kapacitor 24
- 15. 10/17/2016 13 Scaling Testing the framework g Resiliency Pilot teams Analytics Alerting & Reporting 25 Culture of Who has Encourage Adoption Culture of Reuse? Who has Responsibility? Market It Train Them 26
- 21. 10/17/2016 19 Dynamic Scaling II Jenkins λ ES Logic Call λ λ λ λ λ λ λ λ λ Kinesis S3or Jmeter,Gatling, Artillery (Java) Distribute Scripts, Data λ 37 API’s API’s ES KibanaAPI GW λ λλ λ λ λ λ λ λ λ λ λ λ λ λ 38 GW Gatling Jmeter API’s