Monitoring Redefined - Austrian Testing Board
Download as PPTX, PDF1 like155 views
The document discusses the importance of redefining monitoring strategies in technology, highlighting four core KPIs: business, availability, errors, and performance. It emphasizes that monitoring must evolve to address modern application architectures and dynamic environments, focusing on service flows and optimization. Additionally, it outlines challenges in DevOps adoption and calls for better collaboration and data consistency across business and development teams.
Monitoring Redefined - Austrian Testing Board
- 1. Klaus Enzenhofer Director Technology Strategy Monitoring Redefined klaus-enzenhofer @kenzenhofer
- 5. confidential
- 7. The 4 Core KPIs of Monitoring
- 10. confidential Watch your business success! Business KPI
- 11. What‘s next?
- 15. Adonair
- 16. Payment Service Transaction Service Login Service Balance Service User Service Customer View SLA 00:00 23:59 ? 98% 100% 99% 95% 99%
- 18. What‘s next?
- 20. confidential
- 21. confidential
- 23. 25 Example of error: What you see here, is the CUSS status 309 Approximately 20 minutes before 309 there is the last customer interaction The fix: Improved the code to prevent this freeze situation.
- 24. 26 CK – Business KPI Dashboard
- 26. What‘s next?
- 27. 4.5 sec 15 sec Why?
- 28. Network Same Page 4.5 sec 15 secSanity Check Browser CheckChrome 49 Chrome Mobile 33 Server Side Local WLANLocal WLAN Only difference is Browser & Device
- 29. confidential Why did they look at the performance on the mobile device?
- 30. Change in their compensations plan!
- 31. Contract SLA: Average Response Time < 3 sec User on Desktop + Mobile
- 32. Good idea?!
- 34. Let‘s take a look at the timings! Navigation Start: 0 ms Domain Lookup End: 269 ms Connect End: 330 ms Response Start: 517 ms Response End: 518 ms Dom Loading: 519 ms Dom Interactive: 519 ms DomContentLoaded Event End: 520 ms Dom Complete: 520 ms
- 35. 0.5 sec 0.5 sec Developer
- 36. User
- 38. 285 Resources for an initial Page Load 151 CSS and 121 JavaScript files
- 39. ~200 Resources had larger Header than Body
- 40. The CDN bill exploded!
- 44. Back Home
- 45. Back Home
- 46. HTTP Archive – Transfer Size Trend http://httparchive.org/trends.php Average Size ~2 500 KB By 1.6 € per 100 KB 40 € to get started!!!!
- 48. confidential Monitoring needs to cover: Business Results Availability Errors Performance
- 51. Monitoring used to be about looking at dashboards …
- 52. Process Memory (GB) CPU Graphs (%)
- 53. .. and about analyzing logs & exceptions …
- 55. But the apps and services we build have transformed to something more dynamic…
- 56. confidential Develop Ship Deploy Run Scale Compute nodejs mongo db netty cassandra redis ansible jenkins puppet chef docker cloudfoundry rh openshift rh atomic rocket core os rancher kvm busybox mesos marathon kubernetes swarm Amazon azure openstack mesosphere calico weave eureka/hystrix A whole new technology stack & polyglot development Amazon DynamoDB AWS Lambda AWS CodeDeploy Amazon EC2 Container Services Amazon EC2 AWS Elastic Beanstalk Amazon API Gateway
- 58. Granularity Doc Processor Doc Transformer Doc Signer Doc Encryption Doc Shipment Document Encryption is carved out at a separate service. May not be the best option to run it as a separate service Documents
- 60. Tightly coupled. Really Distributed?
- 61. confidential Inefficient Service Flow (drawing parallels to Web Performance Optimization)
- 62. SFPO (Service Flow&Performance Optimization) has to teach us how to optimize (micro)service dependencies through Service Flows
- 63. Especially useful to identify: inefficient 3rd party services, recursive call chains, N+1 Query Patterns, loading too much data, no data caching, … -> sounds very familiar to WPO
- 64. Classical cascading effect of recursive service calls!
- 65. THIS IS WHY monitoring had to transform as well
- 66. 2major releases/year customers deploy & operate on-prem 26 major releases/year 500 prod deployments/day self-service online sales SaaS & Managed 2011 2016 sprint releases (continuous-delivery) 1h: Code -> Prod6months major/minor release
- 67. Monitoring as Pipeline & Platform Feature Dev Perf/Test Ops Biz Faster Innovation with Quality Gates Faster Acting on Feedback Unit Perf Cont. Perf New Deploy New Capability CI CD Remove/Promote Triage/Optimize Update Tests Innovate/Design $$$ Lower Costs Happy Users
- 68. acting as Engineers Role of Dynatrace DevOps Team Dynatrace Managed/SaaS Orchestration Layer DynatracePipeline Visualization Deployment Timeline Log Overview using Dynatrace Log APIJIRA Integrations & Product Managers
- 69. Shift-Left Continuous Performance with Dynatrace “Performance Signature” for Build Nov 16 “Performance Signature” for Build Nov 17
- 70. Learnings when scaling DevOps Pipelines Feature Team A Feature Team B Feature Team X Improve “Efficiency” Cloud Ops Ensure “Operational Service” PM/Biz Improve“Business”
- 71. Dynatrace Transformation by the numbers 26 500 Releases / Year Deployments / Day 31000 60h Unit & Int Tests / hour UI Tests per Build More Quality ~120 340 Code commits / day Stories per sprint More Agile 93% Production bugs found by Dev More Stability 450 99.998% Global EC2 Instances Global Availability
- 72. High Performers vs Low Performers: Speed Gap Closing but Quality Gap Increasing https://puppet.com/resources/whitepaper/2017-state-devops-report/
- 73. BizDevOps Adoption Challenges Technical Complexity DevOps promotes choice: “the best stack for your problem” Bad Data & Code Quality DevOps today mainly driven by Biz “faster to market” but not “quality to market” Data & Department Silos DevOps promotes small & agile: “2 Pizza Teams”, “Services”, “Containers” IDG Research: April 2017 - http://www.computerwoche.de/a/digitale-kundenbeziehung-keine-halben-sachen,3330524,2 https://www.dynatrace.com/blog/devops-adoption-challenges-from-around-the-world/
- 74. The reason why: Different Perspective from Biz and DevOps
- 76. Marketing Analysts Executives Search Engine Optimization Security team Business AnalyticsFraud Detection UX-Designer App Owner CxO Customer Success Team
- 78. Biz View: Airline – Platinum Member Traveling Book Flight Check- In Stop in Lounge Inflights Max Platinum Going on a Trip
- 79. Team Individual Pipe Cycle Time Monitoring Geo Service Team Weekly Product Service Team Every Sprint Book Service Team Daily Auth Service Team On-Demand Mobile App Team Monthly Dev View: Airline – Platinum Member Traveling
- 80. Team Individual Pipe Payment Service A Check in Service B Passport Service C Baggage Service D Check in Service X ISSUE! Max Platinum Can Not Check In!
- 81. Silo #4 Silo #3 Silo #2 Silo #1 Are we making MONEY with Max? Which digital touchpoints is MAX using? System Availability Errors PerformanceBusiness Result Digital Touchpoints: Mobile App Desktop Web Kiosk App PoS-System Voice Interfaces (Alexa,...) Rich Client App System Availability Errors PerformanceBusiness Result System Availability Errors PerformanceBusiness Result System Availability Errors PerformanceBusiness Result Silo #6 Silo #7 Silo #8
- 82. 12-01-2011IAR - Version 0.91 8 4
- 83. confidential 85
- 84. 86
- 85. confidential So what should we do now?
- 86. confidential Have a BIG vision
- 87. confidential We need to answer the same questions for ALL touchpoints System Availability Errors PerformanceBusiness Result Digital Touchpoints: Mobile App Desktop Web Kiosk App PoS-System Voice Interfaces (Alexa,...) Rich Client App System Availability Errors PerformanceBusiness Result System Availability Errors PerformanceBusiness Result System Availability Errors PerformanceBusiness Result
- 88. Digital Touchpoints: Mobile App Desktop Web Locations: Vienna, Austria Store Salzburg Check-in Terminal A FRA Construction Site ABC, India Device: Mobile Mobile Broswer Kiosk App PoS System Voice Interfaces (Alexa,...) Rich Client App Smart Watch ATM Car Entertainment System TV
- 89. confidential OpsDev Biz Collaboration based on Consistent Data
- 90. Act tomorrow locally! Establish a quality gate beyond functional health Introduce monitoring early in the pipeline Chart your money making step/action Take a look the 4 Key KPIs and check them Make the KPIs available to others Start with a minimal DevOps Check your monitoring solution future readiness No Monitoring in place? – Checkout Dynatrace
- 91. Klaus Enzenhofer Director Technology Strategy Monitoring redefined klaus-enzenhofer @kenzenhofer