Monitoring 改造計畫：流程觀點
21 likes2,041 views
The document discusses key insights from William Yeh's presentation on monitoring in DevOps at the 2016 Summit. It emphasizes the importance of process monitoring, risk management, and leveraging technology for effective monitoring practices. Various tools and strategies are suggested to create streamlined workflows and enhance feedback loops in DevOps environments.
![sum(
irate(
node_netstat_TcpExt_TCPTimeWaitOverflow[1m]
)
) by (ec2tag_Service)
counter
gauge
aggregate
TCP Timeout
node_netstat_TcpExt_TCPTimeWaitOverflow[1m]
irate(
node_netstat_TcpExt_TCPTimeWaitOverflow[1m]
)
grouping](https://image.slidesharecdn.com/monitoring-aprocessperspective-160705155230/85/Monitoring-92-320.jpg)
![avg by (ec2tag_Service) (
irate(
node_cpu{job="node", mode="idle"}[1m]
)
)
counter
gauge
aggregate
CPU Utilization
100 - (
* 100)](https://image.slidesharecdn.com/monitoring-aprocessperspective-160705155230/85/Monitoring-94-320.jpg)
Monitoring 改造計畫：流程觀點
- 1. William Yeh DevOps Summit 2016 (2016-07-06) Monitoring Monitoring: a Process Perspective
- 3. CRT (Current Reality Tree) #6 #10 ( ) best practice #3 #5 ( ) ( )( ) AND AND AND AND #1#2 #4 #7 #8 #9 #11 Murphy exists AND AND × AND DevOps AND AND AND AND AND AND AND AND Op AND http://www.slideshare.net/williamyeh/devops-63711710
- 7. Risk management • Threats • avoid • transfer • mitigate 7 • Opportunities • exploit • enhance • share 👍👎 http://www.slideshare.net/williamyeh/whoscall-realtime-monitoring
- 8. William Yeh DevOps Summit 2016 (2016-07-06) Monitoring Monitoring: a Process Perspective
- 12. #5 Part 2
- 18. What to change To What to change How to cause the change
- 19. CRT (Current Reality Tree)
- 21. DevOps
- 24. TOC CCPM
- 25. Stephen R. Covey
- 28. What get measured, get done. Peter Drucker
- 29. Policy What get measured, get done.
- 30. Policy
- 31. Policy
- 35. What to change To What to change How to cause the change
- 36. Adrian Cockcroft
- 39. CloudFront ELB API servers MongoDB Cloud Manager CloudWatch log in S3 StatsD BigQuery
- 40. CloudFront ELB API servers MongoDB Cloud Manager CloudWatch log in S3 StatsD BigQuery
- 41. CloudFront ELB API servers MongoDB Cloud Manager CloudWatch log in S3 BigQuery
- 42. CloudFront ELB API servers MongoDB Cloud Manager CloudWatch log in S3 BigQuery
- 45. Lean Change Canvas Commitment Wins/Benefits Urgency Target State Success Criteria Vision Communication Action Change Recipients FYI: http://kojenchieh.pixnet.net/blog/post/442550432-firstthing_of_agile_promotion FYI: http://leankit.com/blog/2015/02/lean-change-method/ Monitoring Q1 (brainstorming) 2016-Jan-06 Iteration #1 TO DO LIST details Augmented
- 46. Lean Change Canvas gency Target State Success Criteria Vision Communication Action Monitoring Q1 (brainstorming) 2016-Jan-06 Iteration #1
- 47. What to change To What to change How to cause the change
- 48. Lean Change Canvas gency Target State Success Criteria Vision Communication Action Monitoring Q1 (brainstorming) 2016-Jan-06 Iteration #1 Flow Tech Monitoring
- 51. TOC
- 53. “The Three Ways” Create fast flow of work from Dev into IT Ops. Shorten and amplify feedback loops. Create a culture that simultaneously fosters 2 things: 1. continual experimentation, learning from failure. 2. repetition and practice is the prerequisite to mastery. Create fast flow of work from Dev into IT Ops. Shorten and amplify feedback loops.
- 56. VPC CloudFront ELB API servers DB Simplified version
- 57. CloudFront ELB API servers DB ELB API servers DB Microservices Simplified version
- 58. Flow
- 62. Flow
- 63. Flow
- 65. Flow
- 67. Overview
- 69. API servers
- 70. DB servers
- 72. Lean Change Canvas gency Target State Success Criteria Vision Communication Action Monitoring Q1 (brainstorming) 2016-Jan-06 Iteration #1 Flow
- 73. TOC Flow TOC
- 75. Lean Change Canvas gency Target State Success Criteria Vision Communication Action Monitoring Q1 (brainstorming) 2016-Jan-06 Iteration #1 Tech
- 76. Personal Preferences • Golang • Microservices • Composability • OSS ecosystem of server technologies
- 77. Personal Preferences • Golang • Microservices • Composability • OSS ecosystem Runtime dependency william Ansible
- 78. Personal Preferences • Golang • Microservices • Composability • OSS ecosystem Scalability Overhead
- 85. Personal Preferences • Golang • Microservices • Composability • OSS ecosystem Node/system metrics exporter AWS CloudWatch exporter Blackbox exporter Collectd exporter Consul exporter Graphite exporter HAProxy exporter InfluxDB exporter JMX exporter Memcached exporter Mesos task exporter MySQL server exporter SNMP exporter StatsD exporter cAdvisor Doorman Etcd Kubernetes-Mesos Kubernetes RobustIRC SkyDNS Weave Flux Aerospike exporter Apache exporter BIG-IP exporter BIND exporter Ceph exporter CouchDB exporter Django exporter Google's mtail log data extractor Heka dashboard exporter Heka exporter IoT Edison exporter Jenkins exporter knxd exporter Meteor JS web framework exporter Minecraft exporter module Mirth Connect exporter MongoDB exporter Munin exporter New Relic exporter Nginx metric library NSQ exporter OpenWeatherMap exporter Passenger exporter PgBouncer exporter PostgreSQL exporter PowerDNS exporter RabbitMQ exporter RabbitMQ Management Plugin exporter Rancher exporter Redis exporter RethinkDB exporter rTorrent exporter scollector exporter SMTP/Maildir MDA blackbox prober Speedtest.net exporter SQL query result set metrics exporter Ubiquiti UniFi exporter Varnish exporter Zookeeper exporter
- 86. CloudFront ELB API servers MongoDB Cloud Manager CloudWatch log in S3 StatsD BigQuery
- 87. ELB API servers MongoDB Cloud Manager CloudWatch
- 92. sum( irate( node_netstat_TcpExt_TCPTimeWaitOverflow[1m] ) ) by (ec2tag_Service) counter gauge aggregate TCP Timeout node_netstat_TcpExt_TCPTimeWaitOverflow[1m] irate( node_netstat_TcpExt_TCPTimeWaitOverflow[1m] ) grouping
- 93. gauge aggregate Memory Used 1 - node_memory_MemFree/node_memory_MemTotal grouping avg( 1 - node_memory_MemFree/node_memory_MemTotal ) by (ec2tag_Service)
- 94. avg by (ec2tag_Service) ( irate( node_cpu{job="node", mode="idle"}[1m] ) ) counter gauge aggregate CPU Utilization 100 - ( * 100)
- 95. avg( request_time_summary ) by (ec2tag_Service, quantile) summary aggregate Latency grouping Customized metrics with Fluentd plugin for Prometheus
- 96. Conclusion
- 100. William Yeh DevOps Summit 2016 (2016-07-06) Monitoring Monitoring: a Process Perspective