Metrics stack 2.0
0 likes7,652 views
The document discusses the evolution and implementation of metrics within Vimeo, emphasizing the shift towards self-describing and standardized metrics to improve clarity and usability. It covers various concepts, including the use of tags, aggregation methods, and the need for structured metrics for better graphing and alerting. The conclusion advocates for the adoption of metrics 2.0 to enhance data interpretation and application across different systems.
![Dashboard definition
queries = [
'cpu usage sum by core',
'mem unit=B !total group by type:swap',
'stack network unit=b/s',
'unit=B (free|used) group by =mountpoint'
]](https://image.slidesharecdn.com/metrics20-140404135543-phpapp01/85/Metrics-stack-2-0-87-320.jpg)
![/etc/carbon/storageaggregation.conf
[min]
pattern = .min$
aggregationMethod = min
[max]
pattern = .max$
aggregationMethod = max
[sum]
pattern = .count$
aggregationMethod = sum
[default_average]
pattern = .*
aggregationMethod = average](https://image.slidesharecdn.com/metrics20-140404135543-phpapp01/85/Metrics-stack-2-0-98-320.jpg)
Metrics stack 2.0
- 12. “Dieter” ?
- 13. Peter Deter→
- 14. Terminology sync
- 18. ● stats.hits.vimeo_com ● stats_counts.hits.vimeo_com
- 35. Metrics 2.0
- 36. Old: ● information lacking ● fields unclear & inconsistent ● cumbersome strings / trees ● forbidden characters New: ● Selfdescribing ● Standardized ● all dimensions in orthogonal tagspace ● Allow some useful characters
- 38. Main advantages: ● Immediate understanding of metric meaning (ideally) ● Minimize time to graphs, dashboards, alerting rules
- 43. Carbontagger: ... service=foo.instance=host.target_type=gauge.type=calculatio n.unit=B 123 1234567890 … Statsdaemon: ..unit=B..unit=B... unit=B/s→ ..unit=ms..unit=ms.. unit=ms stat=mean→ → unit=ms stat=upper_90 → ...
- 52. Compare 80 traffic amongt servers site:api.vimeo.com unit=Req/s port=80 group by none avg over 10M
- 74. unit=MB
- 77. { server=dfvimeodfs1 plugin=diskspace mountpoint=_srv_node_dfs5 unit=B type=used target_type=gauge }
- 102. ● stats.timers.render_time.histogram.bin_0.01 ● stats.timers.render_time.histogram.bin_0.1 ● stats.timers.render_time.histogram.bin_1 unit=Freq_abs bin_upper=1→ ● stats.timers.render_time.histogram.bin_10 ● stats.timers.render_time.histogram.bin_50 ● stats.timers.render_time.histogram.bin_inf ● stats.timers.render_time.lower unit=ms stat=lower→ ● stats.timers.render_time.mean unit=ms stat=mean→ ● stats.timers.render_time.mean_90 ...→ ● stats.timers.render_time.median ● stats.timers.render_time.std ● stats.timers.render_time.upper ● stats.timers.render_time.upper_90
- 103. Also.. ● graphite API functions such as "cumulative", "summarize" and "smartSummarize" ● Graph renderers
- 105. From: dygraphs.com
- 113. Metric types ● gauge ● count & rate ● counter ● timer
- 118. gauge ● Multiple values in same interval ● “sticky”
- 120. Count & Rate
- 121. Counter
- 122. Timer..
- 125. Timer..
- 126. ● What should a metric be? ● Stickyness? ● Behavior on no packets received ● Behavior on multiple packets received
- 128. Conclusion ● Building graphs, setting up alerting cumbersome ● Esp. changing information needs (troubleshooting, exploring, ..) ● Esp. Complicated information needs → PAIN ● Structuring metrics ● Selfdescribing metrics ● Standardized metrics ● Native metrics 2.0 ● → BREEZE
- 129. Conclusion ● Metrics can be so much more usable and useful. Let's talk about tagging, standardisation, retaining information throughout the pipeline. ● Converting information needs into graph defs, alerting rules ● GraphExplorer, carbontagger, statsdaemon, … ● Graphiteng (native metrics 2.0) ● Metrics 2.0 in your apps, agents, aggregators? ● Build out structured metrics library