Combining Logs, Metrics, and Traces for Unified Observability
1 like810 views
The document discusses evolving complexities in monitoring architectures and the need for a unified observability approach that integrates logs, metrics, and APM tools. It emphasizes the importance of a centralized platform that provides insights through real-time dashboards and machine learning while reducing tool silos. Additionally, it outlines Elastic's solutions for logging, metrics, and APM, promoting an open approach and transparent pricing models aligned with business value.
Combining Logs, Metrics, and Traces for Unified Observability
- 1. Logs, Metrics, and APM for Unified Observability Asawari Samant Senior Director, Product Marketing June 23, 2020
- 2. Monitoring Complexity Evolving Architectures = ↑ Monitoring Complexity
- 3. Development Team Ops: Log Monitoring Uptime Response Time Uptime Tool Ops: Infra Monitoring Web Logs App Logs Database Logs Container Logs Log Tool Ops: Service Monitoring Real User Monitoring Txn Perf Monitoring Distributed Tracing APM Tool Container Metrics Host Metrics Database Metics Network Metrics Storage Metrics Metrics Tool Status Quo: Siloed Collection of Tools
- 5. How many tools does your org currently use for monitoring your systems?
- 6. APM Data Uptime DataMetrics DataLog Data Elastic Approach to Observability Uptime Response Time Web Logs App Logs Database Logs Container Logs Real User Monitoring Txn Perf Monitoring Distributed Tracing Container Metrics Host Metrics Database Metics Network Metrics Storage Metrics Dev & Ops Teams
- 7. APM Data Uptime DataMetrics DataLog Data Elastic Approach to Observability Dev & Ops Teams Unified Data Model Unified Dashboard Unified Machine Learning Unified Alerting Unification at the data layer
- 8. Pricing aligned with business value Unified Licensing Model PER AGENT $$$$ PER HOST $$$$ PER INGEST $$$$ PER MONITOR $$$$ PER ADD-ON $$$$ • Transparent Single, unified pricing model. No add-ons. • Cloud native No problem using with container workloads and serverless. • Future proof You pay for capacity and are not locked into a specific use case.
- 10. Adopt an open approach to centralized logging Turnkey data ingestion, intuitive search interface
- 11. Turn log events into intelligence Real-time dashboards based on log data, at scale
- 12. Make logs actionable with machine learning Improve analyst efficiency: 10,000 foot view to a single log line
- 13. Meet audit requirements with log lifecycle management Index lifecycle management Policy based data management that optimizes your cluster behind the scenes Hot. Warm. Cold. Frozen.Log archival and re-hydration Robust snapshot management via API or Snapshot Management UI Cold storage with online search Specialized indices for efficient long-term retention of logs Improve analyst efficiency: 10,000 foot view to a single log line
- 15. Elastic Stack as a Metrics Store BKD trees Data structures optimized for numerical time series analysis. Columnar storage Structured data storage, resulting in compact storage and faster analytics Rollups and Index Lifecycle Management Aggregate older data into bigger time buckets Aggregations framework Analytics features to slice and dice data along various dimensions 2012 2016 2014 2018 Prometheus support Support for ingesting data from Prometheus exporters and servers 2019 Improved support for histograms Dedicated histogram data type in Elasticsearch 2020
- 16. Turnkey data onboarding 100s of data sources at your fingertips Visit elastic.co/integrations
- 17. Turn metrics into intelligence Flexible time-series analytics and data visualization
- 18. Make logs more valuable with metrics From KPIs to logs
- 19. Elastic for APM
- 20. 20 Elastic APM Elastic joins forces with Opbeat A next-generation APM solution designed for developers 2017 2018 Distributed tracing Auto-instrumentation and support for OpenTracing, W3C Trace Context header 2020 2019 Elastic APM GA & more agents Agents for Python, Node.js, Ruby, Javascript; Real User Monitoring, Java, … Enterprise-ready free and open APM ● Java ● .NET ● Node.js ● Javascript Language Support ● Python ● Ruby ● Go ● PHP (in dev) • Turnkey agents • Auto-instrumentation for common frameworks • Designed to be lightweight
- 21. 21 Elastic APM Elastic joins forces with Opbeat A next-generation APM solution designed for developers 2017 2018 Distributed tracing Auto-instrumentation and support for OpenTracing, W3C Trace Context header 2020 2019 Elastic APM GA & more agents Agents for Python, Node.js, Ruby, Javascript; Real User Monitoring, Java, … Service Map, annotations Fully features user interface for navigating APM data Enterprise-ready free and open APM
- 22. Track transactions from browser to backend End-to-end distributed tracing
- 23. Understand your dependencies in real time Dependency mapping
- 24. Reduce MTTR by streamlining analyst workflow Navigate traces, metrics, and logs in one UI for faster issue resolution
- 25. Avoid vendor lock-in with open source APM Agents Support for open standards - Jaeger, OpenTracing, OpenMetrics, W3C Trace context
- 26. 26 Demo
- 27. What now? Try it yourself!
- 28. Next up: Workplace Search?