Monitoring HOWTOs
1 like584 views
This document discusses various tools that can help provide precise information about application health and anomalies, alert the right people, and enable automation. It describes how Sensu can query Nginx to check application status and meet SLAs. Anodot detects anomalies in metrics over time for backend services. Storm and Esper enable real-time anomaly detection from streaming data for frontend services. BigPanda provides a central place for alerts and change management. The goal is to quickly identify issues, see their potential causes, and ensure the right people are notified.
Monitoring HOWTOs
- 1. Precise information Alert the right person Automation
- 2. Service is alive • Is my application alive on the minimum number required by my SLA? • 2 out of 5 instances of my-app are not responding to isAlive • my-app requires a minimum of 3 instances to meet the SLA
- 3. Alert Sensu Queries Nginx Alert & SLA ZooKeeper Planned Configuration Service owner Nginx Service Load Balancer Is-alive
- 4. Alert Sensu Queries Nginx Alert & SLA ZooKeeper Planned Configuration Service owner Nginx Service Load Balancer Is-alive Alert the right person Precise information Automation
- 5. Service anomalies BECKEND • Identify unhealthy KPIs per end-points • Abnormal increase in error rate for class.method.get • Abnormal increase in performance or decrease in RPM
- 6. Anomaly Alert Anodot Time series anomaly detection Alerts & graphs statsd Stats aggregation Forwarding metrics JVM servers Metrics library metrics / 1m Graphs
- 7. Anomaly Alert Anodot Time series anomaly detection Alerts & graphs statsd Stats aggregation Forwarding metrics JVM servers Metrics library metrics / 1m Graphs Precise information Alert the right person Automation
- 8. Service anomalies FRONTEND • Users effected or not? How and where? • Success event count is low • Client error increased in Canada from Chrome
- 9. Anomaly Alert STORM & ESPER Realtime streaming processing Metrics / 1m Client flume events Graphs Client flume events Anodot Time series anomaly detection Alerts & graphs
- 10. Anomaly Alert STORM & ESPER Realtime streaming processing Metrics / 1m Client flume events Graphs Client flume events Precise information Alert the right person Automation Anodot Time series anomaly detection Alerts & graphs
- 11. Alerts management • Which active alerts do I have? • What changes could cause the problem? • I have 2 active alerts on MySql and 2 deployments in the last hour
- 12. Alert BigPanda Central alerts & changes Alerts & Changes Changes Integrations Deployments Chef uploads Alerts integrations NewRelic Sensu Nagios PingDom Web UI
- 13. Alert BigPanda Central alerts & changes Alerts & Changes Changes Integrations Deployments Chef uploads Alerts integrations NewRelic Sensu Nagios PingDom Precise information Alert the right person Automation Precise information Web UI
- 14. Questions?