Open-Falcon: A Distributed and High-Performance Monitoring System
5 likes841 views
Open-Falcon is a distributed and high-performance monitoring system, popular in China, designed to overcome the limitations of existing systems like Zabbix by offering better scalability, performance, and flexibility. It features an architecture that allows for easy scaling, historical data retention, and a user-oriented interface, with abundant APIs and support for Grafana integration. The system has a growing community and ecosystem, demonstrating widespread adoption among various sectors.
![Grafana PR#3787
[feature] Add Open-Falcon datasource](https://image.slidesharecdn.com/open-falconadistributedandhigh-performancemonitoringsystem-170601133003/85/Open-Falcon-A-Distributed-and-High-Performance-Monitoring-System-3-320.jpg)
Open-Falcon: A Distributed and High-Performance Monitoring System
- 1. Open-Falcon A Distributed and High-Performance Monitoring System Yao-Wei Ou & Lai Wei 2017/05/22
- 2. Let us begin with a little story…
- 3. Grafana PR#3787 [feature] Add Open-Falcon datasource
- 4. “I'm sorry but we will not merge any new datasources unless they are very popular.” - Grafana
- 7. Open-Falcon,now is one of the most popular monitoring systems in China.
- 8. About us Laiwei ✦ Technical director of DiDi, China. ✦ Founder of Open-Falcon software. ✦ Core maintainer of Open-Falcon community. ✦ Focus on service reliability, DevOps, Cloud computing, etc. 7 Yaowei ✦ Director of WiFire oversea R&D center. ✦ Core maintainer of Open-Falcon organization. ✦ Leads the development of CDN monitoring system in Fastweb. ✦ Focus on CDN and Blockchain.
- 9. Outline ‣ Motivation ‣ Features ‣ Architecture ‣ Comparison ‣ Community 8
- 10. Motivation(1/3) There are already so many outstanding open source monitoring systems. Why do we reinvent a wheel? And how does it become the most popular monitoring system in China? 9
- 11. Motivation(2/3) ‣ Zabbix •difficult to scale out (~2000) • Database can grow very large, very quickly if not tuned properly. • not very resource friendly: a lot of connections will be made. • we have to build several zabbix cluster to deal with the rapid business grow. •a bit of a learning curve • Tuning and tweaking can be a lot of work • more complex to setup • hard to extend • manually configuration ‣ 2015 open sourced by Xiaomi SRE, China, under Apache License Version 2.0 •to replace Zabbix 10
- 12. Motivation(3/3) ‣ key factors of an enterprise class monitoring system. 11 Scalability Performance User- Oriented Flexibility High Availability
- 13. Features
- 14. Scalability ‣ Scalable monitoring system is necessary to support rapid business growth. Each module of Open-Falcon is super easy to scale horizontally. ‣ Supports up to hundreds of million transactions per minute (query/ judge/store/search). ‣ Can easily support over 100,000 hosts. 13
- 15. Performance ‣ With RRA(Round Robin Archive) mechanism, the one-year history data of 100+ metrics could be returned in just few seconds. ‣ stores 10+ years historical metrics. 14
- 16. High Availability ‣ No critical single point of failure ‣ Easy to operate and deploy 15
- 17. Flexibility ‣ Falcon-agent has already 400+ built-in server metrics. Users can collect their customized metrics by writing plugins or just simply run a script/program to relay metrics to falcon-agent. ‣ Extensive architecture. ‣ Customizable metrics. ‣ Abundant APIs. 16
- 18. Efficiency ‣ For easier management of alerting rules, Open-Falcon supports strategy, expression, template inheritance, and multiple alerting method, and callback for recovery. ‣ Auto discovery of endpoints and counters. ‣ API support. 17
- 19. User-Oriented ‣ Supports Grafana Datasource. 18 ‣ Open-Falcon could present multi-dimension graph, including user-defined dashboard/screen.
- 20. Grafana with Open-Falcon Screenshot (1/3)
- 21. Grafana with Open-Falcon Screenshot (2/3)
- 22. Grafana with Open-Falcon Screenshot (3/3)
- 23. Architecture
- 28. Center Status 24
- 29. Agent1 Agent2 Agent3 Agent4 … AgentN UICDashboard Portal2 2 2 AlarmQuery Graph 20 Transfer 60 Judge 20 5 1 HBS Sender Link 5 1 1 Before:Too many modules Nodata Aggregator Tasks Alarm FE
- 30. After
- 32. After Alarm API Gateway Graph Transfer (Queue) Judge HBS (Control) UICDashboard Portal Alarm FE Dashboard
- 33. After Central Status Alarm API Gateway Graph Transfer (Queue) Judge HBS (Control) UICDashboard Portal Alarm FE Falcon-Plus Dashboard
- 34. After Central Status Alarm API Gateway Graph Transfer (Queue) Judge HBS (Control) UICDashboard Portal Alarm FE Falcon-Plus Dashboard Agent1 Agent2 Agent3 Agent4 … AgentN
- 35. Grafana After Central Status InfluxDB Cassandra OpenTSDB ELK Alarm API Gateway Graph Transfer (Queue) Judge HBS (Control) UICDashboard Portal Alarm FE Falcon-Plus Dashboard Agent1 Agent2 Agent3 Agent4 … AgentN
- 37. Design Philosophy 27 MICROSERVICES 10 major modules individual deployed
- 38. Design Philosophy 27 PUSH agent as a push proxy MICROSERVICES 10 major modules individual deployed
- 39. Design Philosophy 27 PUSH agent as a push proxy RRDTOOL with consistent hash MICROSERVICES 10 major modules individual deployed
- 40. Design Philosophy 27 BINARY Go static binary PUSH agent as a push proxy RRDTOOL with consistent hash MICROSERVICES 10 major modules individual deployed
- 41. Design Philosophy 27 BINARY Go static binary PUSH agent as a push proxy RRDTOOL with consistent hash DEPLOYMENT mass deployment by ops-updater MICROSERVICES 10 major modules individual deployed
- 42. Design Philosophy 27 BINARY Go static binary PUSH agent as a push proxy RRDTOOL with consistent hash GRAFANA open-falcon datasource DEPLOYMENT mass deployment by ops-updater MICROSERVICES 10 major modules individual deployed
- 43. Comparison
- 44. Compared to Prometheus 29 OPEN-FALCON PROMETHEUS Abundant APIs Metrics API Push Model: Auto Discovery Pull Model: Manual configuration Easy to scale out Harder to scale out simple alert management of own dashboard Alertmanager offers grouping, deduplication and silencing functionality Faster query performance of RRA Slower, Recording rules Simple shellscript as plugin A bit learning curve to write exporter and collector Limited expression PromQL
- 45. Compared to Prometheus 29 OPEN-FALCON PROMETHEUS Abundant APIs Metrics API Push Model: Auto Discovery Pull Model: Manual configuration Easy to scale out Harder to scale out simple alert management of own dashboard Alertmanager offers grouping, deduplication and silencing functionality Faster query performance of RRA Slower, Recording rules Simple shellscript as plugin A bit learning curve to write exporter and collector Limited expression PromQL
- 46. Compared to Prometheus 29 OPEN-FALCON PROMETHEUS Abundant APIs Metrics API Push Model: Auto Discovery Pull Model: Manual configuration Easy to scale out Harder to scale out simple alert management of own dashboard Alertmanager offers grouping, deduplication and silencing functionality Faster query performance of RRA Slower, Recording rules Simple shellscript as plugin A bit learning curve to write exporter and collector Limited expression PromQL
- 47. Compared to Prometheus 29 OPEN-FALCON PROMETHEUS Abundant APIs Metrics API Push Model: Auto Discovery Pull Model: Manual configuration Easy to scale out Harder to scale out simple alert management of own dashboard Alertmanager offers grouping, deduplication and silencing functionality Faster query performance of RRA Slower, Recording rules Simple shellscript as plugin A bit learning curve to write exporter and collector Limited expression PromQL
- 48. Compared to Prometheus 29 OPEN-FALCON PROMETHEUS Abundant APIs Metrics API Push Model: Auto Discovery Pull Model: Manual configuration Easy to scale out Harder to scale out simple alert management of own dashboard Alertmanager offers grouping, deduplication and silencing functionality Faster query performance of RRA Slower, Recording rules Simple shellscript as plugin A bit learning curve to write exporter and collector Limited expression PromQL
- 49. Compared to Prometheus 29 OPEN-FALCON PROMETHEUS Abundant APIs Metrics API Push Model: Auto Discovery Pull Model: Manual configuration Easy to scale out Harder to scale out simple alert management of own dashboard Alertmanager offers grouping, deduplication and silencing functionality Faster query performance of RRA Slower, Recording rules Simple shellscript as plugin A bit learning curve to write exporter and collector Limited expression PromQL
- 50. Compared to Prometheus 29 OPEN-FALCON PROMETHEUS Abundant APIs Metrics API Push Model: Auto Discovery Pull Model: Manual configuration Easy to scale out Harder to scale out simple alert management of own dashboard Alertmanager offers grouping, deduplication and silencing functionality Faster query performance of RRA Slower, Recording rules Simple shellscript as plugin A bit learning curve to write exporter and collector Limited expression PromQL
- 51. Compared to Prometheus 29 OPEN-FALCON PROMETHEUS Abundant APIs Metrics API Push Model: Auto Discovery Pull Model: Manual configuration Easy to scale out Harder to scale out simple alert management of own dashboard Alertmanager offers grouping, deduplication and silencing functionality Faster query performance of RRA Slower, Recording rules Simple shellscript as plugin A bit learning curve to write exporter and collector Limited expression PromQL
- 52. Community
- 53. Ecosystem (1/2) 31 ✦ Banking ✦ IaaS, SaaS ✦ CDN ✦ O2O ✦ Social ✦ Entertainment ✦ …
- 54. Ecosystem (2/2) 32 OS Plugins Switch Hadoop HBase Docker Redis MongoDB GPU RabbitMQ HAProxy Nginx JMX LVS Tomcat WebSphere IIS UI
- 56. Summary
- 57. 100+ companies
- 58. 40,000+ servers
- 60. 5,000+ users
- 61. 3 seconds