Graylog
2 likes1,934 views
The document is a presentation by Diwakar Upadhyay introducing Graylog, a free and open-source log management system that addresses issues present in existing monitoring solutions. It outlines the system's architecture, how to send logs using various protocols, and use cases illustrating its ability to extract and analyze log data. The presentation highlights Graylog's features, including pattern detection, interactive visualization, and dynamic queries.
Graylog
- 2. About me Diwakar Upadhyay 26 Year old, Lead Engineer at KelltonTech Gurugram India Skype:derrickindia
- 3. Agenda 1. Why Graylog? 2. What is Graylog? 3. How to send your logs? 4. Architecture 5. Extract Information & Analytics 5. FAQ
- 4. Why Graylog ? Free and open source log management system Existing monitoring solutions (Nagios,Zabbix) have problems : -Some of them lack of APIs -Ingegration with configration managment is time consuming -Do not scale well -High Availability is not considered by the System Architecture Pattern detection,interactive visualization,dynamic queries,anomaly detection,more sharing
- 6. What does Graylog? Receives messages from muliple input protocols GELF via HTTP/UDP/TCP,Syslog, .... Assign messages to stream Trigger user-defined alerts per streams Stores messages in ElasticSearch for graphing Provides messages to different outputs based on streams Uses MongoDB to store metadata and alerts
- 7. How to send your logs? 1.Classic syslog via TCP/UDP 2.GELF via TCP/UDP 3.Write your own input plugin
- 8. GELF Graylog Extended Log Format – Lets you structure your logs Many libraries for different systems and language available
- 9. Example GELF message { 'short_message': 'Something went wrong', 'host':'some-host-1.example.org', 'full_message':'Stacktrace and message', 'file':'some controller', 'line':7, '_from_load_balancer':'lb-3', '_user_id':9001, '_http_response_code':500 }
- 11. Have fields like user_id,http_response_code, processed_controller,processed_action, ... Use Case 1 Give your in-house developers access to the logs they produce : _oauth_consumer_key = ‘acbd18db4cc2f85cedef654fccc4a4d8’ _http_response_code = ^(4|5).* Extract Information & Analytics
- 12. ...and let them see the errors they produce with one click on the “Errors from app” stream. _http_response_code = 404 _from_lb = ‘lb-3’ all.distribution({_from_lb}, _http_response_code = 404) > lb-1 (5732), lb-2 (69), lb-3 (45), lb-4 (22) How many signups did you have today? all.count(_processed_controller = ‘Session’, _processed_action = ‘create’, _http_return_code = 301) > 294358 ...and how many failed? Graph this! all.count(_processed_controller = ‘Session’, _processed_action = ‘create’, _http_return_code = 200) > 10452
- 13. ThanksDiwakar