![Search query syntax
• status:active
• title:(quick OR brown)
• title:(quick brown)
• author:"John Smith«
• book.*:(quick brown)
• _missing_:title
• _exists_:title
• qu?ck bro*
• name:/joh?n(ath[oa]n)/
• Fuzziness :
• quikc~ brwn~ foks~
• quikc~1
• Proximity seach:
• "fox quick"~5
• Ranges
• date:[2012-01-01 TO 2012-
12-31]
• count:{10 TO *] = count:>10
• Boosting
• quick^2 fox
• Boolean operators
• quick brown +fox –news
• Avoid
• ((quick AND fox) OR
(brown AND fox) OR
fox) AND NOT news
• *ing](https://image.slidesharecdn.com/elkstack-160709012616/85/ELK-Elasticsearch-Logstash-and-Kibana-Stack-for-Log-Management-16-320.jpg)
ELK Elasticsearch Logstash and Kibana Stack for Log Management
- 1. ELK STACK Master the data BY EL MAHDI BENZEKRI
- 2. PLAN Ourproblem Old solutions limits Other solutions ELK Stack overview Logstash/Elasticsearch/Kibana Demo
- 3. Who are we?
- 4. Our problem Lot of users Lots of data Lots of systems Integrated collection of logs
- 5. Old solutions limits • Tail & grep impossible with multiple systems • DBs dont scale and its difficult to extract statistics • Syslog : • Only log history • Not flexible : syslog client to syslog server
- 7. Other solutions - Biggest tool - 537 apps - High cost and complexity - Up to ~10000$ SPLUNK - Easy setup - Establish baseline and notify - Sensitive data - Data transmition overhead - Complex pricing strategy SUMO LOGIC (SAAS)
- 8. ELK Stack - Open source - Easy install - Mature components - Complexity of three different products - Diffrent machines in production
- 9. Logstash Over 200 plugins and all pluggable and mixable Unify and democratize data from different sources and into different destinations Data collection engine
- 11. Logstash popular plugins - Date - mutate - Grok - Multiline FILTER - File - Stdin - syslog INPUT - Elasticsearch - File - Mongodb - Email - stdout OUTPUT
- 12. Elasticsearch • Build on top of Apache Lucene(java) • Developer-Friendly, RESTful API • High Availability • Massively Distributed • Real-Time data and Advanced Analytics(1s from indexing to searching)
- 15. Kibana Real time dashboards Designed to interact with elasticsearch data Full JS and HTML5 analytics and visualization platform
- 16. Search query syntax • status:active • title:(quick OR brown) • title:(quick brown) • author:"John Smith« • book.*:(quick brown) • _missing_:title • _exists_:title • qu?ck bro* • name:/joh?n(ath[oa]n)/ • Fuzziness : • quikc~ brwn~ foks~ • quikc~1 • Proximity seach: • "fox quick"~5 • Ranges • date:[2012-01-01 TO 2012- 12-31] • count:{10 TO *] = count:>10 • Boosting • quick^2 fox • Boolean operators • quick brown +fox –news • Avoid • ((quick AND fox) OR (brown AND fox) OR fox) AND NOT news • *ing
- 17. Demo architecture TOMCAT Logs Accounts.json REST API File input plugin
- 18. 01110100 01101000 01100001 01101110 01101011 00100000 01111001 01101111 01110101 :D