Splunk .conf2011: Search Language: Intermediate
Download as PPT, PDF1 like1,038 views
This document provides an overview of Splunk capabilities including knowledge objects, tags, event types, saved searches, alerts, and the search pipeline. It demonstrates how to use these features to better organize and analyze IT data through examples such as monitoring server activity, detecting suspicious login attempts, and tracking software sales. Advanced searching techniques including comparison operators, stats, and transaction commands are also explained to help users leverage Splunk's powerful search language.
Splunk .conf2011: Search Language: Intermediate
- 1. Search Language - Intermediate Karen Hodges, Sr. Instructor
- 2. Karen Hodges – Senior Instructor – Splunk Over 20 years of experience in software training and education in: UNIX System Administration Intergraph GIS Systems Relational Database Management Systems BMC Remedy Mortgage Fraud Detection Real Property Title Search Splunk Your presenter . . .
- 3. Knowledge Objects Tags Event types Saved searches and alerts Advanced searching techniques Comparison operators The search pipeline Topics
- 5. Type in keywords, hit return, get results . . . Splunk as “Search Engine”
- 6. Splunk allows you to “store” knowledge alongside your IT data Institutional knowledge For example: server function or device location Learned knowledge For example: identify crash precursors or suspicious activity patterns You store these in Splunk using Knowledge Objects So Much More than a “Search Engine”
- 7. Server names aren’t always very helpful! Sometimes they pack too much information into the name Sometimes they make them reflect their hobbies/obsessions Scenario – Confusing Server Names
- 8. Tags are metadata you can add to field values Knowledge Objects – Tags to the Rescue
- 9. Search all hosts tagged as “ webfarm ” Using Tags
- 10. IT data is full of strange and confusing message Some are alarming! Some are low key, but should be alarming Scenario – So Many Different Needles and Hays
- 11. Event types are fields based on a search – similar to a saved search Knowledge Objects – Event Types
- 12. For example: 2 events in linux_secure Save event types to differentiate these 2 events pwd_fail_known and pwd_fail_unknown Event Type Example - Different Events
- 13. For example: 2 different types of firewalls CheckPoint firewall “action=reject” Netscreen firewall “action=deny” Event Type Example – Same Event
- 14. Using Event Types Use the eventtype as you would any other field
- 15. Servers and devices run 24/7 Hackers, bugs and crashes (oh my!) are lurking 24/7 Humans aren’t 24/7 – they need things like sleep, vacations, lunch, or just a few minutes away from staring at a screen in a freezing cold server room! Scenario – 24/7 Monitoring
- 16. Searches can be run on a schedule and be setup to “do something” based on the results We call these Alerts Splunk Alerts Never Sleep!
- 17. Hackers need a user name AND a password to log in to your systems Public web pages often contain names of CEOs, sales folks, etc. splunk.com is no exception Alerting Scenario – Public User Logins
- 18. Since only certain users appear on the web page, we can give those users the tag=publicID We can use the “ pwd_fail_known ” Event Type we created earlier Leverage Tagging and Event Types
- 19. Craft the search that searches for login attempts from public users then create the alert Click next to define alert conditions Craft Your Search and Create the Alert
- 20. You can specify alert conditions which will trigger the alert In our case we are looking for four or more login attempts since after that legitimate users are locked out Alert Conditions
- 21. Can send email, create RSS feed, or trigger shell script We have opted to have the results included in our email so we can evaluate the severity of the attack easily Tracking allows us to view fired alerts in the Alert manager Alert Actions
- 22. Use the Alerts menu item in the main Splunk navigation to display the Alerts manager window. Click Results to view the events that triggered the alert Click Edit to edit the alert settings Alert Manager Failed Logins Failed Logins
- 24. Comparison operators make your searches more exacting Splunk’s full-featured search language permits you to organize and analyze data in amazing ways! So Much More than a “Search Engine”- Part II
- 25. Comparison operators != > < <= >= Towards More Sophisticated Searches
- 26. Search is a data generating command You can organize and analyze data using the search pipeline The Search Pipeline sourcetype=syslog ERROR | top user | fields - percent Fetch events from disk that match Remove column showing percentage Intermediate results table Intermediate results table Final results table Disk Summarize into table of top 10 users
- 27. After the search command use the “|” symbol to pipe your search results to a subsequent command For example here we are changing the sort order to sort by user name descending – grouping all the logins together Organize and Analyze Your Data
- 28. We’ve already seen sort, there are many MANY more . . . dedup removes duplicates Weeding out duplicate entries makes results easier to use AND keeps statistical operations more pure regex allows you filter your results using a regular expression REGEX gurus can filter using all the ?’s and *’s they want! transaction allows you to group your events by a certain field and time range See all the web pages your boss visited in the past hour from your proxy data Data Processing Commands
- 29. When you type in a command after the | symbol Splunk’s Search Assistant provides an instant mini “man page” Splunk Makes Using its Search Language Easy
- 30. The table command is useful for visually organizing events Columns are displayed in the same order of fields entered in the command Column headers are field names Rows are field values Each row represents an event View Events in a Table
- 31. The top command finds the most common values of a given field Returns top 10 results by default Automatically returns a count and percentage Adding limit=# after the top command returns the specified number of results Top Scenario – Getting Top Site Visitors
- 32. count returns the number of occurrences of a given field The by clause returns a count for each field value of a named field Stats Scenario – Counting Product Sales
- 33. Online trading activity is captured in a log file which includes each trader’s unique identification Company policy requires that we monitor each trader’s activity in hourly chunks, but the trades are all jumbled up together making it hard to spot patterns in each trader’s trades Transaction Scenario – Monitor Trading Activity
- 34. Use transaction to group each trade by TradeID Set your time span to an hour and your max pause to one hour in case some traders only have one or two trades per hour Use Transaction to Group Your Trades
- 35. Event types and tags are excellent ways to capture existent knowledge as well as knowledge learned from using Splunk Splunk’s search language includes many powerful commands which allow you to organize and analyze your data easily Summary
- 36. You’ve just seen some of the many ways Splunk can be used to leverage the intelligence in your IT data Further your Splunk education with official Splunk training Using Splunk – Gets deeper into basic search, alerts, knowledge objects, quick reports and more… Searching and Reporting with Splunk – Takes you to the next level leveraging statistical operations and reporting in Splunk Congratulations!
- 37. August 15, 2011 Questions? Karen Hodges, Sr. Instructor