SplunkLive! Zürich 2016 - Getting Started
0 likes313 views
This document provides an overview and introduction to Splunk. It discusses what Splunk is, how to get started with Splunk including downloading, installing and the basic web interface. It also covers searching machine data, extracting fields, alerts and dashboards. The document then discusses deployment and integration topics such as scaling Splunk, forwarding data, roles, indexing and searching across locations. It concludes with information on support and the Splunk community.
![Extract Fields
Interactive Field Extractor
props.conf
[mysourcetype]
REPORT-myclass = myFields
transforms.conf
[myFields]
REGEX = ^(w+)s
FORMAT = myFieldLabel::$1
Configuration File
• manual field extraction
• delim-based extractions
Rex Search Command
... | rex field=_raw "From: (?<from>.*)
To: (?<to>.*)"](https://image.slidesharecdn.com/gettingstartedwithsplunk-breakoutsession-alain-160512092138/85/SplunkLive-Zurich-2016-Getting-Started-29-320.jpg)
SplunkLive! Zürich 2016 - Getting Started
- 1. Copyright © 2016 Splunk Inc. Alain Gutknecht Senior Sales Engineer alain@splunk.com Getting Started
- 10. Install Splunk Start Splunk • WIN: Program FilesSplunkbinsplunk.exe start (services start) • *NIX: /opt/splunk/bin/splunk start www.splunk.com/download • 32 or 64 Bit? • Indexer or Universal Forwarder?
- 12. Splunk Licenses Free Download Limits Indexing to 500MB/day • Enterprise Trial License expires after 60 days • Reverts to Free License Features Disabled in Free License • Multiple user accounts and role-based access controls • Distributed search • Forwarding to non-Splunk Instances • Deployment management • Scheduled saved searches and alerting • Summary indexing Other License Types • Enterprise, Forwarder, Trial
- 13. Default installation on: http://localhost:8000 Splunk Web Basics Browser Support • Internet Explorer 9, 10 and 11 • Firefox (latest) • Safari (latest) • Chrome (latest)
- 14. Splunk Web Basics continued… Splunk Home • Provides Interactive portal to the Apps & data. • Explore Splunk Enterprise: 1 – Product Tours 2 – Add Data 3 – Splunk Apps 4 – Splunk Docs Splunk Apps • Default Search & Reporting App • Provide different contexts for your data out of sets of views, dashboards, and configurations • You can create your own!
- 15. Optional: add some test data Download the sample file, follow this link and save the file to your desktop, then unzip: http://www.splunkbook.com (Using Splunk Book) To add the file to Splunk: – From the Welcome screen, click Add Data. – Click From files and directories on the bottom half of the screen. – Select Skip preview. – Click the radio button next to Upload and index a file. – Click Save.
- 16. Search Basics
- 17. current view global stats app navigation time range picker Selecting Data Summary: • Host • Source • Sourcetype start search search box
- 18. Searching Search > * Select Time Range • Historical, custom, or real-time Select Mode • Smart, Fast, Verbose Using the timeline • Click events and zoom in and out • Click and drag over events for a specific range
- 19. Everything is searchable Everything is searchable • * wildcards supported • Search terms are case insensitive • Booleans AND, OR, NOT – Booleans must be uppercase – Implied AND between terms – Use () for complex searches • Quote phrases fail* fail* nfs error OR 404 error OR failed OR (sourcetype=access_*(500 OR 503)) "login failure"
- 20. Example Search:
- 21. Search Assistant Contextual Help - advanced type-ahead History - search - commands Search Reference - short/long description - examples suggests search terms updates as you type shows examples and help toggle off / on
- 22. Searches can be managed as asynchronous processes Jobs can be • Scheduled • Moved to background tasks • Paused, stopped, resumed, finalized • Managed • Archived • Cancelled Job Management Modify Job Settings pause finalize delete
- 23. Search Commands Search > error | head 1 Search results are “piped” to the command Commands for: • Manipulating fields • Formatting • Handling results • Reporting
- 24. Over 130 Commands! splunk.com > Documentation > Search Reference abstract accum addcoltotals addinfo addtotals af analyzefields anomalies anomalousvalue append appendcols ar associate audit autoregress bin bucket chart cluster collect common contingency convert correlate counttable crawl ctable dbinspect dedup delete delta diff discretize erex eval eventcount eventstats excerpt extract file fillnull folderize format gentimes head highlight iconify input inputcsv inputlookup iplocation join kmeans kv kvform loadjob localize localop lookup macro makecontinuous makemv maketable map metadata multikv mvcombine mvexpand nomv outlier outlierfilter outputcsv outputlookup outputtext overlap rangemap rare regex relevancy rename replace reverse run savedsearch savedsplunk script scrub selfjoin sendemail set sichart sirare sistats sitimechart sitop slc stash strcat streamstats sumindex summaryindex tail test timechart top transaction transam trendline typeahead typelearner typer uniq untable xmlkv xmlunescape xpath xyseries http://www.splunk.com/base/Documentation/latest/SearchReference/SearchCheatsheet
- 26. Fields Default fields • host, source, sourcetype, linecount, etc. • View on left panel in search results or all in field picker Where do fields come from? • Pre-defined by sourcetypes • Automatically extracted key-value pairs • User defined
- 27. Sources, Sourcetypes, Hosts • Host - hostname, IP address, or name of the network host from which the events originated • Source - the name of the file, stream, or other input • Sourcetype - a specific data type or data format
- 28. Extract Fields Interactive Field Extractor • Regular Expression or Delimeteres • Creates Regular Expression for you! • preview/validate
- 29. Extract Fields Interactive Field Extractor props.conf [mysourcetype] REPORT-myclass = myFields transforms.conf [myFields] REGEX = ^(w+)s FORMAT = myFieldLabel::$1 Configuration File • manual field extraction • delim-based extractions Rex Search Command ... | rex field=_raw "From: (?<from>.*) To: (?<to>.*)"
- 30. Tagging and Event Typing Eventtypes for more human-readable reports • to categorize and make sense of mountains of data • punctuation helps find events with similar patterns Search > eventtype=failed_login instead of Search > “failed login” OR “FAILED LOGIN” OR “Authentication failure” OR “Failed to ………………authenticate user” Tags are labels • apply ad-hoc knowledge • create logical divisions or groups • tag hosts, sources, fields, even eventtypes Search > tag=web_servers instead of Search > host=“apache1.splunk.com” OR host=“apache2.splunk.com” OR …………….host=“apache3.splunk.com”
- 32. Saved Searches Leverage Searches for future Insights! • Reports • Dashboards • Alerts • Eventtypes Add a Time Range Picker • Preset • Relative • Real-time • Date-Range • Date & Time Range • Advanced
- 34. Alerting Continued… Searches run on a schedule and fire an alert • Example: Run a search for “Failed password” every 15 min over the last 15 min and alert if the number of events is greater than 10 Searches are running in real-time and fire an alert • Example: Run a search for “Failed password user=john.doe” in a 1 minute window and alert if an event is found
- 35. Alerting Actions • Send email • Execute a script • Webhook • Create your own custom Alert Action!
- 37. Reporting results of any search Define your Search and set your time range, accelerate you search and more Choose the type of chart (line, area, column, etc) and other formatting options Build reports from
- 38. Reporting Examples • Use wizard or reporting commands (timechart, top, etc) • Build real-time reports with real-time searches • Save reports for use on dashboards
- 41. Manager Settings For All of that Cool Stuff You Just Created (and more!) • Permissions • Saved Searches/Reports • Custom Views • Distributed Splunk • Deployment Server • License Usage….
- 45. Understanding the Universal Forwarder Forward data without negatively impacting production performance. Universal Forwarder Regular (Heavy) Forwarder Monitor All Supported Inputs ✔ ✔ Routing, Filtering, Cloning ✔ ✔ Splunk Web ✔ Python Libraries ✔ Event Based Routing ✔ Scripted Inputs ✔ ✔ Heavy Forwarder is nothing else than a normal Splunk Instance with activated forwarding.
- 48. Delivers Mission-Critical Availability • Data replication – maintain searchability even if servers go down • Multi-site capable – maintain searchability even if a site goes down • Search Affinity – optimized searches by fetching from the closest/fastest location REPLICATION Portland Datacenter New York Datacenter Clustering
- 51. Integrate Users and Roles Problem Investigation Problem Investigation Problem Investigation Save Searches Share Searches LDAP, AD Users and Groups Splunk Flexible Roles Manage Users Manage Indexes Capabilities & Filters NOT tag=PCI App=ERP … Map LDAP & AD groups to flexible Splunk roles. Define any search as a filter. Integrate authentication with LDAP and Active Directory.
- 52. FrozenWARM COLDHOT Index How the Data is Stored and Aged Hot – Newest buckets of data that are still open for write Warm – Recent data but closed for writing (read only) Cold – Oldest data, commonly on cheaper, slower storage Frozen –No longer searchable, commonly archived or deleted data
- 55. Thank You