Security Onion - Brief

Editor's Notes

• #3: Security Onion is a network security monitoring system that provides full context and forensic visibility into the traffic it monitors. At it's heart it is designed to make deploying multiple complex open source tools simple via a single package, reducing what would normally take days to weeks of work to minutes. Featuring Bro IDS, your choice of Snort or Suricata, Sguil analyst console, ELSA, Squert, Snorby and capME web interfaces, and the ability to pivot from one tool to the next seamlessly provides the most effective collection of network security tools available in a single package.  
• #4: Its like having a SIEM solution, but with ALL of the associated forensic data to each logged event. NSM gives us the ability to be able to detect and respond to events by having all the necessary information at hand to investigate.
• #5: Take an IDS sig and pivot into full ascii transcript or drop into wireshark to get full details of that traffic.For example, if we’re investigating some sort of drive by malware we now have the ability to grab the traffic from the packet captures and alerts and what not .. And not only see the traffic flows, but be able to carve the files out for later analysisWe can take a windows admin off the street .. Give them this tool, and they can start responding and investigating incidents
• #6: One of the big features of Security Onion is it’s ease of installationWe can run it as a live CD just to see what all the fuss is about .. Maybe need use it to do some basic analysis or whateverThe Quick Setup process automatically configures most of the applications using Snort and Bro to monitor all network interfaces by default. This setup method is used when the IDS server and the IDS sensor are configured on the same system. The Quick Setup process also configures and enables Sguil, Squert and Snorby. Advanced Setup allows more control over the setup of Security Onion. This process is used when an analyst wants to configure a system to: • Install either a Sguil server, Sguil sensor, or both • Select either Snort or Suricata IDS engine • Selecting an IDS ruleset, Emerging Threats, Snort VRT, or both • Configure network interfaces monitored by the IDS Engine and Bro
• #7: Pulledpork is just another tool you can use to keep your snort rules up to date. Like the name says Pulledpork just pulls the Snort rules you need. Not just the VRT rules but also rules from other sources like Emerging ThreatsVRT – Sourcefire Vulnerability Research Team
• #8: Traditionally SecOnion is used as a detection only system, but there is the ability to drop traffic and turn it into an IPS of sorts. Also have the ability to use FWSnort which will pass snort rules and create automated IPTables rules to block traffic under certain conditions (someone ping sweeping or port scanning)
• #9: Expanded the alert to show some detail – sensor – alert id – date/time – source details – destination details – actual alertShow rule (know whats triggered)Show packet data (see whats actually triggered the alert)Ability to use reverse DNS (test environment was disabled)
• #10: Ascii transcript – can also pivot into wiresharkBasic email alert – can be customisedCategorisation of events – can create an analysts console based just upon a certain category of alerts (help desk or windows admins/network admins)
• #11: Security Onion comes with numerous dashboards and web interfaces that are highly customisable.If for whatever reason you find that none of the tools can produce the required output, all the raw data is stored that can be queried however you want – grepawksedcsv headache
• #12: Only shown a few tools included in SecOnionSguil provides one of the best security analyst consoles available in terms of function and utility. Squert and Snorby provide dashboards to Sguil and Snort respectively, and ELSA provides a Splunk-like interface to the log data from Bro & OSSEC.Other toolsabciphttp://sourceforge.net/projects/abcip/"A simple packet crafting tool that turns text commands into pcaps. Optionally build a DAQ and Snort can directly read commands or raw payload data - no pcap required. Packets can exhibit any flaw or anomaly desired. Syntax is flexible and powerful."argushttp://www.qosient.com/argus/"Argus is a data network transaction auditing tool that categorizes network packets that match the libpcap filter expression into a protocol-specific network flow transaction model. Argus reports on the transactions that it discovers, as periodic network flow data, that is suitable for historical and near real-time processing for forensics, trending and alarm/alerting."barnyard2http://www.securixlive.com/barnyard2/"Barnyard2 is an open source interpreter for Snort unified2 binary output files. Its primary use is allowing Snort to write to disk in an efficient manner and leaving the task of parsing binary data into various formats to a separate process that will not cause Snort to miss network traffic."bittwisthttp://bittwist.sourceforge.net/"Bit-Twist is a simple yet powerful libpcap-based Ethernet packet generator. It is designed to complement tcpdump, which by itself has done a great job at capturing network traffic."Brohttp://bro-ids.org/"Bro is a powerful network analysis framework that is much different from the typical IDS you may know."chaosreaderhttp://chaosreader.sourceforge.net/"Chaosreader is a freeware tool to fetch application data from snoop or tcpdump logs. Supported protocols include TCP, UDP, IPv4, IPv6, ICMP, telnet, FTP, HTTP, SMTP, IRC, X11, and VNC."Daemonloggerhttp://www.snort.org/snort-downloads/additional-downloads#daemonlogger"Daemonlogger™ is a packet logger and soft tap developed by Martin Roesch."driftnethttp://www.ex-parrot.com/~chris/driftnet/"Driftnet is a program which listens to network traffic and picks out images from TCP streams it observes."dsniffhttp://www.monkey.org/~dugsong/dsniff/"dsniff is a collection of tools for network auditing and penetration testing. dsniff, filesnarf, mailsnarf, msgsnarf, urlsnarf, and webspy passively monitor a network for interesting data (passwords, e-mail, files, etc.). arpspoof, dnsspoof, and macof facilitate the interception of network traffic normally unavailable to an attacker (e.g, due to layer-2 switching). sshmitm and webmitm implement active monkey-in-the-middle attacks against redirected SSH and HTTPS sessions by exploiting weak bindings in ad-hoc PKI."Dumbpighttp://leonward.wordpress.com/dumbpig/"Dumbpig is an automated bad-grammarsik detector for snort rules. It parses each rule in a file and reports on badly formatted entries, incorrect usage, and alerts to possible performance issues. It should be considered as work in progress and all users should only work with the latest code available."ELSAhttp://code.google.com/p/enterprise-log-search-and-archive/"ELSA is a centralized syslog framework built on Syslog-NG, MySQL, and Sphinx full-text search. It provides a fully asynchronous web-based query interface that normalizes logs and makes searching billions of them for arbitrary strings as easy as searching the web. It also includes tools for assigning permissions for viewing the logs as well as email based alerts, scheduled queries, and graphing."fwsnorthttp://cipherdyne.org/fwsnort/"fwsnort parses the rules files included in the SNORT ® intrusion detection system and builds an equivalent iptablesruleset for as many rules as possible. fwsnort utilizes the iptables string match module (together with a custom patch that adds a --hex-string option to the iptables user space code which is now integrated with iptables) to detect application level attacks."Hoggerhttp://code.google.com/p/hogger/"Hogger leverages nmap scan files to create a Host Attribute Table for you in the XML format that Snort needs to tune your pre-processors."hpinghttp://www.hping.org/"hping is a command-line oriented TCP/IP packet assembler/analyzer. The interface is inspired to the ping(8) unix command, but hping isn't only able to send ICMP echo requests. It supports TCP, UDP, ICMP and RAW-IP protocols, has a traceroute mode, the ability to send files between a covered channel, and many other features."httpryhttp://dumpsterventures.com/jason/httpry/"httpry is a specialized packet sniffer designed for displaying and logging HTTP traffic. It is not intended to perform analysis itself, but to capture, parse, and log the traffic for later analysis. It can be run in real-time displaying the traffic as it is parsed, or as a daemon process that logs to an output file. It is written to be as lightweight and flexible as possible, so that it can be easily adaptable to different applications."hunt"Advanced packet sniffer and connection intrusion. Hunt is a program for intruding into a connection, watching it and resetting it. Note that hunt is operating on Ethernet and is best used for connections which can be watched through it. However, it is possible to do something even for hosts on another segments or hosts that are on switched ports."inundatorhttp://inundator.sourceforge.net/"Inundator is a multi-threaded, queue-driven, anonymous intrusion detection false positives generator with support for multiple targets."labreahttp://labrea.sourceforge.net/labrea-info.html"LaBrea takes over unused IP addresses, and creates virtual servers that are attractive to worms, hackers, and other denizens of the Internet. The program answers connection attempts in such a way that the machine at the other end gets "stuck", sometimes for a very long time."mergecaphttp://www.wireshark.org/docs/man-pages/mergecap.html"Mergecap is a program that combines multiple saved capture files into a single output file specified by the -w argument. Mergecap knows how to read libpcap capture files, including those of tcpdump, Wireshark, and other tools that write captures in that format."ncathttp://nmap.org/ncat/"Ncat is a feature-packed networking utility which reads and writes data across networks from the command line."netsed"The network packet altering stream editor NetSED is small and handful utility designed to alter the contents of packets forwarded thru your network in real time. It is really useful for network hackers in following applications: black-box protocol auditing - whenever there are two or more proprietary boxes communicating over undocumented protocol (by enforcing changes in ongoing transmissions, you will be able to test if tested application is secure), fuzz-alike experiments, integrity tests - whenever you want to test stability of the application and see how it ensures data integrity, other common applications - fooling other people, content filtering, etcetc - choose whatever you want to. It perfectly fits ngrep, netcat and tcpdump tools suite."netsniff-nghttp://netsniff-ng.org/"netsniff-ng is a free, performant Linux networking toolkit."NetworkMinerhttp://www.netresec.com/?page=NetworkMiner"NetworkMiner is a Network Forensic Analysis Tool (NFAT) for Windows. NetworkMiner can be used as a passive network sniffer/packet capturing tool in order to detect operating systems, sessions, hostnames, open ports etc. without putting any traffic on the network. NetworkMiner can also parse PCAP files for off-line analysis and to regenerate/reassemble transmitted files and certificates from PCAP files."nftrackerhttps://github.com/gamelinux/nftracker"nftracker is a networks sniffing daemon that will read a pcap file or sniff a network interface and look for files that traverse your network. nftracker is session oriented, and will print out the files seen in a session."ngrephttp://ngrep.sourceforge.net/"ngrep strives to provide most of GNU grep's common features, applying them to the network layer. ngrep is a pcap-aware tool that will allow you to specify extended regular or hexadecimal expressions to match against data payloads of packets. It currently recognizes IPv4/6, TCP, UDP, ICMPv4/6, IGMP and Raw across Ethernet, PPP, SLIP, FDDI, Token Ring and null interfaces, and understands BPF filter logic in the same fashion as more common packet sniffing tools, such as tcpdump and snoop."nmaphttp://nmap.org/"Nmap ("Network Mapper") is a free and open source (license) utility for network exploration or security auditing. Many systems and network administrators also find it useful for tasks such as network inventory, managing service upgrade schedules, and monitoring host or service uptime. Nmap uses raw IP packets in novel ways to determine what hosts are available on the network, what services (application name and version) those hosts are offering, what operating systems (and OS versions) they are running, what type of packet filters/firewalls are in use, and dozens of other characteristics. It was designed to rapidly scan large networks, but works fine against single hosts. Nmap runs on all major computer operating systems, and official binary packages are available for Linux, Windows, and Mac OS X. In addition to the classic command-line Nmap executable, the Nmap suite includes an advanced GUI and results viewer (Zenmap), a flexible data transfer, redirection, and debugging tool (Ncat), a utility for comparing scan results (Ndiff), and a packet generation and response analysis tool (Nping)."oinkmasterhttp://oinkmaster.sourceforge.net/"Oinkmaster is a script that will help you update and manage your Snort rules. It is released under the BSD license and will work on most platforms that can run Perl scripts, e.g. Linux, BSD, Windows, Mac OS X, Solaris, etc. Oinkmaster can be used to update and manage the VRT licensed rules, the community rules, the bleeding-snort rules and other third party rules, including your own local rules."OSSEChttp://www.ossec.net/"OSSEC is an Open Source Host-based Intrusion Detection System. It performs log analysis, file integrity checking, policy monitoring, rootkit detection, real-time alerting and active response."ostinatohttp://code.google.com/p/ostinato/"Ostinato is an open-source, cross-platform network packet crafter/traffic generator and analyzer with a friendly GUI. Craft and send packets of several streams with different protocols at different rates."p0fhttp://lcamtuf.coredump.cx/p0f3/"P0f is a tool that utilizes an array of sophisticated, purely passive traffic fingerprinting mechanisms to identify the players behind any incidental TCP/IP communications (often as little as a single normal SYN) without interfering in any way. Version 3 is a complete rewrite of the original codebase, incorporating a significant number of improvements to network-level fingerprinting, and introducing the ability to reason about application-level payloads (e.g., HTTP)."pcapcathttp://blog.kiddaland.net/dw/pcapcat"This script reads a PCAP file and prints out all the connections in the file and gives the user the option of dumping the content of the TCP stream."ptunnelhttp://www.cs.uit.no/~daniels/PingTunnel/"Ptunnel is an application that allows you to reliably tunnel TCP connections to a remote host using ICMP echo request and reply packets, commonly known as ping requests and replies. At first glance, this might seem like a rather useless thing to do, but it can actually come in handy in some cases."Reassemblerhttp://isc.sans.edu/diary.html?storyid=13282"If you provide reassembler.py with a pcap that contains fragments, it will reassemble the packets using each of the 5 reassembly engines and show you the result."scapyhttp://www.secdev.org/projects/scapy/"Scapy is a powerful interactive packet manipulation program. It is able to forge or decode packets of a wide number of protocols, send them on the wire, capture them, match requests and replies, and much more. It can easily handle most classical tasks like scanning, tracerouting, probing, unit tests, attacks or network discovery (it can replace hping, 85% of nmap, arpspoof, arp-sk, arping, tcpdump, tethereal, p0f, etc.). It also performs very well at a lot of other specific tasks that most other tools can't handle, like sending invalid frames, injecting your own 802.11 frames, combining technics (VLAN hopping+ARP cache poisoning, VOIP decoding on WEP encrypted channel, ...), etc."sguilhttp://sguil.sourceforge.net/"Sguil (pronounced sgweel) is built by network security analysts for network security analysts. Sguil's main component is an intuitive GUI that provides access to realtime events, session data, and raw packet captures. Sguil facilitates the practice of Network Security Monitoring and event driven analysis. The Sguil client is written in tcl/tk and can be run on any operating system that supports tcl/tk (including Linux, BSD, Solaris, MacOS, and Win32)."Sniffithttp://sniffit.sourceforge.net/"SniffIt is a Distribted Sniffer System, which allows users to capture network traffic from an unique machine using a graphical client application. This feature is very useful in switched networks, where traditional sniffers only allow users to sniff their own network traffic."Snorbyhttp://snorby.org/"Snorby is a ruby on rails web application for network security monitoring that interfaces with current popular intrusion detection systems (Snort, Suricata and Sagan). The basic fundamental concepts behind Snorby are simplicity, organization and power. The project goal is to create a free, open source and highly competitive application for network monitoring for both private and enterprise use."Snorthttp://www.snort.org/"Snort® is an open source network intrusion prevention and detection system (IDS/IPS) developed by Sourcefire. Combining the benefits of signature, protocol, and anomaly-based inspection, Snort is the most widely deployed IDS/IPS technology worldwide. With millions of downloads and nearly 400,000 registered users, Snort has become the de facto standard for IPS."SnortValidatorhttp://doc.emergingthreats.net/bin/view/Main/SnortValidator"SnortValidator is a tool that analyzes snort rules and searches for certain syntactic and semantic errors. It aims to supplement Snort itself, which has a very weak error checking at some points. Hence, SnortValidator detects many things that Snort will silently accept, but that will for sure not work. Additionally, it detects some common semantic problems that indicate wrong usage of keywords that will certainly not do what you actually intended."Squerthttp://www.squertproject.org/"Squert is a web application that is used to query and view event data stored in a Sguil database (typically IDS alert data). Squert is a visual tool that attempts to provide additional context to events through the use of metadata, time series representations and weighted and logically grouped result sets. The hope is that these views will prompt questions that otherwise may not have been asked."ssldumphttp://www.rtfm.com/ssldump/"ssldump is an SSLv3/TLS network protocol analyzer. It identifies TCP connections on the chosen network interface and attempts to interpret them as SSLv3/TLS traffic. When it identifies SSLv3/TLS traffic, it decodes the records and displays them in a textual form to stdout. If provided with the appropriate keying material, it will also decrypt the connections and display the application data traffic."sslsniffhttp://www.thoughtcrime.org/software/sslsniff/"sslsniff is designed to create man-in-the-middle (MITM) attacks for SSL/TLS connections, and dynamically generates certs for the domains that are being accessed on the fly. The new certificates are constructed in a certificate chain that is signed by any certificate that is provided. sslsniff also supports other attacks like null-prefix or OCSP attacks to achieve silent interceptions of connections when possible."Suricatahttp://www.openinfosecfoundation.org/index.php/download-suricata"The Suricata Engine is an Open Source Next Generation Intrusion Detection and Prevention Engine. This engine is not intended to just replace or emulate the existing tools in the industry, but will bring new ideas and technologies to the field."tcpdumphttp://www.tcpdump.org/"Tcpdump prints out a description of the contents of packets on a network interface that match the boolean expression. It can also be run with the -w flag, which causes it to save the packet data to a file for later analysis, and/or with the -r flag, which causes it to read from a saved packet file rather than to read packets from a network interface. In all cases, only packets that match expression will be processed by tcpdump."tcpickhttp://tcpick.sourceforge.net/"tcpick is a textmode sniffer libpcap-based that can track, reassemble and reorder tcp streams. Tcpick is able to save the captured flows in different files or displays them in the terminal, and so it is useful to sniff files that are transmitted via ftp or http. It can display all the stream on the terminal, when the connection is closed in different display modes like hexdump, hexdump + ascii, only printable charachters, raw mode and so on. Available a color mode too, helpful to read and understand better the output of the program. Actually it can handle several interfaces, including ethernet cards and ppp. It is useful to keep track of what users of a network are doing, and is usable with textmode tools like grep, sed, awk."tcpreplayhttp://tcpreplay.synfin.net/"Tcpreplay is a suite of GPLv3 licensed tools written by Aaron Turner for UNIX (and Win32 under Cygwin) operating systems which gives you the ability to use previously captured traffic in libpcap format to test a variety of network devices. It allows you to classify traffic as client or server, rewrite Layer 2, 3 and 4 headers and finally replay the traffic back onto the network and through other devices such as switches, routers, firewalls, NIDS and IPS's. Tcpreplay supports both single and dual NIC modes for testing both sniffing and inline devices."tcpslicehttp://sourceforge.net/projects/tcpslice/"tcpslice is a tool for extracting portions of packet trace files generated using tcpdump's -w flag. It can combine multiple trace files, and/or extract portions of one or more traces based on time."tcpstathttp://www.frenchfries.net/paul/tcpstat/"tcpstat reports certain network interface statistics much like vmstat does for system statistics. tcpstat gets its information by either monitoring a specific interface, or by reading previously saved tcpdump data from a file."tcpxtracthttp://tcpxtract.sourceforge.net/"tcpxtract is a tool for extracting files from network traffic based on file signatures."traceroute-circlhttps://github.com/CIRCL/traceroute-circl"traceroute-circl is an extended traceroute to support the activities of CSIRT (or CERT) operators. Usually CSIRT team have to handle incidents based on IP addresses received."tsharkhttp://www.wireshark.org/docs/man-pages/tshark.html"TShark is a network protocol analyzer. It lets you capture packet data from a live network, or read packets from a previously saved capture file, either printing a decoded form of those packets to the standard output or writing the packets to a file. TShark's native capture file format is libpcap format, which is also the format used by tcpdump and various other tools."u2boathttp://www.snort.org/Part of Snort, u2boat converts unified2 files to pcaps.u2spewfoohttp://www.snort.org/Part of Snort, u2spewfoo converts unified2 files to text.udptunnelhttp://www.cs.columbia.edu/~lennox/udptunnel/"UDPTunnel is a small program which can tunnel UDP packets bi-directionally over a TCP connection. Its primary purpose (and original motivation) is to allow multi-media conferences to traverse a firewall which allows only outgoing TCP connections."Vortexhttp://sourceforge.net/projects/vortex-ids/"Vortex is a near real time IDS and network surveillance engine for TCP stream data. Vortex decouples packet capture, stream reassembly, and real time constraints from analysis. Vortex is used to provide TCP stream data to a separate analyzer program."Wiresharkhttp://www.wireshark.org/"Wireshark is a GUI network protocol analyzer. It lets you interactively browse packet data from a live network or from a previously saved capture file. Wireshark's native capture file format is libpcap format, which is also the format used by tcpdump and various other tools."xpipeshttp://sourceforge.net/projects/vortex-ids/"Utilized by Vortex to facilitate highly parallel analysis. Xpipes borrows much of its philosophy (and name) from xargs. Like xargs it reads a list of data items (very often filenames) from STDIN and is usually used in conjunction with a pipe, taking input from another program. While xargs takes inputs and plops them in as arguments to another program, xpipes takes inputs and divides them between multiple pipes feeding other programs."Xplicohttp://www.xplico.org/"The goal of Xplico is extract from an internet traffic capture the applications data contained. For example, from a pcap file Xplico extracts each email (POP, IMAP, and SMTP protocols), all HTTP contents, each VoIP call (SIP), FTP, TFTP, and so on. Xplico isn’t a network protocol analyzer. Xplico is an open source Network Forensic Analysis Tool (NFAT)."xprobe2http://xprobe.sourceforge.net/"xprobe2 is an active operating system fingerprinting tool with a different approach to operating system fingerprinting. xprobe2 relies on fuzzy signature matching, probabilistic guesses, multiple matches simultaneously, and a signature database."Zenmap
• #13: Installation takes minutes – even a Windows admin can knock this upSnort or Suricata for signature-based detection, SnortVRT and/or Emerging Threats signatures available for use.Bro IDS provides visibility into the haystack.Have full context of all activity detected from the host involved. What domains a host queries, SSL certificates it's used, files downloaded, FTP/SMTP/IRC activity? All contextual questions that can help determine whether a signature-based alert is an event or an incident.Full packet capture means that we know exactly what a host did. Sguil and its integration with other tools in Security Onion, such as Network Miner and Wireshark in addition to ELSA, Squert and Snorby via capME, allow usto look at the evidence of a network attack frame by frame exactly as it happened.It’s loaded with tonnes of tools to monitor the network. It's free, well except for the hardware. But it will help you save a lot of money you might otherwise waste on other tools. Maybe we can spend some of that money to make out analysts better??