Clarkson joshua white - ids testing - spie 2013 presentation - jsw - d1
1 like818 views
This document summarizes a study that quantitatively analyzed the performance of the Snort and Suricata intrusion detection systems. The authors developed a testing framework involving over 8000 tests of the two systems using different rulesets, configurations, workloads and core counts. Initial results showed Suricata's performance dropping at 4 cores, while Snort maintained consistent single-threaded performance. Feedback from the Open Information Security Foundation and Snort.org led to configuration changes that improved Suricata's threading and implemented parallelization for Snort. Further results sections provide additional details on the comparative performance of Snort and Suricata under different testing conditions.
Clarkson joshua white - ids testing - spie 2013 presentation - jsw - d1
- 1. Quantitative Analysis of Intrusion Detection Systems: Snort and Suricata Joshua S. White Thomas T. Fitzsimmons Jeanna N. Matthews, PhD
- 2. Outline • • • • • • IDS Testing Background Snort / Suricata Our Method Analysis Results Conclusion / Future Work
- 3. Background • Given competing claims, an objective head-to-head comparison of the performance of both the Snort and Suricata Intrusion Detection Systems is needed.
- 4. Snort • Open source IDS • Open-source community and corporate support from SourceFire • Single-threaded, uses a rule-based language combining signature, protocol and anomaly inspection methods • http://www.snort.org/ * Snort, www.snort.org, Snort is a Registered Trademark of SourceFire Inc. ** Snort Logo Trademark of SourceFire Inc.
- 5. Suricata Open source IDS Open Information Security Foundation (OISF) Multi-threaded, native IPv6, Snort syntax, Unified2 output, Statistical anomaly detection, File extraction, High-speed Regex, IP reputation, Hardware and GPU Acceleration http://www.openinfosecfoundation.org/
- 6. Method • Be different than existing testing systems – PytBull, 300 Tests, Aimed at rule validation • Pytbull.sourceforge.net • Focus on testing performance – CPU, Memory, Scaling, PPS Processing • Initial system consisted of 2800 LOC written in Bash – 36 Hrs to process • Current framework 650 LOC written in Python – 6.5 Hrs to process
- 8. Test Details • Snort and Suricata • 10 x Workloads • 4 x Ruleset Configurations – Snort VRT Free, ET-Free, ET-Pro, No-Rules • 2 x IDS Configurations – Default and Optimized • 10 x Core Configurations – 1,2,3,4,5,6,8,12,18,24 • Each Test Run 5 Times • Total of 8000 tests • Additional 600 Live Replay Tests
- 9. Initial Results • Baseline tests – PPS graph – Suricata 1.2 performance drop at 4 Cores • Even when using optimized configuration – Snort consistent single threaded performance
- 10. Initial Results Continued • Suggested changes asked for advice – The OISF “Victor Julien” • Max-Pending-Packets hard coded to 1000 – Changes now include variable configuration up to USHRT_MAX (65535) • Developed on dual and quad-core systems, threading didn't consider keeping like flows together in clusters of cores – Changes now include CPU-Affinity settings in configuration files » This includes sticking like flows to single core cluster, keeps inter-CPU communication bottlenecks down
- 11. Initial Results Continued • Snort.org – Single threaded performance seemed to be major limitation • Companies like Bivio ran custom parallelized version – “Anonymous” at Sourcefire gave us tips for implementing a standard parallelized version on regular hardware » Not an easy solution to implement even for us » Somewhat buggy startup at times » Solved the single threading issue – Blogs suggested replacing standard regex (Aho-Corasick Binary NFA) with (Aho-Corasick)
- 12. Initial Results Continued • Snort.org – Single threaded performance seemed to be major limitation • Companies like Bivio ran custom parallelized version – “Anonymous” at Sourcefire gave us tips for implementing a standard parallelized version on regular hardware » Not an easy solution to implement even for us » Somewhat buggy startup at times » Solved the single threading issue – Blogs suggested replacing standard regex (Aho-Corasick Binary NFA) with (Aho-Corasick)
- 13. Results
- 14. Results
- 15. Results
- 16. Next Gen Results
- 17. Thanks! • Contact: – Joshua S. White PhD Candidate whitejs@clarkson.edu Clarkson University