SlideShare a Scribd company logo
1
You Don’t Know What You Can’t See:
Network Security Monitoring in ICS
Chris Sistrunk Senior Consultant
Rob Caldwell Principal Consultant
S4x15
© Mandiant, A FireEye Company. All rights reserved. 2
Agenda
§  Overview of NSM
§  Instrumenting an ICS
§  Examples and Case Study
§  Tools
§  Conclusion
§  Questions
© Mandiant, A FireEye Company. All rights reserved.
If ICS are so
vulnerable, why
haven’t we seen
more attacks?
© Mandiant, A FireEye Company. All rights reserved. 4
Two Key Reasons
1.  Intention
2.  Visibility
© Mandiant, A FireEye Company. All rights reserved. 5
Intention
Why are targeted attacks different?
•  It’s a “Who”, not a “What”…
•  They are Professional, Organized & Well Funded…
•  If You Kick Them Out They Will Return
© Mandiant, A FireEye Company. All rights reserved. 6
Visibility
We are not looking!
“Prevention is ideal, but Detection is a must…”
© Mandiant, A FireEye Company. All rights reserved. 7
Visibility
© Mandiant, A FireEye Company. All rights reserved. 8
The IOC problem
There are numerous sources of IOCs, which are a means to describe threat
data like evidence of compromise/activity, attacker methodology, or malware.
For example, from the recent “Ongoing
Sophisticated Malware Campaign” from
ICS-CERT.
What do you do with this?
Most ICS operators have no capability to
consume IOCs, much less generate them for
“information sharing”.
Common sources of ICS IOCs are ICS-CERT, US-CERT, and
many of the recent “vendor” reports.
© Mandiant, A FireEye Company. All rights reserved. 9
Network Security Monitoring
“The collection, analysis, and escalation of indications and
warnings to detect and respond to intrusions. NSM is a way
to find intruders on your network and do something about
them before they damage your enterprise.”
- The Practice of Network Security Monitoring
Cliff Stoll
“Stalking the
Wily Hacker”
1988
Todd Herberlein
et al.
“A Network
Security
Monitor”
1990
US Air Force
Defense
Information
Systems
Agency
Lawrence
Livermore
National Lab
Early 1990s
NetRanger
RealSecure
Snort
and many
others
Late 1990s -
early 2000s
Formal
definition of
NSM
2002
© Mandiant, A FireEye Company. All rights reserved. 10
The NSM Cycle
Collection
DetectionAnalysis
•  Model for action, based on
network-derived data
•  Requires people and process, not
just technology
•  Focuses on the adversary, not the
vulnerability
© Mandiant, A FireEye Company. All rights reserved.
Methods of Monitoring
§  Network tap – physical device which relays a copy of
packets to an NSM server
§  SPAN or mirrored ports – switch configuration which
sends copies of packets to a separate port where NSM
can connect
§  Host NIC – configured to watch all network traffic flowing
on its segment
§  Serial port tap – physical device which relays serial
traffic to another port, usually requires additional
software to interpret data
Fluke Networks
Stratus Engineering
© Mandiant, A FireEye Company. All rights reserved.
Types of Data Collected
§  Full content data – unfiltered collection of packets
§  Extracted content – data streams, files, Web pages, etc.
§  Session data – conversation between nodes
§  Transaction data – requests and replies between nodes
§  Statistical data – description of traffic, such as protocol
and volume
§  Metadata – aspects of data, e.g. who owns this IP
address
§  Alert/log data – triggers from IDS tools, tracking user
logins, etc.
© Mandiant, A FireEye Company. All rights reserved.
Difficulties for NSM
§  Encrypted networks
§  Widespread NAT
§  Devices moving between network segments
§  Extreme traffic volume
§  Privacy concerns
Issues that most ICS do not face!
© Mandiant, A FireEye Company. All rights reserved.
Example ICS
14
Enterprise/IT
DMZ
Plant
Control
Web
Historian or
other DB
DCS HistorianHMI
PLCs,
Controllers,
RTUs, PACs
© Mandiant, A FireEye Company. All rights reserved.
Anatomy of an Attack
15
Over all Mandiant attack investigations,
only a little more than half of victim computers have malware on them.
While attackers often use malware to gain an initial foothold,
they quickly move to other tactics to execute their attacks.
EVIDENCE OF COMPROMISE
Initial Compromise Establish Foothold Escalate Privileges Internal Recon Complete Mission
Move
Laterally
Maintain
Presence
Unauthorized
Use of Valid
Accounts
Known &
Unknown
Malware
Command &
Control Activity
Suspicious
Network Traffic
Files Accessed
by Attackers
Valid Programs
Used for Evil
Purposes
Trace Evidence
& Partial Files
© Mandiant, A FireEye Company. All rights reserved. 16
Attacker Objectives
Attacker’s goals:
§  Damage equipment
§  Affect or steal process info
§  Cause safety or compliance issue
§  Pivot from vulnerable ICS to
enterprise
Attacker’s options:
§  Gain physical access to an ICS
host
§  Gain remote access to an ICS
host
§  Compromise a highly-privileged
client machine with access to the
ICS network
Enterprise/IT
Plant DMZ
Control
Web
Historian or
other DB
SCADA HistorianHMI
PLCs,
Controllers,
RTUs, PACs
© Mandiant, A FireEye Company. All rights reserved.
NSM Collection
17
•  Firewall Logs
•  Netflow Data
•  NIDS/HIDS
•  Full packet capture or NetFlow
•  Windows Logs and syslog
•  SNMP (CPU % etc.)
•  Alerts from security agents
(AV, whitelisting, etc.)
DMZ
Plant
Control
Web
Historian or
other DB
DCS HistorianHMI
PLCs,
Controllers,
RTUs, PACs
Enterprise/ITEnterprise technology collectors Logs and/or Agent
Network sensors Logs only
© Mandiant, A FireEye Company. All rights reserved. 18
What Are We Looking For?
§  Exceptions from baseline (e.g. A talks to B but never C)
§  “Top Talkers”
§  Unexpected connectivity (to Internet, Business network)
§  Known malicious IPs and domains
§  Logins using default accounts
§  Error messages that could correlate to vulnerabilities
§  Unusual system and firewall log entries
§  Host-based IDS or other security system alerts
§  Unexpected file and firmware updates
§  Antivirus alerts
§  And others….
© Mandiant, A FireEye Company. All rights reserved.
•  IDS alerts
•  Anomaly detection
•  Firmware updates, other
commands
•  Login with default credentials
•  High CPU or network bandwidth
•  Door alarms when nobody is
supposed to be working
•  Devices going off-line or behaving
strangely
19
NSM Detection
Analyst looks at detected anomalies
or alerts then escalates to IR
Enterprise/IT
DMZ
Plant
Control
HMI
PLCs,
Controllers,
RTUs, PACs
!
DMZ
Plant
Control
Web
Historian or
other DB
DCS HistorianHMI
PLCs,
Controllers,
RTUs, PACs
© Mandiant, A FireEye Company. All rights reserved. 20
NSM Analysis
Incident responders analyze the
detected anomalies to find evil
Enterprise/IT
DMZ
Plant
Control
HMI
PLCs,
Controllers,
RTUs, PACs
•  Application exploitation
•  Third-party connections (ex. ICCP
or vendor access)
•  ICS-specific communication
protocol attacks (ex. Modbus,
DNP3, Profinet, EtherNet/IP)
•  Remote access exploitation
•  Direct network access due to poor
physical security
•  USB-delivered malware
DMZ
Plant
Control
Web
Historian or
other DB
DCS HistorianHMI
PLCs,
Controllers,
RTUs, PACs
© Mandiant, A FireEye Company. All rights reserved. 21
Top Talkers
FlowBat characterizes NetFlow data, showing which nodes have the most traffic
Web traffic
Web traffic
NetBios
NTP
© Mandiant, A FireEye Company. All rights reserved. 22
Address Spoofing
NetworkMiner can find potential ARP spoofing (as well as many other indicators)
© Mandiant, A FireEye Company. All rights reserved.
Bro IDS Logs
Modbus
DNP3
Bro parses Modbus and DNP3 packets, ELSA consolidates Bro logs
© Mandiant, A FireEye Company. All rights reserved. 24
IDS GUIs
Alerts in Sguil of scanning activity
© Mandiant, A FireEye Company. All rights reserved. 25
Malformed Modbus
Deep packet inspection of Modbus by Wireshark
© Mandiant, A FireEye Company. All rights reserved.
Syslog
Syslog can be configured to send to the SO server, or detected in network
traffic if sent elsewhere.
© Mandiant, A FireEye Company. All rights reserved.
15 minutes of network traffic capture data revealed
external DNS requests (to some dubious hosts…)
Case Study – ICS Operator
27
© Mandiant, A FireEye Company. All rights reserved. 28
Abnormal DNS Traffic
“Strange” DNS requests originating from within the ICS
© Mandiant, A FireEye Company. All rights reserved. 29
Abnormal DNS Traffic
DNS requests shown in ELSA
© Mandiant, A FireEye Company. All rights reserved.
NSM Tools
Security Onion Linux distribution
‒  Easy to install and lots of documentation
§  Full packet capture – Tcpdump/Wireshark/NetworkMiner
§  Extracted content – Xplico/NetworkMiner
§  Session data – Bro/FlowBat
§  Transaction data – Bro
§  Statistical data – Capinfos/Wireshark
§  Meta data – ELSA (Whois)
§  Alert data – Snort, Suricata, Sguil, Snorby
Peel Back the Layers of Your Network
© Mandiant, A FireEye Company. All rights reserved.
Security Onion Tools
© Mandiant, A FireEye Company. All rights reserved. 32
Security Onion Implementation
§  Test in a lab first
§  Select suitable hardware platform
‒  More RAM is better
‒  Bigger hard drive is better (longer retention)
§  Mirrored/SPAN port on router/switch or a good network
tap
§  Select proper placement of SO sensor
‒  The Practice of Network Security Monitoring
‒  Applied Network Security Monitoring
§  Work with the right stakeholders if placing in production
© Mandiant, A FireEye Company. All rights reserved. 33
NetFlow Tools
SiLK & FlowBAT
§  Install on Security Onion with 2 scripts
§  www.flowbat.com
© Mandiant, A FireEye Company. All rights reserved. 34
Takeaways
§ You can implement NSM in ICS
today – without impacting your
operations
§ ICS IoCs are becoming more
common – need tools to look
for them
© Mandiant, A FireEye Company. All rights reserved.
People…
…the most important part of NSM!
§  Gigabytes of data and 1000s of IDS alerts are useless without
interpretation
§  Analyze data collected to understand what’s normal – and
what’s not
§  Identify adversary TTPs and act to disrupt them
Remember, adversaries are a “Who”,
not a “What
© Mandiant, A FireEye Company. All rights reserved. 36
Set the Right Goal
DETECT
ICS network
instrumented with
security technology
and monitored by
security personnel
RESPOND
Effective process for
response to ICS
cyber security
incidents
CONTAIN
Business continuity
and DR planning
consider ICS asset
compromise
Showing evidence of
conformance/compliance
Finding indications of
compromise
© Mandiant, A FireEye Company. All rights reserved.
Redefine the Win
Reconnaissance Weaponization Delivery Exploitation Installation
Command &
Control
Actions on
Objectives
http://papers.rohanamin.com/wp-content/uploads/papers.rohanamin.com/2011/08/iciw2011.pdf
Halting the attacker anywhere
in the cycle stops them from
achieving their objective
© Mandiant, A FireEye Company. All rights reserved.
§  The Cuckoo’s Egg by Cliff Stoll
https://www.youtube.com/watch?v=EcKxaq1FTac
1-hour NOVA Special (1990)
§  The Practice of Network Security Monitoring by Richard
Bejtlich
http://www.nostarch.com/nsm
§  Applied Network Security Monitoring by Chris Sanders &
Jason Smith
http://www.appliednsm.com/
§  The NSM Wiki http://nsmwiki.org
§  Security Onion distribution http://securityonion.net
NSM References/Resources
38
© Mandiant, A FireEye Company. All rights reserved. 39
Questions?
chris.sistrunk@mandiant.com
@chrissistrunk
robert.caldwell@mandiant.com
@robac3

More Related Content

PPTX
Security operation center (SOC)
PPTX
Splunk Enterprise Security
PDF
Cisco Cyber Security Essentials Chapter-1
PDF
DEF CON 23 - NSM 101 for ICS
PPTX
Security Operation Center - Design & Build
PDF
How MITRE ATT&CK helps security operations
PPTX
Fundamentals of Network security
PDF
Building a Next-Generation Security Operations Center (SOC)
Security operation center (SOC)
Splunk Enterprise Security
Cisco Cyber Security Essentials Chapter-1
DEF CON 23 - NSM 101 for ICS
Security Operation Center - Design & Build
How MITRE ATT&CK helps security operations
Fundamentals of Network security
Building a Next-Generation Security Operations Center (SOC)

What's hot (20)

PPT
Malware forensics
PPTX
IT Infrastrucutre Security
PDF
Q radar architecture deep dive
PDF
Cybersecurity in Industrial Control Systems (ICS)
PPTX
IBM Security QRadar
PDF
Overview of the Cyber Kill Chain [TM]
PDF
ETUDE ET MISE EN PLACE D’UNE SOLUTION DE GESTION DE LA SECURITE DU RESEAU : C...
PPSX
Next-Gen security operation center
PDF
2021/0/15 - Solarwinds supply chain attack: why we should take it sereously
PDF
Cyber Kill Chain Deck for General Audience
PPTX
Beginner's Guide to SIEM
PPT
Audit
 
PPTX
4_Session 1- Universal ZTNA.pptx
PPTX
An introduction to SOC (Security Operation Center)
PPTX
Zero Trust Framework for Network Security​
PPTX
Network security - Defense in Depth
PPT
Audit et sécurité des systèmes d'information
PDF
Endpoint Detection & Response - FireEye
PPTX
Zero trust deck 2020
PDF
Security operations center-SOC Presentation-مرکز عملیات امنیت
Malware forensics
IT Infrastrucutre Security
Q radar architecture deep dive
Cybersecurity in Industrial Control Systems (ICS)
IBM Security QRadar
Overview of the Cyber Kill Chain [TM]
ETUDE ET MISE EN PLACE D’UNE SOLUTION DE GESTION DE LA SECURITE DU RESEAU : C...
Next-Gen security operation center
2021/0/15 - Solarwinds supply chain attack: why we should take it sereously
Cyber Kill Chain Deck for General Audience
Beginner's Guide to SIEM
Audit
 
4_Session 1- Universal ZTNA.pptx
An introduction to SOC (Security Operation Center)
Zero Trust Framework for Network Security​
Network security - Defense in Depth
Audit et sécurité des systèmes d'information
Endpoint Detection & Response - FireEye
Zero trust deck 2020
Security operations center-SOC Presentation-مرکز عملیات امنیت
Ad

Viewers also liked (20)

PPTX
BSidesAugusta ICS SCADA Defense
PDF
Monitoring ICS Communications
PDF
Should I Patch My ICS?
PDF
Attacking and Defending Autos Via OBD-II from escar Asia
PDF
Remote Control Automobiles at ESCAR US 2015
PPTX
Vulnerability Inheritance in ICS (English)
PPTX
Dynamic Zoning Based On Situational Activity in ICS (Japanese)
PPTX
Using Assessment Tools on ICS (English)
PPTX
Hacker Halted 2016 - How to get into ICS security
PPTX
ICS Security Training ... What Works and What Is Needed (Japanese)
PDF
Active Directory in ICS: Lessons Learned From The Field
PDF
ICS/SCADA/PLC Google/Shodanhq Cheat Sheet
PDF
Lessons Learned from the NIST CSF
PPTX
Havex Deep Dive (English)
PPTX
ICS Security from the Plant Floor Up - A Controls Engineers Approach to Secur...
PPTX
The RIPE Experience
PDF
Windows Service Hardening
PPTX
Incubation of ICS Malware (English)
PDF
Accelerating OT - A Case Study
PPTX
Internet Accessible ICS in Japan (English)
BSidesAugusta ICS SCADA Defense
Monitoring ICS Communications
Should I Patch My ICS?
Attacking and Defending Autos Via OBD-II from escar Asia
Remote Control Automobiles at ESCAR US 2015
Vulnerability Inheritance in ICS (English)
Dynamic Zoning Based On Situational Activity in ICS (Japanese)
Using Assessment Tools on ICS (English)
Hacker Halted 2016 - How to get into ICS security
ICS Security Training ... What Works and What Is Needed (Japanese)
Active Directory in ICS: Lessons Learned From The Field
ICS/SCADA/PLC Google/Shodanhq Cheat Sheet
Lessons Learned from the NIST CSF
Havex Deep Dive (English)
ICS Security from the Plant Floor Up - A Controls Engineers Approach to Secur...
The RIPE Experience
Windows Service Hardening
Incubation of ICS Malware (English)
Accelerating OT - A Case Study
Internet Accessible ICS in Japan (English)
Ad

Similar to ICS Network Security Monitoring (NSM) (20)

PPTX
Where Are All The ICS Attacks?
PDF
Defcon 23 - Chris Sistrunk - nsm 101 for ics
PDF
CNIT 50: 1. Network Security Monitoring Rationale
PDF
1. Network Security Monitoring Rationale
PPTX
Managing Network Security Monitoring at Large Scale with Puppet - PuppetConf ...
PDF
Network security monitoring with open source tools
PPTX
Network Reliability Monitoring for ICS: Going Beyond NSM and SIEM
PPTX
The Internal Signs of Compromise
PDF
Lessons Learned Fighting Modern Cyberthreats in Critical ICS Networks
PPTX
Evaluación de riesgos asociados al puesto de trabajo: empleados, externos, vi...
PPT
Intrusion Detection System
PDF
FireSIGHT Management Center (FMC) slides
PPTX
Dcit 418-Slide two presentation (1).pptx
PDF
Bit defender ebook_secmonitor_print
DOCX
ebook converter DEMO Watermarks.docx
PDF
Eximbank security presentation
PDF
MMIX Peering Forum and MMNOG 2020: Packet Analysis for Network Security
PDF
stackArmor MicroSummit - Niksun Network Monitoring - DPI
PDF
Ceh v5 module 19 evading ids firewall and honeypot
PPTX
Security and-visibility
Where Are All The ICS Attacks?
Defcon 23 - Chris Sistrunk - nsm 101 for ics
CNIT 50: 1. Network Security Monitoring Rationale
1. Network Security Monitoring Rationale
Managing Network Security Monitoring at Large Scale with Puppet - PuppetConf ...
Network security monitoring with open source tools
Network Reliability Monitoring for ICS: Going Beyond NSM and SIEM
The Internal Signs of Compromise
Lessons Learned Fighting Modern Cyberthreats in Critical ICS Networks
Evaluación de riesgos asociados al puesto de trabajo: empleados, externos, vi...
Intrusion Detection System
FireSIGHT Management Center (FMC) slides
Dcit 418-Slide two presentation (1).pptx
Bit defender ebook_secmonitor_print
ebook converter DEMO Watermarks.docx
Eximbank security presentation
MMIX Peering Forum and MMNOG 2020: Packet Analysis for Network Security
stackArmor MicroSummit - Niksun Network Monitoring - DPI
Ceh v5 module 19 evading ids firewall and honeypot
Security and-visibility

More from Digital Bond (14)

PPTX
The Future of ICS Security Products
PDF
Assessing the Security of Cloud SaaS Solutions
PDF
API Training 10 Nov 2014
PDF
Unidirectional Security Appliances to Secure ICS
PDF
S4xJapan Closing Keynote
PPTX
Survey and Analysis of ICS Vulnerabilities (Japanese)
PPTX
Unsolicited Response - Getting BACnet Off of the Internet (Japanese)
PPTX
Sharing Plant Data with Phones, Tablets and the Cloud (Englsh)
PPTX
Application Whitelisting and DPI in ICS (English)
PDF
Industrial Wireless Security (Japanese)
PDF
S4x14 Session: You Name It; We Analyze It
PDF
Writing ICS Vulnerability Analysis
PDF
HART as an Attack Vector
PDF
PLC Code Protection
The Future of ICS Security Products
Assessing the Security of Cloud SaaS Solutions
API Training 10 Nov 2014
Unidirectional Security Appliances to Secure ICS
S4xJapan Closing Keynote
Survey and Analysis of ICS Vulnerabilities (Japanese)
Unsolicited Response - Getting BACnet Off of the Internet (Japanese)
Sharing Plant Data with Phones, Tablets and the Cloud (Englsh)
Application Whitelisting and DPI in ICS (English)
Industrial Wireless Security (Japanese)
S4x14 Session: You Name It; We Analyze It
Writing ICS Vulnerability Analysis
HART as an Attack Vector
PLC Code Protection

Recently uploaded (20)

PPTX
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
PDF
Review of recent advances in non-invasive hemoglobin estimation
PDF
TokAI - TikTok AI Agent : The First AI Application That Analyzes 10,000+ Vira...
PDF
How UI/UX Design Impacts User Retention in Mobile Apps.pdf
PDF
Dropbox Q2 2025 Financial Results & Investor Presentation
PDF
CIFDAQ's Market Insight: SEC Turns Pro Crypto
PDF
Unlocking AI with Model Context Protocol (MCP)
PDF
NewMind AI Monthly Chronicles - July 2025
PPTX
PA Analog/Digital System: The Backbone of Modern Surveillance and Communication
PDF
Chapter 3 Spatial Domain Image Processing.pdf
PPTX
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
PPTX
20250228 LYD VKU AI Blended-Learning.pptx
PDF
Agricultural_Statistics_at_a_Glance_2022_0.pdf
PDF
Spectral efficient network and resource selection model in 5G networks
DOCX
The AUB Centre for AI in Media Proposal.docx
PDF
NewMind AI Weekly Chronicles - August'25 Week I
PDF
Encapsulation_ Review paper, used for researhc scholars
PDF
Approach and Philosophy of On baking technology
PPTX
Detection-First SIEM: Rule Types, Dashboards, and Threat-Informed Strategy
PDF
Building Integrated photovoltaic BIPV_UPV.pdf
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
Review of recent advances in non-invasive hemoglobin estimation
TokAI - TikTok AI Agent : The First AI Application That Analyzes 10,000+ Vira...
How UI/UX Design Impacts User Retention in Mobile Apps.pdf
Dropbox Q2 2025 Financial Results & Investor Presentation
CIFDAQ's Market Insight: SEC Turns Pro Crypto
Unlocking AI with Model Context Protocol (MCP)
NewMind AI Monthly Chronicles - July 2025
PA Analog/Digital System: The Backbone of Modern Surveillance and Communication
Chapter 3 Spatial Domain Image Processing.pdf
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
20250228 LYD VKU AI Blended-Learning.pptx
Agricultural_Statistics_at_a_Glance_2022_0.pdf
Spectral efficient network and resource selection model in 5G networks
The AUB Centre for AI in Media Proposal.docx
NewMind AI Weekly Chronicles - August'25 Week I
Encapsulation_ Review paper, used for researhc scholars
Approach and Philosophy of On baking technology
Detection-First SIEM: Rule Types, Dashboards, and Threat-Informed Strategy
Building Integrated photovoltaic BIPV_UPV.pdf

ICS Network Security Monitoring (NSM)

  • 1. 1 You Don’t Know What You Can’t See: Network Security Monitoring in ICS Chris Sistrunk Senior Consultant Rob Caldwell Principal Consultant S4x15
  • 2. © Mandiant, A FireEye Company. All rights reserved. 2 Agenda §  Overview of NSM §  Instrumenting an ICS §  Examples and Case Study §  Tools §  Conclusion §  Questions
  • 3. © Mandiant, A FireEye Company. All rights reserved. If ICS are so vulnerable, why haven’t we seen more attacks?
  • 4. © Mandiant, A FireEye Company. All rights reserved. 4 Two Key Reasons 1.  Intention 2.  Visibility
  • 5. © Mandiant, A FireEye Company. All rights reserved. 5 Intention Why are targeted attacks different? •  It’s a “Who”, not a “What”… •  They are Professional, Organized & Well Funded… •  If You Kick Them Out They Will Return
  • 6. © Mandiant, A FireEye Company. All rights reserved. 6 Visibility We are not looking! “Prevention is ideal, but Detection is a must…”
  • 7. © Mandiant, A FireEye Company. All rights reserved. 7 Visibility
  • 8. © Mandiant, A FireEye Company. All rights reserved. 8 The IOC problem There are numerous sources of IOCs, which are a means to describe threat data like evidence of compromise/activity, attacker methodology, or malware. For example, from the recent “Ongoing Sophisticated Malware Campaign” from ICS-CERT. What do you do with this? Most ICS operators have no capability to consume IOCs, much less generate them for “information sharing”. Common sources of ICS IOCs are ICS-CERT, US-CERT, and many of the recent “vendor” reports.
  • 9. © Mandiant, A FireEye Company. All rights reserved. 9 Network Security Monitoring “The collection, analysis, and escalation of indications and warnings to detect and respond to intrusions. NSM is a way to find intruders on your network and do something about them before they damage your enterprise.” - The Practice of Network Security Monitoring Cliff Stoll “Stalking the Wily Hacker” 1988 Todd Herberlein et al. “A Network Security Monitor” 1990 US Air Force Defense Information Systems Agency Lawrence Livermore National Lab Early 1990s NetRanger RealSecure Snort and many others Late 1990s - early 2000s Formal definition of NSM 2002
  • 10. © Mandiant, A FireEye Company. All rights reserved. 10 The NSM Cycle Collection DetectionAnalysis •  Model for action, based on network-derived data •  Requires people and process, not just technology •  Focuses on the adversary, not the vulnerability
  • 11. © Mandiant, A FireEye Company. All rights reserved. Methods of Monitoring §  Network tap – physical device which relays a copy of packets to an NSM server §  SPAN or mirrored ports – switch configuration which sends copies of packets to a separate port where NSM can connect §  Host NIC – configured to watch all network traffic flowing on its segment §  Serial port tap – physical device which relays serial traffic to another port, usually requires additional software to interpret data Fluke Networks Stratus Engineering
  • 12. © Mandiant, A FireEye Company. All rights reserved. Types of Data Collected §  Full content data – unfiltered collection of packets §  Extracted content – data streams, files, Web pages, etc. §  Session data – conversation between nodes §  Transaction data – requests and replies between nodes §  Statistical data – description of traffic, such as protocol and volume §  Metadata – aspects of data, e.g. who owns this IP address §  Alert/log data – triggers from IDS tools, tracking user logins, etc.
  • 13. © Mandiant, A FireEye Company. All rights reserved. Difficulties for NSM §  Encrypted networks §  Widespread NAT §  Devices moving between network segments §  Extreme traffic volume §  Privacy concerns Issues that most ICS do not face!
  • 14. © Mandiant, A FireEye Company. All rights reserved. Example ICS 14 Enterprise/IT DMZ Plant Control Web Historian or other DB DCS HistorianHMI PLCs, Controllers, RTUs, PACs
  • 15. © Mandiant, A FireEye Company. All rights reserved. Anatomy of an Attack 15 Over all Mandiant attack investigations, only a little more than half of victim computers have malware on them. While attackers often use malware to gain an initial foothold, they quickly move to other tactics to execute their attacks. EVIDENCE OF COMPROMISE Initial Compromise Establish Foothold Escalate Privileges Internal Recon Complete Mission Move Laterally Maintain Presence Unauthorized Use of Valid Accounts Known & Unknown Malware Command & Control Activity Suspicious Network Traffic Files Accessed by Attackers Valid Programs Used for Evil Purposes Trace Evidence & Partial Files
  • 16. © Mandiant, A FireEye Company. All rights reserved. 16 Attacker Objectives Attacker’s goals: §  Damage equipment §  Affect or steal process info §  Cause safety or compliance issue §  Pivot from vulnerable ICS to enterprise Attacker’s options: §  Gain physical access to an ICS host §  Gain remote access to an ICS host §  Compromise a highly-privileged client machine with access to the ICS network Enterprise/IT Plant DMZ Control Web Historian or other DB SCADA HistorianHMI PLCs, Controllers, RTUs, PACs
  • 17. © Mandiant, A FireEye Company. All rights reserved. NSM Collection 17 •  Firewall Logs •  Netflow Data •  NIDS/HIDS •  Full packet capture or NetFlow •  Windows Logs and syslog •  SNMP (CPU % etc.) •  Alerts from security agents (AV, whitelisting, etc.) DMZ Plant Control Web Historian or other DB DCS HistorianHMI PLCs, Controllers, RTUs, PACs Enterprise/ITEnterprise technology collectors Logs and/or Agent Network sensors Logs only
  • 18. © Mandiant, A FireEye Company. All rights reserved. 18 What Are We Looking For? §  Exceptions from baseline (e.g. A talks to B but never C) §  “Top Talkers” §  Unexpected connectivity (to Internet, Business network) §  Known malicious IPs and domains §  Logins using default accounts §  Error messages that could correlate to vulnerabilities §  Unusual system and firewall log entries §  Host-based IDS or other security system alerts §  Unexpected file and firmware updates §  Antivirus alerts §  And others….
  • 19. © Mandiant, A FireEye Company. All rights reserved. •  IDS alerts •  Anomaly detection •  Firmware updates, other commands •  Login with default credentials •  High CPU or network bandwidth •  Door alarms when nobody is supposed to be working •  Devices going off-line or behaving strangely 19 NSM Detection Analyst looks at detected anomalies or alerts then escalates to IR Enterprise/IT DMZ Plant Control HMI PLCs, Controllers, RTUs, PACs ! DMZ Plant Control Web Historian or other DB DCS HistorianHMI PLCs, Controllers, RTUs, PACs
  • 20. © Mandiant, A FireEye Company. All rights reserved. 20 NSM Analysis Incident responders analyze the detected anomalies to find evil Enterprise/IT DMZ Plant Control HMI PLCs, Controllers, RTUs, PACs •  Application exploitation •  Third-party connections (ex. ICCP or vendor access) •  ICS-specific communication protocol attacks (ex. Modbus, DNP3, Profinet, EtherNet/IP) •  Remote access exploitation •  Direct network access due to poor physical security •  USB-delivered malware DMZ Plant Control Web Historian or other DB DCS HistorianHMI PLCs, Controllers, RTUs, PACs
  • 21. © Mandiant, A FireEye Company. All rights reserved. 21 Top Talkers FlowBat characterizes NetFlow data, showing which nodes have the most traffic Web traffic Web traffic NetBios NTP
  • 22. © Mandiant, A FireEye Company. All rights reserved. 22 Address Spoofing NetworkMiner can find potential ARP spoofing (as well as many other indicators)
  • 23. © Mandiant, A FireEye Company. All rights reserved. Bro IDS Logs Modbus DNP3 Bro parses Modbus and DNP3 packets, ELSA consolidates Bro logs
  • 24. © Mandiant, A FireEye Company. All rights reserved. 24 IDS GUIs Alerts in Sguil of scanning activity
  • 25. © Mandiant, A FireEye Company. All rights reserved. 25 Malformed Modbus Deep packet inspection of Modbus by Wireshark
  • 26. © Mandiant, A FireEye Company. All rights reserved. Syslog Syslog can be configured to send to the SO server, or detected in network traffic if sent elsewhere.
  • 27. © Mandiant, A FireEye Company. All rights reserved. 15 minutes of network traffic capture data revealed external DNS requests (to some dubious hosts…) Case Study – ICS Operator 27
  • 28. © Mandiant, A FireEye Company. All rights reserved. 28 Abnormal DNS Traffic “Strange” DNS requests originating from within the ICS
  • 29. © Mandiant, A FireEye Company. All rights reserved. 29 Abnormal DNS Traffic DNS requests shown in ELSA
  • 30. © Mandiant, A FireEye Company. All rights reserved. NSM Tools Security Onion Linux distribution ‒  Easy to install and lots of documentation §  Full packet capture – Tcpdump/Wireshark/NetworkMiner §  Extracted content – Xplico/NetworkMiner §  Session data – Bro/FlowBat §  Transaction data – Bro §  Statistical data – Capinfos/Wireshark §  Meta data – ELSA (Whois) §  Alert data – Snort, Suricata, Sguil, Snorby Peel Back the Layers of Your Network
  • 31. © Mandiant, A FireEye Company. All rights reserved. Security Onion Tools
  • 32. © Mandiant, A FireEye Company. All rights reserved. 32 Security Onion Implementation §  Test in a lab first §  Select suitable hardware platform ‒  More RAM is better ‒  Bigger hard drive is better (longer retention) §  Mirrored/SPAN port on router/switch or a good network tap §  Select proper placement of SO sensor ‒  The Practice of Network Security Monitoring ‒  Applied Network Security Monitoring §  Work with the right stakeholders if placing in production
  • 33. © Mandiant, A FireEye Company. All rights reserved. 33 NetFlow Tools SiLK & FlowBAT §  Install on Security Onion with 2 scripts §  www.flowbat.com
  • 34. © Mandiant, A FireEye Company. All rights reserved. 34 Takeaways § You can implement NSM in ICS today – without impacting your operations § ICS IoCs are becoming more common – need tools to look for them
  • 35. © Mandiant, A FireEye Company. All rights reserved. People… …the most important part of NSM! §  Gigabytes of data and 1000s of IDS alerts are useless without interpretation §  Analyze data collected to understand what’s normal – and what’s not §  Identify adversary TTPs and act to disrupt them Remember, adversaries are a “Who”, not a “What
  • 36. © Mandiant, A FireEye Company. All rights reserved. 36 Set the Right Goal DETECT ICS network instrumented with security technology and monitored by security personnel RESPOND Effective process for response to ICS cyber security incidents CONTAIN Business continuity and DR planning consider ICS asset compromise Showing evidence of conformance/compliance Finding indications of compromise
  • 37. © Mandiant, A FireEye Company. All rights reserved. Redefine the Win Reconnaissance Weaponization Delivery Exploitation Installation Command & Control Actions on Objectives http://papers.rohanamin.com/wp-content/uploads/papers.rohanamin.com/2011/08/iciw2011.pdf Halting the attacker anywhere in the cycle stops them from achieving their objective
  • 38. © Mandiant, A FireEye Company. All rights reserved. §  The Cuckoo’s Egg by Cliff Stoll https://www.youtube.com/watch?v=EcKxaq1FTac 1-hour NOVA Special (1990) §  The Practice of Network Security Monitoring by Richard Bejtlich http://www.nostarch.com/nsm §  Applied Network Security Monitoring by Chris Sanders & Jason Smith http://www.appliednsm.com/ §  The NSM Wiki http://nsmwiki.org §  Security Onion distribution http://securityonion.net NSM References/Resources 38
  • 39. © Mandiant, A FireEye Company. All rights reserved. 39 Questions? chris.sistrunk@mandiant.com @chrissistrunk robert.caldwell@mandiant.com @robac3