How to not fail at security data analytics (by CxOSidekick)
0 likes338 views
1. The document discusses the challenges of obtaining security-related data from different sources and transporting it to a central platform for analysis. It addresses questions about data volume, collection methods, filtering and formatting. 2. Setting up a security data pipeline involves determining what data to collect from various host systems, networks, and applications. Data must then be forwarded from collectors to a central platform while managing bandwidth, latency, and failures. 3. Collecting the right security-related data is vital for detecting threats and being able to investigate incidents. The document argues for collecting most available data by default and filtering out exceptions, rather than only collecting predefined types of data.
![[EPS x Avg SPE] x Seconds = Log vol for time period n](https://image.slidesharecdn.com/howtonotfailatsecuritydataanalytics-180622095211/85/How-to-not-fail-at-security-data-analytics-by-CxOSidekick-61-320.jpg)
![[EPS x Avg SPE] x Seconds = Log vol for time period n
What is the time
scale of interest?](https://image.slidesharecdn.com/howtonotfailatsecuritydataanalytics-180622095211/85/How-to-not-fail-at-security-data-analytics-by-CxOSidekick-62-320.jpg)
How to not fail at security data analytics (by CxOSidekick)
- 1. @CxOSidekick Opinions were shared Mistakes were made
- 3. 132 sides … in 15 minutes?
- 4. Theme #1: Vendors and the details they leave out
- 5. Or: How better questions avoid Pointless Offensive Concepts
- 6. More global deployments than our competitors can’t be wrong!
- 7. Hi, vaguely plausible maths guy here. We use all the machine learnings.
- 8. “The product’s good, as long as you don’t believe what the marketing tells you it’ll do; we had to invest 5 years for two analysts to build the skills to use it for real effect.”
- 9. 1. What are the specific detection use cases you cover? 2. What are their limits? 3. What are your high and low bars for false positive rates in customer deployments and why? 4. What attack techniques over what attack surfaces that you cover? 5. What specific data sources, and what fields within those data sources, are vital to solve the detection problem sets you focus on? 6. What is the ideal vs bare minimum baseline sufficient data set and coverage to deliver value? 7. What are your dependencies on config, modules and settings of technologies that deliver input? 8. What visibility do you provide of the logic available (i.e. rule sets / algos) and the rationale for your choices in how you’ve applied that logic? 9. What is the process for tuning and how adaptable is that by the customer? 10.What is the playbook for triage used by your most advanced customers? 11.How long will it be before we see value, how do you define ‘value’, and what are the conditions that must be true for you to stand behind delivering that in ‘n’ number of days?
- 10. Theme #2: Security Data Ops & why you need it
- 11. Behind the GUI, no one can hear you scream
- 13. Deliver on use cases Analytics capability Platform(s?) Data Pipeline Data
- 14. A lot of effort for a feature, not a product
- 15. Solution to problem sets Analytics capability Platform(s?) Data Pipeline Data
- 16. 8 platforms, 4 years, >20m blown, users asking for their money back
- 17. NEVER START WITH THE DASHBOARD
- 18. We are envoys, we take what is offered
- 19. AKA: in a mission where you have a clear target outcome, but an unknowable path to victory in a complex environment… 1. No plan survives 1st contact with the enemy 2. Always be adapting
- 20. Theme #3: There’s getting the data, then getting the data
- 21. Data sludge (SIEMs familiar)
- 22. We sit on top of your SIEM
- 24. Act 1 Getting the data
- 25. A pre-requisite for recognition is visibility
- 27. “My log has something to tell you.”
- 30. Wait, it gets worse
- 31. 1. What’s the level of variation / entropy across sensors? 2. How is your sensor eco-system likely to change over time? 3. What data are sensors generating currently? 4. How consistently, with what coverage across environments? 5. What are the data generation options per sensor? 6. What are the change considerations to get maximum data? 7. What volumes of data will that generate? 8. What’s are the options to transport them (stream, batch)? 9. What are the costs of that to the network? 10. How does that impact your collection criteria of ‘by default’ or ‘by exception’?
- 32. Identify Protect Detect Respond Recover Users Devices Apps Data Network
- 33. Sounil Yu
- 36. § Revenue Engines § Trade Secrets § Executive Image § Compliance § Critical Operations
- 37. Identify Protect Detect Respond Recover Users Devices Apps Data Network Business level view: “I want to detect super-users who attempt to exfiltrate trade secrets (e.g. customer lists)”
- 38. Identify Protect Detect Respond Recover Users Devices Apps Data Network
- 39. Assuming your network isn’t…
- 40. Prod environment? Digital Dev? = risk appetite = constraints on flexible working Back office apps?WAN? Trust zonesLoose Strict
- 41. How far do we promise we’ll get? How far will our data get us?
- 43. MVPD
- 44. Phase 1 Phase 2 Phase 3 Phase 4 Data sets that… … triage to give broad situational awareness … support infil / exfil detection across internal and external attack surfaces … support IR and hunting for threat actor TTPs … support proactive hunting for end-to- end adversary tradecraft Users § HR database § Badge swipe system § Cloud auth / SSO § DC AD Kerberos tickets Devices § Active directory § Vuln scanner § Endpoint software § CMDB § Domain controller events § Workstation events § Server events § Workstation and server processes with full command args § Server & Workstation config baseline § Server & Workstation PowerShell logs Apps § Static /Dynamic scan results § Application access logs § Web server logs Data § DLP alerts § SQL / Database logs Network § Web proxy § DHCP § Netflow § DNS § Firewall § Custom IDS alerts
- 45. Act 2 Getting the data
- 47. Host / Observer Collector Platform Security Data Operations User Need
- 48. Host / Observer #1 Collector #1 Platform #1 Host / Observer #2 Host / Observer #n Collector #n Platform #n Host / Observer #n Collector #2
- 49. Host / Observer #1 Collector #1 Platform #1 Host / Observer #2 Host / Observer #n Collector #n Platform #n Host / Observer #n Collector #2
- 50. not
- 51. Class of Host / Observer Collector Platform Filtered? Aggregated? Metricated? Summarized? All the data? Full collect?
- 52. Class of Host / Observer Collector Platform Filtered? Aggregated? Metricated? Summarized? All the data? What is centralized + available Richness of content Full collect?
- 54. Host Observer Collector Platform Full collect Filtered Aggregated Summarised Metricated
- 55. Host Observer Collector Platform Full collect Filtered Aggregated Summarised Metricated Storage Settings Implied Retention
- 56. Host dimensions
- 57. 1. Mechanism 2. Volume 3. Collect mode 4. Format Host
- 58. 1. Native 2. Agent 3. Role based 1. Mechanism 2. Volume 3. Collect mode 4. Format Host
- 59. 1. Win logs 2. AV logs 3. DHCP logs 1. Mechanism 2. Volume 3. Collect mode 4. Format DHCP Server
- 60. Avg/min/max/ peak a) EPS b) Size / event 1. Mechanism 2. Volume 3. Collect mode 4. Format Host
- 61. [EPS x Avg SPE] x Seconds = Log vol for time period n
- 62. [EPS x Avg SPE] x Seconds = Log vol for time period n What is the time scale of interest?
- 63. Host / Observer Collector Platform How contended is this resource, from a) my top talker device b) at the 95th percentile from all devices?
- 64. 1. Mechanism 2. Volume 3. Collect mode 4. Format 1. Push / pull a) Directly from… b) Indirectly from… 2. On demand only pull Host
- 65. Win Box Directly from… Win logs Collection Server
- 67. 1. Syslog (CEF, Key value pairs) 2. JSON 3. XML 4. Agent telemetry 5. CSV 6. … big long list 1. Mechanism 2. Volume 3. Collect mode 4. Format Host
- 69. Collector 1. Collection decisions 2. Fwd’r agent decisions 3. Spec decisions 4. Pipe decisions
- 70. No collect Full collect Filter Agg Summ Met Collector 1. Collection decisions 2. Fwd’r agent decisions 3. Spec decisions 4. Pipe decisions
- 71. But measure (or summarize, then measure) what you filter out if possible as that tells you the cost if you decided to filter in those event types in future.
- 72. Collector 1. Collection decisions 2. Fwd’r agent decisions 3. Spec decisions 4. Pipe decisions
- 73. Note: this may be contingent on the Collection Modes and Formats of sources considered under the Host section, and/or compatibility with the chosen downstream Platform
- 74. Collector 1. Collection decisions 2. Fwd’r agent decisions 3. Spec decisions 4. Pipe decisions
- 77. Do you feel lucky? Well? Do ya?
- 78. 1. Turn on all the logs 2. Tweak EPS input via config 3. Test spec 4. At fail, increase required resource 5. Measure output Pick a ‘top talker’ 1000 5000 10,000 15,000 30,000 50,000 !?,000 CPU n MEM n NET n DISK IOPS n CPU n MEM n NET n DISK IOPS n x2 x2 EPS Compression performance Utilization Delay
- 79. Level up 1. Turn on all the logs 2. Tweak EPS input via config 3. Test spec 4. At fail, increase required resource 5. Measure output Pick a ‘top talker’ 1000 5000 10,000 15,000 30,000 50,000 !?,000 CPU n MEM n NET n DISK IOPS n CPU n MEM n NET n DISK IOPS n x2 x2 EPS Compression performance Utilization Delay
- 80. Collector 1. Collection decisions 2. Fwd’r agent decisions 3. Spec decisions 4. Pipe decisions 1. Bandwidth 2. Tolerable latency 3. Fail-over
- 81. “I cannot apologize for the cost of the logs.”
- 82. Security logs were probably not factored in
- 83. Delaying/batching by only a few seconds can positively impact compression and/or aggregation
- 84. Throttle Host / Observer Collector Server Platform Muchos logs 4 hrs of logs Possible rate of send Muchos logs
- 85. Fail. Over. Host / Observer Collector Server Platform Muchos logs 4 hrs of logs Possible rate of send Muchos logs
- 87. Platform 1. Cost of data ingress / egress 2. Cost of data processing 3. Expertise to operationalize 4. Integration / API availability
- 88. “I’m technology agnostic other than when it comes to Splunk and Excel.”
- 89. Act 3 Getting me the data
- 90. 1. What is the user need? 2. What system of processes does it exist in? (i.e. context)
- 91. What problem sets drive data collection?
- 93. Scope of all possible log collection
- 94. Breaks into these broad categories Sec Tech Inter-networking Devices Apps Enrichment
- 95. § Telemetry § Alerts § Logs § Databases § Random excel spreadsheets
- 96. Let’s say we’re generating something like this Sec Tech Inter-netwk Devices Apps Enrichment Time
- 98. Sec Tech Inter-netwk Devices Apps Enrichment Dip test for ops Manual xls for 4erly reporting Time Let’s say we’re generating something like this
- 99. Of ‘available’ we’ll be centralizing, (cont. or periodically) between 0-100% Only available locally Not available For any of these
- 100. Of ‘available’ we’ll be centralizing, (cont. or periodically) between 0-100% Only available locally Not available For any of these
- 101. Compliance driven Last 30 daysLast 60 days Ops / assurance driven Only localLocal only default Logs are available Logs may be available Centralised continuous and periodic
- 103. Compliance driven Last 30 daysLast 60 days Ops / assurance driven Local only default Logs are available Logs may be available
- 104. Time to forensicate
- 105. Compliance driven Last 30 daysLast 60 days Ops / assurance driven Local only default Availability of historic logs Need for historic logs Logs are available Logs may be available x x x x x x x x x x x x x x x x x x x x x x
- 106. Compliance driven Last 30 daysLast 60 days Ops / assurance driven Local only default Availability of historic logs Need for historic logs Logs are available Logs may be available x x x x x x x x x x x x x x x x x x x x x x
- 107. X = relevant signals or …. ?
- 108. Compliance driven Last 30 daysLast 60 days Ops / assurance driven Local only default Availability of historic logs Need for historic logs Logs are available Logs may be available Maybe alerts show in SIEM for relevant X? x x x x x x x x x x x x x x x x x x x x x x
- 109. Compliance driven Last 30 daysLast 60 days Ops / assurance driven Local only default Availability of historic logs Need for historic logs Logs are available Logs may be available x x x x x x x x x x x x x x x x x x x x x x x Maybe alerts show in SIEM for relevant X?
- 110. Compliance driven Last 30 daysLast 60 days Ops / assurance driven Local only default Availability of historic logs Need for historic logs Logs are available Logs may be available x x x x x x x x x x x x x x x x x x x x x x x Alerts may show in SIEM for X
- 111. Compliance driven Last 30 daysLast 60 days Ops / assurance driven Local only default Availability of historic logs Need for historic logs Logs are available Logs may be available x x x x x x x x x x x x x x x x x x x x x x x Alerts may show in SIEM for X
- 112. Compliance driven Last 30 daysLast 60 days Ops / assurance driven Local only default Availability of historic logs Need for historic logs Logs are available Logs may be available x x x x x x x x x x x x x x x x x x x x x x x
- 113. This is incident / story driven collection.
- 114. RinseRepeat > > >
- 115. This is why ‘hunt’ today is a fundamentally challenged discipline.
- 116. “We are not focused on technical indicators. We take adversary tradecraft and codify that into a threat analytic.”
- 117. Collect everything by default. Filter or exclude by exception.