Cyber Attacks Spatial Analysis
- 1. Cyber Attacks Analysis Shwetha Narayanan Insight Data Engineering Fellow New York – Summer 2017 Real Time Analysis of Cyber Attack Hotspots
- 2. Motivation
- 3. Streaming Data Source • Anti-virus software companies • Augmented data to scale • 4000 - 6000 events per minute • Scaled up to 100,000 events per minute
- 4. Streaming Data Source • Content –Attack Type •Malware •DDOS •Backdoor –Location Information of victim and attacker
- 5. Metric for Hotspot Analysis Getis - Ord
- 6. Getis - Ord • Used when you have geospatial data • Calculates statistical significant clusters based on a feature • Estimate a Gi Score for every space in the region – Higher Gi score => Significant Hotspot • Compares the feature score of current cell and it’s neighbors with sum of all feature values
- 7. Getis Ord – Gi Score • Steps to Calculate – Divide the space into cells – Accumulate attack counts in each cell – Calculate Gi Score • Blue vs Green – Blue is surrounded by cells of higher attack count 5 3 2 4 1 5 10 14 9 9 10 1 2
- 8. Interactive Query • Find events within a radius of 10 miles – Calculate Bounding box Bounding box • min(x), max(x), min(y), max(y) • Based on earth’s spherical radius at that point
- 9. Data Pipeline Cyber Attacks Streaming Data Source
- 10. Demo
- 11. Kafka Streams Technical Challenges • Streams application should provide Serializers and Deserializers to materialize the data – Read input from stream / Write to stream • Built in serializers are: String, Integer, Long, Double
- 12. How to work with other data formats?
- 13. Deserializer for other data formats
- 14. Creating a Serde - SerializerDeserializer
- 15. Kafka Streams Technical Challenges • Kafka Streams Errors – Internal Topic Error - Cannot create internal topics • User permissions to create topics - Stack Overflow • Set Group ID and Application ID • Used for co-ordinating between instances
- 16. About Me - Shwetha Narayanan • Recently graduated with Masters in Computer Science • Worked for 2 years as a Software Engineer • Co-authored a paper on “Enabling Real time crime intelligence using mobile GIS and prediction methods”, EISIC, 2013
- 18. Screenshots - Cyber Attack Trends
- 19. Getis Score - Calculation
- 20. Bounding Box calculation acos(sin(input_lat) * sin(Lat) + cos(input_lat) * cos(Lat) * cos(Lon - (input_lon))) * 6371 <= 1000;