Building an IP Reputation Engine: Tracking the Miscreants
16 likes9,163 views
The document outlines the development of an IP reputation engine, detailing its purpose, features, and integration with an open-source IP reputation portal. It explains how the engine classifies and scores IP addresses based on their historical behavior, leveraging multiple data sources for accurate assessments. Additionally, it discusses current integrations with the OSSIM platform and the potential for future enhancements to the system.
Building an IP Reputation Engine: Tracking the Miscreants
- 1. Building an IP Reputation engine Tracking the miscreants
- 2. Index 1. What is IP Reputation 2. Open Source IP Reputation Portal 3. How is the engine 4. Feeding the engine 5. Current integrations
- 3. Index 1. What is IP Reputation 1.1. The problem 1.2. What is IP Reputation? 1.3. What is an IP Reputation engine? 1.4. Features of an IP Reputation engine 2. Open Source IP Reputation Portal 3. How is the engine 4. Feeding the engine 5. Current integrations
- 4. The problem Security analyst: “How many of my network connections are going to bad sites?”
- 5. What is IP Reputation? IP Reputation is a summary of the past behavior activity detected on an IP An IP with reputation information add context when a network connection is observed
- 6. What is an IP Reputation engine? An IP Reputation engine is a system to classify and score large sets of IPs, in low or high reputation
- 7. Features of an IP Reputation engine Updated information Accurate values associated to every IP Assign activity classification to every IP Range of detection
- 8. Index 1. What is IP Reputation 2. Open Source IP Reputation Portal 3. How is the engine 4. Feeding the engine 5. Current integrations
- 9. Open Source IP Reputation Portal http://labs.alienvault.com/labs/index.php/projects/open-source-ip-reputation-portal/
- 10. A register in the reputation.data file: <IP>#<RELIABILITY>#<RISK>#<ACTIVITY>#<COUNTRY>#<CITY>#<LAT>,<LON> 1...10 1...10 C&C Open Proxy Malicious Host Phishing Malware Domain Spamming Malware IP Scanning Host 64.44.240.225#4#3#Malicious Host#US#Chicago#41.9287986755,-87.6315002441 194.176.176.82#4#2#Spamming#RO#Bucharest#44.4333000183,26.1000003815 93.183.203.41#3#2#C&C;Malware Domain#UA#Kiev#50.4333000183,30.5167007446 64.141.101.204#1#2#Malware Domain#CA#Calgary#51.0833015442,-114.083297729 https://reputation.alienvault.com/reputation.data
- 11. Index 1. What is IP Reputation 2. Open Source IP Reputation Portal 3. How is the engine 3.1. Architecture design 3.1.1. Server 3.1.2. Agent 3.1.3. URL system 3.2. Scoring system 4. Feeding the engine
- 12. Architecture design Server Database Prefilter URL system Agent IPs/domains URLs Agent DATA IP reputation portal
- 13. Scoring system DNSBL + BULK DOMAINS + DYNAMIC IP DYNAMIC DNS + GOOGLE SAFE BROWSING + FILE-SHARING IP - ALEXA TOP ONE MILLION - HEURISTIC DOMAIN +
- 14. Scoring system DNSBL + $ host 6.6.6.6.zen.spamhaus.org Host 6.6.6.6.zen.spamhaus.org not BULK DOMAINS + found: 3(NXDOMAIN) DYNAMIC IP $ host 2.0.0.127.zen.spamhaus.org 2.0.0.127.zen.spamhaus.org has DYNAMIC DNS + address 127.0.0.10 2.0.0.127.zen.spamhaus.org has address 127.0.0.2 GOOGLE SAFE BROWSING + 2.0.0.127.zen.spamhaus.org has address 127.0.0.4 FILE-SHARING IP - ALEXA TOP ONE MILLION - HEURISTIC DOMAIN +
- 15. Scoring system DNSBL + *.co.be BULK DOMAINS + *.co.cc *.co.com.au DYNAMIC IP *.co.tv *.com.ua DYNAMIC DNS + *.cu.cc GOOGLE SAFE BROWSING + *.cw.cm *.cx.cc FILE-SHARING IP - *.cz.cc ALEXA TOP ONE MILLION - *.cz.tf HEURISTIC DOMAIN +
- 16. Scoring system DNSBL + BULK DOMAINS + $ host 87.216.x.x DYNAMIC IP x.x.216.87.in-addr.arpa domain name pointer x.x.216.87.dynamic.jazztel.es. DYNAMIC DNS + GOOGLE SAFE BROWSING + FILE-SHARING IP - ALEXA TOP ONE MILLION - HEURISTIC DOMAIN +
- 17. Scoring system DNSBL + BULK DOMAINS + *.ath.cx DYNAMIC IP *.dyndns.org DYNAMIC DNS + *.no-ip.biz *.no-ip.info GOOGLE SAFE BROWSING + *.no-ip.org FILE-SHARING IP - ALEXA TOP ONE MILLION - HEURISTIC DOMAIN +
- 18. Scoring system DNSBL + BULK DOMAINS + DYNAMIC IP DYNAMIC DNS + GOOGLE SAFE BROWSING + FILE-SHARING IP - ALEXA TOP ONE MILLION - HEURISTIC DOMAIN +
- 19. Scoring system DNSBL + BULK DOMAINS + DYNAMIC IP DYNAMIC DNS + GOOGLE SAFE BROWSING + FILE-SHARING IP - ALEXA TOP ONE MILLION - HEURISTIC DOMAIN +
- 20. Scoring system DNSBL + BULK DOMAINS + 1, google.com DYNAMIC IP 2, facebook.com 3, youtube.com 4, yahoo.com DYNAMIC DNS + 5, baidu.com 6, wikipedia.org GOOGLE SAFE BROWSING + 7, live.com 8, blogspot.com 9, amazon.com FILE-SHARING IP - 10, twitter.com ... ALEXA TOP ONE MILLION - 999999, panciapiatta.net 1000000, acsysun.co.jp HEURISTIC DOMAIN +
- 21. Scoring system DNSBL + BULK DOMAINS + ypyfp.com.tw jlmjalzjk.gs ewdkddr.me DYNAMIC IP xzasuf.com.pt nnis.co.uk DYNAMIC DNS + qzlx.co.za tuxs.com.ua GOOGLE SAFE BROWSING + upwcbab.tw hkwytkey.pe uzabfgqfk.my FILE-SHARING IP - http://labs.alienvault.com/labs/ index.php/2012/detecting-malware- ALEXA TOP ONE MILLION - domains-by-syntax-heuristics/ HEURISTIC DOMAIN +
- 22. Index 1. What is IP Reputation 2. Open Source IP Reputation Portal 3. How is the engine 4. Feeding the engine 4.1. External sources 4.2. Our sandnet 4.3. AlienVault OTX 5. Current integrations
- 23. Getting data from external sources { Malware Trackers Malicious Hosts lists Open Proxy lists Scanning Hosts lists SPAM Trackers and more...
- 24. Our sandnet Samples Queue Sandbox Sandnet web panel Sandnet { } Database Traffic, rules trigger Traffic, no rules trigger No traffic! IP Reputation Database
- 25. AlienVault OTX is a system for sharing threat intelligence among OSSIM users and AlienVault customers. http://www.alienvault.com/alienvault-labs/open- threat-exchange/
- 28. Index 1. What is IP Reputation 2. What is the Open Source IP Reputation Portal 3. How is the engine 4. Feeding the engine 5. Current integrations 5.1. Integration in OSSIM 5.2. Other integrations
- 29. Integration in OSSIM OSSIM is an Open Source SIEM (Security Information Event Management). A comprehensive compilation of tools that work together to provide a detailed view over each and every aspect of your networks, hosts, physical access devices, server, etc. http://communities.alienvault.com/community A security event manager (SEM) (acronyms SIEM and SIM) is a computerized tool used on enterprise data networks to centralize the storage and interpretation of logs, or events, generated by other software running on the network. http://en.wikipedia.org/wiki/Security_event_manager
- 30. { fprobe, nfSen (flow collector and analyzer) Snort (IDS) + EmergingThreats ruleset OSSEC (HIDS) Nagios (service and infrastructure monitoring) OpenVAS, Nessus (vulnerability assessment) p0f, PADS, arpwatch (passive network monitoring) nmap (network scanning) OCS Inventory NG (host-based inventory) Wireshark, tcpdump (full packet capture) and more...
- 31. { data collection with plugins: routers, firewalls, switches... load balancers, intrusion prevention systems honeypots, web proxies, web application firewalls ...
- 32. OSSIM architecture Find patterns Server Correlation engine Insert events Normalized data Sensors Database Detects new data DATA
- 33. Logic correlation if detected firewall or proxy event + and is an ACCEPT or HTTP code 200 OK event + and the destination IP has a low reputation = alarm <directive id="29001" name="Suspicious communication on SRC_IP" priority="5"> <rule type="detector" name="HTTP connection to low IP reputation destination" plugin_id="1503" plugin_sid="1" reliability="10" occurrence="1" from="HOME_NET" to="!HOME_NET" port_from="ANY" port_to="80,443" to_reputation="true" protocol="TCP"/> </directive>
- 35. Other integrations Snort reputation format Iptables format Squid format Unix (hosts.deny) format More to come: shellscripts, configuration guides, nfSen plugin...
- 36. Future of the IP reputation Live scoring API Predictive IP reputation Extent to domain blocklist
- 37. Conclusions 1. Free to use IP Reputation database 2. Detailed information about the activity and history of every IP through the web portal 3. Continuously updated and maintained using different resources and improved with AlienVault OTX 4. Fully integrated in OSSIM, ready to be easily integrated with another systems
- 38. http://labs.alienvault.com Alberto Ortega Guillermo Grande a0rtega Guillermo aortega@alienvault.com ggrande@alienvault.com