IDS with Artificial Intelligence
3 likes4,492 views
The document discusses using artificial intelligence techniques like neural networks and fuzzy logic for intrusion detection systems. It proposes a neural network-based IDS to detect denial of service and port scan attacks. Measures from physics are used to quantify network traffic patterns. Fuzzy logic is also discussed as a way to implement intrusion detection using approximate human-like reasoning. The goal is to create flexible and adaptive IDS that can recognize new attack patterns.
IDS with Artificial Intelligence
- 1. Intrusion Detection System with Artificial Intelligence Mario Castro Ponce Universidad Pontificia Comillas de Madrid FIST Conference - June 2004 edition Sponsored by: MLP Private Finance IDS with AI marioc@dsi.icai.upco.es FIST Conference - june 2004 edition– 1/28
- 2. Aim of the talk 1. Showing you a different approach to Intrussion Detection based on Artificial Intelligence 2. Contact experts in the field to exchange ideas and maybe creating a (pioneer!!!!) working group IDS with AI marioc@dsi.icai.upco.es FIST Conference - june 2004 edition– 2/28
- 3. Sketch of the talk What is an IDS? Architecture of a Vulnerability Detector Why using A.I.? Neurons and other animals Neural-IDS Fuzzy-Correlator Conclusions IDS with AI marioc@dsi.icai.upco.es FIST Conference - june 2004 edition– 3/28
- 4. What is an IDS? Any hardware, software, or combination of thereof that monitors a system or network of systems for malicious activity IDS with AI marioc@dsi.icai.upco.es FIST Conference - june 2004 edition– 4/28
- 5. What is an IDS? Any hardware, software, or combination of thereof that monitors a system or network of systems for malicious activity Main functions Dissuade Prevent Documentate IDS with AI marioc@dsi.icai.upco.es FIST Conference - june 2004 edition– 4/28
- 6. What is an IDS? Any hardware, software, or combination of thereof that monitors a system or network of systems for malicious activity Main functions Dissuade Prevent Documentate Two kinds of IDS Host based Network based IDS with AI marioc@dsi.icai.upco.es FIST Conference - june 2004 edition– 4/28
- 7. Architecture of a Vulnerability Detector Example: OSSIM n IDS with AI marioc@dsi.icai.upco.es FIST Conference - june 2004 edition– 5/28
- 8. Why using AI? The system manager nightmare: The false positives. IDS with AI marioc@dsi.icai.upco.es FIST Conference - june 2004 edition– 6/28
- 9. Why using AI? The system manager nightmare: The false positives. Then? A.I. for three main reasons Flexibility (vs threshold definition) Adaptability (vs specific rules) Pattern recognition (and detection of new patterns) IDS with AI marioc@dsi.icai.upco.es FIST Conference - june 2004 edition– 6/28
- 10. Why using AI? The system manager nightmare: The false positives. Then? A.I. for three main reasons Flexibility (vs threshold definition) Adaptability (vs specific rules) Pattern recognition (and detection of new patterns) Moreover Fast computing (faster than humans, actually) Learning abilities. IDS with AI marioc@dsi.icai.upco.es FIST Conference - june 2004 edition– 6/28
- 11. Neurons and other animals AI TOOLS Neural Networks Fuzzy Logic Other... IDS with AI marioc@dsi.icai.upco.es FIST Conference - june 2004 edition– 7/28
- 12. Artificial Neural networks Change of paradigm in computing science: Many dummy processors with a simple task to do against one (or few) powerful versatile processors IDS with AI marioc@dsi.icai.upco.es FIST Conference - june 2004 edition– 8/28
- 13. Neurons and artificial neurons IDS with AI marioc@dsi.icai.upco.es FIST Conference - june 2004 edition– 9/28
- 14. Main types of ANN Multilayer perceptrons OUTPUT LAYER INPUT LAYER HIDDEN LAYER Self-organized maps Radial basis neural networks Other IDS with AI marioc@dsi.icai.upco.es FIST Conference - june 2004 edition– 10/28
- 15. Neural IDS Designed for DoS and port scan attacks IDS based on a multilayer perceptron IDS with AI marioc@dsi.icai.upco.es FIST Conference - june 2004 edition– 11/28
- 16. Neural IDS Designed for DoS and port scan attacks IDS based on a multilayer perceptron Designing the tool Analysis Quantification Topology feed−back Learning & validation IDS with AI marioc@dsi.icai.upco.es FIST Conference - june 2004 edition– 11/28
- 17. First scenario: Port scan Pouring rain analogy Packets from the same source @IP 21 22 23 25 80 PORT NUMBERS IDS with AI marioc@dsi.icai.upco.es FIST Conference - june 2004 edition– 12/28
- 18. Second scenario: Denial of Service Pouring rain analogy Packets from the same source @IP 21 22 23 25 80 PORT NUMBERS IDS with AI marioc@dsi.icai.upco.es FIST Conference - june 2004 edition– 13/28
- 19. Measures Visually the difference between them is clear. . . but quantitatively? IDS with AI marioc@dsi.icai.upco.es FIST Conference - june 2004 edition– 14/28
- 20. Measures Visually the difference between them is clear. . . but quantitatively? Measures borrowed from Physics IDS with AI marioc@dsi.icai.upco.es FIST Conference - june 2004 edition– 14/28
- 21. Measures Visually the difference between them is clear. . . but quantitatively? Measures borrowed from Physics Statistical Mechanics Order = Low Entropy Disorder = High Entropy IDS with AI marioc@dsi.icai.upco.es FIST Conference - june 2004 edition– 14/28
- 22. Measures Visually the difference between them is clear. . . but quantitatively? Measures borrowed from Physics Solid State Physics (electronics) ATOMS INSULATOR ATOMS CONDUCTOR IDS with AI marioc@dsi.icai.upco.es FIST Conference - june 2004 edition– 14/28
- 23. Measures Visually the difference between them is clear. . . but quantitatively? Measures borrowed from Physics Packets from the same source @IP Disorder = High Entropy 21 22 23 25 80 PORT NUMBERS CONDUCTOR IDS with AI marioc@dsi.icai.upco.es FIST Conference - june 2004 edition– 14/28
- 24. Measures Visually the difference between them is clear. . . but quantitatively? Measures borrowed from Physics Packets from the same source @IP Order = Low Entropy 21 22 23 25 80 PORT NUMBERS INSULATOR IDS with AI marioc@dsi.icai.upco.es FIST Conference - june 2004 edition– 14/28
- 25. Measures Visually the difference between them is clear. . . but quantitatively? Measures borrowed from Physics Traffic parameters Packets per second Fraction of total packets to a port Inverse of the total number of packets IDS with AI marioc@dsi.icai.upco.es FIST Conference - june 2004 edition– 14/28
- 26. Measures Visually the difference between them is clear. . . but quantitatively? Measures borrowed from Physics Traffic parameters Packets per second Fraction of total packets to a port Inverse of the total number of packets All measures are evaluated within a time window. Parallel time windows: e.g., 15 sec, 30 sec, 5 minutes, 30 minutes IDS with AI marioc@dsi.icai.upco.es FIST Conference - june 2004 edition– 14/28
- 27. Topology ENTROPY PORT SCAN IPR DENIAL OF SERVICE PACKETS/SEC FRACTION OF PACKETS NONE 1/PACKETS IDS with AI marioc@dsi.icai.upco.es FIST Conference - june 2004 edition– 15/28
- 28. Learning and testing TYPE OF ATTACK LEARNING PATTERNS RATE OF SUCCESS SEQUENCIAL SCAN 20 100 % SEQUENCIAL SCAN 50 100 % RANDOM SCAN 20 100 % RANDOM SCAN 50 100 % DoS 20 70 % DoS 50 80 % ALL 20 60 % ALL 50 65 % IDS with AI marioc@dsi.icai.upco.es FIST Conference - june 2004 edition– 16/28
- 29. Learning and testing TYPE OF ATTACK LEARNING PATTERNS RATE OF SUCCESS SEQUENCIAL SCAN 20 100 % SEQUENCIAL SCAN 50 100 % RANDOM SCAN 20 100 % RANDOM SCAN 50 100 % DoS 20 70 % DoS 50 80 % ALL 20 60 % ALL 50 65 % Best choice: Specialized neural detectors IDS with AI marioc@dsi.icai.upco.es FIST Conference - june 2004 edition– 16/28
- 30. Fuzzy Logic Imitates human perception: Approximate reasoning IDS with AI marioc@dsi.icai.upco.es FIST Conference - june 2004 edition– 17/28
- 31. Fuzzy Logic Imitates human perception: Approximate reasoning Example: Air cooler Classical rules: IF Temperature > 25 THEN Switch-on IF Temperature < 21 THEN Switch-off ... IDS with AI marioc@dsi.icai.upco.es FIST Conference - june 2004 edition– 17/28
- 32. Fuzzy Logic Imitates human perception: Approximate reasoning Example: Air cooler Classical rules: IF Temperature > 25 THEN Switch-on IF Temperature < 21 THEN Switch-off ... Fuzzy rules: IF Temperature is high THEN Switch-on IF Temperature is too low THEN Switch-off ... IDS with AI marioc@dsi.icai.upco.es FIST Conference - june 2004 edition– 17/28
- 33. Fuzzy Logic Imitates human perception: Approximate reasoning Example: Air cooler Classical rules: IF Temperature > 25 THEN Switch-on IF Temperature < 21 THEN Switch-off ... Fuzzy rules: IF Temperature is high THEN Switch-on IF Temperature is too low THEN Switch-off ... More sofisticated fuzzy rules: IF Temperature is moderate AND my wife is very pregnant THEN Switch-on ... IDS with AI marioc@dsi.icai.upco.es FIST Conference - june 2004 edition– 17/28
- 34. Term sets and grade of membership Thresholds More than 3000 packets/sec ⇒ Possible DoS More than 5000 packets/sec ⇒ DoS! IDS with AI marioc@dsi.icai.upco.es FIST Conference - june 2004 edition– 18/28
- 35. Term sets: Thresholds 0 1 IDS with AI marioc@dsi.icai.upco.es 0 ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ 1000 ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ low ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ 2000 ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ VOLUME OF TRAFFIC ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ ¡ More than 5000 packets/sec ⇒ DoS! More than 3000 packets/sec ⇒ Possible DoS Term sets and grade of membership FIST Conference - june 2004 edition– 18/28
- 36. Fuzzy correlator: Preliminary work Aim of the research: Use the flexibility and human language features of Fuzzy Logic and include them in the OSSIM Correlation Engine IDS with AI marioc@dsi.icai.upco.es FIST Conference - june 2004 edition– 19/28
- 37. Fuzzy correlator: Preliminary work Aim of the research: Use the flexibility and human language features of Fuzzy Logic and include them in the OSSIM Correlation Engine Status: Preliminary definitions and precedures. IDS with AI marioc@dsi.icai.upco.es FIST Conference - june 2004 edition– 19/28
- 38. More on term sets Input variable: Volume of traffic very low low normal high very high 1 0 0 1000 2000 3000 4000 5000 IDS with AI marioc@dsi.icai.upco.es FIST Conference - june 2004 edition– 20/28
- 39. More on term sets (II) Input variable: Number of visited ports very low low normal high very high 1 0 0 2 4 6 8 10 IDS with AI marioc@dsi.icai.upco.es FIST Conference - june 2004 edition– 21/28
- 40. More on term sets (III) Output variable: DoS Attack? improbable maybe almost sure 1 0 0 0.5 1 Rules (example): IF traffic is high AND number of destination ports is low THEN DoS Evaluating rules gives the required answer ’DoS Attack?’: almost sure IDS with AI marioc@dsi.icai.upco.es FIST Conference - june 2004 edition– 22/28
- 41. OSSIM Correlation Engine Characteristics Depends strongly on timers All the variants of an attack must be coded Cannot detect new attacks Complex sintax IDS with AI marioc@dsi.icai.upco.es FIST Conference - june 2004 edition– 23/28
- 42. Sample scenario: NETBIOS DCERPC ISystemActivator IDS with AI marioc@dsi.icai.upco.es FIST Conference - june 2004 edition– 24/28
- 43. Sample scenario: NETBIOS DCERPC ISystemActivator TIME_OUT IF destination_ports = 135,445 THEN Generate Alarm with Reliability 1 and wait 60 seconds for next rule TIME_OUT AND IF DEST_IP and SRC_IP talk again THEN Alarm, Reliability 3 and wait 60 seconds for next rule AND IF DEST_PORT and SRC_PORT talk again AND plugin_sid=2123 (CMD.EXE) THEN Alarm TIME_OUT Reliability 6 and wait 60 seconds for next rule TIME_OUT AND FINALLY IF plugin_id=2002 and conection lasts more than 10 THEN Alarm with Reliability 10 IDS with AI marioc@dsi.icai.upco.es FIST Conference - june 2004 edition– 25/28
- 44. Fuzzy Correlator revisited: Objectives Going beyond the sequential arrival of packets Integrating different sensors: IDS with AI marioc@dsi.icai.upco.es FIST Conference - june 2004 edition– 26/28
- 45. Fuzzy Correlator revisited: Objectives Going beyond the sequential arrival of packets Integrating different sensors: SNORT Anomaly detection: Abnormal connection to an open port (firewall) Thresholds High traffic at nights or weekends, . . . Neural-IDS Other IDS with AI marioc@dsi.icai.upco.es FIST Conference - june 2004 edition– 26/28
- 46. Fuzzy Correlator revisited: Objectives Going beyond the sequential arrival of packets Integrating different sensors: SNORT Anomaly detection: Abnormal connection to an open port (firewall) Thresholds High traffic at nights or weekends, . . . Neural-IDS Other Defining rules according to Security Manager’s experience IDS with AI marioc@dsi.icai.upco.es FIST Conference - june 2004 edition– 26/28
- 47. Conclusions and open questions AI techniques are Flexible Suitable for pattern recognition Powerful (Neural-IDS) Easy to design (human language) IDS with AI marioc@dsi.icai.upco.es FIST Conference - june 2004 edition– 27/28
- 48. Conclusions and open questions AI techniques are Flexible Suitable for pattern recognition Powerful (Neural-IDS) Easy to design (human language) But there is still a lot of work to do. . . IDS with AI marioc@dsi.icai.upco.es FIST Conference - june 2004 edition– 27/28
- 49. Conclusions and open questions AI techniques are Flexible Suitable for pattern recognition Powerful (Neural-IDS) Easy to design (human language) But there is still a lot of work to do. . . We need more time. We need more people Students Security experts (working group?) And of course. . . IDS with AI marioc@dsi.icai.upco.es FIST Conference - june 2004 edition– 27/28
- 50. Conclusions and open questions AI techniques are Flexible Suitable for pattern recognition Powerful (Neural-IDS) Easy to design (human language) But there is still a lot of work to do. . . We need more time We need more people Students Security experts (working group?) And of course. . . some money to pay it IDS with AI marioc@dsi.icai.upco.es FIST Conference - june 2004 edition– 27/28
- 51. And that’s all folks. . . IDS with AI marioc@dsi.icai.upco.es FIST Conference - june 2004 edition– 28/28