Intrusion Detection with Neural Networks
- 1. Intrusion Detection and Classification Using Neural Networks Antonio Moran, Ph.D. amoran@ieee.org Stockholm University, Sweden May 17, 2013
- 2. Information Security in Computer Networks Information assurance is an issue of serious global concern. Malicious usage, attacks and sabotage have been on the rise. Connecting information systems to public networks (Internet, telephone) magnifies the potential for intrusion and attack.
- 3. Intrusion in Information Systems and Networks Any set of actions that attempt to compromise the integrity, confidentiality or availability of a resource Intrusion Intrusion in Information Systems Any anauthorized access, unauthorized attempt to access, damage, or malicious use of information resources
- 4. Motives to Launch Attacks Force a network to stop a service(s) Steal some information stored in a network To show unhappiness or uneasiness To obtain economical benefits
- 5. Network Attacks liability for compromised customer data Attacks could result in: Liability for compromised customer data Loss of intellectual property Degraded quality of network service Great business loss ………..
- 6. Need for and Intrusion Detection System It is difficult (impossible) to ensure that an information system will be free of security flaws. Computer systems suffer from security vulnerabilities regardless of their purpose, manufacturer or origin. It is technically difficult as well as economically costly, to ensure that computer systems and networks are not susceptible to attacks
- 7. Intrusion Detection in Information Systems Attempting to detect computer attacks by examining data records observed by processes on the same network
- 8. Components of an Intrusion Detection System Information source providing a stream of event records Analysis engine identifying signs of intrusion, attacks or other policy violations Response component generating reactions to assure system correct operation Data Analysis Identification Action
- 9. Types of Information Sources Data from network traffic and packet streams Data from sources internal to a computer. Operating system level Data from running applicationsApplication based Network based Host based
- 10. Categories of Analysis Engine Searching for something defined to be bad. Detect intrusions that follow a well-known patterns of attacks. Can not detect unknown future intrusions. Misuse Detection Searching for something rare or unusual. Analyze system event streams to find patterns of activity appearing to be abnormal. Computationally intensive. Anomaly Detection
- 11. Categories of Analysis Engine Detect known attacks using pre-defined attack patterns and signatures Misuse Detection Detect attacks by observing deviations from the normal behavior of the system Anomaly Detection
- 13. Implementation of Analysis Engine Runs periodically detecting intrusions after the fact. Act in a reactive way. Off-Line Detect intrusions while they are happening allowing a quick response. Computationally expensive (continuous monitoring). On-Line Real-Time
- 14. Dynamic Intrusion Deteccion System Hybrid system using misuse and anomaly detection strategies Not allowing an intruder to train (update) the system incorrectly Running in real-time Updating itself continuously over periods of time
- 15. Types of Network Attacks The attacker makes the computing or memory resources too busy or full to handle legitimate requests or denies legitimate users access Remote to User User to Root Denial of Service Probing (Scanning) The attacker, starting out with access to a normal user account, tries to gain root (superuser) access and privilegies The attacker gains access as a local user of the network The attacker scans the network to gather information or detect vulnerabilities
- 16. Approaches for Anomaly Detection Detecting abnormal activity on a server or network whose magnitude overcome a given threshold. Ex: Abnormal consumption of CPU or memory of one server. Rule-based Measures Statistical Measures Threshold Soft Computing Based on sets of predefined rules that are provided by a network administrator or generated by expert systems. Neural Networks, Fuzzy Logic, Genetic Algorithms, Support Vector Machines. Statistical models based on historical values. Asumptions about the underlying statistical distribution of user behavior. Ex: Hidden Markov Models.
- 17. Rule Based Intrusion Detection liability for compromised customer data Detecting attacks by signature matching. A set of signatures, describing the characteristics of possible attacks, and the corresponding rules are stored. The rules are used to evaluate incoming packet stream and detect hostile traffic. Easy to implement and customize but requires human domain experts to find signatures and their rules. It works for known patterns of attacks Artificial intelligence techniques could be useful
- 18. Rule Based Instrusion Detection IF CountConnection=50 THEN AttackType=’smurf’ Human network administrators usually generate low-complexity rules: IF Src_Byte=0 OR Src_Byte>500 THEN ‘Alert’ same host within 2 sec. IF ip_flags = 0 AND ip_len <=256 AND tcp_csum =0 AND ip_length > 120 AND ip_src <= 1.451703E9 AND tcp_dport <= 82 AND tcp_win <= 23 THEN Malicious. Complex rules can be generated using AI techniques:
- 19. Intrusion Deteccion Systems Intrusion Detection Systems alone will not ensure the security of a computer network Intrusion detection systems must be complemented by firewalls, vulnerability assessment, and a comprehensive security policy
- 20. Intrusion Detection and Clasification Using Neural Networks Application of neural networks in Intrusion Deteccion Systems date back to 1992
- 21. When a Computer Network is Working in Normal / Abnormal State It is difficult to define all the attributes that characterize a normal or abnormal state. Let a neural network discovers the patterns characterizing a normal state and an abnormal state.
- 22. Intrusion Detection and Clasification Using Neural Networks Discover underlying patterns that describe normal user or computer network behavior Use the patterns to determine: The state of the network The type of user Normal Attacked Authorized Intruder Neural Network
- 23. Intrusion Detection and Classification Using Neural Networks Hybrid System Misuse Detection Anomaly Detection Runs in real-time Network Based Packet streams
- 24. Intrusion Detection and Classification Using Neural Networks Two Neural Networks Neural Network for detecting intrusion. State of the network: normal or with intrusion Neural Network for classifying intrusion. Four types of intrusion
- 25. Intrusion Detection and Classification Using Neural Networks Two Neural Networks Neural Network Packet Stream Normal Intrusion Neural Network Intrusion Detection Intrusion Classification Denial of Service User to Root Remote to User Probing
- 26. Neural Network Design Process Data collection Definition of inputs and outputs Input and output data generation Data normalization Selection of neural network structure Neural network training Neural network validation
- 27. What Data To Be Used? Main features (attributes) of network packet stream Take a set of network packets Determine main features to be analyzed from packet header (and packet data)
- 28. ……....Pi+1 Pi+2 PrPk+1 Pk+2 Pk+N PzPi Pz-1Pz-2Pj …… …… Packet stream P Window Window Packets Features Vector Attributes Extraction … Window size: 50 - 500 Features vector size: 10 - 50 Features Extraction of Window Based Packet Stream
- 29. ……....Pi+1 Pi+2 PrPk+1 Pk+2 Pk+N PzPi Pz-1Pz-2Pj …… …… Packet stream P Window Window Packets Features Vector Attributes Extraction … Window size: 50 - 500 Features vector size: 10 - 50 Features of Window Based Packet Stream Features are chosen such that their values change perceivably in normal and intrusive conditions.
- 30. ……....Pi+1 Pi+2 PrPk+1 Pk+2 Pk+N PzPi Pz-1Pz-2Pj …… …… Packet stream P Window Attributes Extraction Number of IP addresses Packet Stream Features Number of protocols and types Network service on destination. http, telnet Number of packets with 0 data length Average data length Average window size Number of packets with 0 window size Number of packets with 0 data length Number of failed login attempts Number of wrong fragments Number of urgent packets Number of data bytes from source to destination Number of data bytes from destination to source Number of file creation operations Number of connections with SYN errors Number of coonections to the same service …….... ……....
- 31. Neural Network for Intrusion Detection Inputs Outputs Window packet features vector 40 features Code for every state of the network Intrusion : 0 1 Normal: 1 0 40 Inputs 2 Outputs (Attack)
- 32. Neural Network Training Data 40 Inputs 2 Outputs 12 24 05 00 02 04 09 14 15 21 08 00……. 0 1 04 21 16 12 10 21 01 17 04 13 19 10……. 1 0 01 13 15 21 12 11 12 11 05 11 06 12……. 1 0 14 14 06 15 08 13 10 11 14 06 08 19……. 0 1 …... …... …... : : : 40 Inputs 2 Outputs : 16000 Pairs vij wjk 10000 Normal 6000 Attack
- 33. Neural Network Training and Validation Training: 16000 input-output pairs Validation: 5000 input (feature vectors) Determining coefficients vij wjk Computing network outputs for every input and determining state of network: normal or attack 40 Inputs 2 Outputs:: : : vij wjk
- 34. Neural Network Validation In validation (testing), inputs are different to those used in training Input 1 Output : 0.85 0.15 1 0 Normal Input 2 Output : 0.11 0.88 0 1 Attack …... 40 Inputs 2 Outputs:: : : vij wjk
- 35. Neural Network Validation Normal 3000 94% 6% Attack 2000 90% 10% Correct Detection Rate Detected as Attack Detected as Normal Number of Tests False positive (normal behavior is rejected) : 6% False negative (attack considered as normal) : 10% Intrusion Detection
- 36. Neural Network for Intrusion Detection It is expected that any significantly deviation from the normal behavior is considered an attack It is expected to perform well detecting unknown intrusions and even zero-day attacks
- 37. Neural Network for Attack Classification From the previous neural network an attack has been detected. Now, it is required to determine the type of attack Denial of Service User to Root Remote to User Probing
- 38. Neural Network for Attack Classification Inputs Outputs Window packet features vector 40 features Code for every type of attack Denial of Service: 1 0 0 0 User to root: 0 1 0 0 Remote to user: 0 0 1 0 Probing: 0 0 0 1 40 Inputs 4 Outputs
- 39. Neural Network Training Data 40 Inputs 4 Outputs 12 24 05 00 02 04 09 14 15 21 08 00……. 0 1 0 0 04 21 16 12 10 21 01 17 04 13 19 10……. 1 0 0 0 01 13 15 21 12 11 12 11 05 11 06 12……. 0 0 0 1 14 14 06 15 08 13 10 11 14 06 08 19……. 0 1 0 0 …... …... …... : : : 40 Inputs 4 Outputs : 6000 Pairs vij wjk
- 40. Neural Network Training and Validation Training: 6000 input-output pairs Validation: 2000 input (feature vectors) Determining coefficients vij wjk Computing network outputs for every input and determining type of attack : : : 40 Inputs 4 Outputs : vij wjk
- 41. Neural Network Validation In validation (testing), inputs are different to those used in training Input 1 Output : 0.85 0.15 0.24 0.01 1 0 0 0 Denial of service Input 2 Output : 0.11 0.08 0.18 0.91 0 0 0 1 Probing …... : : : 40 Inputs 4 Outputs : vij wjk
- 42. Neural Network Validation Denial of Service 600 91% User to Root 500 81% Remote to User 300 69% Probing 600 90% Correct Detection Rate Number of Tests Type of Attack Attack Classification
- 43. Data to Design and Evaluate IDS Systems Own Generation Knowledge Discovery and Data Mining Tools Competition. DARPA KDD Data Base Standard benchmark for intrusion detection evaluations.
- 44. Thank you for your attention! Antonio Moran, Ph.D. amoran@ieee.org