SlideShare a Scribd company logo
MECHANISMS FOR DATABASE
INTRUSION DETECTION AND
RESPONSE
ASHISH KAMRA
Electrical and Computer Engineering
Ph.D DefenseTalk
May 12 2010
Purdue University 1
Thesis Statement
To Build a RealTime Anomaly Detection
and Response Mechanism for
Databases
Integrated with the
Core Database Operations
2
Keywords
3
RealTime Intrusion
Detection
Query
ParseTree
Not
Anomalous
Anomalous
Query
Execution
Intrusion
Response
DBMS Integration
DB Server Process
Intrusion Detection
and Response
MOTIVATION
4
Importance of Data
 Most sensitive and proprietary data resides in
a DBMS
 A database breach can be extremely costly
 Loss of Reputation
 Lawsuits
 …
5
Premise
6
DBMS users and applications possess more
privileges than required to carry out the task
Access control solutions alone
may not enough !
Insider Threat Scenario
7
SSN Credit Card Security Code
DatabaseAdministrators accessing sensitive
application data
Disease Patient Name Diagnosis
Privacy breaches by curious employees
Related Systems
 Oracle DatabaseVault
 3rd Party Database Activity Monitoring
Products
8
Why Anomaly Detection?
 Not possible to “specify” all database
malicious actions
 Key Issue: false positive rate
 We augment the detection mechanism with a
response mechanism to help with false
positives
9
Why DBMS Integration?
 Independence from the underlying network
infrastructure
 think moving to a cloud provider
 More control on response actions in real time
 Nobody else has yet tried it !
10
Organization
 System Components
 Anomalous Access Pattern Detection
 Anomaly Response Mechanism
 Privilege State Based Access Control
 JointThreshold Policy Administration
11
System Components
Query
User
Features Assessment
Profile Creator
Log
Audit
Log
Training
Queries
TRAINING PHASE
Detection Engine Response Engine
Response
Policy Base
Feature Selector
Profiles
Alert
Drop
No Action, Update
Profiles
Privilege
State Based
Access Control
12
Joint
Administration
Model
Thesis Contributions
 AnomalousAccess Pattern Detection in Databases
(ACSAC 2005,VLDB Journal 2008)
 Response Mechanism (SDM 2008,TKDE 2010)
 Response Policy Language
 Policy MatchingAlgorithms
 Privilege State Based Access Control for Fine Grained
Intrusion Response (In submission RAID 2010)
 JointThreshold Administration Model for Response
Policy Administration (TKDE 2010, POLICY 2010)
 Prototype Implementation in the PostgreSQL open
source DBMS
13
ANOMALOUS ACCESS
PATTERN DETECTION
14
Query
User
Features Assessment
Profile CreatorAudit
Log
Training Queries
TRAINING PHASE
Detection EngineFeature Selector
Profiles
Example
Normal SQL Commands
EMPLOYEE
TRAINING
PERFORMANCE
HR SCHEMATABLES
Anomalous SQL Commands
FINANCE SCHEMA
TABLES
COMPENSATION
ACCOUNTS
PAYROLL
Access
Control
Mechanism
Access Granted
Access Granted !!!
15
Key Ideas
 Extract features from the SQL Queries
 Experiment with feature extraction at
different granularity levels
 3 granularity levels considered
 Two Scenarios Considered
 Role basedAnomaly Detection
 UnsupervisedAnomaly Detection
16
SQL Query Representation - I
 Coarse
- Command
- Num tables in the query
- Num columns in the query
17
SQL Query Representation - II
 Medium
- Command
- Tables in the query
- Num columns per table
18
SQL Query Representation - III
 Fine
- Command
- Tables in the query
- Columns in the query
19
Role Based Anomaly Detection
 Profiles per Role
 Profile contain frequency counts for the various features
extracted from the SQL query
 Role Profiles input to the Naïve Bayes Classifier
 One Role One Class
 Single Role Activation Assumption
 Classifier Predicted Role != Activated Role ? Anomaly :
20
Naïve Bayes Model – medium
 DB =T1(c1,c2,c3,c4) ;T2(c1,c2,c3,c4)
P(R = r1 | T1C = 2,T2C = 0, Cmd=Select) P(T1C = 2,T2c = 0, Cmd = Select | R = r1) * P(R =r1)
=P(T1C = 2 | R=r1) * P(T2C = 0 | R=r1) * P(Cmd = Select |
R=r1) * P(R=r1)
Query Under consideration : Select c1,c2 fromT1
Role
Cmd T1C T2C

21
Why Naïve Bayes ?
 Low computational complexity
 Works well in practice even if the attribute
independence condition is not met
 Ease of Implementation
22
Role Based Anomaly Detection on
Real Data
QuipletType False Negative (%) False Positive (%)
c 2.6 19.2
m 2.4 17.1
f 2.4 17.9
23
Implementation in PostgreSQL
Postmaster
Main process
Postgres
Server process
Server
Initialization
Code
Query ParseTree
Query Rewritten
ParseTree
Query
Plan
AccessControl
Enforcement
Query
Executor
Connect
Spawn a new server process
Submit SQL Query
Query results
Collect role login stats
Collect table initialization stats
Collect table/column access stats per role
Waiting
for Query Send detection stats to the
statistics collector process
Anomaly
Detection
Anomaly
Response
Yes
No
24
Feature Extraction Architecture
.
.
Collect stats in
memory resident
data structures
.
.
.
.
.
Send messages to
the statistics
collector
Process
.
Periodically
read stats file
Postgres server process
Query executor .
.
Receive messages
and store stats in
memory resident
data structures
.
.
.
.
.
.
Periodically write
the stats to the
pg_stat file
.
.
Statistics collector process
Stats profile creator
pg_stat
file
writeread
25
Performance Impact - Experimental
Setup
 Base Database Size ( x )
 5 tables with indexes on primary keys
 2 user columns per table (all integer type)
 7 system columns per table
 100 rows per table
 EachTransaction -
 5 select queries
 5 update queries
 5 Insert queries
26
Statistics Collection Overhead
0
2
4
6
8
10
12
14
Overhead(%)
Database Size vs Transaction Processing Overhead
coarse
medium
fine
27
Takeaway: Statistics Collection overhead < 15 %
Detection Algorithm Time
medium
0
5
10
15
20
25
30
35
x (6) 2x (11) 4x (21) 8x (41) 16x (81)
DetectionTime(microseconds)
Database Size vs DetectionTime
28
Detection Algorithm Time
fine
0
50
100
150
200
250
300
350
400
x (51) 2x (121) 4x (321) 8x (961) 16x (3201)
DetectionTime(microseconds)
Database Size vs DetectionTime
29
Detection Process Overhead
0
10
20
30
40
50
60
x(5,2) 2x(10,4) 4x(20,8) 8x(40,16) 16x(80,32)
Overhead(%)
Database Size vs Transaction Processing Overhead
30
Takeaway: Large overhead for fine triplets for large DB size
Unsupervised Anomaly Detection
 No Role Information
 Partition training data in clusters of similar
queries
 K centers
 K means
 Map every user to its representative cluster
31
Unsupervised Anomaly Detection
. .
. . .
. . . .
. . . .
. .
.
. .
. .
. . . .
. .
.
. .
. .
. . .
. . .
. . .
.
. . . .
. . .
C1
C2
C3
C4
U1 C3
U2 C3
U3 C1
U4 C2
U5 C4
U6 C1
U7 C4
U8 C3
U9 C2 32
Detection Methodology
 Apply Naïve Bayes with the cluster as the
class
 Outlier detection in the representative cluster
33
Future Research Directions
 Role BasedAnomaly Detection
 Multiple Role Activation
 Considering Role Hierarchies
 …
 Unsupervised Detection
 Implementation in PostgreSQL
 One Class SupportVector Machine
 …
 Query Result Intrusion Detection
 Machine Learning Issues
 Concept Drift
 Overfitting
 …
34
ANOMALY RESPONSE
MECHANISM
35
Assessment
Log
Response Engine
Response
Policy Base
Alert
Drop
No Action, Update
Profiles
Contributions
 Response policy language
 Response policy matching
 Support for fine grained response actions
 Privilege State BasedAccessControl
 Response policy administration
 JointThresholdAdministration Model
36
Response Policy Language
Interactive Event-Condition-Action Policy Language
ON EVENT
IF CONDITION
THEN INITIAL ACTION
CONFIRM CONFIRMATION ACTION
ON SUCCESS RESOLUTION ACTION
ON FAILURE FAILURE ACTION
37
POLICY ATTRIBUTES
ATTRIBUTE DESCRIPTION
CONTEXTUAL
User The user associated with the request.
Role The role associated with the request.
Client App The client application associated with the request.
Source IP The IP address associated with the request.
Date/Time Date/Time of the anomalous request.
STRUCTURAL
Database The database referred to in the request.
Schema The schema referred to in the request.
ObjType The object types referred to in the request
such as table, view etc
Obj(s) The object name(s) referred in the request
SQLCmd The SQL Command associated with the request
Obj Attr(s) The attributes of the object(s) referred in the request. 38
POLICY PREDICATES
Pr1 Role != DBA
Pr2 Source IP IN 192.168.0.0/16
Pr3 Objs IN {dbo.*}
Pr4 Time BETWEEN 0800 – 1700
Pr5 SQLCmd IN {Select}
Pr7 SQLCmd IN {Insert,Update,Delete}
POLICY CONDITIONS
C1 Pr1 ^ Pr7
C2 Pr2 ^ Pr4 ^ Pr6
39
RESPONSE ACTIONS
ACTION DESCRIPTION
CONSERVATIVE : LOW SEVERITY
NOP No OPeration. This option can be used to filter
unwanted alarms.
LOG The anomaly details are logged.
ALERT A notification is sent.
FINE-GRAINED : MEDIUM SEVERITY
TAINT The request is audited.
SUSPEND The request is put on hold till SUCCESSFUL execution
of a confirmation action.
AGGRESSIVE : HIGH SEVERITY
ABORT The anomalous request is aborted.
DISCONNECT The user session is disconnected.
REVOKE A subset of user-privileges are revoked.
DENY A subset of user-privileges are denied.
40
Response Policy Example
Re-authenticate unprivileged users who are logged from inside the
organization’s internal network for write anomalies to tables in the dbo
schema.
If re-authentication fails, drop the request and disconnect the user else do
nothing.
ON ANOMALY DETECTION
IF Role != DBA and Source IP IN 192.168.0.0/16 and
ObjType = table and Objs IN dbo.* and
SQLCmd IN {Insert,Update,Delete}
THEN SUSPEND
CONFIRM RE-AUTHENTICATE
ON SUCCESS NOP
ON FAILURE ABORT REQUEST, DISCONNECT USER
41
Policy Matching Problem
 AA = {A1, A2, . . . , An} be the set of anomaly attributes.
 POL ={Pol1, Pol2, . . . , Polk} be the set of response policies.
 PR = {Pr1, Pr2, . . . , Prm} be the set of all distinct policy predicates.
 Poli(C) be the policy condition for a policy Poli
 AAS : A1 = a1, A2 = a2, . . . , An = an be the assessment of an
anomaly submitted by the detection mechanism to the response
system.
 A policy Poli is said to matchAAS if Poli(C) = ‘true’ evaluated over
AAS.
The policy matching problem is to find the set
of all policies in POL that match a given
anomaly assessment AAS.
42
Policy Matching Algorithm
S
A1
A2
A3
A4
Pr1
Pr2
Pr3
Pr4
Pr5
Pr6
Pol1
Pol2
Pol3
Pol4
Attribute Nodes Predicate Nodes Policy Nodes
43
RESPONSE ACTION SELECTION
 MOST SEVERE POLICY
 LEAST SEVERE POLICY
44
Implementation in PostgreSQL
 New system catalogs for storing policy
attributes, predicates and definition
 Policy matching invoked for every query
detected as anomalous
45
Policy Matching Overhead
0
50
100
150
200
250
300
350
400
450
12 24 36 42 60 72 84 96 108 120
PolicyMatchingOverhead
(microseconds)
Number of Predicates vs Policy Matching Overhead
46
Cumulative Overhead – Real World
 10 tables, 4 columns per table - under
detection
 24 distinct policy predicates
QuipletType Time (in microseconds)
Fine 130 + 20 = 150
Medium 130 + 6 = 136
47
Fine Grained Intrusion Response
 Goal:To change the access control system to
support response actions such as
 Request suspension
 Request tainting
48
Privilege State Based Access
Control (PSAC)
 Attach a “state” parameter to every privilege
assigned to a user or role
 We have come up with the following 5 states
DENY
SUSPEND
TAINT
GRANT
UNASSIGN
49
PSAC Details
 “DENY” state supports negative authorizations
 “SUSPEND” state supports request suspension
 “TAINT” state supports request tainting
 “GRANT” state supports standard SQL GRANT
 “UNASSIGN” state supports standard SQL
REVOKE
50
State Dominance Relationship
X
means ‘X’ overrides ‘Y’
DENY
SUSPEND
TAINT
UNASSIGN
GRANT
Y
51
State Transitions
+
/
+
+
??
?
/
/
/
+ grant
deny
? suspend
/
unassign
taint
?
TAINT
SUSPEND
DENY
GRANT REVOKE
?
52
Considering Role Hierarchies
 Consider a role hierarchy based on privilege
inheritance
 Parent role inherits the privileges of children
role
 What about privileges in “deny”, “suspend”
and “taint” states?
R_parent
{insert}
R_child
{select}
{select}
53
Privilege Orientation Modes
R8
R5 R6 R7
R2 R3 R4
R1
down / neutral
deny | taint | suspend
up
unassigned | grant
54
Implementation in PostgreSQL
 New SQL commands to support adhoc
privilege state transitions
 Additional Access Control Lists on the DB
objects to support privilege states and
orientation modes
 Re-authentication procedure for a privilege in
“suspend” state
55
Results on Overhead of PSAC
Maintenance – No Role Hierarchy
0
10
20
30
40
50
60
16 32 64 128 256 512
Overhead(microseconds)
ACL Size vs Access Control Enforcement Overhead
BASE
PSAC
56
Results on Overhead of PSAC
Maintenance–With Role Hierarchy
0
20
40
60
80
100
120
16 32 64 128 256 512
Overhead(microseconds)
ACL Size vs Access Control Enforcement Overhead
BASE
PSAC
57
Cumulative Overhead – Real World
 10 tables, 4 columns per table - under
detection
 24 distinct policy predicates
 Size of Acl = 16 (with role hierarchy)
QuipletType Time (in microseconds)
Fine 130 + 20 + 22= 172
Medium 130 + 6 + 22 = 158
58
Policy Administration
 Assumptions
 The response policies maintained as database
objects
 DBAs responsible for policy administration
 The policies need to be created to monitor DBAs
as well
Key Issue: How can a DBA be trusted to
maintain policies to monitor himself?
59
Organization Scenario
 Medium size organizations
 Few Database Administrators managing large
number of databases
 Each DBA has all privileges in the system
60
Joint Threshold
Administration Model
Key Ideas
A single DBA not trusted but threat is mitigated
if the trust is distributed among multiple DBAs
A policy operation is “complete” when
authorized by atleast “k” DBAs
Use of threshold digital signatures to
maintain policy integrity
61
Threshold signatures
.
.
k
l
Signature share
message
Secret key
Secret key
Secret key
Signature share
Signature share
FINAL
SIGNATURE
PUBLIC KEY
VERIFY
62
PRACTICAL RSA THRESHOLD
SIGNATURES – VICTOR SHOUP
 Asynchronous signature share generation
 Asynchronous signature share combining
 Public signature shares
 Efficient signature verification mechanism
63
JTAM Set Up
 Trusted Dealer – DBMS
 “l” = number of DBAs
 For each k, 2 <= k <= l -1
 Generate RSA public-private key pair
 “l” secret key shares using the private key
 private key destroyed after set-up
64
Response Policy Lifecycle
65
Create Response Policy
… Joint Adm by
k Users;
Authorize Response Policy
[Policy ID]Create;
(k-1)th time
Suspend Response Policy [Policy ID];
Drop Response Policy
[Policy ID] ;
Authorize Response Policy
[Policy ID] Drop;
(k-1)th time
Authorize Response Policy
[Policy ID] Suspend;
(k-1)th time
Alter Response Policy
[Policy ID] …. ;
Policy Creation Example
Create Response Policy [Policy Data] Jointly
Administered By k Users;
H(Pol) = SHA1 ( Policy ID,Conditions,
Initial Action(s),Optional Action(s),
k, State).
W(DBA1) : Signature share of DBA 1
PolID PolData k r hash sig shares state final sig
… …. 3 2 H(Pol) W(DBA1) CREATED
66
Policy Activation
Authorize Response Policy [Policy ID] Create;
PolID PolData k r hash sig shares state final sig
… …. 3 1 H(Pol) W(DBA1);
W(DBA3)
CREATED
PolID PolData k r hash sig shares state final sig
… …. 3 0 H(Pol) ACTIVATED Wfinal
Policy authorization by DBA3
Policy authorization by DBA4
67
Attacks and Prevention
 Signature ShareVerification
 Final SignatureVerification
 Malicious Policy Update
 Malicious Policy Deletion
 Signature Replay Attack
 Policy Replay Attack
68
Implementation in PostgreSQL
 Enhanced system catalogs for storing the
signature shares and final signature
 Currently, implemented signature verification
for every policy during the policy matching
procedure
 A dedicated signature verification process
(to be done)
69
Signature Verification Overhead
0
20
40
60
80
100
120
140
160
180
128 256 512 1024
VerificationOverhead
(microseconds)
Num RSA bits vs SignatureVerification Overhead
70
Work in Progress
 Implementation of Joint Administration of
sensitive database commands such as
“grant/revoke, user addition/modification“
 POLICY 2010 demo paper
 Public (open source) release of the all the
techniques implemented
71
Thesis Related Publications - I
1. Elisa Bertino, Ashish Kamra, EvimariaTerzi, AthenaVakali:
Intrusion Detection in RBAC-administered Databases. Applied
Computer Security Applications Conference (ACSAC) 2005
2. Ashish Kamra, EvimariaTerzi, Elisa Bertino: Detecting
anomalous access patterns in relational databases.Very Large
DataBases (VLDB) Journal 2008
3. Ashish Kamra, Elisa Bertino, RimmaV. Nehme: Responding to
Anomalous Database Requests. Secure Data Management
(SDM) Workshop 2008
4. Ashish Kamra, Elisa Bertino: Design and Implementation of a
Intrusion Response System for Relational Database.
Transactions on Knowledge and Data Engineering (TKDE) 2010
72
Thesis Related Publications - II
5. Ashish Kamra, Elisa Bertino: JTAM: A Joint
Threshold Administration Model. POLICY 2010
Demo Paper
6. Ashish Kamra, Elisa Bertino: Database
Intrusion Detection and Response. Recent
Advances in Intrusion Detection (RAID) 2008
Poster
7. Ashish Kamra, Elisa Bertino: Privilege State
Based Access Control for Fine-Grained Intrusion
Response. RAID 2010 - In Submission
73
Other Publications
1. MercanTopkara, Ashish Kamra, Mikhail j. Atallah, Cristina Nita-Totaru:ViWiD
:Visible Watermarking Based DefenseAgainst Phishing. International
Workshop on Digital Watermarking (IWDW) 2005
2. Ji-Won Byun, Ashish Kamra, Elisa Bertino, Ninghui Li: Efficient k -
Anonymization Using ClusteringTechniques. DASFAA 2007
3. Elisa Bertino, Ashish Kamra, James P. Early: Profiling DatabaseApplication to
Detect SQL Injection Attacks. IPCCC 2007 – Invited Paper
4. Ashish Kamra: Mechanisms for Database Intrusion Detection and Response.
SIGMOD PhDWorkshop on Innovative Database Research IDAR 2008
5. Book Chapters
1. Elisa Bertino, Ji-Won Byun, Ashish Kamra. Database Security. Security, Privacy, andTrust
in Modern Data Management 2007. Petkovic, Milan; Jonker, Willem (Eds.)
2. Ashish Kamra, Elisa Bertino. Survey of Machine Learning Methods for Database Security.
Machine Learning in CyberTrust 2009.
74
Work Experience
1. SQL Server DatabaseAdministrator at ITAP from Feb
2006 to Dec 2008.
2. Internship, EMC, Rainfinity – May to Jul 2007
Implemented a RBAC mechanism in their file
virtualization product line.
3. Internship, EMC, NAS Engineering - May to Dec 2009
1. Security analysis of Network Filesystem version 4.1 protocol,
2. Distributed File System Manageability.
4. Working Full-time in EMC’s Integrated Systems and
Components group in the Unified Storage Division since
Jan 11 2010.
75
76
BACKUP slides
77
M-estimate of probability
P(Ai) = freq(A=a) + M(Pinitial(A=a)
total_count(A) + M
78
JTAM Set Up
RSA Public Private Keys
The DBMS chooses p, q as two large prime numbers
such that
p = 2p′ + 1 and q = 2q′ + 1
where p′ and q′ are themselves large primes
Let n = p * q be the RSA modulus. Let m = p′ * q′.The
DBMS also chooses e as the RSA public exponent
such that e > l.
Thus, the RSA public key is PK = (n, e).
The server also computes the private key d $in$ Z such that de =
1 mod m.
79
JTAM Set Up
 Secret Shares
 For this purpose, the DBMS sets a0 = d and
randomly assigns ai from {0, . . . ,m − 1} for 1 <= i <= k −
1.
The numbers {a0 . . . ak−1} define the unique polynomial
p(x) of degree k − 1
For 1 <= i <= l, the server computes the secret share, si,
of each DBA, DBAi, as follows:
si = p(i) mod m.
80
Signature Share Generation
Let x = H(Pol).
W(DBA1) = x2 s1
Where = l!
W(DBA1) does not leak any information about the secret share s1 because of the
intractability of the generalized discrete logarithm problem
81

More Related Content

PDF
Overview on SQL Injection Attacks
DOCX
Eastman_MI530_FinalProjectReport
PPTX
SQL Injection
PPTX
SQL Injection Attacks cs586
PPT
How "·$% developers defeat the web vulnerability scanners
PDF
The Impact of Mislabelling on the Performance and Interpretation of Defect Pr...
PPTX
SQL Injections - A Powerpoint Presentation
PPTX
Machine Learning for Malware Classification and Clustering
Overview on SQL Injection Attacks
Eastman_MI530_FinalProjectReport
SQL Injection
SQL Injection Attacks cs586
How "·$% developers defeat the web vulnerability scanners
The Impact of Mislabelling on the Performance and Interpretation of Defect Pr...
SQL Injections - A Powerpoint Presentation
Machine Learning for Malware Classification and Clustering

What's hot (8)

PPT
Sql injection
PPTX
Improving the Performance of Database-Centric Applications Through Program An...
PDF
Sql injection
PDF
Data Security In Relational Database Management System
PDF
Blackhat Europe 2009 - Detecting Certified Pre Owned Software
PPTX
AppSec Threat Modeling with 5 Agile Design Diagrams Every Project Should Have
PDF
Prevention of SQL Injection Attack in Web Application with Host Language
Sql injection
Improving the Performance of Database-Centric Applications Through Program An...
Sql injection
Data Security In Relational Database Management System
Blackhat Europe 2009 - Detecting Certified Pre Owned Software
AppSec Threat Modeling with 5 Agile Design Diagrams Every Project Should Have
Prevention of SQL Injection Attack in Web Application with Host Language
Ad

Similar to Mechanisms for Database Intrusion Detection and Response (20)

PPT
Raid08 dbir
PPTX
Intrusion Detection System for Applications using Linux Containers
PPT
DIDAR: Database Intrusion Detection with Automated Recovery
PDF
Presentation database security audit vault & database firewall
PDF
Requirement Based Intrusion Detection in Addition to Prevention Via Advanced ...
PDF
Requirement Based Intrusion Detection in Addition to Prevention Via Advanced ...
PPTX
Protect Sensitive Data: Implementing Fine-Grained Access Control in Oracle
PDF
Artem Storozhuk "Building SQL firewall: insights from developers"
PDF
Review of Intrusion and Anomaly Detection Techniques
PPTX
SQLCAT: Addressing Security and Compliance Issues with SQL Server 2008
PPTX
PolicyReplay Talk
PDF
Using PostgreSQL for Data Privacy
PDF
Keep Them out of the Database
PPTX
Geek Sync | Handling HIPAA Compliance with Your Data Access
PDF
Как разработать DBFW с нуля
PPTX
Enterprise-class security with PostgreSQL - 1
PPTX
The Role of Audit Analysis in CyberSecurity
PPTX
Database Firewall from Scratch
PDF
Kg2417521755
PPTX
Security Framework for Multitenant Architecture
Raid08 dbir
Intrusion Detection System for Applications using Linux Containers
DIDAR: Database Intrusion Detection with Automated Recovery
Presentation database security audit vault & database firewall
Requirement Based Intrusion Detection in Addition to Prevention Via Advanced ...
Requirement Based Intrusion Detection in Addition to Prevention Via Advanced ...
Protect Sensitive Data: Implementing Fine-Grained Access Control in Oracle
Artem Storozhuk "Building SQL firewall: insights from developers"
Review of Intrusion and Anomaly Detection Techniques
SQLCAT: Addressing Security and Compliance Issues with SQL Server 2008
PolicyReplay Talk
Using PostgreSQL for Data Privacy
Keep Them out of the Database
Geek Sync | Handling HIPAA Compliance with Your Data Access
Как разработать DBFW с нуля
Enterprise-class security with PostgreSQL - 1
The Role of Audit Analysis in CyberSecurity
Database Firewall from Scratch
Kg2417521755
Security Framework for Multitenant Architecture
Ad

Recently uploaded (20)

PDF
Peak of Data & AI Encore- AI for Metadata and Smarter Workflows
PPTX
Big Data Technologies - Introduction.pptx
PDF
The Rise and Fall of 3GPP – Time for a Sabbatical?
PDF
7 ChatGPT Prompts to Help You Define Your Ideal Customer Profile.pdf
PPTX
sap open course for s4hana steps from ECC to s4
PPTX
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
PDF
Agricultural_Statistics_at_a_Glance_2022_0.pdf
PDF
Build a system with the filesystem maintained by OSTree @ COSCUP 2025
PPTX
MYSQL Presentation for SQL database connectivity
PDF
Approach and Philosophy of On baking technology
PDF
Building Integrated photovoltaic BIPV_UPV.pdf
PPTX
Programs and apps: productivity, graphics, security and other tools
PPT
“AI and Expert System Decision Support & Business Intelligence Systems”
PPTX
Detection-First SIEM: Rule Types, Dashboards, and Threat-Informed Strategy
PDF
Review of recent advances in non-invasive hemoglobin estimation
PPTX
Understanding_Digital_Forensics_Presentation.pptx
PDF
Mobile App Security Testing_ A Comprehensive Guide.pdf
PDF
Dropbox Q2 2025 Financial Results & Investor Presentation
PDF
Unlocking AI with Model Context Protocol (MCP)
PPTX
20250228 LYD VKU AI Blended-Learning.pptx
Peak of Data & AI Encore- AI for Metadata and Smarter Workflows
Big Data Technologies - Introduction.pptx
The Rise and Fall of 3GPP – Time for a Sabbatical?
7 ChatGPT Prompts to Help You Define Your Ideal Customer Profile.pdf
sap open course for s4hana steps from ECC to s4
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
Agricultural_Statistics_at_a_Glance_2022_0.pdf
Build a system with the filesystem maintained by OSTree @ COSCUP 2025
MYSQL Presentation for SQL database connectivity
Approach and Philosophy of On baking technology
Building Integrated photovoltaic BIPV_UPV.pdf
Programs and apps: productivity, graphics, security and other tools
“AI and Expert System Decision Support & Business Intelligence Systems”
Detection-First SIEM: Rule Types, Dashboards, and Threat-Informed Strategy
Review of recent advances in non-invasive hemoglobin estimation
Understanding_Digital_Forensics_Presentation.pptx
Mobile App Security Testing_ A Comprehensive Guide.pdf
Dropbox Q2 2025 Financial Results & Investor Presentation
Unlocking AI with Model Context Protocol (MCP)
20250228 LYD VKU AI Blended-Learning.pptx

Mechanisms for Database Intrusion Detection and Response

  • 1. MECHANISMS FOR DATABASE INTRUSION DETECTION AND RESPONSE ASHISH KAMRA Electrical and Computer Engineering Ph.D DefenseTalk May 12 2010 Purdue University 1
  • 2. Thesis Statement To Build a RealTime Anomaly Detection and Response Mechanism for Databases Integrated with the Core Database Operations 2
  • 5. Importance of Data  Most sensitive and proprietary data resides in a DBMS  A database breach can be extremely costly  Loss of Reputation  Lawsuits  … 5
  • 6. Premise 6 DBMS users and applications possess more privileges than required to carry out the task Access control solutions alone may not enough !
  • 7. Insider Threat Scenario 7 SSN Credit Card Security Code DatabaseAdministrators accessing sensitive application data Disease Patient Name Diagnosis Privacy breaches by curious employees
  • 8. Related Systems  Oracle DatabaseVault  3rd Party Database Activity Monitoring Products 8
  • 9. Why Anomaly Detection?  Not possible to “specify” all database malicious actions  Key Issue: false positive rate  We augment the detection mechanism with a response mechanism to help with false positives 9
  • 10. Why DBMS Integration?  Independence from the underlying network infrastructure  think moving to a cloud provider  More control on response actions in real time  Nobody else has yet tried it ! 10
  • 11. Organization  System Components  Anomalous Access Pattern Detection  Anomaly Response Mechanism  Privilege State Based Access Control  JointThreshold Policy Administration 11
  • 12. System Components Query User Features Assessment Profile Creator Log Audit Log Training Queries TRAINING PHASE Detection Engine Response Engine Response Policy Base Feature Selector Profiles Alert Drop No Action, Update Profiles Privilege State Based Access Control 12 Joint Administration Model
  • 13. Thesis Contributions  AnomalousAccess Pattern Detection in Databases (ACSAC 2005,VLDB Journal 2008)  Response Mechanism (SDM 2008,TKDE 2010)  Response Policy Language  Policy MatchingAlgorithms  Privilege State Based Access Control for Fine Grained Intrusion Response (In submission RAID 2010)  JointThreshold Administration Model for Response Policy Administration (TKDE 2010, POLICY 2010)  Prototype Implementation in the PostgreSQL open source DBMS 13
  • 14. ANOMALOUS ACCESS PATTERN DETECTION 14 Query User Features Assessment Profile CreatorAudit Log Training Queries TRAINING PHASE Detection EngineFeature Selector Profiles
  • 15. Example Normal SQL Commands EMPLOYEE TRAINING PERFORMANCE HR SCHEMATABLES Anomalous SQL Commands FINANCE SCHEMA TABLES COMPENSATION ACCOUNTS PAYROLL Access Control Mechanism Access Granted Access Granted !!! 15
  • 16. Key Ideas  Extract features from the SQL Queries  Experiment with feature extraction at different granularity levels  3 granularity levels considered  Two Scenarios Considered  Role basedAnomaly Detection  UnsupervisedAnomaly Detection 16
  • 17. SQL Query Representation - I  Coarse - Command - Num tables in the query - Num columns in the query 17
  • 18. SQL Query Representation - II  Medium - Command - Tables in the query - Num columns per table 18
  • 19. SQL Query Representation - III  Fine - Command - Tables in the query - Columns in the query 19
  • 20. Role Based Anomaly Detection  Profiles per Role  Profile contain frequency counts for the various features extracted from the SQL query  Role Profiles input to the Naïve Bayes Classifier  One Role One Class  Single Role Activation Assumption  Classifier Predicted Role != Activated Role ? Anomaly : 20
  • 21. Naïve Bayes Model – medium  DB =T1(c1,c2,c3,c4) ;T2(c1,c2,c3,c4) P(R = r1 | T1C = 2,T2C = 0, Cmd=Select) P(T1C = 2,T2c = 0, Cmd = Select | R = r1) * P(R =r1) =P(T1C = 2 | R=r1) * P(T2C = 0 | R=r1) * P(Cmd = Select | R=r1) * P(R=r1) Query Under consideration : Select c1,c2 fromT1 Role Cmd T1C T2C  21
  • 22. Why Naïve Bayes ?  Low computational complexity  Works well in practice even if the attribute independence condition is not met  Ease of Implementation 22
  • 23. Role Based Anomaly Detection on Real Data QuipletType False Negative (%) False Positive (%) c 2.6 19.2 m 2.4 17.1 f 2.4 17.9 23
  • 24. Implementation in PostgreSQL Postmaster Main process Postgres Server process Server Initialization Code Query ParseTree Query Rewritten ParseTree Query Plan AccessControl Enforcement Query Executor Connect Spawn a new server process Submit SQL Query Query results Collect role login stats Collect table initialization stats Collect table/column access stats per role Waiting for Query Send detection stats to the statistics collector process Anomaly Detection Anomaly Response Yes No 24
  • 25. Feature Extraction Architecture . . Collect stats in memory resident data structures . . . . . Send messages to the statistics collector Process . Periodically read stats file Postgres server process Query executor . . Receive messages and store stats in memory resident data structures . . . . . . Periodically write the stats to the pg_stat file . . Statistics collector process Stats profile creator pg_stat file writeread 25
  • 26. Performance Impact - Experimental Setup  Base Database Size ( x )  5 tables with indexes on primary keys  2 user columns per table (all integer type)  7 system columns per table  100 rows per table  EachTransaction -  5 select queries  5 update queries  5 Insert queries 26
  • 27. Statistics Collection Overhead 0 2 4 6 8 10 12 14 Overhead(%) Database Size vs Transaction Processing Overhead coarse medium fine 27 Takeaway: Statistics Collection overhead < 15 %
  • 28. Detection Algorithm Time medium 0 5 10 15 20 25 30 35 x (6) 2x (11) 4x (21) 8x (41) 16x (81) DetectionTime(microseconds) Database Size vs DetectionTime 28
  • 29. Detection Algorithm Time fine 0 50 100 150 200 250 300 350 400 x (51) 2x (121) 4x (321) 8x (961) 16x (3201) DetectionTime(microseconds) Database Size vs DetectionTime 29
  • 30. Detection Process Overhead 0 10 20 30 40 50 60 x(5,2) 2x(10,4) 4x(20,8) 8x(40,16) 16x(80,32) Overhead(%) Database Size vs Transaction Processing Overhead 30 Takeaway: Large overhead for fine triplets for large DB size
  • 31. Unsupervised Anomaly Detection  No Role Information  Partition training data in clusters of similar queries  K centers  K means  Map every user to its representative cluster 31
  • 32. Unsupervised Anomaly Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C1 C2 C3 C4 U1 C3 U2 C3 U3 C1 U4 C2 U5 C4 U6 C1 U7 C4 U8 C3 U9 C2 32
  • 33. Detection Methodology  Apply Naïve Bayes with the cluster as the class  Outlier detection in the representative cluster 33
  • 34. Future Research Directions  Role BasedAnomaly Detection  Multiple Role Activation  Considering Role Hierarchies  …  Unsupervised Detection  Implementation in PostgreSQL  One Class SupportVector Machine  …  Query Result Intrusion Detection  Machine Learning Issues  Concept Drift  Overfitting  … 34
  • 36. Contributions  Response policy language  Response policy matching  Support for fine grained response actions  Privilege State BasedAccessControl  Response policy administration  JointThresholdAdministration Model 36
  • 37. Response Policy Language Interactive Event-Condition-Action Policy Language ON EVENT IF CONDITION THEN INITIAL ACTION CONFIRM CONFIRMATION ACTION ON SUCCESS RESOLUTION ACTION ON FAILURE FAILURE ACTION 37
  • 38. POLICY ATTRIBUTES ATTRIBUTE DESCRIPTION CONTEXTUAL User The user associated with the request. Role The role associated with the request. Client App The client application associated with the request. Source IP The IP address associated with the request. Date/Time Date/Time of the anomalous request. STRUCTURAL Database The database referred to in the request. Schema The schema referred to in the request. ObjType The object types referred to in the request such as table, view etc Obj(s) The object name(s) referred in the request SQLCmd The SQL Command associated with the request Obj Attr(s) The attributes of the object(s) referred in the request. 38
  • 39. POLICY PREDICATES Pr1 Role != DBA Pr2 Source IP IN 192.168.0.0/16 Pr3 Objs IN {dbo.*} Pr4 Time BETWEEN 0800 – 1700 Pr5 SQLCmd IN {Select} Pr7 SQLCmd IN {Insert,Update,Delete} POLICY CONDITIONS C1 Pr1 ^ Pr7 C2 Pr2 ^ Pr4 ^ Pr6 39
  • 40. RESPONSE ACTIONS ACTION DESCRIPTION CONSERVATIVE : LOW SEVERITY NOP No OPeration. This option can be used to filter unwanted alarms. LOG The anomaly details are logged. ALERT A notification is sent. FINE-GRAINED : MEDIUM SEVERITY TAINT The request is audited. SUSPEND The request is put on hold till SUCCESSFUL execution of a confirmation action. AGGRESSIVE : HIGH SEVERITY ABORT The anomalous request is aborted. DISCONNECT The user session is disconnected. REVOKE A subset of user-privileges are revoked. DENY A subset of user-privileges are denied. 40
  • 41. Response Policy Example Re-authenticate unprivileged users who are logged from inside the organization’s internal network for write anomalies to tables in the dbo schema. If re-authentication fails, drop the request and disconnect the user else do nothing. ON ANOMALY DETECTION IF Role != DBA and Source IP IN 192.168.0.0/16 and ObjType = table and Objs IN dbo.* and SQLCmd IN {Insert,Update,Delete} THEN SUSPEND CONFIRM RE-AUTHENTICATE ON SUCCESS NOP ON FAILURE ABORT REQUEST, DISCONNECT USER 41
  • 42. Policy Matching Problem  AA = {A1, A2, . . . , An} be the set of anomaly attributes.  POL ={Pol1, Pol2, . . . , Polk} be the set of response policies.  PR = {Pr1, Pr2, . . . , Prm} be the set of all distinct policy predicates.  Poli(C) be the policy condition for a policy Poli  AAS : A1 = a1, A2 = a2, . . . , An = an be the assessment of an anomaly submitted by the detection mechanism to the response system.  A policy Poli is said to matchAAS if Poli(C) = ‘true’ evaluated over AAS. The policy matching problem is to find the set of all policies in POL that match a given anomaly assessment AAS. 42
  • 44. RESPONSE ACTION SELECTION  MOST SEVERE POLICY  LEAST SEVERE POLICY 44
  • 45. Implementation in PostgreSQL  New system catalogs for storing policy attributes, predicates and definition  Policy matching invoked for every query detected as anomalous 45
  • 46. Policy Matching Overhead 0 50 100 150 200 250 300 350 400 450 12 24 36 42 60 72 84 96 108 120 PolicyMatchingOverhead (microseconds) Number of Predicates vs Policy Matching Overhead 46
  • 47. Cumulative Overhead – Real World  10 tables, 4 columns per table - under detection  24 distinct policy predicates QuipletType Time (in microseconds) Fine 130 + 20 = 150 Medium 130 + 6 = 136 47
  • 48. Fine Grained Intrusion Response  Goal:To change the access control system to support response actions such as  Request suspension  Request tainting 48
  • 49. Privilege State Based Access Control (PSAC)  Attach a “state” parameter to every privilege assigned to a user or role  We have come up with the following 5 states DENY SUSPEND TAINT GRANT UNASSIGN 49
  • 50. PSAC Details  “DENY” state supports negative authorizations  “SUSPEND” state supports request suspension  “TAINT” state supports request tainting  “GRANT” state supports standard SQL GRANT  “UNASSIGN” state supports standard SQL REVOKE 50
  • 51. State Dominance Relationship X means ‘X’ overrides ‘Y’ DENY SUSPEND TAINT UNASSIGN GRANT Y 51
  • 52. State Transitions + / + + ?? ? / / / + grant deny ? suspend / unassign taint ? TAINT SUSPEND DENY GRANT REVOKE ? 52
  • 53. Considering Role Hierarchies  Consider a role hierarchy based on privilege inheritance  Parent role inherits the privileges of children role  What about privileges in “deny”, “suspend” and “taint” states? R_parent {insert} R_child {select} {select} 53
  • 54. Privilege Orientation Modes R8 R5 R6 R7 R2 R3 R4 R1 down / neutral deny | taint | suspend up unassigned | grant 54
  • 55. Implementation in PostgreSQL  New SQL commands to support adhoc privilege state transitions  Additional Access Control Lists on the DB objects to support privilege states and orientation modes  Re-authentication procedure for a privilege in “suspend” state 55
  • 56. Results on Overhead of PSAC Maintenance – No Role Hierarchy 0 10 20 30 40 50 60 16 32 64 128 256 512 Overhead(microseconds) ACL Size vs Access Control Enforcement Overhead BASE PSAC 56
  • 57. Results on Overhead of PSAC Maintenance–With Role Hierarchy 0 20 40 60 80 100 120 16 32 64 128 256 512 Overhead(microseconds) ACL Size vs Access Control Enforcement Overhead BASE PSAC 57
  • 58. Cumulative Overhead – Real World  10 tables, 4 columns per table - under detection  24 distinct policy predicates  Size of Acl = 16 (with role hierarchy) QuipletType Time (in microseconds) Fine 130 + 20 + 22= 172 Medium 130 + 6 + 22 = 158 58
  • 59. Policy Administration  Assumptions  The response policies maintained as database objects  DBAs responsible for policy administration  The policies need to be created to monitor DBAs as well Key Issue: How can a DBA be trusted to maintain policies to monitor himself? 59
  • 60. Organization Scenario  Medium size organizations  Few Database Administrators managing large number of databases  Each DBA has all privileges in the system 60
  • 61. Joint Threshold Administration Model Key Ideas A single DBA not trusted but threat is mitigated if the trust is distributed among multiple DBAs A policy operation is “complete” when authorized by atleast “k” DBAs Use of threshold digital signatures to maintain policy integrity 61
  • 62. Threshold signatures . . k l Signature share message Secret key Secret key Secret key Signature share Signature share FINAL SIGNATURE PUBLIC KEY VERIFY 62
  • 63. PRACTICAL RSA THRESHOLD SIGNATURES – VICTOR SHOUP  Asynchronous signature share generation  Asynchronous signature share combining  Public signature shares  Efficient signature verification mechanism 63
  • 64. JTAM Set Up  Trusted Dealer – DBMS  “l” = number of DBAs  For each k, 2 <= k <= l -1  Generate RSA public-private key pair  “l” secret key shares using the private key  private key destroyed after set-up 64
  • 65. Response Policy Lifecycle 65 Create Response Policy … Joint Adm by k Users; Authorize Response Policy [Policy ID]Create; (k-1)th time Suspend Response Policy [Policy ID]; Drop Response Policy [Policy ID] ; Authorize Response Policy [Policy ID] Drop; (k-1)th time Authorize Response Policy [Policy ID] Suspend; (k-1)th time Alter Response Policy [Policy ID] …. ;
  • 66. Policy Creation Example Create Response Policy [Policy Data] Jointly Administered By k Users; H(Pol) = SHA1 ( Policy ID,Conditions, Initial Action(s),Optional Action(s), k, State). W(DBA1) : Signature share of DBA 1 PolID PolData k r hash sig shares state final sig … …. 3 2 H(Pol) W(DBA1) CREATED 66
  • 67. Policy Activation Authorize Response Policy [Policy ID] Create; PolID PolData k r hash sig shares state final sig … …. 3 1 H(Pol) W(DBA1); W(DBA3) CREATED PolID PolData k r hash sig shares state final sig … …. 3 0 H(Pol) ACTIVATED Wfinal Policy authorization by DBA3 Policy authorization by DBA4 67
  • 68. Attacks and Prevention  Signature ShareVerification  Final SignatureVerification  Malicious Policy Update  Malicious Policy Deletion  Signature Replay Attack  Policy Replay Attack 68
  • 69. Implementation in PostgreSQL  Enhanced system catalogs for storing the signature shares and final signature  Currently, implemented signature verification for every policy during the policy matching procedure  A dedicated signature verification process (to be done) 69
  • 70. Signature Verification Overhead 0 20 40 60 80 100 120 140 160 180 128 256 512 1024 VerificationOverhead (microseconds) Num RSA bits vs SignatureVerification Overhead 70
  • 71. Work in Progress  Implementation of Joint Administration of sensitive database commands such as “grant/revoke, user addition/modification“  POLICY 2010 demo paper  Public (open source) release of the all the techniques implemented 71
  • 72. Thesis Related Publications - I 1. Elisa Bertino, Ashish Kamra, EvimariaTerzi, AthenaVakali: Intrusion Detection in RBAC-administered Databases. Applied Computer Security Applications Conference (ACSAC) 2005 2. Ashish Kamra, EvimariaTerzi, Elisa Bertino: Detecting anomalous access patterns in relational databases.Very Large DataBases (VLDB) Journal 2008 3. Ashish Kamra, Elisa Bertino, RimmaV. Nehme: Responding to Anomalous Database Requests. Secure Data Management (SDM) Workshop 2008 4. Ashish Kamra, Elisa Bertino: Design and Implementation of a Intrusion Response System for Relational Database. Transactions on Knowledge and Data Engineering (TKDE) 2010 72
  • 73. Thesis Related Publications - II 5. Ashish Kamra, Elisa Bertino: JTAM: A Joint Threshold Administration Model. POLICY 2010 Demo Paper 6. Ashish Kamra, Elisa Bertino: Database Intrusion Detection and Response. Recent Advances in Intrusion Detection (RAID) 2008 Poster 7. Ashish Kamra, Elisa Bertino: Privilege State Based Access Control for Fine-Grained Intrusion Response. RAID 2010 - In Submission 73
  • 74. Other Publications 1. MercanTopkara, Ashish Kamra, Mikhail j. Atallah, Cristina Nita-Totaru:ViWiD :Visible Watermarking Based DefenseAgainst Phishing. International Workshop on Digital Watermarking (IWDW) 2005 2. Ji-Won Byun, Ashish Kamra, Elisa Bertino, Ninghui Li: Efficient k - Anonymization Using ClusteringTechniques. DASFAA 2007 3. Elisa Bertino, Ashish Kamra, James P. Early: Profiling DatabaseApplication to Detect SQL Injection Attacks. IPCCC 2007 – Invited Paper 4. Ashish Kamra: Mechanisms for Database Intrusion Detection and Response. SIGMOD PhDWorkshop on Innovative Database Research IDAR 2008 5. Book Chapters 1. Elisa Bertino, Ji-Won Byun, Ashish Kamra. Database Security. Security, Privacy, andTrust in Modern Data Management 2007. Petkovic, Milan; Jonker, Willem (Eds.) 2. Ashish Kamra, Elisa Bertino. Survey of Machine Learning Methods for Database Security. Machine Learning in CyberTrust 2009. 74
  • 75. Work Experience 1. SQL Server DatabaseAdministrator at ITAP from Feb 2006 to Dec 2008. 2. Internship, EMC, Rainfinity – May to Jul 2007 Implemented a RBAC mechanism in their file virtualization product line. 3. Internship, EMC, NAS Engineering - May to Dec 2009 1. Security analysis of Network Filesystem version 4.1 protocol, 2. Distributed File System Manageability. 4. Working Full-time in EMC’s Integrated Systems and Components group in the Unified Storage Division since Jan 11 2010. 75
  • 76. 76
  • 78. M-estimate of probability P(Ai) = freq(A=a) + M(Pinitial(A=a) total_count(A) + M 78
  • 79. JTAM Set Up RSA Public Private Keys The DBMS chooses p, q as two large prime numbers such that p = 2p′ + 1 and q = 2q′ + 1 where p′ and q′ are themselves large primes Let n = p * q be the RSA modulus. Let m = p′ * q′.The DBMS also chooses e as the RSA public exponent such that e > l. Thus, the RSA public key is PK = (n, e). The server also computes the private key d $in$ Z such that de = 1 mod m. 79
  • 80. JTAM Set Up  Secret Shares  For this purpose, the DBMS sets a0 = d and randomly assigns ai from {0, . . . ,m − 1} for 1 <= i <= k − 1. The numbers {a0 . . . ak−1} define the unique polynomial p(x) of degree k − 1 For 1 <= i <= l, the server computes the secret share, si, of each DBA, DBAi, as follows: si = p(i) mod m. 80
  • 81. Signature Share Generation Let x = H(Pol). W(DBA1) = x2 s1 Where = l! W(DBA1) does not leak any information about the secret share s1 because of the intractability of the generalized discrete logarithm problem 81

Editor's Notes

  • #71: With the typical modular exponentiation algorithms used to implement the RSA algorithm, public key operations take O(k2) steps, private key operations take O(k3) steps, and key generation takes O(k4) steps, where k is the number of bits in the modulus. ``Fast multiplication'' techniques, such as methods based on the Fast Fourier Transform (FFT), require asymptotically fewer steps. In practice, however, they are not as common due to their greater software complexity and the fact that they may actually be slower for typical key sizes.