Complex Event Processing

CEP Overview
1. CEP Definitions and Terms
2. Use Case Example Architecture
3. Architecture of our Event(s)
4. CEP Patterns & Rules
5. Temporal Reasoning / Complexity of Rules
6. Implemented Rules
7. Allen’s (5 of 13) Temporal Operator Diagrams
8. Example Output ( & code overview), App Demo.
9. Forward Chaining Inference Engine
10. Backward Chaining Inference Engine
11. Comparison: Forward vs Backward Chaining

Complex Event Processing
Dr. David Luckham's book "The Power of Events“ - 2002.
Event Stream [ needle(s) in haystack(s) ]Event Stream [ needle(s) in haystack(s) ]
Time Zero Time Infinity
e1, e2, e3, e4, e5, e6, e7, e8 ,e9, e10, e11, e12, e13, e14, e15, e16, e17, e18, e19, e20, e21, e22, e23, ….
Definitions:
Events – “something of interest that happens”: (1) AccessAttempt, (2) DataAccessAttempt.
Event Streams – “massive streams of data” (some important, most not).
Temporal Reasoning – “reasoning over time, factoring in time during reasoning”. The
computer must have a way to understand time and reason while incorporating it.
Complex Event Processing – “the task of processing multiple events with the goal of
identifying the meaningful events (over time) within the event cloud (stream).”
* Detection of complex patterns of many events,
* Event Correlation, Abstraction, and Hierarchies,
* Relationships between events: (causality, membership, timing, etc…)
Common Examples:
* Stock Broker Application: Ticker Monitor
* Our Example: Military ID Monitor System

Example Use Case:
ID Events Processing
Events: AccessAttempts and DataAccessAttempts
OFFICE
CAFE
BARRACKS
• No multiple logins permitted.
• The only area where access to restricted data
is permitted.
• No logins from unknown origins permitted.
• No repeat logon failures permitted.
• Accessing restricted data not permitted here
• Accessing restricted data not permitted here.
UNKNOWN
• No logins permitted from here at all.
• No data access of any kind permitted
from here.
• Alert if multiple access failures emanate
from here.
CEP: TRACKING SYSTEM

Architecture of Events: (Java Objects)
public class AccessAttempt {
private String employeeId;
private String location;
private String computerId;
private String action;
private String result;
……….
} // End Class LoginAttempt
public class DataAccessAttempt {
private String dataClassification;
private String computerId;
private String employeeId;
…….
} // End Class DataAccessAttempt
Correlated Attributes
• employeeId
• computerId
private static final String LOGIN = "login";
private static final String LOGOUT = "logout";
private static final String SUCCESSFUL = "successful";
private static final String FAILURE = "failure";
private static final String TOPSECRET = "topsecret";
private static final String SECRET = "secret";
private static final String CLASSIFIED = "classified";
private static final String UNCLASSIFIED = "unclassified";
private static final String OFFICE = "office";
private static final String CAFE = "cafe";
private static final String BARRACKS = "barracks";
private static final String UNKNOWN = "unknown";

Complex Patterns – Our Example:
Important, but not meaningful alone:
Login / Logout – [normal, paired, properly sequenced, singleton]
Data Access – [unclassified – anywhere known]
Data Access: [classified, secret, top secret – with in the office, proper procedure]
Important and Meaningful: – Trigger Alerts
Data Access: [classified, secret, top secret – in any public area, or from unknown places]
Login – from any unknown place.
Multiple Logins of same ID from different places
Repeat Login Failures (X failures over N minutes)
Complex Event Processing
Events: AccessAttempts and DataAccessAttempts

Expert Systems Rule
Forward Chaining Inference Engine
Complexity of Rules which reason using time (Temporal)
Example:
“Multiple Logins”
[t0] [Event A – logIN ID: “234”, “Office”] [t+10m] [Event B – logIN ID: “234”, “Cafe’”] [t+30m] [Event C – logOUT ID: “234”, “Office”]
Notes:
Event A has to happened before Event B, but Event C cannot happen before Event B.
Key Concepts:
Temporal Operators: “not”, “before”, “after”… Accumulate : (reasoning over collections of data)
Drools implements all 13 operators defined by Allen and also their logical complement (negation).
• [ALLEN81] Allen, J.F..An Interval-based Representation of Temporal Knowledge. 1981.
• [ALLEN83] Allen, J.F..Maintaining knowledge about temporal intervals. 1983.
Temporal Operators:
(1) Before, (2) After, (3) Coincides, (4) During, (5) Finishes, (6) Finished By, (7) Includes, (8) Meets, (9) Met
By, (10) Overlaps, (11) Overlapped By, (12) Starts, (13) Started By.

Implemented Rules: Arch Types
Patterns which raise alerts:
1. Concurrent logins (one ID can’t be in 2 places at the same time)
2. Multiple failed attempts during a specific time period for any location
3. Successful logins from any unknown locations
4. Any data access from any unknown location
5. Restricted data access from non-secure area (public areas)
RULE EXAMPLE:
rule "ALERT: Access of TopSecret Data from unkown Location"
when
$loginUnknown : AccessAttempt( location == "unknown", action == "login", $eid : employeeId, $cid : computerId, result == "successful") from entry-point "idsystem"
$dataAccess : DataAccessAttempt( this after $loginUnknown, dataClassification == "topsecret", employeeId == $eid, computerId == $cid ) from entry-point
"idsystem"
then
System.out.println("ALERT: TopSecret Data Breach -> [ " + $dataAccess + " ]");
end

Allen: Temporal Operator Diagrams
Temporal Operators:
(1) Before, (2) After, (3) Coincides, (4) During, (5) Finishes, (6) Finished By, (7) Includes, (8) Meets, (9) Met By, (10) Overlaps, (11)
Overlapped By, (12) Starts, (13) Started By.
Current vs Correlated Events: [Current: “Event of Focus” / Correlated: “Event that is related to”]
Event
A
Event
B
Current Event:
Correlated Event:
AFTER
Finishes
Starts
Event
B
Event
ACurrent Event:
Correlated Event:
BEFORE
Finishes
Starts
DURING
Current Event:
Correlated Event:
Starts
Event
B
Finishes
Event
A
Starts
Event
A
Finishes
Time Window: Mins, Hrs
Event
B
Time 0 Time 00)

Allen: Temporal Operator Diagrams
Event
B
OVERLAPS
Current Event:
Correlated Event:
Starts
Event
B
Finishes
Event
A
Starts
Event
A
Finishes
Temporal Operators:
(1) Before, (2) After, (3) Coincides, (4) During, (5) Finishes, (6) Finished By, (7) Includes, (8) Meets, (9) Met By, (10) Overlaps, (11)
Overlapped By, (12) Starts, (13) Started By.
Current vs Correlated Events: [Current: “Event of Focus” / Correlated: “Event that is related to”]
Event
B
INCLUDES
Current Event:
Correlated Event:
Starts
Event
B
Finishes
Event
A
Starts
Event
A
Finishes
Time 0 Time 00)

CEP Sample Output
Dec 08, 2014 12:54:13 PM com.military.idcard.IDCardEventExample main
INFO: *********************************>>>> Drools CEP ID Card Example
ALERT: Restricted Data Breach -> [ DataAccessAttempt
[dataClassification=secret, computerId=barracks3333, employeeId=3333] ]
[dataClassification=classified, computerId=cafe2222, employeeId=2222] ]
[dataClassification=topsecret, computerId=unknown, employeeId=4444] ]
ALERT: Unknown Location Login -> [ Unknown Location ] : AccessAttempt
[employeeId=4444, location=unknown, computerId=unknown, action=login,
result=successful]
ALERT: TopSecret Data Breach -> [ DataAccessAttempt
[dataClassification=topsecret, computerId=unknown, employeeId=4444] ]
ALERT: Multiple Logins: (Successful) -> [ 1st: Barracks and 2nd: cafe ] - ID: 2023
ALERT: Multiple Logins: (Successful) -> [ 1st: Cafe and 2nd: barracks ] - ID: 1023
ALERT: Multiple Logins: (Successful) -> [ 1st: Office and 2nd: cafe ] - ID: 4023
ALERT: Multiple Logins: (Successful) -> [ 1st: Office and 2nd: cafe ] - ID: 1123
ALERT: Multiple Logins: (Failures) -> CAFE 6
ALERT: Multiple Logins: (Failures) -> BARRACKS 5
ALERT: Breach Attempts - Multiple Failures: (UNKNOWN) 5

Forward Chaining Inference Engine

Backward Chaining Inference Engine

Forward vs Backward Chaining
• Backward chaining (a la Prolog) is more like finding what initial conditions form a path to
your goal. At a very basic level it is a backward search from your goal to find
conditions that will fulfill it.
• Backward chaining is used for interrogative applications (finding items that fulfill certain
criteria) - one commercial example of a backward chaining application might be finding
which insurance policies are covered by a particular reinsurance contract.
• Forward chaining (a la Drools) matches conditions and then generates inferences
from those conditions. These conditions can in turn match other rules. Basically, this
takes a set of initial conditions and then draws all inferences it can from those conditions.
• STUDY OF DIFFERENCE BETWEEN FORWARD AND BACKWARD REASONING
http://www.ijetae.com/files/Volume2Issue10/IJETAE_1012_48.pdf
• Forward chaining means applying rules in a forward direction: recursively applying the
rules over data to generate more data (and applying the rules over that data... I have a
member of po:Person... it must also be a member of foaf:Person... and so it must be
a foaf:Agent and dc:Agent... and so...)
• Backward chaining means applying rules in a backwards manner: taking a goal (e.g., a
query) and recursively working backwards to find more data that can satisfy the goal (I'm
looking for foaf:Agents... I should also look for dc:Agents and foaf:Persons
and po:Persons...)

Complex Event Processing

More Related Content

Viewers also liked (12)

Similar to Complex Event Processing (20)

Recently uploaded (20)

Complex Event Processing