Soa12c launch 5 event processing shmakov eng cr

Copyright © 2014, Oracle and/or its affiliates. 1 All rights reserved.
SOA Suite 12.1.3 Field Enablement
Oracle Event Processing
Enabling Fast Data and
The Internet of Things
Technical Presentation

 Modern Computing Challenges
 Solution Product Overview
 Oracle Event Processing Applications
 Continuous Query Language (CQL)
 Internet of Things (IoT)
 What is Fast Data?
 12c Features
Agenda
Oracle Event Processing 12c
Sales Enablement

The application evaluates if the roaming asset (device) is within a certain
distance of the business. If it is, the system will generate an event to send the
customer a promotional message.
This powerful example encompasses the integration of JMS, Real Time Spatial
Analysis and Big Data integration
Mobile Marketing - OEP in Action: DEMO

Modern Computing Challenges
Does the problem you want to solve have any one or more of the following
conditions?
 Requires high-throughput and low-latency processing
 Continuously streaming data (Log Files, converting Batch to Real Time)
 Real-time correlation between multiple incoming data sources
 Time-sensitive alerts, aggregations and calculations
 Needs to look for patterns in the data stream (Spatial Analysis)
 Identify when some event(s) should have happened and did not.
 Data does not need to be stored, if there is nothing of interest in it
 Problem is more easily solved by analyzing before storing in DB
Conditions for Event Processing Opportunities

Solution Product Overview
Solution involves one or more of the following:
 High Volume
 Continuous Streaming
 Sub-Millisecond Latency
 Disparate Sources
 Time-Window Processing
 Pattern Matching
 Business Event Visualization
OEP
Streaming
Event Data
Alerts,
Actions
Filtering,
Pattern Matching,
Missing Events,
Aggregations,
Correlations,
Calculations,
Geo-Spatial

Solution Product Overview
Inverted Database
 Data is ‘static’
 Queries are ‘dynamic’
• Data (event) is ‘dynamic’
• Queries are ‘static’

POJO
App Frameworks
Spring Services
Stream
Management
Core Event Infrastructure
Complex Event
Processor
Real-Time
Kernel
Data Cartridges
Extended Event Infrastructure
Cluster Management HTTP Pub/Sub Engine
Event Repository
Coherence
Foundation Services
Security Logging
 Processes Streaming Data In-Memory & in
Real-Time (274 OSGi Bundles)
 Event Analysis: Pattern Matching, Missing
Events, Aggregations, Correlations,
Calculations, Coherence Integration,
Spatial Functions, Big Data Integration
 Performs Aggregations and Correlates
Multiple Streams of Data
 Integrates easily with other Oracle
products: Big Data (Hadoop & NoSQL) –
FAST DATA, SOA (EDN), Coherence,
Spatial, RTD, etc.
Streaming Event Data
Alerts, Actions
Solution Overview: Oracle Event Processing (OEP)
OSGi

Solution Overview
Filtering, Correlation & Aggregation
 New stream filtered for specific criteria, e.g. stock price > $22
 Process events from different streams together
 Scrolling, time-based window metrics, e.g. average # of stock trades in
the last hour
Detect Absence of Events and Missing Events
 Event “A” NOT followed by Event “B” within 10 minutes
 Event “A”, Event “B” should occur next, but Event “C” occurs instead

Perform Calculations in Real-Time
Perform Calculations on Data From Sensors:
 Use last X seconds or minutes of data OR
 Use last X data points
 Smooth out “noise” in the data

Input
Adapter
Channel
Input
Adapter
Channel
Business
Logic
(CQL)
Channel
Channel
Channel
Output
Adapter
Output
Adapter
Oracle Event Processing Application
DB
OICDnahQuptatuLapn tbup naatre dsoalaescdps eah tspaestnlreposdr r cscsCo sconeohtnnnreeodtrcale dtitnn hatco tefea i dl f tcalaeoantrwcadinh soageofl,s e udc rcraotcastraen rte s oa l b andetdoi o rwcenafn,en sar tgebrngeac rtemeugdn as e tyddiosi rftneoe crma tnolsydp a itpninma dCta tQbel urpLnse p irmnrfoearcstmceshs apisnnroocgrec sbe ussiense ss logic
Coherence
Business
Logic
(CQL)
Business
Logic
(CQL)

High-Performance In-Memory Data Processing
Input
Adapter
Data
Input
Adapter
Data
Channel
Business
Logic
(CQL)
Channel
Data
Data
Analytics
Channel
Business
Logic
(CQL)
Enrich
Output
Adapter
Data Data Data Data
Analytics: Continuously Sliding Time Windows of Streaming Data, Filtering,
Correlations, Calculations, Aggregations, Pattern Matching, Missing Event Detection,
Spatial Analysis, etc.
Enrichment: Integrate with data from DB, Coherence, NoSQL, Hadoop etc.

Administration & Monitoring
View the data flow of the application
“Event Processing Network” (EPN)

Administration & Monitoring
Monitor throughput and latency between any two nodes in
the application

Oracle Event Processing & Coherence
Challenges Benefits
Handle and correlate events in real-time,
including support for multiple
patterns:
• Pre-processing (buffer OEP)
•Within OEP (to cache reference data)
• Post OEP (to expose processed events to
consuming apps)
• High throughput for storing data
• Aggregation and event querying
• Pattern implementation flexibility
combining two complementary
technologies
Data
Grid
OEP
Consolidated
& in-context
Data

Simple Filtering
SELECT *
FROM inputChannel [NOW]
WHERE eventValue > 10
Continuously calculate the last hour sales by store
SELECT SUM(amount) as salesTotal, storeID
FROM inputChannel [range 60 minutes]
GROUP BY storeID
Calculate the average of the last 2 stock ticks by stock symbol
SELECT AVG(stockPrice) as avgPrice, stockSymbol
FROM inputChannel [PARTITION BY stockSymbol ROWS 2]
GROUP BY stockSymbol
Sample CQL Queries
Filter for events meeting specific
threshold values
Running total of up- to-the-moment
sales by store
Average of the last 2 stock ticks
by symbol

CQL view that joins a cache containing
reference data to an individual streaming
event to provide additional context for further
processing.
Sample CQL Queries
Cache Join Query

Pattern Matching
 Powerful concept that allows identification of complex event patterns
 Defined as regular expressions
PATTERN (X+ Y+)
1 or more X events … … followed by 1 or
more Y events

CQL view filters events for use by a query.
CQL query that starts when an OFF event
occurs and waits 10 seconds to make sure
that it is not followed by an ON event before
sending the OFF event downstream.
Pattern Match Query

Find passengers stuck in security when their flight reaches “final boarding”.
SELECT
stuck.reservationLocator,
'STUCK' as state
FROM PassengerStateEventChannel MATCH_RECOGNIZE (
PARTITION BY
reservationLocator
MEASURES
Entered.reservationLocator AS reservationLocator
PATTERN (CheckIn Entered NotExited*? Final)
DEFINE
CheckIn AS state = 'CHECKIN',
Entered AS state = 'ENTERED',
NotExited AS state != 'EXITED',
Final AS state = 'FINAL'
) AS stuck
Sample CQL Queries
Find passengers who are stuck in
security when their flight is in the
“FINAL BOARDING” process.

Flight is in final boarding!
Find passengers stuck in
security.
Pattern Matching

Find users with no status update for 4 hours
SELECT
M.userWithNoStatus
FROM StatusEventChannel MATCH_RECOGNIZE (
PARTITION BY
user
MEASURES
user AS userwithNoStatus
INCLUDE TIMER EVENTS
PATTERN (A B)
DURATION 4 HOURS
DEFINE
A AS A.user is not null
) AS M
Sample CQL Queries
Missing Events: Finds users who report
a status, but fail to report another
status during the next 4 hours.

A market leader in security and also the innovation leader in
their industry that continuously offer to its customer’s new Smart
Home products and services.
An inventory and anti-jaming system connected through 3G
network to the alarm reception system, devices send keep Alive
(KA) signal in real-time providing Inventory system data and
manage device status (discovered, available, unavailable, error)
through the reception of the KA events. All this will be
connected to business applications and CRM.
Greater real-time insight into customer security assets and
immediate detection of competitive equipment tampering
ensuring customer retention and satisfaction.
Home security Alarm system
equipment disruption monitoring
BENEFITS SOLUTION COMPANY
Deal:
• Oracle Event Processing
• Coherence

Real Time Tracking via
GPS and resource
evaluation in relation to
virtual geographical areas
Real Time Dynamic
definition of virtual
geographical complex
(polygon) areas
Real-Time Spatial Geo-Fencing Solutions

Track school buses on a map
SELECT bus.busId as busId, bus.seq as seq,
com.oracle.cep.cartridge.spatial.Geometry.createPoint(8307, bus.longitude,
bus.latitude) as geom
FROM BusPosStream as bus
Alert when the bus arrives at the bus stop
SELECT systimestamp() as incidentTime, bus.busId as busId, busstop.seq as stopSeq
FROM BusPosGeomStream[NOW] as bus, BusStopRelation as busstop
WHERE CONTAIN@spatial(busstop.geom, bus.geom, 100.0d) = true and bus.busId =
busstop.busId
Sample CQL Queries
Track a bus on a map
Determine when the bus is near
the bus stop

Point-in-Polygon Matching
OEP detects entry into a pre-defined area

Nearest Neighbor
OEP can find the nearest place of interest (e.g. WiFi Hotspot)

Dynamic Query Generation
Business Users Modify Processing in Real-Time
Simple forms can be used to allow users to
add, modify or delete CQL queries at run-time
without stopping the application or
the server allowing existing processing to
continue running while business users
dynamically make modifications.

Dynamic Query Generation

Internet of Things (IoT)

Solution Overview: Oracle Event Processing (OEP)
 Architecture designed and Optimized for Java
Enabled Devices/Gateways (197 OSGi Bundles)
 High-speed real-time device data capture and
analysis helps mitigate the risk of down-time and
helps with continuous process improvement
 Enables local real-time process monitoring to
detect degradation of performance, thus helping
with initiation of alerts or requests for operations
and maintenance actions
 Significantly reduces noise to signal ratio of data
reaching the server, enabling savings in
bandwidth and scalability costs
 Faster time to market for devices and easy over-the-
air updates post-deployment.
OSGi

Oracle Event Processing for Java Embedded
 Runs on devices capable of running
Java SE Embedded
 Key component of Internet of Things
(IoT)
 Smaller, lighter-weight version of the
same OEP product
 No need to learn new skills to
program application for devices
OEP

OEP
OEP
OEP
OEP
OEP
OEP in Device to Data Center Deployments
Real-time local data analysis for real-world event data
• Upstream nodes
perform basic filtering
and aggregation
• Larger servers
downstream perform
complex combining and
correlation across
multiple streams
• OEP in embedded
devices allows initial
processing to be
handled by less
powerful devices near
the origin/edge.
Event Flow
Edge devices Servers Gateways Enterprise Apps
OEP
OEP
OEP
OEP

Oracle Fast Data for Big Data
Fast Data Big Data Infrastructure
Oracle BI
Foundation Suite
Oracle Real-Time
Decisions
Endeca Information
Discovery
Business Analytics
Oracle Event
Processing Oracle Big Data
Connectors
Oracle Data
Integrator
Oracle
Advanced
Analytics
Oracle
Database
Oracle
Spatial
& Graph
Apache
Flume
Oracle
GoldenGate
Oracle
NoSQL
Database
Cloudera
Hadoop
Oracle R
Distribution

1. Big data ≠ Infinite storage
Yes, storage is cheap but it helps to have clean
data, with context and less redundancy
2. Hadoop is batch-oriented and there is
inherent latency
"With the paths that go through Hadoop [at
Yahoo!], the latency is about fifteen minutes […] it
will never be true real-time. " *
Raymie Stata, Yahoo! CTO
(June 2011)
Some Challenges Working with Big Data
: http://www.theregister.co.uk/2011/06/30/yahoo_hadoop_and_realtime/
minutes

1. Filter out noise (ex: data ticks with no
change), add context (by correlating
multiple sources), increase relevance
1. Identify certain critical conditions as you
insert data into the warehouse
Move time-critical analysis to front of process
Get Ahead of the Curve
Filter out,
correlate
Use Event Processing Techniques

Fast Data
Getting Ahead of the Curve
Big Data
ms minutes
Fast Data
Historical depth: deep
Historical depth:
shallow
Example:
analysis of traffic
patterns and
congestion times
for urban
planning
Example:
monitoring of traffic
cameras to ensure
given license plates
are not in use on
multiple vehicles
Add “depth” to your fast data by
merging output of MapReduce
to stream processing

COMPANY OVERVIEW
• Motorola Solutions connects people through technology
• Motorola Solutions serves both enterprise and government
customers with core markets in public safety government
agencies and commercial enterprises
• Global presence with 23,000 employees worldwide in 65
countries with sales in over 100 countries
CHALLENGES/OPPORTUNITIES
• Provide a Fast Data intelligence layer for Big data streaming
video feeds
• Event Processing analyzes Real Time Video meta-data for
distinct patterns of interest to Government agencies and
law enforcement
• Centralized Exalogic implementation
PROJECT OBJECTIVES
• Projects worldwide using a reusable scalable
architecture supporting 2800 or more cameras
• Streaming video meta data interfaces with
IOmniscient processing for Face recognition and
License plate monitoring
• Event patterns for duplicate plates within
temporal period beyond distance capabilities
• Speeding analysis with driver recognition
RESULTS
• Ongoing partner relationship delivering
planned projects in Mexico, Oman
• Prototyping successfully completed with live
video feed data
• Foundational technology for Smart Cities
platform solutions
How Much Data Will Humans Create & Store
This Year (2011)?
1.8 zettabytes being created and replicated (Streaming Video)
57.5 billion 32 GB iPads, How much is that? About $34.4 trillion worth.
http://mashable.com/2011/06/28/data-infographic/
Motorola
Smart IPVS : Vehicle Capacity & Flow Control
WARNING:
Suspect License
plate – CABO .
MEXICO CITY
WARNING: Driver
Criminal Record

JDeveloper IDE for OEP 12c

JDeveloper IDE for OEP 12c
• Create Event Processing Network
• Graphically develop an OEP application
• Drag EPN components to canvas and configure
• Deploy to Configured OEP Servers
• Define OEP Servers and Deployment Profiles

Improved Installer
• Installer and Domain Creation
Wizard Improvements
• Mac OS X is supported for
development.

New and Improved Adapters
New File Adapter
• Doesn’t require the “Load
Generator Utility”.
• Easily configure all the required
parameters when defining the
adapter in Jdev.

CQL Enhancements
Variable Duration For Pattern Match Queries
Duration for the pattern
match query can be
specific to each event.

CQL Pattern Templates
Starts the Query Logic for the Developer

List of 12c Enhancements
• CQL enhancements
• Spatial Improvements
• Application compiler
• Application and CQL testing
• New and Improved adapters
• Public Cartridge API
• Improved Installer
• Improved Coherence Cartridge
• Support for JDeveloper
• EDN integration
• Business Rules integration
• Performance optimizations

Request
Request
Event Data Event Data
SOA Composite
Instances
OEP Application
SOA
EPN
EDN Integration SOA Composite vs.
Event Processing Network

Customer1
$600
Customer1
$600
Customer1
$700
> $1000 per
customer in 24 hours
Customer1
$700
EDN Integration

DEMO

Oracle Event Processing (OEP) Summary
 High-Volume Low-Latency Event Processing Infrastructure
 Event Processing Network (EPN)
 Light-weight Java Application Server (embeddable)
 Easily Customizable
 Integrate with existing infrastructure and other Oracle Products (e.g.
Coherence, Business Activity Monitoring, Database, Big Data
Appliance, Data Mining, Spatial, NoSQL Database etc.)
 Time Management & Pattern Matching
 Continuously Perform Calculations Over Time Windows
 Partition Event Streams By Key Values
 Perform Complex Pattern Matching, Detect Missing Events
 Adjust Core Business Logic in Real-time without Redeploying

Fast Data Business Integrator and User Tooling
• Fast exploration of Real
Time Streams
• Fast definition of Real
Time Event Patterns
• Fast Testing and
Deployment of Projects
• Fast Export to
Development Environment
for more elaborate
prototyping

Summary: When To Use OEP
1 Business Logic Layer for Event-Driven/Coherence Applications
2 High-Volume Business Activity Monitoring Applications
3 Real-Time Spatial Applications
4 Fast Data: Real-Time Requirements with Big Data Infrastructure
5 High Volume Batch to Real-Time Conversion Projects
6 Internet of Things (IoT): Processing Data On and From Devices
7 Pattern Matching / Missing Events / Alerting / Fraud Detection

Soa12c launch 5 event processing shmakov eng cr

More Related Content

What's hot (20)

Viewers also liked (6)

Similar to Soa12c launch 5 event processing shmakov eng cr (20)

More from Vasily Demin (12)

Recently uploaded (20)

Soa12c launch 5 event processing shmakov eng cr