A General Extension System for Event Processing Languages
1 like1,168 views
The document discusses a general extension system for event processing languages. It presents two scenarios that demonstrate how to blend the Continuous Query Language (CQL) with other languages like Java and Oracle Spatial. The proposed architecture includes a type registry and language extensions that allow CQL to interface with other languages while remaining agnostic to language specifics. This achieves extensibility while keeping CQL highly descriptive.
![Scenario 1: DEBS2011
Programming-in-the-small
Oracle CQL
Processor
SELECT *
FROM Text
news [RANGE 1 HOUR],
stock_tick [RANGE 1 HOUR]
WHERE /* join-criteria */
Monday, July 11, 2011](https://image.slidesharecdn.com/debs2011-extensionsystem-110713141039-phpapp01/85/A-General-Extension-System-for-Event-Processing-Languages-4-320.jpg)
![Java Regular Expressions DEBS2011
• Let’s look at a simple solution to the problem using
Java:
Matcher matcher =
Text
Pattern.compile("[.,; ][A-Z][A-Z][A-Z][.,; ]").matcher
(message);
if (matcher.find()) {
Matches three letter
symbol = message.substring(matcher.start() + 1, symbols, separated by
leading and trailing
matcher.end() - 1)); white-spaces
}
Monday, July 11, 2011](https://image.slidesharecdn.com/debs2011-extensionsystem-110713141039-phpapp01/85/A-General-Extension-System-for-Event-Processing-Languages-6-320.jpg)
![Blending CQL with Java DEBS2011
CREATE VIEW filtered_news(message, matcher) AS
SELECT
message,
Pattern.compile("[.,; ][A-Z][A-Z][A-Z][.,; ]").matcher(message) as
matcher
FROM news [RANGE 1 HOUR] Text
SELECT
location, item_description, message
FROM filtered_news, stock_tick[RANGE 1 HOUR]
WHERE
matcher.find() = true AND
filtered_news.message.substring(matcher.start() + 1,
matcher.end() - 1) = stock_tick.symbol
Monday, July 11, 2011](https://image.slidesharecdn.com/debs2011-extensionsystem-110713141039-phpapp01/85/A-General-Extension-System-for-Event-Processing-Languages-7-320.jpg)
![Blending CQL with Java DEBS2011
CQL char conversion
to Java String
CREATE VIEW filtered_news(message, matcher) AS
SELECT
message,
Pattern.compile("[.,; ][A-Z][A-Z][A-Z][.,; ]").matcher(message) as matcher
FROM news [RANGE 1 HOUR]
Text
SELECT
location, item_description, message Method invocation
FROM filtered_news, stock_tick[RANGE 1 HOUR]
WHERE
matcher.find() = true AND Static method
filtered_news.message.substring(matcher.start() + 1, invocation
matcher.end() - 1) = stock_tick.symbol
Monday, July 11, 2011](https://image.slidesharecdn.com/debs2011-extensionsystem-110713141039-phpapp01/85/A-General-Extension-System-for-Event-Processing-Languages-8-320.jpg)
![Blending CQL with Java DEBS2011
‘matcher’ is a Java
Object, and treated as
CREATE VIEW filtered_news(message, matcher) AS
a complex type by
SELECT CQL
message,
Pattern.compile("[.,; ][A-Z][A-Z][A-Z][.,; ]").matcher(message) as matcher
FROM news [RANGE 1 HOUR]
Text
SELECT Overload of CQL ‘+’
location, item_description, message operator to handle
FROM filtered_news, stock_tick[RANGE 1 HOUR] Java Integer
WHERE
matcher.find() = true AND Overload of CQL
filtered_news.message.substring(matcher.start() + 1, equality to handle
matcher.end() - 1) = stock_tick.symbol Java String
Monday, July 11, 2011](https://image.slidesharecdn.com/debs2011-extensionsystem-110713141039-phpapp01/85/A-General-Extension-System-for-Event-Processing-Languages-9-320.jpg)
![Architecture DEBS2011
Pattern.compile("[.,; ][A-Z][A-Z][A-Z][.,; ]").matcher
Pattern.compile("[.,; ][A-Z][A-Z][A-Z][.,; ]").matcher(message)
(message)
• Is this symbol:
• Text
A stream attribute (e.g. message)?
• A stream alias (e.g. FROM S1 as A)
• A CQL function (e.g. SELECT myfunc())
• The l-value of complex type (e.g. myObj.myMethod())
• A constructor
• A (static) method
• A class name
Monday, July 11, 2011](https://image.slidesharecdn.com/debs2011-extensionsystem-110713141039-phpapp01/85/A-General-Extension-System-for-Event-Processing-Languages-10-320.jpg)
![Architecture DEBS2011
Pattern.compile("[.,; ][A-Z][A-Z][A-Z][.,; ]").matcher
Pattern.compile("[.,; ][A-Z][A-Z][A-Z][.,; ]").matcher(message)
(message)
• To be able to semantically compile symbols, we need
to have type information.Text
• However, CQL is not aware of Java, nor should it be.
• There is a need of a generic type-system, and an
extension model for providers to plug-in their own
languages
Monday, July 11, 2011](https://image.slidesharecdn.com/debs2011-extensionsystem-110713141039-phpapp01/85/A-General-Extension-System-for-Event-Processing-Languages-11-320.jpg)
![Architecture DEBS2011
Pattern.compile("[.,; ][A-Z][A-Z][A-Z][.,; ]").matcher
Pattern.compile("[.,; ][A-Z][A-Z][A-Z][.,; ]").matcher(message)
(message)
ComplexType
“Pattern”
Text
Type fields
Registry
methods
Java
Type-System
Monday, July 11, 2011](https://image.slidesharecdn.com/debs2011-extensionsystem-110713141039-phpapp01/85/A-General-Extension-System-for-Event-Processing-Languages-13-320.jpg)
![Architecture DEBS2011
Pattern.compile("[.,; ][A-Z][A-Z][A-Z][.,; ]").matcher
Pattern.compile("[.,; ][A-Z][A-Z][A-Z][.,; ]").matcher(message)
(message)
Pattern
ComplexType
fields Text
Find a “compile” method and its return type
methods
Monday, July 11, 2011](https://image.slidesharecdn.com/debs2011-extensionsystem-110713141039-phpapp01/85/A-General-Extension-System-for-Event-Processing-Languages-15-320.jpg)
![Architecture DEBS2011
Pattern.compile("[.,; ][A-Z][A-Z][A-Z][.,; ]").matcher(message)
Pattern Matcher
ComplexType ComplexType Text
fields fields Store ‘matcher’ as
methods methods a stream/view attribute
Monday, July 11, 2011](https://image.slidesharecdn.com/debs2011-extensionsystem-110713141039-phpapp01/85/A-General-Extension-System-for-Event-Processing-Languages-16-320.jpg)
![Blending CQL with Spatial DEBS2011
CREATE VIEW CustomerLocation-Stream(point, custId) AS
SELECT createPoint@spatial(lng, lat) as point, custId
FROM Location-Stream
point is a spatial type
SELECT loc.custId, shop.id Text
FROM
CustomerLocation-Stream[NOW] AS loc, Shop as shop
WHERE
contain@spatial(shop.geometry, loc.point, 2.0d) ‘spatial’ link points to
Oracle Spatial
‘extension’, where
‘contain’ function
resides
Monday, July 11, 2011](https://image.slidesharecdn.com/debs2011-extensionsystem-110713141039-phpapp01/85/A-General-Extension-System-for-Event-Processing-Languages-20-320.jpg)
A General Extension System for Event Processing Languages
- 1. <Insert Picture Here> A General Extension System for Event Processing Languages Alexandre Alves - Oracle CEP Monday, July 11, 2011
- 2. Agenda DEBS2011 • Scenario 1 • String manipulation (programming-in-the-small) • Blending CQL and Java • Architecture • Scenario 2 Text • Geo-fence (extensibility) • Blending CQL and Spatial • Architecture • Q/A Monday, July 11, 2011
- 3. Scenario 1: DEBS2011 Programming-in-the-small • Correlate security price changes to real-time event news. Text Monday, July 11, 2011
- 4. Scenario 1: DEBS2011 Programming-in-the-small Oracle CQL Processor SELECT * FROM Text news [RANGE 1 HOUR], stock_tick [RANGE 1 HOUR] WHERE /* join-criteria */ Monday, July 11, 2011
- 5. Scenario 1: DEBS2011 Programming-in-the-small • Problem: the NEWS feed is a unstructured (String), hence not trivial to identify its event properties • For example, a naive join criteria is to look for a stock’s symbol • The problem of parsing a String can be solved using Regular Expressions. Text • Regular Expressions and other programming-in-the- small tasks have long been solved by general purpose programming languages • Can we leverage this within an event processing language (e.g. CQL)? Monday, July 11, 2011
- 6. Java Regular Expressions DEBS2011 • Let’s look at a simple solution to the problem using Java: Matcher matcher = Text Pattern.compile("[.,; ][A-Z][A-Z][A-Z][.,; ]").matcher (message); if (matcher.find()) { Matches three letter symbol = message.substring(matcher.start() + 1, symbols, separated by leading and trailing matcher.end() - 1)); white-spaces } Monday, July 11, 2011
- 7. Blending CQL with Java DEBS2011 CREATE VIEW filtered_news(message, matcher) AS SELECT message, Pattern.compile("[.,; ][A-Z][A-Z][A-Z][.,; ]").matcher(message) as matcher FROM news [RANGE 1 HOUR] Text SELECT location, item_description, message FROM filtered_news, stock_tick[RANGE 1 HOUR] WHERE matcher.find() = true AND filtered_news.message.substring(matcher.start() + 1, matcher.end() - 1) = stock_tick.symbol Monday, July 11, 2011
- 8. Blending CQL with Java DEBS2011 CQL char conversion to Java String CREATE VIEW filtered_news(message, matcher) AS SELECT message, Pattern.compile("[.,; ][A-Z][A-Z][A-Z][.,; ]").matcher(message) as matcher FROM news [RANGE 1 HOUR] Text SELECT location, item_description, message Method invocation FROM filtered_news, stock_tick[RANGE 1 HOUR] WHERE matcher.find() = true AND Static method filtered_news.message.substring(matcher.start() + 1, invocation matcher.end() - 1) = stock_tick.symbol Monday, July 11, 2011
- 9. Blending CQL with Java DEBS2011 ‘matcher’ is a Java Object, and treated as CREATE VIEW filtered_news(message, matcher) AS a complex type by SELECT CQL message, Pattern.compile("[.,; ][A-Z][A-Z][A-Z][.,; ]").matcher(message) as matcher FROM news [RANGE 1 HOUR] Text SELECT Overload of CQL ‘+’ location, item_description, message operator to handle FROM filtered_news, stock_tick[RANGE 1 HOUR] Java Integer WHERE matcher.find() = true AND Overload of CQL filtered_news.message.substring(matcher.start() + 1, equality to handle matcher.end() - 1) = stock_tick.symbol Java String Monday, July 11, 2011
- 10. Architecture DEBS2011 Pattern.compile("[.,; ][A-Z][A-Z][A-Z][.,; ]").matcher Pattern.compile("[.,; ][A-Z][A-Z][A-Z][.,; ]").matcher(message) (message) • Is this symbol: • Text A stream attribute (e.g. message)? • A stream alias (e.g. FROM S1 as A) • A CQL function (e.g. SELECT myfunc()) • The l-value of complex type (e.g. myObj.myMethod()) • A constructor • A (static) method • A class name Monday, July 11, 2011
- 11. Architecture DEBS2011 Pattern.compile("[.,; ][A-Z][A-Z][A-Z][.,; ]").matcher Pattern.compile("[.,; ][A-Z][A-Z][A-Z][.,; ]").matcher(message) (message) • To be able to semantically compile symbols, we need to have type information.Text • However, CQL is not aware of Java, nor should it be. • There is a need of a generic type-system, and an extension model for providers to plug-in their own languages Monday, July 11, 2011
- 12. Architecture DEBS2011 CQL Parser Semantic Type Analyzer Registry Text Code Generator Java Type-System Code Executor Monday, July 11, 2011
- 13. Architecture DEBS2011 Pattern.compile("[.,; ][A-Z][A-Z][A-Z][.,; ]").matcher Pattern.compile("[.,; ][A-Z][A-Z][A-Z][.,; ]").matcher(message) (message) ComplexType “Pattern” Text Type fields Registry methods Java Type-System Monday, July 11, 2011
- 14. Architecture DEBS2011 ComplexType fields “Pattern” Type Registry methods Text Java Type-System CQL deals with a ‘complex type’ abstraction, and does not know of its ‘Java Language’ binding. Monday, July 11, 2011
- 15. Architecture DEBS2011 Pattern.compile("[.,; ][A-Z][A-Z][A-Z][.,; ]").matcher Pattern.compile("[.,; ][A-Z][A-Z][A-Z][.,; ]").matcher(message) (message) Pattern ComplexType fields Text Find a “compile” method and its return type methods Monday, July 11, 2011
- 16. Architecture DEBS2011 Pattern.compile("[.,; ][A-Z][A-Z][A-Z][.,; ]").matcher(message) Pattern Matcher ComplexType ComplexType Text fields fields Store ‘matcher’ as methods methods a stream/view attribute Monday, July 11, 2011
- 17. Architecture DEBS2011 Extensible Language stream 1 * attribute Type <<is-of>> ‘matcher’ is an attribute of the view of type ‘java.util.regex.Matcher’ ComplexType Text <<metadata binding>> Extension Java Language CREATE VIEW filtered_news(message, matcher) Java Class Monday, July 11, 2011
- 18. Extensibility DEBS2011 Extensible Language stream 1 * attribute Type <<is-of>> ComplexType Text <<metadata binding>> How do we know which Extension Java Language ‘language extension’ to use? How to provide new Java Class extensions? Monday, July 11, 2011
- 19. Scenario 2: DEBS2011 Spatial Integration • Targeted marketing for a mobile subscriber • CEP application checks if the location of the subscriber is within the distance of a registered shop Text Text Oracle Spatial Shop id: CHAR geometry: SDO_GEOMETRY Monday, July 11, 2011
- 20. Blending CQL with Spatial DEBS2011 CREATE VIEW CustomerLocation-Stream(point, custId) AS SELECT createPoint@spatial(lng, lat) as point, custId FROM Location-Stream point is a spatial type SELECT loc.custId, shop.id Text FROM CustomerLocation-Stream[NOW] AS loc, Shop as shop WHERE contain@spatial(shop.geometry, loc.point, 2.0d) ‘spatial’ link points to Oracle Spatial ‘extension’, where ‘contain’ function resides Monday, July 11, 2011
- 21. Architecture DEBS2011 CQL CQL link specifies language extension, which are plugged into the system as a CQL cartridge Text <<link>> CQL Cartridge Cartridge contain@spatial(shop.geometry, loc.point, 2.0d) Monday, July 11, 2011
- 22. Conclusion DEBS2011 • Blending of CQL with other languages allow for the creation of feature-rich CEP applications while still being highly descriptive • Example: • String manipulation using Java Text • Geo-fencing using Oracle Spatial • Generic frameworks allows for the dynamic plugin of extensions (cartridges) Monday, July 11, 2011