WSO2 Complex Event Processor - Product Overview
1 like583 views
The document discusses the implementation of complex event processing (CEP) systems by various organizations, emphasizing their real-time data analysis capabilities to enhance customer experiences and automate decision-making in IoT platforms. Key examples include Experian's use of CEP for customer behavior analysis, Eurecat's application in retail with iBeacons, and Pacific Controls' Galaxy 2021 platform for building management. The document also highlights technical features such as event filtering, pattern detection, and the ability to create custom operators and extensions for specific business needs.
![’
50
define stream StockStream
(symbol string, volume int, price double);
@name(Filter Query’)
@dist(parallel= ‘3')
from StockStream[price > 75]
select *
insert into HighPriceStockStream ;
@name(‘Window Query’)
@dist(parallel= ‘2')
partition with (symbol of HighPriceStockStream)
begin
from HighPriceStockStream#window.time(10 min)
select symbol, sum(volume) as sumVolume
insert into ResultStockStream ;
end;](https://image.slidesharecdn.com/wso2-cep-410-productoverview-160826120716/85/WSO2-Complex-Event-Processor-Product-Overview-50-320.jpg)
WSO2 Complex Event Processor - Product Overview
- 4. 4
- 5. 5
- 6. 6
- 7. 7
- 8. ™ …
- 12. 12 Experian delivers a digital marketing platform, where CEP plays a key role to analyze in real-time customers behavior and offer targeted promotions. CEP was chosen after careful analysis, primarily for its openness, its open source nature, the fact support is driven by engineers and the availability of a complete middleware, integrated with CEP, for additional use cases. Eurecat is the Catalunya innovation center (in Spain) - Using CEP to analyze data from iBeacons deployed within department stores to offer instant rebates to user or send them help if it detected that they seem “stuck” in the shop area. They chose WSO2 due to real time processing, the variety of IoT connectors available as well as the extensible framework and the rich configuration language. They also use WSO2 ESB in conjunction with WSO2 CEP. Pacific Controls is an innovative company delivering an IoT platform of platforms: Galaxy 2021. The platform allows to manage all kinds of devices within a building and take automated decisions such as moving an elevator or starting the air conditioning based on certain conditions. Within Galaxy2021, CEP is used for monitoring alarms and specific conditions.Pacific Controls also uses other products from the WSO2 platform, such as WSO2 ESB and Identity Server. A leading Airlines uses CEP to enhance customer experience by calculating the average time to reach their boarding gate (going through security, walking, etc.). They also want to track the time it takes to clean a plane, in order to better streamline the boarding process and notify both the air line and customers about potential delays. They evaluated WSO2 CEP first as they were already using our platform and decided to use it as it addressed all their requirements.
- 15. 15
- 16. •Core Event Processing Engine Event Processors •Publishes events to external systems Event Publisher
- 21. 21
- 26. Transformation Filter Composition/A ggregation/Ana lytics Join Multiple Streams Detect Patterns Blacklisting Build Profiles
- 27. 27 Category Operators Event Sequencing e handle out of order events by using a variant of the K-Slack algorithm, which is a well-known solution to handling disorder in event streams, by buffering data until order can be guaranteed. Compensation for missed events is not supported in the current version, but is on the roadmap. Additionally, we can use filtering to reduce noisy events in a stream (based on Kalman filter) Enrichment Enrichment is done via two ways: event tables to access historical data from any JDBC data source, and custom extensions to connect to custom source of data, such as files. Business Logic Scripting can be used to add any business logic to any execution plan. JavaScript, Scala and R are supported out of the box. Additional, customers can easily invoke custom logic through their own operators. Transformation The filter operator can be used to filter streams on a certain set of conditions, which can be combined via and/or - Conditions can be expressed using mathematical operators, regular expressions, string manipulation and logical operators. Additional , queries allow to select information from input stream, project them to output stream or new stream, and replace certain elements
- 28. 28 Category Operators Time Windows Siddhi provides very strong support for time windows, a domain where an SQL-like query language bring much simplicity compared to a programing language. Several types of windows are supported, including sliding and tumbling (batch) windows, time windows starting from a point in time, or CRON-based time windows. Additionally, we support applying streaming processing to events based on the number of events ( length window), the unicity of events or the frequency of events. Aggregation/Correlation Using Join and Pattern operators, we can aggregate and correlate two or more streams of data. Join allows to join events based on condition, while pattern allows to correlate multiple events based on time, logical relationship or event counting. Pattern Matching We detect patterns based on temporal order (based on arrival order), logical relationship (based or the logical relationship of 2 events, or counting (to limit the number of events matching the pattern). The pattern may or may not allow events in between the events the condition. If no foreign event is allowed, the sequence operator must be used. Custom Developers can create their own function, operators , time windows and processing operators. The extensions are written in Java. Once implemented the operators can be used as any other out of the box operator or function. Libraries to support custom operators Developers use the current operators as reference to develop their own, this is one of the key advantages with open source distribution. We deliver dozens of extensions on GitHub which can be adapted by 3rd parties. At the implementation level, implementing an extension just involves extending a well-defined interface. Other operators We support more than 100 custom operators on top of the list above, including geographical operators, for location-based applications, time series, math, natural language processing, integration with machine learning models created in PMML or our own Machine Learning product.
- 29. 29
- 30. 30
- 31. 31
- 32. 32
- 33. 33
- 34. 34
- 35. 35
- 36. 36
- 37. 37
- 38. 38
- 39. 39
- 40. 40
- 41. 41
- 43. •
- 44. •
- 46. 46
- 47. 47
- 50. ’ 50 define stream StockStream (symbol string, volume int, price double); @name(Filter Query’) @dist(parallel= ‘3') from StockStream[price > 75] select * insert into HighPriceStockStream ; @name(‘Window Query’) @dist(parallel= ‘2') partition with (symbol of HighPriceStockStream) begin from HighPriceStockStream#window.time(10 min) select symbol, sum(volume) as sumVolume insert into ResultStockStream ; end;
- 51. 5
- 52. Using Enron Email Data Set 52 Protocol Throughput (events/sec) Standalone 6555 distributed 1 node 3699 distributed: 2 nodes 6627 distributed: 3 nodes 9500 distributed: 4 nodes 11050 •Data Set: • 517,417 emails • mean body size of 1.8KB • largest size 1.92MB
- 54. 54
- 55. ’ 55
- 56. Using Enron Email Data Set 56 Protocol Throughput (events/sec) Thrift 82137 Binary 245000 Protocol Throughput (events/sec) Standalone 6555 distributed 1 node 3699 distributed: 2 nodes 6627 distributed: 3 nodes 9500 distributed: 4 nodes 11050 Simple Use Case •Data Set: • 517,417 emails • mean body size of 1.8 KB • largest size 1.92MB
- 58. 58 Guaranteed SLAs Pattern 1 (Hot/warm) Pattern 2 (Persistence) Pattern 3 (Distributed) Continuity in the availability of the system even in case of failure Guarantee that no event is lost. Stateful queries (for example involving a time window or a sequence), state is not lost despite failures. Not supported in current version (on roadmap) Event is treated only once Not fully supported Not supported by Storm
- 59. 59
- 60. 60
- 61. 61
- 62. 62
- 64. ’ 64
- 65. 65
- 66. 66
- 67. … 67
- 68. 68
- 73. CONTACT US ! Try WSO2 CEP 4.1.0