SlideShare a Scribd company logo

RuleML 2015: When Processes Rule Events

RuleML, profile picture
RuleML

The document outlines a lecture on big data, highlighting key concepts such as complex event processing, data volume, velocity, variety, and veracity. It discusses the importance of data science skills, the optimization of event pattern matching, and methods for predicting traveling times in scheduled transportation. The content integrates theoretical principles with empirical evaluations in big data applications.

Science
1 of 60
Downloaded 15 times
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules When Processes Rule Events Avigdor Gal Technion – Israel Institute of Technology
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Presentation Outline Big data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimizaion Process Mining with Schedules
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Big Data: is it a Storm in a Teacup?
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Big data is a game changer From Theory to Systems: empirical evaluation counts From Systems to Data: large scale empirical evaluation counts
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Who is a Data Scientist? The ability to take data – to be able to understand it, to process it, to extract value from it, to visualize it, to communicate it – that’s going to be a hugely important skill in the next decades. (Hal Varian, Google’s Chief Economist)
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Data Volume: No Longer the Size of a Teacup Volume Table: Big Data Cross Table Big data may be a single dataset with a lot of data
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Data Volume: No Longer the Size of a Teacup Table: Big Data Cross Table Big data may be a single dataset with a lot of data
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Data Velocity: Replacing a Teacup with a Tea Hose Volume Velocity Table: Big Data Cross Table Big data may be data that rapidly changes
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Data Velocity: Replacing a Teacup with a Tea Hose Table: Big Data Cross Table Big data may be data that rapidly changes
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Data Velocity: Replacing a Teacup with a Tea Hose Table: Big Data Cross Table Big data may be data that rapidly changes
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Data Velocity: Replacing a Teacup with a Tea Hose Table: Big Data Cross Table Big data may be data that rapidly changes
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Data Variety: When One Tea Type is Just not Enough Volume Velocity Variety Table: Big Data Cross Table Big data may be a small dataset with many different schemata
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Data Variety: When One Tea Type is Just not Enough Table: Big Data Cross Table Big data may be a small dataset with many different schemata
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Data Veracity: Is it Coffee or Black Tea with Milk? Volume Velocity Variety Veracity Table: Big Data Cross Table Big data may be data with varying levels of trustworthiness
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Data Veracity: Is it Coffee or Black Tea with Milk? Table: Big Data Cross Table Big data may be data with varying levels of trustworthiness
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Data Gathering: where and when to expect the fountain to burst Gathering Volume Velocity Variety Veracity Signal and Event Processing Table: Big Data Cross Table
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Data Gathering: where and when to expect the fountain to burst Table: Big Data Cross Table
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Data Management: Not your typical DBA anymore Gathering Managing Volume Velocity Variety Veracity Cloud Computing, NoSQL, NewSQL Table: Big Data Cross Table
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Data Analytics: When Data Analysis Explodes Multi-Dimensionally Gathering Managing Analyzing Volume Velocity Variety Veracity Data & Process Mining ML, IR, NLP Table: Big Data Cross Table
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Data Visualization: The Machine Offering to Mankind Gathering Managing Analyzing Visualizing Volume Velocity Variety Veracity User Experience Table: Big Data Cross Table
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Data Visualization: The Machine Offering to Mankind Table: Big Data Cross Table
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Events Processes Complex Event Processing Optimization Process Mining with Schedules Big Data Cross Table Gathering Managing Analyzing Visualizing Volume Ev Pro Velocity en ce Variety t ss Veracity s es Table: Big Data Cross Table
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Events Processes Complex Event Processing Optimization Process Mining with Schedules Event Processing Events An event e is an occurrence within a particular system or domain. It is something that has happened, or is contemplated as having happened in that domain. [Etzion and Niblett, 2010] Point-based semantics. An event type E ∈ E is a specification for a set of events that share the same semantic intent and structure. Complex Event Processing Systems: Amit [Adi and Etzion, 2004], SASE [Wu et al., 2006], Cayuga [Demers et al., 2007], CEDR [Barga et al., 2007], ESPER []. DEBS 2016: Oragne County, California
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Events Processes Complex Event Processing Optimization Process Mining with Schedules Event Processing Urban Traffic Management
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Events Processes Complex Event Processing Optimization Process Mining with Schedules Traffic Flow
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Events Processes Complex Event Processing Optimization Process Mining with Schedules Bus Log
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Events Processes Complex Event Processing Optimization Process Mining with Schedules Events and Big Data Volume: 23 Million records per month (∼ 4GB) Velocity: 770,000 new records per day (an event each 2-6 seconds) Variety: Homogeneous Veracity: GPS locations
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Events Processes Complex Event Processing Optimization Process Mining with Schedules Processes Processes Process models describe time dependencies among activities: Business processes Scheduled activities Used as a template for execution by a process engine. A process model can be modeled as a graph containing activity nodes and control nodes: Petri nets [Reisig, 1985] BPMN [bpm, 2011]
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Events Processes Complex Event Processing Optimization Process Mining with Schedules Process Models Bus Log Bus Model s d ω_2 ω_3 ω_i ω_{n-1}
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Events Processes Complex Event Processing Optimization Process Mining with Schedules Between Events and Processes Given processes, detect (complex) events Given events, discover processes
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules From Processes to CEP Optimisation of event pattern matching on three levels Approach based on domain knowledge Results taken from: M. Weidlich, H. Ziekow, A. Gal, J. Mendling, M. Weske - Optimising Event Pattern Matching using Business Process Models. IEEE Transactions on Knowledge and Data Engineering (TKDE), accepted for publication, 2015.
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules From Processes to CEP Thanks Matthias Weidlich for the slides
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Optimization by Transformation Sequentialization Rule
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Optimization by Plan Selection Sequentialization Rule
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Optimization by Early Termination Sequentialization Rule
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Performance Analysis Datasets publicly available process log that contains recorded execution sequences of a paper reviewing process.a The model denes 20 activities. The log comprises 3730 events that are related to 100 process instances. Each event is associated with a timestamp and a reference to an activity of the process model. Process models of a German insurance company. 1021 process models, ranging from 4 to 339 nodes. The average size of the process models is around 23 nodes. The log was simulated using annotations of the process models. a http://www.processmining.org/logs/start
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Performance Analysis
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Performance Analysis
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Complex Events Processing with Processes Gathering ... Volume Velocity Optimization Variety Optimisation in event processing networks Veracity Table: Big Data Cross Table
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Complex Events Processing with Processes ... Analysis Volume Mining of constraints Velocity Variety Veracity Probabilistic mining of constraints Table: Big Data Cross Table
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules From Events to Processes Online Traveling Time Prediction: when Processes Rule Events Using information on bus stops, the prediction of the journey traveling time T( ω1, . . . , ωn , tω1 ) is traced back to the sum of traveling times per segment: T( ω1, . . . , ωn , tω1 ) = T( ω1, ω2 , tω1 ) + . . . + T( ωn−1, ωn , tωn−1 ) where tωn−1 = tω1 + T( ω1, ωn−1 , tω1 ). s d Traveling Time = Drive Time + Delay Time + Stop Time ω_2 ω_3 ω_i ω_{n-1} (Thanks to Arik Senderovich for the slides)
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules From Events to Processes Online Traveling Time Prediction: when Processes Rule Events Using information on bus stops, the prediction of the journey traveling time T( ω1, . . . , ωn , tω1 ) is traced back to the sum of traveling times per segment: T( ω1, . . . , ωn , tω1 ) = T( ω1, ω2 , tω1 ) + . . . + T( ωn−1, ωn , tωn−1 ) where tωn−1 = tω1 + T( ω1, ωn−1 , tω1 ). s d Traveling Time = Drive Time + Delay Time + Stop Time ω_2 ω_3 ω_i ω_{n-1} (Thanks to Arik Senderovich for the slides)
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Prediction: The Snapshot Principle in Single-Station Queues The snapshot principle stems from a heavy-traffic approximation of a queueing system under limits of its parameters, as the workload converges to capacity. Station1 The principle states that the total time in the station (waiting+service) remains constant. In our context, bus that passes through a segment, e.g., ωi, ωi+1 ∈ S × S, will have the same traveling time as another bus that has just passed through that segment (not necessarily of the same type, line, etc.).
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Prediction: The Snapshot Principle in Single-Station Queues The snapshot principle stems from a heavy-traffic approximation of a queueing system under limits of its parameters, as the workload converges to capacity. Station1 The principle states that the total time in the station (waiting+service) remains constant. In our context, bus that passes through a segment, e.g., ωi, ωi+1 ∈ S × S, will have the same traveling time as another bus that has just passed through that segment (not necessarily of the same type, line, etc.).
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Prediction: The Snapshot Principle in Single-Station Queues The snapshot principle stems from a heavy-traffic approximation of a queueing system under limits of its parameters, as the workload converges to capacity. Station1 The principle states that the total time in the station (waiting+service) remains constant. In our context, bus that passes through a segment, e.g., ωi, ωi+1 ∈ S × S, will have the same traveling time as another bus that has just passed through that segment (not necessarily of the same type, line, etc.).
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules The Snapshot Principle in Single-Station Queues Based on the above, we define a single-segment snapshot predictor, Last-Bus-to-Travel-Segment (LBTS), denoted by θLBTS( ωi, ωi+1 , tω1 ). In real-life settings, applicability of the snapshot principle predictors should be tested ad-hoc. The snapshot principle was shown to be of an empirical value in previous research, where queueing techniques were applied to predict delays.
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules The Snapshot Principle in Single-Station Queues Based on the above, we define a single-segment snapshot predictor, Last-Bus-to-Travel-Segment (LBTS), denoted by θLBTS( ωi, ωi+1 , tω1 ). In real-life settings, applicability of the snapshot principle predictors should be tested ad-hoc. The snapshot principle was shown to be of an empirical value in previous research, where queueing techniques were applied to predict delays.
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Snapshot Principle in a Network In our case, the LBTS predictor needs to be lifted to a network setting. The snapshot principle holds for networks of queues, when the routing through this network is known in advance. In scheduled transportation such as buses this is the case as the order of stops (and segments) is predefined: Station1 Station2 Station3 Station5 Station6 Station4 Station7
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Snapshot Principle in a Network In our case, the LBTS predictor needs to be lifted to a network setting. The snapshot principle holds for networks of queues, when the routing through this network is known in advance. In scheduled transportation such as buses this is the case as the order of stops (and segments) is predefined: Station1 Station2 Station3 Station5 Station6 Station4 Station7
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Snapshot Principle in a Network In our case, the LBTS predictor needs to be lifted to a network setting. The snapshot principle holds for networks of queues, when the routing through this network is known in advance. In scheduled transportation such as buses this is the case as the order of stops (and segments) is predefined: Station1 Station2 Station3 Station5 Station6 Station4 Station7
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Snapshot Principle in a Network We define a multi-segment (network) snapshot predictor that we refer to as the Last-Bus-to-Travel-Network or θLBTN ( ω1, ..., ωn , tω1 ), given a sequence of stops (with ω1 being the start stop and ωn being the end stop). According to the snapshot principle in networks we get that: θLBTN ( ω1, ..., ωn , tω1 ) = n i=1 θLBTS( ωi, ωi+1 , tω1 ).
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Snapshot Principle in a Network We define a multi-segment (network) snapshot predictor that we refer to as the Last-Bus-to-Travel-Network or θLBTN ( ω1, ..., ωn , tω1 ), given a sequence of stops (with ω1 being the start stop and ωn being the end stop). According to the snapshot principle in networks we get that: θLBTN ( ω1, ..., ωn , tω1 ) = n i=1 θLBTS( ωi, ωi+1 , tω1 ).
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Snapshot Principle in a Network We define a multi-segment (network) snapshot predictor that we refer to as the Last-Bus-to-Travel-Network or θLBTN ( ω1, ..., ωn , tω1 ), given a sequence of stops (with ω1 being the start stop and ωn being the end stop). According to the snapshot principle in networks we get that: θLBTN ( ω1, ..., ωn , tω1 ) = n i=1 θLBTS( ωi, ωi+1 , tω1 ).
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Performance Analysis Data 8 days of bus data, between September and October of 2014. Each day: approximately 11500 traveled segments. First trip for each day: no associated last travel time. Prediction for line 046A. Data comes from all buses that share segments with line 046A.
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Performance Analysis 10 20 30 40 50 Index of the segment in the trip 100 101 102 103 104 105 106 107 Samplesquareestimationerror 40 50 60 70 80 90 100 110 RootMeanSquareError
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Process Mining with Schedules ... Analysis Volume Better prediction Velocity Segmentation Variety Veracity Table: Big Data Cross Table
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Process Mining with Schedules ... Management ... Volume Velocity Variety Veracity Event Cleaning Table: Big Data Cross Table
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Thank You Avigdor Gal Technion – Israel Institute of Technology
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules A. Adi and O. Etzion. Amit - the situation manager. The International Journal on Very Large Data Bases, 13(2):177–203, May 2004. Roger S. Barga, Jonathan Goldstein, Mohamed H. Ali, and Mingsheng Hong. Consistent streaming through time: A vision for event stream processing. In CIDR [DBL, 2007], pages 363–374. Business Process Model and Notation (BPMN) Version 2.0. Technical report, Object Management Group (OMG), January 2011. CIDR 2007, Third Biennial Conference on Innovative Data Systems Research, Asilomar, CA, USA, January 7-10, 2007, Online Proceedings. www.cidrdb.org, 2007. Alan J. Demers, Johannes Gehrke, Biswanath Panda, Mirek Riedewald, Varun Sharma, and Walker M. White. Cayuga: A general purpose event monitoring system. In CIDR [DBL, 2007], pages 412–422. Opher Etzion and Peter Niblett. Event Processing in Action. Manning Publications Company, 2010.
Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Wolfgang Reisig. Petri Nets: An Introduction, volume 4 of Monographs in Theoretical Computer Science. An EATCS Series. Springer, 1985. Eugene Wu, Yanlei Diao, and Shariq Rizvi. High-performance complex event processing over streams. In SIGMOD ’06: Proceedings of the 2006 ACM SIGMOD international conference on Management of data, pages 407–418, New York, NY, USA, 2006. ACM.

More Related Content

PPTX
Big Process for Big Data @ PNNL, May 2013
Ian Foster
 
PDF
Séminaire Big Data Alter Way - Elasticsearch - octobre 2014
ALTER WAY
 
PPTX
Are you ready for BIG DATA?
Putchong Uthayopas
 
PDF
Lessons from building a stream-first metadata platform | Shirshanka Das, Stealth
HostedbyConfluent
 
PPTX
Big Data, Baby Steps
William Yetman
 
PPTX
Владимир Слободянюк «DWH & BigData – architecture approaches»
Anna Shymchenko
 
PPTX
LendingClub RealTime BigData Platform with Oracle GoldenGate
Rajit Saha
 
PDF
Visualising and Linking Open Data from Multiple Sources
Data Driven Innovation
 
Big Process for Big Data @ PNNL, May 2013
Ian Foster
 
Séminaire Big Data Alter Way - Elasticsearch - octobre 2014
ALTER WAY
 
Are you ready for BIG DATA?
Putchong Uthayopas
 
Lessons from building a stream-first metadata platform | Shirshanka Das, Stealth
HostedbyConfluent
 
Big Data, Baby Steps
William Yetman
 
Владимир Слободянюк «DWH & BigData – architecture approaches»
Anna Shymchenko
 
LendingClub RealTime BigData Platform with Oracle GoldenGate
Rajit Saha
 
Visualising and Linking Open Data from Multiple Sources
Data Driven Innovation
 

What's hot (20)

PDF
Tutorial - Modern Real Time Streaming Architectures
Karthik Ramasamy
 
PPT
My other computer is a datacentre - 2012 edition
Steve Loughran
 
PDF
Project
Sujith C.P
 
PDF
Building a Graph-based Analytics Platform
Kenny Bastani
 
PDF
Introduction to Big Data
IMC Institute
 
PPTX
Stephen Dillon - Fast Data Presentation Sept 02
Stephen Dillon
 
PDF
Big and fast a quest for relevant and real-time analytics
Natalino Busa
 
PDF
WorDS of Data Science in the Presence of Heterogenous Computing Architectures
Ilkay Altintas, Ph.D.
 
PPTX
American Century (Revolution Analytics Customer Day)
Revolution Analytics
 
PDF
Are we reaching a Data Science Singularity? How Cognitive Computing is emergi...
Big Data Spain
 
PPT
Big Graph Analytics on Neo4j with Apache Spark
Kenny Bastani
 
PDF
A Maturing Role of Workflows in the Presence of Heterogenous Computing Archit...
Ilkay Altintas, Ph.D.
 
PPTX
"Demystifying Big Data by AIBDP.org
AIBDP
 
PPSX
De-Mystifying Big Data
Prasad Mavuduri
 
PDF
Big Data Day LA 2016/ Big Data Track - Twitter Heron @ Scale - Karthik Ramasa...
Data Con LA
 
PPTX
Big Data Lessons from the Cloud
MapR Technologies
 
PDF
The architecture of data analytics PaaS on AWS
Treasure Data, Inc.
 
PPTX
Geolocation analysis using HiveQL
Priyanka Kale
 
PPTX
Interactive Realtime Dashboards on Data Streams using Kafka, Druid and Superset
Hortonworks
 
PPT
Introduction to Large Scale Data Analysis with WSO2 Analytics Platform
Srinath Perera
 
Tutorial - Modern Real Time Streaming Architectures
Karthik Ramasamy
 
My other computer is a datacentre - 2012 edition
Steve Loughran
 
Project
Sujith C.P
 
Building a Graph-based Analytics Platform
Kenny Bastani
 
Introduction to Big Data
IMC Institute
 
Stephen Dillon - Fast Data Presentation Sept 02
Stephen Dillon
 
Big and fast a quest for relevant and real-time analytics
Natalino Busa
 
WorDS of Data Science in the Presence of Heterogenous Computing Architectures
Ilkay Altintas, Ph.D.
 
American Century (Revolution Analytics Customer Day)
Revolution Analytics
 
Are we reaching a Data Science Singularity? How Cognitive Computing is emergi...
Big Data Spain
 
Big Graph Analytics on Neo4j with Apache Spark
Kenny Bastani
 
A Maturing Role of Workflows in the Presence of Heterogenous Computing Archit...
Ilkay Altintas, Ph.D.
 
"Demystifying Big Data by AIBDP.org
AIBDP
 
De-Mystifying Big Data
Prasad Mavuduri
 
Big Data Day LA 2016/ Big Data Track - Twitter Heron @ Scale - Karthik Ramasa...
Data Con LA
 
Big Data Lessons from the Cloud
MapR Technologies
 
The architecture of data analytics PaaS on AWS
Treasure Data, Inc.
 
Geolocation analysis using HiveQL
Priyanka Kale
 
Interactive Realtime Dashboards on Data Streams using Kafka, Druid and Superset
Hortonworks
 
Introduction to Large Scale Data Analysis with WSO2 Analytics Platform
Srinath Perera
 
Ad

Viewers also liked (17)

PDF
RuleML 2015 Constraint Handling Rules - What Else?
RuleML
 
PDF
RuleML 2015: Semantics of Notation3 Logic: A Solution for Implicit Quantifica...
RuleML
 
PDF
RuleML2015: How to combine event stream reasoning with transactions for the...
RuleML
 
DOCX
History of Induction and Recursion B
Damien MacFarland
 
PPT
Induction and Decision Tree Learning (Part 1)
butest
 
PDF
RuleML 2015: Ontology Reasoning using Rules in an eHealth Context
RuleML
 
PDF
11X1 T14 08 mathematical induction 1 (2011)
Nigel Simmons
 
PDF
Challenge@RuleML2015 Transformation and aggregation preprocessing for top-k r...
RuleML
 
PDF
Challenge@rule ml2015 rule based recommender systems for the Web of Data
RuleML
 
PDF
Harold Boley: RuleML/Grailog: The Rule Metalogic Visualized with Generalized ...
PhiloWeb
 
PDF
Iteration, induction, and recursion
Mohammed Hussein
 
PDF
Math induction principle (slides)
IIUM
 
PPSX
Mathematical induction
sonia -
 
PPT
Mathematical induction
Sman Abbasi
 
PPTX
5.4 mathematical induction
math260
 
PPTX
Principle of mathematical induction
Kriti Varshney
 
PPT
mathematical induction
ankush_kumar
 
RuleML 2015 Constraint Handling Rules - What Else?
RuleML
 
RuleML 2015: Semantics of Notation3 Logic: A Solution for Implicit Quantifica...
RuleML
 
RuleML2015: How to combine event stream reasoning with transactions for the...
RuleML
 
History of Induction and Recursion B
Damien MacFarland
 
Induction and Decision Tree Learning (Part 1)
butest
 
RuleML 2015: Ontology Reasoning using Rules in an eHealth Context
RuleML
 
11X1 T14 08 mathematical induction 1 (2011)
Nigel Simmons
 
Challenge@RuleML2015 Transformation and aggregation preprocessing for top-k r...
RuleML
 
Challenge@rule ml2015 rule based recommender systems for the Web of Data
RuleML
 
Harold Boley: RuleML/Grailog: The Rule Metalogic Visualized with Generalized ...
PhiloWeb
 
Iteration, induction, and recursion
Mohammed Hussein
 
Math induction principle (slides)
IIUM
 
Mathematical induction
sonia -
 
Mathematical induction
Sman Abbasi
 
5.4 mathematical induction
math260
 
Principle of mathematical induction
Kriti Varshney
 
mathematical induction
ankush_kumar
 
Ad

Similar to RuleML 2015: When Processes Rule Events (20)

PPTX
Rpi talk foster september 2011
Ian Foster
 
PPTX
Big Data By Vijay Bhaskar Semwal
IIIT Allahabad
 
PPTX
TSE_Pres12.pptx
ssuseracaaae2
 
PDF
L21 Big Data and Analytics
Ólafur Andri Ragnarsson
 
PPTX
So Long Computer Overlords
Ian Foster
 
PPTX
Big Data Session 1.pptx
ElsonPaul2
 
PDF
L18 Big Data and Analytics
Ólafur Andri Ragnarsson
 
PPTX
Data mining with big data
Sandip Tipayle Patil
 
PPTX
INN530 - Assignment 2, Big data and cloud computing for management
Simen Smaaberg
 
PPTX
Innovation med big data – chr. hansens erfaringer
Microsoft
 
PPTX
2025 USGS Science Data Management Briefing
N/A
 
PDF
International Journal of Engineering Research and Development (IJERD)
IJERD Editor
 
PDF
1Lecture_01_Introduction to Big Data.pdf
aihtesham
 
PDF
4Lecture_01_Introduction to Big Data.pdf
aihtesham
 
PDF
NPTEL BIG DATA FULL PPT BOOK WITH ASSIGNMENT SOLUTION RAJIV MISHRA IIT PATNA...
SayantanRoy14
 
PPTX
SQL Server 2008 R2 StreamInsight
Eduardo Castro
 
PDF
IICT-Big Data.pdf slideshow information to communication
juicepiladomusambika
 
PDF
IICT-Big Data.pdf slideshow Information to communication technology
juicepiladomusambika
 
PDF
Decision Ready Data: Power Your Analytics with Great Data
DLT Solutions
 
PDF
Oracle Stream Analytics - Simplifying Stream Processing
Guido Schmutz
 
Rpi talk foster september 2011
Ian Foster
 
Big Data By Vijay Bhaskar Semwal
IIIT Allahabad
 
TSE_Pres12.pptx
ssuseracaaae2
 
L21 Big Data and Analytics
Ólafur Andri Ragnarsson
 
So Long Computer Overlords
Ian Foster
 
Big Data Session 1.pptx
ElsonPaul2
 
L18 Big Data and Analytics
Ólafur Andri Ragnarsson
 
Data mining with big data
Sandip Tipayle Patil
 
INN530 - Assignment 2, Big data and cloud computing for management
Simen Smaaberg
 
Innovation med big data – chr. hansens erfaringer
Microsoft
 
2025 USGS Science Data Management Briefing
N/A
 
International Journal of Engineering Research and Development (IJERD)
IJERD Editor
 
1Lecture_01_Introduction to Big Data.pdf
aihtesham
 
4Lecture_01_Introduction to Big Data.pdf
aihtesham
 
NPTEL BIG DATA FULL PPT BOOK WITH ASSIGNMENT SOLUTION RAJIV MISHRA IIT PATNA...
SayantanRoy14
 
SQL Server 2008 R2 StreamInsight
Eduardo Castro
 
IICT-Big Data.pdf slideshow information to communication
juicepiladomusambika
 
IICT-Big Data.pdf slideshow Information to communication technology
juicepiladomusambika
 
Decision Ready Data: Power Your Analytics with Great Data
DLT Solutions
 
Oracle Stream Analytics - Simplifying Stream Processing
Guido Schmutz
 

More from RuleML (20)

PDF
Aggregates in Recursion: Issues and Solutions
RuleML
 
PDF
A software agent controlling 2 robot arms in co-operating concurrent tasks
RuleML
 
PDF
Port Clearance Rules in PSOA RuleML: From Controlled-English Regulation to Ob...
RuleML
 
PDF
Challenge@RuleML2015 Developing Situation-Aware Applications for Disaster Man...
RuleML
 
PDF
Rule Generalization Strategies in Incremental Learning of Disjunctive Concepts
RuleML
 
PDF
RuleML2015 The Herbrand Manifesto - Thinking Inside the Box
RuleML
 
PDF
RuleML2015 PSOA RuleML: Integrated Object-Relational Data and Rules
RuleML
 
PDF
Industry@RuleML2015: Norwegian State of Estate A Reporting Service for the St...
RuleML
 
PDF
A Service for Improving the Assignments of Common Agriculture Policy Funds to...
RuleML
 
PDF
Datalog+-Track Introduction & Reasoning on UML Class Diagrams via Datalog+-
RuleML
 
PDF
RuleML2015: Binary Frontier-guarded ASP with Function Symbols
RuleML
 
PDF
RuleML2015: API4KP Metamodel: A Meta-API for Heterogeneous Knowledge Platforms
RuleML
 
PDF
RuleML2015: Rule-Based Exploration of Structured Data in the Browser
RuleML
 
PDF
RuleML2015: Ontology-Based Multidimensional Contexts with Applications to Qua...
RuleML
 
PDF
RuleML2015: Compact representation of conditional probability for rule-based...
RuleML
 
PDF
RuleML2015: Learning Characteristic Rules in Geographic Information Systems
RuleML
 
PDF
RuleML2015: Using Substitutive Itemset Mining Framework for Finding Synonymou...
RuleML
 
PDF
RuleML2015: User Extensible System to Identify Problems in OWL Ontologies and...
RuleML
 
PDF
RuleML2015: Representing Flexible Role-Based Access Control Policies Using Ob...
RuleML
 
PDF
RuleML2015: Rule Generalization Strategies in Incremental Learning of Disjunc...
RuleML
 
Aggregates in Recursion: Issues and Solutions
RuleML
 
A software agent controlling 2 robot arms in co-operating concurrent tasks
RuleML
 
Port Clearance Rules in PSOA RuleML: From Controlled-English Regulation to Ob...
RuleML
 
Challenge@RuleML2015 Developing Situation-Aware Applications for Disaster Man...
RuleML
 
Rule Generalization Strategies in Incremental Learning of Disjunctive Concepts
RuleML
 
RuleML2015 The Herbrand Manifesto - Thinking Inside the Box
RuleML
 
RuleML2015 PSOA RuleML: Integrated Object-Relational Data and Rules
RuleML
 
Industry@RuleML2015: Norwegian State of Estate A Reporting Service for the St...
RuleML
 
A Service for Improving the Assignments of Common Agriculture Policy Funds to...
RuleML
 
Datalog+-Track Introduction & Reasoning on UML Class Diagrams via Datalog+-
RuleML
 
RuleML2015: Binary Frontier-guarded ASP with Function Symbols
RuleML
 
RuleML2015: API4KP Metamodel: A Meta-API for Heterogeneous Knowledge Platforms
RuleML
 
RuleML2015: Rule-Based Exploration of Structured Data in the Browser
RuleML
 
RuleML2015: Ontology-Based Multidimensional Contexts with Applications to Qua...
RuleML
 
RuleML2015: Compact representation of conditional probability for rule-based...
RuleML
 
RuleML2015: Learning Characteristic Rules in Geographic Information Systems
RuleML
 
RuleML2015: Using Substitutive Itemset Mining Framework for Finding Synonymou...
RuleML
 
RuleML2015: User Extensible System to Identify Problems in OWL Ontologies and...
RuleML
 
RuleML2015: Representing Flexible Role-Based Access Control Policies Using Ob...
RuleML
 
RuleML2015: Rule Generalization Strategies in Incremental Learning of Disjunc...
RuleML
 

Recently uploaded (20)

PDF
Sciences of Europe No 170 (2025)
Sciences of Europe
 
PPTX
EPIDURAL ANESTHESIA ANATOMY AND PHYSIOLOGY.pptx
PrabakaranMP1
 
PPTX
microscope-Lecturecjchchchchcuvuvhc.pptx
jaylaurencenobleza
 
PDF
CAPERS-LRD-z9:AGas-enshroudedLittleRedDotHostingaBroad-lineActive GalacticNuc...
Sérgio Sacani
 
PPTX
Taita Taveta Laboratory Technician Workshop Presentation.pptx
CALVINCEOWINO
 
PPTX
Classification Systems_TAXONOMY_SCIENCE8.pptx
glofernignacio
 
PDF
The scientific heritage No 166 (166) (2025)
The scientific heritage
 
PPT
POSITIONING IN OPERATION THEATRE ROOM.ppt
PrabakaranMP1
 
PDF
HPLC-PPT.docx high performance liquid chromatography
darshanambiga1633
 
PPTX
INTRODUCTION TO EVS | Concept of sustainability
manumaddy15198
 
DOCX
Viruses (History, structure and composition, classification, Bacteriophage Re...
Educator
 
PPT
protein biochemistry.ppt for university classes
bayewodajo27
 
PPTX
Derivatives of integument scales, beaks, horns,.pptx
Dr Showkat Ahmad Wani
 
PPTX
G5Q1W8 PPT SCIENCE.pptx 2025-2026 GRADE 5
Rodel Gordzkie
 
PPTX
BIOMOLECULES PPT........................
vachieagrawal1221
 
PPTX
TOTAL hIP ARTHROPLASTY Presentation.pptx
hararghemssc
 
PPTX
famous lake in india and its disturibution and importance
judesebina
 
PDF
Phytochemical Investigation of Miliusa longipes.pdf
IrfanShahirSharafi
 
PPTX
2Systematics of Living Organisms t-.pptx
RachanaT6
 
PDF
Biophysics 2.pdffffffffffffffffffffffffff
MiraMusleh
 
Sciences of Europe No 170 (2025)
Sciences of Europe
 
EPIDURAL ANESTHESIA ANATOMY AND PHYSIOLOGY.pptx
PrabakaranMP1
 
microscope-Lecturecjchchchchcuvuvhc.pptx
jaylaurencenobleza
 
CAPERS-LRD-z9:AGas-enshroudedLittleRedDotHostingaBroad-lineActive GalacticNuc...
Sérgio Sacani
 
Taita Taveta Laboratory Technician Workshop Presentation.pptx
CALVINCEOWINO
 
Classification Systems_TAXONOMY_SCIENCE8.pptx
glofernignacio
 
The scientific heritage No 166 (166) (2025)
The scientific heritage
 
POSITIONING IN OPERATION THEATRE ROOM.ppt
PrabakaranMP1
 
HPLC-PPT.docx high performance liquid chromatography
darshanambiga1633
 
INTRODUCTION TO EVS | Concept of sustainability
manumaddy15198
 
Viruses (History, structure and composition, classification, Bacteriophage Re...
Educator
 
protein biochemistry.ppt for university classes
bayewodajo27
 
Derivatives of integument scales, beaks, horns,.pptx
Dr Showkat Ahmad Wani
 
G5Q1W8 PPT SCIENCE.pptx 2025-2026 GRADE 5
Rodel Gordzkie
 
BIOMOLECULES PPT........................
vachieagrawal1221
 
TOTAL hIP ARTHROPLASTY Presentation.pptx
hararghemssc
 
famous lake in india and its disturibution and importance
judesebina
 
Phytochemical Investigation of Miliusa longipes.pdf
IrfanShahirSharafi
 
2Systematics of Living Organisms t-.pptx
RachanaT6
 
Biophysics 2.pdffffffffffffffffffffffffff
MiraMusleh
 

RuleML 2015: When Processes Rule Events

  • 1. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules When Processes Rule Events Avigdor Gal Technion – Israel Institute of Technology
  • 2. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Presentation Outline Big data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimizaion Process Mining with Schedules
  • 3. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Big Data: is it a Storm in a Teacup?
  • 4. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Big data is a game changer From Theory to Systems: empirical evaluation counts From Systems to Data: large scale empirical evaluation counts
  • 5. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Who is a Data Scientist? The ability to take data – to be able to understand it, to process it, to extract value from it, to visualize it, to communicate it – that’s going to be a hugely important skill in the next decades. (Hal Varian, Google’s Chief Economist)
  • 6. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Data Volume: No Longer the Size of a Teacup Volume Table: Big Data Cross Table Big data may be a single dataset with a lot of data
  • 7. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Data Volume: No Longer the Size of a Teacup Table: Big Data Cross Table Big data may be a single dataset with a lot of data
  • 8. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Data Velocity: Replacing a Teacup with a Tea Hose Volume Velocity Table: Big Data Cross Table Big data may be data that rapidly changes
  • 9. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Data Velocity: Replacing a Teacup with a Tea Hose Table: Big Data Cross Table Big data may be data that rapidly changes
  • 10. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Data Velocity: Replacing a Teacup with a Tea Hose Table: Big Data Cross Table Big data may be data that rapidly changes
  • 11. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Data Velocity: Replacing a Teacup with a Tea Hose Table: Big Data Cross Table Big data may be data that rapidly changes
  • 12. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Data Variety: When One Tea Type is Just not Enough Volume Velocity Variety Table: Big Data Cross Table Big data may be a small dataset with many different schemata
  • 13. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Data Variety: When One Tea Type is Just not Enough Table: Big Data Cross Table Big data may be a small dataset with many different schemata
  • 14. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Data Veracity: Is it Coffee or Black Tea with Milk? Volume Velocity Variety Veracity Table: Big Data Cross Table Big data may be data with varying levels of trustworthiness
  • 15. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Data Veracity: Is it Coffee or Black Tea with Milk? Table: Big Data Cross Table Big data may be data with varying levels of trustworthiness
  • 16. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Data Gathering: where and when to expect the fountain to burst Gathering Volume Velocity Variety Veracity Signal and Event Processing Table: Big Data Cross Table
  • 17. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Data Gathering: where and when to expect the fountain to burst Table: Big Data Cross Table
  • 18. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Data Management: Not your typical DBA anymore Gathering Managing Volume Velocity Variety Veracity Cloud Computing, NoSQL, NewSQL Table: Big Data Cross Table
  • 19. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Data Analytics: When Data Analysis Explodes Multi-Dimensionally Gathering Managing Analyzing Volume Velocity Variety Veracity Data & Process Mining ML, IR, NLP Table: Big Data Cross Table
  • 20. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Data Visualization: The Machine Offering to Mankind Gathering Managing Analyzing Visualizing Volume Velocity Variety Veracity User Experience Table: Big Data Cross Table
  • 21. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Data Visualization: The Machine Offering to Mankind Table: Big Data Cross Table
  • 22. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Events Processes Complex Event Processing Optimization Process Mining with Schedules Big Data Cross Table Gathering Managing Analyzing Visualizing Volume Ev Pro Velocity en ce Variety t ss Veracity s es Table: Big Data Cross Table
  • 23. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Events Processes Complex Event Processing Optimization Process Mining with Schedules Event Processing Events An event e is an occurrence within a particular system or domain. It is something that has happened, or is contemplated as having happened in that domain. [Etzion and Niblett, 2010] Point-based semantics. An event type E ∈ E is a specification for a set of events that share the same semantic intent and structure. Complex Event Processing Systems: Amit [Adi and Etzion, 2004], SASE [Wu et al., 2006], Cayuga [Demers et al., 2007], CEDR [Barga et al., 2007], ESPER []. DEBS 2016: Oragne County, California
  • 24. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Events Processes Complex Event Processing Optimization Process Mining with Schedules Event Processing Urban Traffic Management
  • 25. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Events Processes Complex Event Processing Optimization Process Mining with Schedules Traffic Flow
  • 26. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Events Processes Complex Event Processing Optimization Process Mining with Schedules Bus Log
  • 27. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Events Processes Complex Event Processing Optimization Process Mining with Schedules Events and Big Data Volume: 23 Million records per month (∼ 4GB) Velocity: 770,000 new records per day (an event each 2-6 seconds) Variety: Homogeneous Veracity: GPS locations
  • 28. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Events Processes Complex Event Processing Optimization Process Mining with Schedules Processes Processes Process models describe time dependencies among activities: Business processes Scheduled activities Used as a template for execution by a process engine. A process model can be modeled as a graph containing activity nodes and control nodes: Petri nets [Reisig, 1985] BPMN [bpm, 2011]
  • 29. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Events Processes Complex Event Processing Optimization Process Mining with Schedules Process Models Bus Log Bus Model s d ω_2 ω_3 ω_i ω_{n-1}
  • 30. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Events Processes Complex Event Processing Optimization Process Mining with Schedules Between Events and Processes Given processes, detect (complex) events Given events, discover processes
  • 31. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules From Processes to CEP Optimisation of event pattern matching on three levels Approach based on domain knowledge Results taken from: M. Weidlich, H. Ziekow, A. Gal, J. Mendling, M. Weske - Optimising Event Pattern Matching using Business Process Models. IEEE Transactions on Knowledge and Data Engineering (TKDE), accepted for publication, 2015.
  • 32. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules From Processes to CEP Thanks Matthias Weidlich for the slides
  • 33. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Optimization by Transformation Sequentialization Rule
  • 34. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Optimization by Plan Selection Sequentialization Rule
  • 35. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Optimization by Early Termination Sequentialization Rule
  • 36. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Performance Analysis Datasets publicly available process log that contains recorded execution sequences of a paper reviewing process.a The model denes 20 activities. The log comprises 3730 events that are related to 100 process instances. Each event is associated with a timestamp and a reference to an activity of the process model. Process models of a German insurance company. 1021 process models, ranging from 4 to 339 nodes. The average size of the process models is around 23 nodes. The log was simulated using annotations of the process models. a http://www.processmining.org/logs/start
  • 37. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Performance Analysis
  • 38. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Performance Analysis
  • 39. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Complex Events Processing with Processes Gathering ... Volume Velocity Optimization Variety Optimisation in event processing networks Veracity Table: Big Data Cross Table
  • 40. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Complex Events Processing with Processes ... Analysis Volume Mining of constraints Velocity Variety Veracity Probabilistic mining of constraints Table: Big Data Cross Table
  • 41. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules From Events to Processes Online Traveling Time Prediction: when Processes Rule Events Using information on bus stops, the prediction of the journey traveling time T( ω1, . . . , ωn , tω1 ) is traced back to the sum of traveling times per segment: T( ω1, . . . , ωn , tω1 ) = T( ω1, ω2 , tω1 ) + . . . + T( ωn−1, ωn , tωn−1 ) where tωn−1 = tω1 + T( ω1, ωn−1 , tω1 ). s d Traveling Time = Drive Time + Delay Time + Stop Time ω_2 ω_3 ω_i ω_{n-1} (Thanks to Arik Senderovich for the slides)
  • 42. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules From Events to Processes Online Traveling Time Prediction: when Processes Rule Events Using information on bus stops, the prediction of the journey traveling time T( ω1, . . . , ωn , tω1 ) is traced back to the sum of traveling times per segment: T( ω1, . . . , ωn , tω1 ) = T( ω1, ω2 , tω1 ) + . . . + T( ωn−1, ωn , tωn−1 ) where tωn−1 = tω1 + T( ω1, ωn−1 , tω1 ). s d Traveling Time = Drive Time + Delay Time + Stop Time ω_2 ω_3 ω_i ω_{n-1} (Thanks to Arik Senderovich for the slides)
  • 43. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Prediction: The Snapshot Principle in Single-Station Queues The snapshot principle stems from a heavy-traffic approximation of a queueing system under limits of its parameters, as the workload converges to capacity. Station1 The principle states that the total time in the station (waiting+service) remains constant. In our context, bus that passes through a segment, e.g., ωi, ωi+1 ∈ S × S, will have the same traveling time as another bus that has just passed through that segment (not necessarily of the same type, line, etc.).
  • 44. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Prediction: The Snapshot Principle in Single-Station Queues The snapshot principle stems from a heavy-traffic approximation of a queueing system under limits of its parameters, as the workload converges to capacity. Station1 The principle states that the total time in the station (waiting+service) remains constant. In our context, bus that passes through a segment, e.g., ωi, ωi+1 ∈ S × S, will have the same traveling time as another bus that has just passed through that segment (not necessarily of the same type, line, etc.).
  • 45. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Prediction: The Snapshot Principle in Single-Station Queues The snapshot principle stems from a heavy-traffic approximation of a queueing system under limits of its parameters, as the workload converges to capacity. Station1 The principle states that the total time in the station (waiting+service) remains constant. In our context, bus that passes through a segment, e.g., ωi, ωi+1 ∈ S × S, will have the same traveling time as another bus that has just passed through that segment (not necessarily of the same type, line, etc.).
  • 46. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules The Snapshot Principle in Single-Station Queues Based on the above, we define a single-segment snapshot predictor, Last-Bus-to-Travel-Segment (LBTS), denoted by θLBTS( ωi, ωi+1 , tω1 ). In real-life settings, applicability of the snapshot principle predictors should be tested ad-hoc. The snapshot principle was shown to be of an empirical value in previous research, where queueing techniques were applied to predict delays.
  • 47. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules The Snapshot Principle in Single-Station Queues Based on the above, we define a single-segment snapshot predictor, Last-Bus-to-Travel-Segment (LBTS), denoted by θLBTS( ωi, ωi+1 , tω1 ). In real-life settings, applicability of the snapshot principle predictors should be tested ad-hoc. The snapshot principle was shown to be of an empirical value in previous research, where queueing techniques were applied to predict delays.
  • 48. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Snapshot Principle in a Network In our case, the LBTS predictor needs to be lifted to a network setting. The snapshot principle holds for networks of queues, when the routing through this network is known in advance. In scheduled transportation such as buses this is the case as the order of stops (and segments) is predefined: Station1 Station2 Station3 Station5 Station6 Station4 Station7
  • 49. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Snapshot Principle in a Network In our case, the LBTS predictor needs to be lifted to a network setting. The snapshot principle holds for networks of queues, when the routing through this network is known in advance. In scheduled transportation such as buses this is the case as the order of stops (and segments) is predefined: Station1 Station2 Station3 Station5 Station6 Station4 Station7
  • 50. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Snapshot Principle in a Network In our case, the LBTS predictor needs to be lifted to a network setting. The snapshot principle holds for networks of queues, when the routing through this network is known in advance. In scheduled transportation such as buses this is the case as the order of stops (and segments) is predefined: Station1 Station2 Station3 Station5 Station6 Station4 Station7
  • 51. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Snapshot Principle in a Network We define a multi-segment (network) snapshot predictor that we refer to as the Last-Bus-to-Travel-Network or θLBTN ( ω1, ..., ωn , tω1 ), given a sequence of stops (with ω1 being the start stop and ωn being the end stop). According to the snapshot principle in networks we get that: θLBTN ( ω1, ..., ωn , tω1 ) = n i=1 θLBTS( ωi, ωi+1 , tω1 ).
  • 52. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Snapshot Principle in a Network We define a multi-segment (network) snapshot predictor that we refer to as the Last-Bus-to-Travel-Network or θLBTN ( ω1, ..., ωn , tω1 ), given a sequence of stops (with ω1 being the start stop and ωn being the end stop). According to the snapshot principle in networks we get that: θLBTN ( ω1, ..., ωn , tω1 ) = n i=1 θLBTS( ωi, ωi+1 , tω1 ).
  • 53. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Snapshot Principle in a Network We define a multi-segment (network) snapshot predictor that we refer to as the Last-Bus-to-Travel-Network or θLBTN ( ω1, ..., ωn , tω1 ), given a sequence of stops (with ω1 being the start stop and ωn being the end stop). According to the snapshot principle in networks we get that: θLBTN ( ω1, ..., ωn , tω1 ) = n i=1 θLBTS( ωi, ωi+1 , tω1 ).
  • 54. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Performance Analysis Data 8 days of bus data, between September and October of 2014. Each day: approximately 11500 traveled segments. First trip for each day: no associated last travel time. Prediction for line 046A. Data comes from all buses that share segments with line 046A.
  • 55. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Performance Analysis 10 20 30 40 50 Index of the segment in the trip 100 101 102 103 104 105 106 107 Samplesquareestimationerror 40 50 60 70 80 90 100 110 RootMeanSquareError
  • 56. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Process Mining with Schedules ... Analysis Volume Better prediction Velocity Segmentation Variety Veracity Table: Big Data Cross Table
  • 57. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Process Mining with Schedules ... Management ... Volume Velocity Variety Veracity Event Cleaning Table: Big Data Cross Table
  • 58. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Thank You Avigdor Gal Technion – Israel Institute of Technology
  • 59. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules A. Adi and O. Etzion. Amit - the situation manager. The International Journal on Very Large Data Bases, 13(2):177–203, May 2004. Roger S. Barga, Jonathan Goldstein, Mohamed H. Ali, and Mingsheng Hong. Consistent streaming through time: A vision for event stream processing. In CIDR [DBL, 2007], pages 363–374. Business Process Model and Notation (BPMN) Version 2.0. Technical report, Object Management Group (OMG), January 2011. CIDR 2007, Third Biennial Conference on Innovative Data Systems Research, Asilomar, CA, USA, January 7-10, 2007, Online Proceedings. www.cidrdb.org, 2007. Alan J. Demers, Johannes Gehrke, Biswanath Panda, Mirek Riedewald, Varun Sharma, and Walker M. White. Cayuga: A general purpose event monitoring system. In CIDR [DBL, 2007], pages 412–422. Opher Etzion and Peter Niblett. Event Processing in Action. Manning Publications Company, 2010.
  • 60. Lecture Outline Big Data: the New Playground Events, Processes, and Anything in Between Complex Event Processing Optimization Process Mining with Schedules Wolfgang Reisig. Petri Nets: An Introduction, volume 4 of Monographs in Theoretical Computer Science. An EATCS Series. Springer, 1985. Eugene Wu, Yanlei Diao, and Shariq Rizvi. High-performance complex event processing over streams. In SIGMOD ’06: Proceedings of the 2006 ACM SIGMOD international conference on Management of data, pages 407–418, New York, NY, USA, 2006. ACM.