Complete and Interpretable Conformance Checking of Business Processes

Editor's Notes

• #3: detect deviations in the process execution with respect to the behavior stipulated by a normative model Model maintenance: The first situation suggests that the model needs to be extended to capture the unfitting behavior The second situation suggests that there are paths in the model that have become spurious overtime, meaning they are no longer used and need to be pruned if they are found to have lost relevance Iterative improvement of the process model. First, an initial model is discovered using any discovery algorithm, then conformance checking is measured and based on this result the behavior of the model is refined to better align it to that of the log, e.g. it is restricted to remove paths in the model never observed, in order to avoid over-generalization, and extended to add paths observed in the log but not captured in the model
• #5: Related work MEASURING UNFITTING BEHAVIOR Replay fitness and ICS extension: replay a trace on the Petri net, to detect missing tokens that need to be added to a place to replay the trace, and remaining tokens that remain in the Petri net once the trace has been fully replayed. ICS extension trades accuracy for performance Vanden Bourcke et al. proposes a SESE decomposition to improve performance and offer more localized feedback General limitation: error recovery is performed locally each time an error is detected, thus these methods may not identify the minimum number of differences (errors) to explain the unfitting behavior. This limitation is solved by: - trace alignment, which however still provides feedback at the level of individual traces, rather than behavioral relations between events. MEASURING ADDITIONAL MODEL BEHAVIOR Negative events (Negative Events Precision): additional “negative” events are added and if the model can reply any of these negative events when replaying a trace, then this is a case of extra model behavior. However this method is heuristic, and cannot guarantee that all extra behavior is detected. While there are improvements on performance (the method is exponential as it is exponentially large the number of possible negative events), these still do not guarantee 100% accuracy in detecting extra behavior Trace automata (ETC Conformance): a prefix automaton is built from the log, such that each state in the automaton corresponds to a given trace prefix. Then for each state, by replaying the prefix on the model, a corresponding state in the model is detected, and if the set of enabled transitions in this stage includes events that cannot be taken at that state in the log, then this is detected as an “escaping edge” (i.e. a sink) in the automaton, pinpointing the beginning of extra model behavior and the reply is continued. Two limitations: i) unable to handle duplicate labels and silent transitions, and ii) assumes fully-fitting log. These limitations are solved by the: Alignment-based ETC Conformance: first, an optimal alignment is calculated, including silent moves on model. Then the model-projection of this alignment is used to compute the automaton, after which traces are replayed to detect escaping edges. General limitation of trace automata: they cannot fully characterize the extra behavior in the model, but only pinpoint where this extra behavior takes place and with what task it starts.
• #6: The first statement characterizes the behavior observed in the log but not in the model: in the model, task C is compulsory, while in the log C is skippable The second statement characterizes the behavior observed in the model but not in the log Trace alignment would produce two optimal alignments: One between ABDEH of the log and ABCDHE of the model, the other between ABDFH of the log and ABCDFH of the model. From this one can infer that task C is optional in the log (move on log only). 1) However the number of misaligned traces is often very large, rendering this inference quite hard in practice. Visualizations, e.g. on top of Petri net, and at an aggregate level, can help, but fundamentally the problem is that trace alignment provides feedback at the level of individual traces, not at the level of behavioral relations observed in the log but not captured in the model. 2) Moreover, trace alignment would detect that there is escaping behavior starting with “Request addition information” at a trace prefix finishing with “Notify rejection”, but it will not identify that the extra behavior includes tasks IG and that IGDF is behavior that can be repeated in the model but not in the log. For example, task “Assess application” can be repeated in the model but not in the log.
• #8: Input model in BPMN and event log in XES or MXML BPMN converted to Petri nets using Remco, Chun and Marlon’s technique (Information and Software Technology)
• #9: The Petri net is unfolded using McMillan’s complete prefix unfolding technique From the log, first a set of partially-ordered runs are extracted over a concurrency relation discovered from the log. Then these runs are prefix-merged into a Prime Event Structure (PES)
• #10: The PSP is a representation of a synchronized traversal of two input PESs, such that when a discrepancy is detected it is explicitly recorded and the traversal resumes from a “suitable” configuration in each of the two PESs. If the discrepancy if of type “unfitting log behavior” this will be recorded by a node of the PSP, so we can enumerate all unfitting behavior. To expose additional model behavior, we define a notion of coverage of the PES extracted from the model by the PES extracted from the log. The parts of the model PES not covered by the log PES are the additional model behavior.
• #11: Each discrepancy falls under one of a set of disjoint patterns. For each pattern, we have a verbalization of the difference.
• #12: An occurrence of a task in the business process, as represented by the model or the log Note that conflict is “inherited” by causality. For example, if e # e’ and e’ <= e’’, then e # e’’ Nielsen et al, “Petri nets, event structures and domains, part I”, TCS, 1981
• #18: To simplify, in an event structure we don’t show: Transitive causality relations, e.g. A is causal to E Hereditary conflict, e.g. B and C are in conflict with {g2}E Concurrency: every pair of events that is neither directly nor transitively causally related, nor in conflict of course, is concurrent. E.g. B and C are concurrent Every state of a PES (called “configuration”) is represented by a set of events. A configuration is: causally closed, meaning that for each event e in C, C includes all causal predecessors of e, and conflict-free, meaning that there cannot be any pair of events in conflict within C. e is extension of C if {e} U C is also a configuration. A maximal configuration is a configuration that is maximal w.r.t. set inclusion Lossless representation: the set of maximal configurations of the PES is equivalent to the set of runs inferred from the log, modulo the inaccuracy of the concurrency oracle. The complexity of building such an event structure from a log is cubic on the length of the longest trace. We merge events with the same label (e.g. e0 and f0) and which have same history (same prefix)
• #19: The model is a variation of what we showed before, with C being optional and without loopback after F. Requirements: The Petri net must be safe The silent transitions will be eliminated when constructing the PES
• #20: First, we construct the branching process by prefix-merging all the partially ordered runs induced by the Petri net. We do so because branching processes explicitly represent the same set of behavioral relations as event structures. Transitions in the branching process represent events in the event structure, and the behavioral relations in the branching process can thus be used to generate an event structure. However, the event structure that we create by using a set of inductive rules, includes silent transitions. It has been shown (Abel’s IS paper) that silent transitions can be abstracted away in a behavioral-preserving manner, under the well-known notion of visible-pomset equivalence (which requires that sink events in the event structure are not silent, and we can always add fake ones). In the example, the future of t2 and t4 (C) is isomorphic, which is unfolded in the branching process separately for both t2 and t4. Furthermore, the future of E and F is isomorphic. Therefore, we can safely stop unfolding the branching process once we reach t4, provided that we continue unfolding from t2 and onwards.
• #21: McMillan showed that for a safe net, a prefix of a branching process that unfolds each loop once fully encodes the behavior of the original net. Such prefix is referred to as complete prefix unfolding of the net. We use Esparza’s optimization that has been shown to produce compact unfoldings The trick is to find transitions which have isomorphic futures. When we stop unfolding after t4, we call this the cutoff event. The event with the isomorphic future, t2, is called the corresponding event. The resulting cc-pair (t4,t2) (cutoff-corresponding pair) is grapically depicted here with the red line. Similar for E and F
• #22: We call the PES derived from a complex prefix unfolding PES prefix unfolding or simply prefix PES This translates to a PES prefix unfolding as shown in this slide. Reasoning about possible executions of a PES prefix unfolding is not convenient because some configurations are not explicitly represented. To make it more convenient to explore the configurations of a PES prefix unfolding, we use the “shift” operation on net unfoldings. Given a cc-pair (t4, t2), since the futures of [t4] and [t2] are isomorphic, we can “shift” from one configuration to the other. In other words, the shift operation is a “step” function that allows us to move from one configuration to another. Note that in a PES we retain those tau transitions that are involved in a cc-pair and safely get rid of the rest. The extraction of a complete prefix unfolding from a Petri net is in the worst-case scenario is exponential on the size of the net.
• #23: For loop relations, the advantage of the prefix unfolding and shift operations is clear, as it allows us to identify elementary pomsets (partially ordered multisets), so that elementary loops can be identified and such that the PES prefix is not required to unfold the cyclic behavior infinitely. This corresponds to unfolding every cycle so that it is traversed only once. We will talk about pomsets later, when illustrating how to identify additional (cyclic and acyclic) model behavior.
• #24: Let’s take this log and model, and their corresponding PESs
• #26: Event matching must be Label preserving (the two events must have the same label), and Order preserving (the two events must be consistent with the casual relation in the input PESs). For example, I cannot match two events enabled at a given synchronized state if one is casual to the current event while the other is concurrent, even if these have the same label
• #27: When matching, traversal of enabled events at a given configuration is done in lexicographical order to guarantee determinism.
• #28: we can also match C and D
• #31: In the example, note that C is optional in the log and mandatory in the model ONLY after state {A,B} and not always, e.g. in the model after {A} I can execute D and thus skip C At each synchronized state, the set of enabled events in the two PESs is checked, to identify those that are label-preserving and order-preserving. Label preservation is a simple check, but order-preservation requires a backward traversal of the event structure to check that all causality relations are maintained between each of the enabled events and the configuration path in the event structure. If this is not the case, the algorithm returns the set of events that have a causality discrepancy, the cut-off events being traversed and the set of all causality relations that are violated. This is later used to characterize behavioral mismatches. The PSP contruction aims at finding the optimal matchings (i.e. maximum number of matchings, meaning minimum number of hide) for every maximal configuration of the log PES. Hence, priority is given to the log. A PSP with no hide operations identifies a situation where the log is fully fitting into the model. --- Complexity of PSP construction: we use an A* heuristic to find the optimal number of matches, so worst-case is O(3^(nPES1 x nPES2)) where nPESx is the number of configurations of PESx. 3 is the branching factor – avg number of successors per state (match, lhide and rhide). Indeed each configuration of PES1 is associated with a configuration of PES2 via 3 possible operations
• #32: We only report on immediate causality (not transitive causality) and direct conflict (not inherited conflict) because we want to report each mismatch once: 1. Immediate causality vs concurrency 2. direct conflict vs concurrency direct conflict vs immediate causality Each mismatch occurs in a given context, i.e. a pair of configurations, one for each PES Relation mismatch patterns are O(n) where n is the number of arcs of the PSP (via optimizations of O(n^3)) --- Task absence / insertion is a “catch all” pattern, essentially saying that there is a task at a given configuration in the PES of the log but not in the corresponding configuration in the PES of the model
• #33: The pattern observed is the lhide and rhide of the same label in two different event structures. This happens because each configuration is conflict-free, so two conflicting events cannot co-exist in the same configuration (in our example a2 and b2). Note that it’s not enough to check that we have an lhide and an rhide of the same label, we also need to make sure that the two events being hidden are not order-preserving w.r.t. the enabling state. In our example, a2 is concurrent with a1 while b2 is causally related to b1, with the enabling state being {(a0,b0)A, (a2,b1)B. Given that there can be operations that are commutative, these two hides do not necessarily need to be contiguous. For example here after matching A I can lhide C and then match B. Note that the algorithm for computing the PSP applies partial order reductions from the field of model checking to remove redundant paths which lead to the same maximal configuration due to the concurrent enablement of events.
• #35: To characterize additional model behavior not observed in the log we define a notion of “coverage”: essentially every elementary path (no repetition) and elementary cycle (first and last event of the sequence being the same, and if I remove the last event the cycle becomes an elementary path) of the prefix PES must be covered by a maximal configuration of the log PES, after hide operations have been applied. If this is the case, then there isn’t any additional model behavior. Elementary paths and cycles are borrowed from Graph Theory. The nice thing is that in this way we do not detect a difference between the log and the model if in the log we observe a finite number of iterations of a cycle while in the model the cycle is clearly infinite. This is because we only need the elementary cycle in the prefix PES to be covered by the log PES. --- In the example, paths in the log PES are highlighted in the model PES We can see in this example that the log behavior is perfectly fitting the model, as each of the paths in the PES of the log exist in the PES of the model. However, the PES of the model has a loop with two entry points and two exit points, while the PES of the log doesn’t have any repetitive behavior (each entry point is in fact an exit point).
• #36: In the case of a PES prefix, the set of induced pomsets is infinite when the PES prefix captures cyclic behavior via cc-pairs. Therefore, we cannot enumerate all pomsets of a PES prefix in order to check if each of them is observed in the PES of the log. The set of elementary pomsets collectively cover all the possible pomsets induced by a PES prefix, such that every cycle is traversed only once.
• #39: e ≠ ┴ means that the event is defined in the log σ is the state where the behavioral discrepancy is observed --- Note: in the case of unmatched repetition in the log, the idea is very simple: if we have an lhide of a label that has already been observed in the history of the PSP, that means that we are against repeated log behavior that is not matched by the model.