Where did I go wrong? Explaining errors in process models
1 like1,318 views
The document discusses the verification of process models, focusing on error diagnosis and the need for effective diagnostics in process model verification. It highlights various verification techniques and proposes improvements to reduce the length of diagnostic paths by eliminating spurious and obvious steps, thus identifying essential paths leading to errors. The authors suggest a systematic approach to enhance the diagnosis of process models to achieve better operational semantics and clarity in error tracing.
Where did I go wrong? Explaining errors in process models
- 1. Where did I go wrong? Explaining errors in process models Niels Lohmann @nlohmann Dirk Fahland @dfahland
- 2. Verification of processes and services 2 process model property verification technique diagnostic information
- 3. Verification of processes and services 3 verification technique diagnostic information BPMN Soundness domain-specific high-quality
- 4. Verification of processes and services 4 CMMN Declare WS-BPEL WS-Policy EPC BPMN YAWL Object Life-Cycles GSM Rules Soundness Security Compliance Conformance to … diagnostic information verification technique domain-specific high-quality -moving target -domain-specific approaches too specific to follow
- 5. Verification of processes and services 5 CMMN Declare WS-BPEL WS-Policy EPC BPMN YAWL Object Life-Cycles GSM Rules Soundness Security Compliance Conformance to … diagnostic information verification technique high-quality general purpose
- 6. Model checking general purpose verification approach: 1. formalize model and specification* 2. push a button 6 *can be hidden from the user
- 7. Effectiveness and efficiency -model checking works in reality -successful applications in many domains -very fast: “verify while you model” 7
- 8. Diagnosis -in case of error: outputs target state and produce a witness path -describes how target state can be reached -operational semantics: can be simulated 8 witness path target state
- 9. Diagnosis: the bad PATH process.s00000823##s00006200.inputCriterion.s00001053 fork.s00001071.activate.s00001072 fork.s00001071.fire.s00001078 merge.s00001061.activate.s00001065 merge.s00001061.fire.s00001069 callToTask.s00006202.inputCriterion.s00001053 callToTask.s00006202.outputCriterion.s00001055 callToTask.s00006211.inputCriterion.s00001053 callToTask.s00006211.outputCriterion.s00001055 callToTask.s00006209.inputCriterion.s00001053 callToTask.s00006209.outputCriterion.s00001055 decision.s00001158.activate.s00001072 decision.s00001158.fire.s00001075 merge.s00001160.activate.s00001064 merge.s00001160.fire.s00001069 callToTask.s00006203.inputCriterion.s00001053 callToTask.s00006203.outputCriterion.s00001055 callToTask.s00006214.inputCriterion.s00001053 callToTask.s00006214.outputCriterion.s00001055 callToTask.s00006213.inputCriterion.s00001053 callToTask.s00006213.outputCriterion.s00001055 decision.s00001840.activate.s00001072 decision.s00001840.fire.s00001075 callToTask.s00006201.inputCriterion.s00001053 callToTask.s00006201.outputCriterion.s00001055 decision.s00001123.activate.s00001072 decision.s00001123.fire.s00001075 merge.s00001161.activate.s00001064 merge.s00001161.fire.s00001069 callToTask.s00006208.inputCriterion.s00001053 callToTask.s00006208.outputCriterion.s00001055 decision.s00001157.activate.s00001072 decision.s00001157.fire.s00001073 fork.s00001071.fire.s00001073 merge.s00001061.activate.s00001064 join.s00001163.activate.s00001062 merge.s00001061.fire.s00001069 join.s00001163.activate.s00001064 merge.s00001162.activate.s00001062 merge.s00001162.fire.s00001069 callToTask.s00006210.inputCriterion.s00001053 callToTask.s00006210.outputCriterion.s00001055 decision.s00001159.activate.s00001072 decision.s00001159.fire.s00001073 join.s00001163.activate.s00001065 join.s00001163.fire.s00001069 fork.s00001071.fire.s00001075 merge.s00001160.activate.s00001065 callToTask.s00006207.inputCriterion.s00001053 merge.s00001160.fire.s00001069 callToTask.s00006207.outputCriterion.s00001055 callToTask.s00006203.inputCriterion.s00001053 decision.s00001126.activate.s00001072 callToTask.s00006203.outputCriterion.s00001055 decision.s00001126.fire.s00001073 callToTask.s00006214.inputCriterion.s00001053 callToTask.s00006202.inputCriterion.s00001053 callToTask.s00006202.outputCriterion.s00001055 callToTask.s00006211.inputCriterion.s00001053 callToTask.s00006211.outputCriterion.s00001055 callToTask.s00006209.inputCriterion.s00001053 callToTask.s00006209.outputCriterion.s00001055 decision.s00001158.activate.s00001072 decision.s00001158.fire.s00001075 callToTask.s00006214.outputCriterion.s00001055 callToTask.s00006213.inputCriterion.s00001053 callToTask.s00006213.outputCriterion.s00001055 decision.s00001840.activate.s00001072 decision.s00001840.fire.s00001075 callToTask.s00006201.inputCriterion.s00001053 callToTask.s00006201.outputCriterion.s00001055 decision.s00001123.activate.s00001072 decision.s00001123.fire.s00001073 callToTask.s00006204.inputCriterion.s00001053 callToTask.s00006204.outputCriterion.s00001055 callToTask.s00003714.inputCriterion.s00001053 callToTask.s00003714.outputCriterion.s00001055 callToTask.s00006215.inputCriterion.s00001053 callToTask.s00006215.outputCriterion.s00001055 callToTask.s00006206.inputCriterion.s00001053 join.s00001163.activate.s00001064 callToTask.s00006206.outputCriterion.s00001055 callToTask.s00006205.inputCriterion.s00001053 callToTask.s00006205.outputCriterion.s00001055 merge.s00001161.activate.s00001062 merge.s00001161.fire.s00001069 callToTask.s00006208.inputCriterion.s00001053 callToTask.s00006208.outputCriterion.s00001055 decision.s00001157.activate.s00001072 decision.s00001157.fire.s00001073 join.s00001163.activate.s00001062 callToTask.s00006212.inputCriterion.s00001053 callToTask.s00006212.outputCriterion.s00001055 merge.s00001162.activate.s00001064 merge.s00001162.fire.s00001069 callToTask.s00006210.inputCriterion.s00001053 callToTask.s00006210.outputCriterion.s00001055 decision.s00001159.activate.s00001072 decision.s00001159.fire.s00001073 join.s00001163.activate.s00001065 join.s00001163.fire.s00001069 callToTask.s00006207.inputCriterion.s00001053 callToTask.s00006207.outputCriterion.s00001055 decision.s00001126.activate.s00001072 decision.s00001126.fire.s00001073 STATE decision.s00001126.output.s00001054 : 2 -paths can become very long -length correlates with size of the model -reports all events equally: disregarding importance
- 10. This talk: better diagnosis PATH process.s00000823##s00006200.inputCriterion.s00001053 fork.s00001071.activate.s00001072 fork.s00001071.fire.s00001078 merge.s00001061.activate.s00001065 merge.s00001061.fire.s00001069 callToTask.s00006202.inputCriterion.s00001053 callToTask.s00006202.outputCriterion.s00001055 callToTask.s00006211.inputCriterion.s00001053 callToTask.s00006211.outputCriterion.s00001055 callToTask.s00006209.inputCriterion.s00001053 callToTask.s00006209.outputCriterion.s00001055 decision.s00001158.activate.s00001072 decision.s00001158.fire.s00001075 merge.s00001160.activate.s00001064 merge.s00001160.fire.s00001069 callToTask.s00006203.inputCriterion.s00001053 callToTask.s00006203.outputCriterion.s00001055 callToTask.s00006214.inputCriterion.s00001053 callToTask.s00006214.outputCriterion.s00001055 callToTask.s00006213.inputCriterion.s00001053 callToTask.s00006213.outputCriterion.s00001055 decision.s00001840.activate.s00001072 decision.s00001840.fire.s00001075 callToTask.s00006201.inputCriterion.s00001053 callToTask.s00006201.outputCriterion.s00001055 decision.s00001123.activate.s00001072 decision.s00001123.fire.s00001075 merge.s00001161.activate.s00001064 merge.s00001161.fire.s00001069 callToTask.s00006208.inputCriterion.s00001053 callToTask.s00006208.outputCriterion.s00001055 decision.s00001157.activate.s00001072 decision.s00001157.fire.s00001073 fork.s00001071.fire.s00001073 merge.s00001061.activate.s00001064 join.s00001163.activate.s00001062 merge.s00001061.fire.s00001069 join.s00001163.activate.s00001064 merge.s00001162.activate.s00001062 merge.s00001162.fire.s00001069 callToTask.s00006210.inputCriterion.s00001053 callToTask.s00006210.outputCriterion.s00001055 decision.s00001159.activate.s00001072 decision.s00001159.fire.s00001073 join.s00001163.activate.s00001065 join.s00001163.fire.s00001069 fork.s00001071.fire.s00001075 merge.s00001160.activate.s00001065 callToTask.s00006207.inputCriterion.s00001053 merge.s00001160.fire.s00001069 callToTask.s00006207.outputCriterion.s00001055 callToTask.s00006203.inputCriterion.s00001053 decision.s00001126.activate.s00001072 callToTask.s00006203.outputCriterion.s00001055 decision.s00001126.fire.s00001073 callToTask.s00006214.inputCriterion.s00001053 callToTask.s00006202.inputCriterion.s00001053 callToTask.s00006202.outputCriterion.s00001055 callToTask.s00006211.inputCriterion.s00001053 callToTask.s00006211.outputCriterion.s00001055 callToTask.s00006209.inputCriterion.s00001053 callToTask.s00006209.outputCriterion.s00001055 decision.s00001158.activate.s00001072 decision.s00001158.fire.s00001075 callToTask.s00006214.outputCriterion.s00001055 callToTask.s00006213.inputCriterion.s00001053 callToTask.s00006213.outputCriterion.s00001055 decision.s00001840.activate.s00001072 decision.s00001840.fire.s00001075 callToTask.s00006201.inputCriterion.s00001053 callToTask.s00006201.outputCriterion.s00001055 decision.s00001123.activate.s00001072 decision.s00001123.fire.s00001073 callToTask.s00006204.inputCriterion.s00001053 callToTask.s00006204.outputCriterion.s00001055 callToTask.s00003714.inputCriterion.s00001053 callToTask.s00003714.outputCriterion.s00001055 callToTask.s00006215.inputCriterion.s00001053 callToTask.s00006215.outputCriterion.s00001055 callToTask.s00006206.inputCriterion.s00001053 join.s00001163.activate.s00001064 callToTask.s00006206.outputCriterion.s00001055 callToTask.s00006205.inputCriterion.s00001053 callToTask.s00006205.outputCriterion.s00001055 merge.s00001161.activate.s00001062 merge.s00001161.fire.s00001069 callToTask.s00006208.inputCriterion.s00001053 callToTask.s00006208.outputCriterion.s00001055 decision.s00001157.activate.s00001072 decision.s00001157.fire.s00001073 join.s00001163.activate.s00001062 callToTask.s00006212.inputCriterion.s00001053 callToTask.s00006212.outputCriterion.s00001055 merge.s00001162.activate.s00001064 merge.s00001162.fire.s00001069 callToTask.s00006210.inputCriterion.s00001053 callToTask.s00006210.outputCriterion.s00001055 decision.s00001159.activate.s00001072 decision.s00001159.fire.s00001073 join.s00001163.activate.s00001065 join.s00001163.fire.s00001069 callToTask.s00006207.inputCriterion.s00001053 callToTask.s00006207.outputCriterion.s00001055 decision.s00001126.activate.s00001072 decision.s00001126.fire.s00001073 STATE decision.s00001126.output.s00001054 : 2
- 11. This talk: better diagnosis path essential path distill Why useless?
- 12. Reasons for useless paths 12 detours depth-first search indisputable parts bootstrapping interleavings concurrency
- 13. Running example 13 lack of synchronization
- 14. Reduction: obvious parts -classify transitions -only report points of alternative continuations* 14 * XOR-gateways, events, exceptions, … assume progress of flow
- 15. Reduction: obvious parts 15 t1 t2 t9 t10 t11 t12 t14 t8 t2 t3 t4 t5 “down” “down” “up”
- 16. Non-obvious “core” of a path ≈ 10-25% 16
- 17. Reduction: spurious decisions -can be found by model checking -results: 50%-80% spurious, occasionally no reduction (timeout) 17 p1 p3 p2 p4 p5 p6 p1 p3 p5 p6 genuine decision spurious decision = irrelevant for outcome
- 18. Reasons for useless paths 18 detours depth-first search indisputable parts bootstrapping interleavings concurrency
- 19. Reduction: unorder steps -idea: show independence of steps ( partially ordered runs) -makes synchronization points (milestones) explicit 19 independent steps many paths to same goal state order of steps irrelevant
- 20. Reduction: unorder steps 20 t1 t2 t9 t10 t11 t12 t14 t8 t2 t3 t4 t5 t9 t14 t3
- 21. More aid: preserve reference points 21 t1 t2 t9 t10 t11 t12 t14 t8 t2 t3 t4 t5 t9 t14 t3 p1 p6 p6 t5 t11 t10 t1
- 22. Final: remove obvious/spurious parts 22 t1 t2 t9 t10 t11 t12 t14 t8 t2 t3 t4 t5 t9 t14 t3 p1 p6 p6 t5 t11 t10 t1
- 23. Essential path: find source of error PATH process.s00000823##s00006200.inputCriterion.s00001053 fork.s00001071.activate.s00001072 fork.s00001071.fire.s00001078 merge.s00001061.activate.s00001065 merge.s00001061.fire.s00001069 callToTask.s00006202.inputCriterion.s00001053 callToTask.s00006202.outputCriterion.s00001055 callToTask.s00006211.inputCriterion.s00001053 callToTask.s00006211.outputCriterion.s00001055 callToTask.s00006209.inputCriterion.s00001053 callToTask.s00006209.outputCriterion.s00001055 decision.s00001158.activate.s00001072 decision.s00001158.fire.s00001075 merge.s00001160.activate.s00001064 merge.s00001160.fire.s00001069 callToTask.s00006203.inputCriterion.s00001053 callToTask.s00006203.outputCriterion.s00001055 callToTask.s00006214.inputCriterion.s00001053 callToTask.s00006214.outputCriterion.s00001055 callToTask.s00006213.inputCriterion.s00001053 callToTask.s00006213.outputCriterion.s00001055 decision.s00001840.activate.s00001072 decision.s00001840.fire.s00001075 callToTask.s00006201.inputCriterion.s00001053 callToTask.s00006201.outputCriterion.s00001055 decision.s00001123.activate.s00001072 decision.s00001123.fire.s00001075 merge.s00001161.activate.s00001064 merge.s00001161.fire.s00001069 callToTask.s00006208.inputCriterion.s00001053 callToTask.s00006208.outputCriterion.s00001055 decision.s00001157.activate.s00001072 decision.s00001157.fire.s00001073 fork.s00001071.fire.s00001073 merge.s00001061.activate.s00001064 join.s00001163.activate.s00001062 merge.s00001061.fire.s00001069 join.s00001163.activate.s00001064 merge.s00001162.activate.s00001062 merge.s00001162.fire.s00001069 callToTask.s00006210.inputCriterion.s00001053 callToTask.s00006210.outputCriterion.s00001055 decision.s00001159.activate.s00001072 decision.s00001159.fire.s00001073 join.s00001163.activate.s00001065 join.s00001163.fire.s00001069 fork.s00001071.fire.s00001075 merge.s00001160.activate.s00001065 callToTask.s00006207.inputCriterion.s00001053 merge.s00001160.fire.s00001069 callToTask.s00006207.outputCriterion.s00001055 callToTask.s00006203.inputCriterion.s00001053 decision.s00001126.activate.s00001072 callToTask.s00006203.outputCriterion.s00001055 decision.s00001126.fire.s00001073 callToTask.s00006214.inputCriterion.s00001053 callToTask.s00006202.inputCriterion.s00001053 callToTask.s00006202.outputCriterion.s00001055 callToTask.s00006211.inputCriterion.s00001053 callToTask.s00006211.outputCriterion.s00001055 callToTask.s00006209.inputCriterion.s00001053 callToTask.s00006209.outputCriterion.s00001055 decision.s00001158.activate.s00001072 decision.s00001158.fire.s00001075 callToTask.s00006214.outputCriterion.s00001055 callToTask.s00006213.inputCriterion.s00001053 callToTask.s00006213.outputCriterion.s00001055 decision.s00001840.activate.s00001072 decision.s00001840.fire.s00001075 callToTask.s00006201.inputCriterion.s00001053 callToTask.s00006201.outputCriterion.s00001055 decision.s00001123.activate.s00001072 decision.s00001123.fire.s00001073 callToTask.s00006204.inputCriterion.s00001053 callToTask.s00006204.outputCriterion.s00001055 callToTask.s00003714.inputCriterion.s00001053 callToTask.s00003714.outputCriterion.s00001055 callToTask.s00006215.inputCriterion.s00001053 callToTask.s00006215.outputCriterion.s00001055 callToTask.s00006206.inputCriterion.s00001053 join.s00001163.activate.s00001064 callToTask.s00006206.outputCriterion.s00001055 callToTask.s00006205.inputCriterion.s00001053 callToTask.s00006205.outputCriterion.s00001055 merge.s00001161.activate.s00001062 merge.s00001161.fire.s00001069 callToTask.s00006208.inputCriterion.s00001053 callToTask.s00006208.outputCriterion.s00001055 decision.s00001157.activate.s00001072 decision.s00001157.fire.s00001073 join.s00001163.activate.s00001062 callToTask.s00006212.inputCriterion.s00001053 callToTask.s00006212.outputCriterion.s00001055 merge.s00001162.activate.s00001064 merge.s00001162.fire.s00001069 callToTask.s00006210.inputCriterion.s00001053 callToTask.s00006210.outputCriterion.s00001055 decision.s00001159.activate.s00001072 decision.s00001159.fire.s00001073 join.s00001163.activate.s00001065 join.s00001163.fire.s00001069 callToTask.s00006207.inputCriterion.s00001053 callToTask.s00006207.outputCriterion.s00001055 decision.s00001126.activate.s00001072 decision.s00001126.fire.s00001073
- 24. Results: typical reduced paths 24 2x lack of synchronization improper completion deadlock
- 25. Summary -general purpose verification more user friendly -paths partial order of important decisions -applicable to any verification goal -keep reference points to aid diagnosis Next steps -error localization vs. explanation -detect useless cycles -How should a good diagnosis for $problem look like? 25
- 26. Where did I go wrong? Explaining errors in process models Niels Lohmann