Model-based Detection of Runtime Inconsistencies
Download as PPTX, PDF0 likes62 views
The document presents a model-based approach called IDBoM for detecting runtime inconsistencies. It aims to reduce manual effort by reusing existing software models. The approach was evaluated using a self-driving car case study. Key results include: (1) IDBoM allows full automation of inconsistency detection steps given connected services and models, (2) usability of model interactions is improved, (3) IDBoM can find inconsistencies involving incorrect method calls and message sequences, and (4) execution time scales linearly with model size. The approach provides a reusable solution for automated runtime inconsistency detection using design models.
![7
Methodology: Design Science [1,2]
[1] Wieringa, Roel J. Design science methodology for information systems and software engineering. Springer, 2014
[2] Hevner, Alan R., et al. "Design science in information systems research." MIS quarterly (2004): 75-105
Research Questions
Requirements
Artefact: IDBoM
Dynamic Checking Service (DynCS)
Service-Oriented Querying and Management of Models (SOMQM)
Evaluation
What is the answer to RQs?
Validation
(Are requirements satisfied by artefact?)
Problem
Static/dynamic Analysis
Mapping table
Demonstration Case
Discussion of results
Related Work from LiteratureComparison of Results
Contribution
Is there some improvement?](https://image.slidesharecdn.com/finalpresentation-200928123134/85/Model-based-Detection-of-Runtime-Inconsistencies-7-320.jpg)
![11
Usability of model interactions (RQ 2) [1]
[1] Shackel, B. (2009). Usability–Context, framework, definition, design and
evaluation. Interacting with computers, 21(5-6), 339-346.
Parameter XMI MDT-UML SOMQM
Required setup for accessing
model content
Prerequisites on programming
language
Capabilities for managing
several models
Abstraction level
Requirements on model input
Documentation
Best satisfaction of parameter
Second best satisfaction of parameter
Worst satisfaction of parameter
Results: Usability of model interactions is improved for all
chosen parameters (RQ2)](https://image.slidesharecdn.com/finalpresentation-200928123134/85/Model-based-Detection-of-Runtime-Inconsistencies-11-320.jpg)
![Identified potential parameters
CD: # Classes, Associations, Operations
AD: # Actions, Decision Nodes, Parallel Execution Paths
SD: # Messages
Test data
Test program: check if message sequence is valid with respect to design
model
13
Scalability (RQ 4) – experiment setting (3/3)
[1] Pfleeger, S. L. (1995). Experimental design and analysis in software engineering, part 5:
analyzing the data. ACM SIGSOFT Software Engineering Notes, 20(5), 14-17.](https://image.slidesharecdn.com/finalpresentation-200928123134/85/Model-based-Detection-of-Runtime-Inconsistencies-13-320.jpg)
Model-based Detection of Runtime Inconsistencies
- 1. Business Informatics Group Institute of Software Technology and Interactive Systems Vienna University of Technology Favoritenstraße 9-11/188-3, 1040 Vienna, Austria phone: +43 (1) 58801-18804 (secretary), fax: +43 (1) 58801-18896 office@big.tuwien.ac.at, www.big.tuwien.ac.at Model-based Detection of Runtime Inconsistencies Author: Daniel Lehner Advisors: Manuel Wimmer, Sabine Wolny
- 2. Motivation: traditional approach 2 I have to stop my car!
- 3. Motivation: self-driving car 3 Will my car stop?
- 4. Challenge 4 • Limited information whether system works as expected Static verification: state explosion Testing: mostly done before deployment Runtime monitoring: a lot of manual effort required for setup • For my thesis: How to reduce the manual effort for detecting runtime inconsistencies by reusing already existing software models?
- 5. Research Questions 5 RQ1 (Automation): Automation potential of reusing existing models for inconsistency detection. RQ2 (Usability): Comparative evaluation of the usability of model interaction features. RQ3 (Coverage): Defect kinds which can be found by model-based inconsistency detection. RQ4 (Scalability): Parameters that influence execution time of model- based inconsistency detection.
- 6. 6 Approach: Inconsistency Detection Based on Models (IDBoM) Design Time Does actual behavior match requirements? UML Class Diagram (expected structure) UML Activity Diagram (expected behavior) Check for - inconsistent structure - inconsistent behavior ClassB.getY() Runtime UML Sequence Diagrams (runtime traces) a:ClassAb:ClassB getX() ClassA - getX() ClassB - getY() response
- 7. 7 Methodology: Design Science [1,2] [1] Wieringa, Roel J. Design science methodology for information systems and software engineering. Springer, 2014 [2] Hevner, Alan R., et al. "Design science in information systems research." MIS quarterly (2004): 75-105 Research Questions Requirements Artefact: IDBoM Dynamic Checking Service (DynCS) Service-Oriented Querying and Management of Models (SOMQM) Evaluation What is the answer to RQs? Validation (Are requirements satisfied by artefact?) Problem Static/dynamic Analysis Mapping table Demonstration Case Discussion of results Related Work from LiteratureComparison of Results Contribution Is there some improvement?
- 9. • RQ1 (Automation): • Static analysis (demonstration case) • RQ2 (Usability) Static analysis (comparison framework) • RQ3 (Coverage) Functional testing + static analysis (demonstration case) • RQ 4 (Scalabilty) Dynamic analysis (controlled simulation experiment) 9 Evaluation + applied methods
- 10. Demonstration case: autonomously driving car Requirements: automation of the following steps • Triggering the checking process • Retrieving required information automation after process is triggered • Calculating the checking result from retrieved information • Processing the calculated checking result Results: automation of the required steps is possible Given running and connected services Given existing design time models 10 Automation (RQ1)
- 11. 11 Usability of model interactions (RQ 2) [1] [1] Shackel, B. (2009). Usability–Context, framework, definition, design and evaluation. Interacting with computers, 21(5-6), 339-346. Parameter XMI MDT-UML SOMQM Required setup for accessing model content Prerequisites on programming language Capabilities for managing several models Abstraction level Requirements on model input Documentation Best satisfaction of parameter Second best satisfaction of parameter Worst satisfaction of parameter Results: Usability of model interactions is improved for all chosen parameters (RQ2)
- 12. • Demonstration case: autonomously driving car • Derive inconsistency cases from CD and AD • 23 cases identified, involving Unallowed method calls Incorrect message sequences, with particular regard to • alternative execution paths • parallel execution paths Inconsistencies between software versions Execution time constraints • Results: IDBoM covers 22 cases Execution time constraints not yet supported 12 Coverage (RQ 3)
- 13. Identified potential parameters CD: # Classes, Associations, Operations AD: # Actions, Decision Nodes, Parallel Execution Paths SD: # Messages Test data Test program: check if message sequence is valid with respect to design model 13 Scalability (RQ 4) – experiment setting (3/3) [1] Pfleeger, S. L. (1995). Experimental design and analysis in software engineering, part 5: analyzing the data. ACM SIGSOFT Software Engineering Notes, 20(5), 14-17.
- 14. • Results: Linear expansion of execution time for all tested parameters • Runtime is mainly determined by the number of messages in SD Scalabilty (RQ 4) – Results 14 #Classes in CD #Actions in AD#Associations in CD #Operations in CD#Messages in SD
- 15. 15 Model-based Detection of Runtime Inconsistencies allows End-to-end automation Linear development of execution time Usability of interacting with models improved by Using a service-oriented approach Combining querying and management of models Automating generation of documentation Contribution End-to-end solution for model-based inconsistency detection Including scalability and automation analysis Reusable solution for automated processing of model information with improved usability for developers Conclusion
- 16. 16 Impact of chosen parameters for Improve absolute execution time values, exploiting Parallelization Cloud infrastructure Extension to an industrial setting Future Work
- 17. Model-driven runtime inconsistency detection Questions? Find my Implementation on Github • Inconsistency Checking Framework (Dynamic Checking Service): https://github.com/derlehner/dyncs • Model Interaction Framework (Service-Oriented Management and Querying of Models): https://github.com/derlehner/somqm Contact me daniel.lehner@jku.at https://www.researchgate.net/profile/Daniel_Lehner 17
- 18. Business Informatics Group Institute of Software Technology and Interactive Systems Vienna University of Technology Favoritenstraße 9-11/188-3, 1040 Vienna, Austria phone: +43 (1) 58801-18804 (secretary), fax: +43 (1) 58801-18896 office@big.tuwien.ac.at, www.big.tuwien.ac.at Daniel
- 19. Showcase of self- driving car: Class Diagram
- 20. Showcase of self-driving car: Activity Diagram
- 21. Functionality Example: incorrect method calls
- 22. Functionality Example: alternative execution paths
- 23. Functionality Example: parallel execution paths
- 24. Demonstration