Model Based Software Testing
2 likes1,615 views
This document discusses model-based testing (MBT), including what it is, the MBT process, tools that can be used, and benefits/limitations. MBT involves creating a model of the system under test and using that model to automatically generate test cases. The MBT process includes modelling the system, selecting test requirements, generating abstract test cases from the model, concretizing the tests into executable scripts, and executing the tests against the system. MBT can find faults, reduce costs and time, and improve test quality compared to manual testing. However, it requires skilled modelers and some testing experience to apply effectively.
Model Based Software Testing
- 1. Esin Karaman SWE550 Boğaziçi University 1
- 2. Agenda Software testing process evalution What is model-based testing The process of model-based testing MBT (Model-based testing) Tools Prerequisites for MBT Benefits of MBT Limitations of MBT 2
- 3. «Life is too short for manual testing.» Harry Robinson, test architect for Microsoft’s Engineering Excellence Group and a driving force behind Microsoft’s model-based testing initiative, 1999-2004 3
- 4. Software testing process evalution 4 Manual Testing Capture/Replay Script Base Testing Keyword-Driven Automated Testing Model-Based
- 5. What is model-based testing? Model-based testing is a variant of testing for automatic generation of test cases using models describing some aspects of the software under test (SUT). It can automate the complete test design process, given a suitable model, and produces complete test sequences that can be transformed into executable tests cripts. Model-based testing is the automation of the design of black-box tests 5
- 6. What is model-based testing? The difference from the usual black-box testing is that rather than manually writing tests based on the requirements documentation, we instead create a model of the expected SUT behaviour, which captures some of the requirements. Then the model based testing tools are used to automatically generate tests from that model. 6
- 7. The process of model-based testing Create a model detailed enough to describe the characteristics that we want to test Automatically generate abstract test cases from the model by using a model-based testing tool Transform abstract test cases into into executable test scripts Execute test scripts by a test execution tool 7
- 8. 8 Model-Based Testing Process
- 9. Modelling A model of the SUT is built on the grounds of requirements Includes expected behaviour Must be concise and precise 9
- 10. Modelling (2) Model-based testing uses models of the SUT and its environment. Abstraction - Function - Data M1-M3 is suitable 10
- 11. Modelling (3) Modelling notations State based (or Pre/Post) Notations Transition-based Notations Functional Notations Operational Notations Stochastic Notations Data-Flow Notations 11
- 12. Test requirements selection Used to control the generation of tests Steps: Test selection criteria are defined Test selection criteria are then transformed into test case specifications 12
- 13. Test selection criteria Defining «best» test selection criteria is not possible. Test engineer chooses adequate test selection criteria. Criteria types: Structural model coverage criteria Data coverage criteria Requirements-based coverage criteria Ad-Hoc test case specifications 13
- 14. Test generation Once the model and the test case specification are defined, an abstract test suite is generated. Technology: Random Graph-search algorithms Model-checking Symbolic execution Theorem proving 14
- 15. Test concretization(transformation) Concretize abstract test suite to executable level (test script). This can be done by a MBT Tool using some translation tables supplied by the test engineer. The resulting executable tests may be Junit in Java, or in a dynamic language such as Tcl or Python, or in a dedicated test scripting language. 15
- 16. Test execution Test scripts are executed and outcomes are compared to expected ones. Tests are marked as pass, fail or inconclusive Two approaches: Online Offline 16
- 17. Test execution(2) Online Testing Tests are executed as they are generated MBT tool is tightly coupled to the SUT Slow Offline Testing Tests are generated first, executed afterwards Decouples the generation and execution phases It can be used repeatedly for regression purposes 17
- 18. MBT Tools TorX LTG (LEIRIOS Test Generator) Matlab Simulink V&V JUMBL AETG 18
- 19. Prerequisites for MBT Someone enthusiastic Test execution maturity Modelling skills Access to training, tools, and experts The right kind of application 19
- 20. Benefits of MBT Fault detection : MBT founds roughly the same number of errors as manually designed test suits Reduced testing cost and time Improved test quality Requirements defect detection Requirements evolution 20
- 21. Limitations of MBT The model designers must be expert in the application area. This may require some training costs. Usually used only for functional testing, little experience for other kinds of testing. Sophisticated approach to testing, so it requires a reosonable level of test maturity. It is unwise to start using MBT without some experience with automated test execution. 21
- 22. References Mark Utting, Alexander Pretschner and Bruno Legeard , A Taxonomy of Model-Based Testing, 2006 Bruno Legeard, Mark Utting, Practical Model-Based Testing A Tools Approach, 2007 A. Pretschner, W. Prenninger, S. Wagner, C. Kuhnel, M. Baumgartner, B. Sostawa, One Evaluation of Model-Based Testing and its Automation, 2005 22