MDD and the Tautology Problem: Discussion Notes.
0 likes784 views
The document discusses the limitations of white-box testing and the implications of model-driven development (MDD) in software testing. It highlights issues such as the inability to test missing code and the challenges in verifying correctness and requirements. Additionally, it emphasizes the complexity of adequately testing transformed models due to various factors like semantic equality and assumptions made during code transformation.
MDD and the Tautology Problem: Discussion Notes.
- 1. MDD and the Tautology Problem: Discussion Notes MoDeVVA @ MODELS 2009 October 5, 2009 Robert V. Binder
- 2. The Tautology Effect • Transformation of model into object code is like transformation of source code into object code • An antecedent and its transformation cannot provide additional information necessary to reveal many common faults • White-box testing has well know limitations • Do these apply to implementations produced with MDD?
- 3. White Box Testing Limitations • Cannot test missing code – can’t reveal omissions • Cannot completely infer requirements (intent) from code • Where do we get the oracle? (how to discern correct/incorrect outputs?) • Triggering of side-effects • Platform-specific behavior/interactions • Blind to interaction effects outside the scope of the unit under test • Doesn’t scale • Doesn’t support “non-functional” testing • Possible developer bias in test selection
- 4. Weuyker: Adequacy Axioms • In terms of the antecedent (source code), what must an adequate test suite exercise? • Must at least cover every statement – What is a statement in a model? • Anti-extensionality – The semantic equality of two programs is not sufficient to imply that they should necessarily be tested the same way • Anti-decomposition – Although a program has been adequately tested, it does not necessarily imply that each of its called modules has been adequately tested. • More … • Elaine J. Weyuker, “The Evaluation Of Program-based Software Test Data Adequacy Criteria,” Communications of the ACM, v 31 n 6, June 1988.
- 5. Manna: Limits of Verification • Verification = proof of correctness – We can never be sure that the specifications are correct – No verification system can verify every correct program – We can never be certain that a verification system is correct • Zohar Manna and Richard Waldinger, “The Logic of Computer Programming,” IEEE Transactions on Software Engineering, v SE-4, n 3, May 1978
- 6. Some Observations • Transforming model elements to an executable – Fewer constraints than code expression to machine instructions – Many assumptions baked-in with translator – More fault opportunities • Many functionally equivalent implementations possible, suggests test strategy – M to E1 with T1, M to E2 with T2 – Run same test suite on E1 and E2 – Expect identical results for deterministic behavior – Expect bounded results for non-deterministic behavior