Using task models in model-based testing

USING TASK MODELS IN
MODEL-BASED TESTING
by Rachid Kherrazi
High Tech Campus Eindhoven, Netherland 16-04-2016
© All rights reserved

2
Rachid Kherrazi
Test specialist at Promedico
Test Process Improvement
Model Based Testing
Model Driven Engineering
iSQI Accredited Certified MBT Trainer
Also involved in this project
Anita Wierda User Experience specialist
Neda Noroozi MBT specialist
Arjan van der Meer MDE specialist

Agenda
• Introduction to Model Based
Testing
• What is MBT?
• What are benefits and
challenges?
• Why MBT?
• User Experience and Task Models
• From Task model to Test Model
using UXSpec tool
• Case study and results

IntegrationDesign
Coding
Requirements System test
Test case specification
(design of logical test case)
(reduction of number of test cases by application of test techniques
Test execution
(reporting)
Test case generation
(design of physical test case)
(selection of input values and calculation of expected results)
Test process
Rachid Kherrazi NextBuild 2016 4
3 main steps in test process
Module
Subsystem
System

MBT is the automation of test case generation
Manual Automatic
Test case specification
(design of logical test cases)
Test case execution
(and reporting)
Test case generation
(design of physical test case)
(Actions & expected results)
Modelbasedtesting
Traditionalmanualtesting
Traditionalautomatictesting
Manual C# Scripting
Testing skills + Scripting skills + Modeling skills
Modeling
Lightweight
Modeling languages
Generate
Scripting

Model Based Testing
6Rachid Kherrazi NextBuild 2016
+ better communication using models
+ abstract tests
+ auto design of tests, variety of test
suites
+ model checking
+ early exposure of ambiguities in
spec/design
+ ease of updating of test suites
+ systematic coverage
o automatic execution
o auto regression testing
- modelling overhead
- complexity of the models

Typical issues with traditional testing
• Many problems discovered at a late stage of development
process, often costly to resolve.
• In addition, they affect the planning of development processes
in that they must be resolved 'In between'
Several pre and post release defects not found with traditional
testing, which results in
Several delays in planning
With related quality/cost impact
Typical what we
hear from our
customers

Root Cause
• Unit testing is insufficient
• Single unit may work properly in isolation
• Incorrect interaction between units may cause serious reliability failures
• System-level testing
• Requires model of system behavior
• Behavior is often reactive/nondeterministic
• Implementation is multi-threaded or distributed
• Thread scheduling hard to control
• State space is typically infinite
• Traditional FSM-based testing does not scale

MBT helps to avoid integration issues and post
release defects
Rachid Kherrazi UCAAT 2015 9
Review
Static testing
Unit testing
Manual testing
happy flows
Manual testing
alternative flows
Manual testing
bad weather flows
Typical uncovered
Post release defects
Typical uncovered
Integration issues
This is typical domain
where Model Based
Testing helps
You can try to avoid
integration issues and
post release defects by
adding manual test case,
but it will be complex will
take lot of effort and
finally costly
Increased complexity and effort of testing
Manual exhaustive
exploratory testing

Other Drivers for MBT
Our world evolves….and becomes complex
• Cyber-Physical Systems/EoT
• Machine2Machine
• Digital Transformation
 Increased need to manage complexity
 Increased need to technology enablers
Innovative company’s need innovative tools and approaches to develop
innovative products and services
 Model Based System Engineering
 Model Driven Engineering
 Model Based Testing
Rachid Kherrazi UCAAT 2015 10

Spec Explorer and possible extensions
Explore &
Analyze
Remodel
Generate
Model Model Graph
Test Suite
Test Execute Tool
Nunit
Note: this is not implementation
code, this is the textual Test
model
(describing the interface
behavior)
MBT extension to overcome modeling /scripting skills and reuse available models
UML Extension
Sequence Diagrams
UXSpec
(converts Task Models to SE test
models)

MBT: workflow
Model
Cord
Script
Spec
Explorer
Test
Cases
Cord Scripts:
• Coverage /Slicing /Scenario Selection
Spec Explorer (MBT tool):
• Generate automatically test cases
MBT
Tester

Task Model in human-centered design
This figure shows the
interdependence of human-centered
design activities according to the ISO
9241-210 standard.
Blue rectangles show the 5 key design
activities in an iterative, human-
centered design process.
"UX deliverables" are key deliverables
from the corresponding design
activity.
During Analysis phase, Task Model
is one of the deliverables
Task Model: A description of the
subtasks that have to be carried out
in order to reach the
user’s goals

What are task models, anyway?
• Description of the process a user takes to reach a goal
in a specific domain
• Typically have hierarchical structure
• Number of different task modeling languages
• GOMS
• UAN
• ConcurTaskTrees (CTT)
http://giove.isti.cnr.it/attachments/publications/2003-A1-07.pdf

ConcurTaskTrees
• Developed by Fabio Paterno et al. for the design of user interfaces
• Goals
– Graphical for easy
interpretation
– Concurrent model for
representing UI tasks
– Different task types
• Represent all tasks,
including those
performed by the system

Task Building Process
• Three phases
• Hierarchically decompose the
tasks
• Identify the temporal
relationships among tasks at
same level
• Identify what objects are
manipulated and what actions
can be performed on them,
and assign these to the tasks
as appropriate.
• Temporal Relationships
• T1 [] T2 - Choice
• T1 ||| T2 - Interleaving
• T1 |[]| T2 - Synchronization
• T1 >> T2 - Enabling
• T1 []>> T2 - Enabling with
Information Passing
• T1 [> T2 - Deactivation
• T1* - Iteration
• T1(n) - Finite Iteration
• [T1] - Optional
• T – Recursion

Building/Editing Task Models
• Tools are available
• ConcurTaskTrees
Environment
Rachid Kherrazi NextBuild 2016
17
http://giove.cnuce.cnr.it/ctte.html or Google for “ConcurTaskTrees”
Ref: Paulo Filipe de Jesus Cruz
Example CTT

MBT: workflow
Model
Cord
Script
Spec
Explorer
Test
Cases
Cord Scripts:
• Coverage /Slicing /Scenario Selection
Spec Explorer (MBT tool):
• Generate automatically test cases
MBT
Tester
UXspec
Task
Model
Maria
(CTT)
UX Designer
Tester reuses existing UX Task models to
generate Spec Explorer MBT models

Case study:
Car reservation system
• Navigate to Cars tab
• Add a car to reservation system
• select ‘add Car’
• Enter Car info in the popup window
• Enter mandatory information (fields)
• Select Car Model
• Select a Station
• Enter additional information,
if applicable (Class of Car)
• Select ‘Ok’ to close the window and perform the action only if
“Eindhoven” station is selected
• Select ‘Cancel’ to close the window without performing the action
(for simplicity and demonstration we selected 2 requirements)
(Req. 1)
(Req. 2)

MBT + UXspec: generated Model
15 States
39 Transitions
13 Test cases
Complete Test cases coverage

Perceived benefits of approach
• Translation of design to Concur Task Tree seems straight
forward
• Clear visual way of specifying all interaction details (e.g. all
input fields in a screen) and user interaction flows
• Changes in requirements: Only keep this model up to date, not
have to change all design mockups
• Model can be maintained by others than UX (free tooling)easy
way to create test cases and apply MBT.

Questions?
• MBT is…..
• Strategies, tools and artifacts
• Manage complexity
• Reduces the need of manual or human involvement of interaction
• Avoids spending time in unskilled repetitive error prone or redundant tasks
• Provides bandwidth to Innovate!!

Case study:
perceived benefits
• Pilot has demonstrate the correct working of our tool chain
(feasibility study)
Perceived benefits:
• Enforce team involvements
• Better communication, less misunderstanding
• Increased productivity
• Reducing test specification effort by reusing task models
• Increased test Automation (Saving time and cost)
• Maintainability /Agility
• Easier and safer Maintainability by using task models as one
synchronization point
• Changes in requirements  Update in Task Model  automatic update in
Test cases
Next Steps
• Run project based on real product from industry
• Tool improvements
• Support data handling/configuration aspects in UXSpec,
instead of relying on manual additions in Spec Explorer,
• Supporting other MBT tools.

Results
Manual test design and manual
scripting
(automatic test execution)
MBT+ UXspec
SUT Car reservation system
(2 requirements )
Car reservation system
(2 requirements )
Test Design (logical
test case)
0,5 h for 2 test specification ½ h for creating task model
Test Generation
(physical test case)
0,5 h for 2 scripting the test cases Automatic generation of 13 test cases
Test Execution Automatic execution
(MS UI automation)
Automatic execution
(MS UI automation)
Number of Test
Cases
2 Test cases
(2 test case/hrs)
13 Test cases
(26 test case/hrs)
Coverage 2 scenarios
2 req
All scenarios
2 req
Maintenance
Effort
(change /new
requirements)
1 h for adding test scenario (new
requirement)
½ h for updating task model

Using task models in model-based testing

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to Using task models in model-based testing (20)

Recently uploaded (20)

Using task models in model-based testing