ODD Testing

Obstacle Driven Development
ODD Testing

Testing in History 1
Testing ideas is implicit to science
and technology.
• Testing implemented on products
for many years
• Ideas are assumptions without
sufficient testing
• Todays technology is a result of
centuries of tests
21/06/2016 ©odd.enterprises 2

Testing in History 2
Testing implemented on certain
products for many years.
• Tests must replicate real world
conditions
• Armour designed to be bullet
proof is tested
• Non standard components
required this approach

Cost of Failure
Cost of failure is often greater than
associated costs related to a
successful development.
• Undetected errors may become
very costly
• Increased use of specification can
reduce costs overall
• Cost increases exponentially for
each stage a bug is undetected

Fail Early, Fail Often
Achieving success with ODD is
through identifying, correcting and
preventing failure.
• Undiscovered errors cost 10x
more to fix by next stage
• Errors become expensive to solve
• 2 stages missed ≈ 100x
• 3 stages missed ≈ 1000x

Verification and Validation 1
Verification and validation occurs
between stages with appropriate
adaptions.
Verification and validation concern
the questions:
• Verification
Is it built in the right way?
• Validation
Is it built right?

Verification and Validation 2
Verification and validation
are adapted for each stage.
• Specification
– Verification and
validation (of behaviours)
• Solution
– Testing and design
• Production
– Quality assurance and
control
• Analysis
– Utilisation and
elicitation

ODD Elements
ODD Elements are single
generic parts of a
development.
• Divided into stage,
system and abstraction
level
• Higher level elements
will consist of
combined lower levels
• Each stage contains
different and distinct
elements

Analysis Elements
Analysis links Production
and Specification stages
with tests solved and
created by each level.
• Analysis links
Production by elicitation
of customers
• Analysis links
Specification by
verification of
behaviours

Specification Elements
Specification links
Analysis and Solution
stages with tests
solved and created by
each level.
• Specification links
Analysis through
solving of tests
• Specification links
Solution through
creation of tests

Solution Elements
Solution links Specification
and Production stages with
tests solved and created by
each level.
• Solution links
Specification by design
according to tests
• Solution links
Production by quality
assurance tests
according to solution

Production Elements
Production links Solution
and Analysis stages with
tests solved and created by
each level.
• Production links
Solution through quality
control according to
tests
• Production links
Analysis through
utilisation of products
features

ODD Flow Chart
Flow chart to demonstrate a
generic ODD process.
Problems or obstacles to be
overcome are divided into 4 stages
with appropriate testing.
• Analysis
• Specification
• Solution
• Production

Linking Tests 1
Tests link behaviours with
solutions through testing
and design.
• Solutions designed according to
tests from behaviours
• Each solution is a single element
of a product
• Unit testing is applied
• Test suite created and ran when
changes occur

Linking Tests 2
Testing and design concerns
solutions created from
behaviours of a specification.
• Each solution implements 1 or
more behaviours
• Tests suite ran for any changes
or additions
• Created as with Test Driven
Development

ODD Test Suites
Test suites implemented to
create a solution for software
and identify errors.
• Test suites contain individual
and combined unit tests
• Test suites are intended to be
implemented between all stages
• TDD process extended
throughout development to
create ODD

Linking Behaviours to Situations 1
Decision tree adapts to any
situation and linked to a
specification by creating tests.
• Any situation can be described
by a decision tree
• Diagram appropriate for failure
mode effects and analysis
• Linked to a behaviour which
covers the situation

Linking Behaviours to Situations 2
Each branch of a decision tree
results in a situation to be
covered by one or more
behaviours.
• Each situation is tested to
ensure coverage by specification
• Ensures situations are covered
before creation of solution
• All expected situations should
have an associated behaviour

Linking Solution to Production 1
Solution must be produced with
consistent quality and often for
very large quantities.
• Solution assures and controls
quality of related production
• Tests are created for quality
assurance
• Tests passed give measure for
quality control

Linking Solution to Production 2
Each element of a solution
creates a quality assurance test
which a production element
must pass.
• Quality assurance and control
based on probability
• Assurance will determine an
acceptable failure rate
• Control ensures failure rate is
acceptable

Linking Production to Analysis 1
Linking production to analysis
ensures product features are
utilised and elicited.
• Features of product utilised in
practical situations by customers
• Elicitation after utilisation of
feature to find requirements
• Verifies current solutions and
identifies new obstacles

Linking Production to Analysis 2
Linking production to analysis
has product features used as this
is how a customer sees a
product.
• New feature may cover
requirement but create
another
• Each product element and
level investigated separately

ODD is 3D

ODD without Tests

ODD with Tests

ODD with Passed Tests

ODD Generic Flow Chart
Each stage of ODD is an adaption of
this generic flow chart.
Flow chart is adapted to provide:
• Analysis - Utilisation and
Elicitation
• Specification – Verification and
Validation
• Solution - Testing and Design
• Production – Quality Assurance
and Control

Creating Tests 1
Creation of solutions inspired by
Behaviour Driven Development.
• Tests created by rewriting a
behaviour and designing test
• Design of solution according to
tests reduces debugging
• Testing and design may continue
until all behaviours implemented

Creating Tests 2
Full test suite created using each
behaviour in specification.
• Creating a test first ensures an
objective is understood
• Design according to passing tests
reduces ambiguity
• Passing a test ensures behaviour
is implemented

Flow Chart for Solution
1. Select behaviour which a solution
has to perform.
2. Test created to ensure solution
performs behaviour.
3. Solution designed to pass test.
4. Repeat until solution passes test.
5. Repeat until behaviours tested
and solutions designed.
6. Production stage begins.

Flowchart for Production
1. Select solution which
production is to create.
2. Quality assurance test ensures
production creates solution.
3. Production begins.
4. Repeat until quality is
controlled.
5. Repeat until all production is
quality controlled.
6. Analysis stage begins.

Flowchart for Analysis
1. Select feature to be elicited for
analysis.
2. Utilisation test is created for
feature.
3. Product is utilised by customers.
4. Repeat elicitation until sufficient
feedback obtained.
5. Repeat until all features in
production are tested.
6. Specification stage begins.

Flowchart for Specification
1. Requirement selected to be
covered by behaviour.
2. Verification test is created.
3. Behaviour is specified according
to the test.
4. Repeat until behaviour is
validated.
5. Repeat until all requirements
are covered by behaviours.
6. Solution stage begins.

ODD Materials
ODD is explained in further
presentations.
• Obstacle Driven
Development
• ODD: Requirements Analysis
• ODD: Extending a
Specification
• ODD: Extending TDD
• ODD: Extending V-models
• ODD Is Not Agile or Waterfall
ODD Is Not
Agile or
Waterfall
Obstacle Driven
Development
ODD:
Requirements
Analysis
ODD: Extending
a Specification
ODD: Extending
V-models
ODD:
Extending TDD

Further Information and Questions
• Website
• Presentations
• Facebook
• Twitter
• Email

Legal Stuff
References
Test Driven Development for Embedded C
James Grenning, 2011
Test Driven Development
http://en.wikipedia.org/wiki/Test-driven development
Behaviour Driven Development
http://en.wikipedia.org/wiki/Behavior-driven development
Unit Testing
http://en.wikipedia.org/wiki/Unit testing
Contact us for more information on sources and references.
Disclaimer
The ODD M-model and associated processes are provided by odd.enterprises and may be
used for any purpose whatsoever.
The names odd.enterprises and associated logos should not be used in any representation,
advertising, publicity or other manner whatsoever to endorse or promote any entity that
adopts or uses the model and/or associated processes.
odd.enterprises does not guarantee to provide support, consulting, training or assistance of
any kind with regards to the use of the model and/or processes including any updates.
You agree to indemnify odd.enterprises and its affiliates, officers, agents and employees
against any claim or demand including reasonable solicitors fees, related to your use,
reliance or adoption of the model and/or processes for any purpose whatsoever.
The model is provided by odd.enterprises “as is” and any express or implied warranties,
included but not limited to the implied warranties of merchantability and fitness for a
particular purpose are expressly disclaimed.
In no event shall odd.enterprises be liable for any damages whatsoever, including but not
limited to claims associated with the loss of data or profits, which may result from any
action in contract, negligence or other tortious claim that arises out of or in connection with
the use or performance of the model.

ODD Testing

More Related Content

What's hot (20)

Viewers also liked (14)

Similar to ODD Testing (20)

More from Jonathan Herring (7)

Recently uploaded (20)

ODD Testing