Ch07 1-2-overview graphcoverage

Introduction to Software
Testing
(2nd edition)
Chapter 7.1, 7.2
Overview Graph Coverage
Criteria
Paul Ammann & Jeff Offutt
http://www.cs.gmu.edu/~offutt/softwaretes
t/
Update, January 2016

Ch. 7 : Graph Coverage
Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 2
Four Structures for
Modeling Software
Graphs LogicInput Space Syntax
Input
Models
Integ
Source
Applied
to
DNFSpecs
FSMsSource
Applied to
Use cases
Specs
Design
Source
Applied
to

Covering Graphs (7.1)
• Graphs are the most commonly used structure for
testing
• Graphs can come from many sources
– Control flow graphs
– Design structure
– FSMs and statecharts
– Use cases
• Tests usually are intended to “cover” the graph in
some way

Definition of a Graph
• A set N of nodes, N is not empty
• A set N0 of initial nodes, N0 is not empty
• A set Nf of final nodes, Nf is not empty
• A set E of edges, each edge from one node to
another
– ( ni , nj ), i is predecessor, j is successor

Example Graphs
1
32
4
N0 = { 1}
Nf = { 4 }
E = { (1,2), (1,3),
(2,4), (3,4) }
1
32
4
N0 = { }
Nf = { 4 }
E = { (1,2), (1,3),
(2,4), (3,4) }
10
1
54
8
2
6
9
3
7
N0 = { 1, 2, 3 }
Nf = { 8, 9, 10 }
E = { (1,4), (1,5), (2,5), (3,6), (3, 7), (4,
8), (5,8), (5,9), (6,9), (6,10), (7,10)
(9,6) }
Not a
valid
graph

Paths in Graphs
• Path : A sequence of nodes – [n1, n2, …, nM]
– Each pair of nodes is an edge
• Length : The number of edges
– A single node is a path of length 0
• Subpath : A subsequence of nodes in p is a subpath
of p
108 9
1 2 3
54 6 7
A Few Paths
[ 1, 4, 8 ]
[ 2, 5, 9, 6, 2 ]
[ 3, 7, 10 ]

Test Paths and SESEs
• Test Path : A path that starts at an initial node and
ends at a final node
• Test paths represent execution of test cases
– Some test paths can be executed by many tests
– Some test paths cannot be executed by any tests
• SESE graphs : All test paths start at a single node
and end at another node
– Single-entry, single-exit
– N0 and Nf have exactly one node
1
3
2
74
6
5
Double-diamond graph
Four test paths
[1, 2, 4, 5, 7]
[1, 2, 4, 6, 7]
[1, 3, 4, 5, 7]
[1, 3, 4, 6, 7]

Visiting and Touring
• Visit : A test path p visits node n if n is in p
A test path p visits edge e if e is in p
• Tour : A test path p tours subpath q if q is a subpath
of p
Path [ 1, 2, 4, 5, 7 ]
Visits nodes 1, 2, 4, 5, 7
Visits edges (1, 2), (2, 4), (4, 5), (5, 7)
Tours subpaths [1, 2, 4], [2, 4, 5], [4, 5, 7], [1, 2,
4, 5], [2, 4, 5, 7], [1, 2, 4, 5, 7]
(Also, each edge is technically a subpath)

Tests and Test Paths
• path (t) : The test path executed by test t
• path (T) : The set of test paths executed by the set of
tests T
• Each test executes one and only one test path
– Complete execution from a start node to an final node
• A location in a graph (node or edge) can be reached
from another location if there is a sequence of edges
from the first location to the second
– Syntactic reach : A subpath exists in the graph
– Semantic reach : A test exists that can execute that
subpath
– This distinction will become important in section 7.3

Tests and Test Paths
test 1
test 2
test 3
many-to-
one
test 1
test 2
test 3
many-to-many
Test Path 1
Test Path 2
Test Path 3
Non-deterministic software–the same test can execute
different test paths
Test
Path
Deterministic software–test always executes the same test path

Testing and Covering Graphs (7.2)
• We use graphs in testing as follows :
– Develop a model of the software as a graph
– Require tests to visit or tour specific sets of nodes, edges
or subpaths
• Test Requirements (TR) : Describe properties of test paths
• Test Criterion : Rules that define test requirements
• Satisfaction : Given a set TR of test requirements for a
criterion C, a set of tests T satisfies C on a graph if and only if
for every test requirement in TR, there is a test path in path(T)
that meets the test requirement tr• Structural Coverage Criteria : Defined on a graph just in terms
of nodes and edges
• Data Flow Coverage Criteria : Requires a graph to be
annotated with references to variables

Node and Edge Coverage
• The first (and simplest) two criteria require that each
node and edge in a graph be executed
Node Coverage (NC) : Test set T satisfies node
coverage on graph G iff for every syntactically
reachable node n in N, there is some path p in path(T)
such that p visits n.
Node Coverage (NC) : TR contains each reachable node
in G.
• This statement is a bit cumbersome, so we abbreviate it in
terms of the set of test requirements

Node and Edge Coverage
• Edge coverage is slightly stronger than node
coverage
Edge Coverage (EC) : TR contains each reachable path
of length up to 1, inclusive, in G.
• The phrase “length up to 1” allows for graphs with
one node and no edges
• NC and EC are only different when there is an edge
and another subpath between a pair of nodes (as in
an “if-else” statement)
Node Coverage : TR = { 1, 2, 3 }
Test Path = [ 1, 2, 3 ]
Edge Coverage : TR = { (1, 2), (1, 3), (2, 3) }
Test Paths = [ 1, 2, 3 ]
[ 1, 3 ]
2
3
1

Paths of Length 1 and 0
• A graph with only one node will not have any edges
• It may seem trivial, but formally, Edge Coverage
needs to require Node Coverage on this graph
1
• Otherwise, Edge Coverage will not subsume Node
Coverage
– So we define “length up to 1” instead of simply “length 1”
2
1
• We have the same issue with graphs
that only have one edge – for Edge-
Pair Coverage …

Covering Multiple Edges
• Edge-pair coverage requires pairs of edges, or
subpaths of length 2
Edge-Pair Coverage (EPC) : TR contains each
reachable path of length up to 2, inclusive, in G.
• The phrase “length up to 2” is used to include graphs
that have less than 2 edges
• The logical extension is to require all paths …
2
3
5
6
1
4
Edge-Pair Coverage :
TR = { [1,4,5], [1,4,6], [2,4,5],
[2,4,6], [3,4,5], [3,4,6] }

Covering Multiple Edges
Complete Path Coverage (CPC) : TR contains all paths in
G.
Specified Path Coverage (SPC) : TR contains a set S of
test paths, where S is supplied as a parameter.
Unfortunately, this is impossible if the graph has a
loop, so a weak compromise makes the tester decide
which paths:

Structural Coverage Example
Node Coverage
TR = { 1, 2, 3, 4, 5, 6, 7 }
Test Paths: [ 1, 2, 3, 4, 7 ] [ 1, 2, 3, 5, 6, 5, 7 ]
7
1
3
2
4 5
Edge Coverage
TR = { (1,2), (1, 3), (2, 3), (3, 4), (3, 5), (4, 7), (5, 6), (5,
7), (6, 5) }
Test Paths: [ 1, 2, 3, 4, 7 ] [1, 3, 5, 6, 5, 7 ]
Edge-Pair Coverage
TR = {[1,2,3], [1,3,4], [1,3,5], [2,3,4], [2,3,5], [3,4,7],
[3,5,6], [3,5,7], [5,6,5], [6,5,6], [6,5,7] }
Test Paths: [ 1, 2, 3, 4, 7 ] [ 1, 2, 3, 5, 7 ] [ 1, 3, 4, 7 ]
[ 1, 3, 5, 6, 5, 6, 5, 7 ]
Complete Path Coverage
Test Paths: [ 1, 2, 3, 4, 7 ] [ 1, 2, 3, 5, 7 ] [ 1, 2, 3, 5, 6,
5, 6 ] [ 1, 2, 3, 5, 6, 5, 6, 5, 7 ] [ 1, 2, 3, 5, 6, 5, 6, 5, 6, 5,
7 ] …
6

Handling Loops in Graphs
• If a graph contains a loop, it has an infinite number of
paths
• Thus, CPC is not feasible
• SPC is not satisfactory because the results are
subjective and vary with the tester
• Attempts to “deal with” loops:
– 1970s : Execute cycles once ([4, 5, 4] in previous example, informal)
– 1980s : Execute each loop, exactly once (formalized)
– 1990s : Execute loops 0 times, once, more than once (informal
description)
– 2000s : Prime paths (touring, sidetrips, and detours)

Simple Paths and Prime Paths
• Simple Path : A path from node ni to nj is simple if no
node appears more than once, except possibly the
first and last nodes are the same
– No internal loops
– A loop is a simple path
• Prime Path : A simple path that does not appear as a
proper subpath of any other simple path
Simple Paths : [1,2,4,1], [1,3,4,1], [2,4,1,2],
[2,4,1,3], [3,4,1,2], [3,4,1,3], [4,1,2,4], [4,1,3,4],
[1,2,4], [1,3,4], [2,4,1], [3,4,1], [4,1,2], [4,1,3], [1,2],
[1,3], [2,4], [3,4], [4,1], [1], [2], [3], [4]
Prime Paths : [2,4,1,2], [2,4,1,3], [1,3,4,1], [1,2,4,1],
[3,4,1,2], [4,1,3,4], [4,1,2,4], [3,4,1,3]
2 3
1
4

Prime Path Coverage
• A simple, elegant and finite criterion that requires
loops to be executed as well as skipped
Prime Path Coverage (PPC) : TR contains each prime path
in G.
• Will tour all paths of length 0, 1, …
• That is, it subsumes node and edge coverage
• PPC almost, but not quite, subsumes EPC …

PPC Does Not Subsume EPC
2
3
1
• If a node n has an edge to itself (self edge),
EPC requires [n, n, m] and [m, n, n]
• [n, n, m] is not prime
• Neither [n, n, m] nor [m, n, n] are simple paths
(not prime)
EPC Requirements :
TR = { [1,2,3], [1,2,2], [2,2,3], [2,2,2] }
PPC Requirements :
TR = { [1,2,3], [2,2] }

Prime Path Example
• The previous example has 38 simple paths
• Only nine prime paths
Prime Paths
[1, 2, 3, 4, 7]
[1, 2, 3, 5, 7]
[1, 2, 3, 5, 6]
[1, 3, 4, 7]
[1, 3, 5, 7]
[1, 3, 5, 6]
[6, 5, 7]
[6, 5, 6]
[5, 6, 5]
Execute
loop once
Execute loop
more than once
6
1
3
2
4 5
7
Execute
loop 0 times

Touring, Sidetrips, and Detours
• Prime paths do not have internal loops … test paths
might
• Tour : A test path p tours subpath q if q is a subpath
of p
• Tour With Sidetrips : A test path p tours subpath q
with sidetrips iff every edge in q is also in p in the
same order
• The tour can include a sidetrip, as long as it comes back to
the same node
• Tour With Detours : A test path p tours subpath q
with detours iff every node in q is also in p in the
same order
• The tour can include a detour from node ni, as long as it

Sidetrips and Detours Example
1 32 6
4
5
1 32 6
4
5
Touring with a
sidetrip
1 32 6
4
5
Touring with a
detour
1 2 5 6
3 4
1 2 5
3
4
1 2 3 4
Touring the prime path
[1, 2, 3, 5, 6] without
sidetrips or detours

Infeasible Test Requirements
• An infeasible test requirement cannot be satisfied
– Unreachable statement (dead code)
– Subpath that can only be executed with a contradiction (X > 0 and X <
0)
Practical recommendation—Best Effort Touring
– Satisfy as many test requirements as possible without
sidetrips
– Allow sidetrips to try to satisfy remaining test requirements
• Most test criteria have some infeasible test
requirements
• It is usually undecidable whether all test
requirements are feasible
• When sidetrips are not allowed, many structural
criteria have more infeasible test requirements
• However, always allowing sidetrips weakens the test
criteria

Simple & Prime Path Example
6
1
3
2
4 5
7
Len 0
[1]
[2]
[3]
[4]
[5]
[6]
[7] !
‘!’ means path
terminatesLen 1
[1, 2]
[1, 3]
[2, 3]
[3, 4]
[3, 5]
[4, 7] !
[5, 7] !
[5, 6]
[6, 5]
Len 2
[1, 2, 3]
[1, 3, 4]
[1, 3, 5]
[2, 3, 4]
[2, 3, 5]
[3, 4, 7] !
[3, 5, 7] !
[3, 5, 6] !
[5, 6, 5] *
[6, 5, 7] !
[6, 5, 6] *
‘*’ means path
cycles
Len 3
[1, 2, 3, 4]
[1, 2, 3, 5]
[1, 3, 4, 7] !
[1, 3, 5, 7] !
[1, 3, 5, 6] !
[2, 3, 4, 7] !
[2, 3, 5, 6] !
[2, 3, 5, 7] !
Len 4
[1, 2, 3, 4, 7] !
[1, 2, 3, 5, 7] !
[1, 2, 3, 5, 6] !
Prime Paths
Simple
paths

Round Trips
• Round-Trip Path : A prime path that starts and ends
at the same node
Simple Round Trip Coverage (SRTC) : TR contains at
least one round-trip path for each reachable node in G
that begins and ends a round-trip path.
Complete Round Trip Coverage (CRTC) : TR contains
all round-trip paths for each reachable node in G.
• These criteria omit nodes and edges that are not in
round trips
• Thus, they do not subsume edge-pair, edge, or node
coverage

Data Flow Criteria
• Definition (def) : A location where a value for a
variable is stored into memory
• Use : A location where a variable’s value is accessed
Goal: Try to ensure that values are computed and used
correctly
1
3
2
74
6
5
X = 42
Z = X-8
Z = X*2 Defs: def (1) = {X}
def (5) = {Z}
def (6) = {Z}
Uses: use (5) = {X}
use (6) = {X}
The values given in defs should reach at least one,
some, or all possible uses

DU Pairs and DU Paths
• def (n) or def (e) : The set of variables that are defined by node n
or edge e
• use (n) or use (e) : The set of variables that are used by node n or
edge e
• DU pair : A pair of locations (li, lj) such that a variable v is
defined at li and used at lj
• Def-clear : A path from li to lj is def-clear with respect to variable
v if v is not given another value on any of the nodes or edges in
the path
• Reach : If there is a def-clear path from li to lj with respect to v,
the def of v at li reaches the use at lj
• du-path : A simple subpath that is def-clear with respect to v
from a def of v to a use of v
• du (ni, nj, v) – the set of du-paths from ni to nj
• du (ni, v) – the set of du-paths that start at ni

Touring DU-Paths
• A test path p du-tours subpath d with respect to v if p
tours d and the subpath taken is def-clear with
respect to v
• Sidetrips can be used, just as with previous touring
• Three criteria
– Use every def
– Get to every use
– Follow all du-paths

Data Flow Test Criteria
All-defs coverage (ADC) : For each set of du-paths S =
du (n, v), TR contains at least one path d in S.
All-uses coverage (AUC) : For each set of du-paths to
uses S = du (ni, nj, v), TR contains at least one path d in
S.
All-du-paths coverage (ADUPC) : For each set S = du
(ni, nj, v), TR contains every path d in S.
• Then we make sure that every def reaches all
possible uses
• Finally, we cover all the paths between defs and
uses
• First, we make sure every def reaches a use

Data Flow Testing Example
1
3
2
74
6
5
X = 42
Z = X-8
Z = X*2
All-defs for X
[ 1, 2, 4, 5 ]
All-uses for X
[ 1, 2, 4, 5 ]
[ 1, 2, 4, 6 ]
All-du-paths for X
[ 1, 2, 4, 5 ]
[ 1, 3, 4, 5 ]
[ 1, 2, 4, 6 ]
[ 1, 3, 4, 6 ]

Graph Coverage Criteria
Subsumption
Simple Round
Trip Coverage
SRTCNode
Coverage
NC
Edge
Coverage
EC
Edge-Pair
Coverage
EPC
Prime Path
Coverage
PPC
Complete
Path
Coverage
CPC
Complete Round
Trip Coverage
CRTC
All-DU-Paths
Coverage
ADUP
All-uses
Coverage
AUC
All-defs
Coverage
ADC

Summary 7.1-7.2
• Graphs are a very powerful abstraction for designing
tests
• The various criteria allow lots of cost / benefit
tradeoffs
• These two sections are entirely at the “design
abstraction level” from chapter 2
• Graphs appear in many situations in software
– As discussed in the rest of chapter 7

Ch07 1-2-overview graphcoverage

More Related Content

What's hot (20)

Similar to Ch07 1-2-overview graphcoverage (20)

Recently uploaded (20)

Ch07 1-2-overview graphcoverage