S_22.ppt verification and validation in software testing

1 / 28
CS 425/625 Software Engineering
Verification and Validation
Based on Chapter 19 of the textbook [SE-6] Ian Sommerville,
Software Engineering, 6th
Ed., Addison-Wesley, 2000 and on the
Ch19 PowerPoint presentation available at the book’s web-site:
www.comp.lancs.ac.uk/computing/resources/IanS/SE6/Slides/index.html
November 10, 2003

2 / 28
Outline
 Introduction
 Testing and Debugging
 Verification and Validation Planning
 Software Inspections
 Program Inspections (forms of Software
Inspections)
 Automatic Static Analysis

3 / 28
Introduction…..
 Verification and validation (V&V): the checking and
analysis processes that ensure the software satisfies
its specification and meets the needs of the clients
who are paying for it
 Validation: “Are we building the right product?”
 Verification: “Are we building the product right?”
 Verification involves checking the software conforms
with its specification while the more general process
of validation ensures the software meets the needs of
the clients
 V&V is a whole life-cycle process, encompassing
requirements reviews, design reviews, code
inspections, and program testing

4 / 28
.Introduction.…
 V & V techniques:
V & V techniques:
 Software inspections
Software inspections
 Software testing
Software testing
 Software inspections
 Are static V&V techniques as they do not require the
Are static V&V techniques as they do not require the
software to be executed
software to be executed
 Consist of inspections, automated static analyses, and formal
Consist of inspections, automated static analyses, and formal
verifications of source code or system models
verifications of source code or system models
 Can only check the correspondence between the software
and its specification
 Cannot demonstrate the system is operationally useful
 Cannot check non-functional requirements of the software

5 / 28
..Introduction...
 Software testing
 It is a dynamic V&V technique as it needs an
executable version of the software system
 The system is executed with test data and its
operational behaviour is assessed
 Can reveal the presence of errors, not their absence
 A successful test is a test which discovers one or
more errors
 The V&V technique for checking non-functional
requirements and for verifying system integration
 Should be used in conjunction with static techniques
to provide full V&V coverage

6 / 28
…Introduction..
 Types of software testing
 Defect testing
 Intended to find inconsistencies between a program and its
specification
 Tests designed to discover program faults and defects
 A successful defect test is one which reveals the presence
of defects in a system
 Statistical testing
 Designed for software’s performance and reliability
estimation
 By running tests that reflect actual user inputs and their
frequency, an estimate of operational reliability can be made

7 / 28
….Introduction.
 Static and dynamic V&V [Fig 19.1, Somm00]
Formal
specification
High-level
design
Requirements
specification
Detailed
design
Program
Prototype
Dynamic
validation
Static
verification

8 / 28
…..Introduction
 Verification and validation should establish confidence
that the software is fit for purpose
 This does not mean the software is completely free of
defects; rather, it must be good enough for its intended
use
 The required level of confidence depends on:
 Software’s purpose: the level of confidence depends on
how critical the software is to an organisation
 User expectations: users may have lower expectations
of certain types of software
 Marketing environment: getting a product to market
early may have higher priority than finding its defects

9 / 28
Testing and debugging.
 Defect testing and debugging are distinct
processes
 V&V is concerned with establishing the
existence of defects in a program
 Debugging is concerned with locating and
repairing these errors
 Debugging involves formulating hypotheses
about program behaviour then testing these
hypotheses to find the errors

10 / 28
.Testing and debugging
The debugging process [Fig. 19.2, Somm00]
Locate
error
Design
error repair
Repair
error
Re-test
program
Test
results Specification Test
cases

11 / 28
 The planning of V&V should start early in the
development process
 The plan should balance static verification and
testing, specify testing standards and procedures,
establish checklists for inspections, and define the
software test plan
 Test planning breaks down V&V into a number of
stages, often organized according to the V-model
(shown on next page)
 The focus is on setting standards and procedures for
inspections and testing, not on describing product
tests
V & V Planning..

12 / 28
.V&V Planning.
.V&V Planning.
Requirements
specification
System
specification
System
design
Detailed
design
Module and
unit code
and tess
Sub-system
integration
test plan
System
integration
test plan
Acceptance
test plan
Service
Acceptance
test
System
integration test
Sub-system
integration test
The V-model of development [Fig. 19.3, Somm00]

13 / 28
..V&V Planning
..V&V Planning
 The structure of a software test plan
 The testing process
 Requirements traceability
 Tested items
 Testing schedule
 Test recording procedures
 Hardware and software requirements
 Constraints

14 / 28
Software Inspections.
 Involve people examining software code or models
(representations) with the aim of discovering defects
 Do not require execution of a system thus can be
used throughout the development process
 May be applied to any representation of the system:
requirements, design, test data, etc.
 Very effective technique for discovering errors

15 / 28
.Software Inspections
 In software inspections many different defects can
be discovered in a single review of the source code
or software model
 In testing, one defect may mask another hence
several executions are required
 Software inspections reuse domain and
programming knowledge so reviewers are likely to
have seen the types of error that commonly occur
 Software inspections and software testing are
complementary, not competing techniques (see also
slides 4 and 5)

16 / 28
Program Inspections…….
 Program inspections are a type of software
inspections
 Consist of formal reviews conducted by teams and
intended for program defect detection
 Defects may be logical errors, anomalies in the code
that might indicate an erroneous condition (e.g., an
un-initialized variable), or non-compliances with
standards

17 / 28
.Program Inspections……
 Inspection pre-conditions:
 A precise specification must be available
 Team members must be familiar with the
organisation standards
 Syntactically correct code must be available
 An error checklist should be prepared
 Management must accept that inspection will
increase costs early in the software process
 Management must not use inspections for staff
appraisal

18 / 28
Inspection
meeting
Individual
preparation
Overview
Planning
Rework
Follow-up
..Program Inspections…..
The inspection process [Fig. 19.6, Somm00]

19 / 28
…Program Inspections….
 Program inspection procedure:
 The inspection is planned by the moderator (or chairman)
 The system overview is presented to the inspection team by the
program author (or owner)
 The code and associated documents are distributed to the
inspectors (inspection team) in advance for individual preparation
 The inspection meeting takes place and errors are noted (the
inspection also involves a reader and, possibly, a scribe)
 During re-work modifications are made by the author to repair
discovered errors
 Re-inspection may or may not be required, based on moderator’s
decision

20 / 28
….Program Inspections…
 Inspection teams are
Inspection teams are made up of at least 4
members:
 Author of the code being inspected
 Inspector who finds errors, omissions and
inconsistencies
 Reader who reads the code to the team
 Moderator who chairs the meeting and notes
discovered errors
 Other roles are chief moderator and scribe

21 / 28
…..Program Inspections..
 Inspection checklist
 A checklist of common errors should be used
during the inspection
 This error checklist is programming language
dependent
 The weaker the type-checking of the language,
the larger the checklist
 Examples of possible errors: initialisation,
constant naming, loop termination, array
bounds, etc.

22 / 28
Inspection checks
(fault classes)
[Fig. 19.7, Somm00]
Fault class Inspection check
Data faults Are all program variables initialised before their values
are used?
Have all constants been named?
Should the lower bound of arraysbe 0, 1, or something
else?
Should the upper bound of arrays beequal to the size of
the array or Size -1?
If character strings are used, is a delimiter explicitly
assigned?
Control faults For each conditional statement, is the condition correct?
Is each loop certain to terminate?
Are compound statements correctly bracketed?
In case statements, are all possible cases accounted for?
Input/output faults Are all input variables used?
Are all output variables assigned a value before they are
output?
Interface faults Do all function and procedure calls have the correct
number of parameters?
Do formal and actual parameter types match?
Are the parameters in the right order?
If components access shared memory, do they have the
same model of the shared memory structure?
Storage management
faults
If a linked structure is modified, have all links been
correctly reassigned?
If dynamic storage is used, has space been allocated
correctly?
Is space explicitly de-allocated after it is no longer
required?
Exception
management faults
Have all possible error conditions been taken into
account?

23 / 28
……
…….Program Inspections
.Program Inspections
 Inspection rates:
 500 statements/hour during overview
 125 source statement/hour during individual
preparation
 90-125 statements/hour can be inspected
 Inspection is therefore an expensive process
 Inspecting 500 lines costs about 40 man/hours
effort = £2800

24 / 28
Automated Static Analysis….
 Static analyzers are software tools for source code
processing
 They parse the program text to discover erroneous
conditions
 Very effective as an aid to inspections; supplement
but cannot replace inspections
 Particularly valuable for languages such as C that
have weak typing (many errors can remain
undetected by the compiler)
 Less cost-effective for languages such as Java that
have strong type checking (many errors can be
detected during compilation)

25 / 28
.Automated Static Analysis…
Fault class Static analysis check
Data faults Variables used before initialisation
Variables declared but never used
Variables assigned twice but never used
between assignments
Possible array bound violations
Undeclared variables
Control faults Unreachable code
Unconditional branches into loops
Input/output faults Variables output twice with no intervening
assignment
Interface faults Parameter type mismatches
Parameter number mismatches
Non-usage of the results of functions
Uncalled functions and procedures
Storage management
faults
Unassigned pointers
Pointer arithmetic
Automatic static analysis checks [Fig. 19.8, Somm00]

26 / 28
..Automated Static Analysis..
 Stages of static analysis:
 Control flow analysis. Checks for loops with
multiple exit or entry points, finds unreachable
code, etc.
 Data use analysis. Detects un-initialised
variables, variables written twice without an
intervening assignment, variables which are
declared but never used, etc.
 Interface analysis. Checks the consistency of
routine and procedure declarations and their use

27 / 28
…Automated Static Analysis.
 Stages of static analysis [cont’d]:
 Information flow analysis. Identifies the
dependencies of output variables. Does not
detect anomalies itself but highlights
information for code review (inspection)
 Path analysis. Identifies paths through the
program and sets out the statements executed
in that path. Again, potentially useful in the
review process

28 / 28
LINT static analysis
[Fig 19.9, Somm00]
138% more lint_ex.c
#include <stdio.h>
printarray (Anarray)
int Anarray;
{
printf(“%d”,Anarray);
}
main ()
{
int Anarray[5]; int i; char c;
printarray (Anarray, i, c);
printarray (Anarray) ;
}
139% cc lint_ex.c
140% lint lint_ex.c
lint_ex.c(10): warning: c may be used before set
lint_ex.c(10): warning: i may be used before set
printarray: variable # of args. lint_ex.c(4) :: lint_ex.c(10)
printarray, arg. 1 used inconsistently lint_ex.c(4) :: lint_ex.c(10)
printarray, arg. 1 used inconsistently lint_ex.c(4) :: lint_ex.c(11)
printf returns value which is always ignored

S_22.ppt verification and validation in software testing

More Related Content

Similar to S_22.ppt verification and validation in software testing (20)

More from namrataparopate (10)

Recently uploaded (20)

S_22.ppt verification and validation in software testing