Tool support for testing

Program Studi S1 Sistem Informasi
Fakultas Sains dan Teknologi
Universitas Islam Negeri Sultan Syarif Kasim Riau
ELVIRA MUNANDA

a)Classify different types of test tools according to the test process
activities. (K2)
b)Recognize tools that may help developers in their testing. (Kl)
In this section, we will describe the various tool types in terms of
their general functionality, rather than going into lots of detail. The
reason for this is that, in general, the types of tool will be fairly stable
over a longer period, even though there will be new vendors in the
market, new and improved tools, and even new types of tool in the
coming years.
We will not mention any commercial tools in this chapter. If we
did, this book would date very quickly! Tool vendors are acquired by
other vendors, change their names, and change the names of the tools
quite frequently, so we will not mention the names of any tools or
vendors
TYPES OF TEST TOOL

A. Test tool classification
The tools are grouped by the testing activities or areas that are
supported by a set of tools, for example, tools that support management
activities, tools to support static testing, etc.
There is not necessarily a one-to-one relationship between a type of
tool described here and a tool offered by a commercial tool vendor or an
open-source tool. Some tools perform a very specific and limited function
(sometimes called a 'point solution'), but many of the commercial tools
provide support for a number of different functions (tool suites or
families of tools). For example a 'test management' tool may provide
support for managing testing (progress monitoring), configuration
management of testware, incident management, and requirements
management and traceability; another tool may provide both coverage
measurement and test design support.
TYPES OF TEST TOOL

There are some things that people can do much better or easier than a com-
puter can do. For example, when you see a friend in an unexpected place, say in an
airport, you can immediately recognize their face. This is an example of pattern
recognition that people are very good at, but it is not easy to write software that can
recognize a face.
There are other things that computers can do much better or more quickly than people
can do. For example, can you add up 20 three-digit numbers quickly? This is not easy for
most people to do, so you are likely to make some mistakes even if the numbers are
written down. A computer does this accurately and very quickly. As another example, if
people are asked to do exactly the same task over and over, they soon get bored and
then start making mistakes.
The point is that it's a good idea to use computers to do things that computers
are really good at and that people are not very good at. So tool support is very useful for
repetitive tasks - the computer doesn't get bored and will be able to exactly repeat what
was done before. Because the tool will be fast, this can make those activities much
more efficient and more reliable. The tools can also do things that might overload a
person, such as comparing the contents of a large data file or simulating how the
system would behave.

B. Tool support for management of testing and tests
What does 'test management' mean? It could be 'the management of
tests' or it could be 'managing the testing process'. The tools in this
broad category provide support for either or both of these. The
management of testing applies over the whole of the software
development life cycle, so a test management tool could be among the
first to be used in a project. A test management tool may also manage
the tests, which would begin early in the project and would then
continue to be used throughout the project and also after the system
had been released. In practice, test management tools are typically
used by special- ist testers or test managers at system or acceptance
test level.

C. Tool support for static testing
Review process support tools
The value of different types of review was discussed in Chapter 3. For a
very informal review, where one person looks at another's document and
gives a few comments about it, a tool such as this might just get in the
way. However, when the review process is more formal, when many
people are involved, or when the people involved are in different
geographical locations, then tool support becomes far more beneficial.
It is possible to keep track of all the information for a review process
using spreadsheets and text documents, but a review tool which is
designed for the purpose is more likely to do a better job. For example,
one thing that should be monitored for each review is that the reviewers
have not gone over the document too quickly, i.e. that the checking rate
(number of pages checked per hour) was close to that recommended for
that review cycle. A review process support tool could automatically
calculate the checking rate and flag exceptions. The review process
support tools can normally be tailored for the particular review process
or type of review being done.

Features or characteristics of review process support tools include
support for:
a) a common reference for the review process or processes to use in
different situations;
b) storing and sorting review comments;
c) communicating comments to relevant people;
d) coordinating online reviews;
e) keeping track of comments, including defects found, and providing
statisti cal information about them;
f) providing traceability between comments, documents reviewed and
related documents;
g) a repository for rules, procedures and checklists to be used in reviews,
as well as entry and exit criteria;
h) monitoring the review status (passed, passed with corrections, requires
re- review);
i) collecting metrics and reporting on key factors.

Static analysis tools (D)
The '(D)' after this (and other types of tool) indicates that these
tools are more likely to be used by developers. Static analysis by tools
was discussed in Chapter 3. In this section we give a summary of what
the tools do.
Static analysis tools are normally used by developers as part of the
development and component testing process. The key aspect is that
the code (or other artefact) is not executed or run. Of course the tool
itself is executed, but the source code we are interested in is the input
data to the tool.
Static analysis tools are an extension of compiler technology - in fact
some compilers do offer static analysis features. It is worth checking
what is available from existing compilers or development environments
before looking at pur- chasing a more sophisticated static analysis tool.
Static analysis can also be carried out on things other than software
code, for example static analysis of requirements or static analysis of
websites (for example, to assess for proper use of accessibility tags or
the following of HTML standards).

Static analysis tools for code can help the developers to understand the
structure of the code, and can also be used to enforce coding standards. See
Section 6.2.3 for special considerations when introducing static analysis tools
into an organization.
Features or characteristics of static analysis tools include support to:
a) calculate metrics such as cyclomatic complexity or nesting levels (which can
help to identify where more testing may be needed due to increased risk);
b) enforce coding standards;
c) analyze structures and dependencies;
d) aid in code understanding;
e) identify anomalies or defects in the code (as described in Chapter 3 ).
Modeling tools (D)
Modeling tools help to validate models of the system or software. For
example a tool can check consistency of data objects in a database and can
find inconsistencies and defects. These may be difficult to pick up in testing -
you may have tested with one data item and not realize that in another part of
the database there is conflicting information related to that item. Modeling
tools can also check state models or object models.

Modeling tools are typically used by developers and can help in
the design of the software.
One strong advantage of both modeling tools and static analysis tools is
that they can be used before dynamic tests can be run. This enables any
defects that these tools can find to be identified as early as possible,
when it is easier and cheaper to fix them. There are also fewer defects
left to propa- gate into later stages, so development can be speeded up
and there is less rework. (Of course this is difficult to show, since those
defects aren't there now!)
Note that 'model-based testing tools' are actually tools that
generate test inputs or test cases from stored information about a
particular model (e.g. a state diagram), so are classified as test design
tools
Features or characteristics of modeling tools include support for:
a) identifying inconsistencies and defects within the model;
b) helping to identify and prioritize areas of the model for testing;
c) predicting system response and behavior under various situations, such
as level of load;
d) helping to understand system functions and identify test conditions
using a modeling language such as UML.

E. Tool support for test specification
Test design tools
Test design tools help to construct test cases, or at least test
inputs (which is part of a test case). If an automated oracle is available,
then the tool can also construct the expected result, so it can actually
generate test cases (rather than just test inputs).
For example, if the requirements are kept in a requirements
management or test management tool, or in a Computer Aided Software
Engineering (CASE) tool used by developers, then it is possible to
identify the input fields, including the range of valid values. This range
information can be used to identify bound- ary values and equivalence
partitions. If the valid range is stored, the tool can distinguish between
values that should be accepted and those that should generate an
error message. If the error messages are stored, then the expected
result can be checked in detail. If the expected result of the input of a
valid value is known, then that expected result can also be included in
the test case constructed by the test design tool.

F. Tool support for test execution and logging
Test execution tools
When people think of a 'testing tool', it is usually a test
execution tool that they have in mind, a tool that can run tests.
This type of tool is also referred to as a 'test running tool'. Most
tools of this type offer a way to get started by capturing or
recording manual tests; hence they are also known as
'capture/playback' tools, 'capture/replay' tools or
'record/playback' tools. The analogy is with recording a television
program, and playing it back. However, the tests are not
something which is played back just for someone to watch the
tests interact with the system, which may react slightly differently
when the tests are repeated

G. Tool support for performance and monitoring
The tools described in this section support testing that can be
carried out on a system when it is operational, i.e. while it is running.
This can be during testing or could be after a system is released into
live operation.
H. Tool support for specific application areas (Kl)
In this chapter, we have described tools according to their general
functional classifications. There are also further specializations of
tools within these classifications. For example there are web-based
performance-testing tools as well as performance-testing tools for
back-office systems. There are static analysis tools for specific
development platforms and programming languages, since each
programming language and every platform has distinct characteristics.
There are dynamic analysis tools that focus on security issues, as well
as dynamic analysis tools for embedded systems.Commercial tool sets
may be bundled for specific application areas such as web-based or
embedded systems.

H. Tool support for specific application areas (Kl)
In this chapter, we have described tools according to their
general functional classifications. There are also further
specializations of tools within these classifications. For example
there are web-based performance-testing tools as well as performance-
testing tools for back-office systems. There are static analysis tools for
specific development platforms and programming languages, since
each programming language and every platform has distinct
characteristics. There are dynamic analysis tools that focus on security
issues, as well as dynamic analysis tools for embedded
systems.Commercial tool sets may be bundled for specific application
areas such as web-based or embedded systems.

I. Tool support using other tools
The tools described in this chapter are not the only tools that a
tester can make use of. You may not normally think of a word processor
or a spreadsheet as a testing tool, but they are often used to store test
designs, test scripts or test data. Testers may also use SQL to set up and
query databases containing test data. Tools used by developers when
debugging, to help localize defects and check their fixes, are also
testing tools.
Developers use debugging tools when identifying and fixing
defects. The debugging tools enable them to run individual and localized
tests to ensure that they have correctly identified the cause of a defect
and to confirm that their change to the code will indeed fix the defect.
It is a good idea to look at any type of tool available to you for
ways it could be used to help support any of the testing activities. For
example, testers can use Perl scripts to help compare test results.

A. EFFECTIVE USE OF TOOLS: POTENTIAL BENEFITS AND RISKS
Summarize the potential benefits and risks of test automation and tool
support for testing. (K2)
Recognize that test execution tools can have different scripting tech
niques, including data-driven and keyword-driven. (Kl)
The reason for acquiring tools to support testing is to gain benefits, by using
a software program to do certain tasks that are better done by a computer than
by a person.
Advice on introducing tools into an organization can be found in web arti- cles,
magazines and books such as [Dustin et al., 1999], [Siteur, 2005] and [Fewster
and Graham, 1999].

a)Potential benefits of using tools
b)Risks of using tools
c)Special considerations for some types of
tools
EFFECTIVE USE OF TOOLS: POTENTIAL BENEFITS AND
RISKS

INTRODUCING A TOOL INTO AN
ORGANIZATION
a)State the main principles of introducing a
tool into an organization. (Kl)
b)State the goals of a proof-of-concept or
piloting phase for tool evaluation. (Kl)
c)Recognize that factors other than simply
acquiring a tool are requiredfor good tool
support. (Kl)

INTRODUCING A TOOL INTO AN
ORGANIZATION
a)Main principles
b)Pilot project
a)Success factors

http://sif.uin-suska.ac.id
http://fst.uin-suska.ac.id
http://www.uin-suska.ac.id
Website Resmi Kampus
Universitas Islam Negeri Sultan Syarif Kasim
Riau

Tool support for testing

More Related Content

What's hot (20)

Similar to Tool support for testing (20)

Recently uploaded (20)

Tool support for testing