![Success factors
• Success is not guaranteed or automatic when implementing a testing tool, but many organizations have succeeded. Here are some of the
factors that have contributed to success:
• incremental roll-out (after the pilot) to the rest of the organization;
• adapting and improving processes, testware and tool artefacts to get the best fit and balance between them and the use of the tool;
• providing adequate training, coaching and mentoring of new users;
• defining and communicating guidelines for the use of the tool, based on what was learned in the pilot;
• implementing a continuous improvement mechanism as tool use spreads through more of the organization;
• monitoring the use of the tool and the benefits achieved and adapting the use of the tool to take account of what is learned.
• More information and advice about selecting and implementing tools can be found in [Fewster and Graham, 1999] and [Dustin et al., 1999].](https://image.slidesharecdn.com/chapter6toolsupportfortesting-171211084941/85/Chapter-6-Tool-Support-for-Testing-10-320.jpg)
Chapter 6 Tool Support for Testing
- 1. Tool Support forTesting ZETRYAN SATRIA Program Studi S1 Sistem Informasi Universitas Islam Negeri Sultan Syarif Kasim Riau
- 2. TYPES OFTESTTOOL • Classify different types of test tools according to the test process activities. (K2) • Recognize tools that may help developers in their testing. (Kl) • Test Tool Classification The tools are grouped by the testing activities or areas that are supported by a set of tools, for example, tools that support management activities, tools to support static testing, etc. . There is not necessarily a one-to-one relationship between a type of tool described here and a tool offered by a commercial tool vendor or an open-source tool. Some tools perform a very specific and limited function (sometimes called a 'point solution'), but many of the commercial tools provide support for a number of different functions (tool suites or families of tools). For example a 'test management' tool may provide support for managing testing (progress monitoring), configuration management of testware, incident management, and requirements management and traceability; another tool may provide both coverage measurement and test design support.
- 3. Tool support for management of testing and tests • What does 'test management' mean? It could be 'the management of tests' or it could be 'managing the testing process'. The tools in this broad category provide support for either or both of these. The management of testing applies over the whole of the software development life cycle, so a test management tool could be among the first to be used in a project. A test management tool may also manage the tests, which would begin early in the project and would then continue to be used throughout the project and also after the system had been released. In practice, test management tools are typically used by special- ist testers or test managers at system or acceptance test level. • Test management tools • The features provided by test management tools include those listed below. Some tools will provide all of these features; others may provide one or more of the features, however such tools would still be classified as test management tools.
- 4. Features or characteristics of test management tools include support for: • Management of tests (e.g. keeping track of the associated data for a given set of tests, knowing which tests need to run in a common environment, number of tests planned, written, run, passed or failed); • Scheduling of tests to be executed (manually or by a test execution tool); • Management of testing activities (time spent in test design, test execution, whether we are on schedule or on budget); • Interfaces to other tools, such as: test execution tools (test running tools); incident management tools; requirement management tools; configuration management tools; • Traceability of tests, test results and defects to requirements or other sources; • Logging test results (note that the test management tool does not run tests, but could summarize results from test execution tools that the test manage- ment tool interfaces with); • preparing progress reports based on metrics (quantitative analysis), such as: tests run and tests passed; incidents raised, defects fixed and outstanding.
- 5. Requirements management tools Are requirements management tools really testing tools? Some people may say they are not, but they do provide some features that are very helpful to testing. Because tests are based on requirements, the better the quality of the require- ments, the easier it will be to write tests from them. It is also important to be able to trace tests to requirements and requirements to tests, as we saw in Chapter 2. Features or characteristics of requirements management tools include support for: • storing requirement statements; • storing information about requirement attributes; • checking consistency of requirements; • identifying undefined, missing or 'to be defined later' requirements; • prioritizing requirements for testing purposes; • traceability of requirements to tests and tests to requirements, functions or features; • traceability through levels of requirements; • interfacing to test management tools; • coverage of requirements by a set of tests (sometimes).
- 6. Incident management tools This type of tool is also known as a defect-tracking tool, a defect-management tool, a bug-tracking tool or a bug-management tool. However, 'incident management tool' is probably a better name for it because not all of the things tracked are actually defects or bugs; incidents may also be perceived problems, anomalies (that aren't necessarily defects) or enhancement requests. Also what is normally recorded is information about the failure (not the defect) that was generated during testing - information about the defect that caused that failure would come to light when someone (e.g. a developer) begins to investigate the failure. • Features or characteristics of incident management tools include support for: • storing information about the attributes of incidents (e.g. severity); • storing attachments (e.g. a screen shot); • prioritizing incidents; • assigning actions to people (fix, confirmation test, etc.); • status (e.g. open, rejected, duplicate, deferred, ready for confirmation test, closed); • reporting of statistics/metrics about incidents (e.g. average time open, number of incidents with each status, total number raised, open or closed). • Incident management tool functionality may be included in commercial test management tools.
- 7. Configuration management tools • An example: A test group began testing the software, expecting to find the usual fairly high number of problems. But to their surprise, the software seemed to be much better than usual this time - very few defects were found. Before they cel- ebrated the great quality of this release, they just made an additional check to see if they had the right version and discovered that they were actually testing the version from two months earlier (which had been debugged) with the tests for that earlier version. It was nice to know that this was still OK, but they weren't actually testing what they thought they were testing or what they should have been testing. • Configuration management tools are not strictly testing tools either, but good configuration management is critical for controlled testing, as was described in Chapter 5. We need to know exactly what it is that we are sup- posed to test, such as the exact version of all of the things that belong in a system. It is possible to perform configuration management activities without the use of tools, but the tools make life a lot easier, especially in complex environments.
- 8. Configuration management tools Testware needs to be under configuration management and the same tool may be able to be used for testware as well as for software items. Testware also has different versions and is changed over time. It is important to run the correct version of the tests as well, as our earlier example shows. Features or characteristics of configuration management tools include support for: • storing information about versions and builds of the software and testware; • traceability between software and testware and different versions or variants; • keeping track of which versions belong with which configurations (e.g. operating systems, libraries, browsers); • build and release management; • baselining (e.g. all the configuration items that make up a specific release); • access control (checking in and out).
- 9. Pilot project • One of the ways to do a proof-of-concept is to have a pilot project as the first thing done with a new tool. This will use the tool in earnest but on a small scale, with sufficient time to explore different ways of using the tool. Objectives should be set for the pilot in order to assess whether or not the concept is proven, i.e. that the tool can accomplish what is needed within the current organizational context. • A pilot tool project should expect to encounter problems - they should be solved in ways that can be used by everyone later on. The pilot project should experiment with different ways of using the tool. For example, different settings for a static analysis tool, different reports from a test management tool, differ- ent scripting and comparison techniques for a test execution tool or different load profiles for a performance-testing tool. • The objectives for a pilot project for a new tool are: • to learn more about the tool (more detail, more depth); • to see how the tool would fit with existing processes or documentation, how those would need to change to work well with the tool and how to use the tool to streamline existing processes; • to decide on standard ways of using the tool that will work for all potential users (e.g. naming conventions, creation of libraries, defining modularity, where different elements will be stored, how they and the tool itself will be maintained); • to evaluate the pilot project against its objectives (have the benefits been achieved at reasonable cost?).
- 10. Success factors • Success is not guaranteed or automatic when implementing a testing tool, but many organizations have succeeded. Here are some of the factors that have contributed to success: • incremental roll-out (after the pilot) to the rest of the organization; • adapting and improving processes, testware and tool artefacts to get the best fit and balance between them and the use of the tool; • providing adequate training, coaching and mentoring of new users; • defining and communicating guidelines for the use of the tool, based on what was learned in the pilot; • implementing a continuous improvement mechanism as tool use spreads through more of the organization; • monitoring the use of the tool and the benefits achieved and adapting the use of the tool to take account of what is learned. • More information and advice about selecting and implementing tools can be found in [Fewster and Graham, 1999] and [Dustin et al., 1999].