SOFTWARE TESTING.pptx
- 1. SOFTWARE TESTING UNIT III Prepared by, Dr.T.Thendral, Assistant Professor, SRCW
- 2. OBJECTIVE • What is Software Testing? • Software testing fundamentals • Test Case Design Prepared by, Dr.T.Thendral, Assistant Professor, SRCW
- 3. • Software testing is a critical element of software quality assurance and represents the ultimate review of specification, design, and code generation • e.g., flight control, nuclear reactor monitoring • Software testing fundamentals define the overriding objectives for software testing • Test case design focuses on a set of techniques for the creation of test cases that meet overall testing objectives Prepared by, Dr.T.Thendral, Assistant Professor, SRCW
- 4. SOFTWARE TESTING FUNDAMENTALS • The engineer creates a series of test cases that are intended to "demolish" the software that has been built • software process that could be viewed (psychologically, at least) as destructive rather than constructive • Myth-we were really good at programming • There would be no bugs to catch Prepared by, Dr.T.Thendral, Assistant Professor, SRCW
- 5. Testing Objectives • 1. Process of executing a program with the intent of finding an error • 2. A good test case - finding an as-yet undiscovered error • 3. A successful test is one that uncovers an as- yet-undiscovered error Prepared by, Dr.T.Thendral, Assistant Professor, SRCW
- 6. Testing Principles • Software engineer must understand the basic principles that guide software testing • All tests should be traceable to customer requirements • Tests should be planned long before testing begins • The Pareto principle applies to software testing(80/20%) • Testing should begin “in the small” and progress toward testing “in the large” • Exhaustive testing is not possible • To be most effective, testing should be conducted by an independent third party Prepared by, Dr.T.Thendral, Assistant Professor, SRCW
- 7. Testability • Software testability is simply how easily can be tested • Metrics that could be used to measure testability in most of its aspects Prepared by, Dr.T.Thendral, Assistant Professor, SRCW
- 8. Operability • "The better it works, the more efficiently it can be tested“ • The system has few bugs (bugs add analysis and reporting overhead to the test process) • No bugs block the execution of tests • The product evolves in functional stages (allows simultaneous development) Prepared by, Dr.T.Thendral, Assistant Professor, SRCW
- 9. Observability • Distinct output is generated for each input • System states and variables are visible or queriable during execution • Past system states and variables are visible or queriable (e.g., transaction logs) • All factors affecting the output are visible • Incorrect output is easily identified • Internal errors are automatically detected through self- testing mechanisms • Internal errors are automatically reported • Source code is accessible Prepared by, Dr.T.Thendral, Assistant Professor, SRCW
- 10. Controllability • The better control the software, the more the testing can be automated and optimized • All possible outputs can be generated through some combination of input • All code is executable through some combination of input • Software and hardware states and variables can be controlled directly by the test engineer • Input and output formats are consistent and structured • Tests can be conveniently specified, automated, and reproduced. Prepared by, Dr.T.Thendral, Assistant Professor, SRCW
- 11. Decomposability • "By controlling the scope of testing, we can more quickly isolate problems and perform smarter retesting – The software system is built from independent modules. – Software modules can be tested independently. Prepared by, Dr.T.Thendral, Assistant Professor, SRCW
- 12. Simplicity • The less there is to test, the more quickly we can test it • Functional simplicity (e.g., the feature set is the minimum necessary to meet requirements • Structural simplicity (e.g., architecture is modularized to limit the propagation of faults). • Code simplicity (e.g., a coding standard is adopted for ease of inspection and maintenance) Prepared by, Dr.T.Thendral, Assistant Professor, SRCW
- 13. Stability • "The fewer the changes, the fewer the disruptions to testing. – Changes to the software are infrequent – Changes to the software are controlled – Changes to the software do not invalidate existing tests – The software recovers well from failures Prepared by, Dr.T.Thendral, Assistant Professor, SRCW
- 14. Understandability • The more information we have, the smarter we will test – The design is well understood – Dependencies between internal, external, and shared components are well understood – Changes to the design are communicated – Technical documentation is instantly accessible – Technical documentation is well organized – Technical documentation is specific and detailed Prepared by, Dr.T.Thendral, Assistant Professor, SRCW
- 15. What are the attributes of a “good” test? • A good test has a high probability of finding an error • A good test is not redundant • A good test should be “best of breed” • A good test should be neither too simple nor too complex Prepared by, Dr.T.Thendral, Assistant Professor, SRCW
- 16. TEST CASE DESIGN • The design of tests for software and other engineered products can be as challenging as the initial design of the product itself • A rich variety of test case design methods have evolved for software • Any engineered product (and most other things) can be tested in one of two ways Prepared by, Dr.T.Thendral, Assistant Professor, SRCW
- 17. • Black-box testing - the tester is not concerned with the internal knowledge or Implementation details of the software, but rather focuses on validating the functionality based on the provided specifications or requirements • White-box testing- White box testing is a form of application testing that provides the tester with complete knowledge of the application being tested, including access to source code and design documents Prepared by, Dr.T.Thendral, Assistant Professor, SRCW
- 18. WHITE-BOX TESTING • White-box testing, sometimes called glass-box testing • The software engineer can derive test cases – guarantee that all independent paths within a module have been exercised at least once – exercise all logical decisions on their true and false sides – execute all loops at their boundaries and within their operational bounds, and – exercise internal data structures to ensure their validity Prepared by, Dr.T.Thendral, Assistant Professor, SRCW
- 19. • Logic errors and incorrect assumptions are inversely proportional to the probability that a program path will be executed • We often believe that a logical path is not likely to be executed when, in fact, it may be executed on a regular basis • Typographical errors are random – Each of these reasons provides an argument for conducting white-box tests Prepared by, Dr.T.Thendral, Assistant Professor, SRCW
- 20. BASIS PATH TESTING • Basis path testing is a white-box testing technique first proposed by Tom McCabe • The basis path method enables the test case designer to derive a logical complexity measure of a procedural design • Test cases derived to exercise the basis set are guaranteed to execute every statement in the program at least one time during Prepared by, Dr.T.Thendral, Assistant Professor, SRCW
- 21. Flow Graph Notation • A simple notation for the representation of control flow, called a flow graph (or program graph • The flow graph depicts logical control flow using the notation illustrated in Figure Prepared by, Dr.T.Thendral, Assistant Professor, SRCW
- 22. Prepared by, Dr.T.Thendral, Assistant Professor, SRCW
- 23. • Figure 17.2B maps the flowchart into a corresponding flow graph • A sequence of process boxes and a decision diamond can map into a single node • The arrows on the flow graph, called edges or links, represent flow of control and are analogous to flowchart arrows • Areas bounded by edges and nodes are called regions Prepared by, Dr.T.Thendral, Assistant Professor, SRCW
- 24. FLOW GRAPH Prepared by, Dr.T.Thendral, Assistant Professor, SRCW
- 25. A compound condition occurs when one or more Boolean operators (logical OR, AND, NAND, NOR) is present in a conditional statement Prepared by, Dr.T.Thendral, Assistant Professor, SRCW
- 26. Cyclomatic Complexity • Cyclomatic complexity is a software metric that provides a quantitative measure of the logical complexity of a program • path 1: 1-11 • path 2: 1-2-3-4-5-10-1-11 • path 3: 1-2-3-6-8-9-10-1-11 • path 4: 1-2-3-6-7-9-10-1-11 Prepared by, Dr.T.Thendral, Assistant Professor, SRCW
- 27. • Cyclomatic complexity has a foundation in graph theory and provides us with an extremely useful software metric • Complexity is computed in one of three ways: – 1. The number of regions of the flow graph correspond to the cyclomatic complexity – 2. Cyclomatic complexity, V(G), for a flow graph, G, is defined as V(G) = E -N + 2 – where E is the number of flow graph edges, N is the number of flow graph nodes. – 3. Cyclomatic complexity, V(G), for a flow graph, G, is also defined as V(G) = P + 1 • where P is the number of predicate nodes contained in the flow graph G Prepared by, Dr.T.Thendral, Assistant Professor, SRCW
- 28. • The flow graph has four regions. • 2. V(G) = 11 edges 9 nodes + 2 = 4 • 3. V(G) = 3 predicate nodes + 1 = 4 Prepared by, Dr.T.Thendral, Assistant Professor, SRCW
- 29. Deriving Test Cases • The following steps can be applied to derive the basis set: – Using the design or code as a foundation, draw a corresponding flow graph – Determine the cyclomatic complexity of the resultant flow graph • The cyclomatic complexity, V(G), is determined by applying the algorithms – Determine a basis set of linearly independent paths Prepared by, Dr.T.Thendral, Assistant Professor, SRCW
- 30. Prepared by, Dr.T.Thendral, Assistant Professor, SRCW
- 31. • Each test case is executed and compared to expected results • Once all test cases have been completed, the tester can be sure that all statements in the program have been executed at least once Prepared by, Dr.T.Thendral, Assistant Professor, SRCW
- 32. Graph Matrices • To develop a software tool that assists in basis path testing, a data structure, called a graph matrix, can be quite useful. • A graph matrix is a square matrix whose size (i.e., number of rows and columns) is equal to the number of nodes on the flow graph • Referring to the figure, each node on the flow graph is identified by numbers, while each edge is identified by letters • The graph matrix is nothing more than a tabular representation of a flow graph Prepared by, Dr.T.Thendral, Assistant Professor, SRCW
- 33. Prepared by, Dr.T.Thendral, Assistant Professor, SRCW
- 34. Prepared by, Dr.T.Thendral, Assistant Professor, SRCW
- 35. CONTROL STRUCTURE TESTING • These broaden testing coverage and improve quality of white-box testing • Condition Testing: Condition testing is a test case design method that exercises the logical conditions contained in a program module • A simple condition is a Boolean variable or a relational expression Prepared by, Dr.T.Thendral, Assistant Professor, SRCW
- 36. • If a condition is incorrect, then at least one component of the condition is incorrect • Types of errors in a condition include the following: • • Boolean operator error (incorrect/missing/extra Boolean operators). • • Boolean variable error • • Boolean parenthesis error • • Relational operator error • • Arithmetic expression error Prepared by, Dr.T.Thendral, Assistant Professor, SRCW
- 37. Data Flow Testing • The data flow testing method selects test paths of a program according to the locations of definitions and uses of variables in the program • DEF(S) = {X | statement S contains a definition of X} • USE(S) = {X | statement S contains a use of X • Data flow testing strategies are useful for selecting test paths of a program containing nested if and loop statements • To illustrate this, consider the application of DU testing to select test paths for the PDL that follows Prepared by, Dr.T.Thendral, Assistant Professor, SRCW
- 38. Prepared by, Dr.T.Thendral, Assistant Professor, SRCW
- 39. Loop Testing • Loop testing is a white-box testing technique that focuses exclusively on the validity of loop constructs. • Four different classes of loops: simple loops, concatenated loops, nested loops, and unstructured loops • Simple loops: – 1. Skip the loop entirely – 2. Only one pass through the loop – 3. Two passes through the loop – 4. m passes through the loop where m < n. 5. n 1, n, n + 1 passes through the loop Prepared by, Dr.T.Thendral, Assistant Professor, SRCW
- 40. Prepared by, Dr.T.Thendral, Assistant Professor, SRCW
- 41. BLACK-BOX TESTING • Black-box testing, also called behavioral testing, focuses on the functional requirements of the software • That is, black-box testing enables the software engineer to derive sets of input conditions that will fully exercise all functional requirements for a program Prepared by, Dr.T.Thendral, Assistant Professor, SRCW
- 42. Tests are designed to answer the following questions: • How is functional validity tested? • How is system behaviour and performance tested? • What classes of input will make good test cases? • Is the system particularly sensitive to certain input values? • How are the boundaries of a data class isolated? • What data rates and data volume can the system tolerate? • What effect will specific combinations of data have on system operation? Prepared by, Dr.T.Thendral, Assistant Professor, SRCW
- 43. Graph-Based Testing Methods • The first step in black-box testing is to understand the objects that are modeled in software and the relationships that connect these objects Prepared by, Dr.T.Thendral, Assistant Professor, SRCW
- 44. Prepared by, Dr.T.Thendral, Assistant Professor, SRCW
- 45. Equivalence Partitioning • Equivalence partitioning is a black-box testing method that divides the input domain of a program into classes of data from which test cases can be derived • Equivalence classes may be defined according to the following guidelines: – 1. If an input condition specifies a range, one valid and two invalid equivalence classes are defined – 2. If an input condition requires a specific value, one valid and two invalid equivalence classes are defined – 3. If an input condition specifies a member of a set, one valid and one invalid equivalence class are defined – 4. If an input condition is Boolean, one valid and one invalid class are defined. Prepared by, Dr.T.Thendral, Assistant Professor, SRCW
- 46. Boundary Value Analysis • Boundary value analysis leads to a selection of test cases that exercise bounding values • 1. If an input condition specifies a range bounded by values a and b, test cases should be designed with values a and b and just above and just below a and b. • 2. If an input condition specifies a number of values, test cases should be developed that exercise the minimum and maximum numbers. Values just above and below minimum and maximum are also tested. • 3. Apply guidelines 1 and 2 to output conditions. Test cases should be designed to create an output report that produces the maximum (and minimum) allowable number of table entries. • 4. If internal program data structures have prescribed boundaries (e.g., an array has a defined limit of 100 entries), be certain to design a test case to exercise the data structure at its boundary Prepared by, Dr.T.Thendral, Assistant Professor, SRCW
- 47. Comparison Testing • These independent versions form the basis of a black-box testing technique called comparison testing or back-to-back testing • Comparison testing is not foolproof • If the specification from which all versions have been developed is in error, all versions will likely reflect the error • In addition, if each of the independent versions produces identical but incorrect results, condition testing will fail to detect the error Prepared by, Dr.T.Thendral, Assistant Professor, SRCW
- 48. Orthogonal Array Testing • Orthogonal array testing can be applied to problems in which the input domain is relatively small but too large to accommodate exhaustive testing • Useful in finding errors associated with region faults—an error category associated with faulty logic within a software component Prepared by, Dr.T.Thendral, Assistant Professor, SRCW