Pairwise testing - Strategic test case design
6 likes7,379 views
Pairwise testing is a technique used to test software interactions between input parameters by selecting all possible unique pairs of parameter values. It aims to find bugs caused by interactions between two parameters, which are common, while being more efficient than testing all possible input combinations. Tools can generate pairwise test sets to cover all pairs, while testing far fewer cases than exhaustive testing. However, pairwise testing requires effective partitioning of input values and has limitations when dependencies between more than two parameters exist or inputs are not discrete. It works best when there are many possible input values that can be separated into equivalence classes.
Pairwise testing - Strategic test case design
- 1. Pairwise Tes*ng Strategic Test Case Design www.xboso6.com
- 2. • XBOSo& – Quality and Tes4ng Process Improvement and So&ware Tes4ng Experts – San Francisco, Beijing, Oslo, Amsterdam
- 3. Pairwise Tes*ng The Problem: • Many defects only occur when a combina4on of inputs or events occur that interact with each other. • But to test every combina4on or permuta4on is impossible. The Solu4on: • Pairwise tes4ng normally begins by selec4ng values for the system’s input variables. • These individual values are o&en selected using domain equivalence par44oning. • The values are then permuted to achieve coverage of all the pairings. 3
- 4. Pairwise Tes*ng • The reasoning behind all-‐pairs tes4ng: the simplest bugs in a program are generally triggered by a single input parameter. • The next simplest category of bugs consists of those dependent on interac4ons between pairs of parameters, which can be caught with all-‐pairs tes4ng. • Bugs involving interac4ons between three or more parameters are progressively less common, while at the same 4me being progressively more expensive to find by exhaus4ve tes4ng, which has as its limit the exhaus4ve tes4ng of all possible inputs. 4
- 5. Pairwise Tes*ng Problem: Test System S Inputs X = A,B Y = C,D Z = E,F There are 23 = 8 possible combina4ons 5 System S X Y Z
- 6. Pairwise Tes*ng 6 8 Test Cases You can get rid of Test Case 2 (T2) • {A, -, E} is covered in T1 • { , -} is covered in T4 • { -, } is covered in T6 Problem: • How many test cases can you get rid of? • List the Test cases you would keep. • Time yourself. Test ID X Y Z T1 A C E T2 A D E T3 A C F T4 A D F T5 B C E T6 B D E T7 B C F T8 B D F A, D D, E
- 7. Pairwise Tes*ng Test ID X Y Z T1 A C E T2 A D E T3 A C F T4 A D F T5 B C E T6 B D E T7 B C F T8 B D F 7 Test ID X Y Z T1 A C E T4 A D F T6 B D E T7 B C F 8 Test Cases Pairwise testing 4 Test Cases
- 8. Example 8 How many combinations are there for twelve inputs? - Hexawise.com
- 9. Pairwise Tes*ng 9 - Hexawise.com
- 10. Pairwise Tes*ng 10 - Hexawise.com
- 11. Pairwise Tes*ng 11
- 12. Pairwise Tes*ng Tools To generate the 20 test cases in the previous slide from the possible 3.7M test cases would be extremely 4me consuming. Just a few of the many SW tools to generate the tes4ng pairs. • Hexawise • AllPairs by James Bach, free • PICT (Pairwise Independent Combinatorial Tes4ng) tool • Automa4c Efficient Test Generator or AETG™ 12
- 13. Limita*ons of Pairwise Tes*ng Common tes*ng problems are made worse: • Selec*ng the wrong input: The problems of equivalence par44oning are made worse with a pair of wrong inputs. • The ‘Oracle Problem’: Is a defect properly revealed and do we recognized it when it is revealed? • The input values to the system under test are not discrete. Problems of Pairwise tes*ng: • High probability combina4ons get too lihle ahen4on. • Not knowing the interdependencies between variables. 13
- 14. Pairwise Tes*ng Works well when: • Number of possible parameter values is large • You can effec4vely separate parameter values into equivalence classes. 14
- 15. Pairwise Tes*ng: Cau*on • Pairwise tes4ng is a tool, it cannot replace a skilled experienced tester. • As a tool, it is effec4ve only as much as the tester is effec4ve. • Responsibility for good tes4ng remains with the tester, not the tool. 15
- 16. Pairwise Tes*ng Conclusion: • When used properly, pairwise test set genera4on is an important technique that can help you produce beher so&ware systems. 16