Pairwise Testing

E Pluribus Unum
or
How to use Pairwise Testing
VLADIMIR ARUTIN

What is the purpose of test design techniques ?
Boundary Value Analysis
Equivalence Class Partitioning
Decision Table testing
Cause-Effect Graphing Technique
Error Guessing
Pairwise testing
State transition testing
Use case testing
User story testing
Domain analysis
Syntax testing
Combining technique
Exhaustive Testing

THE MAIN PURPOSE IS
TO ACHIEVE AS MUCH COVERAGE
AS POSSIBLE
USING AS LESS TESTS
AS POSSIBLE

You can split all checkboxes
into groups

What if there is a dependency
between сheckboxes?

LOAN TYPE COMMISSION
CASH LOAN 500
CREDIT CARD 1000
4 tests
100% coverage
10 minutes
Example#2 Bank Loan

But Life Isn’t Easy…
LOAN TYPE Limit ROI COMMISSION M Class Outstanding
CASH LOAN 10000 10 500 6 A 1
CREDIT CARD 20000 30 1000 9 B 2
50000 50 12 C 3
100000 18 D 4
120000 24 …
150000 57
190000 58
200000 59
60

How many tests do we need to achieve
minimum coverage?

OUR CUSTOMER FOUND A BUG
If Class is D and Period is 24 months
Bank approves 200 000 loan amount
BUT should REFUSE!
AND WE HAVE 40% OF SUCH CLIENTS

LOAN TYPE Limit ROI COMMISSION M Class Outstanding
CASH LOAN 10000 10 500 6 A 1
CREDIT CARD 20000 30 1000 9 B 2
50000 50 12 C 3
100000 18 D 4
120000 24 …
150000 57
190000 58
200000 59
60
How many tests will be failed?
5760 tests

How many test inputs are needed to
trigger defects in production?
84%
11%
4%
1%
1 input or 2 inputs (pairwise)
3 inputs
4 inputs
5 or 6 inputs

Let’s optimize our data
• Combine them into Equivalence Classes
• Use Boundary Values
LOAN TYPE Limit ROI COMMISSION Y Class Outstanding
CASH LOAN 10000 10 500 <1 A 1
CREDIT CARD 20000 30 1000 1 B 2
190000 50 >1 C 59
200000 D 60

Combine Pairs
OUTSTANDING LIMIT CLASS PERIOD ROI LOAN TYPE COMMISSION
1 10000
1 20000
1 190000
1 200000
2 10000
2 20000
2 190000
2 200000
59 10000
59 20000
59 190000
59 200000
60 10000
60 20000
60 190000
60 200000

Combine Pairs
1 10000 A
1 20000 B
1 190000 C
1 200000 D
2 10000 B
2 20000 A
2 190000 D
2 200000 C
59 10000 D
59 20000 C
59 190000 A
59 200000 B
60 10000 C
60 20000 D
60 190000 B
60 200000 A

Combine Pairs
1 10000 A <1
1 20000 B 1
1 190000 C >1
1 200000 D <1
2 10000 B 1
2 20000 A >1
2 190000 D 1
2 200000 C <1
59 10000 D >1
59 20000 C <1
59 190000 A 1
59 200000 B >1
60 10000 C 1
60 20000 D >1
60 190000 B <1
60 200000 A 1

Combine Pairs
1 10000 A <1 30
1 20000 B 1 50
1 190000 C >1 50
1 200000 D <1 10
2 10000 B 1 10
2 20000 A >1 50
2 190000 D 1 50
2 200000 C <1 10
59 10000 D >1 50
59 20000 C <1 30
59 190000 A 1 10
59 200000 B >1
60 10000 C 1 10
60 20000 D >1 10
60 190000 B <1 30
60 200000 A 1 50

Combine Pairs
1 10000 A <1 30 CASH LOAN
1 20000 B 1 50 CREDIT CARD
1 190000 C >1 50
1 200000 D <1 10
2 10000 B 1 10 CASH LOAN
2 20000 A >1 50 CREDIT CARD
2 190000 D 1 50 CREDIT CARD
2 200000 C <1 10 CASH LOAN
59 10000 D >1 50 CREDIT CARD
59 20000 C <1 30
59 190000 A 1 10 CASH LOAN
59 200000 B >1 CREDIT CARD
60 10000 C 1 10 CREDIT CARD
60 20000 D >1 10 CASH LOAN
60 190000 B <1 30 CREDIT CARD
60 200000 A 1 50 CASH LOAN

Combine Pairs
1 10000 A <1 30 CASH LOAN 500
1 20000 B 1 50 CREDIT CARD 1000
1 190000 C >1 50 500
1 200000 D <1 10
2 10000 B 1 10 CASH LOAN 1000
2 20000 A >1 50 CREDIT CARD 500
2 190000 D 1 50 CREDIT CARD 500
2 200000 C <1 10 CASH LOAN 1000
59 10000 D >1 50 CREDIT CARD 1000
59 20000 C <1 30 1000
59 190000 A 1 10 CASH LOAN 1000
59 200000 B >1 CREDIT CARD 500
60 10000 C 1 10 CREDIT CARD 1000
60 20000 D >1 10 CASH LOAN
60 190000 B <1 30 CREDIT CARD 500
60 200000 A 1 50 CASH LOAN 1000

106 pairs, 16 tests, about 30 minutes
1 10000 A <1 30 CASH LOAN 500
1 20000 B 1 50 CREDIT CARD 1000
1 190000 C >1 50 500
1 200000 D <1 10
2 10000 B 1 10 CASH LOAN 1000
2 20000 A >1 50 CREDIT CARD 500
2 190000 D 1 50 CREDIT CARD 500
2 200000 C <1 10 CASH LOAN 1000
59 10000 D >1 50 CREDIT CARD 1000
59 20000 C <1 30 1000
59 190000 A 1 10 CASH LOAN 1000
59 200000 B >1 CREDIT CARD 500
60 10000 C 1 10 CREDIT CARD 1000
60 20000 D >1 10 CASH LOAN
60 190000 B <1 30 CREDIT CARD 500
60 200000 A 1 50 CASH LOAN 1000

Let’s add some conditions
• IF ROI < 50% THEN LOAN CAN’T BE 200000
• IF CLASS < B THEN PERIOD <1 YEAR
• IF CASH LOAN THEN COMMISSION=1000
1 10000 A <1 30 CASH LOAN 500
1 20000 B 1 50 CREDIT CARD 1000
1 190000 C >1 50 500
1 200000 D <1 10
2 10000 B 1 10 CASH LOAN 1000
2 20000 A >1 50 CREDIT CARD 500
2 190000 D 1 50 CREDIT CARD 500
2 200000 C <1 10 CASH LOAN 1000
59 10000 D >1 50 CREDIT CARD 1000
59 20000 C <1 30 1000
59 190000 A 1 10 CASH LOAN 1000
59 200000 B >1 CREDIT CARD 500
60 10000 C 1 10 CREDIT CARD 1000
60 20000 D >1 10 CASH LOAN
60 190000 B <1 30 CREDIT CARD 500
60 200000 A 1 50 CASH LOAN 1000
No, God,
please….

Pairwise Independent Combinatorial Testing
(PICT)
NOW MY LIFE WILL
NEVER BE THE
SAME…

REQUIREMENTS
COMPANY: Microsoft, Apple, Samsung, BlackBerry, Xiaomi, Sony
SCREEN: 4.5', 5', 5.2', 5.5', 6', 6.9‘
RESOLUTION: HD, FullHD, 4K
RAM: 2Gb, 3Gb, 4Gb, 6Gb, 8Gb
PROCESSOR MAKE: QS650, MSM8230, QS820, A10, A11
OS: Android, BlackBerry OS, iOS, Windows Phone
NET STANDART: CDMA, GSM, LTE, 3G
STORAGE: 16Gb, 32Gb, 64Gb, 128Gb, 512Gb
Example#3 Mobile Game

REQUIREMENTS
COMPANY: Microsoft, Apple, Samsung, BlackBerry, Xiaomi, Sony
SCREEN: 4.5', 5', 5.2', 5.5', 6', 6.9‘
RESOLUTION: HD, FullHD, 4K
RAM: 2Gb, 3Gb, 4Gb, 6Gb, 8Gb
PROCESSOR MAKE: QS650, MSM8230, QS820, A10, A11
OS: Android, BlackBerry OS, iOS, Windows Phone
NET STANDART: CDMA, GSM, LTE, 3G
STORAGE: 16Gb, 32Gb, 64Gb, 128Gb, 512Gb
Example#3 Mobile Game
(6)
(6)
(3)
(5)
(5)
(4)
(4)
(5)
MINIMUM TEST AMOUNT=6 x 6 x 3 x 5 x 5 x 4 x 4 x 5= 216000

CONDITIONS AND DEPENDENCIES
Can be used: =, <>, >, >=, <, <=, and LIKE
Operator IN The IF, THEN, and ELSE parts of a
predicate may contain multiple terms joined
by logical operators: NOT, AND, and OR.
IF [RAM] in {"2Gb", "3Gb", "4Gb"} OR ([COMPANY] =
"Xiaomi" AND [STORAGE] < "64Gb") THEN [PROCESSOR
MAKE] = "QS650“;
IF [Company] ="Apple" THEN [PROCESSOR
MAKE] IN {"A10", "A11"};
IF[Company] ="Samsung" THEN [SCREEN]= "5'";

Conclusion
• The number of bugs detected by pairwise testing is bigger than
found by checking all values for each parameter.
• Pairwise testing reduces the number of test cases needed
for system checking.
• It accelerates the testing process itself.
• It takes 50 percent less efforts.

Pairwise Testing

More Related Content

Recently uploaded (20)

Featured (20)

Pairwise Testing