Leveraging selenium tests to generate more effective test cases.

Leveraging Selenium
recorded tests to generate
more effective tests

Author: Cu D. Nguyen, PhD
http://selab.fbk.eu/dnguyen/

Tuesday, January 8, 13

Selenium tests
• A Selenium test consists of a sequence of
events and their input parameters.
• Example:

Event

open /CuteNews/

clickAndWait link=Add News link=Add News

type name=title afternoon news

select id=selecsinglecat label=Hot

clickAndWait css=input[type="submit"]

clickAndWait link=Logout


Selenium tests (contd.)
• Often capture/replay specific test scenarios
(or paths)
• Stick with specific input values
• The user might have to copy/paste/
modify recorded tests and specify new
inputs
➜ often focus on some specific functionality
➜ doesn’t scale, costly !


We can do better with Selenium recored
tests!
• How:
1. combining test paths (scenarios) to
create a test machine (a map, as an
analogy) to generate more paths
2. fusing available test inputs to infer input
partitions, and generate new inputs from
that
3. combining new paths + new inputs to
create new tests


I. Inferring test machine


Inferring test machine from available
paths
• Simple example illustrating the value of
combining test paths
Available paths: 1. add, add
2. add, remove, checkout

1. add, add
Results by 2. add, remove, checkout
combining paths: 3. add, checkout
4. add, add, checkout
- + Buy +

that's easy!
Here we are testing a web page that has
want to buy!

only three buttons: add, remove, and
checkout add remove checkout


Inferring test machine (contd.)

• We can do even better: from existing test
paths, we infer a test machine (or in a
technical terminology: a ﬁnite state
machine, FSM)
• Example:
Available paths: Inferred machine:
1. add, checkout add

2. add, remove, checkout remove
add
checkout


Inferring test machine (contd.)

• From the test machine, we can generate
more test paths
• Example:
Inferred machine: Generated test paths
add 1. add, checkout
add remove
2. add, remove, checkout
checkout
4. add, add, add, checkout
5. add, remove, remove, checkout
6. add, remove, add, remove, checkout
7. …...

➜ many more test paths are generated!

2. Inferring test input
partitioning


Test input partitioning
• Input space for even a single variable/parameter is
huge
• Example: price of an object can be any non negative
number.
• Input space of a set of variables/parameters is even
much more larger
• In software testing, we often partition input space of
each input into equivalent classes
• Example: input of x is
classiﬁed into 2 classes:


Inferring test input partitions
• Test input partitions can be speciﬁed by domain experts
• Or inferred automatically (when recorded/logged traces
are available)
• concrete data are recored with events
• machine learning techniques can classify such data into
clusters, input partitions can then be inferred from
them
• Example: from all traces, at the event add(#item), the
parameter #item receives the following values:
{1,2,3,4,1,2,4,3,1}, we can infer that 1 ≤ #item ≤ 4.
Therefore, we can partition the domain of item into 3
classes: #item < 1; 1 ≤ #item ≤ 4; #item > 4


3. Putting them together
to generate more tests


Test generation
• Test machine can generate test paths, each
path is a sequence of events
• The input of each event parameter is
partitioned into classes
• Now, we can combine the two to generate
more test cases
• Use pairwise criteria to reduce the
number of input combination


Example
• Test sequence: add(#item), remove(#item)
• #item has 3 partitions: p1: #item < 1; p2:1
≤ #item ≤ 4; and p3: #item > 4
add remove

p1 p2 p3 p1 p2 p3

• Generated new tests: add(p1), remove(p1)
add(p1), remove(p2)
add(p1), remove(p3)
add(p2), remove(p1)
……………...
Concrete values of #item can be randomly generated based
on the partition information, example #item = 5 in p3

Automated tools


Selen2FSM & M[agi]C
• Available at http://selab.fbk.eu/magic
• Selen2FSM can infer test machines (or
FSMs), and input partitions from existing
Selenium tests
• M[agi]C takes the FSMs and input partitions
and generate executable test cases
• In Selenium Webdriver
• And Robotium for Android applications

More info & support
• Selen2FSM & M[agi]C are developed under the
support of the FITTEST project
• https://www.facebook.com/FITTESTproject
• More info
• Currently we support Selenium for web
testing, and Robotium for Android application
• However, the general idea can be applied for
other type of functional testing as well.
• For support, contact cunduy AT fbk DOT eu


Leveraging selenium tests to generate more effective test cases.

More Related Content

Viewers also liked (8)

Similar to Leveraging selenium tests to generate more effective test cases. (20)

Recently uploaded (20)

Leveraging selenium tests to generate more effective test cases.