Get a Handle on Your Test Data—Starting Now

T9

Test
Data

10/6/16
11:15

Get
a
Handle
on
Your
Test
Data-‐Starting

Now

Presented
by:

Paul
Merrill

Beaufort
Fairmont

Brought
to
you
by:

350
Corporate
Way,
Suite
400,
Orange
Park,
FL
32073

888-‐-‐-‐268-‐-‐-‐8770
·∙·∙
904-‐-‐-‐278-‐-‐-‐0524
-‐
info@techwell.com
-‐
http://www.starwest.techwell.com/

Paul
Merrill

Paul
Merrill
is
principal
software
engineer
in
test
and
founder
Beaufort
Fairmont

Automated
Testing
Services.
Paul
works
with
clients
every
day
to
reduce
risk
to

their
brands,
customers,
and
reputation,
and
to
increase
the
efficacy
of
their
testing

processes.
He
takes
personally
the
company
missionÛÓÛÏto
rid
the
world
of
bad

code.Û_
An
entrepreneur,
tester,
and
software
engineer,
Paul
has
a
unique

perspective
on
launching
and
maintaining
quality
products.
He
co-‐hosts
Reflection

as
a
Service,
a
podcast
about
software
development,
testing,
and
entrepreneurship.

Follow
Paul
on
Twitter
@dpaulmerrill.

9/21/16
1
www.beaufor0airmont.com 984.244.2313 © 2016 Beaufort Fairmont, LLC All rights reserved
Why do we care about the data used by the
system under test?
Data Strategies
PAUL MERRILL
@dpaulmerrill

9/21/16
2
Agenda
!   Deﬁne Terms
!   Iden9fy Constraints
!   Walk Through Examples
Deﬁne Terms

9/21/16
3
Define Terms
Data Strategy - the combina9on of code,
procedure and infrastructure that affect how
tests interact with data to s9mulate the System
Under Test (SUT).
Define Terms
Constraint - a characteris9c that applies a force in
a way that is limi9ng to an automated test or its
environment.

9/21/16
4
Deﬁne Terms
Constraints factor down to:
Policy
Cost
Infrastructure
People
Iden9fy Constraints

9/21/16
5
Examples of Constraints
Shared Environments!
Downtime!
Technical Skills!
Data Source Growth!
Repeatable Tests!
Data Variance!
Example Data Strategies

9/21/16
6
Strategies Have 2 Parts:
!   Crea9onal Strategy - how and when data is
created.
!   Cleanup Strategy - the method by which a data
source is returned to a benign state.
Elementary Approach
!   Crea9on - Tests assume speciﬁc data in SUT -
no crea9onal strategy
!   Clean up - No clean-up strategy, no
management of data

9/21/16
7
Elementary Approach
SUT
ExecuSon 1
ExecuSon 2
ExecuSon 3
Data
Source
Elementary Approach

Pros:
!   Simple
!   No implementa9on
(quick)

Cons:
!   Tests are not repeatable
!   No isola9on of data
!   Does not work on
diﬀerent systems

9/21/16
8
Refresh Data Source Approach
!   Crea9on - Tests assume speciﬁc data in SUT
!   Cleanup - Data managed by refreshing the data
source prior to test execu9on
SUT
ExecuSon 1
ExecuSon 2
ExecuSon 3

9/21/16
9
Pros:
!   Clean test bed (data-
wise) per test
!   Uniform, one-size-fits-all
process
!   No variance of data
!   Less 9me/cost to
implement
!   Total control of data
Cons:
!   Timely
!   Requires Down9me
!   Blocks shared
environments
!   No variance of data
!   Stale data over 9me
!   Doesn’t work on
different systems
Selfish Data Genera9on
!   Crea9on - Tests create the data they need in
SUT
!   Clean up - No management of data aYer
genera9on
!   May be random (e.g. via Faker)
!   Batch process or in test
!   Via SUT or directly in Data Source
!   In-test or pre-test

9/21/16
10
SUT
ExecuSon 1
ExecuSon 2
ExecuSon 3
Pros:
!   Unique data per test
!   Does not block shared
environments
!   No down9me
!   No stale data
!   Tests can run anywhere
!   Tests become repeatable
Cons:
!   Unchecked growth of data
source
!   Requires understanding of
data model
!   Tests need to know about
data

9/21/16
11
Data Genera9on & Batch Cleanup
!   Crea9on - Data generated for each test
!   Cleanup - Data cleaned up aYer test run
!   Clean up may happen in data source or via SUT
SUT
ExecuSon 1
ExecuSon 2
ExecuSon 3

9/21/16
12
Pros:
environments
!   Minimal down9me
!   No stale data
!   Growth of data source kept
in check
!   Tests run anywhere
!   Tests are repeatable
Cons:
!   Requires deep
understanding of data
model
!   Must know what to clean
!   Signiﬁcant scaﬀolding must
be created

Data Genera9on & Immediate Cleanup
!   Crea9on - tests generate their own data
!   Clean up - for each test case immediately aYer
execu9on
!   Uses a TearDown method supplied by the
framework

9/21/16
13
SUT
ExecuSon 1
ExecuSon 2
ExecuSon 3
Pros:
!   Tests are repeatable
environments
!   Minimal down9me
!   No stale data
!   Growth of data source kept in
check
!   No data leY over from other tests
!   Tests run anywhere
Cons:
!   Requires deep understanding of
data model
!   Must know what to clean
!   Signiﬁcant scaﬀolding must be
created

9/21/16
14
Crea9onal Strategies
Elementary! Refresh! Generation!
Shared Env!
Downtime!
Repeatability!
Variance!
Data Growth!
Skills!
Cleanup Strategies
Note: Elementary has no clean-up strategy.
Refresh! Batch! Immediate!
Shared Env!
Downtime!
Repeatability!
Skills!

9/21/16
15
How do we know what data to clean
up?
Deﬁne Terms
Signed Data - data modiﬁed
so as to be easily recognizable.

Username: “TestUser”
Email address: “automated@tes9ng.com”

9/21/16
16
Deﬁne Terms
Marked Data - data denoted
so as to be easily iden9ﬁable.

Addi9onal Field: “isTestData”
Created By User: “TestUser”
Persisted Data
!   Keep a list in memory or in datasource:
–  userIds:
• 12312
• 4322
• 23423
–  emailIds:
• 987
• 423423
• 234

9/21/16
17
In Review
!   Defined Terms
!   Iden9fied Constraints
!   Walked Through Examples
References
!   Icons used freely in accordance with thenounproject terms…
–  Broom Icon - Anna Weiss
–  Database Icon - Cees de Vries
–  Gavel Icon - Eugen Belyakoff
–  Wallet Icon - Margot Nadot
–  Flyover Icon - Vineet Kumar Thakur
–  Group Icon - Alexandra Coscovelnita
–  Signature Icon - Korawan.M
–  Flag Icon - lipi
–  Persisted Data Icon - Stefan Parnarov

9/21/16
18
www.beaufor0airmont.com 984.244.2313
Paul Merrill

My Podcast About SoYware &
Entrepreneurship:
Reﬂec9on As A Service

Wri9ng: beauformairmont.com/blog

Email: paul@beauformairmont.com
Twiner: @dpaulmerrill
Like us on Facebook!
Call us today - free consulta9on
984.244.2313

Get a Handle on Your Test Data—Starting Now

More Related Content

Similar to Get a Handle on Your Test Data—Starting Now (20)

More from TechWell (20)

Recently uploaded (20)

Get a Handle on Your Test Data—Starting Now