BDD/TDD based automation on short-term project on real example, Igor Kokoz

BDD/TDD BASED AUTOMATION ON
SHORT-TERM PROJECT ON REAL
EXAMPLE

TERMS & DEFINITIONS
TDD
BDD
KPI
ROI

INITIAL PROJECT INFO
 BUSINNES AREA: Provision of services
 PRODUCT CATEGORY: Internet resource
 TECH STACK:
 .Net platform
 C#, JavaScript, HTML5
 Project duration:
 1st phase – 2 Months
 2nd phase – 4 Months

INITIAL ESTIMATIONS
 Regression, localization and cross-browser testing
activities:
150 – 180 hours
 TA implementation (basing on project backlog):
 1st phase – up to 40 hours
 2nd phase – up to 130 hours
 Expected ROI: 0.8 – 1.2

RISKS
 NO TIME FOR AUTOMATION TO GET PROFIT:
 Only two regression rounds
 Cross-browser testing could be limited to 1 round
 Number of browsers/devices could be decreased
 AUTOMATION COMPLEXITY:
 Test flows and scenarios could be changed
 UI of the system can be changed
 Some flows could be too complex to automate

AUTOMATION GOALS (INDIRECT BENEFITS)
 Fast feedback on new features
 New tests for automated areas/features
 No need in manual regression
 Continuous quality control
 Adding new browsers/devices to test
scope will take 1-3 hours

PROCESS APPROACHES
 Automate acceptance flows
 TDD/BDD usage:
 Acceptance test flows are defined before start of
feature development
 Automated tests are implemented in parallel with
feature development
 Automated tests are executed just after feature
delivery to test staging
 Passed automated tests are part of DoD for
feature development
 Same process for new features, bug fixes and
CRs
 Tests are integrated to CI

PROCESS APPROACHES
 Tests are executed in CI at least once a day
 Test Automation engineer integration to project:
 Daily meetings
 Involved only for short term automation tasks
during sprint
 Tasks for each sprint are processed as small
fixed price projects
 Main TA KPI: time to get test results after feature
deliver on staging
 Secondary KPI: ROI calculation on each sprint

TECH APPROACHES
SpecFlow based framework:
 Layered structure
 Gherkin for top level test logic description (KDD/BDD)
 Examples used for DDT
 WebDriver for UI interactions
Framework

FRAMEWORK STRUCTURE
FEATURES LAYER
STEPS LAYER
PAGE OBJECTS LAYER SERVICES LAYER MODELS
WEB DRIVER SERVER API
Layers

FRAMEWORK STRUCTURE
 Test cases are written as a user stories describing real
user action flow.
 Test scenarios written on Gherkin language and using
Given, When, Then notation.
Features layer
Given [Some Context]
When [Some Actions is performed]
Then [This should be the result]

FRAMEWORK STRUCTURE
 Test steps represent the level of abstraction between
your Page Objects and higher-level stories
 Divide test implementation from test logic
 DRY
Steps layer

FRAMEWORK STRUCTURE
 PageObject pattern models UI areas as objects within
the test code
 UI area logic is divided from test implementation logic
 DRY
 Driver usage is encapsulated
Page Objects Layer

FRAMEWORK STRUCTURE
 HTTP requests as objects within the test code
 Requests logic is divided from test implementation logic
 DRY
 HTTP request usage is encapsulated
Services layer

TECH APPROACHES
 SpecFlow Context Injection
 Upside down development:
 Start from Features level (steps unimplemented or
mocked)
 Steps implementation without real PageObject (PO and
it’s methods are mocked)
 Page object is last one to be implemented with known
API
 IDE method generation usage
Dev practices

TECH APPROACHES
Dev practices (example)

TA RESULTS
 TEST TYPES COVERED:
 Acceptance testing
 Integration testing
 Regression testing
 PRODUCT INTERFACE INTERACTED:
 Web UI – Test scenario execution,
Test data generation
 Web API – Test scenario execution
 Mobile API – Test scenario execution

TA RESULTS
QUANTITATIVE INDICATORS:
 Total time spent: 95 hours
 Time for scenarios implementation: 65 hours
 Time for tests support: 17 hours
 Framework configuration: 4 hours
 Demo, planning: 9 hours
 Scenarios implemented: 45 scenarios with 145 different
tests
 Average time for 1 scenario: 1,3 hour
 About 300 test runs on CI server

TA RESULTS
 ROI:
 ROI of phase one: about 1.1
 ROI of phase two: about 2
 OTHER BENEFITS:
 Running tests 24/7
 Fast feedback (0,5 – 1 hour after feature was
implemented)
 Continuity
 Speed (run time for all regression tests takes 26
minutes for 145 tests)
 Test data generation

RESULTS
 Successful automation even with occurred
risks
 Product was successfully delivered to
Customer

THANKS FOR YOUR ATTENTION!
Useful books:

BDD/TDD based automation on short-term project on real example, Igor Kokoz

More Related Content

What's hot (20)

Similar to BDD/TDD based automation on short-term project on real example, Igor Kokoz (20)

More from Sigma Software (20)

Recently uploaded (20)

BDD/TDD based automation on short-term project on real example, Igor Kokoz