Automation through the Back Door

W3
Test Automation
5/7/2014 11:30:00 AM
Automation through the Back
Door
Presented by:
Seretta Gamba
Steria Mummert ISS GmbH
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com

Seretta Gamba
Seretta Gamba has more than thirty years of experience in software development. As test
manager at Steria Mummert ISS GmbH, Seretta was charged with improving the test
automation process. After studying the current strategies, she developed a style of keyword-
driven testing and a framework to support it. In 2009, the framework was extended to support
manual testing. Speaking about this at EuroSTAR, Seretta got the attention of Dorothy Graham
who subsequently invited her to contribute a chapter to Dot’s book Experiences of Test
Automation. After reading the entire book, Seretta noticed recurring patterns in solving
automation problems and is currently intent on cataloguing test automation patterns.

26.04.2014
1
www.steria-mummert-iss.de
W3 - Automation Through the Back Door
Seretta Gamba
Steria Mummert ISS Presentation.pptx
© Steria Mummert ISS GmbH
Agenda
26.04.2014 2
The Problem
The “Back Door” Idea
The Implementation
Conclusion and general
discussion

26.04.2014
2
The Problem
26.04.2014 3
The company
The application
The developer team
The Company
26.04.2014 4
Steria Mummert Consulting is one of
the top ten IT companies in Germany
(Counseling, IT-Service). In Germany
and Austria about 1700 employees.
Mummert ISS GmbH was established 2002 as
a 100% subsidiary of Mummert Consulting.
Develops standard software for insurance
companies. About 70 employees.
The Steria Group is one of the
top ten IT-Service companies in
Europe with worldwide about
20000 employees.

26.04.2014
3
The Application
26.04.2014 5
The product is:
a complete software solution for insurance companies
a standard software, that enables flexible management of
the products and processes of an insurance company -
both partner (customer) and portfolio can be easily
maintained
System design:
Completely newly developed standard solution
Customer oriented application structure
The Developer Team
26.04.2014 6
Test
Automation
Team
Product
Team
New Customer
New Customer
New Customer
X
Old Customer
Old Customer
Old Customer

26.04.2014
4
The „Back Door“ Idea
26.04.2014 7
26.04.2014 8
What do we need in order to be able to
automate?
Knowledge of the application under test
Test specifications
Automation strategy
Test Tools

26.04.2014
5
26.04.2014 9
What can manual testing deliver?
Test specifications
Automation strategy
Test Tools
26.04.2014 10
automate?
Test specifications
Automation strategy
Test Tools

26.04.2014
6
Our Automation Strategy
26.04.2014 11
Our Command-Driven framework supports the test
automation team by (mostly) eliminating the need to
program new keywords and by automating some of the
steps
Prerequisites:
Test cases Registration of GUI ControlsCommand Library
26.04.2014 12
Step 1
Capture
Tool
Script
Step 2
Tool
Script
ISS
ISS Test Station
Driver
Data-
Template
Step 3
Driver
Data Suite
Preparation:

26.04.2014
7
26.04.2014 13
Execution
Test suite Capture-Replay Tool Test results
26.04.2014 14
automate?
Can manual testing deliver it?
Can we convince our colleagues to switch over to the
“automation” framework for manual testing?
Can we extract and use the collected Information for
automation?

26.04.2014
8
Supporting manual testing
26.04.2014 15
In order to support manual testing with the automation
framework, we had to find out how testers worked and
devise better support.
The original testing procedure was based on excel-test-
sheets, that would be performed again and again for
regression testing.
Weaknesses of manual testing
26.04.2014 16
Test case
maintenance
• Corrections or
improvements
to the test
template can
get lost
• Test-
specifications
are very
“concise”
Defect
reporting
• Cumbersome
defect
reporting
• Test cases
tend to be
repeated in
order to report
the defects
exactly
Summary
reporting
• No automatic
statistic of the
executed test
cases versus
the fix status
of the found
defects

26.04.2014
9
Supporting manual testing
26.04.2014 17
Many of the available features in the
automation framework could, with no
change, be employed to support manual
testing.
Add or change test cases.
Overview of the available test cases.
Logging of test results.
Result reports or overviews.
Prioritization of the test cases.
Defect reporting.
Availablefeatures
Corrections or
improvements to the
test template can get
lost.
No automatic statistic of
the executed test cases
versus the fix status of
defects found.
Cumbersome defect
reporting.
Test cases tend to be
repeated in order to report
the defects exactly.
Needed for supporting manual testing
26.04.2014 18
Support for manual test
execution
• Test driven by
sequential display of
the selected test cases
• Paperless execution
• Support in:
- Performing tests.
- Maintaining test
cases.
- Creating defect items.
Import test-template
sheets
• Import from the current
excel sheets (csv-files)
to the test suites in the
framework format
• Import functionality
driven by an external
table.

26.04.2014
10
Needed for supporting manual testing
26.04.2014 19
Detail test execution for
developers
• Capture-facility
integrated into the
framework.
• Generation of a capture-
replay script with
simultaneous recording.
• Starts or stops by
pushing a button.
• Recording automatically
attached to defect
tracking item
Save application status to
file
• Integration of SQL-scripts
to extract data from the
database.
The Implementation
26.04.2014 20
Manual testing with the automation
framework
Automating the manual tests

26.04.2014
11
Manual test execution using the test automation
framework
26.04.2014 21
Set preconditions
and export them to file
Start recording
Perform test case
Export post-
conditions to file
Check
test
case
display
test case
OK
Defect
Start test suite
„Harvesting“ Manual Testing
26.04.2014 22
Test cases
After the testers started working with the framework,
creating more automated tests has been much more
rewarding because some steps have already been
performed
Tool script

26.04.2014
12
Automating the manual tests
26.04.2014 23
Step 1
• Call the
framework
functionality
to transform
the
captured
tool-scripts
into
Command-
Driven
scripts
Step 2
• Rework the
command-
Driven
scripts
Step 3
• Create the
new test
procedures
Step 4
• Insert the
test
procedures
to perform
the test
cases in the
test suite
structures
Conclusion
26.04.2014 24
Short summary of our experiences
Next steps
General discussion

26.04.2014
13
Short Summary of our experiences
26.04.2014 25
After some time to allow the testers to get acquainted
with the new process, we started to get the first results.
The testers had a hard time getting used to the new
formats.
The import facility, test-sheet test suite was well
accepted from the very beginning.
The recording feature was very well received by both
testers and developers.
In the beginning the testers were recording either too
much or too little.
Next Steps
26.04.2014 26
and they lived happily ever after. Well, we still have lots
of effort and work to do.
Partially automated
tests.
Sophisticated metrics.
Integration with test
planning and
management tools .

26.04.2014
14
Conclusion
26.04.2014 27
"If you can't get in the front
door, try the back door".
We couldn't do automation
straight on, but found a way
to do it indirectly by
supporting the testers
where they were.

Automation through the Back Door

More Related Content

Viewers also liked (10)

Similar to Automation through the Back Door (20)

More from TechWell (20)

Recently uploaded (20)

Automation through the Back Door