Stf 2019 workshop - enhanced test automation for web and desktop apps

© 2019 Maveryx srl.
All rights reserved.
Enhanced test automation
for Web and Desktop apps
Software Testing Forum 2019
Alfonso Nocella

Maveryx is the Italian testing automation company that
provides one of the most advanced testing automation
solution worldwide.
Introduction
Today
• 14 Released Versions
• 25K Downloads
• 40K Contacts
2009
Open
Source
2010
1° Release
2012
Become a
Company
2013
ATI
Innovation
Award
2015
Trading
with
World
Leader
Company

• Creating and maintaining test artefacts (maps, objects
repositories, recordings, etc…): huge cost in terms of
time and effort;
• Instrumenting the AUT (Application Under Test) code
• AUTs in different technologies need different tools
Challenges (1)

Does automation necessarily be complicated and expensive?
The Application Under
Test is required to start
developing tests
Start late, finish later!
50%
Time
• Acquisition €2K-16K
• Know-how €10K-25K
• Maintenance cost
unpredictable
Costs reduction is
hard to achieve
Cost
Automation does not
replace completely
manual testing
Useless activities
instead of increasing
functional coverage
Quality
Challenges (2)

The Idea
A technology able to operate as a person
– no artefact
– no code instrumentation
– no programming skills.
A technology able to test AUTs from design
– no matter the development technology
– no matter the execution environment

Robot
System Under Observation
Viewer
ModelController
Decision
Support
System
Data
Commands
The Technology

The Maveryx Test Automation Framework
innovates automated testing for:
 Functional Testing
 Regression Testing
 Keyword-Driven Testing
 Data-Driven Testing
 Continuous Integration
The Framework

 No GUI Maps or Object Repositories
 No pre-recording or UI element capture
 No AUT code instrumentation
 No programming skills needed
 Recognition of objects in the User Interface by Images
Inspection «On-The-Fly»

 Inspection & recognition at runtime
 One script to test them all
 Testers will be testers
 Multi-Platform
The Framework is an expert tool operating as a senior human tester
So innovative ?!

A Case Study
Functional
Requirements
Prototype Beta GA
1 9 18 27 36
Beta GA
45
Design
Development
Test Execution
Test Development
Design
Development
Test Record
Test Development
Test Execution
With Maveryx
Without

Test automation runs in
parallel with software
development
Short time-to-market:
early start, earlier finish!
Time
Accuracy improved by:
• test objects recognition
• accommodating
changes
• error recovering
Tests resilient to
frequent changes
Quality
• Std. technologies
• Easy to learn & use
• No maintenance cost
Reduced all the
automation costs
Cost
Challanges won

 Your team can save a lot of
time that can be spent to
increase test coverage
 You can cut effort & costs of
test creation & maintenance,
releasing earlier
 Everyone in your team can
play a significant role
Testers
 No code, or learn and use
complex XPath Locators, or
Matchers, etc…
 No effort to capture and
maintain any Object
Repository, GUI Map, etc…
 Java & C# scripting, or
Keywords & Blocks for non
programmers
Managers

• Taking advantage of the coding skills;
• Using constructs, statements, etc…
• Using the OOP features (e.g. inheritance,
polymorphism, recursion, etc… );
• Using design patterns;
Test Scripting

Scripting: Java Example

Scripting: C# Example

Scriptless Testing
• No programming skills are needed;
• Easy to learn and use;
• Promotes an improved functional coverage;
• Favours the participation of all the stakeholders;

Test Case :
1. Start the Application
2. Enter valid username
3. Enter valid Password
4. Click “Login” button
5. Check the results: “logged in”
6. Click “OK" button
7. Close the Application
Scriptless: Excel Example

Test Case :
5. Check the label: “logged in”
Scriptless: Blockly Example

Data-Driven Testing allows writing the test
cases as scripts those read their data from
external files or db
One script to drive the tests and changing the
data you can create any number of test cases
Data-Driven Testing

Extension plugin mechanism & interfaces
to support custom controls
to add new keyword (action) libraries
API extension
to create new Test API
Add-ons

Create and run a keyword-driven test
1. Create a new Maveryx Test Project
2. Write the test case
3. Run the test

1. Select File → New → Maveryx Test Project
In the Maveryx Test Project window
1. enter the Project name (e.g.
"PasswordDemoTest")
2. in the JRE section make sure that
Java/JRE 8 or higher is selected
2. Click Finish
Create New Test Project

1. Select File → New → Maveryx Test
Class
In the Maveryx Test Class window
1. enter a name for the Package (e.g.
"com.maveryx.demo")
2. enter a Name for the test class /
script (e.g. “PasswordDemoTest”)
2. Click Finish
Create New Test Script

Set the full path (pathName) to
the AUT launch file.
e.g. private final String pathName =
"C:/Maveryx/demo/AUT/PasswordDemo.xml";
The static method
startApplication(pathName)
in class Bootstrap launches the AUT
The static method stop() in class
Bootstrap closes the AUT.
Test Script "stub"

To execute a Java Application-Under-Test it is necessary to create the
related AUT launch file.
Java AUT Launch File

To execute a MFC or .NET Application-Under-Test it is necessary
to create the related AUT launch file.
MFC & .Net AUT Launch File
Set the absolute or relative path to your AUT executable file

To execute a Web Application-Under-Test it is necessary to
create the related AUT launch file.
Web AUT Launch File
Set the path of the browser
you want to use for your tests
Set the URL of the AUT

The Sample AUT

Test Case : TC_01
5. Check the results: Hi "username”
Test Case #001

7. Close the AUT
5. Check the results
4. Click "Login" button
3. Enter Password
2. Enter username
1. Start the AUT
Close
Has_Text
Click
Set_Text
Start
Keywords
Test Case
Identify Keywords

Test Case : TC_01
5. Check the results: “Hi
alfonso”
Keyword Data / Input Description
START AUT lauch file path Launch the AUT
Design Test step 1

Test Case : TC_01
alfonso”
Keyword Data /
Input
Description
SET_TEXT Text Set the text into a text field
Design Test step 2 & 3

Test Case : TC_01
alfonso”
Keyword Data /
Input
Description
CLICK Click the button
Design Test step 4

Test Case : TC_01
alfonso”
7. Close the Application Keyword Data /
Input
Description
HAS_TEXT Text Check whether the given
LABEL has the actual Text
Design Test step 5

Test Case : TC_01
5. Check the results:
“logged in”
Keyword Data /
Input
Description
CLICK Click the button
Design Test step 6

Test Case : TC_01
5. Check the results:
“logged in”
Keyword Data / Input Description
CLOSE Close the AUT
Design Test step 7

a. Create a new object
KeywordDrivenTestManager
b. Call the method run()
specifying the path of the
Excel file
Create the Driver Script

1. Click on the test class or package you
want to run
2. Select File → Run As → JUnit Test
Run a Test Script

Alternatively
By command line KeywordDrivenTestingCLI with the following arguments:
For example:
KeywordDrivenTestingCLI –o "C:Report" "C:Testtest001.xls"
The test script C:Testtest001.xls is executed and the test report is stored
in C:Report
Or:
KeywordDrivenTestingCLI –o "C:Report" –e
"C:Testtest001.xls " "C:Test"
All test scripts in C:Test (including subfolders) are executed except
C:Testtest001.xls and the test report is stored in C:Report

5. Check the label
4. Click "Login" button
3. Enter Password
1. Enter username
Assert
Button
Text
BlocksTest Case
Identify Blocks
Label
SetText
Action
Click
GetText

Test Case : TC_01
4. Check label: “Hi Admin”
Design Test step 2 & 3

Test Case : TC_01
Design Test step 4

Test Case : TC_01
Design Test step 5

Test Case : TC_01
Design Test step 6

Customer Case

The team develops Mission Computing Software for
an airborne command and control platform.
• This project was the major mid-life upgrade and the
team wanted to introduce automated testing.
• The challenge was to find a tool that it could
integrate with a large existing codebase quickly and
efficiently, that would provide a robust framework
for the testing of additional new features.
Boeing Defence Australia

“Maveryx’s unique innovative technology made the tool selection
task less difficult. Their technology, as opposed to other
established GUI automated test tools, promotes efficiency for a
start from scratch test automation solution on a legacy
application by not having to spend a long time developing an
object repository.”
BDA
Paul D. Ellis, Principal Software Engineer at

Nightly GUI automated test runs:
– Unit testing;
– Build Verification Testing;
Weekly collaborative automated test runs.
The code-base is instrumented for memory error
detection and code coverage.
Maveryx is used to drive the execution paths of the
instrumented executables and libraries.
DBA Environment

Thank you

Stf 2019 workshop - enhanced test automation for web and desktop apps

More Related Content

What's hot (20)

Similar to Stf 2019 workshop - enhanced test automation for web and desktop apps (20)

Recently uploaded (20)

Stf 2019 workshop - enhanced test automation for web and desktop apps