Containerized End-2-End Testing - Agile Testing Meetup at Süddeutsche Zeitung Munich 19.01.2017

Containerized End-2-End Testing
+
,Tobias Schneck ConSol Software GmbH

Characteristics of End-2-End Testing
Different types of testing:
Regression tests
Functional approval tests
Parallel tests with GUIs are complex
Stateful tests: user logins, sessions, history
Setup and cleanup of test data
Manual effort > effort for test automation

Advantages of Container Technology
Isolation of environments
Repository for versioning and distribution
Reproducible application environment
Dockerfile, docker-compose.yml
Optimized for parallel execution and cloud systems
Less memory and CPU overhead (shared Linux kernel)
Starting containers on-the-fly
Unique command line interface (orchestration tools)

Containerized GUIs
### start the docker container via x-forwarding
docker run -it -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix:rw rasch/inkscape
### start the docker container with VNC interface
# connect via URL: http://localhost:6911/vnc_auto.html?password=vncpassword
docker run -it -p 5911:5901 -p 6911:6901 consol/ubuntu-xfce-vnc
docker run -it -p 5912:5901 -p 6912:6901 consol/centos-xfce-vnc
docker run -it -p 5913:5901 -p 6913:6901 consol/ubuntu-icewm-vnc:dev
docker run -it -p 5913:5901 -p 6914:6901 consol/centos-icewm-vnc:dev

What's provided by ?
Category
Web tests through HTML selectors
Restricted to browser content
Open Source & Java API
Headless execution
Test writing assistance
(recorder, tag finder)
Automatable / Test result reporting
(CI, DB, monitoring environment)

Monitoring Integration
Nagios OMD Incinga Check_MK

Test Case Structure
// testcase.js
/*************************************
* Initialization
*************************************/
_dynamicInclude($includeFolder);
var testCase = new TestCase( 60, 70);
var env = new Environment();
var appNotepad = new Application( "gedit");
var region = new Region();
/******************************
* Description of the test case
******************************/
try {
//...
/************************************************
* Exception handling and shutdown of test case
**********************************************/
} catch (e) {
testCase.handleException(e);
} finally {
testCase.saveResult();
}

Call all Sahi Functions for Web Testing
// testacase.js
/************************
* Call Sahi Functions
***********************/
_navigateTo( "http://sahi.example.com/_s_/dyn/Driver_initialized" );
_highlight(_link( "SSL Manager" ));
_isVisible(_link( "SSL Manager" ));
_highlight(_link( "Logs"))
_click(_link( "Logs"))
testCase.endOfStep( "Test Sahi landing page" , 5);

Fluent API for UI Testing
// testacase.js
/*** open calculator app ***/
var calculatorApp = new Application( "galculator" ).open();
testCase.endOfStep( "Open Calculator" , 3);
/*** calculate 525 + 100 ***/
var calculatorRegion = calculatorApp.getRegion();
calculatorRegion.type( "525")
.find( "plus.png" )
.click()
.type( "100");
calcRegion.find( "result.png" ).click();
calcRegion.waitForImage( "625", 5);
testCase.endOfStep( "calculate 525 +100" , 20);

Custom Functions
// e.g. excluded into a separate common.js
/**********
* Combine click and highlight
*********/
function clickHighlight ($selector) {
_highlight($selector);
_click($selector);
}
/***************
* Open PDF in native viewer
**************/
var PDF_EDITOR_NAME = "masterpdfeditor3" ;
function openPdfFile (pdfFileLocation ) {
return new Application(PDF_EDITOR_NAME + ' "' + pdfFileLocation + '"').open();
}

Test Definition (Java)
public class FirstExampleTest extends AbstractSakuliTest {
private static final String CITRUS_URL = "http://www.citrusframework.org/" ;
private Environment env;
private Region screen;
@BeforeClass
@Override
public void initTC() throws Exception {
super.initTC();
env = new Environment();
screen = new Region();
browser.open();
}
@Override
protected TestCaseInitParameter getTestCaseInitParameter () throws Exception {
return new TestCaseInitParameter( "test_citrus" );
}
@Test
public void testMyApp() throws Exception {
// ... testcode
}
}
For the Maven dependencies, take a look at the .Java DSL Documentaion

Test Definition (Java)
public class FirstExampleTest extends AbstractSakuliTest {
// ... initializing code
@Test
public void testCitrusHtmlContent () throws Exception {
browser.navigateTo(CITRUS_URL);
ElementStub heading1 = browser.paragraph( "Citrus Integration Testing" );
heading1.highlight();
assertTrue(heading1.isVisible());
ElementStub download = browser.link( "/Download v.*/" );
download.highlight();
assertTrue(download.isVisible());
download.click();
ElementStub downloadLink = browser.cell( "2.6.1");
downloadLink.highlight();
assertTrue(downloadLink.isVisible());
}
@Test
public void testCitrusPictures () throws Exception {
browser.navigateTo(CITRUS_URL);
env.setSimilarity( 0.8);
screen.find( "citrus_logo.png" ).highlight();
env.type(Key.END);
screen.find( "consol_logo.png" ).highlight();
}
}

Sakuli End-2-End Testing Container

Demo - Sakuli Container
# start the docker container
docker run -it -p 5911:5901 -p 6911:6901 consol/sakuli-ubuntu-xfce
docker run -it -p 5912:5901 -p 6912:6901 consol/sakuli-centos-xfce
docker run -it -p 5913:5901 -p 6913:6901 consol/sakuli-ubuntu-xfce-java
docker run -it -p 5914:5901 -p 6914:6901 consol/sakuli-centos-xfce-java
# start in parallel via docker-compose
# use docker-compos.yml from https://github.com/ConSol/sakuli/tree/master/docker
docker-compose up

Bakery Demo
git clone https://github.com/toschneck/sakuli-example-bakery-testing.git
# start jenkins
jenkins/deploy_jenkins.sh
# start OMD montioring
omd-nagios/deploy_omd.sh
# start the build of the application images
bakery-app/app-deployment-docker-compose/deploy_app.sh
#start tests
sakuli-tests/execute_all.sh
#start tests for monitoring
sakuli-tests/execute_all_4_monitoring.sh

What's next?
Headless execution - Linux: VNC & Docker Windows: ?
Cloud-ready container forOpenShift (security, source-2-image)
Video recording of the test execution (error documentation)
Web UI to handle Sakuli test suites
Connect 3rd-party test management tools(HP QC, TestRail, ...)
Improve test result presentation in CI tools
Selenium as an alternative to Sahi
Implement Junit 5 test runner

Links
ConSol/sakuli
ConSol/sakuli-examples
toschneck/sakuli-example-bakery-testing
sakuli@consol.de @sakuli_e2e

Thank you!
Tobias Schneck
ConSol Software GmbH
Franziskanerstraße 38
D-81669 München
Tel: +49-89-45841-100
Fax: +49-89-45841-111
tobias.schneck@consol.
toschneck
info@consol.de
www.consol.de
ConSol

Containerized End-2-End Testing - Agile Testing Meetup at Süddeutsche Zeitung Munich 19.01.2017

More Related Content

What's hot (20)

Similar to Containerized End-2-End Testing - Agile Testing Meetup at Süddeutsche Zeitung Munich 19.01.2017 (20)

More from Tobias Schneck (20)

Recently uploaded (20)

Containerized End-2-End Testing - Agile Testing Meetup at Süddeutsche Zeitung Munich 19.01.2017