End to-end testing from rookie to pro

End-to-end testing: from rookie to pro
Domenico Gemoli - @aberonni

Overview.
● What is end-to-end testing?
● Setup a test suite with WebdriverIO
● WebdriverIO under the hood
● Augment WebdriverIO with 3rd
party tools
● Advanced techniques for
end-to-end testing

End-to-end testing is a technique used to test
whether the flow of an application, from start to
finish, is behaving as expected

● Fail fast and catch bugs before going to
production
● Increase “Developer Confidence” - code
fearlessly and let the tests tell you if something is
broken
● Maintain quality without forcing people to do
numerous rounds of manual testing

1. Push to Git server
2. Build commit in CI server (Travis/Jenkins/TeamCity etc…)
3. Deploy built code to a testing/staging server
4. Run end-to-end test suite on that server
5. Report test status to monitoring tools
Build Deploy Test ReportPush
Ideal CI workflow

Why WebdriverIO?
Written in Javascript
with an easy API to
learn for those with
experience in unit
testing
Based on the
WebDriver protocol
which means solid
drivers and easy-ish
cross-browser testing
Active community that
continuously improves
the framework and
ships new features

Setup a test suite
with WebdriverIO

Learn the basics:
http://webdriver.io/guide.html
Deep dive into the API:
http://webdriver.io/api.html

Prerequisites.
● Basic JS knowledge
● Node
● Java (optional, only for cross-browser testing)
● At least one browser installed

$ mkdir webdriverio-example
$ cd webdriverio-example
$ npm init -y
$ npm install webdriverio --save-dev
$ npx wdio config

End to-end testing from rookie to pro

const { assert } = require('chai');
describe('contact form', function() {
it('opens alert', function() {
// test script
});
});

// test script
browser.url('/');
$('input#email').setValue('charlie@gmail.com');
$('textarea#message').setValue('Hello World!');
$('input[type="submit"]').click();
assert.equal(
browser.alertText(),
`charlie@gmail.com says 'Hello World!'`
);

“This sounds like a lot of work. Isn’t
there an easy way to get this all
setup?”

Augment WebdriverIO
with 3rd party tools

3rd party tools
Use physical or
emulated mobile
devices to run your
tests
Manage your
selenium grid and
record videos of
your tests
Proprietary
solutions to offload
the work of setting
up Zalenium.

A flexible and scalable
container based
Selenium Grid with
video recording, live
preview, basic auth &
dashboard.
Docker and
kubernetes support
out of the box.

Cloud web and mobile
testing platform - test
across on-demand
browsers, operating
systems and real
mobile devices, without
installing or
maintaining virtual
machines, devices or
emulators

Appium is an
open-source tool for
automating native,
mobile web, and
hybrid applications on
iOS mobile, Android
mobile, and Windows
desktop platforms.

Advanced techniques
for end-to-end testing

VRT A11y Video Audio
Take screenshots
during tests and
implement visual
regression testing
Test web page for
a11y violations by
injecting Axe into
the page
Test whether a video
is playing or not, by
using the video’s
‘paused’ attribute
Test whether
audio is playing or
not, by injecting an
extension on the fly
Advanced techniques for testing trickier elements

VRT.
Take screenshots and
compare them with
baseline.
Easily catch visual
regressions and bugs
introduced by generic CSS
changes.

// array of diff results
// one result for each viewport size
const results = browser.checkElement('#footer');
const isExactSameImage = results.every(
result => result.isExactSameImage
);
assert.ok(isExactSameImage, 'footer has not
changed');

A11y testing with Axe.
Inject the axe-core library
into a web page.
Then use the axe-core
library to analyze the
website and detect a11y
violations.

const axeSource = require('axe-core').source;
browser.execute(axeSource);
const res = browser.executeAsync(function(done) {
axe.run(function(err, results) {
if (err) done(err);
done(results);
});
});
assert.lengthOf(res.value.violations, 0);

Video.
The easiest and most
reliable way to test video:
just check the ‘paused’
attribute.
Having tried various
techniques, this is the
simplest and has never
failed me

const isVideoPaused = browser.selectorExecute(
'#video',
// executes the following in the browser context
// videos is an array of elements corresponding
// to the given selector, just retrieve the
// `paused` state of the first video.
videos => videos[0].paused
);
assert.ok(!isVideoPaused, `video is playing`);

Audio.
Inject a chrome extension on
the fly. The extension will
leverage the chrome API to
inject a DOM element when
sound is playing on the
current tab.
Then simply check whether
the DOM element is present.

// snippet from extension
chrome.runtime.onMessage.addListener(request => {
if (request.audible) {
el.className = isPlayingAudioClass;
} else {
el.className = '';
}
});

// element injected by the extension
// if audio is playing
const isPlayingAudio =
browser.isExisting(isPlayingAudioClass);
assert(isPlayingAudio, 'audio is playing');

github.com/aberonni/webdriverio-demo-tests

You
“Shouldn’t we rather invest time in
unit and integration tests?.”

The testing pyramid tricks us into thinking that unit
tests are more important than end-to-end tests,
and that, if we have to choose, we should choose
unit tests.

The truth is that the testing pyramid is telling us
that we should prioritise unit tests. If forced to
choose, you should prefer to rely on end-to-end
tests because they are closest to the customer.

Me again
“Any test is better than no tests.”

End to-end testing from rookie to pro

More Related Content

What's hot (20)

Similar to End to-end testing from rookie to pro (20)

Recently uploaded (20)

End to-end testing from rookie to pro