Collaborating with Developers: How-to Guide for Test Engineers - By Gil Tayar

@giltayar
SUCCESSFULLY COLLABORATING WITH DEVELOPERS:
HOW-TO GUIDE FOR QA PROS AND TEST ENGINEERS
Gil Tayar (@giltayar)
January 2019
This presentation: http://bit.ly/collaborating-with-developers
Github repo: https://github.com/giltayar/collaborating-with-developers
1

@giltayar@giltayar
About Me ● My developer experience goes all the way
back to the ‘80s.
● Am, was, and always will be a developer
● Testing the code I write is my passion
● Currently evangelist and architect @
Applitools
● We deliver Visual Testing tools:
If you’re serious about testing, checkout
Applitools Eyes
● Sometimes my arms bend back
● But the gum I like is coming back in style
@giltayar
2

@giltayar
Stepping in and
Collaborating with
Developers
3

@giltayar
Stepping in and
Collaborating with
Developers
4

@giltayar
Agile Manifesto
● Individuals and interactions over processes and tools
● Working software over comprehensive documentation
● Customer collaboration over contract negotiation
● Responding to change over following a plan
● http://agilemanifesto.org/
7

@giltayar
The “Just Wing It” Approach
8

@giltayar
(Agility)
● Individuals and interactions over
processes and tools
● Working software over
comprehensive documentation
● Customer collaboration over
contract negotiation
● Responding to change over
following a plan
9

@giltayar
10

@giltayar
Working software
over comprehensive documentation
11

@giltayar
Trunk-based Development
12

@giltayar
Trunk-based
Development
13

@giltayar
How do we test in such an environment?
14

@giltayar
No more nightlies
● Tests cannot run “overnight”
● Tests cannot take hours, or even tens of minutes.
● At most a few minutes. 1-3.
15

@giltayar
Developers MUST Test
16

@giltayar
The QA Gateway Must Die
17

@giltayar
Tests must be part of the development cycle
18

@giltayar
Tests must be fast
● Developers can’t wait
● They need to know now that the code runs
● They have to commit now
19

@giltayar
This is the “Waterfall” Method
20

@giltayar
This is better, but not good enough
21

@giltayar
This is true “agile”
22

@giltayar
This is the Essence of “Shift Left”
23

@giltayar
Yay!
Shift Left!
Yay!
24

@giltayar
Developers Don’t Test
26

@giltayar
Why Don’t
Developers Test?
● They’re lazy bums
● They just “wing it”.
● “It’s gonna be
alright”
27

@giltayar
Backend and Frontend Developers
28

@giltayar
Backend
Developers Test
More!
● More years building
methodologies
● Easier
29

@giltayar
Frontend
Developers Test
Less
● It’s a young discipline
● More difficult
30

@giltayar
Frontend Testing is Young
● The whole modern Frontend Stack didn’t exist 5 years ago
○ The previous stack was impossible to test
● The current stack is testable
○ It took time to solidify
● But it has solidified now.
● There is a methodology that is used for frontend testing
31

@giltayar
But Why Frontend Developers?
32

@giltayar
But Why Frontend Developers?
● Closest to the product
● Less tested
● We need to help them
● Best bang for the buck
○ Same tools as E2E
33

@giltayar
And… they’re cooler!
34

@giltayar
Which brings us to the second part...
35

@giltayar
How do we do frontend testing?
37

@giltayar
Let’s start with the language
38

@giltayar
JavaScript isn’t serious
● “JavaScript is a toy language”
● “JavaScript shouldn’t be taken seriously”
● “It’s nice for small programs”
● “0.2 + 0.1 == 0.30000000000000004”
This was true 5 to 10 years ago. Not true now.
(and the last one is true in most languages)
39

@giltayar
I have two quotes for you...
40

@giltayar
Atwood’s Law
If it can be written in JavaScript, it will be written in JavaScript
42

@giltayar
Code Written in JavaScript
● Gmail
● Google Maps
● Twitter UI
● Facebook
● Large parts of server-side Netflix
● My favorite example:
a CPU+hardware emulator that boots Linux
43

@giltayar
The JavaScript Renaissance
JavaScript today is...
● Modern
● Powerful
● Concise
● Functional
● Readable
● Ubiquitous (browser, server, CLI, IoT)
● Has the richest and largest 3rd party library in the world
● ...and is continually evolving
44

@giltayar
Next Thing: Testing Methodology
45

@giltayar
Unit
Integration
E2E
The Testing Pyramid
46

@giltayar
Unit tests...
● Are fast (milliseconds)
● Are not flaky
● Do no I/O or use browser features
● Test only one module, function, or class
● Bring little confidence on their own
● Are perfect for Business Logic testing
48

@giltayar
Integration
Integration Tests
49

@giltayar
Integration tests...
● Are still fast (10-100s milliseconds)
● Are mostly not flaky
● Do I/O and use browser features
● Test a group of modules/classes/functions as they are tested in the
final product
● Bring some level of confidence in the application
● Are perfect for testing whole parts of the application easily
50

@giltayar
E2E tests...
● Are slow (seconds)
● Are flakier
● Browser Automation tests
● Test features end to end
● Bring lots of confidence
52

@giltayar
Unit
Integration
E2E
The Testing Pyramid
Slowness
Flakiness
Confidence
53

@giltayar
Unit
Integration
E2E
Why is Speed Important?
Slowness
Flakiness
Confidence
54

@giltayar
Answer: Development Cycle
What is acceptable for nightly automation test, is not acceptable for
developers
55

@giltayar
Answer: Development Cycle
What is acceptable for nightly automation test, is not acceptable for
developers
Hence the emphasis on unit and integration tests
56

@giltayar
OK, OK, Shift Left, yeah.
But...
57

@giltayar
What’s the Tester’s Role?
● Educate and monitor
○ They are lazy bums, after all. 😉
● Work on the tests with the frontend developers
● Write the real E2E tests
● And… Shift Right. E2E tests in production!
○ Which you can (and should) still do with JS
58

@giltayar
OK, OK. But how?
How do I write
tests?
Show me some
code!
59

@giltayar
Writing Unit Tests
60

@giltayar
Remember….
● Unit tests test only one module, function, or class
● Bring little confidence on their own
● Are perfect for Business Logic testing
● Are very fast (milliseconds)
61

@giltayar
The Function to Test
function factorial (n) {
let result = 1
for (let i = 1; i <= n; ++i) {
result *= i
}
return result
}
module.exports = factorial
62

@giltayar
Whoever uses the function needs to...
const factorial = require('./factorial.js')
...
... factorial(...)
...
63

@giltayar
What do we want to test?
● factorial(0) == 1
64

@giltayar
Test Factorial
const assert = require('assert')
const factorial = require('../../lib/factorial')
assert.strict.equal(factorial(0), 1)
65

@giltayar
Does the browser support `module.exports`? No!
function factorial (n) {
let result = 1
for (let i = 1; i <= n; ++i) {
result *= i
}
return result
}
module.exports = factorial
66

@giltayar
Modular Modern JS
<script
src=bundle.js>
Webpack bundle.js
Production
Code
67

@giltayar
Where can this code run?
The Browser*
* With the help of webpack
68

@giltayar
Where can it also run?
NodeJS
69

@giltayar
Most frontend code today can also run under
NodeJS
707
0

@giltayar
Universal/Isomorphic Code
71

@giltayar
Let’s run it under NodeJS
72

@giltayar
Awkward to Test Like This
● We need a Test Runner
● Just like jUnit, NUnit, pytest, test-unit, … in other languages
● NodeJS has lots of them:
○ Mocha, Jest, Ava, Tape, Jasmine.
○ And the list goes on…
● The most popular are Mocha and Jest
● We’ll be demoing using Mocha
73

@giltayar
Mocha Test
const { describe, it } =
require('mocha')
const { expect } =
require('chai')
const factorial =
require('../../lib/factorial`' )
...
...
describe('factorial', () => {
it('should handle 0' , () => {
expect(factorial( 0)).to.equal(1)
})
})
})
})
74

@giltayar
Let’s run it under Mocha
75

@giltayar
Testable Code
● Separation of Concerns: code does one thing and one thing only
● Separate UI code, I/O code, and logic
● Test logic with unit tests, and the others with integration tests
76

@giltayar
Untestable Code
function writeFactorialToServer (n, filename) {
let result = 1
for (let i = 1; i <= n; ++i) {
result *= i
}
// write result to server
fetch('http://...', {method: 'PUT', body: result.toString()})
}
module.exports = writeFactorial
77

@giltayar
Notice how important speed is...
78

@giltayar
Writing Integration Tests
79

@giltayar
Remember...
● Test a group of modules/classes/functions as they are glued in the
final product
80

@giltayar
Must Run in the Browser?
● Test a group of modules/classes/functions as they are glued in the
final product
81

@giltayar
No! It Can Run Under NodeJS
82

@giltayar
But unfortunately, out of scope
83

@giltayar
For more information...
https://www.youtube.com/watch?v=H_2cMSuNdS8
84

@giltayar
Just a taste...
describe('calculator app component' , function () {
before(function () {
global.window =
new JSDOM(
`<html><body><div id="container"/></div></body></html>` ).window
global.document = window.document
})
85

@giltayar
Just a taste [2]...
it('should work', function () {
ReactDom.render(<CalculatorApp />, document.getElementById('container'))
const digit4Element = document.querySelector('.digit-4')
const operatorMultiply = document.querySelector('.operator-multiply')
const operatorEquals = document.querySelector('.operator-equals')
digit4Element.click()
operatorMultiply.click()
digit4Element.click()
operatorEquals.click()
expect(displayElement.textContent).to.equal('16')
})
86

@giltayar
Using JSDOM for Integration Tests...
● Run in milliseconds
● No need to run a server
● No need to run a browser
● Not flaky
● Debug with any NodeJS debugger
● No sourcemaps
● No build step - just change code and rerun
● Mock XHR using nock - no mock HTTP server
87

@giltayar
Writing E2E Tests
(Browser Automation)
89

@giltayar
We need a browser automation framework...
909
0

@giltayar
● Selenium WebDriver
● TestCafe
● WebDriverIO
● NightWatch
● CasperJS
● Cypress
● Puppeteer
● …
We have lots of them...
91

@giltayar
● Selenium WebDriver
But we’ll use...
92

@giltayar
Serving the Frontend Code
before((done) => {
const app = express()
app.use('/', express.static(__dirname + '/../../dist'))
server = app.listen(8080, done)
})
after(() => {
server.close()
})
93

@giltayar
Initializing WebDriver
before(async () => {
driver = new webdriver.Builder()
.forBrowser('chrome')
.build()
})
after(async () => await driver.quit())
94

@giltayar
The Test
it('should work', async function () {
await driver.get('http://localhost:8080')
const digit4Element = await driver.findElement(By.css('.digit-4'))
const operatorMultiply = await driver.findElement(By.css('.operator-multiply'))
const operatorEquals = await driver.findElement(By.css('.operator-equals'))
await digit4Element.click()
await operatorMultiply.click()
await operatorEquals.click()
await driver.wait(until.elementTextIs(await driver.findElement(By.css('.display')),
'84'))
})
95

@giltayar
Adding Visual Regression
Tests
96

@giltayar
Especially Important For Frontend Developers
97

@giltayar
Visual Regression Testing
Enables frontend developers to ensure that their application always
looks the same
98

@giltayar
Pretty Simple To Implement Atop Automation Tests
● Just add a line that takes a screenshot of the page and compares to previous
baseline screenshot
99

@giltayar
The Test
it('should work', async function () {
await driver.get('http://localhost:8080')
...
await operatorMultiply.click()
await operatorEquals.click()
await driver.wait(until.elementTextIs(
await driver.findElement(By.css('.display')), '84'))
await eyes.checkWindow('After calculating 42 * 2 =')
})
10
0

@giltayar
Summary
● Agile is here: “There is no release, code is always working”
● Old “QA Gateway method” cannot work anymore
● Shift-left to testing during development
● Work with developers for this. Mostly frontend developers
● Understand the language of the frontend developers
○ The test pyramid
○ The advantages and disadvantages of each test type in terms of speed, flakiness, and
confidence
○ JavaScript and modern JavaScript Development
○ The different test runners, browser automation frameworks, etc…
● It’s a whole new world!
10
2

@giltayar
● Intro to frontend testing:
https://hackernoon.com/testing-your-frontend-code-part-v-visual-testing-935864cf
b5c7
● Frontend integration testing: https://www.youtube.com/watch?v=H_2cMSuNdS8
● Assert(JS) Talks:
https://www.youtube.com/playlist?list=PLZ66c9_z3umNSrKSb5cmpxdXZcIPNvK
Gw
● People to follow:
○ Kent C. Dodds
○ Kevin Lamping
○ Me… 😊
Resources
10
3

@giltayar
Questions?
This presentation: http://bit.ly/collaborating-with-developers
Github repo: https://github.com/giltayar/collaborating-with-developers
10
4

Collaborating with Developers: How-to Guide for Test Engineers - By Gil Tayar

More Related Content

What's hot (11)

Similar to Collaborating with Developers: How-to Guide for Test Engineers - By Gil Tayar (20)

More from Applitools (20)

Recently uploaded (20)

Collaborating with Developers: How-to Guide for Test Engineers - By Gil Tayar