SlideShare a Scribd company logo
objectcomputing.com© 2017, Object Computing, Inc. (OCI). All rights reserved. No part of these notes may be reproduced, stored in a retrieval system, or
transmitted, in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior, written
permission of Object Computing, Inc. (OCI)
Make your testing Groovy
GR8Conf US 2017
Paul King
code: https://github.com/paulk-asert/MakeTestingGroovy
slides: https://www.slideshare.net/paulk_asert/make-testing-groovy
©paulk_asert2006-2017
Understanding testing
Test
System under test
Range of testing scenarios
Simple
component
Unit
Testing
Multiple
Pieces
Your
lete
comp-
system
Integration
Testing
Acceptance
Testing
Test
System under test
©paulk_asert2006-2017
Understanding testing
Unit
Utilities
AcceptanceIntegration
Tests
Test Runner/Framework
Driver
©paulk_asert2006-2017
Understanding testing with a DSL
Unit
Utilities
AcceptanceIntegration
Tests
Test Runner/Framework Driver
©paulk_asert2006-2017
Understanding testing with a DSL
Unit
Utilities
AcceptanceIntegration
Tests
Test Runner/Framework Driver
Testing DSL
©paulk_asert2006-2017
Why test with Groovy?
©paulk_asert2006-2017
Why test with Groovy?
It depends …
… but many great reasons! ☺
TL;DR
• Feature-rich frameworks which encourage easy to maintain
tests
• Script all things (devops nirvana)
• Open source avoids being locked-in to vendor acceptance
testing tools
• Great for writing testing DSLs
• Access to huge range of utilitiy libraries
©paulk_asert2006-2017
Why Groovy?
Unit AcceptanceIntegration
Consider audience/”fit”
Utilities
Tests
Test Runner/Framework Driver
Testing DSL
©paulk_asert2006-2017
“People Fit” / “Organization Fit”
People
• Developers (familiarity with languages)
• Testers (tools language familiarity)
• Responsibility to create/run/maintain
• BAs/SMEs (ability to read technical tests)
• Consider both development and maintenance
• Expected feedback from tests (JUnit/Reports)
Organization
• Maturity level
• Degree of automation
• Tools in use
• Need for adaptability
©paulk_asert2006-2017
Why Groovy?
Unit AcceptanceIntegration
Utilities
Tests
Test Runner/Framework Driver
Testing DSL
Very flexible utilities available:
reading excel
regex for reading text
combinations
©paulk_asert2006-2017
Why Groovy?
Unit AcceptanceIntegration
Utilities
Tests
Test Runner/Framework Driver
Testing DSL
Great DSL support
©paulk_asert2006-2017
Why Groovy?
Unit AcceptanceIntegration
Utilities
Tests
Test Runner/Framework Driver
Testing DSL
Less brittle “glue” code
©paulk_asert2006-2017
Why Groovy?
Unit AcceptanceIntegration
Utilities
Tests
Test Runner/Framework Driver
Testing DSL
Great Frameworks:
JUnit, TestNG, Spock
Data-driven, property-based
©paulk_asert2006-2017
Why Groovy?
Unit AcceptanceIntegration
Utilities
Tests
Test Runner/Framework Driver
Testing DSL
Many Java/Groovy drivers:
Geb, HtmlUnit, Selenium
©paulk_asert2006-2017
Looking at testing frameworks
Unit AcceptanceIntegration
Utilities
Tests
Test Runner/Framework Driver
Testing DSL
Testing Frameworks
None
JUnit 3
JUnit 4
JUnit 5
TestNG
Spock
Testing Frameworks
None ???
JUnit 3
JUnit 4
JUnit 5
TestNG
Spock
Testing Frameworks
None ???
• Groovy deems testing so important
that it comes with built-in testing:
• Built-in asserts, mocks
• Built-in JUnit 3 GroovyTestCase and utilities
• Built-in runners for tests
• Bundles JUnit 4
JUnit 3
JUnit 4
JUnit 5
TestNG
Spock
©paulk_asert2006-2017
Built-in assertions
class Converter {
static celsius (fahrenheit) {
(fahrenheit - 32) * 5 / 9
}
}
Temperature scales
http://imgur.com/gallery/KB3nyXX
©paulk_asert2006-2017
Built-in assertions
import static Converter.celsius
assert 20 == celsius(68)
assert 35 == celsius(95)
assert -17 == celsius(0).toInteger()
assert 0 == celsius(32)
class Converter {
static celsius (fahrenheit) {
(fahrenheit - 32) * 5 / 9
}
}
©paulk_asert2006-2017
Built-in assertions
But what about errors?
Groovy’s power Assert mechanism gives a friendly description of
what went wrong
©paulk_asert2006-2017
Built-in assertions
But what about errors?
Groovy’s power Assert mechanism gives a friendly description of
what went wrong
©paulk_asert2006-2017
GroovyTestCase (extends JUnit 3)
import org.junit.Assert
import static Converter.celsius
import static org.junit.Assert.assertEquals
class SimpleGroovyTest extends GroovyTestCase {
void testShouldConvert() {
assert Converter.celsius(68) == 20
assert Converter.celsius(212) == 100, "Should convert boiling"
assertEquals("Should convert freezing", 0.0, celsius(32.0))
assertEquals("Should convert nearly freezing", 0.0, celsius(32.1), 0.1)
}
}
©paulk_asert2006-2017
JUnit4
import org.junit.Test
import static org.junit.Assert.assertEquals
import static Converter.celsius
class SimpleJUnit4Test {
@Test
void shouldConvert() {
assert celsius(68) == 20
assert celsius(212) == 100, "Should convert boiling"
assertEquals("Should convert freezing", 0.0, celsius(32.0))
assertEquals("Also for nearly freezing", 0.0, celsius(32.1), 0.1)
}
}
©paulk_asert2006-2017
JUnit5
@Grab('org.junit.platform:junit-platform-runner:1.0.0-M5')
@Grab('org.junit.jupiter:junit-jupiter-engine:5.0.0-M5')
import org.junit.jupiter.api.Test
import org.junit.platform.runner.JUnitPlatform
import org.junit.runner.RunWith
import static Converter.celsius
@RunWith(JUnitPlatform)
class ConverterJUnit5Tests {
@Test
void freezing() {
assert celsius(32) == 0
}
@Test
void boiling() {
assert celsius(212) == 100
}
}
©paulk_asert2006-2017
JUnit5
@Grab('org.junit.platform:junit-platform-runner:1.0.0-M5')
@Grab('org.junit.jupiter:junit-jupiter-engine:5.0.0-M5')
import org.junit.jupiter.api.Test
import org.junit.platform.runner.JUnitPlatform
import org.junit.runner.RunWith
import static Converter.celsius
@RunWith(JUnitPlatform)
class ConverterJUnit5Tests {
@Test
void freezing() {
assert celsius(32) == 0
}
@Test
void boiling() {
assert celsius(212) == 100
}
}
©paulk_asert2006-2017
Spock - BDD style
@Grab('org.spockframework:spock-core:1.1-groovy-2.4')
import spock.lang.Specification
class GivenWhenThenSpec extends Specification {
def "test adding a new item to a set"() {
given:
def items = [4, 6, 3, 2] as Set
when:
items << 1
then:
items.size() == 5
}
}
©paulk_asert2006-2017
Parameterized import org.junit.Test
import org.junit.runner.RunWith
import org.junit.runners.Parameterized
import org.junit.runners.Parameterized.Parameters
import static Converter.celsius
@RunWith(Parameterized)
class DataDrivenJUnitTest {
private c, f, scenario
@Parameters static scenarios() {[
[0, 32, 'Freezing'],
[20, 68, 'Garden party conditions'],
[35, 95, 'Beach conditions'],
[100, 212, 'Boiling']
]*.toArray()}
DataDrivenJUnitTest(c, f, scenario)
this.c = c
this.f = f
this.scenario = scenario
}
@Test void convert() {
def actual = celsius(f)
def msg = "$scenario: ${f}°F should convert into ${c}°C"
assert c == actual, msg
}
}
©paulk_asert2006-2017
Spock
@Grab('org.spockframework:spock-core:1.1-groovy-2.4')
import spock.lang.*
import static Converter.celsius
class SpockDataDriven extends Specification {
def "test temperature scenarios"() {
expect:
celsius(tempF) == tempC
where:
scenario | tempF || tempC
'Freezing' | 32 || 0
'Garden party conditions' | 68 || 20
'Beach conditions' | 95 || 35
'Boiling' | 212 || 100
}
}
©paulk_asert2006-2017
Spock - Celsius
@Grab('org.spockframework:spock-core:1.1-groovy-2.4')
import spock.lang.*
import static Converter.celsius
class SpockDataDriven extends Specification {
@Unroll
def "Scenario #scenario: #tempFºF should convert to #tempCºC"() {
expect:
celsius(tempF) == tempC
where:
scenario | tempF || tempC
'Freezing' | 32 || 0
'Garden party conditions' | 68 || 20
'Beach conditions' | 95 || 34
'Boiling' | 212 || 100
}
}
©paulk_asert2006-2017
Spock - Celsius
@Grab('org.spockframework:spock-core:1.1-groovy-2.4')
import spock.lang.*
import static Converter.celsius
class SpockDataDriven extends Specification {
@Unroll
def "Scenario #scenario: #tempFºF should convert to #tempCºC"() {
expect:
celsius(tempF) == tempC
where:
scenario | tempF || tempC
'Freezing' | 32 || 0
'Garden party conditions' | 68 || 20
'Beach conditions' | 95 || 34
'Boiling' | 212 || 100
}
}
Assertion frameworks
None
• Groovy’s built-in assert
JUnit 3 assertions
• assertEquals()
Dedicated library
• Hamcrest
• FEST
• AssertJ
• Google truth
assertThat(googleColors).containsNoneOf(PINK, BLACK, WHITE, ORANGE)
assert ![PINK, BLACK, WHITE, ORANGE].any {
color -> color in googleColors }

usually
usually
Assertion frameworks
None
• Groovy’s built-in assert
JUnit 3 assertions
• assertEquals()
Dedicated library
• Hamcrest
• FEST
• AssertJ
• Google truth
assertThat(googleColors).containsNoneOf(PINK, BLACK, WHITE, ORANGE)
assert [PINK, BLACK, WHITE, ORANGE].every {
color -> color !in googleColors }

usually
usually
©paulk_asert2006-2017
Looking at web drivers
Unit AcceptanceIntegration
Utilities
Tests
Test Runner/Framework Driver
Testing DSL
Case Study
Case Study
Case Study
Web testing drivers
None
Regex
XmlSlurper
Cyberneko
JSoup
HttpBuilder
HttpBuilderNG
JMeter
Ersatz
WebTest
HtmlUnit
Geb
WebDriver
Selenium
JWebUnit
Arquillian
Cucumber
JBehave
Serenity
RobotFramework
Concordion
EasyB
Tumbler
FitNesse/Slim
Case Study
def html = new URL('http://localhost:8080').text
assert html.contains('<title>Welcome to SimpBlog</title>')
html.find(~'<title>(.*)</title>') { all, title ->
assert title == 'Welcome to SimpBlog'
}
Case Study
def page = new XmlSlurper().parse('http://localhost:8080/viewPost?id=1')
assert page.body.h1.text().contains('Christmas')
assert page.body.h3[1].text() == 'Category: Home'
assert page.body.h3[2].text() == 'Author: Bart'
assert page.body.table.tr.td.p.text() ==
"Aren't we forgeting the true meaning of this day? You know,
the birth of Santa."
Case Study
@Grab('net.sourceforge.nekohtml:nekohtml:1.9.22')
import org.cyberneko.html.parsers.SAXParser
def parser = new XmlSlurper(new SAXParser())
def page = parser.parse('http://localhost:8080/viewPost?id=1')
assert page.BODY.H1.text().contains('Christmas')
assert page.BODY.H3[1].text() == 'Category: Home'
assert page.BODY.H3[2].text() == 'Author: Bart'
assert page.BODY.TABLE.TBODY.TR.TD.P.text() ==
"Aren't we forgeting the true meaning of this day? You
know, the birth of Santa."
Case Study
HtmlUnit
GUI-Less browser for Java
Lets you retrieve pages, click
links, fill out forms, etc.
HtmlUnit class TestSimpBlogJUnit4 {
def page
@Before
void setUp() {
page = new WebClient().getPage('http://localhost:8080/postForm')
assert 'Welcome to SimpBlog' == page.titleText
}
@Test
void bartWasHere() {
// fill in blog entry and post it
def form = page.getFormByName('post')
form.getInputByName('title').setValueAttribute('Bart was here (HtmlUnit JUnit4)')
form.getSelectByName('category').getOptions().find {
it.text == 'School' }.setSelected(true)
form.getTextAreaByName('content').setText('Cowabunga Dude!')
def result = form.getInputByName('btnPost').click()
// check blog post details
assert result.getElementsByTagName('h1').item(0).
textContent.matches('Post.*: Bart was here.*')
def h3headings = result.getElementsByTagName('h3')
assert h3headings.item(1).textContent == 'Category: School'
assert h3headings.item(2).textContent == 'Author: Bart'
// expecting: <table><tr><td><p>Cowabunga Dude!</p>...</table>
def cell = result.getByXPath('//TABLE//TR/TD')[0]
assert cell.textContent.trim() == 'Cowabunga Dude!'
}
}
Case Study
Geb
@Grab("org.gebish:geb-core:1.1.1")
@Grab("org.seleniumhq.selenium:selenium-chrome-driver:3.4.0")
@Grab("org.seleniumhq.selenium:selenium-support:3.4.0")
import geb.Browser
Browser.drive {
go 'http://localhost:8080/postForm'
assert title == 'Welcome to SimpBlog'
$("form").with {
title = "Bart was here (Geb)"
author = 'Bart'
category = 'School'
content = "Cowabunga Dude!"
btnPost().click()
}
assert $("h1").text().matches("Post d+: Bart was here.*")
assert $("h3")[1].text() == 'Category: School'
assert $("h3")[2].text() == 'Author: Bart'
assert $("p").text() == "Cowabunga Dude!"
}
Geb with pages
class NewPostPage extends Page {
static url = "http://localhost:8080/postForm"
static at = { title == 'Welcome to SimpBlog' }
static content = {
blogTitle { $("form").title() } // !title
blogger { $("form").author() }
label { $("form").category() }
blogText { $("form").content() } // !content
post(to: ViewPostPage) { btnPost() }
}
}
class ViewPostPage extends Page {
static at = { $("h1").text().contains('Post') }
static content = {
mainHeading { $("h1").text() }
categoryHeading { $("h3")[1].text() }
authorHeading { $("h3")[2].text() }
blogText { $("p").text() }
}
}
Geb with pages
class NewPostPage extends Page {
static url = "http://localhost:8080/postForm"
static at = { title == 'Welcome to SimpBlog' }
static content = {
blogTitle { $("form").title() } // !title
blogger { $("form").author() }
label { $("form").category() }
blogText { $("form").content() } // !content
post(to: ViewPostPage) { btnPost() }
}
}
class ViewPostPage extends Page {
static at = { $("h1").text().contains('Post') }
static content = {
mainHeading { $("h1").text() }
categoryHeading { $("h3")[1].text() }
authorHeading { $("h3")[2].text() }
blogText { $("p").text() }
}
}
Browser.drive {
to NewPostPage
assert at(NewPostPage)
blogTitle.value 'Bart was here (Geb pages)'
blogger.value 'Bart'
label.value 'School'
blogText.value 'Cowabunga Dude!'
post.click()
assert at(ViewPostPage)
assert mainHeading ==~ "Post d+: Bart was here.*"
assert categoryHeading == 'Category: School'
assert authorHeading == 'Author: Bart'
assert blogText == "Cowabunga Dude!"
}
Other Web Drivers
Case Study
Case Study
Case Study
Case Study
Case Study:
Cucumber
//…
Given(~/^we are on the create blog entry page$/) { ->
tester = new BlogTester('http://localhost:8080/postForm')
tester.checkTitle 'Welcome to SimpBlog'
}
When(~/^I have entered '([^']*)' as the title$/) { String title ->
formFields.title = title
}
When(~/^I have entered '([^']*)' into the content$/) { String content ->
formFields.content = content
}
When(~/^I have selected '([^']*)' as the category$/) { String category ->
formFields.category = category
}
When(~/^I click the 'Create Post' button$/) { ->
tester.postBlog(formFields)
}
Then(~/^I expect the entry to be posted$/) { ->
tester.checkHeadingMatches formFields.title
tester.checkSubheading 'Category', formFields.category
tester.checkPostText formFields.content
tester.checkSubheading 'Author', formFields.author
}
//…
Given(~/^we are on the create blog entry page$/) { ->
tester = new BlogTester('http://localhost:8080/postForm')
tester.checkTitle 'Welcome to SimpBlog'
}
When(~/^I have entered '([^']*)' as the title$/) { String title ->
formFields.title = title
}
When(~/^I have entered '([^']*)' into the content$/) { String content ->
formFields.content = content
}
When(~/^I have selected '([^']*)' as the category$/) { String category ->
formFields.category = category
}
When(~/^I click the 'Create Post' button$/) { ->
tester.postBlog(formFields)
}
Then(~/^I expect the entry to be posted$/) { ->
tester.checkHeadingMatches formFields.title
tester.checkSubheading 'Category', formFields.category
tester.checkPostText formFields.content
tester.checkSubheading 'Author', formFields.author
}
Feature: Use the blogging system
Scenario: Bart posts a new blog entry
Given we are on the create blog entry page
When I have entered 'Bart was here (Cuke Groovy)' as the title
And I have entered 'Cowabunga Dude!' into the content
And I have selected 'Home' as the category
And I have selected 'Bart' as the author
And I click the 'Create Post' button
Then I expect the entry to be posted
Case Study:
Cucumber
Case Study:
Cucumber
//…
Given(~/^we are on the create blog entry page$/) { ->
tester = new BlogTester('http://localhost:8080/postForm')
tester.checkTitle 'Welcome to SimpBlog'
}
When(~/^I have entered '([^']*)' as the title$/) { String title ->
formFields.title = title
}
When(~/^I have entered '([^']*)' into the content$/) { String content ->
formFields.content = content
}
When(~/^I have selected '([^']*)' as the category$/) { String category ->
formFields.category = category
}
When(~/^I click the 'Create Post' button$/) { ->
tester.postBlog(formFields)
}
Then(~/^I expect the entry to be posted$/) { ->
tester.checkHeadingMatches formFields.title
tester.checkSubheading 'Category', formFields.category
tester.checkPostText formFields.content
tester.checkSubheading 'Author', formFields.author
}
Case Study: Slim/FitNesse
package simpblog
import com.gargoylesoftware.htmlunit.WebClient
class NewBlogPost {
String author, title, content, category
private result
def execute() {
def client = new WebClient()
def page = client.getPage('http://localhost:8080/postForm')
assert 'Welcome to SimpBlog' == page.titleText
def form = page.getFormByName('post')
form.getInputByName('title').
setValueAttribute("$title (Slim)")
form.getSelectByName('author').getOptions().find{
it.text == author }.setSelected(true)
form.getSelectByName('category').getOptions().find{
it.text == category }.setSelected(true)
form.getTextAreaByName('content').setText(content)
result = form.getInputByName('btnPost').click()
}
def mainHeading() {
def m = result.getElementsByTagName('h1').item(0).textContent =~ /Post .*: (.*) ([^)]*)/
m[0][1]
}
Case Study: Slim/FitNesse
package simpblog
import com.gargoylesoftware.htmlunit.WebClient
class NewBlogPost {
String author, title, content, category
private result
def execute() {
def client = new WebClient()
def page = client.getPage('http://localhost:8080/postForm')
assert 'Welcome to SimpBlog' == page.titleText
def form = page.getFormByName('post')
form.getInputByName('title').
setValueAttribute("$title (Slim)")
form.getSelectByName('author').getOptions().find{
it.text == author }.setSelected(true)
form.getSelectByName('category').getOptions().find{
it.text == category }.setSelected(true)
form.getTextAreaByName('content').setText(content)
result = form.getInputByName('btnPost').click()
}
def mainHeading() {
def m = result.getElementsByTagName('h1').item(0).textContent =~ /Post .*: (.*) ([^)]*)/
m[0][1]
}
©paulk_asert2006-2017
Looking at testing utilities
Unit AcceptanceIntegration
Utilities
Tests
Test Runner/Framework Driver
Testing DSL
Testing DSLs
Low-level “DSL/fluent API”
Medium-level DSL
Higher-level DSL
post blog from Bart with title "Bart rulz!" and category School and content "Cowabunga Dude!"
def form = page.getFormByName('post')
form.getInputByName('title').setValueAttribute('Bart was here (HtmlUnit JUnit4)')
form.getSelectByName('category').getOptions().find {
it.text == 'School' }.setSelected(true)
DSLs: Fluent API
class BlogTestCase extends GroovyTestCase {
def page
def lastResult
void setUp() {
page = new WebClient().getPage('http://localhost:8080/postForm')
}
def checkTitle(String title) {
assert title == page.titleText
}
def prepareBlog() {
new PrepareBlogEmpty()
}
// ...
}
DSLs: Fluent API
// ...
def postBlog(Map params) {
def form = page.getFormByName('post')
form.getInputByName('title').setValueAttribute(params.title)
form.getSelectByName('category').options.find {
it.text == params.category
}.setSelected(true)
form.getSelectByName('author').options.find {
it.text == params.author
}.setSelected(true)
form.getTextAreaByName('content').setText(params.content)
lastResult = form.getInputByName('btnPost').click()
}
def checkHeadingMatches(String regex) {
def heading = lastResult.getElementsByTagName('h1').item(0)
assert heading.textContent.matches(regex)
}
// ...
}
DSLs: Fluent API
class TestSimpBlogFluentApi extends BlogTestCase {
void setUp() {
super.setUp()
checkTitle('Welcome to SimpBlog')
}
void testBartWasHere() {
prepareBlog()
.withTitle('Bart was here (HtmlUnit FluentApi)')
.withAuthor('Bart')
.withCategory('School')
.withContent('Cowabunga Dude!')
.post()
checkHeadingMatches 'Post.*: Bart was here.*'
checkSubheading 1, 'Category: School'
checkSubheading 2, 'Author: Bart'
checkPostText 'Cowabunga Dude!'
}
DSLs:
command
chains
class TestSimpBlogDsl extends BlogTestCase {
private the, has, a, with, heading
void setUp() {
super.setUp()
}
def post(_a) {
[
blog: { _with ->
[title: { postTitle ->
[and: { __with ->
[author: { postAuthor ->
[and: { ___with ->
[category: { postCategory ->
[and: { ____with ->
[content: { postContent ->
postBlog(title: postTitle,
author:postAuthor,
content:postContent,
category: postCategory)
} ]} ]} ]} ]} ]} ]} ]}
]
}
def check(_the) {
[
browser: { _has -> [title: { checkTitle it }]},
main: { _heading -> [matches: { checkHeadingMatches it }]},
category: { _has -> [value: { checkSubheading 1, "Category: $it" }]},
author: { _has -> [value: { checkSubheading 2, "Author: $it" }]},
blog: { _has -> [text: { checkPostText it }]}
]
}
// ...
DSLs:
command
chains
// ...
void testBartWasHere() {
check the browser has title 'Welcome to SimpBlog'
post a blog with title 'Bart was here (HtmlUnit DSL)' 
and with author 'Bart' 
and with category 'School' 
and with content 'Cowabunga Dude!'
check the main heading matches 'Post.*: Bart was here.*'
check the category has value 'School'
check the author has value 'Bart'
check the blog has text 'Cowabunga Dude!'
}
}
Testing DSLs
©paulk_asert2006-2017
Testing Utilities
Unit AcceptanceIntegration
Utilities
Tests
Test Runner/Framework Driver
Testing DSL
Testing Utilities
All combinations
All pairs
Property-based testing
GPars
Constraint programming
ModelJUnit
All Combinations
All Combinations Case Study
All Pairs
All Pairs Case Study
©paulk_asert2006-2017
Property-based testing
Agile testing game (TDD)
• Minimum test code to steer design of minimal production code
with desired business functionality but 100% code coverage
• “Grey box” testing
• Rarely used with functional programming
©paulk_asert2006-2017
Property-based testing
Agile testing game (TDD)
• Minimum test code to steer design of minimal production code
with desired business functionality but 100% code coverage
• “Grey box” testing
• Rarely used with functional programming
• Instead validate certain properties
©paulk_asert2006-2017
Property-based testing
@Grab('net.java.quickcheck:quickcheck:0.6')
import static
net.java.quickcheck.generator.PrimitiveGenerators.*
import static java.lang.Math.round
import static Converter.celsius
def gen = integers(-40, 240)
def liquidC = 0..100
def liquidF = 32..212
100.times {
int f = gen.next()
int c = round(celsius(f))
assert c <= f
assert c in liquidC == f in liquidF
}
©paulk_asert2006-2017
Property-based testing with Spock
https://github.com/Bijnagte/spock-genesis
©paulk_asert2006-2017
Property-based testing: spock genesis
@Grab('com.nagternal:spock-genesis:0.6.0')
@GrabExclude('org.codehaus.groovy:groovy-all')
import spock.genesis.transform.Iterations
import spock.lang.Specification
import static Converter.celsius
import static java.lang.Math.round
import static spock.genesis.Gen.integer
class ConverterSpec extends Specification {
def liquidC = 0..100
def liquidF = 32..212
@Iterations(100)
def "test phase maintained"() {
given:
int tempF = integer(-40..240).iterator().next()
when:
int tempC = round(celsius(tempF))
then:
tempC <= tempF
tempC in liquidC == tempF in liquidF
}
…
©paulk_asert2006-2017
Property-based testing: spock genesis
@Grab('com.nagternal:spock-genesis:0.6.0')
@GrabExclude('org.codehaus.groovy:groovy-all')
import spock.genesis.transform.Iterations
import spock.lang.Specification
import static Converter.celsius
import static java.lang.Math.round
import static spock.genesis.Gen.integer
class ConverterSpec extends Specification {
def liquidC = 0..100
def liquidF = 32..212
@Iterations(100)
def "test phase maintained"() {
given:
int tempF = integer(-40..240).iterator().next()
when:
int tempC = round(celsius(tempF))
then:
tempC <= tempF
tempC in liquidC == tempF in liquidF
}
…
…
@Iterations(100)
def "test order maintained"() {
given:
int tempF1 = integer(-273..999).iterator().next()
int tempF2 = integer(-273..999).iterator().next()
when:
int tempC1 = round(celsius(tempF1))
int tempC2 = round(celsius(tempF2))
then:
(tempF1 <=> tempF2) == (tempC1 <=> tempC2)
}
}
©paulk_asert2006-2017
Property-based testing: Case Study
©paulk_asert2006-2017
Property-based testing: Case Study
©paulk_asert2006-2017
Property-based testing: Case Study
GPars
Library classes and DSL allowing
you to handle tasks concurrently:
• Data Parallelism map, filter, reduce functionality
in parallel with parallel array support
• Asynchronous functions extend the Java
executor services to enable multi-threaded
closure processing
• Dataflow Concurrency supports natural
shared-memory concurrency model, using
single-assignment variables
• Actors provide Erlang/Scala-like actors
including "remote" actors on other machines
• Safe Agents provide a non-blocking mt-safe
reference to mutable state; like "agents" in Clojure 89
Case Study with GPars
//@Grab('org.codehaus.gpars:gpars:0.12')
import groovyx.gpars.GParsPool
def testCases = [
['Title 1 (GPars)', 'Home', 'Bart', 'Content 1'],
['Title 2 (GPars)', 'Work', 'Homer', 'Content 2'],
['Title 3 (GPars)', 'Travel', 'Marge', 'Content 3'],
['Title 4 (GPars)', 'Food', 'Lisa', 'Content 4']
]
GParsPool.withPool {
testCases.eachParallel{ title, category, author, content ->
postAndCheck title, category, author, content
}
}
def postAndCheck(String title, String category, String author, String content) {
def tester = new BlogTester('http://localhost:8080/postForm')
tester.postAndCheck title, category, author, content
}
Case Study with GPars
//@Grab('org.codehaus.gpars:gpars:0.12')
import groovyx.gpars.GParsPool
def testCases = [
['Title 1 (GPars)', 'Home', 'Bart', 'Content 1'],
['Title 2 (GPars)', 'Work', 'Homer', 'Content 2'],
['Title 3 (GPars)', 'Travel', 'Marge', 'Content 3'],
['Title 4 (GPars)', 'Food', 'Lisa', 'Content 4']
]
GParsPool.withPool {
testCases.eachParallel{ title, category, author, content ->
postAndCheck title, category, author, content
}
}
def postAndCheck(String title, String category, String author, String content) {
def tester = new BlogTester('http://localhost:8080/postForm')
tester.postAndCheck title, category, author, content
}
Constraint/Logic Programming
Description
• Style of programming where relations between variables are
stated in the form of constraints
• First made popular by logic programming languages such as
Prolog but the style is now also used outside logic programming
specific languages
• Constraints differ from the common primitives of other
programming languages in that they do not specify one or more
steps to execute but rather the properties of a solution to be
found
• Popular libraries used with Groovy supporting constraint
programming include Gecode/J, Choco and tuProlog
• We'll look at Choco as an example
Case Study with Constraint Programming
You have been asked to set up some test cases representing the
Simpsons’ weekly blogging habits
After some careful study you observe the following strange behavior
• They never blog on the same day
• Marge blogs only on a Saturday or Sunday
• Maggie blogs only on a Tuesday or Thursday
• Lisa blogs only on a Monday, Wednesday or Friday
• Bart blogs only on the day after Lisa
• Homer only blogs if noone else blogged the previous day and
doesn't allow anyone to blog the next day
Case Study with Constraint Programming
//@Grab('org.choco-solver:choco-solver:4.0.4')
import org.chocosolver.solver.Model
def m = new Model()
daysOfWeek = ["Sunday", "Monday", "Tuesday", "Wednesday",
"Thursday", "Friday", "Saturday"]
def (SUN, MON, TUE, WED, THU, FRI, SAT) = 0..6
def bart = m.intVar('Bart', 0, 6)
def homer = m.intVar('Homer', 0, 6)
def marge = m.intVar('Marge', 0, 6)
def lisa = m.intVar('Lisa', 0, 6)
def maggie = m.intVar('Maggie', 0, 6)
def authors = [bart, homer, marge, lisa, maggie]
//…
Case Study with Constraint Programming
// They never blog on the same day
m.allDifferent(*authors).post()
// Marge blogs only on a Saturday or Sunday
m.or(m.arithm(marge, "=", SAT), m.arithm(marge, "=", SUN)).post()
// Maggie blogs only on a Tuesday or Thursday
m.or(m.arithm(maggie, "=", TUE), m.arithm(maggie, "=", THU)).post()
// Lisa blogs only on a Monday, Wednesday or Friday
m.or(m.arithm(lisa, "=", MON), m.arithm(lisa, "=", WED), m.arithm(lisa, "=", FRI)).post()
// Bart blogs only on the day after Lisa
m.arithm(bart, "-", lisa, "=", 1).post()
// Homer only blogs if noone else blogged the previous
// day and doesn't allow anyone to blog the next day
m.and(m.distance(homer, marge, "!=", 1),
m.distance(homer, bart, "!=", 1),
m.distance(homer, maggie, "!=", 1),
m.distance(homer, lisa, "!=", 1)).post()
//…
Case Study with Constraint Programming
def solutions = []
while (m.solver.solve()) {
solutions << pad('') + authors.collect {
pad(daysOfWeek[it.value])
}.join()
}
if (solutions) {
println pad("Solutions:") + authors.collect {
pad(it.name)
}.join()
println solutions.join('n')
} else {
println "No Solutions"
}
def pad(s) { s.padRight(12) }Solutions: Bart Homer Marge Lisa Maggie
Thursday Sunday Saturday Wednesday Tuesday
Thursday Saturday Sunday Wednesday Tuesday
Saturday Tuesday Sunday Friday Thursday
Tuesday Saturday Sunday Monday Thursday
ModelJUnit
ModelJUnit
ModelJUnit
Case Study with ModelJUnit
Case Study with ModelJUnit
Case Study with ModelJUnit
Case Study with ModelJUnit
Case Study with ModelJUnit
https://github.com/paulk-asert/MakeTestingGroovy
©paulk_asert2006-2017
Key Testing Practices…
Use testing DSL’s
Look to move up the testing stack
• It used to be all about the driver
• Now the driver is hidden in the
framework or tool stack
Apply good testing practices
• Pareto analysis, bug clusters, mutation testing, test early,
all pairs/equivalence partitions/orthogonal array testing,
risk-based test selection, coding for testability, use CI,
boundary value analysis, defensive programming
More Information: Groovy in Action

More Related Content

PPTX
groovy transforms
PPTX
concurrency gpars
PDF
Industrial Strength Groovy - Tools for the Professional Groovy Developer: Pau...
PPT
A gremlin in my graph confoo 2014
ODP
GPars (Groovy Parallel Systems)
PDF
JVM Mechanics
PPTX
Pattern Matching in Java 14
PDF
Concurrency Concepts in Java
groovy transforms
concurrency gpars
Industrial Strength Groovy - Tools for the Professional Groovy Developer: Pau...
A gremlin in my graph confoo 2014
GPars (Groovy Parallel Systems)
JVM Mechanics
Pattern Matching in Java 14
Concurrency Concepts in Java

What's hot (20)

PDF
Swift and Kotlin Presentation
PPT
bluespec talk
PPT
What's New in Groovy 1.6?
PDF
Scalaz 8 vs Akka Actors
PDF
Use PEG to Write a Programming Language Parser
PPTX
Triton and Symbolic execution on GDB@DEF CON China
PPTX
Blazing Fast Windows 8 Apps using Visual C++
PDF
JVMLS 2016. Coroutines in Kotlin
PDF
Introduction to web programming for java and c# programmers by @drpicox
PDF
Java Performance Puzzlers
PDF
Leveraging Completable Futures to handle your query results Asynchrhonously
PPTX
Codestrong 2012 breakout session hacking titanium
PDF
Rust Workshop - NITC FOSSMEET 2017
PDF
Agile Developer Immersion Workshop, LASTconf Melbourne, Australia, 19th July ...
PDF
Silicon Valley JUG: JVM Mechanics
PDF
STAMP Descartes Presentation
PPTX
Introduction to Rust language programming
PDF
The Ring programming language version 1.5.3 book - Part 189 of 194
PDF
Twins: Object Oriented Programming and Functional Programming
PDF
關於測試,我說的其實是......
Swift and Kotlin Presentation
bluespec talk
What's New in Groovy 1.6?
Scalaz 8 vs Akka Actors
Use PEG to Write a Programming Language Parser
Triton and Symbolic execution on GDB@DEF CON China
Blazing Fast Windows 8 Apps using Visual C++
JVMLS 2016. Coroutines in Kotlin
Introduction to web programming for java and c# programmers by @drpicox
Java Performance Puzzlers
Leveraging Completable Futures to handle your query results Asynchrhonously
Codestrong 2012 breakout session hacking titanium
Rust Workshop - NITC FOSSMEET 2017
Agile Developer Immersion Workshop, LASTconf Melbourne, Australia, 19th July ...
Silicon Valley JUG: JVM Mechanics
STAMP Descartes Presentation
Introduction to Rust language programming
The Ring programming language version 1.5.3 book - Part 189 of 194
Twins: Object Oriented Programming and Functional Programming
關於測試,我說的其實是......
Ad

Viewers also liked (6)

PDF
How You Will Get Hacked Ten Years from Now
PDF
The Next Generation MOP, Jochen Theodorou, GR8Conf 2013
PDF
GR8Conf 2011: Grails Webflow
PPTX
Spring Web flow. A little flow of happiness
PPTX
Building a scalable API with Grails
PDF
Creating and testing REST contracts with Accurest Gradle
How You Will Get Hacked Ten Years from Now
The Next Generation MOP, Jochen Theodorou, GR8Conf 2013
GR8Conf 2011: Grails Webflow
Spring Web flow. A little flow of happiness
Building a scalable API with Grails
Creating and testing REST contracts with Accurest Gradle
Ad

Similar to Make Testing Groovy (20)

PDF
Make Your Testing Groovy
PDF
Verify Your Kubernetes Clusters with Upstream e2e tests
PPTX
Altitude San Francisco 2018: Testing with Fastly Workshop
PDF
BeJUG Meetup - What's coming in the OSGi R7 Specification
PDF
2015 Q4 webrtc standards update
PDF
World-Class Test Automation: You Can Build It Too
PDF
Maximize the power of OSGi in AEM
KEY
Tim Panton - Presentation at Emerging Communications Conference & Awards (eCo...
PPTX
How to lock a Python in a cage? Managing Python environment inside an R project
PPTX
New and cool in OSGi R7 - David Bosschaert & Carsten Ziegeler
PDF
AI For Software Engineering: Two Industrial Experience Reports
PDF
Integration tests: use the containers, Luke!
PDF
GeeCON 2017 - TestContainers. Integration testing without the hassle
PPTX
CI/CD Pipeline to Deploy and Maintain an OpenStack IaaS Cloud
PDF
DEVNET-1148 Leveraging Cisco OpenStack Private Cloud for Developers
PDF
PVS-Studio Is Now in Chocolatey: Checking Chocolatey under Azure DevOps
PPTX
Mastering the Sling Rewriter by Justin Edelson
PPTX
Mastering the Sling Rewriter
PDF
Kirill Rozin - Practical Wars for Automatization
PDF
Testing for fun in production Into The Box 2018
Make Your Testing Groovy
Verify Your Kubernetes Clusters with Upstream e2e tests
Altitude San Francisco 2018: Testing with Fastly Workshop
BeJUG Meetup - What's coming in the OSGi R7 Specification
2015 Q4 webrtc standards update
World-Class Test Automation: You Can Build It Too
Maximize the power of OSGi in AEM
Tim Panton - Presentation at Emerging Communications Conference & Awards (eCo...
How to lock a Python in a cage? Managing Python environment inside an R project
New and cool in OSGi R7 - David Bosschaert & Carsten Ziegeler
AI For Software Engineering: Two Industrial Experience Reports
Integration tests: use the containers, Luke!
GeeCON 2017 - TestContainers. Integration testing without the hassle
CI/CD Pipeline to Deploy and Maintain an OpenStack IaaS Cloud
DEVNET-1148 Leveraging Cisco OpenStack Private Cloud for Developers
PVS-Studio Is Now in Chocolatey: Checking Chocolatey under Azure DevOps
Mastering the Sling Rewriter by Justin Edelson
Mastering the Sling Rewriter
Kirill Rozin - Practical Wars for Automatization
Testing for fun in production Into The Box 2018

More from Paul King (18)

PDF
awesome groovy
PDF
groovy databases
PDF
tictactoe groovy
PDF
groovy rules
PDF
functional groovy
PDF
Agile Testing Practices
PDF
groovy DSLs from beginner to expert
PDF
concurrency with GPars
PDF
groovy and concurrency
PDF
GroovyDSLs
PDF
Atlassian Groovy Plugins
PDF
Dynamic Language Practices
PDF
Make Your Builds More Groovy
PDF
Groovy Power Features
PDF
Groovy Testing Sep2009
PDF
Craig Smith & Paul King Agile Tool Hacking Taking Your Agile Development ...
PDF
Groovy Tutorial
PDF
XML and Web Services with Groovy
awesome groovy
groovy databases
tictactoe groovy
groovy rules
functional groovy
Agile Testing Practices
groovy DSLs from beginner to expert
concurrency with GPars
groovy and concurrency
GroovyDSLs
Atlassian Groovy Plugins
Dynamic Language Practices
Make Your Builds More Groovy
Groovy Power Features
Groovy Testing Sep2009
Craig Smith & Paul King Agile Tool Hacking Taking Your Agile Development ...
Groovy Tutorial
XML and Web Services with Groovy

Recently uploaded (20)

PPTX
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
PDF
Reach Out and Touch Someone: Haptics and Empathic Computing
PPTX
PA Analog/Digital System: The Backbone of Modern Surveillance and Communication
PPTX
A Presentation on Artificial Intelligence
PDF
Machine learning based COVID-19 study performance prediction
PPTX
Digital-Transformation-Roadmap-for-Companies.pptx
PDF
The Rise and Fall of 3GPP – Time for a Sabbatical?
PDF
Modernizing your data center with Dell and AMD
PDF
Encapsulation_ Review paper, used for researhc scholars
PPTX
Big Data Technologies - Introduction.pptx
PDF
Chapter 3 Spatial Domain Image Processing.pdf
PDF
Shreyas Phanse Resume: Experienced Backend Engineer | Java • Spring Boot • Ka...
PDF
Encapsulation theory and applications.pdf
PDF
KodekX | Application Modernization Development
PDF
Build a system with the filesystem maintained by OSTree @ COSCUP 2025
PDF
CIFDAQ's Market Insight: SEC Turns Pro Crypto
PDF
How UI/UX Design Impacts User Retention in Mobile Apps.pdf
DOCX
The AUB Centre for AI in Media Proposal.docx
PDF
Electronic commerce courselecture one. Pdf
PDF
Agricultural_Statistics_at_a_Glance_2022_0.pdf
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
Reach Out and Touch Someone: Haptics and Empathic Computing
PA Analog/Digital System: The Backbone of Modern Surveillance and Communication
A Presentation on Artificial Intelligence
Machine learning based COVID-19 study performance prediction
Digital-Transformation-Roadmap-for-Companies.pptx
The Rise and Fall of 3GPP – Time for a Sabbatical?
Modernizing your data center with Dell and AMD
Encapsulation_ Review paper, used for researhc scholars
Big Data Technologies - Introduction.pptx
Chapter 3 Spatial Domain Image Processing.pdf
Shreyas Phanse Resume: Experienced Backend Engineer | Java • Spring Boot • Ka...
Encapsulation theory and applications.pdf
KodekX | Application Modernization Development
Build a system with the filesystem maintained by OSTree @ COSCUP 2025
CIFDAQ's Market Insight: SEC Turns Pro Crypto
How UI/UX Design Impacts User Retention in Mobile Apps.pdf
The AUB Centre for AI in Media Proposal.docx
Electronic commerce courselecture one. Pdf
Agricultural_Statistics_at_a_Glance_2022_0.pdf

Make Testing Groovy

  • 1. objectcomputing.com© 2017, Object Computing, Inc. (OCI). All rights reserved. No part of these notes may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior, written permission of Object Computing, Inc. (OCI) Make your testing Groovy GR8Conf US 2017 Paul King code: https://github.com/paulk-asert/MakeTestingGroovy slides: https://www.slideshare.net/paulk_asert/make-testing-groovy
  • 3. Range of testing scenarios Simple component Unit Testing Multiple Pieces Your lete comp- system Integration Testing Acceptance Testing Test System under test
  • 5. ©paulk_asert2006-2017 Understanding testing with a DSL Unit Utilities AcceptanceIntegration Tests Test Runner/Framework Driver
  • 6. ©paulk_asert2006-2017 Understanding testing with a DSL Unit Utilities AcceptanceIntegration Tests Test Runner/Framework Driver Testing DSL
  • 8. ©paulk_asert2006-2017 Why test with Groovy? It depends … … but many great reasons! ☺ TL;DR • Feature-rich frameworks which encourage easy to maintain tests • Script all things (devops nirvana) • Open source avoids being locked-in to vendor acceptance testing tools • Great for writing testing DSLs • Access to huge range of utilitiy libraries
  • 9. ©paulk_asert2006-2017 Why Groovy? Unit AcceptanceIntegration Consider audience/”fit” Utilities Tests Test Runner/Framework Driver Testing DSL
  • 10. ©paulk_asert2006-2017 “People Fit” / “Organization Fit” People • Developers (familiarity with languages) • Testers (tools language familiarity) • Responsibility to create/run/maintain • BAs/SMEs (ability to read technical tests) • Consider both development and maintenance • Expected feedback from tests (JUnit/Reports) Organization • Maturity level • Degree of automation • Tools in use • Need for adaptability
  • 11. ©paulk_asert2006-2017 Why Groovy? Unit AcceptanceIntegration Utilities Tests Test Runner/Framework Driver Testing DSL Very flexible utilities available: reading excel regex for reading text combinations
  • 12. ©paulk_asert2006-2017 Why Groovy? Unit AcceptanceIntegration Utilities Tests Test Runner/Framework Driver Testing DSL Great DSL support
  • 13. ©paulk_asert2006-2017 Why Groovy? Unit AcceptanceIntegration Utilities Tests Test Runner/Framework Driver Testing DSL Less brittle “glue” code
  • 14. ©paulk_asert2006-2017 Why Groovy? Unit AcceptanceIntegration Utilities Tests Test Runner/Framework Driver Testing DSL Great Frameworks: JUnit, TestNG, Spock Data-driven, property-based
  • 15. ©paulk_asert2006-2017 Why Groovy? Unit AcceptanceIntegration Utilities Tests Test Runner/Framework Driver Testing DSL Many Java/Groovy drivers: Geb, HtmlUnit, Selenium
  • 16. ©paulk_asert2006-2017 Looking at testing frameworks Unit AcceptanceIntegration Utilities Tests Test Runner/Framework Driver Testing DSL
  • 17. Testing Frameworks None JUnit 3 JUnit 4 JUnit 5 TestNG Spock
  • 18. Testing Frameworks None ??? JUnit 3 JUnit 4 JUnit 5 TestNG Spock
  • 19. Testing Frameworks None ??? • Groovy deems testing so important that it comes with built-in testing: • Built-in asserts, mocks • Built-in JUnit 3 GroovyTestCase and utilities • Built-in runners for tests • Bundles JUnit 4 JUnit 3 JUnit 4 JUnit 5 TestNG Spock
  • 20. ©paulk_asert2006-2017 Built-in assertions class Converter { static celsius (fahrenheit) { (fahrenheit - 32) * 5 / 9 } }
  • 22. ©paulk_asert2006-2017 Built-in assertions import static Converter.celsius assert 20 == celsius(68) assert 35 == celsius(95) assert -17 == celsius(0).toInteger() assert 0 == celsius(32) class Converter { static celsius (fahrenheit) { (fahrenheit - 32) * 5 / 9 } }
  • 23. ©paulk_asert2006-2017 Built-in assertions But what about errors? Groovy’s power Assert mechanism gives a friendly description of what went wrong
  • 24. ©paulk_asert2006-2017 Built-in assertions But what about errors? Groovy’s power Assert mechanism gives a friendly description of what went wrong
  • 25. ©paulk_asert2006-2017 GroovyTestCase (extends JUnit 3) import org.junit.Assert import static Converter.celsius import static org.junit.Assert.assertEquals class SimpleGroovyTest extends GroovyTestCase { void testShouldConvert() { assert Converter.celsius(68) == 20 assert Converter.celsius(212) == 100, "Should convert boiling" assertEquals("Should convert freezing", 0.0, celsius(32.0)) assertEquals("Should convert nearly freezing", 0.0, celsius(32.1), 0.1) } }
  • 26. ©paulk_asert2006-2017 JUnit4 import org.junit.Test import static org.junit.Assert.assertEquals import static Converter.celsius class SimpleJUnit4Test { @Test void shouldConvert() { assert celsius(68) == 20 assert celsius(212) == 100, "Should convert boiling" assertEquals("Should convert freezing", 0.0, celsius(32.0)) assertEquals("Also for nearly freezing", 0.0, celsius(32.1), 0.1) } }
  • 27. ©paulk_asert2006-2017 JUnit5 @Grab('org.junit.platform:junit-platform-runner:1.0.0-M5') @Grab('org.junit.jupiter:junit-jupiter-engine:5.0.0-M5') import org.junit.jupiter.api.Test import org.junit.platform.runner.JUnitPlatform import org.junit.runner.RunWith import static Converter.celsius @RunWith(JUnitPlatform) class ConverterJUnit5Tests { @Test void freezing() { assert celsius(32) == 0 } @Test void boiling() { assert celsius(212) == 100 } }
  • 28. ©paulk_asert2006-2017 JUnit5 @Grab('org.junit.platform:junit-platform-runner:1.0.0-M5') @Grab('org.junit.jupiter:junit-jupiter-engine:5.0.0-M5') import org.junit.jupiter.api.Test import org.junit.platform.runner.JUnitPlatform import org.junit.runner.RunWith import static Converter.celsius @RunWith(JUnitPlatform) class ConverterJUnit5Tests { @Test void freezing() { assert celsius(32) == 0 } @Test void boiling() { assert celsius(212) == 100 } }
  • 29. ©paulk_asert2006-2017 Spock - BDD style @Grab('org.spockframework:spock-core:1.1-groovy-2.4') import spock.lang.Specification class GivenWhenThenSpec extends Specification { def "test adding a new item to a set"() { given: def items = [4, 6, 3, 2] as Set when: items << 1 then: items.size() == 5 } }
  • 30. ©paulk_asert2006-2017 Parameterized import org.junit.Test import org.junit.runner.RunWith import org.junit.runners.Parameterized import org.junit.runners.Parameterized.Parameters import static Converter.celsius @RunWith(Parameterized) class DataDrivenJUnitTest { private c, f, scenario @Parameters static scenarios() {[ [0, 32, 'Freezing'], [20, 68, 'Garden party conditions'], [35, 95, 'Beach conditions'], [100, 212, 'Boiling'] ]*.toArray()} DataDrivenJUnitTest(c, f, scenario) this.c = c this.f = f this.scenario = scenario } @Test void convert() { def actual = celsius(f) def msg = "$scenario: ${f}°F should convert into ${c}°C" assert c == actual, msg } }
  • 31. ©paulk_asert2006-2017 Spock @Grab('org.spockframework:spock-core:1.1-groovy-2.4') import spock.lang.* import static Converter.celsius class SpockDataDriven extends Specification { def "test temperature scenarios"() { expect: celsius(tempF) == tempC where: scenario | tempF || tempC 'Freezing' | 32 || 0 'Garden party conditions' | 68 || 20 'Beach conditions' | 95 || 35 'Boiling' | 212 || 100 } }
  • 32. ©paulk_asert2006-2017 Spock - Celsius @Grab('org.spockframework:spock-core:1.1-groovy-2.4') import spock.lang.* import static Converter.celsius class SpockDataDriven extends Specification { @Unroll def "Scenario #scenario: #tempFºF should convert to #tempCºC"() { expect: celsius(tempF) == tempC where: scenario | tempF || tempC 'Freezing' | 32 || 0 'Garden party conditions' | 68 || 20 'Beach conditions' | 95 || 34 'Boiling' | 212 || 100 } }
  • 33. ©paulk_asert2006-2017 Spock - Celsius @Grab('org.spockframework:spock-core:1.1-groovy-2.4') import spock.lang.* import static Converter.celsius class SpockDataDriven extends Specification { @Unroll def "Scenario #scenario: #tempFºF should convert to #tempCºC"() { expect: celsius(tempF) == tempC where: scenario | tempF || tempC 'Freezing' | 32 || 0 'Garden party conditions' | 68 || 20 'Beach conditions' | 95 || 34 'Boiling' | 212 || 100 } }
  • 34. Assertion frameworks None • Groovy’s built-in assert JUnit 3 assertions • assertEquals() Dedicated library • Hamcrest • FEST • AssertJ • Google truth assertThat(googleColors).containsNoneOf(PINK, BLACK, WHITE, ORANGE) assert ![PINK, BLACK, WHITE, ORANGE].any { color -> color in googleColors }  usually usually
  • 35. Assertion frameworks None • Groovy’s built-in assert JUnit 3 assertions • assertEquals() Dedicated library • Hamcrest • FEST • AssertJ • Google truth assertThat(googleColors).containsNoneOf(PINK, BLACK, WHITE, ORANGE) assert [PINK, BLACK, WHITE, ORANGE].every { color -> color !in googleColors }  usually usually
  • 36. ©paulk_asert2006-2017 Looking at web drivers Unit AcceptanceIntegration Utilities Tests Test Runner/Framework Driver Testing DSL
  • 41. Case Study def html = new URL('http://localhost:8080').text assert html.contains('<title>Welcome to SimpBlog</title>') html.find(~'<title>(.*)</title>') { all, title -> assert title == 'Welcome to SimpBlog' }
  • 42. Case Study def page = new XmlSlurper().parse('http://localhost:8080/viewPost?id=1') assert page.body.h1.text().contains('Christmas') assert page.body.h3[1].text() == 'Category: Home' assert page.body.h3[2].text() == 'Author: Bart' assert page.body.table.tr.td.p.text() == "Aren't we forgeting the true meaning of this day? You know, the birth of Santa."
  • 43. Case Study @Grab('net.sourceforge.nekohtml:nekohtml:1.9.22') import org.cyberneko.html.parsers.SAXParser def parser = new XmlSlurper(new SAXParser()) def page = parser.parse('http://localhost:8080/viewPost?id=1') assert page.BODY.H1.text().contains('Christmas') assert page.BODY.H3[1].text() == 'Category: Home' assert page.BODY.H3[2].text() == 'Author: Bart' assert page.BODY.TABLE.TBODY.TR.TD.P.text() == "Aren't we forgeting the true meaning of this day? You know, the birth of Santa."
  • 45. HtmlUnit GUI-Less browser for Java Lets you retrieve pages, click links, fill out forms, etc.
  • 46. HtmlUnit class TestSimpBlogJUnit4 { def page @Before void setUp() { page = new WebClient().getPage('http://localhost:8080/postForm') assert 'Welcome to SimpBlog' == page.titleText } @Test void bartWasHere() { // fill in blog entry and post it def form = page.getFormByName('post') form.getInputByName('title').setValueAttribute('Bart was here (HtmlUnit JUnit4)') form.getSelectByName('category').getOptions().find { it.text == 'School' }.setSelected(true) form.getTextAreaByName('content').setText('Cowabunga Dude!') def result = form.getInputByName('btnPost').click() // check blog post details assert result.getElementsByTagName('h1').item(0). textContent.matches('Post.*: Bart was here.*') def h3headings = result.getElementsByTagName('h3') assert h3headings.item(1).textContent == 'Category: School' assert h3headings.item(2).textContent == 'Author: Bart' // expecting: <table><tr><td><p>Cowabunga Dude!</p>...</table> def cell = result.getByXPath('//TABLE//TR/TD')[0] assert cell.textContent.trim() == 'Cowabunga Dude!' } }
  • 48. Geb @Grab("org.gebish:geb-core:1.1.1") @Grab("org.seleniumhq.selenium:selenium-chrome-driver:3.4.0") @Grab("org.seleniumhq.selenium:selenium-support:3.4.0") import geb.Browser Browser.drive { go 'http://localhost:8080/postForm' assert title == 'Welcome to SimpBlog' $("form").with { title = "Bart was here (Geb)" author = 'Bart' category = 'School' content = "Cowabunga Dude!" btnPost().click() } assert $("h1").text().matches("Post d+: Bart was here.*") assert $("h3")[1].text() == 'Category: School' assert $("h3")[2].text() == 'Author: Bart' assert $("p").text() == "Cowabunga Dude!" }
  • 49. Geb with pages class NewPostPage extends Page { static url = "http://localhost:8080/postForm" static at = { title == 'Welcome to SimpBlog' } static content = { blogTitle { $("form").title() } // !title blogger { $("form").author() } label { $("form").category() } blogText { $("form").content() } // !content post(to: ViewPostPage) { btnPost() } } } class ViewPostPage extends Page { static at = { $("h1").text().contains('Post') } static content = { mainHeading { $("h1").text() } categoryHeading { $("h3")[1].text() } authorHeading { $("h3")[2].text() } blogText { $("p").text() } } }
  • 50. Geb with pages class NewPostPage extends Page { static url = "http://localhost:8080/postForm" static at = { title == 'Welcome to SimpBlog' } static content = { blogTitle { $("form").title() } // !title blogger { $("form").author() } label { $("form").category() } blogText { $("form").content() } // !content post(to: ViewPostPage) { btnPost() } } } class ViewPostPage extends Page { static at = { $("h1").text().contains('Post') } static content = { mainHeading { $("h1").text() } categoryHeading { $("h3")[1].text() } authorHeading { $("h3")[2].text() } blogText { $("p").text() } } } Browser.drive { to NewPostPage assert at(NewPostPage) blogTitle.value 'Bart was here (Geb pages)' blogger.value 'Bart' label.value 'School' blogText.value 'Cowabunga Dude!' post.click() assert at(ViewPostPage) assert mainHeading ==~ "Post d+: Bart was here.*" assert categoryHeading == 'Category: School' assert authorHeading == 'Author: Bart' assert blogText == "Cowabunga Dude!" }
  • 56. Case Study: Cucumber //… Given(~/^we are on the create blog entry page$/) { -> tester = new BlogTester('http://localhost:8080/postForm') tester.checkTitle 'Welcome to SimpBlog' } When(~/^I have entered '([^']*)' as the title$/) { String title -> formFields.title = title } When(~/^I have entered '([^']*)' into the content$/) { String content -> formFields.content = content } When(~/^I have selected '([^']*)' as the category$/) { String category -> formFields.category = category } When(~/^I click the 'Create Post' button$/) { -> tester.postBlog(formFields) } Then(~/^I expect the entry to be posted$/) { -> tester.checkHeadingMatches formFields.title tester.checkSubheading 'Category', formFields.category tester.checkPostText formFields.content tester.checkSubheading 'Author', formFields.author }
  • 57. //… Given(~/^we are on the create blog entry page$/) { -> tester = new BlogTester('http://localhost:8080/postForm') tester.checkTitle 'Welcome to SimpBlog' } When(~/^I have entered '([^']*)' as the title$/) { String title -> formFields.title = title } When(~/^I have entered '([^']*)' into the content$/) { String content -> formFields.content = content } When(~/^I have selected '([^']*)' as the category$/) { String category -> formFields.category = category } When(~/^I click the 'Create Post' button$/) { -> tester.postBlog(formFields) } Then(~/^I expect the entry to be posted$/) { -> tester.checkHeadingMatches formFields.title tester.checkSubheading 'Category', formFields.category tester.checkPostText formFields.content tester.checkSubheading 'Author', formFields.author } Feature: Use the blogging system Scenario: Bart posts a new blog entry Given we are on the create blog entry page When I have entered 'Bart was here (Cuke Groovy)' as the title And I have entered 'Cowabunga Dude!' into the content And I have selected 'Home' as the category And I have selected 'Bart' as the author And I click the 'Create Post' button Then I expect the entry to be posted Case Study: Cucumber
  • 58. Case Study: Cucumber //… Given(~/^we are on the create blog entry page$/) { -> tester = new BlogTester('http://localhost:8080/postForm') tester.checkTitle 'Welcome to SimpBlog' } When(~/^I have entered '([^']*)' as the title$/) { String title -> formFields.title = title } When(~/^I have entered '([^']*)' into the content$/) { String content -> formFields.content = content } When(~/^I have selected '([^']*)' as the category$/) { String category -> formFields.category = category } When(~/^I click the 'Create Post' button$/) { -> tester.postBlog(formFields) } Then(~/^I expect the entry to be posted$/) { -> tester.checkHeadingMatches formFields.title tester.checkSubheading 'Category', formFields.category tester.checkPostText formFields.content tester.checkSubheading 'Author', formFields.author }
  • 59. Case Study: Slim/FitNesse package simpblog import com.gargoylesoftware.htmlunit.WebClient class NewBlogPost { String author, title, content, category private result def execute() { def client = new WebClient() def page = client.getPage('http://localhost:8080/postForm') assert 'Welcome to SimpBlog' == page.titleText def form = page.getFormByName('post') form.getInputByName('title'). setValueAttribute("$title (Slim)") form.getSelectByName('author').getOptions().find{ it.text == author }.setSelected(true) form.getSelectByName('category').getOptions().find{ it.text == category }.setSelected(true) form.getTextAreaByName('content').setText(content) result = form.getInputByName('btnPost').click() } def mainHeading() { def m = result.getElementsByTagName('h1').item(0).textContent =~ /Post .*: (.*) ([^)]*)/ m[0][1] }
  • 60. Case Study: Slim/FitNesse package simpblog import com.gargoylesoftware.htmlunit.WebClient class NewBlogPost { String author, title, content, category private result def execute() { def client = new WebClient() def page = client.getPage('http://localhost:8080/postForm') assert 'Welcome to SimpBlog' == page.titleText def form = page.getFormByName('post') form.getInputByName('title'). setValueAttribute("$title (Slim)") form.getSelectByName('author').getOptions().find{ it.text == author }.setSelected(true) form.getSelectByName('category').getOptions().find{ it.text == category }.setSelected(true) form.getTextAreaByName('content').setText(content) result = form.getInputByName('btnPost').click() } def mainHeading() { def m = result.getElementsByTagName('h1').item(0).textContent =~ /Post .*: (.*) ([^)]*)/ m[0][1] }
  • 61. ©paulk_asert2006-2017 Looking at testing utilities Unit AcceptanceIntegration Utilities Tests Test Runner/Framework Driver Testing DSL
  • 62. Testing DSLs Low-level “DSL/fluent API” Medium-level DSL Higher-level DSL post blog from Bart with title "Bart rulz!" and category School and content "Cowabunga Dude!" def form = page.getFormByName('post') form.getInputByName('title').setValueAttribute('Bart was here (HtmlUnit JUnit4)') form.getSelectByName('category').getOptions().find { it.text == 'School' }.setSelected(true)
  • 63. DSLs: Fluent API class BlogTestCase extends GroovyTestCase { def page def lastResult void setUp() { page = new WebClient().getPage('http://localhost:8080/postForm') } def checkTitle(String title) { assert title == page.titleText } def prepareBlog() { new PrepareBlogEmpty() } // ... }
  • 64. DSLs: Fluent API // ... def postBlog(Map params) { def form = page.getFormByName('post') form.getInputByName('title').setValueAttribute(params.title) form.getSelectByName('category').options.find { it.text == params.category }.setSelected(true) form.getSelectByName('author').options.find { it.text == params.author }.setSelected(true) form.getTextAreaByName('content').setText(params.content) lastResult = form.getInputByName('btnPost').click() } def checkHeadingMatches(String regex) { def heading = lastResult.getElementsByTagName('h1').item(0) assert heading.textContent.matches(regex) } // ... }
  • 65. DSLs: Fluent API class TestSimpBlogFluentApi extends BlogTestCase { void setUp() { super.setUp() checkTitle('Welcome to SimpBlog') } void testBartWasHere() { prepareBlog() .withTitle('Bart was here (HtmlUnit FluentApi)') .withAuthor('Bart') .withCategory('School') .withContent('Cowabunga Dude!') .post() checkHeadingMatches 'Post.*: Bart was here.*' checkSubheading 1, 'Category: School' checkSubheading 2, 'Author: Bart' checkPostText 'Cowabunga Dude!' }
  • 66. DSLs: command chains class TestSimpBlogDsl extends BlogTestCase { private the, has, a, with, heading void setUp() { super.setUp() } def post(_a) { [ blog: { _with -> [title: { postTitle -> [and: { __with -> [author: { postAuthor -> [and: { ___with -> [category: { postCategory -> [and: { ____with -> [content: { postContent -> postBlog(title: postTitle, author:postAuthor, content:postContent, category: postCategory) } ]} ]} ]} ]} ]} ]} ]} ] } def check(_the) { [ browser: { _has -> [title: { checkTitle it }]}, main: { _heading -> [matches: { checkHeadingMatches it }]}, category: { _has -> [value: { checkSubheading 1, "Category: $it" }]}, author: { _has -> [value: { checkSubheading 2, "Author: $it" }]}, blog: { _has -> [text: { checkPostText it }]} ] } // ...
  • 67. DSLs: command chains // ... void testBartWasHere() { check the browser has title 'Welcome to SimpBlog' post a blog with title 'Bart was here (HtmlUnit DSL)' and with author 'Bart' and with category 'School' and with content 'Cowabunga Dude!' check the main heading matches 'Post.*: Bart was here.*' check the category has value 'School' check the author has value 'Bart' check the blog has text 'Cowabunga Dude!' } }
  • 70. Testing Utilities All combinations All pairs Property-based testing GPars Constraint programming ModelJUnit
  • 74. All Pairs Case Study
  • 75. ©paulk_asert2006-2017 Property-based testing Agile testing game (TDD) • Minimum test code to steer design of minimal production code with desired business functionality but 100% code coverage • “Grey box” testing • Rarely used with functional programming
  • 76. ©paulk_asert2006-2017 Property-based testing Agile testing game (TDD) • Minimum test code to steer design of minimal production code with desired business functionality but 100% code coverage • “Grey box” testing • Rarely used with functional programming • Instead validate certain properties
  • 77. ©paulk_asert2006-2017 Property-based testing @Grab('net.java.quickcheck:quickcheck:0.6') import static net.java.quickcheck.generator.PrimitiveGenerators.* import static java.lang.Math.round import static Converter.celsius def gen = integers(-40, 240) def liquidC = 0..100 def liquidF = 32..212 100.times { int f = gen.next() int c = round(celsius(f)) assert c <= f assert c in liquidC == f in liquidF }
  • 78. ©paulk_asert2006-2017 Property-based testing with Spock https://github.com/Bijnagte/spock-genesis
  • 79. ©paulk_asert2006-2017 Property-based testing: spock genesis @Grab('com.nagternal:spock-genesis:0.6.0') @GrabExclude('org.codehaus.groovy:groovy-all') import spock.genesis.transform.Iterations import spock.lang.Specification import static Converter.celsius import static java.lang.Math.round import static spock.genesis.Gen.integer class ConverterSpec extends Specification { def liquidC = 0..100 def liquidF = 32..212 @Iterations(100) def "test phase maintained"() { given: int tempF = integer(-40..240).iterator().next() when: int tempC = round(celsius(tempF)) then: tempC <= tempF tempC in liquidC == tempF in liquidF } …
  • 80. ©paulk_asert2006-2017 Property-based testing: spock genesis @Grab('com.nagternal:spock-genesis:0.6.0') @GrabExclude('org.codehaus.groovy:groovy-all') import spock.genesis.transform.Iterations import spock.lang.Specification import static Converter.celsius import static java.lang.Math.round import static spock.genesis.Gen.integer class ConverterSpec extends Specification { def liquidC = 0..100 def liquidF = 32..212 @Iterations(100) def "test phase maintained"() { given: int tempF = integer(-40..240).iterator().next() when: int tempC = round(celsius(tempF)) then: tempC <= tempF tempC in liquidC == tempF in liquidF } … … @Iterations(100) def "test order maintained"() { given: int tempF1 = integer(-273..999).iterator().next() int tempF2 = integer(-273..999).iterator().next() when: int tempC1 = round(celsius(tempF1)) int tempC2 = round(celsius(tempF2)) then: (tempF1 <=> tempF2) == (tempC1 <=> tempC2) } }
  • 84. GPars Library classes and DSL allowing you to handle tasks concurrently: • Data Parallelism map, filter, reduce functionality in parallel with parallel array support • Asynchronous functions extend the Java executor services to enable multi-threaded closure processing • Dataflow Concurrency supports natural shared-memory concurrency model, using single-assignment variables • Actors provide Erlang/Scala-like actors including "remote" actors on other machines • Safe Agents provide a non-blocking mt-safe reference to mutable state; like "agents" in Clojure 89
  • 85. Case Study with GPars //@Grab('org.codehaus.gpars:gpars:0.12') import groovyx.gpars.GParsPool def testCases = [ ['Title 1 (GPars)', 'Home', 'Bart', 'Content 1'], ['Title 2 (GPars)', 'Work', 'Homer', 'Content 2'], ['Title 3 (GPars)', 'Travel', 'Marge', 'Content 3'], ['Title 4 (GPars)', 'Food', 'Lisa', 'Content 4'] ] GParsPool.withPool { testCases.eachParallel{ title, category, author, content -> postAndCheck title, category, author, content } } def postAndCheck(String title, String category, String author, String content) { def tester = new BlogTester('http://localhost:8080/postForm') tester.postAndCheck title, category, author, content }
  • 86. Case Study with GPars //@Grab('org.codehaus.gpars:gpars:0.12') import groovyx.gpars.GParsPool def testCases = [ ['Title 1 (GPars)', 'Home', 'Bart', 'Content 1'], ['Title 2 (GPars)', 'Work', 'Homer', 'Content 2'], ['Title 3 (GPars)', 'Travel', 'Marge', 'Content 3'], ['Title 4 (GPars)', 'Food', 'Lisa', 'Content 4'] ] GParsPool.withPool { testCases.eachParallel{ title, category, author, content -> postAndCheck title, category, author, content } } def postAndCheck(String title, String category, String author, String content) { def tester = new BlogTester('http://localhost:8080/postForm') tester.postAndCheck title, category, author, content }
  • 87. Constraint/Logic Programming Description • Style of programming where relations between variables are stated in the form of constraints • First made popular by logic programming languages such as Prolog but the style is now also used outside logic programming specific languages • Constraints differ from the common primitives of other programming languages in that they do not specify one or more steps to execute but rather the properties of a solution to be found • Popular libraries used with Groovy supporting constraint programming include Gecode/J, Choco and tuProlog • We'll look at Choco as an example
  • 88. Case Study with Constraint Programming You have been asked to set up some test cases representing the Simpsons’ weekly blogging habits After some careful study you observe the following strange behavior • They never blog on the same day • Marge blogs only on a Saturday or Sunday • Maggie blogs only on a Tuesday or Thursday • Lisa blogs only on a Monday, Wednesday or Friday • Bart blogs only on the day after Lisa • Homer only blogs if noone else blogged the previous day and doesn't allow anyone to blog the next day
  • 89. Case Study with Constraint Programming //@Grab('org.choco-solver:choco-solver:4.0.4') import org.chocosolver.solver.Model def m = new Model() daysOfWeek = ["Sunday", "Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday"] def (SUN, MON, TUE, WED, THU, FRI, SAT) = 0..6 def bart = m.intVar('Bart', 0, 6) def homer = m.intVar('Homer', 0, 6) def marge = m.intVar('Marge', 0, 6) def lisa = m.intVar('Lisa', 0, 6) def maggie = m.intVar('Maggie', 0, 6) def authors = [bart, homer, marge, lisa, maggie] //…
  • 90. Case Study with Constraint Programming // They never blog on the same day m.allDifferent(*authors).post() // Marge blogs only on a Saturday or Sunday m.or(m.arithm(marge, "=", SAT), m.arithm(marge, "=", SUN)).post() // Maggie blogs only on a Tuesday or Thursday m.or(m.arithm(maggie, "=", TUE), m.arithm(maggie, "=", THU)).post() // Lisa blogs only on a Monday, Wednesday or Friday m.or(m.arithm(lisa, "=", MON), m.arithm(lisa, "=", WED), m.arithm(lisa, "=", FRI)).post() // Bart blogs only on the day after Lisa m.arithm(bart, "-", lisa, "=", 1).post() // Homer only blogs if noone else blogged the previous // day and doesn't allow anyone to blog the next day m.and(m.distance(homer, marge, "!=", 1), m.distance(homer, bart, "!=", 1), m.distance(homer, maggie, "!=", 1), m.distance(homer, lisa, "!=", 1)).post() //…
  • 91. Case Study with Constraint Programming def solutions = [] while (m.solver.solve()) { solutions << pad('') + authors.collect { pad(daysOfWeek[it.value]) }.join() } if (solutions) { println pad("Solutions:") + authors.collect { pad(it.name) }.join() println solutions.join('n') } else { println "No Solutions" } def pad(s) { s.padRight(12) }Solutions: Bart Homer Marge Lisa Maggie Thursday Sunday Saturday Wednesday Tuesday Thursday Saturday Sunday Wednesday Tuesday Saturday Tuesday Sunday Friday Thursday Tuesday Saturday Sunday Monday Thursday
  • 95. Case Study with ModelJUnit
  • 96. Case Study with ModelJUnit
  • 97. Case Study with ModelJUnit
  • 98. Case Study with ModelJUnit
  • 99. Case Study with ModelJUnit
  • 101. ©paulk_asert2006-2017 Key Testing Practices… Use testing DSL’s Look to move up the testing stack • It used to be all about the driver • Now the driver is hidden in the framework or tool stack Apply good testing practices • Pareto analysis, bug clusters, mutation testing, test early, all pairs/equivalence partitions/orthogonal array testing, risk-based test selection, coding for testability, use CI, boundary value analysis, defensive programming