Whitebox testing of Spring Boot applications

About myself
Application Architect @Thomas
Cook
Tech Lead - Platformers team
9+ years Java

About myself
Application Architect @Thomas
Cook
Tech Lead - Platformers team
9+ years Java
*Very happy to leave an amazing teambuilding first to make it for a 5 AM flight back to Kyiv for this presentation

What is our current mission and vision in TC?
We are developing a flexible platform that will be able to make full use of CD
practices to improve time-to-market and be resilient enough to support a variety of
new markets.

Content
1. What we do in Thomas Cook
2. What problems we are facing with testing
3. Alternative frameworks - Spock and JUnit 5
4. Spring Boot testing

Mule -> Spring Boot
TestNG + JUnit 4

Well, at least it’s Spring-based!

TestNG issues
● Test class lifecycle

TestNG issues
● Test class lifecycle:
@BeforeMethod
private void beforeMethod() {
objectUnderTest = null;
MockitoAnnotations.initMocks(this);
….
}

TestNG issues
● Test class lifecycle:
@BeforeMethod
private void beforeMethod() {
objectUnderTest = null;
MockitoAnnotations.initMocks(this);
….
}
*Funnily enough you can use the same lifecycle for JUnit 5 with @TestInstance:
enum Lifecycle
PER_CLASS, PER_METHOD;

TestNG issues
● A problem with Mockito related to the way Mockito used Reflection:
https://github.com/mockito/mockito/issues/810
https://github.com/mockito/mockito/pull/948

Issues with JUnit 4
● Not possible to have several @RunWith

Issues with JUnit 4
● Not possible to have several @RunWith
For that reason a list of custom Rules was implemented on our side:
public class MockitoRule implements MethodRule {
@Override
public Statement apply(Statement base, FrameworkMethod frameworkMethod, Object test) {
settings.test = test;
Statement wrappedStatement = new MockitoInitializationStatement(base, settings);
wrappedStatement = new MockitoVerificationStatement(wrappedStatement, settings);
return wrappedStatement;
}

Spock: based on JUnit runner
No hassle with the runner: What’s more it extends Junit runner so it can run by
the tools you used for your tests before.
@RunWith(Sputnik.class)
public abstract class Specification extends MockingApi
….
public class Sputnik extends Runner implements Filterable, Sortable

Spock: formal semantics
JUnit tests lack formal semantics
The number one reason for using Spock is to make your tests more readable.

Spock: test blocks
Having them in your tests is mandatory.
Otherwise a piece of code like this will not complain:
def "CreateBooking"() {
whatever
}

Spock: test example
def "should fetch Bob and Alice without errors"() {
given:
def response =
mockMvc.perform(MockMvcRequestBuilders.get("/bookings/$id")).andReturn().response
def content = new JsonSlurper().parseText(response.contentAsString)
expect:
response.status == OK.value()
and:
content.passengerName == result
where:
id || result
'5' || "Bob"
'15' || "Alice"
}

Spock: Data Pipes
Can be as advances as this:

Spock: @Shared fields
Objects stored into instance fields are not shared between feature methods. Instead, every feature
method gets its own object. This helps to isolate feature methods from each other, which is often a
desirable goal.
@Shared res = new VeryExpensiveResource()

Spock: parameterization is way better than in JUnit 4
There is a way to do it for JUnit, but it’s an external lib. Otherwise, it’s just too
verbose

Spock: Interactions
A way to express which method invocations are expected to occur:
then: "a rejection email is sent to the customer"
0 * emailService.sendConfirmation(sampleCustomer.getEmailAddress())
1 * emailService.sendRejection(sampleCustomer.getEmailAddress())
//Can also be:
(1..3) * subscriber.receive("hello") // between one and three calls (inclusive)
(1.._) * subscriber.receive("hello") // at least one call
(_..3) * subscriber.receive("hello") // at most three calls

Spock: Interactions - powerful matchers
1 * subscriber./r.*e/("hello") // a method whose name matches the given regular expression
// (here: method name starts with 'r' and ends in 'e')
1 * subscriber.receive(_) // any single argument (including null)
1 * subscriber.receive(*_) // any argument list (including the empty argument list)
1 * _._ // any method call on any mock object
1 * _ // shortcut for and preferred over the above

Spock: Drawbacks
A bit scary versioning:

Groovy: Drawbacks
If everything inside your test/ folder will be Groovy based there’s a good
chance you will want to have some TestUtils/TestBuilders there at some
point. And you will use Groovy for all that.

Groovy: Drawbacks
Groovy: With great power...

Groovy: Drawbacks
It’s similar to java, but there will still be a learning curve and some tricky cases.
My use case - copying one object into another based on SO:
https://stackoverflow.com/questions/9072307/copy-groovy-class-properties/9072974#9072974

Groovy: Drawbacks
My use case - copying one object into another:
https://stackoverflow.com/questions/46952475/copy-object-properties-to-another-object-in-groovy

Groovy: Drawbacks
Comments/Answers:

JUnit 5: module composition
JUnit 5 is composed of several different modules from three different sub-projects.
JUnit 5 = JUnit Platform + JUnit Jupiter + JUnit Vintage

JUnit 5: native support in Spring 5.0
Spring-boot version 1.5.9-RELEASE is based on Spring 4 and the
SpringExtension is only available since Spring 5.

JUnit 5: can run on top of JUnit 4

JUnit 5: reason for re-architecture
● JUnit 4 test discovery and execution were tightly coupled.
● JUnit 4 - java 5 compatible - that’s ridiculous!

JUnit 5: architectural diagram

JUnit 5: all interested parties
This substantially simplifies collaboration between:
● JUnit framework developers;
● Developers writing tests for their products;
● IDE developers;
● Build tool developers.

JUnit 5: powerful ExtendWith
Only one @RunWith but many @ExtendWith.
@ExtendWith(SpringExtension.class)

JUnit 5: asserts are never skipped
Assert are not skipped if one failed:
void getBooking() throws Exception {
String bob = mockMvc.perform(MockMvcRequestBuilders.get("/bookings/5"))
.andReturn().getResponse().getContentAsString();
ObjectMapper objectMapper = new ObjectMapper();
assertAll("Analyzing Bob's record",
() -> assertEquals("NotBob", objectMapper.readValue(bob, Booking.class).getPassengerName()),
() -> assertEquals("USA", objectMapper.readValue(bob, Booking.class).getDestination()),
() -> assertEquals(Double.valueOf(35), objectMapper.readValue(bob, Booking.class).getAge()));
}

Assert are not skipped if one failed:
void getBooking() throws Exception {
ObjectMapper objectMapper = new ObjectMapper();
assertAll("Analyzing Bob's record",
() -> assertEquals("NotBob", objectMapper.readValue(bob, Booking.class).getPassengerName()),
() -> assertEquals("USA", objectMapper.readValue(bob, Booking.class).getDestination()),
() -> assertEquals(Double.valueOf(35), objectMapper.readValue(bob, Booking.class).getAge()));
}
Fail
Fail
Pass

Will require fewer iterations to stabilize your code

JUnit 5: out-of-the-box support for Exceptions
@org.junit.jupiter.api.Test
void testException() {
Executable closureContainingCodeToTest = () -> {throw new ConcurrentModificationException();};
assertThrows(ConcurrentModificationException.class, closureContainingCodeToTest);
}

JUnit 5: Autowiring into a method
@org.junit.jupiter.api.Test
void getBooking(@Autowired MockMvc mockMvc) throws Exception {

JUnit 5: Autowiring into a method
No need to create loads of fields in your test class.
Inject what is needed to a specific test method.

JUnit 5: Support for @Nested classes
@DisplayName("A stack")
class TestingAStackDemo {
@Test
@DisplayName("is instantiated with new Stack()")
void isInstantiatedWithNew() {
...
}
@Nested
@DisplayName("when new")
class WhenNew {
...
}

Spring Boot Testing: Overview
Spring Boot BOM(?) specifies following test-related
dependencies:
● spring-boot-test
● spring-boot-test-autoconfigure
● spring-boot-starter-test
● solr-test-framework
● groovy-test
● groovy-testng
● hibernate-testing
● spring-batch-test
● spring-kafka-test
● spring-ldap-test
● spring-restdocs-webtestclient
● spring-ws-test

Spring Boot BOM(spring-boot-dependencies pom) specifies
following test-related dependencies:
● groovy-test
● groovy-testng
● spring-ws-test

We will mainly focus on these ones:
● groovy-test
● groovy-testng
● spring-ws-test

Only one test-specific starter:

Spring Test: Integration Testing
Spring’s integration testing support has the following primary goals:
● To manage Spring IoC container caching between test execution.
● To provide Dependency Injection of test fixture instances.
● To provide transaction management appropriate to integration testing.
● To supply Spring-specific base classes that assist developers in writing integration
tests.

Spring Test: how are transactions handled?
Your tests might run against a real database. It’s rare, but possible. We are talking
about Integration tests after all.
By default, the framework will create and roll back a transaction for each test.
If a test method deletes the contents of selected tables while running within the
transaction managed for the test, the transaction will rollback by default, and the
database will return to its state prior to execution of the test.

More flexibility is available via @Commit and @Rollback

Not always that straightforward. Probably depends on implicit commits and
DB engines. Will also only work for MOCK transport.

Spring Test: main actors
TestContextManager TestContext
TestExecutionListeners

Spring Test: TestContextManager
Main entry point into Spring test related functionality
Spring
Test
Context

Spring Test: factories for ExecutionListener

Spring Test: factories for ExecutionListener
Will scan your test class, look at its annotations and add its logic accordingly.

Spring Test: Reactive support
The package org.springframework.mock.http.server.reactive contains mock
implementations of ServerHttpRequest and ServerHttpResponse for use in
WebFlux applications.
The WebTestClient builds on the mock request and response to provide support
for testing WebFlux applications without an HTTP server.

Spring Test: static cache
Test suites and forked processes
The Spring TestContext framework stores application contexts in a static cache.
This means that the context is literally stored in a static variable. In other words, if
tests execute in separate processes the static cache will be cleared between each
test execution, and this will effectively disable the caching mechanism.

Spring Boot + Spock: pom file
<plugin>
<groupId>org.codehaus.gmavenplus</groupId>
<artifactId>gmavenplus-plugin</artifactId>
<configuration>
<testSources>
<testSource>
<directory>${project.basedir}/src/test/java</directory>
<includes>
<include>**/*.groovy</include>
</includes>
</testSource>
</testSources>
</configuration>
<executions>
<execution>
<goals>
<goal>compile</goal>
<goal>compileTests</goal>
</goals>
</execution>
</executions>
</plugin>

<dependency>
<groupId>org.spockframework</groupId>
<artifactId>spock-core</artifactId>
<version>1.1-groovy-2.4</version>
<scope>test</scope>
</dependency>
<dependency>
<artifactId>spock-spring</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.codehaus.groovy</groupId>
<artifactId>groovy-all</artifactId>
<version>2.4.13</version>
</dependency>

<dependency>
<artifactId>spock-core</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<artifactId>spock-spring</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.codehaus.groovy</groupId>
<artifactId>groovy-all</artifactId>
</dependency>
Default scope

Spring Boot + Spock
Integration with Spock: Detached mocks via the DetachedMockFactory and
SpockMockFactoryBean classes.
class TestConfigurationForSpock {
private final detachedMockFactory = new DetachedMockFactory()
@Bean
BookingService bookingService() {
detachedMockFactory.Mock(BookingService);
}
}
Then just use: @Import(TestConfigurationForSpock)

Spring Boot + JUnit 5
Will need to include an additional library to use JUnit 5 with Spring Framework 4.3

There’s nothing at all Spock specific in spring-test, but you can find junit.jupiter is
there:

Be careful with your surefire version:

Spring Boot + JUnit 5: pom file
<dependency>
<artifactId>junit-platform-launcher</artifactId>
</dependency>
<dependency>
<artifactId>junit-jupiter-engine</artifactId>
</dependency>
<dependency>
<artifactId>junit-vintage-engine</artifactId>
</dependency>
<dependency>
<artifactId>junit-jupiter-api</artifactId>
</dependency>
<dependency>
<artifactId>junit-jupiter-params</artifactId>
</dependency>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<includes>
<include>**/Test*.java</include>
<include>**/*Test.java</include>
<include>**/*Tests.java</include>
<include>**/*TestCase.java</include>
</includes>
</configuration>
<dependencies>
<dependency>
<groupId>org.junit.platform</groupId>
<artifactId>junit-platform-surefire-provider</artifactId>
<version>${junit.platform.version}</version>
</dependency>
</dependencies>
</plugin>

Spock vs JUnit 5: integration with Spring
Both frameworks provide a class with exact same name: SpringExtension.
However, the name is the only thing they have in common.

Spock vs JUnit 5: integration with Spring
SpringExtension
Spock JUnit 5
Tries to find any Spring-specific annotation on a test
class(spec): ContentHierarchy, BootstrapWith,
ContextConfiguration.
Using Spring-provided MetaAnnotaionUtils, so that it
can traverse class hierarchy.
Spring Test SpringExtention implements all possible
JUnit 5 callbacks(.e.g.: BeforeAllCallback,
AfterAllCallback)
If found, creates a TestContextManager and
delegate to it.
Will either get TestContextManager from a store of
create one.
Attaches Spock-specific listener for managing
Mocks:
testContext.setAttribute(MOCKED_BEANS_LIST,
mockedBeans);
Will wrap TestContextManager calls and delegate to
it. E.g.:
public void beforeAll(ExtensionContext context) {
getTestContextManager(context).beforeTestClass();

Spring Boot: autoconfigure slicing
Auto-configured tests
The spring-boot-test-autoconfigure module includes a number of annotations that can be used to
automatically configure different ‘slices’ of your app for testing purposes.
Examples:
● @WebMvcTest
● @JsonTest
● @DataJpaTest
● @JdbcTest
● @DataMongoTest
● ...

@*Test would normally contain several @AutoConfigure* annotations:
For example:
...
@BootstrapWith(SpringBootTestContextBootstrapper.class)
@OverrideAutoConfiguration(enabled = false)
@TypeExcludeFilters(DataJpaTypeExcludeFilter.class)
@Transactional
@AutoConfigureCache
@AutoConfigureDataJpa
@AutoConfigureTestDatabase
@AutoConfigureTestEntityManager
@ImportAutoConfiguration
public @interface DataJpaTest

To tweak @*Test mechanism, you can use a corresponding @AutoConfigure*
annotation.
For example:
@RunWith(SpringRunner.class)
@DataJpaTest
@AutoConfigureTestDatabase(replace= AutoConfigureTestDatabase.Replace.NONE)
public class ExampleRepositoryTests {
// ...
}

Have you noticed this relatively unremarkable meta annotation:
@OverrideAutoConfiguration(enabled = false)
It will effectively set:
spring.boot.enableautoconfiguration=false
via AbstractTestContextBootstrapper

String Boot: Bootstrapping
@BootstrapWith is used on @SpringBootTest, for example.

@BootstrapWith is used on @SpringBootTest, for example.
● Aware of Spring Boot
application structure;
● Helps SpringBootTest;

As TestContextManager is the main entry point for test frameworks all they need
is to create one:
public TestContextManager(Class<?> testClass) {
this(BootstrapUtils.resolveTestContextBootstrapper(BootstrapUtils.createBootstrapContext(testClass)));
}

Spring Boot: Slicing
Do not litter the application’s main class with configuration settings that are are
specific to a particular area of its functionality.
Extract them into specific @Configuration instead. Otherwise they will be picked
up by all slice tests, which might not be what you want:
@SpringBootApplication
@EnableBatchProcessing - DO NOT DO IT THIS WAY
public class SampleApplication { ... }

Spring Boot: Custom component scanning
Another source of confusion is classpath scanning. Assume that, while you
structured your code in a sensible way, you need to scan an additional package.
Your application may resemble the following code:
@SpringBootApplication
@ComponentScan({ "com.example.app", "org.acme.another" }) - ALSO BAD
public class SampleApplication { ... }
This effectively overrides the default component scan directive with the side effect
of scanning those two packages regardless of the slice that you’ve chosen.

Spring Boot: TypeExcludeFilter
AutoConfigurationExcludeFilter - tells Spring Boot to exclude scanning
autoconfigurations. To use SpringFactoriesLoader instead.
TypeExcludeFilter - is an interesting case. While it’s in spring-boot jar, the doc
actually says: primarily used internally to support spring-boot-test.

Pivotal cares about your application’s tests so much, they put a test-specific
logic into their main Spring Boot module!

Indeed, all subclasses seem to be coming from spring-boot-test-autoconfigure and
spring-boot-test:

What a specific test filter effectively does is:
1. Loads a corresponding annotation. E.g.: WebMvcTest for
WebMvcTypeExcludeFilter, DataJpaTest for DataJpaTypeExcludeFilter;
2. Inside *ExcludeFilter add specific annotations for which you want to enable
ComponentScan. For most *ExcludeFilters it is not required, however. They
will simply rely on SpringFactoriesLoader

@TestConfiguration :
● Unlike a nested @Configuration class which would be used instead of a your
application’s primary configuration, a nested @TestConfiguration class will
be used in addition to your application’s primary configuration;
● When placed on a top-level class, @TestConfiguration indicates that classes
in src/test/java should not be picked up by scanning. Use explicit @Import to
use them.
Spring Boot: overriding configurations

Spring Boot: Mocking and spying beans
For example, you may have a facade over some remote service that’s unavailable
during development.
Spring Boot includes a @MockBean annotation that can be used to define a
Mockito mock for a bean inside your ApplicationContext.
@RunWith(SpringRunner.class)
@SpringBootTest
public class MyTests {
@MockBean
private RemoteService remoteService;

Spring Boot: AutoConfigureMockMvc
There is an option to not start the server at all, but test only the layer below that,
where Spring handles the incoming HTTP request and hands it off to your
controller. That way, almost the full stack is used, and your code will be called
exactly the same way as if it was processing a real HTTP request, but without the
cost of starting the server. To do that we will use Spring’s MockMvc, and we can
ask for that to be injected for us by using the @AutoConfigureMockMvc annotation
on the test case.
https://spring.io/guides/gs/testing-web/

Spring Boot: WebMvcTest
@WebMvcTest
Only web layer is instantiated, not the whole context.
Often @WebMvcTest will be limited to a single controller and used in combination
with @MockBean to provide mock implementations for required collaborators.

Spring Boot: WebMvcTest vs AutoConfigureMockMvc vs
SpringBootTest
The main difference is inside META-INF/spring.factories:
# AutoConfigureMockMvc auto-configuration imports
org.springframework.boot.test.autoconfigure.web.servlet.AutoConfigureMockMvc=
org.springframework.boot.test.autoconfigure.web.servlet.MockMvcAutoConfiguration,
org.springframework.boot.test.autoconfigure.web.servlet.MockMvcSecurityAutoConfiguration,
org.springframework.boot.test.autoconfigure.web.servlet.MockMvcWebClientAutoConfiguration,
org.springframework.boot.test.autoconfigure.web.servlet.MockMvcWebDriverAutoConfiguration
You’ll notice that WebMvcTest has also a @ImportAutoConfiguration *, but there’s
no entry in spring.factories for it.
https://spring.io/blog/2016/08/30/custom-test-slice-with-spring-boot-1-4

Spring Boot: WebMvcTest vs AutoConfigureMockMvc vs
SpringBootTest
The main difference is inside META-INF/spring.factories:
# AutoConfigureMockMvc auto-configuration imports
org.springframework.boot.test.autoconfigure.web.servlet.AutoConfigureMockMvc=
org.springframework.boot.test.autoconfigure.web.servlet.MockMvcAutoConfiguration,
org.springframework.boot.test.autoconfigure.web.servlet.MockMvcSecurityAutoConfiguration,
org.springframework.boot.test.autoconfigure.web.servlet.MockMvcWebClientAutoConfiguration,
org.springframework.boot.test.autoconfigure.web.servlet.MockMvcWebDriverAutoConfiguration
You’ll notice that WebMvcTest has also a @ImportAutoConfiguration *, but there’s
no entry in spring.factories for it.
However it has an exclude filter to reduce the amount of scanned objects.

Spring Boot: SpringBootTest MOCK WebEnvironment
Embedded servlet containers are not started when using this attribute. Can be
used in conjunction with @AutoConfigureMockMvc for MockMvc-based testing of
your application.

Spring Boot: SpringBootTest
RANDOM_PORT/DEFINED_PORT WebEnvironment
A test like this will run perfectly fine with real HTTP transport:
@SpringBootTest(classes = RamlBasedProducerApplication, webEnvironment =
SpringBootTest.WebEnvironment.RANDOM_PORT)
@AutoConfigureMockMvc
@OverrideAutoConfiguration(enabled = true)
class BookingServiceImplSpockTest extends Specification {
@LocalServerPort
private int port;
@Autowired
MockMvc mockMvc;
Spring will inject different ServletContexts into MockMvc depending on webEnv.

Spring Boot: Test utilities
● EnvironmentTestUtils(addEnvironment(env, "org=Spring", "name=Boot"))
● TestRestTemplate(behave in a test-friendly way by not throwing exceptions
on server-side errors)
● MockRestServiceServer(part of Spring Test)

Whitebox testing of Spring Boot applications

More Related Content

What's hot (20)

Similar to Whitebox testing of Spring Boot applications (20)

Recently uploaded (20)

Whitebox testing of Spring Boot applications

Editor's Notes