The Joy of Testing - Deep Dive @ Devoxx Belgium 2024
1 like688 views
The document provides an extensive overview of best practices in unit testing, covering topics such as test qualities, coverage management, mutation testing, and BDD methodologies. It emphasizes the importance of meaningful test names, the use of tools like AssertJ, and strategies for testing legacy code, alongside real-world examples from the speaker's extensive experience. The coding resources mentioned are available on GitHub, highlighting a hands-on approach to learning testing techniques.
![UT 14
Coverage %
Cost
Coverage%
100%
80% 90%
60%
= covrigi [RO]
cover critical code first
🤔 Higher goals for 'domain' package
Minimum Coverage%](https://image.slidesharecdn.com/thejoyoftestingdevoxx-241008182535-1b9be551/85/The-Joy-of-Testing-Deep-Dive-Devoxx-Belgium-2024-9-320.jpg)
![UT 35
Fix a bug like a Pro
1. Reproduce the bug [twice]
2. Write a test for it è FAILS❌
(⚠can be very-very hard in complex, terrible legacy code)
3. log.debug("..{}", var) instead of breakpoint
4. Find the cause > Kill bug > Test è GREEN✅
Avoid vola1le
breakpoint sessions
You can later lower the log level in producIon without restart😎:
using: JMX, Spring Actuator, or Logback config auto-reload to inves9gate unreproducible bugs](https://image.slidesharecdn.com/thejoyoftestingdevoxx-241008182535-1b9be551/85/The-Joy-of-Testing-Deep-Dive-Devoxx-Belgium-2024-28-320.jpg)
![UT 83
Gherkin Best Prac0ces
§Use Ubiquitous Language understood by tech and !tech: PO, BA, QA
§Focus on behaviour (aka "FuncEonal Tests"), hide details behind steps
§Keep steps implementa7on simple; expose relevant parameters
§UI Scenarios (Selenium) can be fragile and slow⚠ => fire at API-level
§DRY: use shared steps, Background:, Scenario Outline, and @tags
§Mock external services for faster and more reliable tests
§Share state between step classes via dep. injecEon: Spring, Guice..
§Prevent state leak: cleanup test environment a]er each scenario
§Version control .features along code
§Runs on CI ⚠ (even if slow)](https://image.slidesharecdn.com/thejoyoftestingdevoxx-241008182535-1b9be551/85/The-Joy-of-Testing-Deep-Dive-Devoxx-Belgium-2024-60-320.jpg)
The Joy of Testing - Deep Dive @ Devoxx Belgium 2024
- 1. The Joy of Tes*ng 3-hours Deep Dive at by Victor Rentea Code is here: h+ps://github.com/victorrentea/unit-tes9ng.git branch: devoxx24
- 2. UT 3 - Evan Spiegel Ever had a great refactoring idea but you were afraid to do it?
- 3. UT 4 Prove it works today, including edge cases Fast Feedback to preserve focus: Start Clicking => Stop Thinking🤤 Refactor without fear tomorrow, be crea7ve, if func'onal-tests(elseJ) Up-to-date Spec running on CI, if func'onal tests Clarify requirements, if test-early Design feedback: Hard to test ≈> poor design, if test-early For coverage%, to get my PR approved 👍; Trap: Coverage Rush⚠ Why Write Tests? ✅ 🔄 🪓 📄 💬 ✨ %
- 4. 👋 I'm Victor Rentea 🇷🇴 Java Champion, PhD(CS) 👨💻 18 years in Java + most languages 👨🏫 10 years of Training / 150+ companies / 20+ countries: - Architecture, Refactoring, Unit Tes2ng (today) - Spring, JPA, Performance, ReacIve European SoKware CraKers Community (on meetup.com) YouTube.com/vrentea - events👆 + conference🎙 Life += 👰 + 👧 + 🙇 + 2*😸 victorrentea.ro I'm drawing on screen with: > ScreenBrush (Mac) > ZoomIt (Win)
- 5. UT 9 🧠 Quali&es of a Unit Test
- 6. UT 10 §Sensible: detects changes in behavior §Func7onal, not coupled to implementaEon (resilient to refactoring) §Specific: fails for one clear reason §Expressive: uses domain terms with a high signal/raEo raEo §Automated: easy to run by anyone with no/minimal manual setup §Repeatable: produces the same outcome if rerun §Isolated vs other tests / external system state §Minimal: includes only relevant stuff (#responsible copy-paste) §Fast, total build Eme < 15-20 min Quali0es of a Test from the book "The Art of Unit Tes.ng" by Roy Osherove
- 8. UT 13 Coverage Rush Management Goal: Coverage > xx%😤 vs 🥴Confused Developers or ☠Legacy The more code in here, the more untested code can be in the project. 😱 Example of Cobra Effect 5000 lines
- 9. UT 14 Coverage % Cost Coverage% 100% 80% 90% 60% = covrigi [RO] cover critical code first 🤔 Higher goals for 'domain' package Minimum Coverage%
- 10. UT 15 Line vs Branch Coverage §Line Coverage = did my tests executed a line? §Branch Coverage = did my tests exercised a path? 💖 💖 💖 - if (condition) è 50% = means condi4on was only true (or false) during tests - condition ? expr1 : expr2 - a && b || c è 100% branch coverage = a, b, c were all false + true in tests - Measured via a -javaagent: that monitors code executed by tests IntelliJ setup
- 11. UT 16 tl;dr Use Branch Coverage
- 12. UT 17 Professionals use coverage analysis to hunt for UNCOVERED code, NOT to celebrate COVERED lines
- 13. UT 18 Talk is cheap Show the me the code_ Code is here: h+ps://github.com/victorrentea/unit-tes9ng.git branch: devoxx24
- 14. UT 19 A new instance of the test class is created for each @Test. So it's ok to leave dirty data in test class' instance fields. #simplicity #1 Basic People don't know about JUnit 🥇
- 16. UT 21 All tests are green. 👏🥳 You break some behavior, but all tests stay Green 🤨 Evergreen Tests Imagine 95% coverage...
- 17. UT 22 Evergreen Tests Coverage% = ? Oups!! 100% If all tested code is deleted, the test remains GREEN 😱 Da-i asa! vs Basculanta🤣 lies to you
- 18. UT 23 §Prod coverage of class X.java = 87% 👍 §XTest.java = 2000-lines §67 x @Mock §200 x Mockito.when 😵💫 including when(aMock.getB()).thenReturn(bMock); §0 x "assert" 😱 §0 x "verify" 😱 Cobra 🐍 Effect
- 19. UT 24 The only way to prove a test is correct? tl;dr it FAILS if (and only if) you BREAK the tested behavior
- 20. UT 25 Muta3on Tes3ng You inten1onally introduce a bug in Produc@on Code You get a MUTANT Tests should FAIL 🎉 (proving they would catch that bug) Run Tests Don't forget⚠ to Undo the bug Correct tests fail <=> Produc@on Code is altered The only way to prove a test is correct?
- 21. UT 26 §Alters tested bytecode and checks tests fail - Using a set of mutators, like "Negate Condi@onals" §But⚠: - Can report false posi@ves (non-issues) - Takes long @me to run if applied on lots of code §Use it: - To learn - On super-cri@cal code (target a single package) Muta3on Tes3ng with pitest
- 22. UT 27 🦙💩 Ever generated a test with AI (LLM)? 🦄 Dream: Gen-AI tests (chaos) but then check them via muta@on-tes@ng (exact) Lower entry-cost when tes@ng legacy 👍
- 23. UT 28 Check that every new test you write fails for a bug in tested logic. (hands-on if not obvious)
- 24. UT 30 Bug-Tests
- 25. UT 31
- 26. UT 32 Breakpoint using step-into, inspect, evaluate, watch variable... ✅ Use for a 1st explora4on
- 27. UT 33 Breakpoint-debugging an issue is an<social behavior Others entering that code tomorrow will suffer too. requires deep focus 🧘 and drains lots of brain energy 🧠 Breakpoint
- 28. UT 35 Fix a bug like a Pro 1. Reproduce the bug [twice] 2. Write a test for it è FAILS❌ (⚠can be very-very hard in complex, terrible legacy code) 3. log.debug("..{}", var) instead of breakpoint 4. Find the cause > Kill bug > Test è GREEN✅ Avoid vola1le breakpoint sessions You can later lower the log level in producIon without restart😎: using: JMX, Spring Actuator, or Logback config auto-reload to inves9gate unreproducible bugs
- 29. UT 36 Why Bother? Bugs S7ck Together
- 30. UT 37 refactoring + tes4ng merge in "islands of safety" Test Around Bugs bug unit-tests around it Tes4ng another nearby bug becomes easier tomorrow Bugs S1ck Together Unit-tes4ng legacy code requires some prep-cleaning
- 32. UT 40 Bugs in Tests
- 33. UT 41 A Bug in a Test assertEquals("Greetings" + name, logic.getGreetings(name)); production code it's missing a " "! The developer copied the logic from produc@on to tests😩 You want Explicit Tests assertEquals("Greetings John", logic.getGreetings("John")); Find the BUG !
- 34. UT 42 Tests should be Simple and Expressive, NOT cryp4c, smarty, and hard to read
- 35. UT 43 Unmaintainable Tests Can make developers : Anger Zone delete test @Disable // comment test // @Test delete asserts flip asserts to pass
- 36. UT 45 Test Name
- 37. UT 47 Test Name void givenAnInactiveCustomer_whenHePlacesAnOrder_itIsActivated() void givenAProgramLinkedTo2Records_whenUpdated_2AuditsAreInserted() Given Then When 🥱
- 38. UT 48 AnInactiveCustomerShould .beActivatedWhenHePlacesAnOrder() Brain-Friendly Test Names Fluent naming hFps://codurance.com/2014/12/13/naming-test-classes-and-methods/ "Given" the class name explains the context shared by tests Test "Then" start with the expected effect to capture the reader a+en9on "When" ... then describe in what condi9on it occurs
- 39. UT 49 AssertJ
- 41. UT 51 assertEquals Never use again Use assertThat (from AssertJ)
- 42. UT 54 §A few fields assertThat(x.getA()).isEqualTo(y.getA()); assertThat(x.getB()).isEqualTo(y.getB()); §Almost all fields assertThat(x) .usingReflective(ignoringFields("a")).isEqualTo(y); §All fields assertThat(toPrettyJson(x)) .isEqualTo(toPrettyJson(y)); How to compare data structures?
- 44. UT 59 Parameterized Tests – Best Prac&ces §Orthogonal parameters - best (low mul4collinearity) Example (for 3 input params): - Good: a={1,2,3}, b={7,8}, c={T,F} = 3x2x2 = 12 valid combina4ons with 3 params - Bad if only 3 valid combina4ons: (1,7,true); (1,8,false); (2,8,true) §Few parameters < 3-5 - More => group them in a class/record with named fields - Complex input structure * many tests => consider file-based tests §Avoid boolean-riddled test - if (param1) excep4on was thrown if (param2) when(mock).. if (param3) verify(mock)..
- 45. UT 60 Tes0ng Styles Maintainability §Output-Based - Assert the returned value - (enough when tested code has no side effects, pure func4on😍) §State-Based - Assert fields of tested stateful object - Assert fields of collaborator objects (eg. passed as params🤨) - Assert external state in DB/Files §CommunicaEon-Based (mocks) - Verify interac4on with collaborators
- 47. UT 63 When you can't an4cipate the impact of a change. §Capture all output of a component in a human readable file - csv, json, xml, txt §Change the component behavior. §Diff 👀 the new output with the old output saved in files §If correct => save the new output in the old file = "approve" Approval Tes3ng
- 49. UT 65 1000 lines of code in a single file 00
- 50. UT 67 Characterization Tests §Dark Techniques to write temporary, ugly tests - Break encapsula4on of tested code (private -> public) - Par4al mock (@Spy) internal methods - Mock sta4cs & control global state §Pre-tesEng refactoring (unguarded by tests) - Exploratory refactoring (eg 30-minute > undo!): look for seams ("fracture planes") - Tiny, safe baby-steps refactoring with IDE, pairing (2) or mobbing (N) §Extract and test code with a clear role = maintainable tests💖 §CharacterizaEon Tests: capture input/output as a black-box Legacy Code Tes&ng Prac&ces
- 51. UT 68 Characterization Tests Some legacy code with no tests runs in production for many years. Therefore, we can assume it is correct. = it became its own spec But we need to refactor that code. 😱 😱 😱 😱 Our Goal = preserve its behaviour (correct or not), not to find bugs. Step 1. Identify & capture the inputs/outputs of the prod code (HARD), example IN= params, results of DB queries/API call, time/random/UUID!.. OUT= returned value, DB insert/update, API calls, sent messages!.. Step 2. Save {inputs + recorded outputs} to files. Step 3. Identify many different inputs to cover all execution paths Get line coverage close to 100%; ± manual mutation testing. PRO😎: Record IN/OUT from production = "Golden Master" Technique (also highlights dead☠ code) Step 4. Refactor guarded by these tests. When done, remove tests?
- 52. UT 69 Shadow TesFng Prod env Staging env Clients traffic traffic ↓ ↓ copy ↓ ↓ ↓ ↓ compare results ↑ ↑
- 53. UT 74 BDD Libraries: Cucumber JBehave BeHat (php) SpecFlow (C#) ... Behavior-Driven-Development (aka Acceptance-Test-Driven-Development)
- 54. UT 75 Have I understood correctly what we need to build?
- 55. UT 76 Bridge the Gap Make sure you implement the right thing Gherkin Language can help
- 56. UT 78 Gherkin Language PO Business // "glue" code can: configure mocks, DB, WireMock, set fields, load files ... @When("Player{int} scores {int} points") public void playerScores (int player, int points) { for (int i = 0; i < points; i++) { tennisScore.addPoint(player); } } @Then("Score is {string}") public void score_is(String expected) throws Throwable { assertThat(tennisScore.score()).isEqualTo(expected); } Cucumber
- 57. UT 80 Behavior-Driven Development Before star;ng the development, developers + testers + domain experts (BA/PO) write together 💖 a series of .feature files that capture (developer = typist, having sketched a few draV scenarios beforehand) the acceptance criteria of the stories to develop. When these .features go green✅ => develop = done🍺 Later, each change request starts with edi;ng .feature files
- 58. UT 81 🌈 Developers' Dream 🦄 Business people will maintain the .features 🚫 It never works. ✅ Works: Pair on them (Dev✍ + Biz💬) ✅ Works: Automa;on-tester
- 59. UT 82 When should you use a .feature? ✅ Hesita2ng/conflic2ng requirements (CYA principle) ✅ Complex rules, to beQer understand via more examples ✅ Business-cri2cal features to avoid expen$ive confusions ✅ Table-like test data (Excel-style) => scenario outline ❌ Business disengages: trivial or too technical features?
- 60. UT 83 Gherkin Best Prac0ces §Use Ubiquitous Language understood by tech and !tech: PO, BA, QA §Focus on behaviour (aka "FuncEonal Tests"), hide details behind steps §Keep steps implementa7on simple; expose relevant parameters §UI Scenarios (Selenium) can be fragile and slow⚠ => fire at API-level §DRY: use shared steps, Background:, Scenario Outline, and @tags §Mock external services for faster and more reliable tests §Share state between step classes via dep. injecEon: Spring, Guice.. §Prevent state leak: cleanup test environment a]er each scenario §Version control .features along code §Runs on CI ⚠ (even if slow)
- 61. UT 84 WASTE Maintaining .feature files that no businessperson approves
- 62. UT 85 When only developers work with Gherkin files consider building a test DSL instead. WriVen in the same language (java, C#...), or in a denser one: kt, py, groovy
- 63. UT 86 Test Helpers Tes<ng Framework Test DSL
- 64. UT 88 Test Domain-Specific Language = a fluent mini-tes@ng framework: Click to see implementa9on
- 65. UT 89 http://spockframework.org/spock/docs/1.3/spock_primer.html Spock tests (.groovy)
- 66. UT 90 Test DSL in Kotlin https://medium.com/porterbuddy/simple-test-dsls-in-kotlin-c6393035a92
- 67. UT 91 Fine-Grained Tests Complex Mocking Fragile vs Refactoring Focus on implementa;on Longer Tests Require Test Helpers Robust Close to spec
- 68. UT 92 JUnit4 Rules JUnit 5 Extensions Cucumber Tags .py decorators ... Test AOP Invisible code runs before and aer your tests 🧙
- 70. UT 94 Not all long tests are like this
- 71. UT 95 Hierarchical Test Fixtures Shared fixture for all tests on this flow Addi4onal fixture for some branches = context @Nested classes Test Subclasses Test AOP
- 72. UT 96 An Inac4ve Customer … with an order … with no orders Hierarchical Fixtures (@Nested in JUnit5) Nested Test Fixtures in JUnit5: https://junit.org/junit5/docs/current/user-guide/#writing-tests-nested
- 73. UT 97 An Inac4ve Customer @Nested With an order @Nested With no orders An Inac4ve Customer With an order … with no orders @Before Hierarchical Fixtures @Before @Before @Before @Before JUnit5 @Nested Extending a test class
- 74. UT 98 The Joy of Testing – key points #1 section - Why do we write Tests? - Coverage% ▶ CustomerValidatorTest - Test Names - ObjectMother: in-mem vs parse.json - Mutation Testing - Bug-testing - assertThat (AssertJ) ▶ - Test AOP with JUnit Extensions ▶ #2 section - Parameterized Tests ▶ TennisScoreTest - Approval Tests (by text inspection🧐) ▶ - Characterization Tests (for legacy code) - Gherkin for Functional Tests ▶ - Hierarchical Test Fixtures ▶ CustomerFacade - Test Qualities
- 75. UT 99 The Joy of Tes*ng Please share with others Code is here: h+ps://github.com/victorrentea/unit-tes9ng.git branch: devoxx24 Thank you!
- 76. UT 100 The Joy of Tes*ng 💡 Share your thoughts: stalk me or at victorrentea@gmail.com 📺 Test-Driven Design Insights, 3h Deep Dive at Devoxx 2023 📺 Tes1ng Microservices: Join the Revolu1on, at Devoxx 2023 📺 YouTube.com/vrentea 👨🏫 Private training ± consultancy: victorrentea.ro/catalog What Next? 💬 Free monthly webinars: victorrentea.ro/community Code is here: h+ps://github.com/victorrentea/unit-tes9ng.git branch: devoxx24