DCDNUG 10/16/2012 Automated testing obstacles pitfalls dangers
Download as PPTX, PDF0 likes390 views
This document summarizes a presentation on overcoming obstacles, pitfalls, and dangers of automated testing. The presentation covers four main obstacles: 1) over-specifying tests with too many assertions, 2) tests becoming coupled to the system clock, 3) database issues causing integration test failures, and 4) test code becoming too repetitive without proper reuse. Examples are provided for each obstacle along with approaches for addressing them like using fewer assertions per test, faking the system time, managing test database state, and implementing better code reuse in test files.
DCDNUG 10/16/2012 Automated testing obstacles pitfalls dangers
- 2. Overcoming the Obstacles, Pitfalls, and Dangers of Automated Testing Stephen D. Ritchie 16-Oct-2012
- 3. Purpose Useful Automated Testing Make Software Better Excella Consulting
- 4. Agenda 1 Motivation 2 Principles 3 Obstacles Excella Consulting -4-
- 5. Chrysler New Yorker Excella Consulting
- 6. First Topic: Motivation Why Write Unit Tests? Why Automate Testing? Excella Consulting
- 7. Agenda 1 Motivation 2 Principles 3 Obstacles Excella Consulting -7-
- 8. Visibility & Insight Problem Detection Advance Warning
- 9. Microscope: Visibility and Insight Excella Consulting
- 10. Smoke Detector: Problem Detection Excella Consulting
- 11. Weather Satellite: Advance Warning Excella Consulting
- 12. Example Perhaps An Example Would Be Helpful Excella Consulting
- 13. Automated Tests Make Sure Software Works As Intended
- 14. Agenda 1 Motivation 2 Principles 3 Obstacles Excella Consulting - 14 -
- 15. Principles Zero Configuration Fast Clear Result Easy To Maintain Excella Consulting
- 16. Automated Testing: Vocabulary • Test-Driven Development (TDD) – Write a Test, Watch the Test Fail – Write Code, Make the Test Pass – Write the Next Test • Behavior-Driven Development (BDD) – Given a Desired Behavior • Intention Checking – The Software Works, As Intended Excella Consulting
- 17. Principles Zero Configuration I can run your tests, You can run mine. Excella Consulting
- 18. Principles Fast All the tests run in just a few minutes Excella Consulting
- 19. Principles Clear Results Pass/Fail Focused Test Excella Consulting
- 20. Principles Easy to Maintain Conventional Brief Excella Consulting
- 21. The Long-Term Goals • Automated Testing – Vigilantly Monitoring the Code • Readability – Have Mercy on Future Developers • Conventional • Short, Clear • Maintainability – Both a Sword and a Shield • Code Works as Intended • Protects Against Regression – Reliable – N+1 is Easy Excella Consulting
- 22. Agenda 1 Motivation 2 Principles 3 Obstacles Excella Consulting - 22 -
- 23. One Primary Assert To Rule Them All Obstacle 1 Over-Specifying Excella Consulting
- 24. One Primary Assert To Rule Them All • Is your effort to refactor and improve code overwhelmed by the time it takes to maintain/update/rewrite all those failing unit tests? – Your test-code could be over specifying things. • Perhaps an example would be helpful … Excella Consulting
- 25. One Primary Assert To Rule Them All • Debate: Only one assertion per test? • Test Method Tests One and Only One Scenario – 1 Primary Assert Verifies and Validates the Scenario • Secondary Asserts – Support Arrangement and Preconditions – Support Post-Conditions • Avoid Asserts that Over Specify – Too Literal => Inhibited Refactoring – Imagined Benefit => Rigidity Excella Consulting
- 26. Four Ways to Fake Time Obstacle 2 Time Crunch Excella Consulting
- 27. Four Ways to Fake Time • Are your test methods starting to fail because the code-under-test is coupled to the system clock? – Your code is too dependent on System.DateTime.Now • Perhaps an example would be helpful … Excella Consulting
- 28. Four Ways to Fake Time • Things to Watch For – Thread Safety • public static class SystemDateTime – Making Your Privates Public // Inject the class dependency on DateTime.Now private DateTime? _now; public DateTime Now { get { return _now ?? DateTime.Now; } set { _now = value; } } Excella Consulting
- 29. Database Killed The Integration Test Obstacle 3 Database Excella Consulting
- 30. Database Killed The Integration Test • Are your automated integration tests failing because of the data in the testing database; the data keeps changing? • Perhaps an example would be helpful … Excella Consulting
- 31. Database Killed The Integration Test • Automated Testing Persistence – NDbUnit – SQL Server Express – NHibernate • Surface Testing – Data Access Layer: API Surface – Liberates Refactoring Excella Consulting
- 32. In Test Code, Do Repeat Yourself ... Do Repeat Yourself Obstacle 4 Unhelpful … Reuse Repetition Coupling Excella Consulting
- 33. In Test Code, Do Repeat Yourself ... Do Repeat Yourself • Do you have an explosion of test methods, with the ratio of test code to code-under-test that’s way too high? – Your test-code is too DRY in some places and … – Not DRY enough in all the right places • Perhaps an example would be helpful … Excella Consulting
- 34. In Test Code, Do Repeat Yourself ... Do Repeat Yourself • Data-Drive Test Cases – One Test Method – Many Test Scenarios • Repeat Code in a “TestsContext” Class – Sidecar Approach • Use Helper Classes – Extension methods – Composition • Keep Inheritance in Reserve – Overall Testing Framework Excella Consulting
- 35. Agenda 1 Motivation 2 Principles 3 Obstacles Excella Consulting - 35 -
- 36. Of Course It’s Safe … After You Excella Consulting
- 37. Further Discussion Any questions? Any comments? Excella Consulting
- 38. Shameless Self Promotion Time! 40% off eBook at apress.com Use promo code: UGoCTNoV Offer ends 10-Nov-2012 Excella Consulting
- 39. Contact Me • Twitter: @ruthlesshelp • Email: stephen.ritchie@excella.com • Blog: http://ruthlesslyhelpful.net • LinkedIn: http://www.linkedin.com/in/sritchie Excella Consulting
- 40. Slides and Examples • Slideshare: http://www.slideshare.net/ruthlesshelp • Github: http://github.com/ruthlesshelp Excella Consulting