Testing: Heaven or Hell
This document discusses different types of software testing and provides examples of both good ("heaven") and bad ("hell") practices for each type. It outlines Brian Marick's four testing quadrants: unit tests (Quadrant 1), functional tests (Quadrant 2), exploratory tests (Quadrant 3), and non-functional tests (Quadrant 4). Examples of good practices include writing unit tests with code, focusing prototypes on solving problems, getting customer feedback, and testing at production scale. Examples of bad practices include writing unit tests long after code, not isolating dependencies, failing to use the product as intended, and load testing in different configurations than production. The overall message is that testing effectiveness depends on matching the test to the
Testing: Heaven or Hell
- 1. Testing: Heaven or Hell OpenWest 2014
- 2. About Me
- 3. What is a test? ● Automated? ● Code?
- 4. Why Testing? ● Find bugs ● Feel confident in the code ● Product problems ● Technology problems
- 5. What is a test? ● Agile Testing Quadrants – Brian Marick – Agile Testing ● Lisa Crispin ● Janet Gregory
- 7. Quadrant 1 ● Unit Tests ● Component Tests
- 8. Quadrant 1 Hell ● User service rewrite – Focus on business – Written long after code – Filled with dependencies
- 9. Quadrant 1 Hell ● JavaScript Testing – Tests complete if they pass – Manual tests
- 10. Quadrant 1 Heaven ● Revenue Dashboard – Focus on unit – Write with code
- 11. Quadrant 1 Heaven ● Clipboard functionality – Isolate dependencies – Write tests to fail
- 12. Benefits ● Find technical bugs early ● Refactoring ● Documentation ● Design ● Quick feedback
- 14. Quadrant 2 ● Functional Tests ● Examples ● Story Tests ● Prototypes ● Simulations
- 15. Quadrant 2 Hell ● Documentation Search – Make it up as you go – Don't worry about the problem being solved
- 16. Quadrant 2 Hell ● Documentation Search – Make it up as you go – Don't worry about the problem being solved
- 17. Quadrant 2 Heaven ● Prototypes – Know what you need to do – Ask for examples
- 18. Quadrant 2 Heaven ● Clipboard refactoring – Focus on solving the problem – Test as the end user
- 19. Benefits ● Universally understood ● Focuses on customer ● Defines done-ness
- 21. Quadrant 3 ● Exploratory Tests ● Scenarios ● Usability Testing ● User Acceptance Testing ● Alpha/Beta
- 22. Quadrant 3 Hell ● Test each piece separately – Never use the product as a user would
- 23. Quadrant 3 Hell? ● Angry customers ● Negative feedback at sprint review
- 24. Quadrant 3 Heaven ● Customer feedback ● Suggestions at sprint review ● Work as a user
- 25. Benefits ● Determines next steps ● Explore the system as a whole ● Negative feedback can be positive
- 27. Quadrant 4 ● Performance Testing ● Load Testing ● Security Testing ● “-ility” Testing – Reliability – Maintainability – Compatibility
- 28. Quadrant 4 Hell
- 29. Quadrant 4 Hell ● Google Apps Invitations – Don't think about volume ● Load testing on dev environment – Test load in a different configuration
- 30. Quadrant 4 Heaven ● Third-party security testing ● Production environment for testing
- 31. Benefits ● Addresses soft requirements ● Think about technology at scale ● Test boundaries of technology
- 32. What is a test? ● Anything that attempts to disprove your assumptions
- 34. Testing Heaven or Hell ● Match tests to assumptions
- 35. Questions?