DaKiRy_QAStandUp_Діана Пінчук "Testing oracles"
0 likes217 views
An oracle is a heuristic mechanism used to recognize problems by generating expected results, though oracles are fallible. Two common oracle classifications are FEW HICCUPS and Doug Hoffman's taxonomy. FEW HICCUPS is useful for designing tests and logging convincing bug reports, while Doug Hoffman's taxonomy can be used to develop automated tests. Testing oracles help detect defects, and emotions can also potentially serve as an oracle.
DaKiRy_QAStandUp_Діана Пінчук "Testing oracles"
- 1. Testing oracles Diana Pinchuk, QA Engineer
- 2. Agenda • Testing Oracles • Oracles classification • Emotions as oracle • Summary
- 3. Oracles An oracle is a heuristic principle or mechanism by which we recognize a problem An oracles is a method of generation of the expected result And that, since they’re heuristics, oracles are fallible and context-dependent. No oracle comes with a guarantee that it’s giving you the right answer. An oracle can only point you to a possible problem.
- 4. Full and partial oracles Full oracle - a mechanism which can provide expected result with 100% sure. Partial oracle - a mechanism which cannot be fully reliable in providing expected result, but requires less resources for usage.
- 5. Failures when using oracles Mishit - an oracle didn’t find the problem when it exists False alarm - an oracle found the problem which doesn’t exist
- 6. Oracles classification FEW HICCUPS by J.Bach & M.Bolton Doug Hoffman’s Approach Your own oracles Emotions and feelings
- 7. FEW HICCUPS Oracle Description History Correspondence to the previous versions Image Correspondence to the Company image Comparable products Correspondence to comparable products Claims Correspondence to requirements (promises) User expectations Correspondence to user’s expectations Product Correspondence to other parts of the product Purpose Correspondence to the product purposes Statutes and Standards Correspondence to the statutes and standards
- 8. FEW HICCUPS Oracle Description Familiarity Discrepancy with similar problems Explainability Correspondence to behaviour which can be explained World Accordance to beliefs about the world
- 9. Doug Hoffman’s taxonomy Oracle Constraint oracle Regression oracle Self-verifying data oracle Physical model oracle Oracle Calculation oracle Inverse oracle Reference program Oracle Business model oracle Statistical model oracle State model oracle Interaction model oracle
- 10. Comparison of the both classifications Areas of usage Strengths Weaknesses FEW HUCCUPS Logging convincing bug reports Test design Mnemonic Flexibility Not suitable for autotests developing Doug Hoffman’s taxonomy Test design Developing of autotests Oracles are well programmed Concreteness Descriptions are not well structured
- 11. And more... Emotions can be used as oracle Tester’s emotions are like smoke detector
- 12. Summary • An oracles is a mechanism of generation of the expected result • 2 most popular oracles classifications are (FEW HICCUPPS) and Doug Hoffman’s taxonomy • Both classifications help us to detect defects and design our tests • FEW HICCUPPS is successfully used for logging of convincing bug reports • Doug Hoffman’s taxonomy can be used for developing autotests • There are many ideas of the testing oracles usage. For example, you can use your emotions