Fundamentals of testing
- 1. “Fundamentals of testing” MUHAMAD IQBAL SAKTI 11453101976 PROGRAM STUDI S1 SISTEM INFORMASI FAKULTAS SAINS DAN TEKNOLOGI UIN SUSKA RIAU 2017 http://sif.uin-suska.ac.id/ http://fst.uin-suska.ac.id/ http://www.uin-suska.ac.id/
- 2. WHY IS TESTING NECESSARY? 1 Describe, with examples, the way in which a defect in software can cause harm to a person, to the environment or to a company. (K2) 2 Distinguish between the root cause of a defect and its effects. (K2) 3 Give reasons why testing is necessary by giving examples. (K2) 4 Describe why testing is part of quality assurance and give examples of how testing contributes to higher quality. (K2) 5 Recall the terms 'mistake', 'defect', 'fault', 'failure' and the corresponding terms 'error' and 'bug'. (K1) 6 Explain the fundamental principles in testing. (K2) FUNDAMENTALS OF TESTING
- 3. Do our mistakes matter? Let's think about the consequences of mistakes. We agree that any human being, programmers and testers included, can make an error. These errors may produce defects in the software code or system, or in a document. If a defect in code is executed, the system may experience a failure. So the mistakes we make matter partly because they have consequences for the products for which we are responsible. FUNDAMENTALS OF TESTING
- 4. When do defects arise? In Figure 1.1 we can see how defects may arise in four requirements for a product. Requirement 1 is implemented correctly - we understood the customer's requirement, designed correctly to meet that requirement, built cor- rectly to meet the design, and so deliver that requirement with the right attributes: functionally, it does what it is supposed to do and it also has the right non-functional attributes, so it is fast enough, easy to understand and so on. Requirement 2 is fine until the software is coded, when we make some mistakes and introduce defects. Probably, these are easily spotted and corrected during testing, because we can see the product does not meet its design specification. Requirement 3 are harder to deal with; we built exactly what we were told to but unfortunately the designer made some mis- takes so there are defects in the design. Unless we check against the require- ments definition, we will not spot those defects during testing. When we do notice them they will be hard to fix because design changes will be required. Requirement 4 were introduced during the definition of the requirements; the product has been designed and built to meet that flawed requirements definition. If we test the product meets its requirements and design, it will pass its tests but may be rejected by the user or customer. FUNDAMENTALS OF TESTING
- 5. What is the cost of defects? As well as considering the impact of failures arising from defects we have not found, we need to consider the impact of when we find those defects. The cost of finding and fixing defects rises considerably across the life cycle; think of the old English proverb 'a stitch in time saves nine'. This means that if you mend a tear in your sleeve now while it is small, it's easy to mend, but if you leave it, it will get worse and need more stitches to mend it. FUNDAMENTALS OF TESTING
- 6. WHAT IS TESTING? Recall the common objectives of testing. (K1) Describe the purpose of testing in software development, maintenance and operations as a means to find defects, provide confidence and infor mation, and prevent defects. (K2) As you read this section, you'll encounter the terms code, debugging, development of software, requirement, review, test basis, test case, testing and test objective. FUNDAMENTALS OF TESTING
- 7. Testing Principle - Early testing Testing activities should start as early as possible in the software or system development life cycle and should be focused on defined objectives. We can use both dynamic testing and static testing as a means for achieving similar test objectives. Both provide information to improve both the system to be tested, and the development and testing processes. We mentioned above that testing can have different goals and objectives, which often include: - finding defects; - gaining confidence in and providing information about the level of quality; - preventing defects. FUNDAMENTALS OF TESTING
- 8. Testing Principle - Defect clustering A small number of modules contain most of the defects discovered during pre-release testing or show the most operational failures. 1.2.5 Focusing on defects can help us plan our tests Reviewing defects and failures in order to improve processes allows us to improve our testing and our requirements, design and development processes. One phenomenon that many testers have observed is that defects tend to cluster. This can happen because an area of the code is particularly complex and tricky, or because changing software and other products tends to cause knock-on defects. Testers will often use this information when making their risk assessment for planning the tests, and will focus on known 'hot spots'. FUNDAMENTALS OF TESTING
- 9. TESTING PRINCIPLES Explain the fundamental principles in testing. (K2) In Sections 1.1 and 1.2, we have introduced a number of testing principles and brief explanations. These are listed in Table 1.2, for you to read over to remind yourself about them. These principles have been suggested over the past 40 years and offer general guidelines common for all testing. FUNDAMENTALS OF TESTING Principle 1: Testing shows presence of defects Testing can show that defects are present, but cannot prove that there are no defects. Testing reduces the probability of undiscovered defects remaining in the software but, even if no defects are found, it is not a proof of correctness. Principle 2: Exhaustive testing is impossible Testing everything (all combinations of inputs and preconditions) is not feasible except for trivial cases. Instead of exhaustive testing, we use risks and priorities to focus testing efforts. Principle 3: Early testing Testing activities should start as early as possible in the software or system development life cycle and should be focused on defined objectives. Principle 4: Defect clustering A small number of modules contain most of the defects discovered during pre-release testing or show the most operational failures. Principle 5: Pesticide paradox If the same tests are repeated over and over again, eventually the same set of test cases will no longer find any new bugs. To overcome this 'pesticide paradox', the test cases need to be regularly reviewed and revised, and new and different tests need to be written to exercise different parts of the software or system to potentially find more defects. Principle 6: Testing is context dependent Testing is done differently in different contexts. For example, safety-critical software is tested differently from an e-commerce site. Principle 7: Absence-of-errors fallacy Finding and fixing defects does not help if the system built is unusable and does not fulfill the users' needs and expectations.
- 10. FUNDAMENTAL TEST PROCESS Recall the fundamental test activities from planning to test closure activities and the main tasks of each test activity. (K1) In this section, you'll also encounter the glossary terms confirmation testing, exit criteria, incident, regression testing, test basis, test condition, test coverage, test data, test execution, test log, test plan, test strategy, test summary report and testware. As we have seen, although executing tests is important, we also need a plan of action and a report on the outcome of testing. Project and test plans should include time to be spent on planning the tests, designing test cases, preparing for execution and evaluating status. The idea of a fundamental test process for all levels of test has developed over the years. Whatever the level of testing, we see the same type of main activities happening, although there may be a different amount of formality at the different levels, for example, component tests might be carried out less formally than system tests in most organizations with a less documented test process. The decision about the level of formality of the processes will depend on the system and software context and the level of risk associated with the software. So we can divide the activities within the fundamental test process into the following basic steps: • planning and control; • analysis and design; • implementation and execution; • evaluating exit criteria and reporting; • test closure activities. FUNDAMENTALS OF TESTING
- 11. THE PSYCHOLOGY OF TESTING Recall that the success of testing is influenced by psychological factors: (K1) • clear objectives; • a balance of self-testing and independent testing; • recognition of courteous communication and feedback on defects. Contrast the mindset of a tester and that of a developer. (K2) In this section, we'll discuss the various psychological factors that influence testing and its success. These include clear objectives for testing, the proper roles and balance of self-testing and independent testing, clear, courteous com- munication and feedback on defects. We'll also contrast the mindset of a tester and of a developer. You'll find a single Syllabus term in this section, independent testing, and the glossary term, independence. FUNDAMENTALS OF TESTING