01. foundamentals of testing
- 1. “Fundamentals of testing” Graham et al (2006) Oleh : Tricia karina (11453201712) Program Studi S1 Sistem Informasi Universitas Islam Negeri Sultan Syarif Kasim Riau http://sif.uin-suska.ac.id/ http://fst.uin-suska.ac.id/ http://www.uin-suska.ac.id/
- 2. Fundamentals of testing In this chapter, we will introduce you to the fundamentals of testing: why testing is needed; its limitations, objectives and purpose; the principles behind testing; the process that testers follow; and some of the psychological factors that testers must consider in their work. By reading this chapter you'll gain an understanding of the fundamentals of testing and be able to describe those fundamentals.
- 3. WHY IS TESTING NECESSARY? • Introduction In this section, we're going to kick off the book with a discussion on why testing matters. We'll describe and illustrate how software defects or bugs can cause problems for people, the environment or a company. We'll draw important distinctions between defects, their root causes and their effects. We'll explain why testing is necessary to find these defects, how testing promotes quality, and how testing fits into quality assurance. In this section, we will also introduce some fundamental principles of testing.
- 4. • Software systems context Not all software systems carry the same level of risk and not all problems have the same impact when they occur. A risk is something that has not happened yet and it may never happen; it is a potential problem. We are concerned about these potential problems because, if one of them did happen, we'd feel a negative impact. When we discuss risks, we need to consider how likely it is that the problem would occur and the impact if it happens. For example, whenever we cross the road, there is some risk that we'll be injured by a car. The likelihood depends on factors such as how much traffic is on the road, whether there is a safe crossing place, how well we can see, and how fast we can cross. The impact depends on how fast the car is going, whether we are wearing protective gear, our age and our health. The risk for a particular person can be worked out and therefore the best road-crossing strategy. • Causes of software defects but it is difficult for people to find their own mistakes while building a product. Defects in software, systems or documents may result in failures, but not all defects do cause failures. We could argue that if a mistake does not lead to a defect or a defect does not lead to a failure, then it is not of any importance - we may not even know we've made an error. Our fallibility is compounded when we lack experience, don't have the right information, misunderstand, or if we are careless, tired or under time pressure. we are more likely to make errors when dealing with perplexing technical or business problems, complex business processes, code or infrastructure, changing technologies, or many
- 5. • Role of testing in software development, maintenance and operations We have seen that human errors can cause a defect or fault to be introduced at any stage within the software development life cycle and, depending upon the consequences of the mistake, the results can be trivial or catastrophic. Rigorous testing is necessary during development and maintenance to identify defects, in order to reduce failures in the operational environment and increase the quality of the operational system. This includes looking for places in the user interface where a user might make a mistake in input of data or in the interpretation of the output, and looking for potential weak points for intentional and malicious attack. Executing tests helps us move towards improved quality of product and service, but that is just one of the verification and validation methods applied to products. Processes are also checked, for example by audit. A variety of methods may be used to check work, some of which are done by the author of the work and some by others to get an independent view.
- 6. • Testing and quality Testing helps us to measure the quality of software in terms of the number of defects found, the tests run, and the system covered by the tests. We can do this for both the functional attributes of the software (for example, printing a report correctly) and for the non-functional software requirements and characteristics (for example, printing a report quickly enough). • How much testing is enough? Testing everything (all combinations of inputs and preconditions) is not feasible except for trivial cases. Instead of exhaustive testing, we use risks and priorities to focus testing efforts.
- 7. WHAT IS TESTING? • The driving test – an analogy for software testing We have spent some time describing why we need to test, but we have not dis- cussed what testing is. What do we mean by the word testing? We use the words test and testing in everyday life and earlier we said testing could be described as 'checking the software is OK'. That is not a detailed enough definition to help us understand software testing. Let's use an analogy to help us: driving tests. • Defining software testing Let's break the definition down into parts; the definition has some key phrases to remember. The definition starts with a description of testing as a process and then lists some objectives of the test process.
- 8. • Software test and driving test compared We can see that the software test is very like a driving test in many ways, although of course it is not a perfect analogy! The driving examiner becomes the software tester. The driver being examined becomes the system or software under test, and you'll see as we go through this book that the same approach broadly holds. • When can we meet our test objectives? Testing activities should start as early as possible in the software or system development life cycle and should be focused on defined objectives. • Focusing on defects can help us plan our tests Reviewing defects and failures in order to improve processes allows us to improve our testing and our requirements, design and development processes. One phenomenon that many testers have observed is that defects tend to cluster. This can happen because an area of the code is particularly complex and tricky, or because changing software and other products tends
- 9. • The defect clusters change over time If the same tests are repeated over and over again, eventually the same set of test cases will no longer find any new bugs. To overcome this 'pesticide paradox', the test cases need to be regularly reviewed and revised, and new and different tests need to be written to exercise different parts of the software or system to potentially find more defects. • Debugging removes defects When a test finds a defect that must be fixed, a programmer must do some work to locate the defect in the code and make the fix. In this process, called debugging, a programmer will examine the code for the immediate cause of the problem, repair the code and check that the code now executes as expected. • If we don't find defects does that mean the users will accept the software? Finding and fixing defects does not help if the system built is unusable and does not fulfill the users' needs and expectations.
- 10. TESTING PRINCIPLES • Testing can show that defects are present, but cannot prove that there are no defects. Testing reduces the probability of undiscovered defects remaining in the software but, even if no defects are found, it is not a proof of correctness. • Testing everything (all combinations of inputs and preconditions) is not feasible except for trivial cases. Instead of exhaustive testing, we use risks and priorities to focus testing efforts. • Testing activities should start as early as possible in the software or system development life cycle and should be focused on defined objectives. • A small number of modules contain most of the defects discovered during pre-release testing or show the most operational failures. • If the same tests are repeated over and over again, eventually the same set of test cases will no longer find any new bugs. To overcome this 'pesticide paradox', the test cases need to be regularly reviewed and revised, and new and different tests need to be written to exercise different parts of the software or system to potentially find more defects.
- 11. FUNDAMENTAL TEST PROCESS • Introduction In this section, we will describe the fundamental test process and activities. These start with test planning and continue through to test closure. For each part of the test process, we'll discuss the main tasks of each test activity. In this section, you'll also encounter the glossary terms confirmation testing, exit criteria, incident, regression testing, test basis, test condition, test coverage, test data, test execution, test log, test plan, test strategy, test summary report and testware. • Test planning and control During test planning, we make sure we understand the goals and objectives of the customers, stakeholders, and the project, and the risks which testing is intended to address. This will give us what is sometimes called the mission of testing or the test assignment.
- 12. • Test analysis and design Test analysis and design is the activity where general testing objectives are trans- formed into tangible test conditions and test designs. During test analysis and design, we take general testing objectives identified during planning and build test designs and test procedures (scripts). • Test implementation and execution During test implementation and execution, we take the test conditions and make them into test cases and testware and set up the test environment. This means that, having put together a high-level design for our tests, we now start to build them. We transform our test conditions into test cases and procedures, other testware such as scripts for automation.
- 13. • Evaluating exit criteria and reporting Evaluating exit criteria is the activity where test execution is assessed against the defined objectives. This should be done for each test level, as for each we need to know whether we have done enough testing. Based on our risk assess- ment, we'll have set criteria against which we'll measure 'enough'. These criteria vary for each project and are known as exit criteria. • Test closure activities During test closure activities, we collect data from completed test activities to consolidate experience, including checking and filing testware, and analyzing facts and numbers. We may need to do this when software is delivered. We also might close testing for other reasons, such as when we have gathered the infor- mation needed from testing, when the project is cancelled, when a particular milestone is achieved, or when a maintenance release or update is done.
- 14. THE PSYCHOLOGY OF TESTING • Independent testing – who is a tester? The mindset we want to use while testing and reviewing is different from the one we use while analyzing or developing. By this we mean that, if we are building something we are working positively to solve problems in the design and to realize a product that meets some need. However, when we test or review a product, we are looking for defects in the product and thus are critical of it. • Why do we sometimes not get on with the rest of the team? As well as independence, separation of the tester role from the developer role is also done to help focus effort and to provide the benefits of trained and professional testing resources. In many organizations, earlier stages of testing are carried out by the developers and integrators and later stages independently, either by a specialist test group or by the customers.
- 15. THANKYOUTHANKYOU