QA Basics and PM Overview

Editor's Notes

• #15: Errors in use of the system: Errors in use of the system or product or application may arise because of the following reasons: –          Inadequate knowledge of the product or the software to the tester. The tester may not be aware of the functionalities of the product and hence while testing the product there might be some defects or failures. –          Lack of the understanding of the functionalities by the developer. It may also happen that the developers may not have understood the functionalities of the product or application properly. Based on their understanding the feature they will develop may not match with the specifications. Hence this may result into the defect or failure. Environmental conditions: Because of the wrong setup of the testing environment testers may report the defects or failures. As per the recent surveys it has been observed that about 40% of the tester’s time is consumed because of the environment issues and this has a great impact on quality and productivity. Hence proper test environments are required for quality and on time delivery of the product to the customers. Intentional damage: The defects and failures reported by the testers while testing the product or the application may arise because of the intentional damage. Potential consequences of earlier errors: Errors found in the earlier stages of the development reduce our cost of production. Hence it’s very important to find the error at the earlier stage. This could be done by reviewing the specification documents or by walkthrough. The downward flow of the defect will increase the cost of production.
• #17: We can see that Requirement 1 is implemented correctly – we understood the customer’s requirement, designed correctly to meet that requirement, built correctly to meet the design, and so deliver that requirement with the right attributes: functionally, it does what it is supposed to do and it also has the right non-functional attributes, so it is fast enough, easy to understand and so on. With the other requirements, errors have been made at different stages. Requirement 2 is fine until the software is coded, when we make some mistakes and introduce defects. Probably, these are easily spotted and corrected during testing, because we can see the product does not meet its design specification. The defects introduced in Requirement 3 are harder to deal with; we built exactly what we were told to but unfortunately the designer made some mistakes so there are defects in the design. Unless we check against the requirements definition, we will not spot those defects during testing. When we do notice them they will be hard to fix because design changes will be required. The defects in Requirement 4 were introduced during the definition of the requirements; the   product has been designed and built to meet that flawed requirements definition. If we test the product meets its requirements and design, it will pass its tests but may be rejected by the user or customer. Defects reported by the customer in acceptance test or live use can be very costly. Unfortunately, requirements and design defects are not rare; assessments of thousands of projects have shown that defects introduced during requirements and design make up close to half of the total number of defects.
• #18: If the error is made and the consequent defect is detected in the requirements phase then it is relatively cheap to fix it. Similarly if an error is made and the consequent defect is found in the design phase then the design can be corrected and reissued with relatively little expense. The same applies for construction phase. If however, a defect is introduced in the requirement specification and it is not detected until acceptance testing or even once the system has been implemented then it will be much more expensive to fix. This is because rework will be needed in the specification and design before changes can be made in construction; because one defect in the requirements may well propagate into several places in the design and code; and because all the testing work done-to that point will need to be repeated in order to reach the confidence level in the software that we require. It is quite often the case that defects detected at a very late stage, depending on how serious they are, are not corrected because the cost of doing so is too expensive.
• #19: Defect life cycle is a cycle which a defect goes through during its lifetime. It starts when defect is found and ends when a defect is closed, after ensuring it’s not reproduced. Defect life cycle is related to the bug found during testing. New:  When a defect is logged and posted for the first time. It’s state is given as new. Assigned:  After the tester has posted the bug, the lead of the tester approves that the bug is genuine and he assigns the bug to corresponding developer and the developer team. It’s state given as assigned. Open:  At  this state the developer has started analyzing and working on the defect fix. Fixed:  When developer makes necessary code changes and verifies the changes then he/she can make bug status as ‘Fixed’ and the bug is passed to testing team. Pending retest:  After fixing the defect the developer has given that particular code for retesting to the tester. Here the testing is pending on the testers end. Hence its status is pending retest. Retest:  At this stage the tester do the retesting of the changed code which developer has given to him to check whether the defect got fixed or not. Verified:  The tester tests the bug again after it got fixed by the developer. If the bug is not present in the software, he approves that the bug is fixed and changes the status to “verified”. Reopen:  If the bug still exists even after the bug is fixed by the developer, the tester changes the status to “reopened”. The bug goes through the life cycle once again. Closed:  Once the bug is fixed, it is tested by the tester. If the tester feels that the bug no longer exists in the software, he changes the status of the bug to “closed”. This state means that the bug is fixed, tested and approved. Duplicate: If the bug is repeated twice or the two bugs mention the same concept of the bug, then one bug status is changed to “duplicate“. Rejected: If the developer feels that the bug is not genuine, he rejects the bug. Then the state of the bug is changed to “rejected”. Deferred: The bug, changed to deferred state means the bug is expected to be fixed in next releases. The reasons for changing the bug to this state have many factors. Some of them are priority of the bug may be low, lack of time for the release or the bug may not have major effect on the software. Not a bug:  The state given as “Not a bug” if there is no change in the functionality of the application. For an example: If customer asks for some change in the look and field of the application like change of colour of some text then it is not a bug but just some change in the looks of the  application
• #21: 1) Testing shows presence of defects: Testing can show the defects are present, but cannot prove that there are no defects. Even after testing the application or product thoroughly we cannot say that the product is 100% defect free. Testing always reduces the number of undiscovered defects remaining in the software but even if no defects are found, it is not a proof of correctness. 2) Exhaustive testing is impossible: Testing everything including all combinations of inputs and preconditions is not possible. So, instead of doing the exhaustive testing we can use risks and priorities to focus testing efforts. For example: In an application in one screen there are 15 input fields, each having 5 possible values, then to test all the valid combinations you would need 30  517  578  125  (515) tests. This is very unlikely that the project timescales would allow for this number of tests. So, accessing and managing risk is one of the most important activities and reason for testing in any project. 3) Early testing: In the software development life cycle testing activities should start as early as possible and should be focused on defined objectives. 4) Defect clustering: A small number of modules contains most of the defects discovered during pre-release testing or shows the most operational failures. 5) Pesticide paradox: If the same kinds of tests are repeated again and again, eventually the same set of test cases will no longer be able to find any new bugs. To overcome this “Pesticide Paradox”, it is really very important to review the test cases regularly and new and different tests need to be written to exercise different parts of the software or system to potentially find more defects. 6) Testing is context dependent: Testing is basically context dependent. Different kinds of sites are tested differently. For example, safety – critical software is tested differently from an e-commerce site. 7) Absence – of – errors fallacy: If the system built is unusable and does not fulfil the user’s needs and expectations then finding and fixing defects does not help.
• #22: Testing is a process rather than a single activity. This process starts from test planning then designing test cases, preparing for execution and evaluating status till the test closure. So, we can divide the activities within the fundamental test process into the following basic steps: 1)    Planning and Control: Test planning has following major tasks:i.  To determine the scope and risks and identify the objectives of testing.ii. To determine the test approach.iii. To implement the test policy and/or the test strategy. (Test strategy is an outline that describes the testing portion of the software development cycle. It is created to inform PM, testers and developers about some key issues of the testing process. This includes the testing objectives, method of testing, total time and resources required for the project and the testing environments.).iv. To determine the required test resources like people, test environments, PCs, etc.v. To schedule test analysis and design tasks, test implementation, execution and evaluation.vi. To determine the Exit criteria we need to set criteria such as Coverage criteria. (Coverage criteria are the percentage of statements in the software that must be executed during testing. This will help us track whether we are completing test activities correctly. They will show us which tasks and checks we must complete for a particular   level of testing before we can say that testing is finished.)  Test control has the following major tasks:i.  To measure and analyze the results of reviews and testing.ii.  To monitor and document progress, test coverage and exit criteria.iii.  To provide information on testing.iv.  To initiate corrective actions.v.   To make decisions. 2)  Analysis and Design: Test analysis and Test Design has the following major tasks:i.   To review the test basis. (The test basis is the information we need in order to start the test analysis and   create our own test cases. Basically it’s a documentation on which test cases are based, such as requirements, design specifications, product risk analysis, architecture and interfaces. We can use the test basis documents to understand what the system should do once built.)ii.   To identify test conditions.iii.  To design the tests.iv.  To evaluate testability of the requirements and system.v.  To design the test environment set-up and identify and required infrastructure and tools. 3)  Implementation and Execution:During test implementation and execution, we take the test conditions into test cases and procedures and other testware such as scripts for automation, the test environment and any other test infrastructure. (Test cases is a set of conditions under which a tester will determine whether an   application is working correctly or not.)(Testware is a term for all utilities that serve in combination for testing a software like scripts, the test environment and any other test infrastructure for later reuse.) Test implementation has the following major task:i.  To develop and prioritize our test cases by using techniques and create test data for those tests. (In order to test a software application you need to enter some data for testing most of the features. Any such specifically identified data which is used in tests is known as test data.)We also write some instructions for carrying out the tests which is known as test procedures.We may also need to automate some tests using test harness and automated tests scripts. (A test harness is a collection of software and test data for testing a program unit by running it under different conditions and monitoring its behavior and outputs.)ii. To create test suites from the test cases for efficient test execution.(Test suite is a collection of test cases that are used to test a software program   to show that it has some specified set of behaviours. A test suite often contains detailed instructions and information for each collection of test cases on the system configuration to be used during testing. Test suites are used to group similar test cases together.)iii. To implement and verify the environment. Test execution has the following major task:i.  To execute test suites and individual test cases following the test procedures.ii. To re-execute the tests that previously failed in order to confirm a fix. This is known as confirmation testing or re-testing.iii. To log the outcome of the test execution and record the identities and versions of the software under tests. The test log is used for the audit trial. (A test log is nothing but, what are the test cases that we executed, in what order we executed, who executed that test cases and what is the status of the test case (pass/fail). These descriptions are documented and called as test log.).iv. To Compare actual results with expected results.v. Where there are differences between actual and expected results, it report discrepancies as Incidents. 4)  Evaluating Exit criteria and Reporting:Based on the risk assessment of the project we will set the criteria for each test level against which we will measure the “enough testing”. These criteria vary from project to project and are known as exit criteria.Exit criteria come into picture, when:— Maximum test cases are executed with certain pass percentage.— Bug rate falls below certain level.— When achieved the deadlines. Evaluating exit criteria has the following major tasks:i.  To check the test logs against the exit criteria specified in test planning.ii.  To assess if more test are needed or if the exit criteria specified should be changed.iii.  To write a test summary report for stakeholders. 5)  Test Closure activities:Test closure activities are done when software is delivered. The testing can be closed for the other reasons also like: When all the information has been gathered which are needed for the testing. When a project is cancelled. When some target is achieved. When a maintenance release or update is done. Test closure activities have the following major tasks:i.  To check which planned deliverables are actually delivered and to ensure that all incident reports have been resolved.ii. To finalize and archive testware such as scripts, test environments, etc. for later reuse.iii. To handover the testware to the maintenance organization. They will give support to the software.iv To evaluate how the testing went and learn lessons for future releases and projects.
• #23: Comparison of the mindset of the tester and developer:The testing and reviewing of the applications are different from the analysing and developing of it. By this we mean to say that if we are building or developing applications we are working positively to solve the problems during the development process and to make the product according to the user specification. However while testing or reviewing a product we are looking for the defects or failures in the product. Thus building the software requires a different mindset from testing the software. The balance between self-testing and independent testing:The comparison made on the mindset of the tester and the developer in the above article is just to compare the two different perspectives. It does not mean that the tester cannot be the programmer, or that the programmer cannot be the tester, although they often are separate roles. In fact programmers are the testers. They always test their component which they built. While testing their own code they find many problems so the programmers, architect and the developers always test their own code before giving it to anyone. However we all know that it is difficult to find our own mistakes. So, programmers, architect, business analyst depend on others to help test their work. This other person might be some other developer from the same team or the Testing specialists or professional testers. Giving applications to the testing specialists or professional testers allows an independent test of the system. This degree of independence avoids author bias and is often more effective at finding defects and failures. There is several level of independence in software testing which is listed here from the lowest level of independence to the highest:i.  Tests by the person who wrote the item.ii.  Tests by another person within the same team, like another programmer.iii.  Tests by the person from some different group such as an independent test team.iv.  Tests by a person from a different organization or company, such as outsourced testing or certification by an external body. Clear and courteous communication and feedback on defects between tester and  developer:We all make mistakes and we sometimes get annoyed and upset or depressed when   someone points them out. So, when as testers we run a test which is a good test from our viewpoint because we found the defects and failures in the software. But at the same time we need to be very careful as how we react or report the defects and failures to the programmers. We are pleased because we found a good bug but how will the requirement analyst, the designer, developer, project manager and customer react. The people who build the application may react defensively and take this reported defect as personal criticism. The project manager may be annoyed with everyone for holding up the project. The customer may lose confidence in the product because he can see defects. Because testing can be seen as destructive activity we need to take care while reporting our defects and failures as objectively and politely as possible. The balance between self-testing and independent testing
• #24: Benefits of independence testing: An independent tester can repeatedly find out more, other, and different defects than a tester working within a programming team – or a tester who is by profession a programmer. While business analysts, marketing staff, designers, and programmers bring their own assumptions to the specification and implementation of the item under test, an independent tester brings a different set of assumptions to testing and to reviews, which often helps in exposing the hidden defects and problems An independent tester who reports to senior management can report his results honestly and without any concern for reprisal that might result from pointing out problems in coworkers’ or, worse yet, the manager’s work. An independent test team often has a separate budget, which helps ensure the proper level of money is spent on tester training, testing tools, test equipment, etc. In addition, in some organizations, testers in an independent test team may find it easier to have a career path that leads up into more senior roles in testing. Risks of independence and integrated testing: There is a possibility that the testers and the test team can get isolated. This can take the form of interpersonal isolation from the programmers, the designers, and the project team itself, or it can take the form of isolation from the broader view of quality and the business objectives (e.g., obsessive focus on defects, often accompanied by a refusal to accept business prioritization of defects). This leads to communication problems, feelings of unfriendliness and hostility. Lack of identification with and support for the project goals, spontaneous blame festivals and political backstabbing. Even well-integrated test teams can suffer problems. Other project stakeholders might come to see the independent test team – rightly or wrongly – as a bottleneck and a source of delay. Some programmers give up their responsibility for quality, saying, ‘Well, we have this test team now, so why do I need to unit test my code?’
• #26: Good design – looks and style: It is very important to have a good design. The application or product should meet all the requirement specifications and at the same time it should be user friendly. The customers are basically attracted by the good looks and style of the application. The right color combinations, font size and the styling of the texts and buttons are very important. Good functionality – it does the job well: Along with the good looks of the application or the product it’s very important that the functionality should be intact. All the features and their functionality should work as expected. There should not be any deviation in the actual result and the expected result. Reliable – acceptable level of breakdowns or failure: After we have tested for all the features and their functionalities it also very important that the application or product should be reliable. For example: There is an application of saving the students records. This application should save all the students records and should not fail after entering 100 records. This is called reliability. Consistency: The software should have consistency across the application or product. Single software can be multi dimensional. It is very important that all the different dimensions should behave in a consistent manner. Durable – lasts as long as it should: The software should be durable. For example if software is being used for a year and the number of data has exceed 5000 records then it should not fail if number of records increases. The software product or application should continue to behave in the same way without any functional breaks. Good after sales service: Once the product is shipped to the customers then maintenance comes into the picture. It is very important to provide good sales services to keep the customers happy and satisfied. For example if after using the product for six months the customer realizes to make some changes to the application then those changes should be done as fast as possible and should be delivered to the customers on time with quality. Value for money: It’s always important to deliver the product to the customers which have value for money. The product should meet the requirement specifications. It should work as expected, should be user friendly. We should provide good services to the customers. Other than the features mentioned in the requirement specifications some additional functionality could be given to the customers which they might not have thought of. These additional functionalities should make their product more user friendly and easy to use. This also adds value for money.
• #27: Verification is done at the starting of the development process. It includes reviews and meetings, walk-throughs, inspection, etc. to evaluate documents, plans, code, requirements and specifications. Suppose you are building a table. Here the verification is about checking all the parts of the table, whether all the four legs are of correct size or not. If one leg of table is not of the right size it will imbalance the end product. Similar behavior is also noticed in case of the software product or application. If any feature of software product or application is not up to the mark or if any defect is found then it will result into the failure of the end product. Hence, verification is very important. It takes place at the starting of the development process. It answers the questions like: Am I building the product right? Am I accessing the data right (in the right place; in the right way). It is a Low level activity Performed during development on key artifacts, like walkthroughs, reviews and inspections, mentor feedback, training, checklists and standards. Demonstration of consistency, completeness, and correctness of the software at each stage and between each stage of the development life cycle. Advantages of Software Verification : Verification helps in lowering down the count of the defect in the later stages of development. Verifying the product at the starting phase of the development will help in understanding the product in a better way. It reduces the chances of failures in the software application or product. It helps in building the product as per the customer specifications and needs.
• #28: Validation is done at the end of the development process and takes place after verifications are completed. It answers the question like: Am I building the right product? Am I accessing the right data (in terms of the data required to satisfy the requirement). It is a High level activity. Performed after a work product is produced against established criteria ensuring that the product integrates correctly into the environment. Determination of correctness of the final software product by a development project with respect to the user needs and requirements. Advantages of Validation: During verification if some defects are missed then during validation process it can be caught as failures. If during verification some specification is misunderstood and development had happened then during validation process while executing that functionality the difference between the actual result and expected result can be understood. Validation is done during testing like feature testing, integration testing, system testing, load testing, compatibility testing, stress testing, etc. Validation helps in building the right product as per the customer’s requirement and helps in satisfying their needs. Validation is basically done by the testers during the testing. While validating the product if some deviation is found in the actual result from the expected result then a bug is reported or an incident is raised. Not all incidents are bugs. But all bugs are incidents. Incidents can also be of type ‘Question’ where the functionality is not clear to the tester. Hence, validation helps in unfolding the exact functionality of the features and helps the testers to understand the product in much better way. It helps in making the product more user friendly.