Testing
- 1. Ome Testing: Testing is a process of executing a program with the intent of finding an error. The purpose is reducing post release expenses. Primary Benefits of Testing: A good test has high probability of finding an as yet undiscovered error. Secondary Benefit: To check whether the functions appear to be working according to specifications. Debug: Is a process of locating exact cause of error and removing that cause. QA: QA involves in the entire software development process – monitoring and improving the process and making sure any agreed upon standards and procedures are followed and ensuring that problems are found and dealt with. QA is about having an overall development and management process that provides right environment for ensuring quality of final product. QA is best carried out on process. QA should take place at every stage of SDLC. QC: is about checking at the end of some development process (E.g.- a design activity ) that we have built quality in i.e. that we have achieved the required quality with our methods. QC is like testing a module against requirement specification or design document , measuring response time , throughput etc. QC is best carried out on products QC should be done at end of every SDLC i.e. when product building is complete. Verification: Typically involves the reviews and meetings to evaluate plans, docs, code requirements and specifications. This can be done with checklists, issue lists, walkthroughs and inspections. Validation: Typically involves the actual testing after verification. validation checks that the system meets the customer’s requirements at the end of the life cycle The major purpose of verification and validation activities is to ensure that software design, code, and documentation meet all the requirements imposed on them. Walkthroughs: This is the informal meeting for evaluation purpose. Inspection: Its just formal walkthrough.
- 2. Test Design: This document records what need to tested in the Application. This should abstract from Requirement specification, Design spec. Test Procedures: This records how to execute the set of test cases, specifying the exact process to be followed to conduct each of test cases. Test Case: Is a document that describing input, action or event and expected response to determine whether feature of application working correctly or not. Developing Test Cases: - Design test cases for unit test event based on objectives of test - Should be specifications derived tests - Should consider Equivalence partitioning - State Transitions testing - Design test cases that include abnormal situations and data. - Design test cases for all known and expected error conditions - Design test cases to be easily duplicated - Each test case must be described input and predicted result and conditions under which the test has been run. Test Case design Test Case ID: It is unique number given to test case in order to be identified. Test description: The description if test case you are going to test. Revision history: Each test case has to have its revision history in order to know when and by whom it is created or modified. Function to be tested: The name of function to be tested. Environment: It tells in which environment you are testing. Test Setup: Anything you need to set up outside of your application for example printers, network and so on. Test Execution: It is detailed description of every step of execution. Expected Results: The description of what you expect the function to do. Actual Results: pass / failed If pass - What actually happen when you run the test. If failed - put in description of what you've observed. Test Case Design Techniques: White box techniques: Branch Testing, Condition Testing. Black box testing techniques are Boundary value analysis, Equivalence partitioning. Equivalence Partitioning: Is the process of taking all of the possible test values and planning them into classes (groups). Test cases should be designed to test one value from each group. So that it uses fewest test cases to cover maximum input requirements.
- 3. Basis Path Testing: A white box test case design technique that uses the algorithmic flow of the program to design tests. Boundary Value Analysis: This is a selection technique where the test data lie along the boundaries of the input domain. Branch Testing: In branch testing, test cases are designed to exercise control flow braches or decision points in a unit. Cause Effect Graph: A graphical representation of inputs and the associated outputs effects, which can be used to design test cases. Characteristics of a Good Test: They are likely to catch bugs, no redundant , not too simple or too complex. Test case is going to be complex if you have more than one expected results. Test Plan: Cont 1. Introduction 2. Scope – Test Items, Features To be tested, Features Not to be Tested 3. Approach 4. Test Environment 5. Schedule and Testing Tasks 6. Item pass/fail 7. Suspension Criteria 8. Risk and Contingencies 9. Test Deliverables 10. Approves of the Plan Introduction: Summary of the items and features to be tested. References to related documents. Scope: Test Items: Names of all the items (software) being tested are listed here and their version too. These include Programs, Batch files, Configuration files and control tables. Features To Be Tested: A detailed list of functions or features will be prepared from reviewing Detail Design Document, Functional specification and Requirement specifications. Features Not To Be Tested: If any feature is not being tested, its should be clearly mentioned here with reason for not being tested. Eg: Sending financial transactions over a private network may not be tested due to non availability of infrastructure. Approach: For each major group features specify the type of testes like regression, stress etc., Specify mojor activities, tools, techniques. Test Environment: Describe test environment minimal and optimal needs. This would include Hardware, Network, Communications, OS and printers and Security and other tools.
- 4. Schedule and Tasks: Provide a detailed schedule of testing activities and responsibilities. Indicate dependencies and time frames for testing activities. Item Pass/Fail criteria: Specify the criteria to be used to determine whether each test item has passed or failed. Suspension criteria: Criteria to be used to suspend the testing activity. Criteria to be used resume the testing activity Risk and Contingencies: Identify the high risk assumptions of the test plan. Specify the contingency plans for each Test Deliverables: Identify the deliverables documents: test plan, test design specifications, test case specifications, test procedure documents, test item transmittal reports, test logs, test incidental reports and test summary reports. And also identify test input and output data. Approvals: Specify the names and titles of all persons who must approve the plan Provides space for signatures and dates Test Techniques: Black box Testing: This testing is completely based on specifications and requirements and not bothered about internal logic of testing. White box Testing: This is completely based on internal logic of code. Tests are based on coverage of code statements, paths, branches and conditions. Test Phase Levels: Unit Testing: This is the most micro scale of testing, to test particular function or module. This will done by programmers only. Incremental Integration: Continuous testing of the application as new functionality is added. Integration testing: Testing the combined parts of the application to determine if they functioning correctly all together. The parts can be code modules, individual applications or client server applications.
- 5. Functional Testing: Functionality testing is performed to verify that a software application performs and functions correctly according to design specifications. By inputting both valid and invalid data, we verify that the output of these features and functional areas meets predefined expectations System Testing: Black box type testing based on overall requirement specifications. Acceptance Testing: Final testing based on the end user or customer acceptance criteria. Determining if the software is satisfactory to a customer. Its of two types Alpha testing and Beta testing. Other Tests: Alpha Testing: This type of testing is conducted at the developer’s site by a customer. The developer makes note of the errors and usage problems. Beta Testing: Conducted at one or more customer sites by the end user(s) of the software. Unlike alpha testing, the developer is generally not present. Adhoc Testing: This is also called as Monkey testing. No presumptions are made in the advancement for the testing of the remaining features based on the following results due to a lack of repeatable tests. End to End Testing: This is similar to system testing in a environment that mimics real world use, such as interacting network communications, databases and network environment or interacting with other hardware, applications or systems Recovery Testing: Forces the software to fail in a variety of ways and verifies that recovery is properly performed. If recovery is automatic, re-initialization, check-pointing mechanisms, data recovery, and restart are each evaluated for correctness. If recovery requires human intervention, the mean time to repair is evaluated to determine whether it is within acceptable limits. Sanity Testing/Smoke Testing: This is initial testing effort to determine if the new software is performing enough to accept major testing. Regression Testing: Re testing after bug fixes or modification of software or environment. We don’t know how much retesting is needed near the end of application. For this automated tools will be used. Load Testing: Testing the application under heavy loads. Such as testing web application under range of loads to determine at what point of time the system response time degrades or fails. Stress Testing: This can described as the functional testing the system under unusually heavy loads, heavy repetitions of same actions or inputs and input of large numerical values and large complex queries to database. Performance: Load and Stress testing combindley called as performance testing. Compatibility Testing: Compatibility Testing test the functionality and performance of a software application across multiple platform configurations. This type of testing typically uncovers compatibility issues with operating systems, other software applications, and hardware components.
- 6. Bottom-up testing: An approach to integration testing where the lowest level components are tested first, then used to facilitate the testing of higher level components. The process is repeated until the component at the top of the hierarchy is tested. The components used are called ‘Drivers’. Top-down testing: An approach to integration testing where the component at the top of the component hierarchy is tested first, with lower level components being simulated by stubs. Tested components are then used to test lower level components. The process is repeated until the lowest level components have been tested. Software Quality: Quality software is reasonably bug free, delivered on time, within budget limit, meets requirements and is maintainable. CMM: Capability maturity model is developed by SEI. This is model of 5 levels of organization maturity, that determine effectiveness of delivering quality software. CMM1: Characterized by chaos, periodic panics and heroic efforts required by individuals to successfully complete the project. Success may not be repeated. CMM2: software project tracking, requirements management, realistic project plan and configuration management process are in place. In this case success is repeatable. CMM3: standard software development and maintenance processes are integrated throughout the organization. A software engineering process group is oversees the process in the organization, and training programs. CMM4: Metrics are used to track productivity, processes and products. Project performance is predictable and quality is consistently high. CMM5: It focus on continuous improvement in process. It can predict impact of new technologies and new processes and can be implemented when its required. ISO: International organization for standardization(9001-2000). This concerns quality systems assessed by outside auditors. IEEE: Institute of Electrical and Electronics Engineering. IEEE/ANSI 829 – Software Test Documentation IEEE/ANSI 1008 – Software Unit Testing IEEE/ANSI 730 – Software Quality Assurance Plans
- 7. ISO: ISO = 'International Organization for Standards' - The ISO 9001, 9002, and 9003 standards concern quality systems that are assessed by outside auditors, and they apply to many kinds of production and manufacturing organizations, not just software. The most comprehensive is 9001, and this is the one most often used by software development organizations. It covers documentation, design, development, production, testing, installation, servicing, and other processes. ISO 9000-3 (not the same as 9003) is a guideline for applying ISO 9001 to software development organizations. The U.S. version of the ISO 9000 series standards is exactly the same as the international version, and is called the ANSI/ASQ Q9000 series. The U.S. version can be purchased directly from the ASQ (American Society for Quality) or the ANSI organizations. To be ISO 9001 certified, a third-party auditor assesses an organization, and certification is typically good for about 3 years, after which a complete reassessment is required. Note that ISO 9000 certification does not necessarily indicate quality products - it indicates only that documented processes are followed. (Publication of revised ISO standards are expected in late 2000; see http://www.iso.ch/ for latest info.) Software Life Cycle: It includes aspects such as initial concept, requirement analysis, functional design, internal design, documentation planning, test planning, coding documentation, testing, retesting, and maintenance. Configuration Management: It covers the processes used to control, coordinate and track: Code, documentation, requirements, problems and change requests, designs and changes made to them, and who makes changes to them. The purpose of software configuration management is to identify all the interrelated components of software and to control their evolution throughout the various life cycle phases. Control Version As an application evolves over time, many different versions of its software components are created, and there needs to be an organized process to manage changes in the software components and their relationships Change Control Change control is the process by which a modification to a software component is proposed, evaluated, approved or rejected, scheduled, and tracked. Business Requirements: This document describes users needs for the application. Functional Requirements: This would be primarily derived from Business Requirements. This describes functional needs. Prototype: This is look and feel representation of the application that is proposed. This basically shows placements of the fields and modules and generic flow of the application. Test Strategy: A Test strategy is a statement of the overall approaches to the testing, identifying what levels of testing are to be applied and the techniques, methods and tools to be used. A test strategy should ideally be organization wide, being applicable to all of an organizations projects.
- 8. Here levels refers Unit, Integration and system etc Methods refers to dynamic and static. Techniques refers to white box and black box testing Test Methodology: This is actual implementation we use for each and every project. This may defer from project to project. For example we may mention in test strategy that we use load test but in the methodology of a particular project we may not perform it Test Policy: Test Policy is a management definition to testing for a department. Its specifies milestones to be achieved. Test policy contains these side headings a. Def of testing b. Testing System: Methods through which testing will be achieved c. Evaluation: How information services management will measure and evaluate testing d. Standards: The standards against which testing will be measured Test Life Cycle: Test Analysis(Risk Analysis), Test plan , Designing Test cases, Executing Test cases, Maintaining test logs, Reporting Risk management is the process of measuring, or assessing risk and then developing strategies to manage the risk. In ideal risk management, a prioritization process is followed whereby the risks with the greatest loss and the greatest probability of occurring are handled first Risk Analysis: Schedule Risk: Factors that may affect the schedule of testing Technology Risk: Risks on hardware and software of the application Resource Risk: Test Team availability on slippage of the project schedules. Support Risk: Clarification required on specifications and availability of personnel for the same Bug: Something abnormal to the expected behavior is called a bug Error: A software bug found before software release is called a Error Defect: A software bug found after the software release is send to customer Fault: Since a defect is a software bug found in production. Every defect leads to a fault to the system SDLC: Waterfall Model: This approach breaks the development cycle down into discrete phases, each with rigid sequential begin and end. Each phase is fully completed before the next is started. Once a phase is completed, and it never goes back to change it. The phase are Requirements, Designing, coding, Testing and maintenance.
- 9. Iterative Model: This model does not start with full specifications of requirements. Instead development begins by specifying and implementing just part of the software, which can be reviewed in order to identify further requirements. This process is repeated, producing a new version of the software for each cycle of the model Progressive Model: Testing Methods: Static Testing: Static testing approaches are time independent. They do not necessarily involve either manual or automated execution of product. Example includes syntax testing, structured walkthroughs, and inspections. Dynamic Testing: Dynamic testing techniques are time dependent and involve executing specific sequence of instructions on paper. For example Boundary testing is a dynamic testing technique that requires the execution of test cases on the computer with a specific focus on the boundary values associated with the inputs or outputs of the program. Test Metrics: Test metric is a measurable unit used to measure the productivity and quality of the system produced. Generally classified into size oriented metrics - based on lines of code (KLOC) Function oriented metrics - we use Function point metric to calculate the functionality delivered by a system For example these are the test metrics u can say as example *Test Coverage* = Number of units (KLOC/FP) tested / total size of the system *Test cost (in %)* = Cost of testing / total cost *100 *Cost to locate defect* = Cost of testing / the number of defects located *Defects detected in testing (in %)* = Defects detected in testing / total system defects*100 *Acceptance criteria tested* = Acceptance criteria tested / total acceptance criteria If u want to mention your company testing department performance we use Defect removal Efficiency having formulae: (No. of defects found by testing team/ (No. of defects found by testing team + No. of defects found by Customer))*100 For example testing department has found 90 defects and customer has found 10 defects then DRE: (90/ 100 )*100 = 90% defect removal efficiency
- 10. IF the case is reverse then our defect removal efficiency will be only 10% Cyclocomplex: This metric is an indication of the number of 'linear' segments in a method (i.e. sections of code with no branches) and therefore can be used to determine the number of tests required to obtain complete coverage. It can also be used to indicate the psychological complexity of a method. A method with no branches has a Cyclomatic Complexity of 1 since there is 1 arc. This number is incremented whenever a branch is encountered. In this implementation, statements that represent branching are defined as: 'for', 'while', 'do', 'if', 'case' (optional), 'catch' (optional) and the ternary operator (optional). The sum of Cyclomatic Complexities for methods in local classes is also included in the total for a method. Check In (or) Check Out Check in means changing or modifying or adding the Information present in Main Repository when other person check out the changes will be reflected. Bug leakage; If a bug which has to be found in the requirement phase is found in the design phase is said to be bug leakage.