![SOFTWARE QUALITY
There are various software quality models have
been proposed for Software Quality and its
related attributes
Two most popular models are
CMM [Software Capability Maturity Model]
ISO [International Standard Organization]
In the field of Testing two popular models are
available,
TPI [Test Process Improvement]
TMM [Test Maturity Model]
16](https://image.slidesharecdn.com/unit1-basicconceptsoftestingquality-150901043949-lva1-app6892/85/Unit-1-basic-concepts-of-testing-quality-16-320.jpg)
![1. STATIC ANALYSIS [ROLE OF TESTING]
It is based on the examination of a number of documents,
requirements documents,
software models,
design documents
source code.
It also includes code review, inspection, walk-through,
algorithm analysis, and proof of correctness.
It does not involve actual execution of the code under
development.
Instead, it examines code and reasons over all possible
behaviours that might arise during run time.
For Ex., Compiler optimizations are standard static analysis.24](https://image.slidesharecdn.com/unit1-basicconceptsoftestingquality-150901043949-lva1-app6892/85/Unit-1-basic-concepts-of-testing-quality-24-320.jpg)
![2. DYNAMIC ANALYSIS [ROLE OF TESTING]
It involves actual program execution with input values.
During execution, it observes the behavioural and
performance properties of the program.
For practical considerations, a finite subset of the input
set can be selected.
Therefore, in testing, we observe program behaviours
and reach a conclusion about the quality of the system.
Careful selection of a finite test set is crucial to reaching
a reliable conclusion.
25](https://image.slidesharecdn.com/unit1-basicconceptsoftestingquality-150901043949-lva1-app6892/85/Unit-1-basic-concepts-of-testing-quality-25-320.jpg)
![ROLE OF TESTING [CONT…]
Both Analysis techniques are complementary
in nature, and for better effectiveness, both
must be performed repeatedly and alternated.
Practitioners and researchers need to remove
the boundaries between static and dynamic
analysis and create a hybrid analysis that
combines the strengths of both approaches
26](https://image.slidesharecdn.com/unit1-basicconceptsoftestingquality-150901043949-lva1-app6892/85/Unit-1-basic-concepts-of-testing-quality-26-320.jpg)
Unit 1 basic concepts of testing & quality
- 1. BASIC CONCEPT OF TESTING AND QUALITY UNIT - 1 1 Prepared By: Ravi J Khimani, CSE Department, SLTIET, Rajkot
- 2. QUALITY REVOLUTION Quality – critical issue in product development In the past, near about 1940, Quality Revolution began. Due to – Global Competition, OutSourcing, Off- Shoring and increasing customer expectation. Developing quality product in tighter schedule is big issue for companies. 2
- 3. QUALITY REVOLUTION In old approach, Efforts to improve quality is centered around the end of product development. In new approach, improve quality encompasses all phases: from Requirement Analysis to Final Delivery of Product. 3
- 4. QUALITY REVOLUTION An effective quality process must focus on: Paying attention to customer expectation Making efforts to continuously improve quality. Integrate measurement with design and development. Develop system level perspectives with concentrating on methodology and process. Eliminate waste through continuous improvement 4
- 5. QUALITY REVOLUTION Quality movement started around 1940-1950 William Edward Deming, gives literature on Statistical Quality Control (STQ) STQ is discipline based on measurements and statistics. In which, decisions are made and plans developed on the collection and evaluation of facts and data. 5
- 6. QUALITY REVOLUTION Deming gave Shewhart Method for STQ. In which, he gave Plan – Do – Check – Act (PDCA) cycle, also known as Shewhart Cycle. 6
- 7. QUALITY REVOLUTION Between 1950 – 1970, companies came up with innovative principle, known as “Lean Principle”. Which is, “A systematic approach to identify and eliminate waste through continuous improvement, flowing the product at the pull of the customer expectations.” For Example, to compress the time period in banking payments. 7
- 8. QUALITY REVOLUTION In the era of 1950, Joshep M Juran of US proposed raising level of Quality Management from Manufacture Dept. to overall organization. That Quality Management is known as Total Quality Control (TQC). It includes companywide activities, audits, quality circle and promotions of quality 8
- 9. QUALITY REVOLUTION That TQC emphases on, Quality comes first not short-term profits. Customer comes first not the producers. Decisions are based on Data and Facts. Management is participatory and respectful of all employees Management is driven by cross-functional committees covering product planning, design, sales, marketing, manufacturing, purchasing, distribution. 9
- 10. QUALITY REVOLUTION One of the TQC method developed by Japan, is known as Ishikawa OR cause-and-effect method. It says, product quality comes from four causes, materials, 10
- 11. QUALITY REVOLUTION Traditionally, the TQC and Lean Concept are applied to Manufacturing Department. The software development process uses these concepts through “Software Capability Maturity Model (CMM)” model, to guide the production of qualitative software. Which provides framework to discuss software production issues. 11
- 12. SOFTWARE QUALITY What is Software Quality? Quality is a complex concept – means different things to different people. Kitchenham and Pfleeger discuss software quality in five different ways: Transcendental View: It envisages quality as something that can be recognized but is difficult to define. It can be also applied to complex area of regular life. Not specific to only Software Quality. 12
- 13. SOFTWARE QUALITY User View: It perceives quality as fitness for purpose. Key question: “Does product satisfy user needs?” Manufacturing View: Here quality level is determined by considering product’s specification. Product View: Quality is viewed as inherent characteristics. Like Internal Characteristics, external characteristics. 13
- 14. SOFTWARE QUALITY Value Based View: Depends on amount a customer is willing to pay for it. In mid-1970, McCall, Richards and Walters were gave the software quality in two terms, Quality Factors Quality Criteria 14
- 15. SOFTWARE QUALITY A quality Factor represents behavioral characteristics of system. For example, Correctness, Reliability, Efficiency, Testability, Maintainability, Modularity and Reusability. A Quality Criteria Is an attribute of Quality Factor that can be related to Software Development. 15
- 16. SOFTWARE QUALITY There are various software quality models have been proposed for Software Quality and its related attributes Two most popular models are CMM [Software Capability Maturity Model] ISO [International Standard Organization] In the field of Testing two popular models are available, TPI [Test Process Improvement] TMM [Test Maturity Model] 16
- 17. FAILURE, FAULT, ERROR AND DEFECT In software testing, we can find references to these four terms. In fault – tolerant computing community, Failure: A failure is said to occur whenever the external behaviour of a system does not confirm to that prescribed in the system specification. Error: An error is a state of the system. In the absence of any corrective action by the system, an error state could lead to a failure which would not be attributed to any event subsequent to the error. 17
- 18. FAILURE, FAULT, ERROR AND DEFECT Fault: A fault is the adjudged (true) cause of an error. A fault may remain undetected for a long time, until some event activates it. When an event activates a fault, it brings the program into an intermediate error state. If computation is allowed to proceed from an error 18
- 19. FAILURE, FAULT, ERROR AND DEFECT in fault-tolerant computing, corrective actions can be taken to take a program out of an error state into a desirable state such that subsequent computation does not eventually lead to a failure. The process of failure manifestation can therefore be succinctly represented as a behaviour chain as follows: 19
- 20. FAILURE, FAULT, ERROR AND DEFECT In software, a system may be defective due to design issues; certain system states will expose a defect, resulting in the development of faults are defined as incorrect signal values or decisions within the system. In practical purpose, both term, Defect and Fault are synonymous terms. 20
- 21. FAILURE, FAULT, ERROR AND DEFECT For Example, Consider a small organization. Defects in the organization’s staff promotion policies can cause improper promotions, viewed as faults. The resulting ineptitudes & dissatisfactions are errors in the organization’s state. The organization’s personnel or departments probably begin to malfunction as result of the errors, in turn causing an overall degradation of performance. The end result can be the organization’s failure to achieve its goal. 21
- 22. ROLE OF TESTING Testing plays an important role in achieving and assessing the quality of a software product . we can improve the quality of the products as we repeat a test - find defects – fix cycle during development. But how can we perform system level tests before releasing product? 22
- 23. ROLE OF TESTING Friedman and Voas say, “software testing is a verification process for software quality assessment and improvement.” Software quality assessment is divided into two categories, Static Analysis Dynamic Analysis 23
- 24. 1. STATIC ANALYSIS [ROLE OF TESTING] It is based on the examination of a number of documents, requirements documents, software models, design documents source code. It also includes code review, inspection, walk-through, algorithm analysis, and proof of correctness. It does not involve actual execution of the code under development. Instead, it examines code and reasons over all possible behaviours that might arise during run time. For Ex., Compiler optimizations are standard static analysis.24
- 25. 2. DYNAMIC ANALYSIS [ROLE OF TESTING] It involves actual program execution with input values. During execution, it observes the behavioural and performance properties of the program. For practical considerations, a finite subset of the input set can be selected. Therefore, in testing, we observe program behaviours and reach a conclusion about the quality of the system. Careful selection of a finite test set is crucial to reaching a reliable conclusion. 25
- 26. ROLE OF TESTING [CONT…] Both Analysis techniques are complementary in nature, and for better effectiveness, both must be performed repeatedly and alternated. Practitioners and researchers need to remove the boundaries between static and dynamic analysis and create a hybrid analysis that combines the strengths of both approaches 26
- 27. OBJECTIVES OF TESTING The stakeholders in a test process are the programmers, the test engineers, the project managers, and the customers. A stakeholder is a person or an organization who influences a system’s behaviours or who is impacted by that system. Different stakeholders view a test process from different perspectives, as follows, 27
- 28. OBJECTIVES OF TESTING It does work: While implementing a program unit, the programmer may want to test whether or not the unit works in normal circumstances. The programmer gets much confidence if the unit works to his or her satisfaction. The same idea applies to an entire system as well—once a system has been integrated, the developers may want to test whether or not the system performs the basic functions. Here, for the psychological reason, the objective of testing is to show that the system works, rather than it does not work. 28
- 29. OBJECTIVES OF TESTING It does not work: Once the programmer (or the development team) is satisfied that a unit (or the system) works to a certain degree, more tests are conducted with the objective of finding faults in the unit (or the system). Here, the idea is to try to make the unit (or the system) fail. Reduce the risk of failure: Software systems contain faults, which cause the system to fail. “failing from time to time” gives rise of failure rate. As faults are discovered and fixed during more tests, Thus, a higher level objective of performing tests is to bring29
- 30. OBJECTIVES OF TESTING Reduce the cost of testing: The different kinds of costs associated with a test process include the cost of designing, maintaining, and executing test cases, the cost of analyzing the result of executing each test case, the cost of documenting the test cases, and the cost of actually executing the system and documenting it. Therefore, the less the number of test cases designed, the less will be the associated cost of testing. However, producing a small number of arbitrary test cases is not a good way of saving cost. Objective – more test cases – more reliable software. Its define effectiveness of test cases. 30
- 31. TEST CASE Basic Form, it’s a pair of, <input, expected outcome> In stateless system, outcome depends only on the current input. For example, calculate square root of number, A compiler for C language. 31
- 32. TEST CASE In state-oriented system, outcome depends on current input as well as current state of system. For this test case consists of Sequence of <input, expected outcome> pairs, according to state. It also consists of Decision and Timing factor. For example, ATM Machine, 32
- 33. EXPECTED OUTCOME Outcome of program execution is complex entity, which includes, Value produced State change (program or database) Set of values interpreted together for valid outcome. ORACLE: is important concept of Test Design. Oracle is entity – Program, Process, Human Expert or body of data – that tells us the expected outcome of test. 33
- 34. EXPECTED OUTCOME Expected Outcome should be computed while designing test cases. Before program execution and with selected inputs. Will eliminate implementation biasing. The process is of – execute program – observe actual output – verify actual output – use that verified output in subsequent test 34
- 35. CONCEPT OF COMPLETE TESTING It means “there are no undiscoverable faults at the end of test phase”. It’s near impossible, due to Input domain is to large Program may have large number of state based on valid and invalid inputs. Timing constraint Design (for program) issues too complex Impossible to create all possible execution environments like weather, temperature, altitude, pressure, etc… “Input value which is valid but is not property timed is called inopportune Input”. 35
- 36. CENTRAL ISSUE IN TESTING Impossible to test complete input domain to discover faults. Select subset of Domain is best thing to test program. 36
- 38. TEST ACTIVITIES Identify objective to be tested Clear purpose must be associated with every test case. Select Inputs Based on specification, source code and expectation Compute the expected outcome Can be done from high-level understanding of the38
- 39. TEST ACTIVITIES Set up execution environment of program Prepare right execution environment For ex, Initialize local system, external to program like making network connection, right database system available. For ex, Initialize any remote, external system like distributed system to run client code. Execute the Program With selected input and observe actual output. Test Coordination concept is used in synchronizing different components of Test Cases, like different physical location. 39
- 40. TEST ACTIVITIES Analyze Test Result Comparison should be precise. After analyzing, Test Verdicts should be assigned, like Pass, Fail, inconclusive. Test Report must be written, contains reasons of failure of test cases analysis of failure A indication to actual outcome, test case, input values tested, environment of 40
- 41. VERIFICATION AND VALIDATION Two similar concept Verification Determine that product of given development phase satisfies the requirements established before starting of that phase. Like requirement specification, design specification, code, user manual. “Activities that check the correctness of a development phase are called Verification 41
- 42. VERIFICATION AND VALIDATION Validation Activities that determine and confirm that product meets its desired use. Aim at confirming that product meets user’s expectations. Focus on final product. Should be executed at early stage of development cycle, not at the end of development cycle. 42
- 43. VERIFICATION VS. VALIDATION Verification Validation Confirming that Building product correctly Confirming that building correct product Review interim works like specification, design specification, code, user manual. Is performed at the end of development phase of each part of system. Verification considers quality attributes like consistency, correctness, completeness. Validation considers only correctness and satisfaction. Can be applied through Statistic Analysis Techniques like inspection, walkthrough, reviews, checklists, Can be applied by running system in its actual environment using verity of Tests. 43
- 44. CORRECTNESS Correctness from software engineering perspective can be defined as the stickiness to the specifications that determine how users can interact with the software and how the software should be have when it is used correctly. If the software behaves incorrectly, it might take considerable amount of time to achieve.44
- 45. CORRECTNESS Below are some of the important rules for effective programming which are consequences of the program correctness theory. Defining the problem completely. Develop the algorithm and then the program logic. Reuse the proved models as much as possible. Prove the correctness of algorithms during the design phase. Developers should pay attention to the clarity and simplicity of your program. Verifying each part of a program as soon as it is developed.45
- 46. SOURCES OF INFORMATION FOR TEST CASES A software development process generates a large body of information, such as requirements specification, design document, and source code. In order to generate effective tests at a lower cost, test designers analyze the following sources of information: Requirements and functional specifications. Source code Input and output domains Operational Profiles Fault model [Error Guessing, Fault Seeding, Mutation 46
- 47. TEST LEVELS Performed at different levels involving complete system. Software goes through four stages: Unit, Integration, System and Acceptance level testing. First three inside developing organization Last one is outside developing organization These four stages of testing is known as V- 47
- 48. 48
- 49. CONTROL FLOW GRAPH It’s a graphical representation of program unit. Three symbols are used. 49
- 50. CONTROL FLOW GRAPH A rectangle represents sequential computation. A diamond box represents decision point. Two branches one decision box has, True or False. A small circle represents a merge point. We are not labeling merge point. 50
- 51. TESTING TYPES Black Box Testing Black-box testing is a method of software testing that examines the functionality of an application based on the specifications. It is also known as Specifications based testing. Independent Testing Team usually performs this type of testing during the software testing life cycle. This method of test can be applied to each and every level of software testing. There are different techniques involved in Black Box testing, like Equivalence Class, Boundary Value Analysis, Domain Tests, Orthogonal Arrays, Decision Tables, State Models, All-51
- 52. TESTING TYPES White Box Testing White box testing is a testing technique, that examines the program structure and derives test data from the program logic/code. The other names of white box testing are clear box testing, open box testing, logic driven testing or path driven testing or structural testing. White Box Testing Techniques: Statement Coverage - This technique is aimed at exercising all programming statements with minimal tests. Branch Coverage - This technique is running a series of tests to ensure that all branches are tested at least once. Path Coverage - This technique corresponds to testing all possible paths which means that each statement and branch is covered. 52
- 53. TESTING TYPES Model Based Testing Model-based testing is a software testing technique in which the test cases are derived from a model that describes the functional aspects of the system under test. It makes use of a model to generate tests that includes both offline and online testing. Importance Unit testing wont be sufficient to check the functionalities To ensure that the system is behaving in the same sequence of actions. Model-based testing technique has been adopted as an integrated part of the testing process. Commercial tools are developed to support model-based testing. 53
- 54. TESTING TYPES Interface Testing Interface Testing is performed to evaluate whether systems or components pass data and control correctly to one another. It is to verify if all the interactions between these modules are working properly and errors are handled properly. Verify that communication between the systems are done correctly Verify if all supported hardware/software has been tested Verify if all linked documents be supported/opened on all platforms Verify the security requirements or encryption while communication happens between systems Check if a Solution can handle network failures between Web site and application serve 54
- 55. TESTING TYPES Unit Testing Tests individual program units like function, class After surety of units working properly, modules are assembled to larger subsystem. By Software Developer. Integration Testing Performed on assembled units of system in unit testing. By Software Developer and Integration test 55
- 56. TESTING TYPES System Testing Object is to construct a reasonably stable system that can withstand to System-Level Testing. System level testing includes functionality testing, security testing, reliability testing, stability, stress, performance, load testing. Aim to discover most of the faults and verify fixes are working. System testing composed of create test plan, design test suit, prepare test environments, executing tests. 56
- 57. TESTING TYPES Regression Testing Perform throughout life cycle of system development. Performed when component of system is modified. Idea to verify that modification does not affect existing work and does not introduce new faults. It is distinct level of testing Sub-phase of unit, integration and system-level testing. 57
- 58. TESTING TYPES Acceptance Testing After completion of all above testing, product is delivered to customer. Customer performs their own series of tests, called acceptance tests. Objective of this testing to measure Quality of Product. Key notion is customer’s expectations from the system. 58
- 59. TESTING TYPES Two kinds of Acceptance Testing User Acceptance Testing (UAT): done by user to ensure system performs contractual acceptance criteria. Business Acceptance Testing (BAT): done by supplier’s development organization, to ensure that system will pass UAT. 59