SE - Lecture 8 - Software Testing State Diagram.pptx
- 2. Agenda Software Testing Types of Software Testing Testing Roles Released Software State Diagram 2
- 4. Introduction Software Testing is a method to check whether the actual software product matches expected requirements and to ensure that software product is defect free. It involves execution of software/system components using manual or automated tools to evaluate one or more properties of interest. The purpose of software testing is to identify errors, gaps or missing requirements in contrast to actual requirements. 3/1/20XX 4
- 5. 5 What is testing? + Testing is a process to help identify correctness, completeness, security, quality, reliability, in fact to determine if the software is useful for its intended purpose + Determine if things work as expected Documentation Processes Programs + "Program testing can be used to show the presence of bugs, but never to show their absence!" E.D.Dijkstra
- 6. Why Software Testing Important? + Though testing itself costs money, companies can save millions per year in development and support if they have a good testing technique and QA processes in place. Early software testing uncovers problems before a product goes to market. The sooner development teams receive test feedback, the sooner they can address issues such as: • Architectural flaws • Poor design decisions • Invalid or incorrect functionality • Security vulnerabilities • Scalability issues Sample footer text 3/1/20XX 6
- 7. 7 Faults, Defects and Debugging + We test to find faults - a difference from expected behaviour + Faults are caused by defects in the software + When a fault is discovered, you have to classify its urgency Colours not displayed correctly AI in a computer game not as good as hoped ATM dispensing incorrect amounts of cash + If you want to locate a programming fault you need to debug your program + Once located you still have to decide what to do. You may... decide to ignore it realise that the requirements are incorrect fix it
- 8. 8 Software Testing + Testing can:- Show errors in coding Give us greater confidence in our software (but software can still be discontinuous) Measure performance Indicate quality + Testing is NOT a substitute for quality, but it can reveal poor quality systems
- 9. 9 Testing & State Diagram + Software Verification, + Validation and Testing + Black Box + White Box Testing
- 10. 10 Testing roles + The following are likely on a testing project Tester Test Designer Test/QA Team Lead Test Build Manager Test Configuration Manager Test automater/automation developer Test administrator
- 12. 12 What to test? + You should test everything (if you are concerned about it being correct!) Documentation + translations! Procedures + Installation + Fault logging & tracking + Payment Programs + Unit testing + Integration testing + System testing Backups! + Do you back up your computer? + Do your backups work?
- 13. 13 What to test? + Every aspect in the development of a computer system can be tested + Requirements Do they represent reality/can they be physically implemented? Have previous systems with these requirements been built/did they have problems (feasibility analysis) Are they self-consistent? Check with the user + Analysis Are the relationships really present in the problem? Is it possible to meet all requirements with the model?
- 14. 14 What to test? + Design Will the system be able to implement the system within given time constraints? Can the design be built within budget? + Implementation Does each class work as specified? Do the classes work together properly? + Documentation Can the user understand it, and use it to solve problems? Does it match the functionality of the system?
- 15. 15
- 16. Testing Category Types of Testing Functional Testing Unit Testing Integration Testing Smoke UAT ( User Acceptance Testing) Localization Globalization Interoperability Non- Functional Testing Performance Endurance Load Volume Scalability Usability So on Maintenance Regression Maintenance 16 Note: This is not the complete list as there are more than 150 types of testing types and still adding. Also, note that not all testing types are applicable to all projects but depend on the nature & scope of the project.
- 17. 17 Unit Testing + Unit testing Checking low level modules work in isolation Modules typically work with other modules, so we have to create a framework to support the modules. Modules that are called can be replaced with simple versions that are easy to code up, and return values that allow you to test sufficiently. Similarly you may need to write code to call the module you are testing. This is often called a test harness.
- 18. 18 Integration testing + Once each unit has been tested, we start combining units to form larger groups. + Integration testing tests whether the components communicate with each other properly, and provide appropriate behaviour. + Need to establish pre/post conditions/invariants + Need to identify requirements that the component is meant to satisfy
- 19. 19 System testing + System testing tests all of the components together as they are expected to be delivered to the customer. + These tests are based against the use cases and scenarios created described during requirements. + See wikipedia link for a list of the different types of testing that can be done.
- 20. 20 Testing strategies in Software Engineering + The best tests are those that discover faults. As we can’t test for every possibility, we have to choose some tests over others, and hope they are representative. We have to have a measure by which we can choose tests This requires an understanding of what faults are likely. Different fault classes can be tested in different ways. + The most common types software errors not knowing the language typographical errors errors in problem solving logic algorithms that provide poor results in some cases single or critical input values that yield unanticipated results data structure defects in design or translation of the design misinterpretation of specifications
- 21. 21 Not knowing the language + In 1990, a bug in the New York telephone network switching software caused a cascading failure. + The programmer didn’t understand “break” properly. In the software (written in C), there was a long statement: do { switch (…) { case …: break; case …: if (…) break; else … } while (…); Example Software Error https://www.latimes.com/archives/la- xpm-1990-01-17-fi-215-story.html
- 22. 22 Typographical errors + Errors where mistyping or accidentally choosing the wrong symbol. + C is very vulnerable as many character sequences are so similar to other valid sequences. if (a = b) - should be if (a==b) ? printf ("a & b is %dn", a & b); printf ("a && b is %dn", a && b); & and && are very different! Example Software Error
- 23. 23 Algorithms that provide poor results in some cases + Computing with floating point numbers is especially vulnerable to the order in which items are manipulated. Example Software Error
- 24. 24 Single or critical input values that yield unanticipated results + USS Yorktown was experimenting with a computer controlled system for propulsion. + A Microsoft database stored information for the control systems + A user entered in 0 into a field that wasn’t validated, resulting in a divide by 0 error. + The failure spread, and locked the entire system. + The ship had to be towed back to port. Example Software Error
- 25. 25 Testing for common errors + We need to check that common problems are addressed in our tests. + Some tests are done by the compiler - e.g. passing correct parameters to a function, checking assignment statements, checking syntax. + Some languages are better than others at providing support for this. if (a = b) {…} C doesn’t prevent this, but Java (mostly) does. + Primitive languages like C often require the help of static code checking tools, such as lint.
- 26. 26 Two types of testing + We can examine the modules (classes) in two ways + White box testing Write tests for the internals of the class - the if statements, while statements, algorithms etc. Perform code reviews/write the code in pairs. Involves a second person checking your program before you start performing run-time tests. + Black Box testing Ignores the internal implementation details and concentrates on the service that the modules offer. Also tests sequences of calls into the object.
- 27. 27 Black box testing + Black-box testing, a tester doesn’t have any information about the internal working of the software system. + Black box testing is a high level of testing that focuses on the behavior of the software. + It involves testing from an external or end-user perspective. Black box testing can be applied to virtually every level of software testing: unit, integration, system, and acceptance.
- 28. 28 White box testing + White box testing looks in detail at the code that makes up a function. Typical places to inspect are branching points where decisions are made... if statements, loops, multiway branches (switch statements). + Boundary conditions for variables can also be tested. For example, a variable to hold a month value should be checked for being outside of range. + Boundary conditions for types, such as integer overflow conditions, should also be checked.
- 29. 29 Testing for common faults + “Bugs lurk in corners and congregate at boundaries” + Boundary conditions - those areas where the behaviour of the system changes dramatically from one state to another - are those that have the greatest potential for error. + The behaviour of the if statement below changes when “a” has values of 9 and 10. A test case should ensure that both “a” is set to both values. + Here we don’t test the if, but the actions. Sometimes we test the logic of the if (especially when it is more complicated). if (a < 10) { x (); } else { y (); } Should be tested with ‘a’ set to 9, 10, 11
- 30. 30 Automating testing + Software needs to be tested every time any change is made. + Small changes can have unforeseen effects, so it is wise to check all of the software. + The modern method of software development encourages lots of small changes to ensure software is always kept runnable. + This results in lots of testing - the only solution is to automate the process. + Automated testing can help identify problems earlier.
- 31. 31 How useful is testing? + "The most powerful weapon in your toolbox is comprehensive unit testing" + It's this kind of thinking that causes too many of the problems you are trying to prevent. + According to “Programming Productivity” by Capers Jones, “Unit testing will only find 25% of the defects. At its best, unit testing finds 50% of the problems. Compare this to, for example, formal inspections, which find 60% of defects on average, 70% at best. Modelling and prototyping (build one to throw away) can find up to 80% of defects.” + None of these methods will build a completely defect-free program, but by combining a number of them, you can get extremely close (99%)
- 32. 32 Testing roles + Some companies separate the role of testing from that of developer because people are blind to the problems in their own code integration testing requires testing the interaction between developer’s code the job of testing is too large for one person The developer still does preliminary testing to ensure their code works to spec. The code is then released to the testers who run their own testing on it. + The human aspects of a separate testing team often results in trouble Testers are often the last in line in the product release cycle. Any delay from the developers means they work rush hours under extreme pressure
- 33. Testing Roles
- 34. 34 Testing roles + The following are likely on a testing project Tester Test Designer Test/QA Team Lead Test Build Manager Test Configuration Manager Test automater/automation developer Test administrator
- 35. 35 Fault tracking + If you want to correct faults you need to identify them (result of testing, or general use) record what their status is identify what versions of software they occur in + Fault tracking software help automate this Roundup TestTrackPro
- 37. 37 How much testing should you do? + Eventually testing has to come to an end. How do you decide when that is? + You need to establish release criteria... You need to define criteria that you can check before agreeing to release Define success for your software Software may contain faults, but it should provide value for your customers Determine what your company needs Ensure the criteria are, like requirements, "SMART" - Specific, Measurable, Attainable, Relevant, Trackable
- 38. 38 Example release criteria + All code must compile and build for all platforms + Zero high priority bugs + For all open bugs, documentation in release notes with workarounds + All planned QA tests run, at least ninety percent pass + Number of open defects decreasing for last three weeks. + Feature x unit tested by developers, system tested by QA, verified with customers A, B before release + Ready to release by June 1 Ref: 2
- 39. 39 Summary + Testing is never a substitute for developing software with a good process. + Testing is used to identify a program’s divergence from the requirements + Not all faults need to be fixed. + Automated testing simplifies the process of testing, and can help development.
- 41. UML and State Diagram 41
- 42. What is State Diagram + An object responds differently to the same event depending on what state it is in. State machine diagrams are usually applied to objects but can be applied to any element that has behavior to other entities such as: actors, use cases, methods, subsystems systems and etc. and they are typically used in conjunction with interaction diagrams (usually sequence diagrams). Sample footer text 3/1/20XX 42
- 43. State Diagram State Machine Diagrams are often used for deriving testing cases. 43 https://www.visual-paradigm.com/guide/uml-unified-modeling-language/about-state-diagrams/
- 44. How To Write Test Scenarios • As a tester, you can follow these five steps to create Test Scenarios- Step 1: Read the Requirement Documents like Business Requirements, System Requirements, Functional Requirement, of the System Under Test (SUT). You could also refer uses cases, books, manuals, etc. of the application to be tested. Step 2: For each requirement, figure out possible users actions and objectives. Determine the technical aspects of the requirement. Ascertain possible scenarios of system abuse and evaluate users with hacker’s mindset. Step 3: After reading the Requirements Document and doing your due Analysis, list out different test scenarios that verify each feature of the software. Step 4: Once you have listed all possible Test Scenarios, a Traceability Matrix is created to verify that each & every requirement has a corresponding Test Scenario Step 5: The scenarios created are reviewed by your supervisor. Later, they are also reviewed by other Stakeholders in the project. 44
- 46. Additional Read + https://www.guru99.com/back-box-vs-white-box-testing.html + https://www.softwaretestinghelp.com/software-testing-exercises-to- test-your-skills/ Sample footer text 3/1/20XX 46