Why do we test software?
Download as PPTX, PDF0 likes268 views
The document discusses why software testing is important. It notes that software today controls many safety critical systems and embedded devices. Software failures can have catastrophic consequences, costing lives and billions of dollars in losses. Testing aims to find faults early in the development process and prevent failures by verifying software meets requirements. The costs of inadequate testing are high, so testing is necessary to improve quality, reduce costs from bugs, and ensure customer satisfaction.
![6
public static int numZero (int [ ] arr)
{ // Effects: If arr is null throw NullPointerException
// else return the number of occurrences of 0 in arr
int count = 0;
for (int i = 1; i < arr.length; i++)
{
if (arr [ i ] == 0)
{
count++;
}
}
return count;
}
Fault: Should start
searching at 0, not 1
Test 1
[ 2, 7, 0 ]
Expected: 1
Actual: 1
Test 2
[ 0, 2, 7 ]
Expected: 1
Actual: 0
Error: i is 1, not 0, on
the first iteration
Failure: none
Error: i is 1, not 0
Error propagates to the variable count
Failure: count is 0 at the return statement](https://image.slidesharecdn.com/whytest-210310135714/85/Why-do-we-test-software-6-320.jpg)
Why do we test software?
- 1. Why Do We Test Software? MD. SHAFIUZZAMAN ASSISTANT PROFESSOR DEPT. OF CSE, JUST
- 2. Software defines behavior network routers, finance, switching networks, other infrastructure Today’s software market: is much bigger is more competitive has more users Embedded Control Applications airplanes, air traffic control spaceships watches ovens remote controllers Agile processes put increased pressure on testers Programmers must unit test – with no training or education! Tests are key to functional requirements – but who builds those tests ? 2 – PDAs – memory seats – DVD players – garage door openers – cell phones
- 3. 3
- 4. Software Fault : A static defect in the software. Faults in software are equivalent to design mistakes in hardware. Software Failure : External, incorrect behavior with respect to the requirements or other description of the expected behavior Software Error : An incorrect internal state that is the manifestation of some fault 4
- 5. A patient gives a doctor a list of symptoms Failures The doctor tries to diagnose the root cause Fault The doctor may look for anomalous internal conditions (high blood pressure, irregular heartbeat, bacteria in the blood stream) Errors 5
- 6. 6 public static int numZero (int [ ] arr) { // Effects: If arr is null throw NullPointerException // else return the number of occurrences of 0 in arr int count = 0; for (int i = 1; i < arr.length; i++) { if (arr [ i ] == 0) { count++; } } return count; } Fault: Should start searching at 0, not 1 Test 1 [ 2, 7, 0 ] Expected: 1 Actual: 1 Test 2 [ 0, 2, 7 ] Expected: 1 Actual: 0 Error: i is 1, not 0, on the first iteration Failure: none Error: i is 1, not 0 Error propagates to the variable count Failure: count is 0 at the return statement
- 7. The Term Bug Bug is used informally Sometimes speakers mean fault, sometimes error, sometimes failure … often the speaker doesn’t know what it means ! 7 BUG
- 8. Spectacular Software Failures 8 Intel’s Pentium FDIV fault : Public relations nightmare THERAC-25 radiation machine : Poor testing of safety-critical software can cost lives : 3 patients were killed Mars Polar Lander crash site? THERAC-25 design Ariane 5: exception-handling bug : forced self destruct on maiden flight (64-bit to 16-bit conversion: about 370 million $ lost) NASA’s Mars lander: September 1999, crashed due to a units integration fault Ariane 5 explosion : Millions of $$
- 9. 9 • Healthcare website : Crashed repeatedly on launch—never load tested • Boeing A220 : Engines failed after software update allowed excessive vibrations • Toyota brakes : Dozens dead, thousands of crashes • Northeast blackout : 50 million people, $6 billion USD lost … alarm system failed • Boeing 737 Max : Crashed due to overly aggressive software flight overrides (MCAS)
- 10. Northeast Blackout of 2003 10 Affected 10 million people in Ontario, Canada Affected 40 million people in 8 US states Financial losses of $6 Billion USD 508 generating units and 256 power plants shut down The alarm system in the energy management system failed due to a software error and operators were not informed of the power overload in the system
- 11. Costly Software Failures 11 NIST report, “The Economic Impacts of Inadequate Infrastructure for Software Testing” (2002) – Inadequate software testing costs the US alone between $22 and $59 billion annually – Better approaches could cut this amount in half Huge losses due to web application failures – Financial services : $6.5 million per hour (just in USA!) – Credit card sales applications : $2.4 million per hour (in USA) In Dec 2006, amazon.com’s BOGO offer turned into a double discount 2007 : Symantec says that most security vulnerabilities are due to faulty software
- 12. Testing in the 21st Century More safety critical, real-time software Embedded software is ubiquitous … check your pockets Enterprise applications means bigger programs, more users Paradoxically, free software increases our expectations ! Security is now all about software faults Secure software is reliable software The web offers a new deployment platform Very competitive and very available to more users Web apps are distributed Web apps must be highly reliable 12
- 13. Validation & Verification Validation : The process of evaluating software at the end of software development to ensure compliance with intended usage Verification : The process of determining whether the products of a given phase of the software development process fulfill the requirements established during the previous phase IV&V stands for “independent verification and validation” 13
- 14. Cost of Late Testing 14 60 50 40 30 20 10 0 Fault origin (%) Fault detection (%) Unit cost (X) Software Engineering Institute; Carnegie Mellon University; Handbook CMU/SEI-96-HB-002 Assume $1000 unit cost, per fault, 100 faults
- 15. Summary: Why Do We Test Software ? 15 A tester’s goal is to eliminate faults as early as possible • Improve quality • Reduce cost • Preserve customer satisfaction
Editor's Notes
- #15: 28-Oct-2010, at GTAC, added the animation to demonstrate increasing the number of faults found early, thereby decreasing the number of faults found late, and finally saving money. Lots of it! This animation is fairly complicated … must practice first!!