Niels Malotaux - Help We Have a QA Problem!
Download as PPT, PDF0 likes638 views
The document addresses a QA problem faced by a testing team due to an overwhelming amount of modules to test and insufficient personnel, leading to a lengthy regression testing process. It emphasizes the importance of implementing the PDCA cycle to improve the testing process, reduce delays, and enhance collaboration between developers and testers. Ultimately, the team successfully resolved their issues through empowerment and adapting their approach, resulting in increased satisfaction among customers and more efficient project management.
Niels Malotaux - Help We Have a QA Problem!
- 1. 1QA Problem 2009 Niels Malotaux +31-30-228 88 68 niels@malotaux.nl www.malotaux.nl Help ! We have aHelp ! We have a QA Problem !QA Problem !
- 2. 2QA Problem 2009 We have a QA problem ! • Large stockpile of modules to test (hardware, firmware, software) • You shall do Full Regression Tests • Full Regression Tests take about 15 days each • Too few testers (“Should we hire more testers ?”) • Senior Tester paralyzed • Can we do something about this?
- 3. 3QA Problem 2009 The essential ingredient: the PDCA Cycle (Shewhart Cycle - Deming Cycle - Plan-Do-Study-Act Cycle - Kaizen) !
- 4. 4QA Problem 2009 Instead of complaining about a problem … (Stuck in the Check-phase) Let’s do something about it ! (Moving to the Act-phase)
- 5. 5QA Problem 2009 Objectifying and quantifying the problem is a first step to the solution Estim 17 8 14 11 9 17 4 1 1 1 3 1.1 3 0.1 18 106 Line Activity 1 Package 1 2 Package 2 3 Package 3 4 Package 4 (wait for feedback) 5 Package 5 6 Package 6 7 Package 7 8 Package 8.1 9 Package 8.2 10 Package 8.3 11 Package 8.4 12 Package 8.5 13 Package 8.6 14 Package 8.7 15 Package 8.8 totals Customer Will be done (now=22Feb) HT Chrt BMC McC? Ast ? Cli Sev ? Chrt 24 Feb Chrt Yet 28 Feb Yet 24 Mar Cli After 8.5 OK Ast Alter native Junior tester Devel opers 2 17 4 5 10 7 5 4 3 5 3 10 10 1 3 1 1 1 1 1.1 3 0.1 18 47 32 36
- 6. 6QA Problem 2009 TimeLine Selecting the priority order of customers to be served • “We’ll have a solution at that date … Will you be ready for it ?” Another customer could be more eagerly waiting • Most promising customers wk 9 10 11 12 13 14 15 16 17 13 delivery cust a delivery cust b,c delivery cust a,d start (all done)
- 7. 7QA Problem 2009 Result • Tester empowered • Done in 9 weeks • So called “Full Regression Testing” was redesigned • Customers systematically happy and amazed • Kept up with development ever since • Increased revenue Recently: • Tester promoted to product manager • Still coaching successors how to plan
- 8. 8QA Problem 2009 • Evo (short for Evolutionary...) uses PDCA consistently • Applying the PDCA-cycle actively, deliberately, rapidly and frequently, for Product, Project and Process, based on ROI and highest value • Combining Planning, Requirements- and Risk-Management into Result Management • We know we are not perfect, but the customer shouldn’t be affected • Evo is about delivering Real Stuff to Real Stakeholders doing Real Things “Nothing beats the Real Thing” • Projects seriously applying Evo, routinely conclude successfully on time, or earlier Evo
- 9. 9QA Problem 2009 Who is the customer of Testing and QA ? • Deming: • Quality comes not from testing, but from improvement of the development process. Testing does not improve quality, nor guarantee quality. It’s too late. The quality, good or bad, is already in the product. You cannot test quality into a product. • Developers are the customer • Testers help developers to become perfect • Testing is a project to run alongside and synchronized to the development project • Therefore, it must be organised like any other project
- 10. 10QA Problem 2009 Universal Project Goal • Providing the customer with • what he needs • at the time he needs it • to be satisfied • to be more successful than he was without it • Constrained by (win - win) • what the customer can afford • what we mutually beneficially and satisfactorily can deliver • in a reasonable period of time
- 11. 11QA Problem 2009 Evo Project Planning Evolutionary Project Management (Evo) • Plan-Do-Check-Act • The powerful ingredient for success • Business Case • Why we are going to improve what • Requirements Engineering • What we are going to improve and what not • How much we will improve: quantification • Architecture and Design • Selecting the optimum compromise for the conflicting requirements • Early Review & Inspection • Measuring quality while doing, learning to prevent doing the wrong things • Weekly TaskCycle • Short term planning • Optimizing estimation • Promising what we can achieve • Living up to our promises • Bi-weekly DeliveryCycle • Optimizing the requirements and checking the assumptions • Soliciting feedback by delivering Real Results to eagerly waiting Stakeholders • TimeLine • Getting and keeping control of Time: Predicting the future • Feeding program/portfolio/resource management Zero Defects Attitude
- 12. 12QA Problem 2009 The aim of Testing • Being done as soon as the development is done • Well, almost • Excuses, excuses, excuses • The developers are always late (Evo developers live up to their promises) • The developers don’t take us seriously (Evo developers ask testers for help) • The developers don’t inject enough defects (now testing becomes a challenge) • Helping development to be successful
- 13. 13QA Problem 2009 TimeLine • Cutting the work into chunks • Estimating • Adding up (averages the uncertainties !) • Usually doesn’t fit in the available time • Find strategies to solve the dilemma • Select ‘optimum’ strategy • Predict what will happen when • Learn and repeat every week, keeping predictions up-to-date • Allow Portfolio Management to manage rather than to game
- 14. 14QA Problem 2009 Predicting what will be done when Calibr factor 1.0 1.0 1.0 1.4 1.4 1.4 1.4 1.4 1.4 Calibr still to 1 2 1 4.2 1.4 4.2 5.6 7.0 9.8 Ratio real/es 1.0 1.2 3.0 3.5 1.0 Spent Still to spend 2 0 5 1 3 0 3 2 4 1 Estim 2 5 1 2 5 3 1 3 4 5 7 Line Activity 1 Activity 1 2 Activity 2 3 Activity 3 4 Activity 4 5 Activity 5 6 Activity 6 7 Activity 7 8 Activity 8 ↓ ↓ 16 Activity 16 17 Activity 17 18 Activity 18 Date done 30 Mar 2009 1 Apr 2009 2 Apr 2009 9 Apr 2009 10 Apr 2009 16 Apr 2009 2 Jun 2009 11 Jun 2009 25 Jun 2009
- 15. 15QA Problem 2009 What do we do if we see we won’t make it on time ? If it doesn’t fit ... count backwards Value Still to EarnEarned Value
- 16. 16QA Problem 2009 Deceptive options • Hoping for the best (fatalistic) • Going for it (macho) • Working Overtime (fooling ourselves) • Moving the deadline • Parkinson’s Law • Work expands to fill the time for its completion • Student Syndrome • Starting as late as possible, only when the pressure of the FatalDate is really felt
- 17. 17QA Problem 2009 The Myth of the Man-Month 1 2 3 4 5 6 87 9 10 11 12 13 14 15 16 1 2 3 4 5 6 8 7 9 10 11 12 13 14 intuition people x time = constant Man-Month Myth reality (Putnam) project duration number of people lower cost shorter time nine mothers area Economic optimum? Brooks’ Law (1975) Adding people to a late project makes it later
- 18. 18QA Problem 2009 Saving time We don’t have enough time, but we can save time without negatively affecting the Result ! • Efficiency in what (why, for whom) we do - doing the right things • Not doing what later proves to be superfluous • Efficiency in how we do it - doing things differently • The product • Using proper and most efficient solution, instead of the solution we always used • The project • Doing the same in less time, instead of immediately doing it the way we always did • Continuous improvement and prevention processes • Constantly learning doing things better and overcoming bad tendencies • Efficiency in when we do it - right time, in the right order • TimeBoxing - much more efficient than FeatureBoxing
- 19. 19QA Problem 2009 More• www.malotaux.nl/Booklets 1 Evolutionary Project Management Methods (2001) Issues to solve, and first experience with the Evo Planning approach 2 How Quality is Assured by Evolutionary Methods (2004) After a lot more experience: rather mature Evo Planning process 3 Optimizing the Contribution of Testing to Project Success (2005) How Testing fits in 3a Optimizing Quality Assurance for Better Results (2005) Same as Booklet 3, but for non-software projects 4 Controlling Project Risk by Design (2006) How the Evo approach solves Risk by Design (by process) 5 TimeLine: How to Get and Keep Control over Longer Periods of Time (2007) Replaced by Booklet 7 6 Human Behavior in Projects (2008) Human Bahavioural aspects of Projects 7 How to Achieve the Most Important Requirement (2008) Planning of longer periods of time, what to do if you don’t have enough time 8 Help ! We have a QA Problem ! (2009) The story of this presentation • www.malotaux.nl/nrm/Insp Inspection pages
- 20. 20QA Problem 2009 Niels Malotaux Help ! We have aHelp ! We have a QA Problem !QA Problem !
Editor's Notes
- #9: What is Evo Short for Evolutionary Development/Delivery/Project ManagementEvo is a label we use for successful methods to deliver Quality On Time. Until now all the successful methods have an Evolutionary aspect in them. So we use the Evolutionary label. In short: Evo. Deliberately going through the PDCA cycle rapidly and frequently, for product, project and processPlan - Do - Check - Act cycle, also called the Shewhart cycle or Deming cycle. Do is what we normally do. Most of us Plan, more, or less. Usually we “have no time” for the Check and Act parts. We use this cycle on everything: the Product (what is really needed and possible within the budget), the Project (how to learn to do things better) and even the Process: what doesn’t work is discarded: no burocracy. Continuously thinking what to do, in which order, to which level of detail for nowWhat we have done until this very moment cannot be changed any more. What we have, we have. What we haven’t, we haven’t. What we thought last week what we should do does not matter. Based on what we know NOW: What is the best to do NOW, in which order (priority!) to which level of detail for now, because if we do more detail than is necessary NOW, we will have wasted time if we later find out that we should have done something different. Methods for efficiently and effectively running development projects, delivering Quality On TimeEvo projects deliver routinely Quality On Time. Delivering what the user needs at the time he needs itThat is what pays our salary