Better Estimation Through Estimation Process Improvement - Dan Galorath
Download as PPTX, PDF2 likes1,168 views
The document discusses the importance of estimation in agile programs and aims to improve estimation processes to prevent project failures. It outlines various estimation techniques, the concept of estimation maturity, and identifies root causes of poor estimates in agile projects. Additionally, it emphasizes the need for organizations to assess their estimation capabilities and develop a structured estimation process to enhance project outcomes.
Better Estimation Through Estimation Process Improvement - Dan Galorath
- 1. Better Estimation Through Estimation Process Improvement Copyright 2015 Galorath Incorporated Getting to the Next Level in Cost Estimation
- 2. Key Points © 2015 Copyright Galorath Incorporated 2 Agile programs need estimation but many try to avoid it Improve estimates to mitigate a root cause of project failure Estimation Maturity Assessment leading into the Process Library that David is developing.
- 3. © 2015 Copyright Galorath Incorporated 3 ESTIMATION & PLANNING: An Estimate Defined • An estimate is the most knowledgeable statement you can make at a particular point in time regarding: • Effort / Cost • Schedule • Staffing • Risk • Reliability • Estimates more precise with progress • A WELL FORMED ESTIMATE IS A DISTRIBUTION 3
- 4. © 2015 Copyright Galorath Incorporated 4 The Planning Fallacy (Kahneman & Tversky, 1979, Updated 2003) • Judgment errors are systematic & predictable • Manifesting bias rather than confusion • Judgment errors made by experts and laypeople alike • Errors continue when estimators aware of their nature • Optimistic due to overconfidence ignoring uncertainty • Underestimate costs, schedule, risks • Overestimate benefits of the same actions • Root cause: Each new venture viewed as unique • “inside view” focuses on components rather than outcomes of similar completed actions • even ventures may appear entirely different Temper With Outside View E.G. SEER, data, etc.
- 5. © 2015 Copyright Galorath Incorporated 5 Reference Class Forecasting (adapted from http://www.slideshare.net/assocpm/a-masterclass-in-risk) • Best predictor of performance is actual performance of implemented comparable projects (Nobel Prize Economics 2002) • Provide an “outside view” focus on outcomes of analogous projects • Reference Class Forecasting attempts to force the outside view and eliminate optimism and misrepresentation • Choose relevant “reference class” completed analogous projects • Compute probability distribution • Compare range of new projects to completed projects
- 6. © 2015 Copyright Galorath Incorporated 6 Example: Parametric Estimate Compared With History
- 7. © 2015 Copyright Galorath Incorporated 7 ROI Analysis of A New System Cost of capital 8.0% Initial Investment Year 1 Year 2 Year 3 Year 4 Year 5 Year 6 Year 7 Total Ownership Investment $100,000 $100,000 Increase/(dec.) in revenue ($40,000) $60,000 $110,000 $100,000 $100,000 $150,000 $150,000 $630,000 Increase/(dec.) in op. exp. $90,000 $70,000 $70,000 $22,000 $24,000 $27,000 $28,000 $331,000 Cash Flow ($100,000) ($130,000) ($10,000) $40,000 $78,000 $76,000 $123,000 $122,000 $199,000 PV of Cash Flow ($100,000) ($120,370) ($8,573) $31,753 $57,332 $51,724 $77,511 $71,186 $60,563 NPV 60,563 $60,563 IRR 13.5% 13.5% ROI 121% 121.1% A Complete ROI analysis should analysis risk and uncertainty as well as likely
- 8. Key Points © 2015 Copyright Galorath Incorporated 8 Agile programs need estimation but many try to avoid it
- 9. © 2015 Copyright Galorath Incorporated 9 Disparaging Estimating Is Agile Sport • Conferences, blogs, tweets: open season for anyone suggests teams should estimate • And heaven help anyone who says that an estimate might be accurate (viable within a range) • Some who believe in estimation are hitting back • Even Scrum-Planning Poker-Burndown school of estimation decry estimation useful to management • Misses the real questions: • How should business plan cost & schedule • When is it useful to estimate? And when is it a waste of time and effort? • In what circumstances are estimates viable? And how can achieve those • How to deal with risk and uncertainty
- 10. © 2015 Copyright Galorath Incorporated 10 The Agile “Life Cycle” (Scrum Example) • Focus is on what features can be delivered per iteration • Not fully defined what functionality will be delivered at the end? • Iterations are often called a “Sprint” • Risk and growth not not explicitly addressed… increases in backlog
- 11. © 2015 Copyright Galorath Incorporated 11 Anatomy of a User Story • A User Story identifies: • The User • What the user wants to do with a feature • Value gained by the user • So a User Story can use the following format: As a (user role) I want (feature/achieve something) So that (describe value) • As a customer I want to track my order from the time I place it until it is delivered so that I know the status of my order at all times.
- 12. © 2015 Copyright Galorath Incorporated 12 User Stories And Other Requirements • User Stories are requirements, but sometime more information is needed • Since you want to keep User Stories light and negotiable, you may have to supplement them with information as the story progresses. • Other types of requirements include: • Business rules • Regulations • More detail • Consider referencing these requirements on the bottom of the User Story card
- 13. © 2015 Copyright Galorath Incorporated 13 Root Causes Of Bad Estimates In Agile Projects • Team not really doing Agile • Immature process • No one with previous experience, i.e.: no Scrum Master • No training in the process being used • Not considering risk • Management gets in the way • Micromanage the burn down chart • Want to use velocity as productivity • Bad Sizing • Pushback on using function points or traditional functional size • Trying to count function points of every Sprint • Bad Story Counting • Trying to use counts across teams • Using historical story point counts for new work
- 14. © 2015 Copyright Galorath Incorporated 14 Root Causes Of Bad Estimates In Agile Projects - Continued • Bad User Stories • Ambiguous stories • Stories with unchallenged assumptions • Untestable stores • Teams combine iterations • Work exceeds the sprint so the sprint is exceeded • Team locks into a repeatable time-box that is not practical • Lack of good definition for “Done” • No clear definition of when to move on or to keep going • Team afraid to stop a sprint when “done” (literally “boxed”) • Product Owner or Customer non-responsive • No way to remove ambiguity • No way to push back against priorities
- 15. © 2015 Copyright Galorath Incorporated 15 Agile Story Points Defined • Measure relative effort required to implement the story • Points are assigned to User Stories • Those point values are later used to estimate • Represents story size based on how hard a story is to implement • Many would say that a story point is an arbitrary measurement that depends on the team. • There is some truth to that statement • We can manage that to some extent
- 16. Story Point Relative Scale • But what am I rating… Size, Weight, Temperament? • How likely is my wife to allow me to have one 5 1 1 3 10 8 7 © 2015 Copyright Galorath Incorporated 16
- 17. Letting Story Points Determine Duration/Iterations • What does the customer want…. A Schedule! • But to provide one you need to know the teams Story Point “velocity” • If the schedule is too long then some functionality could be removed – or other adjustments made © 2015 Copyright Galorath Incorporated 17
- 18. © 2015 Copyright Galorath Incorporated 18 Effective Productivity Needs A Model Average Productivity (or range) Size (Story Points,, NESMA, FP, Etc.) Team size Constraints Complexity Technology Effective / Expected Productivity Just using an average productivity is inadequate for estimating. Effective productivity generally needs a model.
- 19. © 2015 Copyright Galorath Incorporated 19 Array of SEER Tools Support Agile • Agile Knowledge Bases • Specify by teams experience • Story Point Sizing Template • Manage team experience and different team velocities • Activity and Labor Naming Schemas • Custom life cycle activity naming • Describe the roles of team members • User Story Management • Load stories into database for easy Sprint alignment
- 20. © 2015 Copyright Galorath Incorporated 20 System Level Agile Estimates Are Quick 1. Choose Domain & Agile Knowledge Bases 2. Enter or use analogy Sizing (Example = “T shirt” user stories) 3. Enter expected staff per spring 4. If total ownership costs desired enter years of maintenance
- 21. © 2015 Copyright Galorath Incorporated 21 Not Limited To User Stories • User Stories are the predominant Agile method used to capture desired product features • Can just as easily use Function Points, or Use Case Points, or any other volume collection approach • Team Velocity per Sprint is the key • As the intent of Agile is to “discover” functionality – for the estimate - need to know how fast it can be delivered – but you choose the sizing approach • Story Points per sprint (Some user stories may be bigger than a sprint) • Function Points per sprint • Use Case Points per sprint • Requirements per sprint • Features per sprint (feature must fit in 1 sprint) • Even Lines of Code per sprint
- 22. © 2015 Copyright Galorath Incorporated 22 SEER Includes A Story Point Size (Via Proxy) • Included to provide a simple example • Since story points have no consistent scale across programs • Proxy needs to be tuned to the team story point scale and the velocity • SEER Proxy Builder creates relationships that are used transparently in the model
- 23. Key Points © 2015 Copyright Galorath Incorporated 23 Estimation Maturity Assessment leading into the Process Library that David is developing.
- 24. Where Is Your Organization On the Galorath Estimation Maturity Scale? Estimation Bias Mitigation Begins at Level 2, Solid at Level 3
- 25. The Desire for Cost Estimation Process Maturity is Growing • Over 90% of our commercial clients identify their organization at a maturity Level 0 to Level 1 • Many have performed CMMI or SEI assessments • Less than 1% have performed an internal assessment of their “Estimation Maturity” • Most describe rigorous estimating or parametric tools as being above their estimation capabilities or “Too sophisticated” • Those that want to improve are not certain of how to proceed
- 26. © 2015 Copyright Galorath Incorporated 26 Questions About Your Current Process • Do you typically develop a ROM first? • Back of the envelope • Compare new effort to previous similar work • Assemble ad-hoc team to generate estimate • Start with a formal process • How much of your estimates are for new work? • Never done the work before • Usually bid only what is similar to what we know • Nearly all of it is the same code – just different use • What is the life cycle of the estimate • Becomes the foundation for the formal schedule • Revisit estimate every milestone • Throw it away
- 27. © 2015 Copyright Galorath Incorporated 27 Understand Current Practices • What estimating process currently performed? • Tool driven • Home grown spreadsheets • Wet finger • What best describes your development practices? • Traditional Waterfall • RUP/Spiral/Incremental • Seat of the pants • How do you manage projects/proposal data? • Spreadsheets • MS Project • Other Tools?
- 28. © 2015 Copyright Galorath Incorporated 28 How Do You Measure Risk? • What probability of success is used? • 50%, 80%, 100% confidence? • Is your planned productivity feasible? • Do you measure productivity? Across teams? • Do you evaluate risks associated with: • Personnel Capabilities/Availability • Requirements Stability/Complexity • Code Growth and Product Reusability • Full System Integration Testing/User Acceptance • Anticipated Defects at Delivery • Maintenance or Ongoing Support Cost • What is your Technical Debt? • How many defects are percolating in the code?
- 29. © 2015 Copyright Galorath Incorporated 29 Playing the “What If” Game • With a new estimate do you go back and say • Can we do this X months faster? • Can we do this for X dollars less? • How can we improve quality? • During a project do you ask: • When behind – what can we change to catch up? • Are my captured defects matching the projected rate? • What can I do to pull in my estimated completion date? • How confident am I to finish on time/in budget? • What productivity am I getting out of my staff?
- 30. © 2015 Copyright Galorath Incorporated 30 Moving to the Next Level • All of the previous questions are easy to answer when a mature estimation process is in place • To increase in maturity requires a few simple questions be answered: • Where are we now? • What level is sufficient for our corporate strategy? • What is the plan going forward? • Are we politically motivated to implement the plan? • Deploy an Estimation Process Improvement Program will accelerate process development • Developing an Estimation Center of Excellence accelerates and enforces good practices • Centralized learning, data collection, process control
- 32. Assessment Process Defined (Plan is refined a each level achieved)
- 33. Services Tied to Maturity Levels
- 34. Products Tied to Estimation Maturity
- 35. Key Points © 2014 Copyright Galorath Incorporated 35 Agile programs need estimation but many try to avoid it Improve estimates to mitigate a root cause of project failure Estimation Maturity Assessment leading into the Process Library that David is developing.
- 36. © 2015 Copyright Galorath Incorporated 36 Galorath Contact: www.galorath.com Dan Galorath, +1310-414-3222 ex 614; galorath@galorath.com Steve Robinson +44(0)2077 889 042; srobinson@galorath.com