Future of Software Analysis & Measurement_CAST
2 likes1,195 views
The document discusses the future of software analysis and measurement (SAM), emphasizing its role in managing software projects effectively by addressing cost, effort, and risk estimation. It highlights key contributions from industry experts and the importance of combining measurement with management practices to improve outcomes and reduce common project failures. By utilizing methodologies like CMMI and agile techniques, organizations can enhance software performance, governance, and ultimately drive business value.
Future of Software Analysis & Measurement_CAST
- 1. The Future of Software Analysis & Measurement
- 2. The Future of Software Analysis & Measurement
- 3. Daniel D. Galorath 40+ years Software & Software Management Experience Founder & CEO Galorath Incorporated, SEER By Galorath 1980 MBA Management, California State Universities 1984 Began working software estimation, planning & control 2009 Society of Cost Estimation and Analysis (SCEA) Lifetime Achievement award 2001 ISPA Freiman Award, lifetime achievement award 2006 Book: Software Sizing, Estimation, and Risk Management
- 4. SEER® Empowering Project Estimation, Planning & Control Since 1988 SEER parametrically estimates project cost, effort, duration, cost & risk Answer Fundamental Management Questions How much effort is required to complete an activity? How much calendar time is needed to complete an activity? What is the total cost of an activity? How reliable will it be? What can we do better? What If? www.galorath.com
- 5. David Herron Business Development Manager and VP of Knowledge Solution Services for David Consulting Group Consulting and coaching services for a variety of IT organizations throughout the US and Canada Acknowledged authority in the areas of performance measurement, process improvement and organizational change management Noted author and lecturer; co-authored several books on topics relating to IT performance measurement and Function Point Analysis
- 6. David Consulting Group DCG is an international IT process improvement and measurement company managing value-driven engagements with companies and government agencies around the world Software Process Improvement Utilizing CMMI, Six Sigma, Lean and Agile methods Software Sizing Using IFPUG Function point Counting and alternative sizing techniques Software Measurement Providing roadmap planning, estimation models, performance benchmarks and outsourcing SLA support IT Performance Improvement Improving IT operations through ITIL and IT Governance
- 7. Dr. Bill Curtis Senior Vice President and Chief Scientist Industry luminary responsible for influencing CAST’s scientific and strategic direction Best known for leading development of the Capability Maturity Model (CMM) which has become the global standard for evaluating the capability of software development organizations. Prior to joining CAST, Co-Founder of TeraQuest, the global leader in CMM- based services, Directed the Software Process Program at the Software Engineering Institute (SEI) at Carnegie Mellon University.
- 8. CAST Application Intelligence Platform Most enterprises measure everything but the product delivered to the business CAST Application Intelligence Platform (AIP) measures the product itself Process Product Robustness Performance Time & Quality Security CAST AIP Planning Duration &Size Changeability Estimation Transferability Scheduling Size Time Tracking Cost Tracking Requirements Function & Effort & Budget Scope Earned Value User Acceptance Usability
- 9. QUESTIONS What are the single most important business drivers that lead an organization to embark on a SAM initiative? How does software analysis and measurement relate to business outcomes? What organizational behaviors are most favorable for a successful SAM program? What is the relationship between SAM and IT governance? How can measurement and analysis interact with estimating to provide improved visibility and information to make the best management decisions regarding software? How can anyone provide a viable estimate software development, software maintenance, or continuing innovation work and what use it is? How can we determine which software projects with have the highest ROI?
- 10. What are the single most important business drivers that lead an organization to embark on a SAM initiative?
- 11. Human Nature Optimism in Software HBR Article explains this Phenomenon: Humans seem hardwired to be optimists We routinely exaggerate benefits and discount costs Delusions of Success: How Optimism Undermines Executives' Decisions (Source: HBR Articles | Dan Lovallo, Daniel Kahneman | Jul 01, 2003) Solution: Temper with “outside view” Software Analysis & Measurement can temper Viable estimation can temper Don’t remove optimism, but balance optimism & realism 11
- 12. IT Failures are Pervasive: And Even Successful Projects may not be Cutter Consortium Software Project Survey: 62% overran original schedule by more than 50% 64% more than 50% over budget 70% had critical product quality defects after release Standish Group CHAOS Report 46% challenged 19% failed 35% successful ~$875 billion spent on IT ~$300 billion spent on IT projects ~$57 billion wasted annually Ever increasing “Technical Debt” ROI of Applied Software Analysis & Measurement is HUGE
- 13. Manual Estimates Human reasons for error (Software Analysis & Measurement can help) Desire for “credibility” motivates overestimate behavior (80% probability?) So must spend all the time to be “reliable” Better approach force 50% probability & have “buffer” for overruns Technical pride causes underestimates Buy-in causes underestimates 13
- 14. Important Business Drivers for Structural Measurement Poor software 1 from suppliers Embarrassing 2 disasters High cost of 3 maintenance
- 15. How does software analysis and measurement relate to business outcomes?
- 16. Goal Question Metric Approach Goal Organizational Goal Question Development Contractors Organizations Metric Combine goal-orientation bottoms up, decision-support & other operational management techniques Going to weather.com and deciding to bring an umbrella is decision support
- 17. How Does Measurement Relate to Business Outcomes Business Tactical Objectives Application Health Value Maximize standards compliance Factors Improve software readability TRANSFERABILITY Minimize business Reduce vendor lock-in allows new teams to quickly begin working with an application risks Reduce cost of ownership Cost Reduce injected defects Reduce application mistakes CHANGEABILITY makes an application easier and Minimize IT costs Reduce application rework quicker to modify Ease team handoffs Minimize liquidated damages Maximize business Maximize application availability ROBUSTNESS improves application stability & agility Minimize degraded service reduces injecting new defects Reduce learning curves Risk Reduce modification effort Optimize work Accelerate new function delivery PERFORMANCE productivity Reduces degraded response Maximize application scalability times and increases scalability Maximize speed of response Maximize Maximize information retrieval SECURITY customer loyalty Minimize unwanted breaches affects an application’s ability to Maximize customer confidence prevent unauthorized intrusions Maximize information protection
- 18. What organizational behaviors are most favorable for a successful SAM program?
- 19. Key Organizational Behaviors Trust but verify Viable measurement and analysis Answering the right questions Having actions based on measurement & analysis Measure what can make a difference Measure to the audience needs (Goal / Question / Metrics)
- 20. Generating the Business Value Side of the Equation (Benefits) The business owns benefit calculations IT should participate Exception: projects solely improving internal IT Beware of subjectivity translating soft benefits Use probability and risk Increased Revenue Intangible Benefits Increased Profit Reduction in cost Internal benefits (people, processes, cash out)
- 21. Conditions that Favor Structural Measurement Low Fed up maturity managers Disciplined Mature processes High Quantitative maturity management
- 22. What is the relationship between SAM and IT governance?
- 23. The Gap Between IT & Management Needs Financial Business/ IT Organization Customer Vision & Business Process Governance Level Strategy Innovation / Growth Customer Focus CoBIT… 6σ Goals Goal Attainment Questions Answers IT GQM Metrics Measurement Governance Data Collection PSM Software Development & Maintenance (Source: Fraunhofer)
- 24. How Structural Measurement is Tied to IT Governance The Evolution of Governance Govern the Govern the Govern the Department Project Product Budget Schedule Structural headcount defects quality
- 25. How can measurement and analysis interact with estimating to provide improved visibility and information to make the best management decisions regarding software?
- 26. What a Parametric Model can Tell You Firm Fixed Price? Feel lucky? What is likely to happen Understand the risk before you commit! 26
- 27. SAM Feeds Estimation and Provides Insight & Choice Size Effort S A SEER-SEM M Software O Estimation Schedule u Complexity Model t p u Risk t Maintainability/ s Defect Potential Reliability /Defects People, Process, Technology Maintenance Effort Required Constraints Cost, schedule, risk Should we update or Historical redevelop? Results (SAM Outputs) What is the risk of continuing with software with increasing “technical debt”?
- 28. Looking at Maintenance and Technical Debt
- 29. Software Estimation Basic Model & Associated Metrics Effective Technology Cte Development Technology Legacy, Maintenance Maintenance/ Specifics & Block Change Constraints Development Effort K People Process and/or Process Block Changes As Redevelopment Staffa Defects Effective complexity & Count (Qi Qr) D Constraints Software Size Size St Stakeholder Delivered ReuseDIT Requirements Development (Effective Se Software & Total St) Size Se Process (work units) Start Finish Calendar Time On-going Iterations of Effort (ACWP or Spent) Progress (BCWP or Earned Value) Defects (Qi Qr ) Growth (Sg)
- 30. Fundamental Metrics Size Effort AKA Volume, Mass AKA Work, Labor Units: Source Lines of Code Units: Staff Months, Staff Hours (SLOC); Function Points (FP) Use Cases Cost New versus rework AKA Budget, Money COTS & Packages Units: $, other currencies Effective Technology Staffing AKA Productivity Potential, AKA Manpower Loading Efficiency Units: FTE People Units: none Defects Time AKA Reliability, Quality AKA Duration, Schedule Units: Defect Count Units: Calendar Months, Calendar Weeks
- 31. Other Key Metrics Help Track Project Performance Track defect discovery and removal rates against expected rates Heath and Status Indicator shows status and trends from the previous snapshot Thresholds are user definable Increased defect reporting rate shows a worsening trend
- 32. When & Why to Collect Data When to Collect Why People Don’t Want to 1. Up-front: when scoping new Provide Data project data from completed They could be proven wrong projects It could be used against them 2. In-Process: During development for management, Data often doesn’t exist to identify issues and progress Even if processes dictate data 3. Post Mortem: Upon requirements development completion to If it exists it may not be clean improve corporate history It may give away corporate repository productivity & bid strategy 4. In Service: During maintenance to continue learning & improving
- 33. How can anyone provide a viable estimate software development, software maintenance, or continuing innovation work and what use it is?
- 34. 10 Step System Estimation Process 2011 1. Establish Estimate Scope 10. Track Project Throughout Development 2. Establish Technical 9. Document Estimates Baseline, Ground and Lessons Rules, Assumptions Learned 8. Generate a Project Plan 4. Refine Technical Baseline Into Estimable Components 6. Validate Business Case Costs & Benefits (go / no go) 4. Collect data / estimation inputs 6. Quantify Risks and Risk Analysis 5. Estimate Baseline Cost, Schedule, Affordability Value
- 35. Estimation Organizational Maturity V1.7 Manual effort Level 0 Informal or no estimating estimating without a process Level 1 Direct Task Estimation Spreadsheets Ad Hoc Process Simple model Formal (Size * Productivity) or Some Level 2 Sizing (e.g. function Direct Task Estimation informal SEER Use measurement & analysis Informal Process points) Robust Formalized Rigorous Parametric Level 3 Formal Sizing Parametric estimation (SEER) Estimate vs. actual capture Multiple Estimate Process measurement & analysis planning & Control Risk Management Repeatable process Robust Parametric Process Rigorous Level 4 Formal sizing Repeatable process parametric estimating (SEER) measurement & analysis estimation with tracking & control Risk Management improvement via lessons learned Robust Parametric Rigorous Continuous Level 5 Formal sizing Repeatable process parametric estimating (SEER) measurement & analysis estimation with tracking & control Risk Management process improvement Why should we care? Maturity is related to estimate viability… With better estimation process, projects more likely to be successful in execution
- 36. ESTIMATION & PLANNING: An Estimate Defined An estimate is the most knowledgeable statement you can make at a particular point in time regarding: Effort / Cost Schedule Staffing Risk Reliability Estimates more precise with progress A WELL FORMED ESTIMATE IS A DISTRIBUTION 36
- 37. Avoid Surprises with Estimation Process & Tools Challenged projects Would you still go forward if you knew Schedule would be significantly longer? Cost would be dramatically higher? Probably: but perhaps more insight could identify mitigation Plan functionality differently Certainly you could notify stakeholders of real costs Ensure staffing is appropriate for the constraints Failed Projects Would you start a project you knew was unaffordable? Or if schedule was completely unrealistic? If knowing up-front could you do something about it? often better to kill project before it begins than waste resources & let the organization down
- 38. How can we determine which software projects with have the highest ROI?
- 39. Software Projects Must Return Business Value “Economics is primarily a science of choice… software economics should provide methods for analyzing the choices software projects must make.” Leon Levy “Base choices on those providing the maximum business value to the organization” Eli Goldratt Measurement and its uses such as estimating and defect analysis help this science of choice Some say business value is not our problem While others generally need to perform benefit analysis We need to build systems that optimize the business Make IT part of the solution If IT & measurement don’t generate sufficient profit, money will go elsewhere
- 40. Software & IT Systems are about Business Value Cost Value
- 41. An ROI Analysis of an Upgrade: Software Analysis & Measurement Provides Valve Cost Can we do better? Will stakeholders tolerate a loss for 3 years? What is the risk?
- 42. Measurement Manifesto for Software & IT Measurement Perspective Management & Stakeholder Perspective We measure to ultimately Speak so I can understand produce business value to the Give me actionable items organization Don’t just give me problems… MEASURED Measurement Give me solutions should not ultimately be a cost Help me make the best decisions The analysis of measurements so we can produce business produces decisions that value produce business value Measurement itself is not the answer…. Management decision making, better performance, quality improvements and better serving stakeholders is
- 43. L a nmo ea o t A T er r bu C S w w c ss f aec m w .a tot r.o w bo .a tot aec m lgc ss f r.o w w w fc b o .o c so q a t w . e o kc m/a tn u ly a i w w sd s aen t a tot ae w . ie h r.e/ ss f r l c w w w t ie.o O Q a t w . t r m/ n u ly wt c i