Software Reliability CMM-DFSS
Download as PPT, PDF3 likes1,492 views
This document discusses the deployment of Design for Six Sigma (DFSS) in software at Philips Consumer Lifestyle, highlighting two approaches: one from Bruges related to Capability Maturity Model (CMM) Level 4, and the other from Bangalore employing traditional DFSS practices. It outlines lessons learned, the significance of Critical to Quality (CTQ) metrics, and the importance of integrating non-functional requirements and customer needs throughout the software development lifecycle. The document concludes with the challenges faced in software quality management and the need for better requirement specifications from the marketing team.
Software Reliability CMM-DFSS
- 1. Deployment of DFSS in software – experience within Philips Consumer Lifestyle Guy Van Hooveld CTO Office
- 2. Objectives of this presentation Give insight in two different approaches to apply DFSS in a software environment with Philips Consumer Lifestyle Bruges TV experience – link between DFSS and CMM L4 Bangalore experience – link between DFSS and milestone driven software project management Conclude on the gains and limitations of those approaches based on the current experience.
- 3. The TV experience in Bruges
- 4. CMM model ______ KPA _____ ( Key Process Area ) CMM Level 1 CMM Level 2 RM Requirements Management SLC Software lifecycle PP Project Planning PTO Project Tracking and Oversight SM Subcontract Management QA/SCV Quality Assurance/Software Compliance Verification SCM Software Configuration Management CMM Level 3 OPF Organisation Process Focus OPD Organisation Process Definition TP Training Program IM Integrated Management PE Product Engineering IC Intergroup Coordination PR Peer Reviews CMM Level 4 QPM Quantitative Process Management SQM Software Quality Management CMM Level 5 DP Defect Prevention TCM Technology Change Management PCM Process Change Management
- 5. CMM model Level 4 SQM : definition The purpose of Software Quality Management is to develop a quantitative understanding of the quality of the project's software products and achieve specific quality goals. Software Quality Management involves defining quality goals for the software products, establishing plans to achieve these goals, and monitoring and adjusting the software plans, software work products, activities, and quality goals to satisfy the needs and desires of the customer and end user for high quality products.
- 6. Quality goals to satisfy the needs and desires of the customer and end user for high quality products. The reference framework in the software is CMM (L4) Theoretically no actual DFSS deployment as such in the software, but: Use of VOC to derive requirements Use of derived CTQs to measure software quality Focus on non-functional requirements CMM philosophy applied + classical software reliability engineering applied Fault prevention Reviews Requirements management FMEAs / risk management Architecture Fault tolerance Reboots / watchdogs … Fault detection/correction Verification/validation Root cause analysis with feedback loop etc… In fact a lot of commonality with DFSS, close to an actual DFSS deployment.
- 7. What is SW Quality ? (choices made for a specific project) Stability Critical resources : e.g. Memory Budget tracking CPU load, latency, bandwidth Performance : e.g. startup time, zapping time, UI SDE warnings : Koala, compiler QAC warnings Asserts Debugging NFR conformity (validation) Reviews : Defect densities
- 8. SQM in a TV project Step 1. - create plan : SW_Quality_attribute_baseline Step 2. - deploy the plan measurements & reporting ( by product, subsystems, suppliers) Step 3. - corrective actions on deviations
- 9. Plan for a TV project : overview of CTQ’s x x Critical res. : Memory Budget tracking x Defect densities x NFR conformity (validation) x Debugger warnings x x Asserts x x QAC warnings x x SDE warnings : Koala, compiler x x Performance : startup, zapping, UI x x Critical res.: CPU load, latency, bandwidth x Stability Subsystem Platform Product CTQ metrics on the Plan
- 10. Example (zoom-in) : Performance (prod), QAC (subsys)
- 11. TV project : quality parameters Reporting Example :
- 12. Right design vs design right Right design (what does the consumer want ?) DFSS in requirements process Requirements process – functionality vs quality Support from development to marketing Design right (how do we implement it correctly ?) Software processes DFSS in development
- 13. Lessons learned The CMM L4/DFSS approach has worked to improve quality Approach made non-functional requirements more visible When marketing is not capable of providing exact requirements (logical for technical constraints for instance), development can help suggesting CTQs to start discussion. FMEAs not efficient enough in the software (risks can be listed but more difficult to make things measurable/w tolerance cf mechanical/electrical) A number of measurements reflect mainly process quality, not product quality (defects density) Supplier management: improve control over critical parameters from our suppliers : Stability, NFR, robustness. for some SW-suppliers this is handled as well as possible through provisions in requirements (NFR) and work agreements (measurement & reporting) for others : there are still examples where this is difficult to achieve. SW CTQ's (and NFR's) are twofold : those that contribute mainly to end-product (Stability, performance) those that contribute mainly to development trajectory (warnings…) Outside SW, only CTQ's contributing to the end-product are considered
- 15. Approach in Bangalore More classical DFSS deployment (cf Bruges example) Alignment between DFSS DIDOV phases and project milestones Translation of DFSS to software tasks This has been applied in numerous projects (wireless audio, DVD-recorder, …) Clear results with respect to customer satisfaction, reduced cost of non-quality, first time right design .
- 16. DFSS Approach in Software Life cycle CONQ reduction Listen to Consumer needs (Voice of consumer) Translate needs into CTQs with targets and specification limits Flow down the top-level CTQ (Y) into lower level inputs Xs Select Best design to meet the CTQ Predict Capability on “Y” based on current Xs Optimize gap between predicted and desired Verify CTQs on target, Factory Monitor CTQs in Field FMEA & Mistake proofing Req. Mgt Arch, Des & Imp Int. Test Maint Sw Life-cycle D I D O v M
- 17. Requirements Architecture / Top Level Design Detail Design Implementation Module testing Integration Testing Alpha testing DFSS mapping to V-model and SPEED Feasibility Beta testing Maintenance VPH CS PRS DR CR MPR Programming Conception Development Maturity Maintenance Define Identify Design Optimize Verify Monitor
- 18. Quality process - Software Mapping Quality process based on 4 main principles Identify the Voice of the Customer (VOC) Obtain & prioritize customer requirements, Derive CTQs Manage Critical parameters Define targets for CTQs, Flow-down CTQs Flow-up CTQs through predictions, Establish control plan Establish Robust & Reliable Design Identify sources of variation, de-sensitize design Optimize Performance, Verify & Validate robustness Manage risks & problems Anticipate risk, counter or mitigate them Detect and solve problems
- 19. Quality - Software Mapping 1- Identify the Voice of customer (VOC) Kano Risk Benefit VPH CRS CTQs , CFs HECE, UIS, FRS
- 20. Quality - Software Mapping 2- Manage Critical Parameters CTQ flow down CTQ – Targets & Limits CTQ flow-up & predictions Control Plan - Dashboard
- 21. Quality - Software Mapping 3- Establish robust & reliable design Arch/Des choices CTQ Trade-offs CTQ optimisation, Noise de-sensitization Transfer functions
- 22. Quality - Software Mapping 4 – Manage Risks & Problems User/CEC tests Test coverage Stress Tests Field Tests FMEA PMI, PR/CR tracking Verify CTQs
- 23. Gains Software engineering (addressing the pain areas) Better Requirements Management in terms of specifications, prioritization, traceability Focus on non-functional aspects of usability, interoperability, performance, robustness Importance of Execution architecture (State-transitions, CPU loading, Memory) Increase in Design effort, upfront design discussions automatically reduce rework Early involvement of test team – CTQ measurement method CTQs as leading indicators of product quality as against PMI Cross-functional approach Reduction in communication barrier – way of working with marketing e.g. challenging specifications, Kano, risk-benefit analysis Common language of CTQs with other disciplines for synergy e.g. hardware, electrical End-user perspective Development community sensitive to VOC E.g. needs versus wants Measurement focus (variation rather than average), best case-worst case spectrum System understanding (Product perspective) Transfer functions triggers to understand the system better (may expose competency) Understanding of noise and its effect on the system (usage environment) Trigger on “what could fail” (FMEA, mistake-proofing) improves robustness Soft Aspects Mindset of “first time right” instead of “build-test-fix”
- 24. Issues Everything is linked to CTQs so there is a chance of completely missing important ones Does not compensate for domain competency (CTQ flow down, FMEA etc) Many requirements in software fall under Yes/No, Pass/Fail category so limit setting limits, target becomes fuzzy (Critical factors rather than CTQs) Out of limits need not strictly mean defective as in case of hardware e.g. start-up time, hence predicting DPMO (defects per million opportunities) may not reflect reality Random failures due to only software are rare due to which concept like MTBF for software alone is questionable (however makes sense at product level) No concept of samples – the same piece of code is corrected and used, so hard-core statistical concepts cannot be applied at software level only (however can be done at product level) Configuration Management has to be addressed in the traditional way No direct tools available to control regression and side-effects in software Additional effort needs to be budgeted in the initial phases (confidence of Project) With more and more projects going the ODM way (Black-box model) – not clear on how to select/manage suppliers on software CTQs Marketing community still needs to be educated to be able to specify CTQs in a quantifiable way and as part of the requirements specification itself