Experiences with Data Feedback - Better Software 2004 - Ben Linders
0 likes2,797 views
The document discusses the importance of feedback in improving product quality and R&D performance, emphasizing that measurements should lead to actionable insights. Key experiences from Ericsson highlight that involving teams in the data analysis process fosters understanding and enables timely actions, ultimately enhancing performance and product outcomes. It outlines success factors, pitfalls, and concludes that effective feedback leads to better decision-making and improved organizational performance.
Experiences with Data Feedback - Better Software 2004 - Ben Linders
- 1. Rev A 2004-06-28 1 Make Whatâs Counted Count: Experiences with Data Feedback. 2004 Better Software Conference, September 30, San Jose, USA Ben Linders Operational Development & Quality Ericsson R&D, The Netherlands ben.linders@ericsson.com, +31 161 24 9885
- 2. Rev A 2004-06-28 2 Overview ⢠Why feedback? ⢠Experiences â Product Quality â R&D performance ⢠Key Success factors & Pitfalls ⢠Conclusions Feedback: Bridging the gap between data and actions!
- 3. Rev A 2004-06-28 3 Ericsson, The Netherlands ⢠Benelux Market Unit & Worldwide Main R&D Design Center ⢠R&D: Intelligent Networks â Strategic Product Management â Product marketing & technical sales support â Provisioning & total project management â Development & maintenance â Integration & Verification â Customization â Supply & support ⢠1300 employees, of which 350 in R&D Measurements on all R&D levels
- 4. Rev A 2004-06-28 4 Measurements: Analyze Data, lots of data, and nothing but data ⌠how can you get a meaning out of it? We tried: ⢠Historical data: takes long to build up, much effort before data can be used ⢠Industry data: hard to get, often too general ⢠âBrute forceâ SPC: conclusions didnât match with our perception and insight of the situation
- 5. Rev A 2004-06-28 5 Measurements: Actions The purpose of measuring is ⌠to take actions! Insufficient actions: ⢠No insight in causes ⢠Debates on the measurement ⢠Insufficient responsibility for results
- 6. Rev A 2004-06-28 6 Effectiveness of Measurements Change needed: Show relation between measurements and daily work, people should get insight in their own performance Get people involved from definition until results Assure that âvital few actionsâ are done Co-operation: Line/projects â Operational Development!
- 7. Rev A 2004-06-28 7 Feedback: Definition âInformation about the past, delivered in the present, which may influence future behaviorâ Seashore, Seashore & Weinberg, 1992 Analyze to understand current performance Change behavior to reach better results âInformation about collected data delivered to the people who have been doing the work, in order to support their understanding of the situation at hand and help them to take the needed actionsâ Linders, 2004
- 8. Rev A 2004-06-28 8 Feedback: Concepts People are capable of analyzing their own performance and results, you just have to provide them with the right data Empowerment: Make decisions at the lowest possible level Assure that you have valid data, before drawing conclusions Donât use data to punish people Feedback is a means to discuss data in an open atmosphere, enabling early conclusions and actions!
- 9. Rev A 2004-06-28 9 Feedback: Deployment ⢠Feedback should be: â on something that is considered important â quick and frequent â specific, valid, and understandable Start small, with a team that is open for it and willing to try Get feedback on how you are doing feedback
- 10. Rev A 2004-06-28 10 Experience 1: Measuring Product Quality Old approach: Quality Engineer gathered data, did analysis, and presented conclusions to design and test teams (often in a big report) Drawback: Teams didnât understand the data Data available when the project was finished: Too late No insight how measurement related to their work Teams didnât feel the need for changes Result: Hardly any improvement of product or process quality
- 11. Rev A 2004-06-28 11 Project Defect Model ⢠Project Defect Model: â to control quality of the product during development â and improve development/inspection/test processes ⢠Way of working: â Estimate # defects made during development per phase â Estimate defect detection rate per phase â Track the estimates against actual # defects found
- 12. Rev A 2004-06-28 12 Project Defect Model: Feedback New approach: Quality Engineer provides the model, and coaches the teams in estimating, tracking actuals, and drawing conclusions Benefits: Teams develop understanding on their way of working Teams have better insight on their progress/results Teams feel involved, itâs their data/conclusions Result: Conclusions (problem/risk) lead to early actions Teams get recognized for good results
- 13. Rev A 2004-06-28 13 Experience 2: Steering R&D performance Old approach: Quality Engineer collected target data, and presented conclusions to the management (or reported on them). Drawback: Much debates about the data/measurement No insight in causes when targets were not met Blame and denial Result: Metrics didnât support controlled improvement of the performance
- 14. Rev A 2004-06-28 14 Balanced Scorecard ⢠Balanced Scorecard: â comprehensive set of measurable targets â from different focus areas ⢠Way of working: â Collection of data by the Quality Engineers â Feedback/Interview sessions with managers ⢠Show the raw data ⢠Ask management for explanation ⢠Have management draw conclusions and take actions â Document the conclusions/actions with the data
- 15. Rev A 2004-06-28 15 Balanced Scorecard: Feedback New approach: Quality Engineer shows the raw data, signals trends, anomalies, and red flags, asks critical questions; working towards actions Benefits: Management knows what happened, together with Quality Engineer they can pinpoint and analyze the causes of insufficient performance Management feel involved, itâs their data/conclusions Result: Managers take earlier action Management Team focuses on show stoppers, while still keeping overview of total performance
- 16. Rev A 2004-06-28 16 Key Success Factors Data collected must relate to the organizationâs goals Management Support is crucial Quality Engineers have a central role People providing the data should be rewarded Order of data â analysis â conclusions â actions â predictions Communicate, communicate, communicate!
- 17. Rev A 2004-06-28 17 Pitfalls People might distrust data, say that it is wrong: Ask them for correct data (but donât try to be perfect: optimize!) People sometimes do not want to participate in analysis Make clear that only they can draw conclusions People sometimes do not want to take actions: Top-down goal setting, assure they accept the targets People are wary of change Start with less critical measurement, communicate successes
- 18. Rev A 2004-06-28 18 Conclusions Feedback improves effectiveness of measurements: â Earlier insight in performance & risks â Involvement of those whose work is measured â Actions taken by teams and middle management Make whatâs counted count!
- 19. Rev A 2004-06-28 19 Further reading References â What did you say? The art of giving and receiving feedback. Seashore, Seashore & Weinberg, Douglas Charles press, 1992 â Getting things done when you are not in charge. Geoffrey M. Bellman, Fireside, 1993. â How to talk about work performance: A feedback primer. Esther Derby, Crosstalk December 2003, page 13-16. Papers â Controlling Product Quality During Development with a Defect Model, Proceedings ESEPG 2003 â Make whatâs counted count, Better Software magazine, march 2004 Ben Linders, Ericsson R&D, The Netherlands ben.linders@ericsson.com, +31 161 24 9885