Data Driven Decision Making for Product Development
1 like457 views
The document discusses the importance of data-driven decision-making in organizations, emphasizing the need to break down problems, frame them effectively, and quantify metrics for better understanding. It outlines various categories of knowledge, such as known knowns, known unknowns, and unknown unknowns, to highlight the challenges of identifying and addressing issues within teams. The presentation provides a framework for improving team dynamics and decision-making processes through iterative learning and measurement.
Data Driven Decision Making for Product Development
- 1. Data Driven Decision Making By Denise Yu & Zoe Vance @deniseyu21
- 2. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Hello SpringOne Platform! 2
- 3. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Why is this topic important? Velocity is a vector: a function of both speed and direction. Breaking down, framing and quantifying problems helps ensure that we’re always moving in the right direction. 3 
- 4. Data in service of decision making
- 5. 5  Tools to help you take a problem, break it down, quantify it, understand it and make a decision Goal
- 6. Quantitative vs Qualitative Sales increased 10% Marketing reached 150 fewer people this quarter Development speed increased 15% Users are enjoying the new premium features We sent too any promotional emails last quarter Our new processes helped the team move quickly
- 7. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Known Knowns Source: https://www.filmhousecinema.com/showing/mountain
- 8. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Known knowns: We all know we have a problem There’s an issue on the team and everyone knows about it BUT there is a lack of consensus on how to tackle it 8 
- 9. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Our pipelines were taking ages to run 9  example Concourse deployment pipeline (concourse-ci.org)
- 10. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Break the problem down and quantify 1 0  Source: Maria Ntalla, Maya Rosecrance, Sarah Connor, Jack Newberry, Oliver Stenbom
- 11. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Keep improving and measuring 11  Source: Maria Ntalla, Maya Rosecrance, Sarah Connor, Jack Newberry, Oliver Stenbom
- 12. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Takeaways 1 2  Decision enabled: Focus attention on a problem, break it down so it’s addressable and quantifiable Metrics chosen: Time overall, time per task
- 13. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Known Unknowns Source: Maria Ntalla, Maya Rosecrance, Sarah Connor, Source: https://www.minube.net/photos/place/695291
- 14. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Known unknowns: Something is wrong, but we’re not quite sure what Situations where the issues are unsurfaced but there may be some awareness 1 4 
- 15. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 1 5 
- 16. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Assess the value of what you’re trying to learn 1 6  We didn’t really understand the number or what it meant when we hit/didn’t hit it.
- 17. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ We realised we had different types of users! 1 7  One group of users built new things with frequently-pulled data Another group copied data (huge infrequent pulls from data sources)
- 18. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Surfaced bigger existential questions and debate 1 8  What are we building? For who? What do we want to be? Measure our success against that goal
- 19. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Takeaways 1 9  Decision enabled: Type of product we want to be for whom. Metrics chosen: Basic starter metrics Patterns in user behaviors* * complemented with qualitative information!
- 20. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Unknown Unknowns Source: https://ua.wallpapers-fenix.eu
- 21. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Unknown unknowns: We don’t even know we have a problem Surface and acting on nebulous issues 2 1 
- 22. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ How is the team doing? 2 2  Spotify Health Check Source: labs.spotify.com/ 2014/09/16/squad- health-check-model/
- 23. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 2 3  Tools & Technologies that we use Concourse BOSH Go Github
- 24. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Concourse BOSH Go Github ... How important for our success? (1-3) 3 3 1 2 2 4 
- 25. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Concourse BOSH Go Github ... How important for our success? (1-3) 3 3 1 2 Where are we now? (1-3) 1 2 1 2 2 5 
- 26. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Concourse BOSH Go Github ... How important for our success? (1-3) 3 3 1 2 Where are we? (1-3) 1 2 1 2 2 6 
- 27. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Concourse 2 7  Where are we as a team now? (1-3) 3 What worked?
- 28. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Takeaways 2 8  Decision enabled: Spend more time becoming a stronger team Metrics chosen: Self-reported assessment
- 29. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Framework to apply this to your work
- 30. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Step 1: Identify a problem (ex: Stinky Fish) 3 0  What are the biggest pains? What problem, if we continue to ignore it, will become the biggest pains down the line?
- 31. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Step 2: Break down and frame the one problem 3 1  Inaccessible Big Nebulous Accessible Small Addressable
- 32. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Step 3: Measure quantitatively to learn qualitatively Measurements you choose don’t need to be perfect or a target. But they should be good enough to provoke conversations and learnings - as you learn more, you can decide if you need high-fidelity data or just a lightweight insight. 3 2  Source: http://www.goodwinandgoodwin.com/
- 33. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Based on your learnings, iterate and improve how you measure and assess Step 4: Iterate & Learn 3 3 
- 34. Unless otherwise indicated, these slides are © 2013-2018 Pivotal Software, Inc. and licensed under a Creative Commons Attribution-NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/
- 35. Take a hard problem Frame it Break it down Measure it Understand it 3 5 
- 36. Thanks to… Maria Ntalla Therese Stowell Ruth Maclean 3 6 
- 37. > Stay Connected. Talk to us! zvance@pivotal.io & dyu@pivotal.io #springone@s1p