2016 metrics-as-culture
6 likes3,846 views
Measurement is integral to organizational culture and innovation, as it allows for objective testing of ideas and practices. Effective metrics shape behavior, inform incentives, and ensconce decision-making, emphasizing the need to measure relevant outcomes while avoiding unnecessary metrics. Ultimately, organizations that prioritize measurement foster a culture of learning and experimentation, leading to improved performance and collaboration.
2016 metrics-as-culture
- 3. @nicolefv
- 4. @nicolefv "As good as Amazon was, it did not yet have a culture that fully embraced measurement and debate. I think building this culture is the key to innovation. Creativity must flow from everywhere. Whether you are a summer intern or the CTO, any good idea must be able to seek an objective test, preferably a test that exposes the idea to real customers. Everyone must be able to experiment, learn, and iterate. Position, obedience, and tradition should hold no power. For innovation to flourish, measurement must rule.” - Greg Linden http://glinden.blogspot.com/2006/04/early-amazon-shopping-cart.html
- 6. @nicolefv Evaluating well-designed and executed experiments that were designed to improve a key metric, only about 1/3 were successful at improving the key metric! Online Experimentation at Microsoft, Kohavi et al. http://stanford.io/130uW6X
- 10. @nicolefv@nicolefv Sometimes, we just measure ALL THE THINGS. But STOP and ask yourself: • What does this tell me? • How can I use this to make a decision? • How does this relate to an outcome? • Does it still make sense to measure and track this metric? • What are the unintended consequences of implementing this? NOT EVERYTHING THAT CAN BE COUNTED… COUNTS
- 13. @nicolefv
- 15. @nicolefv WESTRUM ORGANIZATIONAL CULTURE PATHOLOGICAL Power-oriented BUREACRATIC Rule-oriented GENERATIVE Performance-oriented Low cooperation Modest cooperation High cooperation Messengers shot Messengers neglected Messengers trained Responsibilities shirked Narrow responsibilities Risks are shared Bridging discouraged Bridging tolerated Bridging encouraged Failure leads to scapegoating Failure leads to justice Failure leads to inquiry Novelty crushed Novelty leads to problems Novelty implemented Westrum, R. (2004). A typology of organisational cultures. Quality and safety in health care, 13(suppl 2), ii22-ii27.
- 16. @nicolefv HOW TO MEASURE WESTRUM ORGANIZATIONAL CULTURE IN A STATISTICALLY VALID & RELIABLE WAY Rate how strongly you agree (7) or disagree (1) to the following statements: • On my team, information is actively sought. • On my team, failures are learning opportunities, and messengers of them are not punished. • On my team, responsibilities are shared. • On my team, cross-functional collaboration is encouraged and rewarded. • On my team, failure causes enquiry. • On my team, new ideas are welcomed. Forsgren, N., J. Humble (2016). “The Role of Continuous Delivery in IT and Organizational Performance.” In the Proceedings of the Western Decision Sciences Institute (WDSI) 2016, Las Vegas, NV.
- 17. @nicolefv WHAT CAN I DO? • Read and share Data Driven, by @dpatil and @hmason • Ask others about metrics: • Is it happening? Where? • Is it important? • Think about your own metrics: • Start with outcomes: what’s important? • Then think about what influences your outcomes • Define and communicate
- 18. @nicolefv METRICS HELP US COMMUNICATE KEY: • Give them meaningful names • Define them well • Communicate them often and across boundaries
- 19. @nicolefv WE’VE TALKED ABOUT: MEASUREMENT IS CULTURE • You can’t improve what you don’t measure • Always measure things that matter • That which is measured gets managed • If you capture only one metric, you know exactly what will be gamed • Metrics inform our incentives… and shape behavior • Not everything that can be counted... Counts • Just because something is hard to measure doesn’t mean it isn’t worth measuring