Customer-centric Metrics
- 1. Customer-centric Metrics Mark McBride October 4th, 2017
- 2. Mark • Twi)er - pla/orm/API team during the great de-monolithing • Nest - led service engineering, launched their developer program • Turbine Labs - turning pracCces developed at Twi)er/Nest/etc. into products you can buy • @mccv on the Twi)er
- 4. Care vs. Anxiety • Caring about whether your system is working is the heavy burden of being a grown up soLware professional • Caring about things you can’t quanCfy leads to anxiety • Caring about too many things leads to anxiety • Caring about a tractable number of quanCfiable things frees you from anxiety
- 5. A Twi)er Metrics History • 2009 - Ganglia • Open source • Limited scalability • URL based UI • Limited exploraCon • Chart unCl the system falls over
- 6. A Twi)er Metrics History • Early 2010 - Reunion • Custom built for diagnosis • ExploraCon oriented • Limited scalability • Only look at what you need to resolve the incident at hand
- 7. A Twi)er Metrics History • 2010 - Cuckoo/Viz • Custom built for scalability • Dashboard oriented • Highly scalable • Charts for everything
- 8. A Twi)er Metrics History • The cardinality of services exploded • Blindness was replaced with cogniCve overload
- 9. Switch Focus to Customers, Not System Internals
- 10. Simplified View of Service Health • What do customers care about? • Success Rate - I want it to work • Latency - I want it to be fast • RPS - I want to be able to use this much of the service • What don’t customers care about? • CPU burn, disk uClizaCon, leader elecCons, etc. etc. etc.
- 11. Success Rate • Stuff obviously needs to work • Necessary but not sufficient • “Nines don’t ma)er if customers aren’t happy” - @mipsyCpsy
- 12. Request Rate • A contract about how much service you’re willing to provide • This is oLen implicit • It’s sCll important
- 13. Latency • Don’t (just) measure medians • Measure the experience of your worst N% of customers, then make them happy
- 14. Puhng it All Together • A simple view of customer happiness • This is not a diagnosCc view • This lets you know whether you need to start diagnosing • More importantly, it quickly lets you know that you don’t need to worry
- 15. Not All AcCons are Equal
- 16. SegmentaCon • Consider an online shop • Customers want to look at stuff • Customers want to buy stuff • Aggregates Hide InformaCon • Looks >>> buys
- 17. SegmentaCon • Break down health by acCon • This gives you a clearer view of all the ways customers could be unhappy • Scan quickly to ensure your service is ok, get back to building features
- 18. But I Don’t Serve Customer Traffic
- 19. Teammates Become Customers • Teammates • Have a relaCvely deep understanding of your system • Are willing to be flexible with system changes • Customers • Don’t understand the internal details of your system • Unwilling to adapt to incidents with your system
- 20. Teammates Become Customers • As teams grow, specializaCon necessarily becomes the norm • TreaCng customers like teammates is going to hurt
- 21. A Twi)er History • 2011 - Viz Standards • Pick good metrics • Show them for your service • Measure them for all your dependent services
- 22. Embrace Customerhood • Manage your internal systems like they serve customers • Strong abstracCon boundaries (APIs) • Measure customer experience, not system internals
- 23. Embrace Customerhood • This means you get to manage your internal dependencies like you’re a customer! • You don’t have to worry about what’s behind the API • But you can easily observe the behavior of the API you’re calling • Common metrics create a common language for discussing system behavior • Common language reduces anxiety team wide
- 24. Observability the Easy Way
- 25. Network AbstracCon Boundaries • Now that everything is behind APIs, everybody is talking over the network • Proxies make this a programmable abstracCon boundary • Proxies give you a common point of observaCon and control • CapabiliCes span languages and runCmes • CapabiliCes can be applied globally
- 26. The Network • Stop thinking of the network as (only) a tax • AbstracCon boundaries are good things • Programmable abstracCon boundaries are be)er • Behind that boundary there is freedom to move • Across those boundaries you can observe and control service interacCon
- 28. • Watch the metrics that make happy customers • Watch metrics for all customer acCons • Develop a common language for discussing system health • Treat teammates like customers • Use your network as a common point of observability and control Todos