Experimenting in the Enterprise Agile 2013 Conference
24 likes5,059 views
This document provides an overview of lean startup methodology for experimenting with new business ideas and products. It discusses key lean startup concepts like validating business model assumptions through customer experiments, mapping experiments to assumptions, and selecting high-learning experiments that can be run quickly. Metrics for experiments should be auditable, actionable and accessible. Customer development is emphasized through techniques like concierge testing and focus groups to better understand customer needs and problems.
Experimenting in the Enterprise Agile 2013 Conference
- 1. Experimenting Enterprise © 2013 BigVisible Solutions in the
- 2. 2© 2012 BigVisible Solutions David J Bland Startups consulted: 18 Twitter followers: 2179 Iterations of Coaching: 215 Speaker Vanity Metrics Brian Bozzuto Blogs & Articles Written: 77 Presentations Delivered: 46 Teams coached: 54
- 3. © 2013 BigVisible Solutions 3 This session doesn’t guarantee a 100 million dollar success, but it should help prevent a 100 million dollar failure.
- 4. © 2012 BigVisible Solutions 4 A business model used to be static and survived many years. 38 13 4 3 2 0 10 20 30 40 Radio TV Internet iPod Facebook Years to Reach a Market of 50 Million
- 5. © 2012 BigVisible Solutions 5
- 6. © 2012 BigVisible Solutions 6 The Lean Startup provides a scientific approach to creating and managing startups and get a desired product to customers’ hands faster. -Eric Ries
- 7. Entrepreneur Intrapreneur © 2013 BigVisible Solutions 7
- 8. Entrepreneur Intrapreneur Skills © 2013 BigVisible Solutions 8
- 9. © 2013 BigVisible Solutions 9 This is great in theory, but it’s not for B2B. This is great in theory, but I’m a visionary. EntrepreneurIntrapreneur
- 10. © 2013 BigVisible Solutions 10 Define the value proposition for: Source: Adapted from Steve Blank Buyer User Decider Buyer User Decider B2C B2B
- 11. © 2013 BigVisible Solutions 11 Are your users/customers: Source: Adapted from Steve Blank Aware of problem? Actively seeking a solution? Hacking together a solution? Have $$$ for your solution?
- 12. © 2013 BigVisible Solutions 12 Can We Build It? Should We Build It?
- 13. © 2013 BigVisible Solutions 13 Requirements = Hypotheses
- 14. © 2013 BigVisible Solutions 14 Source: Adapted from Cynefin Simple “Everyone Knows That” Complicated “Let’s consult the expert” Complex “Whoa, it all makes sense now…” Chaos “That still doesn’t make sense” Work with experts to inform our hypothesis Run experiments to gain new knowledge
- 15. © 2013 BigVisible Solutions 15 Safe Fail != Fail Safe
- 16. © 2012 BigVisible Solutions 16 Experiment with product & process
- 17. © 2013 BigVisible Solutions 17 If you experiment just to see what happens, then you’ll succeed at seeing what happens.
- 18. © 2013 BigVisible Solutions 18 Be careful how you structure your experiments
- 19. © 2013 BigVisible Solutions 19 Image Source: Eric Ries The Lean Startup Then here Start here Last here
- 20. © 2013 BigVisible Solutions 20 Image Source: Eric Ries The Lean Startup Now Execute Quickly
- 21. © 2012 BigVisible Solutions 21 Experiment Template For <customer segment> I predict <outcome> when I run <experiment>
- 22. © 2012 BigVisible Solutions 22 Experiment Example For new customers, I predict 10% increase in conversion when I offer a “no questions asked” cancellation policy
- 23. © 2012 BigVisible Solutions 23 Experiment Guidelines • Customer Segment (who) • Up to / Limit Of (how many) • Length of Time (how long) • Financial Exposure (how much) • Amplification & Dampening Measures
- 24. © 2013 BigVisible Solutions 24 Experiment Dashboard ID Hypothesis Description Status Next Steps 01a We predict that adding a refund feature will increase conversions Story 13c adds refund label and call to action button that puts customer in contact with call center agent. Complete 0.2% increase in conversion on cohort. Meeting with VP of Ops to discuss strategy.
- 25. © 2013 BigVisible Solutions 25 “This is all great, but how do we know what experiments to run?”
- 26. © 2012 BigVisible Solutions 26 Iterative Product + Static Business Model Failed Organization
- 27. © 2012 BigVisible Solutions 27Source: Alex Osterwalder
- 28. © 2012 BigVisible Solutions 28 ? ? ??? ? ? ? ? ? ? ? “Let’s map out our existing business model” Source: Adapted from Alex Osterwalder tip: tell a story
- 29. © 2012 BigVisible Solutions 29 ? ? ??? ? ? ? ? ? ? ? “Let’s validate our riskiest assumptions” Source: Adapted from Alex Osterwalder
- 30. © 2012 BigVisible Solutions 30 This is most likely your Riskiest Assumption Source: Adapted from Alex Osterwalder
- 31. © 2012 BigVisible Solutions 31 This is most likely your other Riskiest Assumption Source: Adapted from Alex Osterwalder
- 32. © 2012 BigVisible Solutions 32 What happens when we subtract costs from revenue? Source: Adapted from Alex Osterwalder
- 33. © 2013 BigVisible Solutions 33 “Okay, now we have a big list of assumptions, how do we decide where to start?”
- 34. © 2013 BigVisible Solutions 34 experiment experiment experiment experiment experiment experiment experiment “Let’s dot vote!” (you could, but that’s pretty subjective)
- 35. © 2013 BigVisible Solutions 35 time learning experiment experiment experiment experiment experiment experiment experiment experiment experiment experiment experiment experiment experiment (ok better, but what are they tied to?)
- 36. © 2013 BigVisible Solutions 36 Now we have a focusing mechanism focustime time learninglearning experiments experiments experiments experiments
- 37. © 2013 BigVisible Solutions 37 And assumptions that need validation focus experiments experiments experiments experiments time time learninglearning
- 38. © 2013 BigVisible Solutions 38 This is Experiment Mapping focus experiments experiments experiments experiments time time learninglearning
- 39. © 2013 BigVisible Solutions 39 Customer Segment is our focus focus experiments experiments experiments experiments time time learninglearning Customer Segment
- 40. © 2013 BigVisible Solutions 40 Will they pay for our product? focus experiments experiments experiments experiments time time learninglearning Customer Segment They will pay for our product
- 41. © 2013 BigVisible Solutions 41 Can we scale to meet demand? focus experiments experiments experiments experiments time time learninglearning Customer Segment They will pay for our product We can scale to meet demand
- 42. © 2013 BigVisible Solutions 42 Can we use our distribution channels? focus experiments experiments experiments experiments time time learninglearning Customer Segment They will pay for our product We can use existing distribution channels We can scale to meet demand
- 43. © 2013 BigVisible Solutions 43 Are we solving a real need? focus experiments experiments experiments experiments time time learninglearning Customer Segment They will pay for our product Our product solves a need they have We can use existing distribution channels We can scale to meet demand
- 44. © 2013 BigVisible Solutions 44 Which of these is the riskiest? focus experiments experiments experiments experiments time time learninglearning Customer Segment They will pay for our product Our product solves a need they have We can use existing distribution channels We can scale to meet demand
- 45. © 2013 BigVisible Solutions 45 Can we do a concierge test? focus experiments experiments experiments experiments time time learninglearning Customer Segment They will pay for our product Our product solves a need they have We can use existing distribution channels We can scale to meet demand Concierge test with 10 customers
- 46. © 2013 BigVisible Solutions 46 We could build a prototype focus experiments experiments experiments experiments time time learninglearning Customer Segment They will pay for our product Our product solves a need they have We can use existing distribution channels We can scale to meet demand Concierge test with 10 customers Build a prototype
- 47. © 2013 BigVisible Solutions 47 Maybe run a focus group? focus experiments experiments experiments experiments time time learninglearning Customer Segment They will pay for our product Our product solves a need they have We can use existing distribution channels We can scale to meet demand Concierge test with 10 customers Build a prototype Run a focus group with 80 people
- 48. © 2013 BigVisible Solutions 48 Prototype will take time focus experiments experiments experiments experiments time time learninglearning Customer Segment They will pay for our product Our product solves a need they have We can use existing distribution channels We can scale to meet demand Concierge test with 10 customers Build a prototype Run a focus group with 80 people
- 49. © 2013 BigVisible Solutions 49 Focus groups take a long time too focus experiments experiments experiments experiments time time learninglearning Customer Segment They will pay for our product Our product solves a need they have We can use existing distribution channels We can scale to meet demand Concierge test with 10 customers Build a prototype Run a focus group with 80 people
- 50. © 2013 BigVisible Solutions 50 Concierge test is quick & we’ll learn focus experiments experiments experiments experiments time time learninglearning Customer Segment They will pay for our product Our product solves a need they have We can use existing distribution channels We can scale to meet demand Concierge test with 10 customers Build a prototype Run a focus group with 80 people
- 51. © 2013 BigVisible Solutions 51 “Let’s swarm on this experiment” high learning short timeframe
- 52. © 2013 BigVisible Solutions 52 Just because you can measure it, does not mean you should measure it.
- 53. © 2013 BigVisible Solutions 53 Auditable Actionable Accessible Source: Eric Ries The Lean Startup
- 54. © 2013 BigVisible Solutions 54 Auditable “Be able to defend your metrics with confidence”
- 55. © 2013 BigVisible Solutions 55 Auditable? zombie project metrics
- 56. © 2013 BigVisible Solutions 56 Actionable “Does this metric incentivize a change in behavior?”
- 57. © 2013 BigVisible Solutions 57 Actionable? Facebook likes won’t influence your dog
- 58. © 2013 BigVisible Solutions 58 Actionable Number of steps should influence your lifestyle
- 59. 0 50 100 150 200 250 1 2 3 4 5 6 7 8 9 10 11 Velocity(Points) Iteration Team Velocity per Iteration Delta Planned Actual Avg Last 3 © 2013 BigVisible Solutions 59 Actionable? What does this tell you about your product?
- 60. © 2013 BigVisible Solutions 60 Accessible “Can you easily access and share this data?”
- 61. © 2013 BigVisible Solutions 61 Accessible? Not accessible
- 62. © 2013 BigVisible Solutions 62 Accessible Accessible Image Source: Lean.st
- 63. © 2013 BigVisible Solutions 63 Source: Dave McClure Pirate Metrics Acquisition Activation Retention Referral Revenue
- 64. © 2013 BigVisible Solutions 64 Source: Dave McClure Pirate Metrics Acquisition Activation Retention Referral Revenue
- 65. © 2013 BigVisible Solutions 65 “Metrics are people, too” Source: Eric Ries The Lean Startup
- 66. © 2013 BigVisible Solutions 66 Aware Hopeful Satisfied Passionate Source: Adapted from Cooper & Vlaskovits
- 67. © 2013 BigVisible Solutions 67 Aware Hopeful Satisfied Passionate Acquisition Activation Retention Referral
- 68. © 2012 BigVisible Solutions 68 Start small.
- 69. © 2012 BigVisible Solutions 69 When you have stories around an uncertain feature, add a hypothesis that states the expected outcome on a specific metric.
- 70. © 2012 BigVisible Solutions 70 If you have a Kanban Board, add a Validate Column to the very right of it.
- 71. © 2012 BigVisible Solutions 71 Problem Solution Process Known Known Waterfall or Agile Known Unknown Agile Unknown Unknown Lean Startup
- 72. © 2012 BigVisible Solutions 72 Where are the unknown / unknown’s in your enterprise?
- 73. David J Bland Experimentor Brian Bozzuto Principal 73© 2013 BigVisible Solutions Thank You!
Editor's Notes
- #59: Overall, pedometer users increased their physical activity by 26.9% over baseline. An important predictor of increased physical activity was having a step goal such as 10,000 steps per day (P = .001). When data from all studies were combined, pedometer users significantly decreased their body mass index by 0.38 (95% CI, 0.05-0.72; P = .03)." Promoting Lifestyle Physical Activity: Experiences with the First Step Program.Am J LifestyleMed. 2009 Jul 1;3(1 Suppl):508-548.
- #60: Overall, pedometer users increased their physical activity by 26.9% over baseline. An important predictor of increased physical activity was having a step goal such as 10,000 steps per day (P = .001). When data from all studies were combined, pedometer users significantly decreased their body mass index by 0.38 (95% CI, 0.05-0.72; P = .03)." Promoting Lifestyle Physical Activity: Experiences with the First Step Program.Am J LifestyleMed. 2009 Jul 1;3(1 Suppl):508-548.