Get Buy-In to Invest in Experimentation by Nate Weisz - The Lean Startup Conference 12/9/14

Editor's Notes

• #2: I’m coming from a professional services background so we have a bunch of product development teams building software products for our clients. We feel that direct user feedback is critical to product development – but found that our client’s leadership teams don’t always agree
• #3: We tried to create a practice around user testing, user research - didn’t take off, client leadership didn’t see the value, they knew everything: “just focus on building!”
• #4: But we found something magic buried in our delivery process in agile approach to building software – we found a great venue for getting direct user feedback and how to do that in such a way it convinces our clients leadership team that this feedback is gold
• #5: What we found is related to Sprint demo – end of two week sprint, a meeting where we show working software – attendees = client leadership, our team, and hopefully some representative users. In prof serv we have advantage of having a lot of different teams working with a lot of different clients.
• #6: We noticed Anti pattern – a lot of teams doing sprint demos wrong – not the right people because stakeholders don’t know what they don’t know – and our demos were one-way we show features
• #7: But demos are supposed to be about feedback!! We have this opportunity every two weeks to get feedback and to many of our teams it’s more about showing progress
• #8: And this is my POINT – by taking the sprint demo – and elevating it by applying techniques I will share – we not only improve product – but we eventually used all this goodness in aggregate towards larger goal of convincing client leadership it’s worth some effort into this.
• #9: Technique - Logistics We try to get representative users at appropriate points in addition to product/biz stakeholders, set expectations in advance that we need their feedback.
• #10: Technique - Plan we ask our teams to prepare questions, Know your areas of uncertainty, , ask them
• #11: Technique - Plan - Example iPad app for defense contractor doing aircraft maint. Organization hesitant to make senior mechanics and shop QA available. Demos were with stakeholder
• #12: Technique - Plan - Example First E2E design prototype, finally got the right people, still one-way underwhelming. lucky we had questions prepared - they opened up
• #13: Technique - Plan - Outcome Next demo improvements. Stakeholders eyes open on how well that worked - talked about it all the time and later did time study in shop
• #14: Technique - Drive Instead of traditional agile where dev team drives demo - make users do it – we ask our teams to create a step by step script - simple but engagement through the roof
• #15: Technique - Drive – Example pricing engine for digital publisher, many touchpoints with accounting -- but only sr mgmt in development
• #16: Technique - Drive - Example got accounting involved - still not much coming out. Asked accounting to perform tasks - watch out as feedback flow -- scary amount in fact
• #17: Technique - Drive - Outcome Discovered significant modifications necessary - put pieces back together and accounting drove again - success and all were glad and agreed v. bad without that
• #18: Technique Summary – these simple – we show don’t tell instead of hand waving and describing how important
• #19: Conclusion They will believe – they see in several micro cases how much value direct user feedback brings – how much we learn and improve
• #20: We find client leadership more receptive to dedicated user testing and research - In one case we ended up working with a client on a time study. Also it gives us countless real stories to tell to new clients and convince them up-front prototyping
• #21: Found these techniques effective and because they are a natural outgrowth of agile approach I think they will be effective for many of you. Thank you!