Evolving the Digital Experience at Pace with Experimentation - Chris Gibbins
0 likes248 views
This document discusses how experimentation and iterative design are essential for digital businesses to evolve with changing customer needs. It advocates embedding A/B testing into the design process to rapidly test hypotheses. Examples are provided of how a retailer improved filtering visibility and simplified product pages through multiple design variations tested concurrently. Continuous experimentation is likened to a "natural selection" process that delivers the best designs. The document concludes that an experimental approach to creativity is best for customer experience optimization.
Evolving the Digital Experience at Pace with Experimentation - Chris Gibbins
- 1. Evolving the Digital Experience at Pace with Experimentation Chris Gibbins, Director of UX & Optimisation, Biglight 2019
- 4. We help our clients to continually evolve & innovate the customer experience
- 5. 2018 “Retail Armageddon” 2,594 stores in administration
- 6. The way people shop is changing And so are their needs & expectations
- 7. Retailers not evolving fast enough! Lazy design practices, one of the contributing factors
- 8. Copying others - “JL’s checkout looks good!” Which variation do you copy?
- 9. ‘Everyone else does it like this’ syndrome! “How the Hamburger Menu Almost Led to a Triple-Digit Million Pound Loss” https://bit.ly/2wi3WCN Joe Pendlebury
- 10. Blindly following “best practice” when it comes to design solutions - Approach / methods - Design solutions
- 11. Making stuff up Playing it safe Far too much…
- 12. Alexander McQueen: “You don’t move forward if you play it safe”
- 13. So retailers need to…
- 14. Focus on solving real customer problems more effectively & more quickly than the competition
- 15. Embed experimentation into the design process
- 16. Digital is an extremely malleable medium
- 17. Digital + A/B/Personalisation technologies = continuous feedback from your customers
- 18. Digital, the perfect medium for experimentation!
- 19. How we do this
- 20. Uncovering customer problems & opportunities Rapid design & A/B testing of hypothesised solutions Rollout of winning experiences Customer Experience Optimisation (CXO) A B C B
- 21. Lean UX Process Test 3 or 4 “Prototypes” (or design variations) at the same time Test multiple concepts A/B Testing live on the Website or App Learn from all your users + Get statistically significant results Our CXO Process Iterate Rollout the winners immediately Ideation (e.g. crazy 8’s) & the ‘wildcard’ concept Research & analysis
- 22. Crazy 8’s ideation and the “wild card” concept 8 ideas wildcard V1 V2 V3 V4
- 23. A/B testing results from a real experiment wildcard
- 24. The numbers from one of our CXO programmes One of our cross-functional delivery teams Rapid design & build (2 week sprints) 45 experiments in the year 14 winners (31% success rate) 3 design variations per experiment (on average) 135 designs completed! Around 100 designs didn’t work! Project Manager UX Specialist UI Designer Analyst Developer Yearly output:
- 25. Jeff Bezos: “Our success at Amazon is a function of how many experiments we do per year, per month, per week, per day,”
- 26. Case Study 1: Solving a simple customer problem identified on the Hobbs website
- 27. Filtering?
- 28. “On other websites there’s usually a thing that allows you to see only the blue dresses!” User 2 who couldn’t find the filters Filtering?
- 29. Problem Statement: Users completely missed the filtering options due to the poor visibility and affordance. But conversion rate was considerably higher for those who used filters.
- 30. What if users could actually find the filters? Original Design idea 1
- 31. What if users could actually find the filters? Original Design idea 1 Design idea 2
- 32. What if users could actually find the filters? Original Design idea 1 Design idea 2 Design idea 3 If you had to guess and go with just one?
- 33. We ran all 3 in an A/B test Original Variation 1 Variation 2 Variation 3 +10% (significant)Filter use: +7% (significant)+14% (significant) -4% (significant)Conversion Rate: -2% (not significant)double digit % increase (significant) WINNER
- 34. Guessing can be very expensive Original Variation 1 +10% (significant)Filter use: -4% (significant)Conversion Rate: For an average retailer who makes £30 million on mobile With 80% conversions going through the PLP The wrong decision (i.e. guessing V1) could cost -£960,000 per year
- 35. You can’t afford NOT to build A/B testing into your design process?
- 36. The “Natural selection” of the best design!
- 37. Case Study 2: The evolution of the Furniture Village digital experience through experimentation
- 38. The CXO Backlog full of Problem Statements, hypotheses and new ideas The CXO roadmap
- 39. Experimentation example 1: Improving the visibility of the navigation and search bar on desktop
- 41. Problem Statement: Users struggled to see the top categories in the navigation due to the poor visibility of the navigation. Search was an important feature for users however this also lacks prominence.
- 42. Variation 1 The Control Variation 2 +19% increase in Add-to-Basket rate WINNER
- 44. Experimentation example 2: A series of more adventurous A/B experiments to reimagine the product details pages
- 45. Problem Statement: The presentation of information and options was overwhelming for users. Poor visibility of sizes, storage options and add-ons, resulting in low uptake.
- 46. 3 design variations tested Control (original PDP) Winning variation
- 47. The Winning Variation Problem Statement: Still room for improvement to simplify the design. Also we observed interaction issues with the tabbed navigation.
- 48. 2 design variations tested The New Control Winning variation
- 49. Luke Wroblewski: “Design is never done”
- 50. The next experiment going live in 2 weeks The New Control Variation 1 - Product images exposed
- 51. Final thoughts
- 52. Conceptual Creativity Experimental Creativity Paul Cézanne Pablo Picasso David Galenson's theory on creativity, 2006 Figuring it out in advance e.g.: Figuring it out through trial and error (experimentation) e.g.: or WINNING APPROACH FOR CX Conceptualists know what they want and know when they’ve created it Experimentalists never know when their work is finished!
- 53. Liberating
- 54. Thank You Chris Gibbins Email: chris.gibbins@biglight.co.uk