Data and Design: BFFs or Frenemies?
Download as PPTX, PDF0 likes629 views
The document discusses the interplay between data and design in UX, emphasizing the importance of not solely relying on data to inform design decisions. It highlights potential traps in design practices, such as optimizing for local maximums and measuring the wrong metrics, while advocating for a continuous learning approach to design. Additionally, it shares insights from NPR's experiences in using data to enhance user engagement and improve digital interfaces.
Data and Design: BFFs or Frenemies?
- 1. Data and Design: BFFs or Frenemies? Steve Mulder @muldermedia
- 2. 2 UX & Design Research Testing Analytics
- 3. 3
- 4. 4
- 5. 5
- 6. 6
- 7. 7 Trap #1 We design by blindly following data
- 8. 8
- 9. 9 Trap #2 We measure the wrong thing
- 10. 10
- 11. 11 Trap #3 We become optimizers instead of designers
- 12. 12 The danger of optimizing to a local maximum Local maximum Optimization via A/B or usability testing
- 13. 13
- 14. 14 1. Data is the platform from which we leap
- 15. 15 For NPR, ratings data tell us about listening
- 16. 16 Third-party research validates trends in media fragmentation and digital adoption
- 17. 17 Channels proliferate, presenting both opportunity and challenge “Radio isn’t going away, it’s going everywhere.”
- 18. 18 Our own research and personas reveal changing consumer behaviors & opportunities
- 19. 19 It’s not about building things that users know to ask for
- 20. 20 New digital interfaces in familiar locations (connected car)
- 21. 21 Experiments with new listening experiences Exploring experiences that meet needs users can’t yet articulate
- 22. 22 2. Data helps us know how high we’ve jumped and whether to keep going
- 23. 23 The Facebook experiment: Local station stories geo-targeted on NPR’s Facebook stream
- 24. 24 Local stories saw consistently higher engagement and grew local audience
- 25. 25 Pivot and double-down: We created a workflow tool so we can scale this offering
- 26. 26 3. It’s less about achieving goals and more about continuous learning
- 27. 27 Yes, this sounds like Lean Startup thinking Assumption/hypoth esis Minimum Viable Product Analytics, research, testing
- 28. 28 Designing choice in the new NPR app Original design: Give users more control/choices Hypothesis to test: Fewer immediate choices + simplicity = longer listening Results: Listening time up 8%
- 29. 29 Usability testing (and third-party research) on the infamous hamburger nav
- 30. 30 Walking the narrow path
- 31. 31
Editor's Notes
- #3: Qualitative and quantitative User interviews, surveys, third-party research Usability testing, A/B testing, concept testing http://www.flickr.com/photos/chicagobart/4650478963/
- #6: UX and design community: fundamental fear and skepticism of research & metrics Can feel like data’s purpose is to mug design and steal creativity from us, restrict what we do http://www.flickr.com/photos/spencerbbclark/6483073463/
- #7: http://www.flickr.com/photos/kerryburnout/3826128002/
- #8: False: Research = doing what users tell you to do Driving directions, prescriptive, restrictive Lycos marketing survey on colors
- #9: Sony boombox color What users say isn’t always what they do
- #10: Or measuring only one thing Everything hinges on what you decide to measure http://www.adprint.ro/files/executions/8/356_topinterior_mag_sofa_low.jpg_900.jpg
- #11: Former Soviet Union: managers of glass plants rewarded by tons of sheet glass produced. Most plants produced glass so thick it was nearly opaque. Changed to square meters of glass. Led to glass so thin that it was easily broken. *Unintended consequences* For us: Session conversion vs longer-term loyalty effects Upsells Leads to false optimization, taking us in the wrong direction What we choose to measure becomes what we reward -> what we do, what we are http://compliance.saiglobal.com/community/know/blogs/item/4392-unintended-consequences-and-perverse-incentives
- #12: Instead of using our creative skills, it’s just about optimization and making very small adjustments Becoming machinists, tending to the robots that make decisions Cult of A/B testing Doug at Google on testing blues http://www.flickr.com/photos/seattlemunicipalarchives/4089729743/
- #13: What is it about A/B testing that makes it so easy to fall into this trap?
- #15: It’s the baseline we start from, not the end It provides context and constraints that focus us before freeing us http://www.flickr.com/photos/bfishadow/4407857237/
- #18: Opportunity: reaching new audience where they live Challenge: More competitors for “earshare” than ever before
- #19: Channel usage by segment Likelihood to consider public radio Brand perception and affinity: NPR vs station vs program BUT all this data doesn’t tell us exactly what to do
- #21: Smartphone app works with Ford Sync, voice control
- #22: App in private beta Mobile version of NPR Infinite Player Remixing public radio content, algorithms and editorial curation for a lean-back or lean-forward experience
- #23: Measure how high we've jumped (and whether it's worth continuing investment) It should inspire us to reach higher Without overly limiting our design choices http://www.flickr.com/photos/10710442@N08/8265087493/in/photolist-dAmJyv-aoYjU9-aoYk3q-LJAUR
- #25: Initially: manual process – concierge MVP
- #27: Mindset: the goal is learning There is no “finished” http://www.deviantart.com/morelikethis/artists/198956566?view_mode=2
- #28: Flips the product development model on its head (build first, then figure out what to measure)
- #30: Findability and discoverability Testing alone didn’t answer the question But it gave us insights so WE could answer it
- #31: Falling into the cult of data vs Falling into the cult of ego Without data, design is just guessing Why is it so hard for teams to walk the narrow path?
- #32: Data doesn’t give us the answers, it helps us ask better questions