Data-Driven Design for User Experience
2 likes437 views
The document discusses the importance of data-driven design in user experience, highlighting the integration of hypotheses, causal data, and decision-making. It covers the significance of understanding data ecosystems, research methods, and the role of both big and small data in UX design. Additionally, it addresses the challenges of decision fatigue and the need for tools and methods for effective user analytics and testing.
Data-Driven Design for User Experience
- 1. Emi Kwon UX Lead @MetLife Japan Why Data-Driven Design for User Experience?
- 2. Why Data-Driven Design for User Experience? Data-Driven Design for User Experience
- 3. Data-Driven Design for User Experience The decision fatigue What if I was wearing a face mask? What if I was not wearing a face mask? What if________? We make some ### decisions every day …and a lot of it is based on ### hypotheses
- 4. Hypothesis >> Causal data >> Optimized decision (design!) (a) Hypothesis + (b) causal data + (c) decision = change with an impact (with less decision fatigue)
- 5. Data-Driven Design for User Experience Data ecosystem — Data lake, data warehouse…what does it mean for UX? Small data and big data — The opportunities and pitfalls Research method basics — Qualitative, quantitative or triangulated User analytics and A/B testing What about COVID-19 and remote usability testing? Q&A and Discussion Contents
- 6. Unstructured data Semi-structured data Structured data Data that has no inherent structure and is usually stored as different types of files E.g. Text documents, images, videos Textual data files with an apparent pattern, enabling analysis E.g. JSON and XML files Data having a defined data model, format and structure E.g. Database Data structure Data-Driven Design for User Experience
- 7. Data lake vs data warehouse Data-Driven Design for User Experience
- 8. Data lake vs data warehouse + data mart Data-Driven Design for User Experience
- 9. Not Customer Facing Customer Facing Sales App 1 Sales Engagement tool TTL admin tool Sales App 2 New Wave pilot ATS Post-Transaction tool Sales Website New Wave for Customer Transaction App Customer App2 Smart Simulator Marketing Website Video Chat Chatbot Customer Micro-site Customer App 1 Sales Customer Sales Admin New Wave App New Wave Web Pilot Existing Proposed Active Inactive SimplePath PIC Kim PIC John PIC James PIC Wendy PIC Jane PIC Mark PIC Bryan PIC Jane PIC Jack PIC Henry PIC Tom PIC Anthony PIC TBC PIC Wendy Mapping out your data ecosystem can be like… Data-Driven Design for User Experience PIC TBC PIC Henry PIC Jack PIC TBC PIC Annie PIC TBC
- 10. Gender Age Group Per Page Use Time Per Page # Logins # Login Errors # Registrat’n # Reg Completed # Reg Bailed Avg Reg Time Sales App 1 Now: ✖ Future: ⃝ Now: ✖ Future: ⃝ ⃝ ⃝ ⃝ ⃝ ⃝ ⃝ ⃝ ＊ Sale App 2 △ △ ⃝ ⃝ ⃝ ⃝ ✖ ✖ ✖ ✖ Sales App 3 △ △ ⃝ ⃝ ⃝ ⃝ ? ? ? ? SmartPath ? ? ⃝ ⃝ ? ? ✖ ✖ ✖ ✖ NewWave App 1 ? ? ⃝ ⃝ ? ? ✖ ✖ ✖ ✖ NewWave App 2 △ △ ⃝ ⃝ ? ? ? ? ? ? NewWave Web 1 △ △ ⃝ ⃝ ✖ ✖ ✖ ✖ ✖ ✖ NewWave Web 2 ? ? ⃝ ⃝ ✖ ✖ ✖ ✖ ✖ ✖ Admin tool 1 ✖ ✖ ⃝ ⃝ ✖ ✖ ✖ ✖ ✖ ✖ Admin tool 2 ✖ ✖ ⃝ ⃝ ✖ ✖ ✖ ✖ ✖ ✖ △ : should, but no defined requirement as yet ＊ : for individual pages or for registration as a whole? Mapping out your data attributes can be like… Data-Driven Design for User Experience
- 11. Q: What do you want to get from doing data-driven design? ✨ Data-Driven Design for User Experience 1) To understand my users better 2) Data is the new oil; no way I can do design without being data-driven 3) To add something data-driven to my portfolio/ work experience 4) Not sure yet; will see what I can learn and go from there 5) Others (Feel free to write your answer in the chat 🙂)
- 12. Data-Driven Design for User Experience Data ecosystem — Data lake, data warehouse…what does it mean for UX? Small data and big data — The opportunities and pitfalls Research method basics — Qualitative, quantitative or triangulated User analytics and A/B testing What about COVID-19 and remote usability testing? Q&A and Discussion Contents
- 13. Data-Driven Design for User Experience The data fatigue.. So much data is generated. So much data is recorded. So much data is accessible. In this data lies many many stories…and among them, many many stories of users… Analysis paralysis..
- 14. Data-Driven Design for User Experience https://www.youtube.com/watch?v=TzxmjbL-i4Y Big Data!!!
- 15. Goals, Location, Data structure, Data preparation, Longevity, Reproducibility… Volume, Velocity, Variety + Value and Veracity 3 ways big data is different from small data Data-Driven Design for User Experience
- 16. Big data and small data… what does it mean for UX design? Data-Driven Design for User Experience https://www.kaushik.net/avinash/best-web-analytics-tools-quantitative-qualitative/ Call center log Call center log , Sales and revenue data
- 17. Data vs Insight Data-Driven Design for User Experience Insight Organized, structured, categorized, useful, condensed, calculated Individual facts, figures, signals, measurements + Context + Meaning + Perspective Data Information Knowledge Understanding, integration, applied, reflected upon, actionable, accumulated, principles, patterns, decision-making process Idea, learning, notion, concept, synthesized, compared, thought- out, discussed Decision risk Value Volume, velocity & variety
- 18. Data lake >> Data warehouse >> > UX Data mart >> Data-driven design decisions The way we (would like to) fertilize our design…
- 19. Call center log Call center log What if only big data exists? Data-Driven Design for User Experience Usage data Only big data ? Call center log
- 20. Call center log Call center log Data-Driven Design for User Experience Behavioral insights Only small data ? What if only small data exists?
- 21. What if only small or big data exists? Data-Driven Design for User Experience You believe you know what users want… Without knowing what their critical contexts are….
- 22. Data-Driven Design for User Experience #multiplicity “The quest for a "single source of the truth" on the web is futile… …the quest for a single tool/source to answer all your questions will ensure that your business will end up in a ditch, and additionally ensure that your career (from the Analyst to the web CMO) will be short-lived… | https://www.kaushik.net/avinash/big-data-imperative-driving-big-action/
- 23. Multiplicity of data leads to good perspective-taking Data-Driven Design for User Experience You know the real risk lies in somewhere not so visible… So you will continue to look out for more evidence
- 24. Q: Is big data important in doing your work (design, production, development…)? ✨ Data-Driven Design for User Experience 1) A lot; using big data can make or break a deal in my line of work 2) To some extent; it’s good to have big data around 3) Hm…. might be, but no specific use case yet 4) I don’t care for big data 5) Never a deal-breaker
- 25. Data-Driven Design for User Experience Data ecosystem — Data lake, data warehouse…what does it mean for UX? Big data and small data — The opportunities and pitfalls Research method basics — Qualitative, quantitative or triangulated User analytics and A/B testing What about COVID-19 and remote usability testing? Q&A and Discussion Contents
- 26. 26 User research & test methods Data-Driven Design for User Experience Why What
- 27. Data-Driven Design for User Experience Qualitative research_Interview flow • Sampling & Participant recruiting • Drafting questionnaire • Data collection • Data analysis • Survey report • Cross-functional action plan • Development • Conduct interview • Team de-brief
- 28. Qualitative analysis tools Data-Driven Design for User Experience Tag and organize your interview notes https://www.aureliuslab.com/ https://evernote.com/
- 29. 29 Data-Driven Design for User Experience Qualitative analysis_Thematic analysis & Word cloud mapping
- 30. • Samping & Participant recruiting • Drafting questionnaire • Data collection • Data analysis • Survey report • Report share-out • Building survey form • Distribution of the survey 30 • Cross-functional action plan • Development Survey 2.0 https://www.qualtrics.com/ Quantitative research_Survey flow Data-Driven Design for User Experience https://www.jotform.com/table-templates/rainbow-sheet Survey 1.0 https://www.surveymonkey.com/
- 31. 31 Data-Driven Design for User Experience Quantitative analysis_Describe, Relate, Regression, PivotTable https://www.qualtrics.com/support/stats-iq/analyses/describe-data/
- 32. 32 Quantitative analysis tools Data-Driven Design for User Experience
- 33. Data-Driven Design for User Experience Data ecosystem — Data lake, data warehouse…what does it mean for UX? Small data and big data — The opportunities and pitfalls Research method basics — Qualitative, quantitative or triangulated User analytics and A/B testing What about COVID-19 and remote usability testing? Q&A and Discussion Contents
- 34. User analytics…what about it? Data-Driven Design for User Experience Dimension • Content information • Conversion funnels • Navigation paths • Visitor details Strategy • Acquisition • Adoption • Engagement Metrics • # of users and sessions • Average session duration • Average pages per session • New/ Returning visitors • Bounce Rate • … 34 Dimensions Metrics Flow visualization Adobe Analytics dashboard
- 35. User analytics…what can they do? Data-Driven Design for User Experience • Real-time events • Heatmaps • Session replay videos • A/B testing
- 36. Data-Driven Design for User Experience Building it together and socializing your UX data
- 37. A/B testing Data-Driven Design for User Experience What do to • Getting a testing tool • Define your UX success metrics • Define hypothesis to be tested • Define variables and learning • Create the variables • Measure and activate • Analyze the results • Document learning & Evangelize • Ideate & Repeat https://www.optimizely.com/ https://www.convertize.com/ab-test-significance/ https://business.adobe.com/products/target/adobe-target.html Control Treatment
- 38. https://www.optimizely.com/insights/blog/how-obama-raised-60-million-by-running-a-simple-experiment/ How Obama raised $60 million by running a simple A/B test 6 Media + 4 Button = 24 combinations.. 40.6% lift in sign-up rate
- 39. Data-Driven Design for User Experience Numbers can inform you of what is happening, but they won’t tell you why something is happening. User analytics and A/B testing_The pitfalls #multiplicity Now you need to dig deeper with qualitative data.
- 40. Data-Driven Design for User Experience Data ecosystem — Data lake, data warehouse…what does it mean for UX? Small data and big data — The opportunities and pitfalls Research method basics — Qualitative, quantitative or triangulated User analytics and A/B testing What about COVID-19 and remote usability testing? Q&A and Discussion Contents The why
- 41. What about COVID-19 and remote usability testing? https://bootcamp.uxdesign.cc/remote-usability-test-heres-all-the-tools-you-need-403717336552
- 42. Data-Driven Design for User Experience Remote usability testing • Sampling & Participant recruiting • Drafting questionnaire • Data collection • Data analysis • Survey report • Building survey form • Distribution of the survey • Cross-functional action plan • Development • Conduct interview • Team de-brief
- 43. Mon Tue 8/10 8/17 8/24 8/31 9/7 9/14 8/11 8/18 8/25 9/1 9/8 9/15 8/12 8/19 8/26 9/2 Emi OOO 9/9 9/16 8/13 8/20 8/27 Emi OOO 9/3 9/10 9/17 8/14 8/21 8/28 9/4 9/11 9/18 Tue Wed Thu Fri Test scenario (E) Eye-tracking hands-on (S) Eye-tracking concept study (W) Eye-tracking test (E, J, H) Data-driven UX monthly call w/ Biz, Data Science Test scenario (E, H) Test mockup (H) & Compliance Form (E) Executive summary (E, M) Test analysis BA sharing (J) Data collection (H) Compliance Form (E) User test (E, W) Data collection (E, W) Thematic analysis (E) Test analysis BA sharing TBC (E) Demo drill (E,S) Projects overview Sales App 1 (Emi, Tom) An app description is an app store optimized product definition. It greatly influences your product's success. There are three components of an app's 'definition': its Name, its Description in the marketplace, and Screenshots. Sales App 2 (James, Hannah) An app description is an app store optimized product definition. It greatly influences your product's success. Continuous touchpoint: whenever you need, you can get it done, fast and easy. NewWave Website pilot (Michael, Wendy) An app description is an app store optimized product definition. It greatly influences your product's success. Continuous touchpoint: whenever you need, you can get it done, fast and easy. Test scenario TBC Eye-tracking demo w/ DevOps Data-driven design calendar Data-Driven Design for User Experience
- 44. But don’t be a user test ninja… Every day A/B test with 6 design variants plus… Try all these methods this and next month… No, you can’t! It will just show your lack of confidence! https://en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect
- 45. Dunning-Kruger Effect Time will come when you hit this mark… Every day A/B test with 6 design variants plus… ry all these methods this and next month… https://en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect
- 46. Hypothesis >> Causal data >> Optimized decision (design!) (a) Hypothesis + (b) causal data + (c) decision = change with an impact (with less decision fatigue)
- 47. Data-Driven Design for User Experience Make informed decisions with causal data What if I was wearing a face mask? What if I was not wearing a face mask? What if________? We make some ### decisions every day …and a lot of it is based on ### hypotheses
- 48. Data-Driven Design for User Experience Q&A and Discussion Data-Driven Design for User Experience