Applying Science to Personas: Merging Small Sample Qualitative Insights with Large Sample Quantitative Analysis
Download as PPTX, PDF5 likes932 views
This document describes a mixed methods approach to creating more scientific personas. It involves conducting qualitative interviews and observations, surveying a large sample, identifying segments using statistical clustering, determining variables that differentiate segments, predicting segment membership with a typing tool, and personifying the segments. It provides an example of creating online grocery personas for Walmart by following these steps, which identified 4 main personas from the data.
Applying Science to Personas: Merging Small Sample Qualitative Insights with Large Sample Quantitative Analysis
- 1. MeasuringU 2017 How To Make Personas More Scientific Jeff Sauro, PhD | Chelsea Meenan, PhD | Jan Moorman, MA
- 2. MeasuringU 2017 Anna 2 Profile Interests Golfing, Fishing & Camping Pets 1 Cat Children None Passions Movies, comedy and music Education Some college Profession Accounting assistant Social Profile Free thinker, smoker, drinks socially I'm a very easy going laid back person. I like to just go with the flow. Age 28
- 3. MeasuringU 2017 Lucy 3 Profile Interests Scrapbooking Pets Dog, Cat and Ferret Children Two Passions Enjoying time with friends, family and pets Education Some college Profession Mom and zoo-keeper Social Profile Self-Reliant I unwind with technology in my free time. Age 46
- 6. MeasuringU 2017 6 Do personas actually improve the development process? Do personas really help deliver better user experiences and more successful products?
- 7. MeasuringU 2017 7 How generalizable are the personas? How many personas should there be? What details should be collected and displayed? What variables differentiate the personas?
- 8. MeasuringU 2017 Mixed Methods Approach: Exploratory Sequential Design 8 Qualitative Collection & Analysis Quantitative Collection & Analysis Interpretation
- 9. MeasuringU 2017 1. Conduct qualitative interviews and observation 9
- 10. MeasuringU 2017 2. Survey a large sample of users or prospects 10
- 11. MeasuringU 2017 3. Identify the segments 11
- 12. MeasuringU 2017 4. Determine key variables that differentiate segments 12 39% 77% 85% 92% 10% 43% 76% 78% 37% 55% 73% 93% It’s a difficult to find time I enjoy grocery shopping I like to try new interesting items 0 0 0 100% 100% 100%
- 13. MeasuringU 2017 5. Predict segment membership using a typing tool 13
- 14. MeasuringU 2017 6. Personify or qualify your segments 14 Personify Go deep to get insights. Conduct qualitative interviews & observations. Qualify What variables matter? What percent of the population?
- 16. MeasuringU 2017 Goal: Align with Walmart Marketing’s US focus 26 Million ‘busy family’ households Millennials and Gen X families 2+ kids Ages: 20 to 50 65% middle class 53% with college degrees 37% multicultural (18% Hispanic)
- 17. MeasuringU 2017 Goals: Investigate our persona related hypotheses There is a distinct persona that is an ‘early adopter’ of online grocery. The ‘busy family’ marketing segment is comprised of several personas. Grocery shopping is an integral part of everyday living… grocery personas map to individuals.
- 18. MeasuringU 2017 1. Conduct qualitative interviews and observation 18 online diary study in-home visit and shopalong
- 19. MeasuringU 2017 Goal: Recruiting for diversity 19 Walmart Competitor Aspirational 3 Distinct groups
- 20. MeasuringU 2017 Overall adoption journeyIndividual Shopping Trip Personas: finding dimensions 20 Peak of Inflated Expectations Technology Trigger Trough of Disillusionment Slope of Enlightenment Plateau of Productivity 2 Analyses yielded 30+ key behaviors and attitudes
- 21. MeasuringU 2017 Personas: finding patterns (behavioral/attitude) 21 Key behavior/attitude dimensions12
- 22. MeasuringU 2017 Quant study goals: Determine relation (if any) to marketing breakdown of ‘busy family’ segment into 4 types. Verify qualitative personas, then create typing tool for future recruiting Determine prevalence of the online grocery personas
- 23. MeasuringU 2017 2. Survey a large sample of users or prospects 23 Customers/competitors/aspirational Existing Personas Deep dive into dimensions Uncover additional concepts Narrow the scope Quantify field work Generalize Develop a typing tool Dimension Reduction Population Identification Item Generation Outcomes
- 24. MeasuringU 2017 2. Survey a large sample of users or prospects Ages: 22 - 70 HH Income: Above $50,000 Open to shopping at Walmart Responsible for grocery shopping Aspirational OR currently buying groceries online with Walmart or competitors Beyond “busy family” demographics 4K completes Walmart Online Grocery Customers PRIZM segment typing available 1K completes General US Population Measuring U panel 1K completes General US Population Harris Poll panel PRIZM segment typing available 2K completes
- 25. MeasuringU 2017 3. Identify the segments Latent Class Analysis (LCA) identifies unobservable subgroups within a population 25
- 26. MeasuringU 2017 3. Identify the segments 26 Latent Class Analysis (LCA) identifies unobservable subgroups within a population Ex. What makes a good presentation? Types of Presentations Visuals Long/Short PowerPoint About UX The presenter likes hockey Manifest Variables Latent Class Variable
- 27. MeasuringU 2017 3. Identify the segments 27 What LCA can tell us How many groups of shoppers? What qualities characterize the groups? Who are the customers? Children Convenience Urban Time Value Manifest Variables Latent Class Variable
- 28. MeasuringU 2017 3. Identify the segments 28 The Data Children Convenience Urban Low Cost Time 1 2.00 1.00 2.00 1.00 1.00 2 2.00 2.00 2.00 1.00 1.00 3 2.00 1.00 1.00 1.00 2.00 4 1.00 1.00 2.00 2.00 1.00 5 2.00 2.00 1.00 2.00 2.00 6 1.00 2.00 2.00 1.00 2.00 7 2.00 1.00 2.00 2.00 1.00 8 2.00 1.00 1.00 2.00 2.00 9 1.00 2.00 2.00 2.00 1.00 10 1.00 1.00 2.00 1.00 1.00 11 1.00 1.00 1.00 1.00 2.00 12 1.00 1.00 2.00 2.00 1.00
- 29. MeasuringU 2017 3. Identify the segments 29 How many classes? Model fit and theory: try it, compare model fit, use theory
- 30. MeasuringU 2017 4. Determine variables that differentiate segments Which manifest variables should I use? • Beyond basic behaviors and demographics Analysis provides the probability that members of each class had of endorsing each dimension. Then, measure where each persona falls on additional dimensions 30 No Yes Class 1: 0.896 0.104 Class 2: 0.3836 0.6164 Class 3: 0.5293 0.4707 Class 4: 0.2554 0.7446 Attitude 1
- 31. MeasuringU 2017 5. Predict segment membership using a typing tool Classify new respondents • Create a formula with an abbreviated set of questions 31
- 32. MeasuringU 2017 6. Qualified 32 Initial Analysis Based on full set of responses 4 main personas contained significant differences in key characteristics to qual personas. • One additional persona emerged (urban, unmarried male) Based on ‘Busy Family’ demographics → apples to apples comparison
- 33. MeasuringU 2017 6. Qualified Correction to qualitative strength of a dimensional characteristic. Survey question inability to capture nuances perceived in the field research. Mixed method - disparity between a research observation and survey self evaluation. 33 Focused Analysis Showed strong correlation with qualitative findings – yet we found 3 types of differences:
- 35. MeasuringU 2017 35 Becky (Class A, 22%) Profile Key driver Enjoys grocery shopping Likes to buy products from lots of different stores Ideal online grocery Willing to pay more for convenience Will spend a little more for something unusual Cares about automatically adding items from previous orders Benefits of online grocery Convenience I have more time to do other things Avoiding the physical store Enjoyment of grocery shopping 76% say they enjoy grocery shopping 19% say one reason they hesitated before signing up for online grocery is that they enjoy going to the store themselves Effort to save on groceries Believes that saving money on groceries is important but exerts lowest effort of all 4 clusters. Low minimum order amount isn’t important for online groceries Effort towards meal planning Is focused on making weeknight dinners less stressful (36%) Depends on what she has in the kitchen (29%) Effort in making shopping list Makes a written list I'm a very easy going laid back person. I like to just go with the flow. Age 28 69% 31%
- 36. MeasuringU 2017 6 Components To Make Personas More Scientific 36 1.Conduct qualitative interviews and observations. 2.Survey a large sample of users and/or prospects. 3.Identify segments using a statistical clustering technique. 4.Determine key variables that differentiate segments. 5.Predict segment membership using a typing tool. 6.Personify or qualify your segments.
- 37. MeasuringU 2017 About MeasuringU MeasuringU is a quantitative research firm based in Denver, Colorado focusing on quantifying the user experience. Remote UX Testing Platform (Desktop & Mobile) UX Research Measurement & Statistical Analysis Eye Tracking & Lab Based Testing UX Boot Camp Aug 16th-18th denverux.com MeasuringU.com @MeasuringU
Editor's Notes
- #6: Personas are a popular research method. Their concept has been around for a while and popularized when applied to UX research design by Alan Cooper in the Inmates are Running the Asylum. I remember when cooper’s book came out 1999, it was the “Design Thinking” ho topic of the .com rise and fall. But unlike the .dot coms, the concepts has remained. As of this year, some 70% of UX researchers report using them, a figure that’s remained constant or growing for the last decade. But while their popularity remains, so do lingering questions about their validity. Common questions and concerns include.
- #7: The Persona Lifecycle: Keeping People in Mind Throughout Product Design recommends brining life-sized cutouts to meetings and also here https://www.snapapp.com/blog/best-practices-building-buyer-personas-experts
- #8: The Persona Lifecycle: Keeping People in Mind Throughout Product Design recommends brining life-sized cutouts to meetings and also here https://www.snapapp.com/blog/best-practices-building-buyer-personas-experts
- #9: Mixed methods image here Our approach to personas is to use a mixed-methods approach that leverages the techniques from a segmentation analysis <link> while maintaining the rich qualitative details that a traditional persona provides. This approach provides the best of both worlds: rich detailed information that describes statistically reliable clusters that are also generalizable. It has six components:
- #10: Through careful observation and inquiry, identify challenges, problems, attitudes, and behaviors that characterize and differentiate groups of users. The number of participants you interview and observe is a function of how common the behaviors and attitudes you observe are. Here is a primer on conducting qualitative research. For the grocery study, the team had already conducted extensive interviews and observations and synthesized the findings into four personas they wanted to validate.
- #14: Using multiple
- #17: Tony Rogers, chief marketing officer for Walmart U.S. Marketing has divided shoppers into four categories: busy families, active savers, older unconnected and urban connected. Busy Family group wields an annual retail spend of $600 billion, or one-third of the industry’s total $1.8 trillion expected this year. He said busy families register the highest with Walmart’s core mission to save people time and money. A desire to be able to compare apples to apples with personas and marketing target market – what are the personas in this segment, and do these hold true in the other segments (which is what we would assume).
- #18: Understanding unmet needs
- #20: We interviewed 24 individuals: 12 were what we termed aspirational online grocery customers – those likely to try in the next 3-6 months and 12 current online grocery customers (of any online grocery retailer).
- #21: Cooper style persona process: visual cluster analysis along key behavioral and aspirational attribute dimensions. Initial assumptions about dimensions that would prove useful did not pan out. We were stuck, until . . . Identified a way to systematically discover meaninful dimensions to use by shopping journey analyses: We looked at two visualizations of their journey – along the adoption curve and then individual shopping trips At each step in the journey we asked the 3 empathy mapping questions: What is the person doing? What are they thinking? How are they feeling?
- #22: Cooper style persona process: visual cluster analysis along key behavioral and aspirational attribute dimensions. Initial assumptions about dimensions that would prove useful did not pan out. We were stuck, until . . . Identified a way to systematically discover meaninful dimensions to use by shopping journey analyses: We looked at two visualizations of their journey – along the adoption curve and then individual shopping trips At each step in the journey we asked the 3 empathy mapping questions: What is the person doing? What are they thinking? How are they feeling?
- #23: Personas are for designers, when you add the quant it’s for business as well – if our hypothesis about early adopters is correct then where are we leaving $ on the table by not understanding and addressing the needs of non-early adopters.
- #24: Item generation: Chelsea and Jan discussing how we came up with the questions in survey Using current customers, prospects and multiple panel sources Range of customers (validating existing segmentation) Balancing existing stakeholder interests Dimension reduction Item Generation (nuances) N > 4000 Survey took about 13 minutes to complete, around 50 questions *Personas are based on participants aged 22-40, who make decisions with a partner/spouse, and have children (n = 577) Goals Focused on replicating field work and identifying other relevant dimensions to understand the customers
- #25: Using current customers, prospects and multiple panel sources N > 4000 *Personas are based on participants aged 22-40, who make decisions with a partner/spouse, and have children (n = 577) Note to MU: I think it is relevant to talk about the different populations – and how those surveyed differed (broader) than the ‘busy family’ demographics. Explanation – they included some demographics that were not included in the study: urban, older, unmarried, no kids.
- #26: http://keltonglobal.com/blog-post/facing-reality-when-it-comes-to-typing-segments/ Based on patterns of Categorical data
- #27: “latent” because it cannot be directly observed. Ex. Interesting, dynamic, impactful, etc. We’re looking for common patterns among the members of a class Unlikely to cluster meaningfully with other variables: Do the presenters like cat videos
- #28: Organize customers into classes To select the number of classes for the model, specify and run a 2-class model and repeat with 3 classes, 4 classes..., up to the highest plausible number of classes. From the results, information about fit (including log likelihood, degrees of freedom, G2, AIC, BIC, CAIC, etc.) are compared to identify the optimal model. Also, the bootstrap likelihood ratio test can be used to compare models. In LCA, the responses of all participants to all items are analyzed. A specified latent class model is fit to the data, and the parameter estimates are obtained. Once the number of classes is selected, the output includes the probability of a response to EACH grocery shopping behavior item in the inventory for each latent class. In other words, you will see the probability that members of each class had of engaging in each grocery shopping behavior.
- #29: To select the number of classes for the model, specify and run a 2-class model and repeat with 3 classes, 4 classes..., up to the highest plausible number of classes. From the results, information about fit (including log likelihood, degrees of freedom, G2, AIC, BIC, CAIC, etc.) are compared to identify the optimal model. Also, the bootstrap likelihood ratio test can be used to compare models. In LCA, the responses of all participants to all items are analyzed. A specified latent class model is fit to the data, and the parameter estimates are obtained. Once the number of classes is selected, the output includes the probability of a response to EACH grocery shopping behavior item in the inventory for each latent class. In other words, you will see the probability that members of each class had of engaging in each grocery shopping behavior.
- #30: PARSIMONY INDICES (AIC, BIC) To select the number of classes for the model, specify and run a 2-class model and repeat with 3 classes, 4 classes..., up to the highest plausible number of classes. From the results, information about fit (including log likelihood, degrees of freedom, G2, AIC, BIC, CAIC, etc.) are compared to identify the optimal model. Also, the bootstrap likelihood ratio test can be used to compare models. In LCA, the responses of all participants to all items are analyzed. A specified latent class model is fit to the data, and the parameter estimates are obtained. Once the number of classes is selected, the output includes the probability of a response to EACH grocery shopping behavior item in the inventory for each latent class. In other words, you will see the probability that members of each class had of engaging in each grocery shopping behavior. The Vuong-Lo-Mendell-Rubin test has a p-value of .1457 and the Lo-Mendell-Rubin adjusted LRT test has a p-value of .1500.
- #31: Nuances that make personas come to life can make it difficult to develop an accurate typing tool. Martin Eichholz http://keltonglobal.com/blog-post/facing-reality-when-it-comes-to-typing-segments/ In LCA, the responses of all participants to all items are analyzed. A specified latent class model is fit to the data, and the parameter estimates are obtained. Once the number of classes is selected, the output includes the probability of a response to EACH grocery shopping behavior item in the inventory for each latent class. In other words, you will see the probability that members of each class had of engaging in each grocery shopping behavior. Theory and probabilities From here, you get the likelihood of membership in each segment, and can place members into the most likely class. BUT what if you want to classify new customers? Self-reported behaviors, attitudes, and demographics
- #32: Chelsea: Easy part now that you have weights from an existing sample, you “type” new samples into personas Predict a categorical variable Minimum number of dimensions to describe differences between groups (UCLA) Run Stepwise to see which variables are the most important. Then compare the full model with just the ones you want to retain. For example 18 down to 5 and compare the percentage predicted correctly (e.g. 80% vs, 50%). You’ll get coefficients for the formula identifying the likelihood of belonging to each class. The highest likelihood is the class the participant belongs to. 80% of original grouped cases correctly classified
- #33: JAN Example of rich personas Initial analysis was done on the full set of responses. There were significant differences in key characteristics, so we requested reanalysis based on demographics use for the persona study (age range, families with kids still at home). (VENN) Second analysis showed strong correlation with qualitative findings – yet there were differences. Discussed with MU and found 3 types of differences between quant and qual findings: Quantitative result corrects the qualitative strength of a dimensional characteristic. Reexamining the qual rationale, able to find evidence to support quant results. Quantitative results differ, but believe survey question lacked the nuance perceived in the field research and used to set the relative strength. Quantitative results differ from qualitative. We believe the difference between are simply the disparity between an outsiders evaluation and the survey self evaluation. Both are important. The qual results show relative positioning with the other personas, but the survey self-evaluation must be part story about who this persona is.
- #34: JAN Quantitative result corrects the qualitative strength of a dimensional characteristic. Reexamining the qual rationale, able to find evidence to support quant results. Quantitative results differ, but believe survey question lacked the nuance perceived in the field research and used to set the relative strength. Quantitative results differ from qualitative. We believe the difference between are simply the disparity between an outsiders evaluation and the survey self evaluation. Both are important. The qual results show relative positioning with the other personas, but the survey self-evaluation must be part story about who this persona is.
- #35: JAN
- #36: 22% of the subsample on which the clusters are based 21% of the entire sample