Survey analysis in a nutshell with Jeffrey Henning
1 like514 views
Jeffrey Henning of Researchscape International presented on survey analysis and best practices for analyzing survey data. He discussed the importance of considering analysis before designing and administering a survey to ensure the right questions are asked to answer the main research question. He also emphasized reviewing each question individually and looking at relationships between questions before analyzing responses. Finally, Henning stressed telling stories with the data rather than just presenting analysis, and leaving the data behind to focus on the insights and meaning.
Survey analysis in a nutshell with Jeffrey Henning
- 1. Please tweet! @jhenning @LoveStats #mrx Survey Analysis in a Nutshell (A Peanut Shell) Presented by Jeffrey Henning President at Researchscape International Moderated by Annie Pettit Chief Research Officer at Peanut Labs
- 2. Please tweet! @jhenning @LoveStats #mrx Our Agenda Thinking about analysis once the results have come in is too late! Before Analysis Start looking at each question in isolation, in detail. It’s better together! Group some questions together, and contrast questions, too. By Each Question InterrelationshipsCBA
- 3. Please tweet! @jhenning @LoveStats #mrx BEFORE ANALYSIS
- 4. Please tweet! @jhenning @LoveStats #mrx Before You Analyze Before Completing the questionnaire design, make sure you are answering the big question. And adding the little questions you’ll need. BC1
- 5. Please tweet! @jhenning @LoveStats #mrx What’s the new price? Will they buy it? Are they happy? THEBIGQUESTION What do we do next?
- 6. Please tweet! @jhenning @LoveStats #mrx True Stories... “What media market do they live in?” “Maybe we can estimate it from their area code, assuming they haven’t moved.” “Can you crosstab by county economic tier?” “I guess we can lookup their county from their zip code, and the tier from their county...” “Let’s analyze the results by the annual revenue of the companies the respondents work for.” “We didn’t ask that.” THElittleQUESTIONS
- 7. Please tweet! @jhenning @LoveStats #mrx Before You Analyze Before Completing the questionnaire design, make sure you are answering the big question. And adding the little questions you’ll need. BC Before Counting Each response, make sure you want to include each response. Clean your data! BCE21
- 8. Please tweet! @jhenning @LoveStats #mrx Before You Analyze Before Completing the questionnaire design, make sure you are answering the big question. And adding the little questions you’ll need. BC Before Counting Each response, make sure you want to include each response. Clean your data! After Data is in hand, think of the big picture. Plan your timeline accordingly. BCE AD321
- 9. Please tweet! @jhenning @LoveStats #mrx Love the Data but Leave the Data Behind Data Price = $2.99 Analysis An Answer
- 10. Please tweet! @jhenning @LoveStats #mrx BY EACH QUESTION
- 11. Please tweet! @jhenning @LoveStats #mrx Please rate your overall satisfaction with each of the following: Sample Size: 206 (71% of Respondents)
- 12. Please tweet! @jhenning @LoveStats #mrx “Top 2 Box” Sums Top 2 Values Sample Size: 206 (71% of Respondents)
- 13. Please tweet! @jhenning @LoveStats #mrx Means are Mean Median Corrects for Skew without Requiring Judgment Calls $4.69 $0.99 $2.00 $99.00 Hedges and Shah BMC Bioinformatics 2003 4:31 doi:10.1186/1471-2105-4-31
- 14. Please tweet! @jhenning @LoveStats #mrx Why do you install new apps on your mobile device? Sample Size: 38 (4% of Respondents)
- 15. Please tweet! @jhenning @LoveStats #mrx Why do you install new apps on your mobile device? • "I install new apps when I need to be entertained or when I need a tool that I don't currently have handy such as a calculator or ruler." • "I always want to be the first to try great apps that I will want to keep. Many of the new apps that I solve help me solve a problem and I can convey this to my friends, which makes me look good.” • "Most of the time it's for entertainment purposes. I like having several app options in case I am bored. Other times, it's because I need a specific app for something (map, flight finder, photo editor, etc.)" • "To be able to do things more efficiently, or to try and discover new games, or to use apps with more functions and a better interface." • "I usually download apps that add functionality to my phone, such as a flashlight app or a sensor reading app. I don't download many game apps, and I almost never download an app developed for a single business, like a banking app.”
- 16. Please tweet! @jhenning @LoveStats #mrx INTERRELATIONSHIPS
- 17. Please tweet! @jhenning @LoveStats #mrx Cross-tabulate Questions Traditional Cross-tabs Draw Eye Away from Important Cells
- 18. Please tweet! @jhenning @LoveStats #mrx Use Cell Comparisons When Possible 86% of iPad Users vs. 69% of Android Phone Users Bought App in Past Month or Less
- 19. Please tweet! @jhenning @LoveStats #mrx Average Related Questions into Indices Customer Experience Index, Price Comparison Index, Customer Satisfaction Index, and a Customer Loyalty Index CUSTOMER EXPERIENCE Effectiveness Enjoyability Ease PRICE COMPARISONS To Expectations To Quality To Competitors CUSTOMER LOYALTY Repurchase Likelihood Reluctance to Switch Recommend Likelihood CUSTOMER SATISFACTION Satisfaction Expectations Comparison with Ideal Source: Researchscape International
- 20. Please tweet! @jhenning @LoveStats #mrx Indices Highly Correlate to Loyalty Behavior Rank CESL Metric Correlation Shared Variance 1 Customer Loyalty Index 0.762 58% 2 CSAT/Loyalty Index 0.751 56% 3 Champion Model 0.749 56% 4 Likelihood to Repurchase 0.730 53% 5 Likelihood to Recommend 0.728 53% 6 Index of Indices 0.685 47% 7 CSAT 0.639 41% 8 Apostle Model 0.604 36% 9 TNS Loyalty Model 0.590 35% 10 Reluctance to Switch 0.585 34% 18 NPS 0.502 25% 22 Ease 0.402 16%
- 21. Please tweet! @jhenning @LoveStats #mrx Apostle Model Contrast Questions with Quadrant Analyses • Jones and Sasser pioneered their own loyalty segmentation • For satisfaction scale, use CSAT question or ACSI’s 3 questions • For loyalty, use likelihood to repurchase or a loyalty index
- 22. Please tweet! @jhenning @LoveStats #mrx Apostle Model Contrast Questions with Quadrant Analyses • Jones and Sasser pioneered their own loyalty segmentation • For satisfaction scale, use CSAT question or ACSI’s 3 questions • For loyalty, use likelihood to repurchase or a loyalty index
- 23. Please tweet! @jhenning @LoveStats #mrx Beyond Quadrant Analysis Tells a Story 53eig.ht/1JkdRVi
- 24. Please tweet! @jhenning @LoveStats #mrx “Stories are just data with a soul.” Story Synthesis Analysis Data - Brene Brown, TEDx Houston, 2010
- 25. Please tweet! @jhenning @LoveStats #mrx Summary Thinking about analysis once the results have come in is too late! Before Analysis Start looking at each question in isolation, in detail. It’s better together! Group some questions together, and contrast questions. By Each Question InterrelationshipsCBA
- 26. Please tweet! @jhenning @LoveStats #mrx Q&A: Contact Information Annie Pettit Chief Research Officer annie@peanutlabs.com ca.linkedin.com/in/AnniePettit/ twitter.com/LoveStats facebook.com/AnniePettit Jonathan Cheriff Director of Sales & Marketing jonathan.cheriff@peanutlabs.com Find PeanutLabs on LinkedIn Facebook Twitter YouTube DIY POLLINGDIY SUPPORT Jeffrey Henning President jhenning@researchscape.com www.researchscape.com twitter.com/jhenning DIY SAMPLE