PDMA Event - Creating Effective Surveys w/Self Service Tools - October 2014
3 likes565 views
The document provides a guide on creating effective surveys using self-service tools, highlighting planning, selecting appropriate tools, questionnaire design, sample sizing, testing, reporting, and improvements. Key considerations include avoiding bias, ensuring representative sample sizes, and leveraging online reporting for data analysis. It also offers a special promotion for a free trial of SurveyGizmo.
PDMA Event - Creating Effective Surveys w/Self Service Tools - October 2014
- 1. Don’t Monkey Around With Surveys Creating Effective Surveys Using Self-Service Tools October 16, 2014 Mike Pritchard Steve Boespflug
- 2. Discussion topics •How you use self-service tools •Proper planning prevents p$%^ poor performance •Things to look for in a self-service tool •Your questions – what we can cover today •Questionnaire •Sampling •Testing •Reporting •Improvements 10/16/2014 © 2014, 5 Circles Research, all rights reserved 2 Tool How you use it
- 3. Use of surveys and tools •http://5circles.pdma-oct-2014.sgizmo.com/s3/ 10/16/2014 © 2014, 5 Circles Research, all rights reserved 3
- 4. What do you do 10/16/2014 © 2014, 5 Circles Research, all rights reserved 4
- 5. Planning and doing 10/16/2014 © 2014, 5 Circles Research, all rights reserved 5
- 6. Choosing a self-service survey tool Class of tool High-end Confirmit, Qualtrics Mid-range SurveyWriter, SurveyGizmo, Survey Analytics QuestionPro LimeSurvey Low-end Add-on Constant Contact Swiftpage Google Docs 10/16/2014 © 2014, 5 Circles Research, all rights reserved 6 SurveyMonkey? What features are important to you? •Question types •Mobile support •Logic •Integration •Add enhancements yourself •Online reporting •Analysis inside tool •Export •Pricing model •Branding •Support •??
- 7. Your questions •1. What kind of research is a good fit for self-administered surveys and what kind of research is not? •2. How can I avoid bias in survey questions? •3. How large a sample size is needed to be statistically valid? •4. Can I ask open-ended questions or should all questions be yes/no, scale and multiple choice? •5. How much time should the survey be limited to in order to keep people occupied? •6. For scale questions, when is it best to use 1 to 10 versus 1 to 5? •7. What is the best way to pre-test a survey? 10/16/2014 © 2014, 5 Circles Research, all rights reserved 7
- 8. Questionnaire •Questionnaire design is Art, Science, and Experience •DON’T design in the survey tool •Use a questionnaire document approved by the team to program from –Prioritize questions based on objectives –Question types –Logic to improve data quality and survey taker experience –Flow is important –Think about accessibility, use of different devices –Minimize bias through wording and randomizing options •Numeric reporting values (coded options) rather than text 10/16/2014 © 2014, 5 Circles Research, all rights reserved 8
- 9. Sample Sizes •Sample size calculations assume representative samples. They rarely are. •Consumer studies. Sample of 400 is a good minimum standard, yielding reliability of ±5% at a confidence level of 95%. Actual minimum is 384, so 400 allows for a few rejects. •Business studies. Sample of 200 is a good minimum standard, yielding reliability of ±7% at a confidence level of 95%. Actual minimum is 196, so 200 allows for a couple of rejects. •Larger sample sizes allow: –Increased ability to discern differences among subgroups –Advanced analysis. Sample of at least 800 is good for segmentation •Smaller universes don’t reduce sample size requirements as much as we’d like. Just as bad, we need to reach a high percentage of the audience. 10/16/2014 © 2014, 5 Circles Research, all rights reserved 9 Universe Sample for ±5% / 95% Percentage of universe Sample for ±5% / 95% Percentage of universe 100,000 or more 384 N/A 196 NA 4,000 350 8.8% 187 4.7%
- 10. Acquiring sample 10/16/2014 © 2014, 5 Circles Research, all rights reserved 10
- 11. Testing the survey 1.Test with the document –Question text clear –Question options exclusive, exhaustive, “other” and “none” as appropriate –Put yourself in the mind of the survey taker, but check with people from the real target audience –Go through the logic •Test programming – all paths and boundary conditions •Use the tool to enhance testing –URL queries to show hidden values, change paths, –Capture testing comments inside the real survey •Client / stakeholders check •Soft launch –Limited number of responses –Test all the way to the data. Pause to review, including survey taker comments •Full launch 10/16/2014 © 2014, 5 Circles Research, all rights reserved 11
- 12. Reporting •Online reporting for monitoring •Online reporting for real reporting – if you can’t avoid it –How do you clean and filter data? –Clients reacting too soon –Inadequate tools for statistical testing, etc. •Export for more capable tools –SPSS format, even if you don’t use SPSS –Excel –R –Tableau 10/16/2014 © 2014, 5 Circles Research, all rights reserved 12
- 13. Improvements •Ask survey takers to rate the survey and provide comments •Review other issues you didn’t think of earlier, possibilities that came up. 10/16/2014 © 2014, 5 Circles Research, all rights reserved 13
- 14. Special Offer from SurveyGizmo •Sign-Up for a SurveyGizmo Account and Receive a 60-day Free Trial 1.Go to www.SurveyGizmo.com 2.Click on the pink button "Try Building a Survey" 3.In the pop-up window enter your email address and password. In the field titled "What's the title of your survey" enter "5Circles". This will act as a promo code giving you the 60-day trial rather than the standard 14-day free trial. 10/16/2014 © 2014, 5 Circles Research, all rights reserved 14
- 15. Questions? Mike Pritchard 425-968-3883 mikep@5circles.com www.5circles.com