What makes good research
0 likes216 views
The document outlines essential components of good research, emphasizing the importance of asking the right questions and the proper design of surveys. It provides various tips for effective question formulation, audience identification, and reporting methodologies, while also acknowledging common pitfalls in research. The document concludes by underscoring the need to clearly define key audiences and gather relevant data to inform strategic objectives.
What makes good research
- 1. What makes good research? Michele Madden
- 2. 2 Good research is…. Asking the right questions… …of the right people… … at the right time
- 3. 3 Key questions to ask What do you want to know? Why do you need this information? What decisions will you take once you have it? Have you already made the decision? Do you have the resources to implement the findings? Is this the right time?
- 4. 4 Top tips for survey and question design • Be clear in what you’re asking – if needed, provide prompts as to the time, location and scope of what you want to know. Only ask one question at a time – E.g. “Have you donated money?” • Scales need to be balanced, intuitive and match the question being asked – E.g. How much did you enjoy the concert you attended? Options: Really enjoyed it, Quite enjoyed it, Don’t know, Wouldn’t go again, Hated it • Don’t ask multiple things within one question – E.g. “Did you enjoy this concert and would you come again?” • Don’t ask questions beyond the expected scope of the respondent’s knowledge – health charities often try to do this! – “Do you approve of the NHS’s decision to not provide Ibrutinib as a standard chemotherapy treatment?” – “Which of the following is most important to you when giving to an armed forces charity?” I don’t! • Avoid jargon. If you must include it, explain but in as neutral a way as possible. Have a look here: https://www.surveymonkey.com/blog/2015/02/11/5-common-survey-mistakes-ruin-your-data/
- 5. 5 More considerations • Do not ask leading questions! – “How much do you think you have improved over the last term?” • Don’t jump around topics – consider the natural flow of the survey – Explain what’s happening to the respondent • Subtly reassure respondents it’s ok to answer negatively – “Some people feel at ease about being around and interacting with people with a disability or sensory impairment condition, while others find it very uncomfortable. Please select the statement you think would apply to you most when thinking about people with the following conditions/symptoms” • If you’ve told respondents what the survey is about, try not to ask questions that seem unrelated • Demographics can influence how you answer – have them at the end unless needed for quotas Online Surveys: • Consider the need for required questions • Progress bars work best at the bottom and without numbers!
- 6. 6 Tools • Basic counting – calls, service users, downloads etc. • Is there any data that is already being collected that you can use? Remember that not all measures need to be research based; number of unique visitors to your website, staff turnover or sickness are valid measures. • SWOT, PESTLE, Competitor mapping (audience, theme and reach) o Ad hoc and on-going • Critical friends • Existing research, o trends, stats and generic • Your own knowledge o ad hoc feedback o who impressed/disappointed you • Assessment of previous performance • One key question – include in different places
- 7. 7 Reporting • Be honest and upfront about any weaknesses in the research methodology – people more likely to believe you • Don’t overuse complex graphs and metrics, make stats and facts accessible in the narrative • Make it impactful, but not sensationalist ⁻ don’t exaggerate, take things out of context, twist the findings – comes back to bite you! ⁻ be careful with percentages and quotes ⁻ think about the participants – would they want to be represented like this? • Case studies and quotes ⁻ Ensure they are representative of the data and not the ‘strongest’ or ‘weakest’ example of someone’s feelings
- 8. 8 There is no ‘perfect’ methodology but these are the 5 things we most often see go wrong… 1. Not planning properly; research planned with no reference to internal timetables – planning, mailing, budgeting, or not involving the right people 2. Unrealistic timescales – too fast or too slow, or being led by internal deadlines “I’ve promised the trustees….” 3. Over-estimating your ability to reach the target audience; how many people will be willing to take part – ensure sufficient resources are available to recruit, skewed recruitment – need to work hard to get people to attend/take part who are representative of the wider group 4. Underestimating how long set up and analysis takes - research is not just about the focus group or questionnaire, the thinking before and after are as important, these elements always get squeezed 5. Communicating the findings badly
- 10. 10 Define key audiences Identify existing knowledge Identify gaps Gather knowledge Set baselines/ targets Identifying key audiences - process
- 11. 11 Stage 1: audience identification • Be clear whether you are doing this exercise at organisation, department or team level. • Identify your key audiences, e.g. Donors, staff, commissioners, influencers, potential supporters (women 35-55). Be as specific as possible. • Ensure that your first go at this includes all the people you can think of. Involve other people in the process, this will ensure that you are including audiences that you may not be close to or aware of. • Fill in as much information as you can about these audiences; their size, value etc. This will help you decide their relative importance. • Thinking about your strategic objectives for the next 5 years, prioritise your audiences A, B and C. Grade very small audiences as C unless they are absolutely critical to what you do
- 12. 12 General public segments Influencers MPs Journalists Donors Commission ers Campaigners Staff volunteers beneficiaries members Stage 1: audience identification
- 13. 13 1. Audience identification who are your key audiences (and why are they key?) Audience Approx no engaged with us Market size (if known) Strategic priorities Amount spent/ received Priority A,B,C Individual donors 300,000 on database 800,000 Increase income by 10% over next 2 years £425,000/ £1.23m A
- 14. 14 2. What do you know about them? Audience Current knowledge Source Gap Comments
- 15. 15 3. Identify key audience knowledge and engagement Audience:……………………………. Awareness Understanding Affinity Engagement Other What Do you want to measure? How Will you measure it? When Will you measure it? How frequently? Detail What specific question, piece of data will you use? Current Target
- 16. 16 2-6 Tenter Ground Spitalfields London E1 7NH www.nfpsynergy.net +44 (0)20 7426 8888 Michele.madden@nfpsynergy.net nfpsynergy nfpsynergy nfpsynergy Registered office: 2-6 Tenter Ground Spitalfields London E1 7NH. Registered in England No. 04387900. VAT Registration 839 8186 72
- 17. Wales networking group Audience Insight Wales Networking Group 05 February 2020 Cardiff #CCwales
- 18. Visit the CharityComms website to view slides from past events, see what events we have coming up and to check out what else we do: www.charitycomms.org.uk