Using evidence
0 likes101 views
The document emphasizes the importance of evidence-based research (EBR) in decision-making to ensure effectiveness and avoid common misconceptions. It highlights that research evidence can inform what works, where, and for whom, while cautioning against over-reliance on expert opinion, which can be biased. Ultimately, it advocates for careful selection and communication of relevant evidence to inform better practices in various fields.
Using evidence
- 2. Why we need to study this? • Having evidence to prove or refute ideas is a scientific way to promote critical thinking and progress in ideation. • Helps us ensure value of what we are doing. • Research evidence is better than expert opinion alone. • Using evidence-based research (EBR) can ensure the value of a new study. 2
- 3. Research Evidence • Can help you understand what works, where, why and for whom. • Can also tell you what doesn’t work, and you can avoid repeating the failures of others by learning from unsuccessful programs. • Evidence also challenges what we might think is common sense. • Example: Increasing the amount of police on the street to reduce crime. The evidence does not support this. More informed police patrolling the streets might make the public feel safer, but it can actually take police away from solving crimes. Source • When budgets are tight, one cannot afford to waste money on policies and ideas that don’t work. • Another example: Getting laptop and high-tech gizmos into every school may be a good headline-grabbing commitment, but it does little in itself to benefit young people’s learning. • Research by OECD has found that frequent use of computers in schools is often connected with lower academic results. Source
- 4. Throwing money at most problems will of course often do some sort of good. But could that money have been spent more effectively on something cheaper?
- 5. The importance of evidence • Whether it is a police station, a school classroom or the boardroom of a charity, evidence can help you make better decisions. • It is helpful not only in frontline service delivery, but also in creating smarter organizations – charities, local authorities, government departments etc.
- 6. What is Evidence-Based Research? 9/3/20XX Presentation Title 6 • Use of prior research in a systematic and transparent way to inform a new study so that it is answering questions that matter in a valid, efficient, and accessible manner. • A single study can very rarely (if ever) provide a definitive answer to the question investigated. • Placing a new idea, in the context of relevant previous research is key. • The risk of bias is high if only a subset of earlier studies is included in the ideation of the work.
- 7. Evidence Pyramid • Depicts the evidential strength of different research designs. • Studies with the highest internal validity, characterised by high degree of quantitative • Analysis, review, analysis, and stringent scientific methodology, are at the top of the • Pyramid. • Observational research and expert opinion • Reside at the bottom of the pyramid.
- 8. • Any model of good decision-making should be wary of putting professional judgment on a pedestal as experts can sometimes get it horribly wrong. The four elements of evidence-based management
- 9. The Nesta innovation spiral
- 10. The trouble with professional opinion • Sometimes professional judgement can fail to get it right. • A study by the American psychologist and political writer Professor Philip E. Tetlock found the predictions of most experts were wrong. • He gathered 80,000 expert predictions and compared them to what actually happened. • The results were devastating. Academics, government officials, journalists and other pundits performed worse than ‘dart-throwing monkeys’ in forecasting the future. • Indeed, those specialists who had more detailed subject knowledge seemed to perform even worse than average. • Source
- 11. Cognitive bias • Experts can sometimes carry a “heavy burden of prejudices, preconceptions and even partiality”. • As humans we are ‘predictably irrational’ and may experience upto 150 cognitive biases that distort our thinking. • Cognitive bias refers to our inability to be entirely objective, which may manifest as perceptual distortion, inaccurate judgements and illogical and/of irrational interpretations. • It means we should be highly cautious about the accuracy of expert decisions. • Cognitive bias is a tendency to see the evidence that fits with what we believe, and to ignore or discount what doesn’t.
- 13. Its not just about experts, it is about everyone who is selective in their use of research evidence. We look for evidence that fits our beliefs.
- 15. What evidence should you choose? • Not all evidence is equal. Some is stronger-and more relevant to your challenge-than others. • Think about the appropriateness of your evidence. • You need not know every type of approach. • For the non-specialist ‘consumer’ of research, its more important – and easier- to understand the assumptions that underpin these ways of doing research.
- 16. In the context of Innovation….
- 17. Judging the quality of research • Another way to help you choose which sort of research you need is to ask a different question : what research can you trust? • What is good enough evidence to fir you needs? • People often cite peer-reviewed journals as THE EVIDENCE. • In a famous paper, John Ioannidis from Stanford University caused a stir by arguing that ‘most published findings are probably false’. • He examined the most cited papers (1,000+ citations) in the best regarded medical journals in the world – largely drawn from The Lancet, the New England Journal of Medicine and the Journal of the American Medical Association. • Of those with claims of efficacy whose results had been tested, 41 per cent were either found to be wrong, or the impact was much smaller than the original study suggested. • This is not to say that peer-reviewed evidence is useless. Its vital, but keep your objectivity afloat.
- 19. The importance of repetition and corroboration • How many studies need to be included for you to be comfortable that a strong body of evidence exists? • A couple of good studies if they are good? Or may be dozens, or hundreds? • There is no magic number of studies. • But the size of the body of evidence is important: there is strength in numbers, and we must have repetition and corroboration.
- 20. Where should you look for evidence? • Established international and national organization reports and studies. Credibility is established if you choose organizations known for their rigor in scientific investigation. • Peer-reviewed studies. • Newspaper reports (Guardian) • National and International Ministry websites. • Depending on the nature of your idea/topic, see which one of the above fits. • Example: https://www.pewresearch.org/
- 21. How to communicate your findings? • Evidence rarely speaks for itself. • Involve stakeholders. • Make recommendations as specific as possible. • Make your message Easy, Attractive, Social and Timely. • Easy: simplify the message. • Attractive: use images or personalization. • Social: use anecdotes, and real people. • Timely: make your timing right, prompt people when they are likely to be most receptive.
- 23. Summary • Implement changes or advice based on evidence that is as strong as possible (and vice versa, be wary of changes or advice if the evidence is weak). • Be evidence-aware in your decision making. • Beware of potential cognitive biases in your decision making. • Think about the right type of research to suit your needs. • Type of evidence must match your question. • Be smart in how you communicate your evidence.