How to write a 4* impact case study
Download as PPTX, PDF0 likes752 views
The document outlines guidelines for writing a 4-star impact case study, emphasizing the need for underpinning research of at least 2-star quality published after 2000 and impacts demonstrated between August 2013 and July 2020. It stresses the importance of evidence-based testimonials and clear connections between research and impacts, highlighting differences between high and low-scoring case studies. Moreover, it provides insights into qualitative and quantitative aspects that contribute to obtaining a high score in evaluations, particularly in the context of REF2021.
How to write a 4* impact case study
- 1. HOW TO WRITE A 4* IMPACT CASE STUDY
- 2. 1. Case study fields remain, but with metadata and 5 pages 2. Underpinning research must be 2* quality and published >2000 3. Impacts must occur between Aug 2013 and July 2020 4. Minimum 2 case studies up to 20 FTE, then 1 per 15 FTE 5. Impacts from a body of research (e.g. collaborative project) must demonstrate substantive contribution from institution 6. Public engagement is a pathway, not an impact 7. Underpinning research 2* as a body with quality justification 8. All evidence submitted with case studies and independently verifiable (testimonials should be based on evidence, not opinion) 9. Continuation case studies have 1) no significant new underpinning research, AND 2) similar impacts and beneficiaries to those in 2014 10. A researcher’s outputs & impact can be submitted to different UoAs 11. Panel A will treat qualitative evidence and continuation case studies without prejudice (despite consultation showing disciplinary biases) Impact in REF2021: a summary
- 3. Has the bar been raised? Will it be harder to get 4* in REF2021? •Definitely •Probably •Probably not •A 4* case study in 2014 should be equivalent to a 4* case study in 2021 The REF2021 bar Vote now
- 4. •Investment in evidence: •Longer-term (sometimes longitudinal) evidence •Cause and effect relationships proven across causal chains •Evidence-based testimonials (but also testimonial carnage) The REF2021 bar
- 5. Before you start: • Ethics approval • Informed consent Evidence-based testimonials Format: • Part catch up and offer of further help • Part evaluation 1. Significance 2. Reach 3. Attribution 4. What could we have done better? Transcribe interview and draft testimonial
- 6. 1. Case study fields remain, but with metadata and 5 pages 2. Underpinning research must be 2* quality and published >2000 3. Impacts must occur between Aug 2013 and July 2020 4. Minimum 2 case studies up to 20 FTE, then 1 per 15 FTE 5. Impacts from a body of research (e.g. collaborative project) must demonstrate substantive contribution from institution 6. Public engagement is a pathway, not an impact 7. Underpinning research 2* as a body with quality justification 8. All evidence submitted with case studies and independently verifiable (testimonials should be based on evidence, not opinion) 9. Continuation case studies have 1) no significant new underpinning research, AND 2) similar impacts and beneficiaries to those in 2014 10. A researcher’s outputs & impact can be submitted to different UoAs 11. Panel A will treat qualitative evidence and continuation case studies without prejudice (despite consultation showing disciplinary biases) Impact in REF2021 Open mic
- 7. What made a 4*case study in REF2014? Based on PhD research by Bella Reichard @BellaReichard based on quantitative analysis of 217 and qualitative analysis of 180 of the highest and lowest scoring cases, spread across Panels A, B, C and D
- 9. Quantitative linguistic analysis 1. Highly-rated case studies provided specific, high-magnitude and well-evidenced articulations of significance and reach • 84% of high-scoring cases articulated significant and far-reaching benefits, compared to 32% of low-scoring cases, which typically focused on pathway Phrases more common in high- scoring: • Significance and reach (specific and high): in England and, in the US, the UK’s, millions of, long-term, the government’s, the department of, the House of Commons, for the first time, prime minister, select committee Phrases more common in low-scoring: • Significance and reach (non-specific or low): a number of, a range, nationally and internationally, in local, of local, the north, city council, policy and practice, an impact on, impact on the, the impact • Pathways to impact: has been disseminated, disseminated through, dissemination of, been disseminated, and workshops, the event, the book • Beneficiaries (not benefits): and community, practitioners and, group of, members of the Qualitative analysis
- 10. • 97% of high-scoring cases clearly linked the underpinning research to claimed impacts, compared to 50% of low-scoring case studies • 42% high-scoring policy cases described policy and implementation, compared to 17% in low Phrases more common in high-scoring • Attribution between research and impact: cited in the, (was) used to inform, to improve the, led to the, resulting in, showing that, was subsequently, produced by, reported in, evidence for, cited in, led by 2. Highly-rated case studies established links between research (cause) and impact (effect) convincingly Phrases more common in low-scoring • Research outputs/process: the paper, peer- reviewed, journal of, et al, research project, this research has, by Dr, of Dr, research team • Attribution between research and pathways: work has, has informed, through the 0 10 20 30 40 50 60 70 80 Generally high quality corroborating evidence Some questionable quality evidence Vague and/or not clearly linked to impacts Number of case studies High-scoring Low-scoring Quantitative linguistic analysis Qualitative analysis • Coh-metrix analysis shows higher-scoring cases had more explicit causal connections between ideas and more logical connective words (and, or, but) than low-scoring cases
- 11. 3. Highly-rated case studies were easy to understand and well written • Low-scoring cases more likely to have academic phrasing: in relation to, in terms of, the way(s) in which • Flesch Reading Ease score, out of 100, was 30.0 on average for 4* and 27.5 on average for 1*/2* (all “college-graduate” difficulty). Panels C & D high-scoring case studies significantly easier to read than low-scorers • High scoring cases had more sub- headings (especially pronounced when comparing high to low cases in Panel D) • High-scoring cases used more direct, plain language, had fewer expressions of uncertainty or hedging statements, and were less likely to contain unsubstantiated or vague use of adjectives to describe impacts Qualitative analysis Quantitative linguistic analysis
- 12. •Add quick/simple questions to chat to start •Open mic: pose challenging questions for discussion • Context: what is your research and impact? • Challenge: what are you stuck on? • Share screen so we can see details (if relevant) • Initial response from Mark • Additional responses from Mark and group via chat • Wider discussion with group (answer in chat or open mic) • Concluding response from case study author Case study surgery Open mic