An overview of the REF
2 likes545 views
The document provides an overview of the Research Excellence Framework (REF) process. It discusses the purpose and assessment framework of the REF, including the criteria for evaluating outputs, impact, and environment. It also outlines the key stages of the REF process, from developing submission guidelines to expert review by panels. Submissions will include staff details, up to 4 research outputs per person, impact templates and case studies, environment data and templates. The results will be published in December 2014 and help determine £2 billion in annual research funding.
An overview of the REF
- 1. An overview of the REF Vicky Jones REF Deputy Manager Vitae: Preparing for the REF
- 2. Presentation outline • Overview • Staff • Outputs • Impact • Environment • Submissions
- 3. Overview: Purpose of the REF • The REF is a process of expert review • It replaces the RAE as the UK-wide framework for assessing research in all disciplines • Its purpose is: - To inform research funding allocations by the four UK HE funding bodies (approximately £2 billion per year) - Provide accountability for public funding of research and demonstrate its benefits - To provide benchmarks and reputational yardsticks
- 4. Overview: The assessment framework Overall quality Outputs Impact Environment Maximum of 4 outputs Impact template and Environment data and per researcher case studies template 65% 20% 15%
- 5. Overview: The REF process Criteria phase Submissions phase Assessment phase 2011 2012-13 2014 • Develop and publish • HEIs submit Codes of • Panels assess Guidance on submissions practice submissions (Jul) • Launch the REF • Publish outcomes • Develop, consult on and submissions system Dec 2014 publish Panel criteria (Jan 2012) • Submission deadline 29 Nov 2013
- 6. Overview: Guidance and criteria Comprehensive information and guidance is set out in: • Assessment framework and guidance on submissions (July 2011): - Sets out the information required in submissions and the definitions used • Panel criteria and working methods (Jan 2012): - Sets out how panels will assess submissions The above documents set out the official guidelines for the REF. These slides provide a summary of key points but do not provide or replace the official guidelines.
- 7. Overview: Submissions • Each HEI may submit in any or all of the 36 units of assessment (UOAs) • Each submission in a UOA provides evidence about the activity and achievements of a ‘submitted unit’ including: - Staff details (REF1a/b/c) - Research outputs (REF2) - Impact template and case studies (REF3a/b) - Environment data (REF4a/b/c) - Environment template (REF5) • A submitted unit may, but need not, comprise staff who work within a single ‘department’ or organisational unit
- 8. Overview: Assessment • Submissions will be assessed by 36 sub-panels working under the guidance of 4 main panels • Panels will carry out the assessment according to the published criteria and working methods Sub-panel responsibilities Main panel responsibilities • Contributing to the panel • Developing the panel criteria criteria and working methods and working methods • Assessing submissions and • Ensuring adherence to the recommending the outcomes criteria/procedures and consistent application of the overall assessment standards • Signing off the outcomes
- 9. Staff
- 10. Staff: Staff selection and circumstances • HEIs are responsible for selecting eligible staff whose outputs are to be included in their REF submissions • Each HEI is required to develop, document and apply a code of practice on the fair selection of staff • Number of outputs can be reduced without penalty where an individual’s circumstances have constrained their ability to work productively or produce four outputs in the REF period • We have sought to make these arrangements as clear and consistent as possible with due regard to confidentiality
- 11. Staff: Individual staff circumstances • Up to four outputs must be listed against each individual • The number of outputs can be reduced without penalty where an individual’s circumstances have constrained their ability to work productively or produce four outputs in the REF period • We have sought to make these arrangements as clear and consistent as possible, with due regard to confidentiality
- 12. Outputs
- 13. Outputs: Research outputs • Panels will assess the quality of research outputs through a process of expert review • All forms of output that embody research will be assessed on an equal footing • Panels will assess the quality of outputs, not the contribution of individual researchers • A co-authored output may be listed against one or more individuals that made a substantial research contribution to it (no more than twice within the same submission) • Institutions may request ‘double-weighting’ for outputs of extended scale and scope
- 14. Outputs: Additional information • Several sub-panels will make use of citation data as a minor component to inform peer-review • HEIs will be provided access to the Scopus citation data (in the relevant UOAs) through the REF submission system • Several sub-panels invite additional information from the HEI to inform judgements • Panels will not use journal impact factors, rankings or lists or the perceived standing of the publisher
- 15. Outputs: Assessment criteria The criteria for assessing the quality of outputs are originality, significance and rigour* Four star Quality that is world-leading in terms of originality, significance and rigour Quality that is internationally excellent in terms of originality, Three star significance and rigour but which falls short of the highest standards of excellence Two star Quality that is recognised internationally in terms of originality, significance and rigour One star Quality that is recognised nationally in terms of originality, significance and rigour Quality that falls below the standard of nationally recognised Unclassified work. Or work which does not meet the published definition of research for the purposes of this assessment * Each main panel provides descriptive account of the criteria
- 16. Impact
- 17. Impact: Definition of impact • Impact is defined broadly for the REF: an effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia • Panels recognise that impacts can be manifest in a wide variety of ways, may take many forms and occur in a wide range of spheres, in any geographic location • Panels provide examples of impact relevant to their disciplines, intended to stimulate ideas - not as exhaustive or prescriptive lists
- 18. Impact: Some examples of impact Enhanced professional Improved health or standards, ethics, guidelines Improved risk welfare outcomes or training management Public debate has been shaped or Improved quality, More effective Improved business informed by research accessibility or efficiency of a management or performance public service workplace practices A social enterprise Research has enabled initiative has been Changes to the Production costs have stakeholders to challenge created design or delivery of reduced conventional wisdom the school curriculum Improved forensic Enhanced preservation, Improved access to methods or expert Policy debate or decisions conservation or presentation justice, employment systems have been influenced or of cultural heritage or education shaped by research Improved management or Research has informed conservation of natural Organisations have Jobs have been public created or protected resources adapted to changing understanding, values, attitud cultural values es or behaviours Enhanced corporate The policies or activities of Changes to Levels of waste have social responsibility NGOs or charities have been legislation or reduced policies informed by research regulations New forms of artistic Changes in Enhanced technical A new product has expression or changes to professional practice standards or been commercialised creative practice protocols
- 19. Impact: Submission requirements • Sets out the submitted unit’s general Impact template approach to supporting impact from (REF3a) its research: • Approach to supporting impact during 20% of the the period 2008 to 2013 impact sub-profile • Forward strategy and plans • Specific examples of impacts already achieved, that were underpinned by Case studies the submitted unit’s research: (REF3b) • 1 case study per 10 FTE staff 80% of the submitted (plus 1 extra) impact • Impacts during 2008 to 2013; sub-profile underpinned by research since 1993
- 20. Impact: Case studies • Each case study should: - Clearly describe the underpinning research, who undertook it and when - Provide references to the research and evidence of quality - Explain how the research led/contributed to the impact - Clearly identify the beneficiaries and define the impact - Provide evidence/indicators of the impact - Provide independent sources of corroboration • All the material required to make a judgement should be included in the case study • Submitted case studies need not be representative of activity across the unit: pick the strongest examples
- 21. Impact: Assessment criteria The criteria for assessing impacts are reach and significance* Four star Outstanding impacts in terms of their reach and significance Three star Very considerable impacts in terms of their reach and significance Two star Considerable impacts in terms of their reach and significance One star Recognised but modest impacts in terms of their reach and significance The impact is of little or no reach and significance; or the impact Unclassified was not eligible; or the impact was not underpinned by excellent research produced by the submitted unit * Each main panel provides descriptive account of the criteria
- 22. Environment
- 23. Environment: Environment template • Each submission to include a completed template: - Overview - Research strategy - People, including: - staffing strategy and staff development - research students - Income, infrastructure and facilities - Collaboration and contribution to the discipline or research base • The ‘panel criteria’ request specific types of evidence under each heading, and indicate how much weight they will attach to each component
- 24. Environment: Environment data • All submissions to include data on: - Research doctoral degrees awarded (REF4a) - Research income (REF4b) - Research income in-kind (REF4c) • Definitions are aligned with HESA returns; the data relate to the ‘whole unit’ - not just submitted staff • Some sub-panels request specific additional data, to be included within the environment template (REF5) • Data will be considered by panels alongside the narrative information provided in the relevant section of the environment template
- 25. Environment: Assessment criteria The criteria for assessing the environment are vitality and sustainability* Four star An environment that is conducive to producing research of world-leading quality, in terms of its vitality and sustainability An environment that is conducive to producing research of Three star internationally excellent quality, in terms of its vitality and sustainability An environment that is conducive to producing research of Two star internationally recognised quality, in terms of its vitality and sustainability An environment that is conducive to producing research of One star nationally recognised quality, in terms of its vitality and sustainability Unclassified An environment that is not conducive to producing research of nationally recognised quality * Each main panel provides a descriptive account of the criteria
- 26. Submissions
- 27. Pre-submission • Codes of practice Submission by Response from funding body by 27 April 2012 6 July 2012 31 July 2012 12 October 2012 • Requests for multiple submissions or case studies requiring security clearance Request by Response from REF team by 27 April 2012 8 June 2012 28 September 2012 9 November 2012 7 December 2012 18 January 2013 • Survey of submission intentions Invitation and guidance Online survey open July 2012 Early Oct – early Dec 2012
- 28. Overview: The submission system • All submissions must be made through the REF submission system: Pilot available to all HEIs: Sep 2012 Open for submissions: Jan–Nov 2013 • Each HEI to set up system users and user permissions • All data may be entered onto the system and/or bulk imported • HEI contacts have seen a demonstration of the system • User guidance and support will be provided
- 29. Access to REF4 data • We will provide HESA data on research income and doctoral degrees awarded in stages: - May 2012: Data for 2008-09, 2009-10 and 2010-11 - Apr 2013: Data for 2011-12 - Doctoral degrees data for 2012-13 will be available from HESA when the student record is submitted • Data on research income-in-kind will be provided by the Research Councils and health research funders • Institutions will need to allocate these data to the appropriate UOAs; or use their own sources • The submission system will validate submitted data against the data we provided, at HEI level • stem
- 30. Further information www.ref.ac.uk (includes all relevant documents) Enquiries from staff at HEIs should be directed to their nominated institutional contact (see www.ref.ac.uk for a list) Other enquiries to info@ref.ac.uk