SlideShare a Scribd company logo
Frameworks for responsible metrics
Challenges & Solutions
Dr Elizabeth Gadd
1. Frameworks are the beginning,
not the end, of a RM journey.
Solutions:
• Communities of practice?
• Codes of practice?
• Pre-agreed evaluative frameworks?
– Henk Moed, The Bibliomagician
2. Frameworks need an owner
• Solutions:
• Frameworks should specify who is responsible
for implementation
• Champions?
3. Frameworks only cover the
metrics under our control, not those
outwith
Solutions:
• Funders and suppliers should also work within
frameworks for responsible use of metrics
• Work as a sector to improve or abandon
nonsensical world university rankings
4. Responsible peer review is just as
important as responsible metrics.
The frameworks don't address this.
Solutions:
• Sector-wide guidance on
– responsible peer review
– the right balance between metrics and peer review
5. Frameworks need to apply to our
people as much as our metrics
Solutions:
• Bibliometric competencies exist for data and
analysis domains
• Do we need minimum competency levels across
evaluation and policy domains too?

More Related Content

PPTX
The turning tide a new culture of research metrics - David Sweeney
PPT
The perspective of Research Councils UK
PPTX
The culture of research metrics in research organisations
PPTX
LWEC presentation on the Ecosystem Approach and the National Ecosystem Assess...
PPTX
Ecosystem Assessments at the Local Scale - Dr. Claire Brown (UNEP-WCMC)
PDF
Tom McBride - Evidence-informed policy making - 26 June 2017
PDF
Assessing the Impact of the Academic Library: From theory to practice.
PPTX
"Impact2: through the power of collaboration. How we increased our impact by ...
The turning tide a new culture of research metrics - David Sweeney
The perspective of Research Councils UK
The culture of research metrics in research organisations
LWEC presentation on the Ecosystem Approach and the National Ecosystem Assess...
Ecosystem Assessments at the Local Scale - Dr. Claire Brown (UNEP-WCMC)
Tom McBride - Evidence-informed policy making - 26 June 2017
Assessing the Impact of the Academic Library: From theory to practice.
"Impact2: through the power of collaboration. How we increased our impact by ...

What's hot (20)

PPTX
Exploring Innovative and Effective Methods for IQA
PPTX
Exploring Innovative and Effective Methods for IQA - Research Framework
PPT
NISO Apr 29 Virtual Conference: Development of specific definitions for alter...
PPTX
Application process
PPTX
Data Policy Frameworks/Joseph Wafula
PPTX
Open Discovery Initiative, Laura Morse, ALA Annual 2017
PPTX
Embedding ORCID across researcher career paths
PPTX
NISO Altmetrics Initiative, ALA Update January 21, 2017
PDF
PROVIA Research Agenda
PPT
Mid-Year Report Information
PPTX
UKRI's simpler and better funding programme
PDF
Scientific Equipment Policy Change through Facilitated Advocacy Nighisty Ghezae
PDF
Glyn Jones presentation at the Administrative Data Research Centre Wales laun...
PPTX
‘Evidence-based forestry’: Constructing bridges that connect science, policy...
PDF
Research Data Management - EPSRC’s Perspective
PDF
Global Water Studio OESEO
PDF
Los Angeles County WIC Programs: Developing Infrastructure for Partnered Rese...
PPTX
Performance Measurement: Ensuring Accuracy, Transparency and Relevance
PPTX
Case studies for open science
Exploring Innovative and Effective Methods for IQA
Exploring Innovative and Effective Methods for IQA - Research Framework
NISO Apr 29 Virtual Conference: Development of specific definitions for alter...
Application process
Data Policy Frameworks/Joseph Wafula
Open Discovery Initiative, Laura Morse, ALA Annual 2017
Embedding ORCID across researcher career paths
NISO Altmetrics Initiative, ALA Update January 21, 2017
PROVIA Research Agenda
Mid-Year Report Information
UKRI's simpler and better funding programme
Scientific Equipment Policy Change through Facilitated Advocacy Nighisty Ghezae
Glyn Jones presentation at the Administrative Data Research Centre Wales laun...
‘Evidence-based forestry’: Constructing bridges that connect science, policy...
Research Data Management - EPSRC’s Perspective
Global Water Studio OESEO
Los Angeles County WIC Programs: Developing Infrastructure for Partnered Rese...
Performance Measurement: Ensuring Accuracy, Transparency and Relevance
Case studies for open science
Ad

Similar to Frameworks for responsible metrics: Challenges and solutions (20)

PDF
February 2019 CoP Webinar - Building Maturity - using the Change Management M...
PDF
Knowledge Management in Healthcare Analytics
PPTX
Grp 1 art 23 - emerg trend in od
PPT
Marcus Evans 2012 - Actionable Segmentations
PPTX
Evaluability Assessments and Choice of Evaluation Methods
PPTX
PPTX
Unit 1 PM.pptx
PDF
National Adaptation Plans Thailand - Theory of Multi-Criteria Analysis
PDF
Webinar slide-deck-enterprise-architecture-capability-assessments
PPTX
Positive Youth Development (PYD) Framework toolkit application modules - Dr B...
PPT
The nature of program evaluation
PDF
Guiding Agile Solution Delivery with the ArchiMate Language
PPT
A process maturity model for requirements engineering
PPTX
1 a jane deal
 
PDF
Building Organisational Change Capability Seminar Info Webinar September 2015
PPTX
PMI PROFESSIONAL IN BUSINESS ANALYSIS (PMI-PBA) 2025 Trainer Materials Free ...
PPTX
Bmsqms revisedwithchevrons
PPTX
Gcwabaza, Lungelo - Conference Presentation Draft 5
DOC
Performance Management - Herman Augnis
February 2019 CoP Webinar - Building Maturity - using the Change Management M...
Knowledge Management in Healthcare Analytics
Grp 1 art 23 - emerg trend in od
Marcus Evans 2012 - Actionable Segmentations
Evaluability Assessments and Choice of Evaluation Methods
Unit 1 PM.pptx
National Adaptation Plans Thailand - Theory of Multi-Criteria Analysis
Webinar slide-deck-enterprise-architecture-capability-assessments
Positive Youth Development (PYD) Framework toolkit application modules - Dr B...
The nature of program evaluation
Guiding Agile Solution Delivery with the ArchiMate Language
A process maturity model for requirements engineering
1 a jane deal
 
Building Organisational Change Capability Seminar Info Webinar September 2015
PMI PROFESSIONAL IN BUSINESS ANALYSIS (PMI-PBA) 2025 Trainer Materials Free ...
Bmsqms revisedwithchevrons
Gcwabaza, Lungelo - Conference Presentation Draft 5
Performance Management - Herman Augnis
Ad

More from Higher Education Funding Council for England (20)

PDF
Learning gain in international perspective: Towards a more reliable model for...
PDF
Ethics and researcher integrity: Institutional considerations for learning ga...
PDF
The UEA learning gain project: Learning gain and confidence gain in the class...
PDF
Engaging students in the national mixed methodology learning gain project
PDF
Are assessment scores good proxies of estimating learning gains?
PDF
Progress with HELGA: Can it provide a sector measure of learning gain? - Jame...
PDF
Learning gain in the national context - Yvonne Hawkins
PDF
HEFCE's learning gain programme: Update on current activity - Ruby Gatehouse
PPTX
Financial health of the sector and related developments - Madeleine Atkins
PPTX
Improving access and student success in higher education - Chris Millward
PPTX
Consultation on the Office for Students regulatory framework - Susan Lapworth
PPTX
UKRI, Research England and the REF - David Sweeney
PPTX
Initial consultation event: Timing and publication
PPTX
Initial consultation event: Introduction
PPTX
Initial consultation event: Content workshop
PPTX
A university challenge: Perspectives on university contributions to place-bas...
PPTX
Challenges, opportunities, and financial health overview - Madeline Atkins
PPTX
Universities as core partners in realising the Industrial Strategy - Luke Geo...
PPTX
REF 2021 and Research England - David Sweeney
PPTX
Next steps for TEF - Chris Millward
Learning gain in international perspective: Towards a more reliable model for...
Ethics and researcher integrity: Institutional considerations for learning ga...
The UEA learning gain project: Learning gain and confidence gain in the class...
Engaging students in the national mixed methodology learning gain project
Are assessment scores good proxies of estimating learning gains?
Progress with HELGA: Can it provide a sector measure of learning gain? - Jame...
Learning gain in the national context - Yvonne Hawkins
HEFCE's learning gain programme: Update on current activity - Ruby Gatehouse
Financial health of the sector and related developments - Madeleine Atkins
Improving access and student success in higher education - Chris Millward
Consultation on the Office for Students regulatory framework - Susan Lapworth
UKRI, Research England and the REF - David Sweeney
Initial consultation event: Timing and publication
Initial consultation event: Introduction
Initial consultation event: Content workshop
A university challenge: Perspectives on university contributions to place-bas...
Challenges, opportunities, and financial health overview - Madeline Atkins
Universities as core partners in realising the Industrial Strategy - Luke Geo...
REF 2021 and Research England - David Sweeney
Next steps for TEF - Chris Millward

Recently uploaded (20)

PDF
Pre independence Education in Inndia.pdf
PDF
Supply Chain Operations Speaking Notes -ICLT Program
PDF
Saundersa Comprehensive Review for the NCLEX-RN Examination.pdf
PDF
Abdominal Access Techniques with Prof. Dr. R K Mishra
PPTX
school management -TNTEU- B.Ed., Semester II Unit 1.pptx
PDF
Physiotherapy_for_Respiratory_and_Cardiac_Problems WEBBER.pdf
PDF
Classroom Observation Tools for Teachers
PPTX
PPH.pptx obstetrics and gynecology in nursing
PPTX
Institutional Correction lecture only . . .
PPTX
Renaissance Architecture: A Journey from Faith to Humanism
PPTX
GDM (1) (1).pptx small presentation for students
PDF
Basic Mud Logging Guide for educational purpose
PDF
Computing-Curriculum for Schools in Ghana
PDF
VCE English Exam - Section C Student Revision Booklet
PPTX
human mycosis Human fungal infections are called human mycosis..pptx
PPTX
Introduction_to_Human_Anatomy_and_Physiology_for_B.Pharm.pptx
PPTX
Pharmacology of Heart Failure /Pharmacotherapy of CHF
PDF
grade 11-chemistry_fetena_net_5883.pdf teacher guide for all student
PPTX
master seminar digital applications in india
PPTX
Microbial diseases, their pathogenesis and prophylaxis
Pre independence Education in Inndia.pdf
Supply Chain Operations Speaking Notes -ICLT Program
Saundersa Comprehensive Review for the NCLEX-RN Examination.pdf
Abdominal Access Techniques with Prof. Dr. R K Mishra
school management -TNTEU- B.Ed., Semester II Unit 1.pptx
Physiotherapy_for_Respiratory_and_Cardiac_Problems WEBBER.pdf
Classroom Observation Tools for Teachers
PPH.pptx obstetrics and gynecology in nursing
Institutional Correction lecture only . . .
Renaissance Architecture: A Journey from Faith to Humanism
GDM (1) (1).pptx small presentation for students
Basic Mud Logging Guide for educational purpose
Computing-Curriculum for Schools in Ghana
VCE English Exam - Section C Student Revision Booklet
human mycosis Human fungal infections are called human mycosis..pptx
Introduction_to_Human_Anatomy_and_Physiology_for_B.Pharm.pptx
Pharmacology of Heart Failure /Pharmacotherapy of CHF
grade 11-chemistry_fetena_net_5883.pdf teacher guide for all student
master seminar digital applications in india
Microbial diseases, their pathogenesis and prophylaxis

Frameworks for responsible metrics: Challenges and solutions

  • 1. Frameworks for responsible metrics Challenges & Solutions Dr Elizabeth Gadd
  • 2. 1. Frameworks are the beginning, not the end, of a RM journey. Solutions: • Communities of practice? • Codes of practice? • Pre-agreed evaluative frameworks? – Henk Moed, The Bibliomagician
  • 3. 2. Frameworks need an owner • Solutions: • Frameworks should specify who is responsible for implementation • Champions?
  • 4. 3. Frameworks only cover the metrics under our control, not those outwith Solutions: • Funders and suppliers should also work within frameworks for responsible use of metrics • Work as a sector to improve or abandon nonsensical world university rankings
  • 5. 4. Responsible peer review is just as important as responsible metrics. The frameworks don't address this. Solutions: • Sector-wide guidance on – responsible peer review – the right balance between metrics and peer review
  • 6. 5. Frameworks need to apply to our people as much as our metrics Solutions: • Bibliometric competencies exist for data and analysis domains • Do we need minimum competency levels across evaluation and policy domains too?