SlideShare a Scribd company logo
Assessing the value of open
access agreements
Researcher to Reader
Wednesday 22 February 2023
Laura Wong, Jisc
Essential metrics, metadata, and infrastructure to support consortial
evaluation of open access agreements
Open Access agreements at Jisc
Assessing the value of open access agreements - Researcher to Research
2023
2
How do we assess value?
•What do we mean by value?
•When do we want to assess value?
•Where do usage statistics fit in?
Assessing the value of open access agreements - Researcher to Research
2023
3
How can usage statistics support value questions?
Negotiation
•Which titles in the publisher’s portfolio are most
important in the UK context? Which are “high value”?
•To what extent are UK needs being met by OA content?
•How are patterns of use changing over time? Are we
seeing more usage of open compared to closed content?
Assessing the value of open access agreements - Researcher to Research
2023
5
Flipped journals
•What’s the global impact of flipped journals?
•What are the usage trends over the last couple of years
at journal and portfolio level? UK vs. global? Open vs.
closed content?
•Where does the UK sit in the international landscape?
What is its contribution to flipping titles to open access?
Assessing the value of open access agreements - Researcher to Research
2023
6
Institutional budgets and justifying spend
•What happens when we can no longer use the
subscriptions budget to fund these deals and initiatives?
•How do we secure budget and justify spend? How do we
continue demonstrating value for money and return on
investment?
•What benefits are we seeing from OA initiatives?
•What is the reach of our publication output?
Assessing the value of open access agreements - Researcher to Research
2023
7
Impact of policies
•How does the usage of funded research outputs
compare to non-funded outputs?
Assessing the value of open access agreements - Researcher to Research
2023
8
What does this mean for how we work with usage data?
Assessing the value of open access agreements - Researcher to Research
2023
9
•There’s still interest in…
•Paywalled usage
•Usage aggregated by title or journal
•Usage aggregated by publisher
•Usage attributed to institutions
•Annual trends
•But also interest in…
•Global usage of both open and paywalled
content
•Total usage of open content in the UK (not
just attributed to the institution)
•Global reach and country of use
Based on those key questions
A different approach
Holistic view
Indirect
value
Beyond the
publisher
platform
Slicing item
and global
data
Assessing the value of open access agreements - Researcher to Research
2023
10
Why item level reporting is important
Assessing the value of open access agreements - Researcher to Research
2023
11
Huge
Item
Report
COUNTER filters
• Articles published under
agreement
• Articles published by
associated authors
• Funded articles
• License type
• Articles on a platform
• Year of publication
Total usage
by…
• Single item
Supporting data
Assessing the value of open access agreements - Researcher to Research
2023
12
We need details
about:
• What has been
published
• Where was it
published
• Author affiliation
• Who funded it
• Under what licence
• Publication type
From places such as:
• Publisher systems
• Crossref
• Open Alex
• Open Access
Switchboard
• Databases such as
Scopus, WoS and
Dimensions
Matching on identifiers
such as:
• ISSN/ISBN
• DOI
• OCRID
• Funder ID
• Organisational IDs
13
OA
agreements
analytics
and
evaluation
Data management
Automate processes to collect, collate,
clean and verify the required data
Shared understanding
Share approaches monitoring and
evaluation and build an evidence
base to support conversations
Data analysis
Data expertise of the team to work with
the dataset to find insights
Additional datasets
Supplement and enhance data with
Crossref, Unpaywall and Dimensions
Presentation
Provide robust and comparable
metrics with contextual information
in a digestible format
Accessible and re-usable
datasets
Provide datasets in an easy and
interoperable format
Assessing the value of open access agreements - Researcher to Research
2023
What are Jisc doing with global and item usage stats?
COUNTER item
reports through
JUSP, our
collection service
Prototype data
visualisations
using publisher
supplied global
usage reports
Global usage of
items in
repositories
through IRUS
Case study
using IRUS stats
in analysis of an
agreement with
self-archiving
element
Assessing the value of open access agreements - Researcher to Research
2023
14
And OA monographs…
Global and granular: COUNTER for OA
RESEARCHER TO READER, FEBRUARY 2023 | TASHA MELLINS-COHEN
Release 5
accommodates
OA
Release 5.1
optimised for
OA
The standard
since 2003
Global
All usage
Non-attributed
Pay
walle
d
Free
to
read
OA
Attributed
Pay
walle
d
Free
to
read
OA
Granular
Item Report
Title and
Database
Reports
Platform
Report
Global AND
Granular
All
usage
Per
article
Global
Item
Report
Impact metrics
METRICS: HANDLE WITH CARE

More Related Content

PPTX
A new role for COUNTER usage data in open access agreements
PDF
UKSG 2023 - A new role for COUNTER usage data in open access agreements.pdf
PPTX
The New Dimensions in Scholcomm: How a global scholarly community collaborati...
PDF
Hawkins "Monitoring Usage of Open Access Long-Form Content"
PDF
Ratan "Are we there yet? Keeping the promise of open science"
PDF
RDA-UK/FREYA workshop
PDF
How to measure the impact of Research ?
A new role for COUNTER usage data in open access agreements
UKSG 2023 - A new role for COUNTER usage data in open access agreements.pdf
The New Dimensions in Scholcomm: How a global scholarly community collaborati...
Hawkins "Monitoring Usage of Open Access Long-Form Content"
Ratan "Are we there yet? Keeping the promise of open science"
RDA-UK/FREYA workshop
How to measure the impact of Research ?

Similar to Assessing the value of OA agreements (20)

PDF
Evaluating Transitional Agreements with Article Level Metadata
PPTX
Charleston 2023 - Batten down the hatches: Navigating the complexities of Ope...
PPTX
Expanding the Possible: What’s New and Upcoming in Standards and Technologies...
PDF
Author_Workshop_Maximizing_your_success_OA_26_July_2023_updated.pdf
PDF
What do funders want from research infrastructure? Let’s link the grants with...
PDF
UKSG 2023 - The read & publish user journey: delivering a unified reader & au...
PDF
Bosman and Kramer Research assessment and metadata"
PDF
UKSG 2024 - A critical review of transitional agreements in the UK: why, how,...
PPTX
Research analytics service - ARMA study tour
PPTX
Building a Trusted Framework - Kevin Hawkins, University North Texas
PDF
Qualitative and Quantitative Methods in Electronic Collection Management
PDF
L cwebinar russell_wise_feb26-2015
PDF
Researcher profiles and metrics that matter
PDF
5 ways to demonstrate value using usage statistics
PPTX
Lagace - Copyright Clearance Center April 2, 2015
PPTX
Frederick Friend: Where we are now in opening research results and data
PPTX
ORCID - UK PIDs for Open Access - progress update
PPTX
Quality Assurance for Journal Guidance
PPTX
Lessons from Journal Research Data Policy Registry Pilot
PPTX
Love Your Code Workshop Introduction_Corti_Engeli
Evaluating Transitional Agreements with Article Level Metadata
Charleston 2023 - Batten down the hatches: Navigating the complexities of Ope...
Expanding the Possible: What’s New and Upcoming in Standards and Technologies...
Author_Workshop_Maximizing_your_success_OA_26_July_2023_updated.pdf
What do funders want from research infrastructure? Let’s link the grants with...
UKSG 2023 - The read & publish user journey: delivering a unified reader & au...
Bosman and Kramer Research assessment and metadata"
UKSG 2024 - A critical review of transitional agreements in the UK: why, how,...
Research analytics service - ARMA study tour
Building a Trusted Framework - Kevin Hawkins, University North Texas
Qualitative and Quantitative Methods in Electronic Collection Management
L cwebinar russell_wise_feb26-2015
Researcher profiles and metrics that matter
5 ways to demonstrate value using usage statistics
Lagace - Copyright Clearance Center April 2, 2015
Frederick Friend: Where we are now in opening research results and data
ORCID - UK PIDs for Open Access - progress update
Quality Assurance for Journal Guidance
Lessons from Journal Research Data Policy Registry Pilot
Love Your Code Workshop Introduction_Corti_Engeli
Ad

More from JUSPSTATS (20)

PPTX
Role of JUSP in gathering usage statistics and informing decision making
PPTX
JUSP fundamentals for further education
PPTX
JUSP webinar: Reporting with COUNTER R4 and R5 data
PPTX
COUNTER R4 to R5 - transition and comparison with JUSP - updated
PPTX
JUSP report features update: title master report filtering and database stand...
PPTX
Working with COUNTER Release 5 reports in JUSP - 30 April 2019
PPTX
JUSP and COUNTER Release 5 - update
PPTX
Getting started with JUSP
PPTX
JUSP for further education
PPTX
Update and Q&A
PPTX
Data visualisations in JUSP
PPTX
Database and platform reports in JUSP
PPTX
JUSP workshop - update and experience exchange
PPTX
JR2 and JR5 reports in JUSP
PPTX
Workshop manchester-june2016
PPTX
Workshop manchester-may2016-morning
PPTX
Journal and ebook usage statistics with JUSP
PDF
JUSP for CPD25 eCollections Management workshop
PPTX
JUSP at Brunel University
PPTX
Knowledge Base+ and JUSP
Role of JUSP in gathering usage statistics and informing decision making
JUSP fundamentals for further education
JUSP webinar: Reporting with COUNTER R4 and R5 data
COUNTER R4 to R5 - transition and comparison with JUSP - updated
JUSP report features update: title master report filtering and database stand...
Working with COUNTER Release 5 reports in JUSP - 30 April 2019
JUSP and COUNTER Release 5 - update
Getting started with JUSP
JUSP for further education
Update and Q&A
Data visualisations in JUSP
Database and platform reports in JUSP
JUSP workshop - update and experience exchange
JR2 and JR5 reports in JUSP
Workshop manchester-june2016
Workshop manchester-may2016-morning
Journal and ebook usage statistics with JUSP
JUSP for CPD25 eCollections Management workshop
JUSP at Brunel University
Knowledge Base+ and JUSP
Ad

Recently uploaded (20)

PPT
ISS -ESG Data flows What is ESG and HowHow
PDF
annual-report-2024-2025 original latest.
PPTX
Computer network topology notes for revision
PPTX
Introduction-to-Cloud-ComputingFinal.pptx
PPTX
IB Computer Science - Internal Assessment.pptx
PPTX
Introduction to Firewall Analytics - Interfirewall and Transfirewall.pptx
PPTX
The THESIS FINAL-DEFENSE-PRESENTATION.pptx
PDF
Foundation of Data Science unit number two notes
PDF
TRAFFIC-MANAGEMENT-AND-ACCIDENT-INVESTIGATION-WITH-DRIVING-PDF-FILE.pdf
PPTX
Supervised vs unsupervised machine learning algorithms
PDF
168300704-gasification-ppt.pdfhghhhsjsjhsuxush
PDF
.pdf is not working space design for the following data for the following dat...
PPTX
DISORDERS OF THE LIVER, GALLBLADDER AND PANCREASE (1).pptx
PDF
Lecture1 pattern recognition............
PDF
22.Patil - Early prediction of Alzheimer’s disease using convolutional neural...
PPTX
advance b rammar.pptxfdgdfgdfsgdfgsdgfdfgdfgsdfgdfgdfg
PPTX
Database Infoormation System (DBIS).pptx
PPT
Quality review (1)_presentation of this 21
PPTX
Introduction to Knowledge Engineering Part 1
PPTX
Introduction to Basics of Ethical Hacking and Penetration Testing -Unit No. 1...
ISS -ESG Data flows What is ESG and HowHow
annual-report-2024-2025 original latest.
Computer network topology notes for revision
Introduction-to-Cloud-ComputingFinal.pptx
IB Computer Science - Internal Assessment.pptx
Introduction to Firewall Analytics - Interfirewall and Transfirewall.pptx
The THESIS FINAL-DEFENSE-PRESENTATION.pptx
Foundation of Data Science unit number two notes
TRAFFIC-MANAGEMENT-AND-ACCIDENT-INVESTIGATION-WITH-DRIVING-PDF-FILE.pdf
Supervised vs unsupervised machine learning algorithms
168300704-gasification-ppt.pdfhghhhsjsjhsuxush
.pdf is not working space design for the following data for the following dat...
DISORDERS OF THE LIVER, GALLBLADDER AND PANCREASE (1).pptx
Lecture1 pattern recognition............
22.Patil - Early prediction of Alzheimer’s disease using convolutional neural...
advance b rammar.pptxfdgdfgdfsgdfgsdgfdfgdfgsdfgdfgdfg
Database Infoormation System (DBIS).pptx
Quality review (1)_presentation of this 21
Introduction to Knowledge Engineering Part 1
Introduction to Basics of Ethical Hacking and Penetration Testing -Unit No. 1...

Assessing the value of OA agreements

  • 1. Assessing the value of open access agreements Researcher to Reader Wednesday 22 February 2023 Laura Wong, Jisc Essential metrics, metadata, and infrastructure to support consortial evaluation of open access agreements
  • 2. Open Access agreements at Jisc Assessing the value of open access agreements - Researcher to Research 2023 2
  • 3. How do we assess value? •What do we mean by value? •When do we want to assess value? •Where do usage statistics fit in? Assessing the value of open access agreements - Researcher to Research 2023 3
  • 4. How can usage statistics support value questions?
  • 5. Negotiation •Which titles in the publisher’s portfolio are most important in the UK context? Which are “high value”? •To what extent are UK needs being met by OA content? •How are patterns of use changing over time? Are we seeing more usage of open compared to closed content? Assessing the value of open access agreements - Researcher to Research 2023 5
  • 6. Flipped journals •What’s the global impact of flipped journals? •What are the usage trends over the last couple of years at journal and portfolio level? UK vs. global? Open vs. closed content? •Where does the UK sit in the international landscape? What is its contribution to flipping titles to open access? Assessing the value of open access agreements - Researcher to Research 2023 6
  • 7. Institutional budgets and justifying spend •What happens when we can no longer use the subscriptions budget to fund these deals and initiatives? •How do we secure budget and justify spend? How do we continue demonstrating value for money and return on investment? •What benefits are we seeing from OA initiatives? •What is the reach of our publication output? Assessing the value of open access agreements - Researcher to Research 2023 7
  • 8. Impact of policies •How does the usage of funded research outputs compare to non-funded outputs? Assessing the value of open access agreements - Researcher to Research 2023 8
  • 9. What does this mean for how we work with usage data? Assessing the value of open access agreements - Researcher to Research 2023 9 •There’s still interest in… •Paywalled usage •Usage aggregated by title or journal •Usage aggregated by publisher •Usage attributed to institutions •Annual trends •But also interest in… •Global usage of both open and paywalled content •Total usage of open content in the UK (not just attributed to the institution) •Global reach and country of use Based on those key questions
  • 10. A different approach Holistic view Indirect value Beyond the publisher platform Slicing item and global data Assessing the value of open access agreements - Researcher to Research 2023 10
  • 11. Why item level reporting is important Assessing the value of open access agreements - Researcher to Research 2023 11 Huge Item Report COUNTER filters • Articles published under agreement • Articles published by associated authors • Funded articles • License type • Articles on a platform • Year of publication Total usage by… • Single item
  • 12. Supporting data Assessing the value of open access agreements - Researcher to Research 2023 12 We need details about: • What has been published • Where was it published • Author affiliation • Who funded it • Under what licence • Publication type From places such as: • Publisher systems • Crossref • Open Alex • Open Access Switchboard • Databases such as Scopus, WoS and Dimensions Matching on identifiers such as: • ISSN/ISBN • DOI • OCRID • Funder ID • Organisational IDs
  • 13. 13 OA agreements analytics and evaluation Data management Automate processes to collect, collate, clean and verify the required data Shared understanding Share approaches monitoring and evaluation and build an evidence base to support conversations Data analysis Data expertise of the team to work with the dataset to find insights Additional datasets Supplement and enhance data with Crossref, Unpaywall and Dimensions Presentation Provide robust and comparable metrics with contextual information in a digestible format Accessible and re-usable datasets Provide datasets in an easy and interoperable format Assessing the value of open access agreements - Researcher to Research 2023
  • 14. What are Jisc doing with global and item usage stats? COUNTER item reports through JUSP, our collection service Prototype data visualisations using publisher supplied global usage reports Global usage of items in repositories through IRUS Case study using IRUS stats in analysis of an agreement with self-archiving element Assessing the value of open access agreements - Researcher to Research 2023 14
  • 16. Global and granular: COUNTER for OA RESEARCHER TO READER, FEBRUARY 2023 | TASHA MELLINS-COHEN
  • 17. Release 5 accommodates OA Release 5.1 optimised for OA The standard since 2003

Editor's Notes

  • #19: How it works: Total usage / global usage is built up of non-attributed and attributed usage of content on publisher platforms – that is, the usage we CAN link to institutions, and that which we CAN’T link to institutions, wherever they are in the world. Within that split, whether content is paywalled or free to read or OA is a secondary question. For subscription content report providers break this down to show only attributed usage, for a single institution or consortium For OA content, report providers don’t have to do that breakdown – that is, they can (AND SHOULD) report global usage Great, but not necessarily granular…