Measuring Open Access- Current State of the Art
1 like561 views
The document discusses the current state of open access (OA) publishing, detailing its definitions, measurement challenges, and the impact of OA on research dissemination. It highlights the roles of librarians and researchers in promoting OA, outlines methods for measuring OA availability, and presents statistical data on OA growth and its scientific impact. The conclusion emphasizes the importance of maximizing the diffusion of OA papers due to their greater impact compared to subscription-based publications.
Measuring Open Access- Current State of the Art
- 1. Measuring Open Access - Current State of the Art by Éric Archambault, D.Phil. President and CEO, Science-Metrix and 1science ESSS 2015 - Leuven
- 2. 2 The OA revolution is firmly in motion Librarians can play a key role: Traditional role – percolation New role – diffusion Researchers too – be fruitful and multiply OA in academic publications: complex beast Understanding the OA universe is key to useful measurement BACKGROUND
- 3. 3 Definitions Key vantage points Measuring OA Results Conclusions SYNOPSIS
- 4. 4 Budapest Open Access Initiative (2002) “The literature that should be freely accessible online is that which scholars give to the world without expectation of payment. Primarily, this category encompasses their peer-reviewed journal articles, but it also includes any unreviewed preprints that they might wish to put online for comment or to alert colleagues to important research findings. There are many degrees and kinds of wider and easier access to this literature. By "open access" to this literature, we mean its free availability on the public internet, permitting any users to read, download, copy, distribute, print, search, or link to the full texts of these articles, crawl them for indexing, pass them as data to software, or use them for any other lawful purpose, without financial, legal, or technical barriers other than those inseparable from gaining access to the internet itself. The only constraint on reproduction and distribution, and the only role for copyright in this domain, should be to give authors control over the integrity of their work and the right to be properly acknowledged and cited.” DEFINITIONS
- 5. 5 Green OA The main idea behind Green is self-archiving Archiving can be done in institutional and thematic repositories Gold OA The main idea behind Gold is that journal publishers make papers available There are Gold journals (cover-to-cover) but also Gold papers published in subscription- based journals (a.k.a. “hybrid journals”) DEFINITIONS
- 6. 6 Complexity of OA definition and measurement notably due to Embargoes Transiency Rights of all kind (to self-archive, to crawl, to recombine, to use commercially, etc) Discoverability DEFINITIONS
- 7. 7 Rules of involvement in OA VANTAGE POINTS
- 8. 8 Directories/registries of repositories Directory of OA journals VANTAGE POINTS
- 9. 9 Free or open source repository software DSpace EPrints Archimede, DAITSS, Dienst, Enterprise-Wide Digital Repository and Archive, ETD-db, eXtensible Text Framework, Fedora, Greenstone, Invenio, IRPlus, Keystone Digital Library Suite, MOAI, Omeka, OPUS, PubMan, WEKO, PeerLibrary Source: http://oad.simmons.edu/oadwiki/Free_and_open-source_repository_software VANTAGE POINTS
- 10. 10 Key repositories arXiv.org – the mothership PubMed Central / Europe PubMed Central Aggregators OpenAire BASE CORE A typical repository hosted by the Umeå universitet Library VANTAGE POINTS
- 11. 11 Despite, or perhaps because of, all the sources of OA available, it is very difficult to measure the availability of OA Here, we are concerned about the availability of peer-reviewed articles published in scholarly journals Why – this is what policies and mandates are preoccupied with BIBLIOMETRICS – PROPORTION OF OA
- 12. 12 Bottom-up measurement One would have to harvest all the sources available and de-duplicate results The main problem is how to determine reliably that items (1) were published (as opposed to an un- submitted manuscript) (2) are peer-reviewed (3) answer the “so what” question (you found there were 14,325,678 papers, so what?) BIBLIOMETRICS – PROPORTION OF OA
- 13. 13 Top-down measurement One would have to find an exhaustive bibliographic database of peer-reviewed articles and verify the availability of all papers Main problems: (1) there is no such database (2) extremely tedious to check all of them (3) how do you actually do that BIBLIOMETRICS – PROPORTION OF OA
- 14. 14 Top-down measurement - Sampling Considering the enormous task at hand, most authors have resorted to using sampling and search engines Harnad and team sampled articles from the Web of Science Björk and team sampled articles from Scopus Archambault and team sampled articles from Scopus and used multiple techniques as well as search engines BIBLIOMETRICS – PROPORTION OF OA
- 15. 15 Dealing with search engines Use user-friendly meta-search engines such as DuckDuckGo or DogPile Try to stay below the radar using mainstream search engines Neither solution feels remotely confortable Other solution is to build a dedicated infrastructure to facilitate OA discovery (this is the solution used by 1science) BIBLIOMETRICS – PROPORTION OF OA
- 16. 16 Divergence from the real measure is due to Capacity to design instrument that provides true value (function of recall and retrieval precision) Capacity to increase statistical significance through large samples SAMPLING AND METROLOGY
- 17. 17 A true positive (tp) in the present case is a paper known to be available in OA which is found by the harvesting instrument developed for the current project. A true negative (tn) is an article which is not available for free and is not found by the instrument. False positives and negatives (fp and fn) are the converse of the later. Retrieval precision, also called positive predictive value, provides an estimation of how frequently the instrument finds correct positive results and is calculated as follows: Retrieval Precision = 𝑡𝑡 𝑡𝑡+𝑓𝑓 Recall, also called true positive rate or sensitivity, is the capacity to correctly identify a large proportion of the positive records: Recall = 𝑡𝑡 𝑡𝑡+𝑓𝑓 Knowing the precise characteristics in terms of true and false positives and negatives allows for the computation of an adjustment score, which can then be applied to recalibrate the results to obtain a truer measure, one that corrects the limits of the instrument. The adjustment made in the previous study is based on the following formula: Adjustment = 𝑡𝑡+𝑓𝑓 𝑡𝑡+𝑓𝑓 SAMPLING AND METROLOGY
- 18. 18 Statistical precision can be assessed with the margin of error (ME). For a proportion (p) where the population is finite and known (which is the case here as the population from which we are sampling is the Scopus database), (N) is not systematically much larger than the sample size (n), and in which the values are discrete (for example, papers are discrete as one does not publish one third of a paper), given a critical score Z (which will be set at 0.95 in the study), ME is calculated as follows: 𝑀𝑀 = 𝑍 𝑝 1−𝑝 𝑁−𝑛 𝑛 𝑁−1 + 0.5 𝑛 SAMPLING AND METROLOGY
- 19. 19 The harvesting engine developed by Science-Metrix searches specific sites, including Scielo, PubMed Central, Research Gate and CiteSeerX It also uses a locally hosted version of the metadata of large-scale specialised repositories such as arXiv It systematically harvests metadata from institutional repositories listed in ROAR and OpenDOAR Finally, and in addition, a portion of the harvesting engine works in the cloud and searches for freely available papers MEASURING THE % OF OA PAPERS
- 20. 20 For Gold Journal OA articles, an estimate of the proportion of papers was made from the random sample by matching the journals that were known to be Gold to the year a paper was published Journals were obtained from the Directory of Open Access Journals (DOAJ) and the list of OA journals in PubMed Central This was done by matching journals’ ISSN, E-ISSN and names from Scopus to the relevant records in the sample MEASURING THE % OF OA PAPERS
- 21. 21 Evolution of the proportion of OA scientific papers as measured in April 2013 and April 2014, 1996–2013 RESULTS 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 1994 1996 1998 2000 2002 2004 2006 2008 2010 2012 2014 %ofpapersavailableinOA Adjusted OA April 2014 Adjusted OA April 2013 Measured OA April 2014 Measured OA April 2013
- 22. 22 Translation of OA availability between April 2013 and April 2014 RESULTS y = 2E-21e0.0234x R² = 0.976 y = 3E-17e0.0186x R² = 0.9473 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 %ofpapersavailableinOA Adjusted OA April 2014 Adjusted OA April 2013
- 23. 23 OA backfilling between April 2013 and April 2014 of papers published in 1996–2011 RESULTS y = 2E-112e0.1335x R² = 0.9976 0 20,000 40,000 60,000 80,000 100,000 120,000 1994 1996 1998 2000 2002 2004 2006 2008 2010 2012 2014 NumberofOApapersbackfilled betweenApril2013andApril2014
- 24. 24 Growth of the number of papers available in OA as measured in April 2014, 1996–2013 RESULTS y = 2E-73e0.09x R² = 0.9971 0 100,000 200,000 300,000 400,000 500,000 600,000 700,000 800,000 900,000 1,000,000 1994 1996 1998 2000 2002 2004 2006 2008 2010 2012 2014 NumberofpapersinOA Adjusted OA Measured OA
- 25. 25 Scientific impact of OA and non-OA papers published in 1996–2011 RESULTS 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 Averageofrelativecitations (ARC,1=worldavergae) OA All Papers Not OA
- 26. 26 Impact contest by OA type by field, 2009–2011 N.B. Here, Gold refers to full-Gold journals, not to Gold papers in hybrid journals RESULTS 1st place 2nd place 3rd place Least impact Type ARC Type ARC Type ARC Type ARC Agriculture, Fisheries & Forestry Green OA 1.57 Other OA 1.32 Not OA 0.88 Gold OA 0.51 Biology Other OA 1.37 Green OA 1.30 Not OA 0.69 Gold OA 0.47 Biomedical Research Other OA 1.23 Green OA 1.10 Gold OA 0.91 Not OA 0.65 Built Environment & Design Green OA 1.56 Other OA 1.28 Not OA 0.86 Gold OA 0.19 Chemistry Other OA 1.34 Green OA 1.28 Not OA 0.95 Gold OA 0.34 Clinical Medicine Other OA 1.56 Green OA 1.08 Gold OA 0.64 Not OA 0.63 Communication & Textual Studies Other OA 1.82 Green OA 1.51 Not OA 0.66 Gold OA 0.73 Earth & Environmental Sciences Green OA 1.46 Other OA 1.26 Gold OA 0.98 Not OA 0.72 Economics & Business Green OA 1.46 Other OA 1.30 Not OA 0.71 Gold OA 0.22 Enabling & Strategic Technologies Green OA 1.68 Other OA 1.53 Not OA 0.83 Gold OA 0.52 Engineering Green OA 1.84 Other OA 1.38 Not OA 0.83 Gold OA 0.55 General Arts, Humanities & Social Sciences Green OA 1.74 Other OA 1.49 Not OA 0.73 Gold OA 0.13 General Science & Technology Green OA 2.56 Other OA 2.24 Gold OA 0.69 Not OA 0.11 Historical Studies Green OA 2.37 Other OA 1.61 Not OA 0.76 Gold OA 0.37 Information & Communication Technology Green OA 1.62 Other OA 1.36 Gold OA 0.76 Not OA 0.69 Mathematics & Statistics Green OA 1.35 Other OA 1.11 Not OA 0.75 Gold OA 0.67 Philosophy & Theology Green OA 1.72 Other OA 1.63 Gold OA 0.86 Not OA 0.72 Physics & Astronomy Green OA 1.43 Gold OA 1.18 Other OA 1.04 Not OA 0.73 Psychology & Cognitive Sciences Other OA 1.35 Green OA 1.31 Not OA 0.66 Gold OA 0.59 Public Health & Health Services Other OA 1.38 Green OA 1.30 Not OA 0.76 Gold OA 0.71 Social Sciences Green OA 1.54 Other OA 1.44 Not OA 0.76 Gold OA 0.52 Visual & Performing Arts Green OA 2.16 Other OA 1.86 Not OA 0.77 Gold OA 0.29 Total Green OA 1.53 Other OA 1.36 Not OA 0.76 Gold OA 0.61 Field
- 27. 27 OA is a fast-moving phenomenon It is also quite complex to understand and to measure Uptake of OA limited by heterogeneity and challenges in discovery CONCLUSION
- 28. 28 Growth of OA should be understood to comprise two main aspects: Organic growth as more publishers, researchers and librarians increasingly make freshly published papers freely available “Backfilling” of already published papers by researchers and librarians and dis-embargoing of previously locked papers by publishers contribute to a translation of the availability curve CONCLUSION
- 29. 29 On average, openly accessible papers have a decidedly greater impact In 7 fields, publishing in subscription-based journals and not self-archiving is the worst possible strategy In these fields, Gold journals surpass in impact publishing in subscription-based journals with no self-archiving, even if these Gold journals are much younger and less established No longer adequate to publish and forget papers One has to actively market papers and think of post- publishing communication strategies Considering the high value of the knowledge contained in papers, and their high public cost, working to maximise diffusion and uptake is the least one can do CONCLUSION
- 30. 30 Visit 1science to learn about our solution to radically facilitate the discovery and use of peer-reviewed open access papers Visit Science-Metrix to learn about our evaluation and measurement activities THANK YOU