![Questions? http://openurlquality.niso.org http://www.niso.org/workrooms/openurlquality @nisoiota on twitter Rafal Kasprowski Electronic Resources Librarian Rice University, Fondren Library MS 44 Houston, TX 77005 [email_address] http://www.slideshare.net/rkaspro](https://image.slidesharecdn.com/kasprowskinasig11postpres-110611013204-phpapp01/85/IOTA-NASIG-2011-Measuring-the-Quality-of-OpenURL-Links-39-320.jpg)
IOTA @ NASIG 2011: Measuring the Quality of OpenURL Links
- 1. NISO’s IOTA Initiative Measuring the Quality of OpenURL Links NASIG Annual Conference St. Louis, MO June 2 – 5, 2011 Rafal Kasprowski, Rice University
- 2. Agenda In the Beginning: Full-text linking and Advent of OpenURL IOTA: Created in response to OpenURL linking problems IOTA’s analytical approach Community-derived reports comparing quality of vendor OpenURLs Concept of the OpenURL Quality Index IOTA & KBART: relationship & joint initiative Community involvement in IOTA: necessary for best outcomes
- 3. Before OpenURL: Proprietary Linking Certain A&I database providers (e.g., CSA, PubMed) offered full-text linking option for a select number of content providers. Libraries manually activated full-text linking with providers they had subscriptions with. A&I --> Full Text
- 4. Proprietary Linking: Pros and Cons Linking had to be activated manually by libraries for each full-text provider. A&I providers offering this option were few. Selection of full-text providers was limited. But... Once set up, the static links to full texts were accurate. Problem source pinpointed easily: A&I --> Full Text
- 5. Advent of OpenURL Objective: Deliver full texts unrestrained by proprietary silos. Open standard generating dynamic links at time of request. A-Z list (e.g., e-journal, e-books): Knowledge base (KB) with library's holdings. Replaces librarian as intermediary in linking. Indicates provider of "appropriate copy" A&I ("Source") --> A-Z list ("KB") --> Full Text ("Target")
- 6. A, Bernand , et al. " A versatile nanotechnology to connect individual nano-objects for the fabrication of hybrid single-electron devices. " Nanotechnology 21 , no. 44 ( November 5, 2010 ): 445201 . Academic Search Complete , EBSCO host (accessed October 24, 2010). OpenURL: resolver, syntax, linking nodes http://ps4ps6lm2r.search.serialssolutions.com/?issn=0957-4484&volume= 21 &issue= 44 &date= 20101105 &spage= 445201 &title= Nanotechnology &atitle= A+ versatile+nanotechnology+to+connect+individual+nano-objects+for+the+ fabrication+of+hybrid+single-electron+devices. &aulast= A++Bernand Source Citation Target OpenURL (Source OpenURL structured similarly)
- 7. Pros & Cons of OpenURL Pros: KB/Resolver vendors took over most of the linking setup: Less work for libraries and providers. Participation by A&I platforms and full-text providers exceeded proprietary linking: OpenURL scales better. Cons: Dynamic linking less predictable than static linking: more difficult to pinpoint cause of link failures. OpenURL linking not improved significantly last 10 years. No systematic method exists to benchmark OpenURLs.
- 8. Identifying source of problem… "72% of respondents to the online survey either agreed or strongly agreed that a significant problem for link resolvers is the generation of incomplete or inaccurate OpenURLs by databases (for example, A&I products)." Culling, James. 2007. Link Resolvers and the Serials Supply Chain: Final Project Report for UKSG, p.33. http://www.uksg.org/sites/uksg.org/files/uksg_link_resolvers_final_report.pdf. Defining methodology for addressing problem… Recently, researchers have indicated the need for metadata quality metrics , including: completeness; accuracy; conformance to expectations; logical consistency and coherence. Bruce, Thomas R. and Hillmann, Diane I. 2004. The Continuum of Metadata Quality: Defining, Expressing, Exploiting. In Metadata in Practice . Ed. Diane I. Hillmann and Elaine L. Westbrooks. Chicago: American Library Association, pp. 238-256.
- 9. Année philologique OpenURL Study 2008 Cornell study led by Adam Chandler* Problem: Too often links sent from Aph did not successfully resolve to requested resource. Objective: Examine quality of OpenURLs offered to users by Aph in order to improve the linking. Aph Study investigated: Faulty citation metadata from source database. Method to evaluate the OpenURLs. *Chandler, Adam. 2009. Results of L’Année philologique online OpenURL Quality Investigation: Mellon Planning Grant Final Report. http://metadata.library.cornell.edu/oq/files/200902%20lannee-mellonreport-openurlquality-final.pdf.
- 10. Scoring System & Aph Study Outcomes Concept of scoring in Aph study (based on B. Hughes study)* establish a baseline for comparison; results to be shared with data providers; develop a best practice. Problem analysis in Aph study limited to: source link presence/absence of citation metadata elements Results: OpenURL quality model: compares elements in Aph OpenURLs to those of other providers. No scoring was achieved for Aph , but model is first step towards scoring system. *Hughes, Baden. 2004. Metadata Quality Evaluation: Experience from the Open Language Archives Community. In Digital Libraries: International Collaboration and Cross-Fertilization. Ed. Zhaoneng Chen et al. Berlin: Springer-Verlag, 2004, pp. 320-329.
- 11. Creation of IOTA NISO: Accepts proposal to take Aph Study to wider community. New OpenURL quality metrics initiative formed in Jan. 2011. Branded as: I mproving O penURLs T hrough A nalytics. Basic Assumptions: Results are achieved through an analytical investigation of how OpenURL links work. Practical Goal: Not the OpenURL standard is addressed, but the links (OpenURLs) generated by standard. Selective changes to OpenURLs will lead to significant improvement in linking success rate. Motto: "small changes. big improvements"
- 12. IOTA Desired Outcomes Produce qualitative reports that will help OpenURL providers quickly compare their OpenURL quality to that of their peers. Develop community-recognized index for measuring the quality of OpenURL links generated by content providers. Method: fair; transparent; scalable across all OpenURLs and their providers.
- 13. How is comparing OpenURLs useful? Content providers generating OpenURLs can: compare their OpenURLs with other providers; make improvements to their OpenURLs. Institutions can: compare OpenURL providers; make local adjustments to OpenURL setup. Link resolver vendors can: compare OpenURL providers; change their OpenURL provider settings: Link resolvers; Web-scale discovery products.
- 14. Report types Source reports Viewing how a particular (1) vendor or (2) database A. uses OpenURL elements (element frequency) B. formats OpenURL elements (pattern frequency) Element / Pattern reports Viewing how a particular (1) element or format A. is used across vendors B. is used across databases
- 15. Running reports
- 16. Reports: Log file providers
- 17. Report: source (vendor or database)
- 18. Report: source = vendor
- 19. Report: element, source = vendor
- 20. Report: pattern, source = vendor
- 21. Report: source = database
- 22. Report: element, source = database
- 23. Report: pattern, source = database
- 24. Report: element and pattern frequency
- 25. Report: element & pattern frequency: Vendor option
- 26. Report: element/pattern frequency: Choosing Metric
- 27. Report: element/pattern by vendor
- 28. Report: element & pattern frequency: Database option
- 29. Report: element/pattern by database
- 30. Reporting System: improvements underway Consolidating variant instances of databases and vendors if the same; Separating article-like requests from book-like requests Either/Or situation: most resources do not offer both formats Once separation is completed, users will be given corresponding options to select OpenURL data by format: ARTICLE or BOOK These improvements will also benefit the accuracy of the OpenURL scoring system.
- 31. OpenURL Quality Index: initial version 1. Core elements: Any element contained in IOTA's OpenURL reporting system; 13M OpenURLs already obtained from libraries content providers. 2. Scoring system based on assumption : Correlation exists between # of core elements ("OpenURL completeness") & ability of OpenURLs to link to specific content. 3. Weighting assigned to core elements: Based on relative importance spage vs atitle issn vs jtitle doi/pmid vs date, etc.
- 32. OpenURL Quality Index: vendor rating
- 33. Work in Progress Element weighting still in progress: E.g., importance of identifiers (doi, pmid) vs bibliographic data (issn, volume, spage). Currently, IOTA focuses on OpenURLs from citation sources only. OpenURL quality is also influenced by: knowledge base, resolver, full-text provider (target). High "completeness" score of OpenURLs not always indicative of "success" in linking to full texts Combination of multiple indexes along linking nodes may provide more complete picture.
- 34. IOTA & KBART: NISO working groups IOTA Deals with issues specific to OpenURL linking; Seeks improvements in OpenURL elements used by: OpenURL providers. KBART “ Knowledge Bases And Related Tools” Deals with data issues at the KB level Seeks improvements in data exchange practices between: content providers (e.g. OpenURL providers); product vendors (e.g. link resolver vendors); subscription agents.
- 35. IOTA & KBART: related through OpenURL IOTA node: analyzing data sent from OpenURL source to link resolver. KBART node: creating formatting best practices for data sent from content providers to knowledge base (and link resolver) vendors.
- 36. KBART-IOTA joint initiative KBART-IOTA node : Exploring together the third source of failures: link-to (or target) syntax and behavior which couples link resolvers to content providers Collaboration begun in March 2011 is meant to address OpenURL quality in a broader context.
- 37. How can I get involved? If you are an OpenURL provider: Contribute data to IOTA Review the IOTA data This data is meant to help you make improvements in your OpenURL linking If you are a librarian: Contribute data to IOTA Spread the word to vendors about IOTA OpenURL data contact: Adam Chandler, alc28@cornell.edu
- 39. Questions? http://openurlquality.niso.org http://www.niso.org/workrooms/openurlquality @nisoiota on twitter Rafal Kasprowski Electronic Resources Librarian Rice University, Fondren Library MS 44 Houston, TX 77005 [email_address] http://www.slideshare.net/rkaspro