RAMP: Repository Analytics and Metrics Portal
Download as PPTX, PDF1 like1,194 views
The document discusses the development of the RAMP (Repository Analytics and Metrics Portal) to improve the accuracy of usage metrics for institutional repositories. It highlights issues with existing web analytics tools, particularly Google Analytics, which underreport citable content downloads. The presentation outlines new methodologies and features of RAMP designed to enhance reporting and provide consistent metrics for academic libraries.
RAMP: Repository Analytics and Metrics Portal
- 1. Kenning Arlitsch, Dean @kenning_msu Patrick OBrien, Semantic Web Research Director @sempob RAMP: Repository Analytics and Metrics Portal Accurately measuring use of institutional repositories ARL Assessment Forum Atlanta, GA, January 20, 2017
- 2. Agenda 1. A New Reporting Model 2. Accuracy of Analytics Reporting Tools Page Tagging Log file 3. RAMP New Prototype Web Service for IR reporting
- 3. A New Reporting Model Page Type Definition Examples Citable Content Downloads Non-HTML scholarly content that may be formally cited in the research process ● Publication (.pdf) ● Presentation (.ppt) ● Data Sets (.csv) Item Summary HTML pages to help user decide to download the full publication ● Title & Abstract ● Item Metadata Ancillary HTML pages that provide general information or navigation ● Search Results ● Browse by Author ● Statistics
- 8. Accuracy of Analytics Reporting Tools
- 9. Two Classes of Web Analytics HTML Analytics Service (SaaS) 1 Log Files 2Page Tagging {JavaScript}
- 10. Page Tagging Page Tagging does not track non-HTML CCD HTML Non-HTML
- 11. Show Google Scholar direct link graphic
- 12. Google Analytics use in academic libraries Tested 279 academic library websites ARL DLF OCLC-RLP 90% US libraries contained Google tracking code
- 13. 51% 54% 52% 63% 35% 27% 20% 20% 14% 19% 28% 17% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% ~51% to 63% of Citable IR Activity is Unreported by Google Analytics 01/05/16 - 05/17/16 (days=134) Citable Content Downloads Item Sumary PV Ancillary PV Most IR activity is Citable Content Downloads
- 14. 54.3% 46.2% 47.9% 42.5% 45.7% 53.8% 52.1% 57.5% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Most IR Activity is Unreported by Google Analytics 01/05/16 - 05/17/16 (days=134) Reported Invisible Most IR Activity Unreported by Google Analytics
- 15. Web Analytics Accuracy Risks Risks Analytics Method Area Types Page Tagging Log Files OverCount Visits Low High Downloads Low High Page Views Low Low UnderCount Visits Medium Medium Downloads High Low Page Views Low Low
- 16. Data set: Jan 5 - May 17, 2016 (n = 134 days) Study Participant IR Platform URL Montana State University ScholarWorks DSpace scholarworks.montana.edu McMaster University MacSphere DSpace macsphere.mcmaster.ca University of New Mexico LoboVault DSpace repository.unm.edu University of Utah USpace CONTENTdm uspace.utah.edu
- 17. Page Tagging does not track non-HTML Citable Content Downloads Non-HTML Search Analytics Does!
- 18. Typical Page Tagging (GA) Data Types Page Type Analytics Pages Events Citable Content Downloads - ✅ Item Summary ✅ - Ancillary ✅ -
- 19. Montana Method Includes Google Search Console Page Type Analytics Search Console Pages Events Search Analytics Citable Content Downloads - ✅ ✅ Item Summary ✅ - - Ancillary ✅ - -
- 20. GA Ancillary Page Views and Item Summary Page Views vs CCD
- 21. All four IR using Google Analytics Page Type Analytics Search Console Pages Events Search Analytics Citable Content Downloads - 26,355 562,933 Item Summary 284,303 - - Ancillary 201,793 - - CCD Tracking Improvement 2,000%
- 22. Montana Method Challenges ❖ Missing non-Google direct link CCD events ● Yahoo ● Bing ● Email ● FB ❖ GSC limits time and access ● Moving 90-day window ● Granular data = programing skills to access API
- 23. RAMP: Repository Analytics & Metrics Portal
- 24. RAMP - Repository Analytics and Metrics Portal Cloud Web service Currently accessible and free During grant period (through November 2017) No training or configuration required Consistent method and terminology Benchmarking across time and organization Request access - mixterj@oclc.org
- 27. Publications Published: Patrick OBrien, Kenning Arlitsch, Leila Sterman, Jeff Mixter, Jonathan Wheeler, and Susan Borda. “Undercounting File Downloads from Institutional Repositories,” Journal of Library Administration, vol. 56, no. 7, 2016 Forthcoming: Patrick OBrien, Kenning Arlitsch, Leila Sterman, Jeff Mixter, Jonathan Wheeler, and Susan Borda. “RAMP: Repository Analytics and Metrics Portal: A Prototype Web Service that Accurately Counts Item Downloads from Institutional Repositories,” Accepted by Library Hi Tech, expected early 2017 Proposal funded by IMLS: ”Measuring Up: Assessing Accuracy of Reported Use and Impact of Digital Repositories” - scholarworks.montana.edu/xmlui/handle/1/8924
- 28. Undercounting Research Team ❖Montana State University ●Kenning Arlitsch, Dean @kenning_msu ●Patrick OBrien, Semantic Web Research Director @sempob ●Leila Sterman, Scholarly Communication Librarian @calamityleila ●Susan Borda, Digital Technologies Librarian @mutanthumb ❖OCLC Research ●Jeff Mixter, Software Engineer @jeffmixter ❖University of New Mexico ●Jonathan Wheeler, Data Curation Librarian