Measure All the (Web Archiving) Things!
- 1. Measure All the (Web Archiving) Things! Nicholas Taylor Web Archiving Service Manager Stanford University Libraries Archive-It Partner Meeting August 18, 2015
- 2. how many more websites are we archiving? “Library_01.jpg” by British Library
- 3. crawl report list Archive-It: “Crawls for Account #198”
- 4. seeds for individual crawl Archive-It: “Seeds for Crawl #99435”
- 5. download seed list Archive-It: “Seeds for Crawl #99435”
- 7. whew, that was easy!
- 8. oh, wait a minute… seed lists are per crawl well, how many crawls are there? • 6 accounts • oldest active since 2007 • 30+ collections • hundreds of crawls
- 9. count and average not enough • seeds move in and out of crawls • seeds have different frequencies • new seeds w/ new URLs for old seeds • “university website” is many seeds plus • non Archive-It web archiving activity “Dichotomic Maples” by francoismi under CC BY-NC-SA 2.0
- 10. “what gets measured, gets managed” “Gudauri still life” by Carsten ten Brink under CC BY-NC-ND 2.0
- 11. why measure? • advocacy/outreach • service modeling • program assessment • policy making • staffing assessment • grant support • prioritization • risk assessment “Measuring river depth” by epeirogenic under CC BY-NC 2.0
- 12. what to measure? • How to handle the data volume? • What is the usage of web archives? • How much does web archiving cost? • How to assure the quality of archived content? • How to secure institutional buy-in? • How much loss have resources suffered? • What is the impact of policy requirements?
- 13. community-valued metrics 0% 10% 20% 30% 40% 50% 60% Volume Usage Cost Quality Buy-in Loss Policy Percentage of organizations NDSA: “Web Archiving in the United States: a 2013 Survey”
- 14. volume • websites – captured – preserved – described • data – captured – preserved • objects – captured – preserved “typography jumble” by Bill Dickinson under CC BY-NC 2.0
- 15. usage • web analytics – visitors – visits – referers • actual use cases (who + how many?) – research – teaching – institutional legacy – compliance “113/365 Days: A page from my heart” by LaughingRhoda under CC BY-NC-ND 2.0
- 16. cost • external – out-payments for web archiving services – quota utilization • internal – staff time, by activity – storage “Largest square from a dollar bill” by origami_madness under CC BY-NC 2.0
- 17. performance • accessioning throughput • service request turnaround • collections/websites w/ discovery records • time to regenerate full-text index “Lower rack” by Andy Melton under CC BY-SA 2.0
- 18. community-valued…metrics? 0% 10% 20% 30% 40% 50% 60% Volume Usage Cost Quality Buy-in Loss Policy Percentage of organizations NDSA: “Web Archiving in the United States: a 2013 Survey”
- 19. “not everything that counts can be counted” “Ten Floods, Twenty-Five Trees, Nineteen Bubbles...” by Flood G. under CC BY-NC-ND 2.0
- 20. quality • use case-specific? • benchmark to ideal or to limits of tools? • quantifiable metrics? • existing metrics as proxies for quality? • sampling approach? • not just missing content but also collected junk NYARC: “I. Introduction - NYARC Documentation”
- 21. buy-in • unique nominators? • projects w/ web archiving component? • budgetary commitments? • resource commitments? • charge for service? • testimonials? “The Play” by Ryan Hyde under CC BY-SA 2.0
- 22. loss UK Web Archive: “Ten years of the UK Web Archive: What have we saved?”
- 23. policy • first capture under embargo • opt-out requests • takedown requests • external environment “We apologise for any convenience - Update” by Alan Stanton under CC BY-SA 2.0
- 24. better measures, measuring better “Line Art Project #2 VIS3 UCSD” by Mandy Jouan under CC BY-NC-ND 2.0