SPEC Kit 303 in the UK and Ireland:
Download as PPT, PDF1 like433 views
The document presents a survey on performance measurement and assessment practices in SCONUL member libraries in the UK and Ireland, indicating a high engagement with performance measurement since the late 1980s. It highlights a reliance on user surveys and various assessment methods, alongside the need for further training in data analysis and survey design. Comparative observations reveal differences in governance and culture between ARL and SCONUL libraries, emphasizing a commitment to delivering value through systematic evaluation and assessment practices.
SPEC Kit 303 in the UK and Ireland:
- 1. SPEC Kit 303 in the UK and Ireland: a survey of performance measurement and assessment practice in SCONUL member libraries Selena Killick, Tracey Stanley and J. Stephen Town
- 2. SPEC Kit 303 in the UK and Ireland
- 3. Summary • Background, methodology and approach • Findings • Comparative observations • Conclusions
- 4. Background, methodology and approach
- 5. Origins and process • Web survey of performance measurement and assessment activities in academic and research libraries • The ARL SPEC Kit 303 on Library Assessment was published in December 2007 • 60% response rate amongst 123 ARL libraries • Conducted by Stephanie Wright and Linda White
- 6. Rationale for UK version • The aim in both cases was to provide ‘an overview of precisely how library assessment activities are being implemented and developed’ within member libraries • Assistance with best practice for developing performance measurement programmes • Awareness of tools, techniques and structures • Direct comparison between ARL and SCONUL libraries
- 7. Findings
- 8. Sample & characteristics • 77 libraries (43% of SCONUL membership but 60% of University institutions) • Majority engaged with PM from late ‘80s onwards • User surveys were first assessment activities in most cases • Rationale was internal and user driven
- 9. PM Activities in use Range of 3-19 of listed methods; median of 10; average of 10.6 • Statistics (96%) • Suggestions (91%) • Data mining (72%) • Outcome evaluation (67%) • Benchmarking (63%) • KPIs (63%)
- 10. Least used • Value/ROI assessment • Impact assessment • Balanced scorecard • Physical orientation studies • Mystery shopper studies
- 11. Functions assessed Every one of 27 library functions reported as assessed by at least six respondents • Enquiry services (92%) • Electronic resources (92%) • Circulation (89%) • Acquisitions, ILL and Web site (all 84%) • Information literacy and online catalogue (82%)
- 12. Organisation • 1 respondent has a f/t coordinator • 26% have p/t coordination • 25% through Committees • 9% within a specific department • Majority of posts and committees created since 2000
- 13. Outcomes and improvements include … • Opening hours most frequent improvement • Web site • IT facilities • Reshelving processes • E-resources • Space • Staff structure
- 14. Strategy and development • 79% have strategic commitment to evaluation, and most have a plan … but 51% have no particular training • Further training needed on – Data analysis tools (Atlas ti) – Understanding survey techniques – Survey design methodology
- 15. Culture of assessment • Results used to improve library (75%) • Evaluation for service quality (69%) • Assessment is a library priority (67%) • Staff development is adequate (13%) • Staff have necessary skills (26%) • Staff accept responsibility (34%)
- 17. ARL & SCONUL ARL • North America (US & Canada) • Selective membership of large scale research libraries • 123 members • Tradition of measurement SCONUL • The British Isles (UK & Ireland) • Inclusive membership of all higher education institutional libraries • 180 members (=129 Univ) • Tradition of measurement
- 18. Basic comparisons SPEC Kit 303 • 73 of 123 (60%) • 99% active • 91% customer driven • 29% accreditation driven • Majority survey first • Improvement 76% • No particular training 29% UK & Ireland • 77 of 129 (60%) • 100% active • 84% customer driven • 9% accreditation driven • Majority survey first • Improvement 75% • No particular training 51%
- 19. Variation in methods • User interface usability testing figures strongly within ARL libraries, and used frequently to test web sites (the most assessed area) • Internally developed surveys used widely in the SCONUL sample, including for the web site
- 20. Organisation • More full time coordinators in ARL (16% vs 1) • More departments charged with assessment (13% vs 9%) • Fewer part time and adhoc committees in ARL, although adhoc teams a feature in both contexts
- 21. Development and culture • 71% support for training in ARL • Strong senior management commitment in both, but not necessarily translating to the organisation as a whole in either
- 22. Conclusions
- 23. Speculative reasons for divergence • Governance differences – UK Public service context for advocacy and reporting • Quality assurance pressures – NSS and other review pressures accentuate particular aspects of library performance (at the expense of others?) • Technique availability – Variation between ARL and SCONUL initiatives and products • Culture – Depth of ‘research’ and data reliance in US? – Local creativity and pragmatism in the UK & Ireland?
- 24. Conclusions • SPEC Kit approach was transferable to the UK & Irish context (and potentially beyond) • Revealed details of performance measurement and evaluation in this context • Provides a tool for international comparison • ‘Assessment’ not recognised as a synonym for Performance Measurement, but this did not affect responses
- 25. Afterword The richness of data and the range of activities described on both sides of the Atlantic demonstrate a very strong commitment in libraries to delivering value to their communities, through measurement and assessment, and an enthusiasm for using any techniques which will assist in the process of developing a customer focused culture