Summon Post Implementation Interviews
1 like92 views
This executive summary presents findings from a usability study on the 'Summon' webscale discovery layer, highlighting user experiences from interviews with seven members of the KAUST community. Participants primarily prefer Google Scholar for article searches due to perceived relevance and faster results, finding Summon results cluttered and overwhelming. Recommendations include improving the library website's structure, enhancing user training, and adopting features like bento box searching to better cater to user needs.
Summon Post Implementation Interviews
- 1. Summon Post Implementation Interviews Study - Executive Summary Prepared by Rindra Ramli. E-Resources Specialist (Lead) University Library. Executive Summary: This usability study (interviews) was conducted to elicit information on our community’s search experience using Summon. Summon is our new webscale discovery layer that was launched in May 2016. A total of 7 KAUST community members participated in the study: • Faculty Member: 1 • Students (PhD and MSc): 5 • Postdoc Researchers: 1 Methodology: • Email invitations were sent out to the community. • ‘Guerilla’ tactic was also deployed where the interviewer roved around the library during peak period(s) to find willing participants to be interviewed. Participants were given Tamimi Food Vouchers as a token our appreciation. Findings: • Google Scholar is the go-to source for articles. According to the participants, they obtain more relevant and faster (search return speed) hits compared to using Summon • Most of the participants searched for articles as compared to ebook / ebook chapters when using Summon • Most used Summon to search for library’s print or electronic books collection. An interesting comment: Google Scholar is not efficient enough to provide this information for them. • Participants found that the search results in Summon are often too cluttered; giving rise to information overload. Most of them are also unaware of the filter features that allow participants to filter out the ‘noise’ from their search results.
- 2. • The search result return speed is another cause for concern (Summon). Perhaps a more directed search approach would assist our users to get their information faster. We could have tab / radio button options on our website to allow our users to search for articles; ebook / ebook chapters and print / AV materials separately. Another approach is the Bento Box search approach that has been deployed by several libraries: NCSU Library, Singapore Polytechnic Library, Duke University Libraries. Bento searching allows the user to see the results categorized into sub-divisions such as articles, websites, research guides and other resources. • Though some of the participants mentioned that they have attended our library training(s), there is not much evidence/co-relation whether this has impacted their search experience in Summon • One of the participants mentioned that we (library) should not be competing with Google Scholar as most of the users are already accustomed to it. • Another participant thought that our library website search only library website information NOT our subscribed electronic resources collections. Recommendations: Long Term • Environmental scanning of the webscale discovery field. On the area of Google Scholar and Google Books, there is a possibility of uploading our print / AV collections into Google. However, this requires further study: https://scholar.google.com/intl/en/scholar/libraries.html • Usability / user experience study (User Testing) on our library website using Camtasia recordings / Note taking. This would give us an in-depth understanding of our users’ search experience on our library website as a whole. • Card Sorting study: This study will help us understand how users categorized ‘things’ within the website. One of the major objectives of this study is to improve the information architecture of the website. • More focus group interview study. This is different from the one-on-one interviews conducted. Focus Groups may involve more participation among users and generate more useful ideas.
- 3. • Study on the impact of our library training(s) or training video(s) on the search experience of our community. • Promotional / Marketing plan: Plan(s) to promote Koral / library website as well as other library online services / features. Short Term • Forming Ad-Hoc library teams to address some of these issues: Web & Discovery team, Marketing/Promo Team etc. • Quick fixes on our library website: Include brief descriptive textual explaining available features on the website. • Quick links to our home-made videos, popular e-resources, subject specialist information among others would give ours users information at their fingertips. • Using feedback from library training(s) / informal chats, we can use this data to improve the quality and prominence of our online resources information. • Based on statistics taken from Summon, 360 products, Libguides and Libanswers, we can make short term quick fixes such as including Live chat features on our website, point users to most commonly accessed Libguides and so forth.
- 4. Appendix: Raw Data. (1) KAUST Faculty member - Past site work very well - Uses library website to search for book titles / ebook titles - Searches for known items - Uses Google Scholar: Articles o (Much faster speed) o Google Scholar: relevant results - Uses arxiv.org for pre-printed articles - Library should not be competing with Google Scholar - Doesn’t use library website to search for articles - Pattern: Search for articles 99% compared to 1% of ebooks / books (2) PhD student - Uses Library website to search for ebooks - Search for articles / ebook chapters - Uses library website instead of Google Scholar - Aware of ILLDD - Attended Library Training: How to do literature search & Endnote - Aware of the ease of searching in Google Scholar - More Library How-to videos: Searching for information - Summon Results too cluttered; too much to digest - Summon Results: Info Overload - Aware of filter feature in Summon - Felt that directed search will be more helpful than open ended - Too much noise in Summon results - Perhaps, put the filter options on the library homepage (3) PhD Student - Frequent user of library website - Much experience in using the old website - New Library website has more features - Google Scholar user - Endnote user - Most often, search for articles followed by ebooks - Prefers Google Scholar over library site - Uses Google Scholar to search for relevant articles. Uses Library website if nothing relevant turns up in Google Scholar results - Bookmark specific databases for future access
- 5. - Prefer to search ‘open search’ than deep searching - Felt that library training is about 20% effective in assisting research work - Endnote user; felt that Endnote is helpful in obtaining materials - Do we have useful streaming videos apart from JoVE and HSTalks for eg NPTel??? (4) MSc Student - Uses library website only if cannot find anything in Google Scholar - Search in Summon do not reveal relevant results - Google Scholar provides more relevant hits compared to Summon (Research area: Chemical Engineering) - Felt that Library website good for searching books, book chapters - Felt that ScienceDirect provides more relevant results - Felt that relevancy in results is a major factor when comparing Summon and Google Scholar (5) Post-Doc - Uses Summon - Article search - Initial impression that library website searches only for information on library website NOT subscribed e-resources such as ejournal articles and ebook chapters - Powerful tool but need to know how to search to be able to exploit the full features of Summon - Searchbox should have more descriptive text - Before Summon, never use the library website. Started only when Summon comes into the picture - Include advance search options on the library homepage - Search heavily for articles/papers more than book chapters - Big time Google scholar user - Not aware of Summon filter options - Lib Trainings: Not attended - But attended library orientation for staff: found this useful - Attended Endnote training class - Most often, learn from peers in the labs. - Recommend that we have more video tutorials on lib subject areas: allow them to learn on their own time and pace - Impressed with ILDD service
- 6. (6) Visiting Student - Uses library website to find books - More theoretical work - Google Scholar user to find articles NOT for books - Uses book often to get information - Summon provide deep linking to ebook chapters: Appreciate this feature - Prefer the current simplified searchbox on the library website (7) PhD Student - Google Scholar as the first stop option to search for information followed by Library website - Uses library website to search for book / ebooks - Attended library training / orientation about once or twice - Felt that there was lot of information shared during the class, but felt that none was applied in research work - Unaware of Summon filter options - Indiscriminate search