A comparison of two digital libraries based on pre-established criteria
- 1. Stephen J. Stose June 19, 2010 1 IST 676: Assignment 2 In what follows is a general outline for a method of evaluating digital libraries. This paper has two goals: 1) to refine and continue to develop effective criteria for the evaluation of digital libraries, and 2) the use of these criteria in the practice of evaluating two specific digital libraries. Two federated digital libraries are evaluated across five criteria of evaluation. The criteria were adapted from multiple sources, most principally from that of Saracevic (2005) and Choudhury (2002). While the criteria presented here do not necessarily follow the structure of these previous authors, they do attempt to incorporate and re-organize the same multiple dimensions generally accepted as essential. These dimensions are listed in bold, each followed by a series of questions that attempt to manifest each concept. They do not constitute orthogonal dimensions. Yet they do seek to represent psychologically distinct experiences of a typical user’s episode with the digital library resource. For this reason the criteria attempts to model this user flow. That is to say that upon entry, the user has 1) a first impression, followed by 2) a judgment of its aesthetics and overall design, which incorporates signposts (links etc) that allow/disallow its 3) usability; entry to the digital objects themselves is facilitated or constrained by the exposure and effectiveness of the sites 4) taxonomic system, which lead to the 5) digital content itself a user hopefully can contextualize and learn from. Such a description of an episode is of course a user-centric ideal, and exposes the author’s own preferences towards good digital library design (i.e., how he believes a site should foster navigation and resource discovery). However, it also serves as a model for further research, given that these propositions regarding user-centric episodes and the criteria by which they should be judged are themselves testable and hence falsifiable. University of Wyoming Digital Initiative The Arizona Memory Project http://digital.uwyo.edu/ http://azmemory.lib.az.us/ The overall quality The front page immediately invites you A very brief time here assures you that of organization: Is into its various “recently digitized” one interface is the rule. The front page the content collections. Navigating from the main quickly divulges a lot, in a friendly immediately interface into separate collection manner. It has teasers of some of its accessible and interfaces is realistic, but there is seldom a content (usually collections inviting with clarity clear way back to the collection options. highlighted), to attract the user inside. It in its presentation? Many collections are federated within one boasts of teacher resources that, within Does its overall space (LUNA), which provides coherency seconds, show you the availability of structure serve its if the user is lucky to land there. Others lesson plans for teachers. You see stated goals? What lead one quite astray into university quickly that this is some kind of impression does the programs with uncertain connection to federated system with many institutions, first 15 seconds of this digital initiative. This fragmentation and a quick click on ‘browse’ or entry leave? is disorienting, a fact the ‘about’ section ‘search’ reveals much of the metadata at failed to allay. The search feature on the one’s fingertips. An extremely attractive front page was both ineffective and start. misleading. The portal’s overall quality is therefore very low. Design: Is the site The main page attractively The design is simple, yet effective. attractive and presented/exposed the various collections There are few items in the menu, and simple to visually available up front. If lucky, most these are repeated with short navigate? Does the collections lead to a separate LUNA explanations within the main content. design assist the workspace (a DAM), which is very More specialized features (‘help’, ‘my
- 2. Stephen J. Stose June 19, 2010 2 IST 676: Assignment 2 user in orienting its attractive, simple and customizable, and favorites’, and ‘my preferences’) are contents? Is the federates all searching for collections less conspicuous, but available design stable and within its domain. If unlucky, other immediately. The site uses only one psychologically collections lead to very distinct spaces interface throughout, and a fluid width coherent and this can be very disorienting and allows use of your entire screen. The throughout, or does organizationally distinct. The front page is site is attractive to navigate, mostly it require continual simple and visually attractive, but the because it is simple with links that visual and menu options usually just offer a series of resolve with the information they boast. conceptual re- links that lead users into a separate web The ‘help’ and ‘about’ are long html orientation as one interface, thereby visually and documents, many times unadvised in navigates? psychologically placing them again in a information architecture, but a quick new space. Thus, the design lacks an menu upfront immediately breaks up umbrella space and visual guideposts that this content by scrolling you to each organize collection subspaces and ensure question listed in its menu. Very navigational cohesion. effective, even if a trifle bland. Usability: Is the If lucky to land inside LUNA, where This is an incredibly simple and resource easy to many of the collections reside, the users effective digital library, which speaks use and effortless to must learn one mere interface in order to well of OCLC’s CONTENTdm, its navigate? Do the navigate across its collections. Links underlying architecture. This is a links resolve in resolve very quickly and the navigational federated digital library, and the links expected places, menu is stable, but a user does need to illustrate this effectively. Within one and is the user experiment a while to learn how they click of the front page you can read correctly oriented function. It is not immediately clear when about the contributing institutions, and to retrace this navigation is within or between within two clicks (‘browse>’select a space? Are collections, but this price of federated collection from the list’) you can view navigational cohesion is resolved soon enough. There the items just from that institution’s signposts stable is very effective user support, but using collection. One click on ‘favorites’ or and informative? Is this resource effectively is not effortless, ‘my preferences’ shows the user how the user aware of given its power to afford users many the interface can be changed to the various features organization powers: media groups, accommodate personalization for available with presentations, API embedding, and other improved usage. The help section is sufficient guidance options given registration. The site may available, but given the ease of this in their operation? be too powerful at the cost of ordinary interface, almost superfluous, I dare and quick presentational capacity for say. A user has no issue at all finding more ordinary user groups. what s/he needs, or just browsing. System and The main taxonomy is the collection-level A taxonomy of categories is metadata: Does a categorization. The “Search” feature on immediately available by clicking on taxonomy of the main page is useless. If within LUNA, ‘browse’. While this list is quite sparse categories enable the advanced search has excellent (broken into ‘collections’, ‘topics’, effective resource dropdown fields with extensive metadata ‘formats, and ‘time periods’ only), it discovery? Is the and Boolian options, and allows federated broadness is indicative of the metadata exposed searching within or amongst all abstraction needed to properly federate to users to enable collections. Simple searching reveals the collections of over 90 institutions. effective search? faceted options in a sidebar, expanding or This is a wonderful way to get users Do queries return refining the results. It is difficult to find into the content immediately, and listed presentable results the “browse by category” section in order beside each thumbnail is basic ‘subject’ digestible to human to immediately learn about the items and ‘description’ metadata, which users? Are queries available, but when found the “what”, allows for quick digestion of its
- 3. Stephen J. Stose June 19, 2010 3 IST 676: Assignment 2 handled efficiently? “who”, and “when” categories do provide relevancy before going further. Facets Are there faceted a needed conceptual framework for the in the sidebar allow narrowing the options for collection’s holdings that might serve the fields. When choosing an image, much continued user better if constantly available in a more metadata unfolds beneath the refinement? sidebar. Queries provide results with image itself, or when doing an advance various grid and object resize options, search. Queries are handled with item-level summaries available immediately, such that you forget you below or upon hovering over each item. are amongst 67,000 digital objects. Digital content: This single-interface federation (in The links ‘my preferences’ or ‘my Does informative LUNA) is excellent for combining high- favorites’ allows you to change the descriptive (e.g., quality items across/within the collection, screen defaults (sort order, background historical) material but contains little collection-level color, grid/list options), such that the contextualize the information, leaving little to learn about content itself appears according to each digital objects? Are the history and context of each collection. user’s desire. You learn a lot very the objects Attempts to return to the main portal for quickly regardless of the format, as informatively collection-level guidance are difficult and beneath each thumbnail are effective labeled at the item- otherwise unhelpful, as “about” section descriptors, and beneath each image and collection- discusses issues more relevant to itself the metadata is co-referenced with level, to make it university members, and the “exhibits” links to that reveal all the objects within easy to learn? Is and other menu items show links that lead that ‘category’. Browsing is fun, which the content a high to unfamiliar space. Individual items are implies you are learning and engaged in digital quality, and well documented in the left sidebar with a the content. Checkboxes beside each can users complete listing of technical, descriptive, object allows you to collect content manipulate (e.g., and administrative metadata. Users can (‘my favorites’). Image size is limited, collate, zoom, print, add/collate items to a workspace or but you can request larger sizes. The itemize) the presentation space, zoom and compare, if best feature is the item-level to objects? they care to learn these features (and collection-level coherency. Regardless register). This plethora of multimedia of how you came upon an item, it is co- features overshadows the storytelling and referenced to information regarding its contextualized nature a digital library also collection. Thus, one learns about the needs to represent. item and its context easily and effectively. Is the digital The site has potential, once loose ends are This digital library is fantastic, one of library mission collected. The site fails to serve as an the best federated sources I have used, supported? umbrella portal except that the first page and very worthy of emulation. It allows users to enter specific collections informs on the item- and collection- (with little way back, however). The level, integrates these seamlessly within LUNA federated interface is an excellent one interface, while still recognizing the multimedia tool that exhibits great control collection has a separate institutional over item-level metadata, but it fails to provenance. The mission is completely incorporate collection-level information, supported. Besides a few design issues and probably sacrifices historical regarding attractiveness (menu fonts) storytelling of the items within each and fuzzy header images, it is top-class collection for multimedia options ordinary in terms of usability. users may find difficult.
- 4. Stephen J. Stose June 19, 2010 4 IST 676: Assignment 2 A brief conclusion is warranted. Obviously the Wyoming Digital Initiative is under construction. This I know, as I know one of the directors (iSchool Graduate Ben Goldman). For this reason, the analysis may not be completely fair. However, given that I mostly concentrated on the federated LUNA system, some words of direct comparison may be granted. While the power and complexity of the LUNA system is quickly apparent, these benefits have as costs the joy of browsing I experienced within the Arizona Memory Project. Too much complexity may instill only confusion, and given that most visitors just wish to browse and perhaps do basic educational research, the power afforded within the LUNA system is perhaps a trifle unnecessary, if not at odds with its educational purpose. In my opinion, if and when a user requires high-level multimedia capabilities, these should be secondary to a digital library’s public and educational (i.e., information quality should be first). The LUNA system does raise the bar for digital library architecture. This power, however, should be secondary to a good experience. Saracevic, Tefko (2005). How were digital libraries evaluated? Presented at Libraries in the Digital Age (LIDA), Dubrovnik and Mljet, Crotia, May 30-June. Choudhury, G.S.; Hobbs, B.; M Lorie, Flores, N.E. (2002). A Framework for Evaluating Digital Library Service. D-Lib Magazine July/August 2002. Volume 8 Number 7/8