Making the Leap Towards Linked Data

Editor's Notes

• #3: First I want to give you all some background about the American Museum of Natural History. 4 city blocks … Besides space for public exhibitions, the many building that make up the museum hold offices for administration, education, communication, and research, among others.
• #4: We have 5 Science Divisions, which includes the Research Library. These divisions also hold separate departments for specific disciplines, such as Mammalogy and Ornithology, both under Vertebrate Zoology.
• #5: Within Science, there are 34.5 million specimens and artifacts, plus approximately 25,000 linear feet of archives! A lot of stuff.
• #6: We know these numbers because starting in 2005 the museum has been identifying and assessing materials for risk. The Library entered the scene during the 3rd Phase of an IMLS-funded Risk Assessment project, which started with Research collections in 2005 – specimens in jars, etc – around 2.5 million objects at that time, then moved onto objects on display in permanent exhibits (22,000 items), and for Phase 3, beginning 2011, we gathered a basic inventory of all the archives in the Museum’s science departments which included the library. For the same year, the Library was fortunate enough to receive additional funding from the Council on Library and Information Resources (CLIR) to create minimal catalog records for manuscripts, department records, photographic prints and slides. These combined efforts produced over 3,000 records for archives across the science departments. HUGE. Which gave us the legs to do our current project of writing more in-depth finding aids and contextual notes for expeditionary field work.
• #7: The finding aids provide more detailed descriptions for archives collections about field work supported by the museum. Using the inventories created from the IMLS project, we were able to identify and prioritize collections from the Science Departments to process. EXPEDITIONS EXPEDITIONS EXPEDITIONS This project has created about 50 finding aids close to 100 rich entity records, as well as over 3,000 minimal EAC records for expeditions and personnel. Minimal = birth and death dates with brief notes about affiliation to the museum for persons, places and personnel along with purpose of research for scientific expeditions.
• #9: Here are examples of the kinds of material people brought back to the museum from the field. Documentation not only on various formats, but also for various disciplines. Back in 2011 when we were still at the stage of gathering basic data about collections, it became clear that archives from expeditions dispersed into their respective departments with no reference to material outside their silos. Some material came to the library, but there was a lot of important documentation scattered around the museum, by and large, documents that support Science collections, ie Specimens. We saw an opportunity to make connections across departments, and virtually restore provenance while providing some meaningful context. Great potential for discovery within the institution of these related resources. Biodiversity Herpetologist making note of weather conditions, provide valuable information for climate change
• #13: So this is great! We can create a virtual network of archives through its people and expeditions using EAC as the hub.
• #14: And how do we encode all of this descriptive data into EAC and EAD? Our interns work from templates built in accessible programs, such as Word and Excel. Finding aids, once completed and reviewed, get input into Archivists’ Toolkit where EAD xml can be simply exported. For lengthy container lists we use an Excel template to wrap the information in EAD tags. The encoded container lists then goes into a blank EAD xml file that can be ingested into Toolkit and merged with its parent collection. Methodology thanks to PACSCL (Philadelphia Area Consortium of Special Collections Libraries)! A consortium of Special Collection Libraries in Philadelphia. EAC is done a little differently. Our EAC template uses a macros (thanks to Nick!) that converts table data into structured and valid EAC-CPF xml files. Further schema validation is done in oXygen and for purposes of readability, I wrote a very basic stylesheet in xslt to transform the EAC into html that can be read and opened in any web browser.
• #15: We have a way to create descriptive data and encode it in the metadata standards we chose for our records. And we can store those files temporarily on the network or AT. What we need is a tool for managing those records over time and providing a public access point so that they can be discovered.
• #16: Ideally the management system would ingest our xml files and offer various administrative features in order to manage the records as they are enriched or added to over time. One of our current challenges is managing workflow and file versioning. A good system would alleviate our file management woes!
• #17: The CLIR project team along with other Library staff came up with a set of Functional Requirements for an Archives Management System. We outlined what we needed for Data Creation, Management, Dissemination, and Technical Administration. Using this as a roadmap, we systematically analyzed a few systems. These were ArchivesSpace, AtoM (Access to Memory) and xEAC which I will get to in a minute. What we discovered is that neither ArchivesSpace nor AtoM have enough support for EAC to adopt them for entity records, though they both have good support for EAD finding aids.
• #19: Despite ASpace’s lack of EAC support, for collection descriptions and accessions, we preferred it over AtoM. Because of it’s lack of EAC support, we decided to implement xEAC along with it. A suite of apps for archives. New funded efforts to push all Special Collections descriptions into ASpace, including the inventories from Science created by the 2011/12 IMLS project.
• #20: We are investigating RDF Triplestores. xEAC publishes RDF out of the box; we may be able to use a stylesheet such as ead2rdf for ead conversion.
• #21: Data is positioned to connect important context to affiliated objects scattered throughout the museum.
• #22: With xEAC and ASpace, we can make meaningful links between entities and archives based on the basic inventories, full finding aids (which is even better since they offer more description of a collection’s contents), and eac records, both fully and minimally described.
• #23: We have loftier goals where linked data is key. As more EAC records become available, we can start to widen our scope to make links with other collections in the Library, like the digital images in Omeka, or within the institution linking to specimens in the Museum science departments. Beyond that we’d also like to share our EAC records with the wider community by submitting our entities to SNAC, an International Authority Cooperative, where further connections can be made outside our institution.
• #24: In the future, we hope to provide a platform that allows other AMNH departments to use our linked data to connect any object in the museum …. Work on this. And now Jen is going to get more specific about how we can link assets in the library.
• #25: AS Iris mentioned, linking resources is THE DREAM. With the implementation of xEAC and Archives Space, the library is truly becoming a repository of relationships. Through standardization we can identify broad categories within the Museum’s history and then branch out to related content across various systems (read the list). The library can be a source of standardization of these entities and demonstrate the value of speaking the same language. We have started to work with other departments to get their working lists of these entity types to combine and standardize them with ours. We have also started implementing this vocabulary sharing in our department - standardizing controlled vocabularies for the above entities for image cataloging, archives management and bibliographic cataloging makes linking related content easier.
• #26: Here is a use case for using linked data and controlled entity names within the library using The Whitney South Sea Expedition. The Whitney South Sea Expedition of the American Museum of Natural History remains a monumental undertaking of focused natural history specimen collecting. Active research spanned from 1920 to 1941. The many participants of the expedition were tasked with systematically exploring Polynesia, Melanesia and Micronesia for previously undiscovered or little-known bird specimens.
• #27: Before wide adoption by the museum as a whole, the library must be able to demonstrate how the EAC framework can be used to link various resources in our own department. Here is an overview of the applications the Library currently uses to manage it’s assets and data. The Library has traditionally relied on custom built applications to meet the requirements for various digital projects, but this model became unsustainable. The library has slowly evolved to an Open Source application shop. We look for open source solutions that meet the majority of our requirements, have a robust, supportive and active user community, support the metadata standards we use, and utilize APIs. In the Whitney South Seas example, I want to demonstrate how assets and data from all of these applications can be related to each other using EAC records as an authority resource.
• #28: The following diagram demonstrates how each resource becomes a potential gateway for wider and deeper research for the scholar and the casual user. In this example, a photograph of American Ornithologist, Rollo Howard Beck and ship engineer in workshop with frigate bird specimen, aboard schooner the France during the Whitney South Sea Expedition is connected to the EAC record for the Whitney South Sea Expedition and EAC record for Rollo Howard Beck. Users interested in Rollo Beck can then find related resources through the EAC record two way links will lead the user to Dspace and Beck’s Journal from the Whitney South Sea Expedition and back again. In Sierra, the library’s online catalog, there are bibliographic records for the France’s ship logbook and for Beck’s Journals, both of which would link back to the EAC records for the Whitney South Seas Expedition and Rollo Beck. A blog entry in the WordPress site is also discoverable through the Whitney South Sea Expedition. Lastly there is an EAD Finding Aid for Hall and Diorama Construction at the AMNH and the Whitney Hall of Oceanic Birds EAC record would link to that. For this example, we tried to identify the specimen being handled by Beck in the image – Through this framework, if the specimen was identified in a photograph, it could be linked to a specimen in one of the dioramas in the hall if it was on display or to a specimen record in KE-EMU the museum’s collection management software
• #29: Traditionally, the library has relied on physical servers to deploy applications and databases. With xEAC and Archives Space, we partnered with our IT department to determine what would best fit our requirements. We were able to get a virtual machine up and running relatively quickly. The IT Dept. installed required software packages via RPM repos and I installed the Archives Space and xEAC. The next phase of the project will be to research and select a Triplestore Triplestores store semantic facts in the form of subject - predicate - object using the Resource Description Framework. RDF is a standard model for data publishing and interchange on the Web. RDF has features that facilitate data merging even if the underlying schemas differ.
• #30: Ideally would would like to push the EAC record and EAD records to other museum departments Nick is now going to talk about how …
• #31: This entire process is just part of a larger one to build towards better long-term information management, or as we have nicknamed that mythical ideal, OAIS. I’m sure everyone is familiar with this diagram and how it flattens our incredibly complex problems into neat shapes and lines. What we’ve been presenting here is just one of those complex problems, and I’ll use the OAIS diagrams to help explain how we’ve thought about it.
• #32: So in an ideal OAIS there is are producers creating information, users consuming information, and an organization in the middle that manages, describes, and presents that information in ways that preserve that information long-term. In this diagram, I’m using AMNH as the organization. Ideally, it’s as simple as that, but an organization like AMNH can’t be contained in those six boxes
• #33: Instead, as Iris and Jen have shown, we are a hydra made of many, many departments each handling its own data. Just for metadata, Anthropology uses Oracle, Earth and Planetary Sciences uses its own systems, other departments use KE Emu, and in the library there’s another multitude of systems. Omeka for images, xEAC for EAC-CPF, ASpace for archival collections, DSpace for digital objects, etc. Each department has very specific disciplinary needs in order to manage its information in a way that helps them accomplish their goals best.
• #34: This runs into a problem when we get to the access side of our systems. Now, users aren’t interacting with the American Museum of Natural History, they’re interacting with each department or collection. If we’ve all developed custom information management systems, it shouldn’t be surprising when each has its own access mechanism.
• #35: In order for our audiences to be able to access our collections, they need to understand both where to find the access mechanism and how to use. And our users aren’t just visitors to the museum’s galleries. We have online users planning a visit students researching a topic scientists studying specimens staff creating exhibits the list goes on and the access mechanisms multiply. Users are no longer interacting with the museum as a single authoritative resource, but with its tangle of constituent parts.
• #36: To fulfill our goals to make collections accessible, we need to make access simpler. At the same time, we can’t force everyone’s worfklows to suddenly shift into a single platform. Those workflows developed to deal with practical problems and are now deeply embedded in each department. It’s too big of a task to create that sudden shift. We have to be flexible to our current systems, and our future ones. When new workflows and systems are created to respond to new needs, our existing systems should be able to accommodate those changes. Which is why we’re looking at linked data. Of course, this diagram is a little disingenuous. We don’t expect users to be fluent in SPARQL, RDF, and data manipulation. Instead we’re looking at using linked data to provide the backbone for other access services. Instead of maintaining separate access services for each system, now we can start evaluating access services built for different audiences.
• #37: I’ll return to the original monolithic diagram. As Jen and Iris showed, collection context can be distributed across the museum. It makes sense that we try to make it possible to discover that context, rather than requiring users to know where additional clues could exist. To make this possible, linked data becomes a key component of the data management strategy. We still haven’t addressed how we might change ingest, storage, and access mechanisms in ways that preserve departmental needs and and simplify overall management. However, those solutions will depend on the links created within the metadata.