Improving access to special collections by automating descriptive metadata creation
- 1. Improving Access to Special Collections by Automating Descriptive Metadata Creation Anna Neatrour, Metadata Librarian Jeremy Myntti, Head of Digital Library Services Betsey Welland, Manuscripts Archivist Jessica Breiman, AV Archivist #ula2016
- 2. Covering today Background information about Special Collections and Digital Library Services Overview of ways we are processing data: • Extracting data from finding aids for item level description in digital collections • Extracting and Analyzing Names/Subjects in EAD • EAD to MARC record transform through MARCEdit • Techniques for working with legacy data Resource page where you can download examples P0790 Shipler Studio Photograph Collection
- 3. Special Collections at Marriott Library • 6 departments: Print & Journal, Rare Books, Manuscripts, Photos, AV, University Records • 4 depts. produce finding aids; each department produces own finding aids even if items come from same donor • Finding aids published in Archives West; 4,000 and counting • Onsite storage; storage in Automated Retrieval Center; and offsite building P0244 Olive Woolley Burton Photograph Collection
- 4. Special Collections at Marriott Library • Despite the size of our collections, we have no Archival Management System • Finding aids encoded by hand in XML • Former process: EADs were printed out and hand-delivered to librarians in cataloging • Doesn’t make sense for metadata librarians to recreate item-level work that has already been done. • Sought help of XSLT and MARCEdit wizards! University of Utah Archival Photograph Collection; Departments -- Computer Science
- 5. Technical background • XSLT - Extensible Stylesheet Language Transformation - Used for transforming XML documents into other formats • Excel formulas - allow you to extract and concatenate data in a variety of ways • Low cost/free tools like Oxygen and MarcEdit • Examples, tips, and learning resources at the end of the presentation! P0206 Rocky Mountain Power and Light Company Photograph Collection
- 6. Extracting descriptive metadata from EAD Finding Aids P0790 Shipler Studio Photograph Collection
- 7. Finding Aids
- 8. Container area of EAD = item level metadata
- 9. Structure of EAD makes it easy to extract data
- 10. XSLT (Extensible Stylesheet Language Transformations)
- 11. Using Oxygen to extract metadata Currently we process this file with formulas in Excel We could rewrite the XSLT to handle this sort of transformation too
- 12. Excel for additional processing See our sample spreadsheet for formulas you can repurpose!
- 13. Extracting and Analyzing Names/Subjects in EAD P0016 Pro-Utah, Inc. Photograph Collection
- 14. XSLT to extract data
- 15. Create spreadsheets of the data
- 16. Deduplicate to see most commonly used values
- 18. Uses for this type of data ● Identify NACO work to be completed ● Find inconsistencies in name/subject usage ● Identify typos or other problems to fix P0413 Alan K. Engen Photograph Collection
- 19. EAD to MARC using XSLT and MarcEdit P0206 Rocky Mountain Power and Light Company Photograph Collection
- 20. Generating MARC records from EAD Since we don’t have an archival management system to handle this automatically, this is a functional workaround. Take EAD, run transform in MarcEdit with local stylesheet, edit the resulting draft MARC record to meet local standards.
- 21. XSLT to transform XML to MARC
- 22. Generating MARC records from EAD
- 23. Use MARC Tools to load custom XSLT
- 24. Generate MARC record for editing
- 25. Working with Legacy Data P0305 University of Utah Archives Photograph Collection – A-Fa. -- Thomas Stockham
- 26. Working with legacy data • Improves access to collections • Gives archivists more accurate data on items/formats in the collection • Helps archivists assess which items most in need of preservation/digitization University of Utah Archival Photograph Collection; Departments -- Computer Science
- 27. AV Archives Legacy Metadata (the “before” shot) ● Hundreds of documents ● Still in WordPerfect format ● Somewhat structured
- 28. A little better with Notepad++
- 29. Even better with Excel...
- 30. Contents List in EAD Item level, no series, boxes, etc.
- 31. Conclusion • Link to resource site: https://sites.google.com/site/specialcollectionsmetadat • Repurposing data can help streamline processes and speed up descriptive metadata creation. • Eliminate time spent reformatting or copying and pasting information. • Doesn’t require a great deal of technical background to implement these solutions. • If you are doing a great deal of copying and pasting there is probably an easier, more efficient way of doing this work. • Google your problem, you will be surprised at the helpful resources you can find. • http://www.libraryworkflowexchange.org/ - collects resources in this area. P0244 Olive Woolley Burton Photograph Collection
Editor's Notes
- #2: Jessica Breiman
- #3: Jessica Breiman
- #4: Jessica
- #5: Jessica Breiman
- #6: ANNA
- #7: ANNA - I’m going to talk a little bit here about how we use the existing descriptive information that we have in EADs in order to create item level descriptive information in our digital library.
- #8: ANNA Here’s an example of one of the EADs that has been used for this process
- #9: ANNA when we want to repurpose EAD data, we are mainly looking at wanting to use items from the container level.
- #10: ANNA Who doesn’t like looking at XML! Here’s another view of the data before it is repurposed.
- #11: ANNA - XSLT for this is pretty simple, grabbing the data in specific tags from the EAD file. We will give you our sample XSLT to download at the end of the presentation. We do this type of transformation work in the XML editor Oxygen.
- #12: ANNA - Example of text file output from Oxygen.
- #13: ANNA Import text file into excel, then use formulas to split out the data. Like putting creator name in last, first format, extracting out the birth date, capturing extent. All of this was originally in one paragraph in the container list..
- #14: Jeremy’s notes Since we don’t have an archival management system that keeps track of our EADs and everything is hand coded XML, it is difficult to keep up to date with standardizing the forms of names and subjects used in these EADs. We wanted to find a way to extract the names and subjects from all 4000 EADs so we could analyze them to find inconsistencies, to reconcile them against LCNAF and LCSH, and to discover the most often used names/subjects.
- #15: Jeremy’s notes This XSLT will extract the different types of names and subjects from the EAD control access fields, including the type of heading (e.g. personal name, corporate body, topical subject, form/genre, etc.), the controlled vocabulary that the heading should reside in (LCNAF, LCSH, ArchivesWest), and the cataloging rules used (AACR2, RDA, DACS).
- #16: Jeremy’s notes After running that XSLT on all of our EADs and compiling it into one large spreadsheet, we had something like this. After sorting the data, we could see where duplicate headings should have been used, but there were minor variations such as missing a death date or including different subdivisions with the subjects.
- #17: Jeremy’s notes We then deduplicated the spreadsheet to get a count of how many EADs use each of the different headings. This was interesting to see so we could discover what the most commonly used terms were for names and subjects in our records.
- #18: Jeremy’s notes We then used OpenRefine to reconcile the data against the Library of Congress Name Authority File and Library of Congress Subject Headings. This helped us identify the names that may have been updated in the LCNAF from AACR2 to RDA that need to be updated in our EADs. For example, the first name listed in this spreadsheet is “Smith, Joseph, 1805-1844” but since that LCNAF record has been updated for RDA, we knew that there were 28 EADs that needed “Jr.” added to the heading.
- #19: Jeremy’s notes Some of the major outcomes of this project were that were were able to identify names that we could potentially do NACO work on if they didn’t reconcile to a record in the LCNAF, we were able to identify inconsistencies in records where the same name or subject have been used but in different forms, and we were able to find simple typos or other problems that could be fixed to make the data cleaner.
- #20: Betsey:
- #21: Betsey: Since we do not have an archival management system to handle the generation of MARC records from EAD, Anna helped to create a workaround that utilizes a software program called MarcEdit. If you are unfamiliar with this program, it was created by Terry Reese and is opensource. There will be a link available if you would like to gain access to the program.
- #22: Betsey: Anna also performed the task of creating the XSLT stylesheet that will help generate the MARC record. This, in my mind since I am not familiar with XSLT is the hardest step of the entire process. Luckily, Anna created one for us that we would be happy to share with anyone if you would like an example.
- #23: Betsey: Once the program has been installed, and you have both your XSLT stylesheet and your Encoded Archival Description (EAD) xml file, simply select the MARC tools option.
- #24: Betsey: Using the MARC Tools, make sure that within the “Functions” option that your specific stylesheet is selected and that the “input” file is the EAD file that you would like to transfer and the “output” file will be saved as a MARC file.
- #25: Betsey Once the MARC file has been generated it is essential to work with a cataloger who will perform quality control over all files. Currently, I work with a cataloger who reviews the files that I generate. He checks the file, includes content if necessary, and makes corrections, and then uploads the file to Alma to be discoverable on Primo along with WorldCAT.
- #26: JESSICA
- #27: JESSICA
- #28: JESSICA
- #29: JESSICA
- #30: JESSICA
- #31: JESSICA
- #32: Jessica