The References of References: Enriching Library Catalogs via Domain-Specific Reference Mining
0 likes408 views
The document discusses a project aimed at enriching library catalogs through domain-specific reference mining, specifically focusing on the history of Venice. It highlights the limitations faced by humanities scholars due to inadequate access to relevant literature and the proposal to utilize reference monographs for creating a more comprehensive bibliographic database. The project encountered challenges in reference extraction and disambiguation but found success in creating a co-citation network from the references identified.
![Motivation - the Scholar
Issues: lack of data [Sula and Miller, 2014] leads to absence of
services: estimated coverage of Web of Science for Humanities
circa 13% [Mingers and Leydesdorff, 2015].
3
Sciences:
Google Scholar
English
mainly papers
Lower-cost information
gathering
Humanities:
no Google Scholar-like system
multiple languages
mainly monographs
Higher-cost information
gathering](https://image.slidesharecdn.com/colavizzaromanello-bir2016final-160331194632/85/The-References-of-References-Enriching-Library-Catalogs-via-Domain-Specific-Reference-Mining-3-320.jpg)
![Motivation - the Footnote
How humanists cite? Footnotes [see e.g. Hellqvist, 2009]
4](https://image.slidesharecdn.com/colavizzaromanello-bir2016final-160331194632/85/The-References-of-References-Enriching-Library-Catalogs-via-Domain-Specific-Reference-Mining-4-320.jpg)
![Motivation - the Archive
Approximately half citations to primary sources [Wiberley Jr., 2009]
5](https://image.slidesharecdn.com/colavizzaromanello-bir2016final-160331194632/85/The-References-of-References-Enriching-Library-Catalogs-via-Domain-Specific-Reference-Mining-5-320.jpg)
![Reference Extraction/Parsing
15
[Klinkhammer b-i-secondary-full] [Lutz, i-secondary-full] [L’occupazione i-
secondary-full] [tedesca i-secondary-full] [in i-secondary-full] [Italia i-secondary-full]
[1943-1945, i-secondary-full] [Torino, i-secondary-full] [Bollati i-secondary-full]
[Boringhieri i-secondary-full] [1993 i-secondary-full].
Klinkhammer Lutz, L’occupazione tedesca in Italia 1943-1945,
Torino, Bollati Boringhieri 1993.
[Klinkhammer author] [Lutz, author] [L’occupazione title] [tedesca title]
[in title] [Italia title] [1943-1945, title] [Torino, publicationplace] [Bollati
publisher] [Boringhieri publisher] [1993 publicationyear].](https://image.slidesharecdn.com/colavizzaromanello-bir2016final-160331194632/85/The-References-of-References-Enriching-Library-Catalogs-via-Domain-Specific-Reference-Mining-15-320.jpg)
The References of References: Enriching Library Catalogs via Domain-Specific Reference Mining
- 1. Giovanni Colavizza Matteo Romanello (@mr56k) Frédéric Kaplan (@frederickaplan) The References of References: Enriching Library Catalogs via Domain-Specific Reference Mining 1
- 2. Goal 2 Empowering scholars in the Humanities with better IR systems
- 3. Motivation - the Scholar Issues: lack of data [Sula and Miller, 2014] leads to absence of services: estimated coverage of Web of Science for Humanities circa 13% [Mingers and Leydesdorff, 2015]. 3 Sciences: Google Scholar English mainly papers Lower-cost information gathering Humanities: no Google Scholar-like system multiple languages mainly monographs Higher-cost information gathering
- 4. Motivation - the Footnote How humanists cite? Footnotes [see e.g. Hellqvist, 2009] 4
- 5. Motivation - the Archive Approximately half citations to primary sources [Wiberley Jr., 2009] 5
- 6. Motivation - the Scholar reloaded 6
- 7. Proposal: Enriching library catalogs 7 Use reference monographs, the “canon” of the domain, to extract references to the rest of the literature and enrich library catalogs.
- 8. Project: Linked Books Focused on a case study/domain: the history of Venice. Partners so far: • Ca’ Foscari University Library System • Biblioteca Marciana • Istituto Veneto di Scienze, Lettere ed Arti • Archivio di Stato di Venezia • EPFL 8
- 10. Corpus selection 10 Result: 1904 monographs, 701 with a structured list of references. Use the means of the library: 1- Consultation shelves 2- Dewey and subject classification 3- Scholarly bibliographies 4- Keyword search
- 11. The Pipeline - Digitization 11
- 12. Digitization 12 1,904 monographs + ~1,000 journal issues
- 13. The Pipeline - Annotation/Extraction/Parsing 13
- 14. Annotation 14 • annotated 27% of 701 monographs (with reference list) • 3.8% of all digitized pages (with references) • annotators identified 33 citation styles, divided into 6 families • Yes, humanities scholars love customized reference styles!
- 15. Reference Extraction/Parsing 15 [Klinkhammer b-i-secondary-full] [Lutz, i-secondary-full] [L’occupazione i- secondary-full] [tedesca i-secondary-full] [in i-secondary-full] [Italia i-secondary-full] [1943-1945, i-secondary-full] [Torino, i-secondary-full] [Bollati i-secondary-full] [Boringhieri i-secondary-full] [1993 i-secondary-full]. Klinkhammer Lutz, L’occupazione tedesca in Italia 1943-1945, Torino, Bollati Boringhieri 1993. [Klinkhammer author] [Lutz, author] [L’occupazione title] [tedesca title] [in title] [Italia title] [1943-1945, title] [Torino, publicationplace] [Bollati publisher] [Boringhieri publisher] [1993 publicationyear].
- 17. Extraction/Parsing - Confusion Matrix 17 null author title abbrev. (E) monograph (E) Task 1 F1 score (avg) 0.806 class=“null” 0.609 Task 2 F1 score (avg) 0.842 class=“end abbreviated” 0.242
- 18. The Pipeline - Lookup 18
- 19. Lookup 19 1. Against OPAC SBN (via API) Steps: 1. search candidates by title 2. match reference metadata 3. assign each candidate a confidence score 4. return set of candidates Evaluation: • 2k references (out of 181k) • 41.7% no candidates • 58.3% with candidates: • 72.3% -> first candidate correct Goal: disambiguation of references Issues: • OCR errors -> impact on search by title (low recall) • API as a “black box” + bottleneck of search by title
- 20. Lookup 20 2. Against metadata of digitized books Lookup Goal: verify cohesiveness of digitized corpus Method: • based on SBN lookup • but lookup against digitization metadata • tuned to maximize precision • returns 1 or no matches Evaluation*: • 500 references (out of 181k) • precision ~ 1.00 • recall > 0.95 Result: • only 7% of references extracted from 701 monographs point inwards (i.e. towards the 1904 monographs)
- 21. 21 Core of the discipline co-citation network from extracted references* giant component = 59% of selected corpus books in the giant component -> core of reference works on history of Venice giant component -> 32.5% with only works in consultation
- 22. Conclusions and Outlook 22 data- and citation-driven approach to assess and exploit, from an IR point of view, domain-specific library holdings on the history of Venice next big challenge: extraction, consolidation and disambiguation of references contained within footnotes (journals)
- 23. Giovanni Colavizza Matteo Romanello (@mr56k) Frédéric Kaplan (@frederickaplan) Thank you! go.epfl.ch/linkedbooks 23