Improved Discoverability of Digital Objects in Institutional Repositories Using Controlled Vocabularies
0 likes190 views
The document discusses the challenges of discoverability of digital objects in institutional repositories due to inconsistent metadata and the non-use of controlled vocabularies, which hinder effective searching and browsing. It outlines a methodology involving a situational analysis, usability studies, and the implementation of a multi-label subject classification model to enhance discoverability. The findings indicate that integrating controlled vocabularies significantly improves the usability and effectiveness of institutional repositories.
![15/30
September 27–30 , 2021 ACM/IEEE Joint Conference on Digital Libraries (JCDL 2021)
Methodology: Multi-label Subject Classifier
Title + Abstract
F1 Score Hamming
Loss
Jaccard
Score
SGDClassifier TF-IDF 0.540 0.005 0.431
[...] [...] [...] [...] [...]
● Multi-label subject classification model implemented using
arXiv CoRR dataset and validated using the UCT@ CS
Document archive
○ Titles and Abstracts used as input features](https://image.slidesharecdn.com/talks-jcdl21-controlledvocabularies-210916014538/85/Improved-Discoverability-of-Digital-Objects-in-Institutional-Repositories-Using-Controlled-Vocabularies-15-320.jpg)
![20/30
September 27–30 , 2021 ACM/IEEE Joint Conference on Digital Libraries (JCDL 2021)
Results and Discussion: Comparative
Analysis (1/2)
● SUS average scores
○ [66.2] Baseline
○ [68.9] Intervention](https://image.slidesharecdn.com/talks-jcdl21-controlledvocabularies-210916014538/85/Improved-Discoverability-of-Digital-Objects-in-Institutional-Repositories-Using-Controlled-Vocabularies-20-320.jpg)
![23/30
September 27–30 , 2021 ACM/IEEE Joint Conference on Digital Libraries (JCDL 2021)
Results and Discussion: Multi-label Subject
Classification Model—Implementation
Title + Abstract
F1 Score Hamming
Loss
Jaccard
Score
SGDClassifier TF-IDF 0.540 0.005 0.431
[...] [...] [...] [...] [...]
● Approaches used: Binary Relevance, Classifier Chains and
One-Versus-Rest
● Estimators: MultinomialNB vs SGDClassifier vs RandomForest
● Features: TF vs TF-IDF; Title vs Abstract vs Title + Abstract](https://image.slidesharecdn.com/talks-jcdl21-controlledvocabularies-210916014538/85/Improved-Discoverability-of-Digital-Objects-in-Institutional-Repositories-Using-Controlled-Vocabularies-23-320.jpg)
![[1] Phiri, L. (2018). Research Visibility in the Global South: Towards
Increased Online Visibility of Scholarly Research Output in
Zambia. IEEE International Conference in Information and
Communication Technologies.
[2] Phiri, L. (2020). A Multi-Faceted Multi-Stakeholder Approach for
Increased Visibility of ETDs in Zambia. Cadernos BAD, (1).
DOI: 10.1017/S0269888910000032
[3] Phiri, L. (2020). Automatic classification of digital objects for
improved metadata quality of electronic theses and dissertations
in institutional repositories. International Journal of Metadata,
Semantics and Ontologies, 14(3), 234-248.
DOI: 10.1504/IJMSO.2020.112804
Bibliography](https://image.slidesharecdn.com/talks-jcdl21-controlledvocabularies-210916014538/85/Improved-Discoverability-of-Digital-Objects-in-Institutional-Repositories-Using-Controlled-Vocabularies-28-320.jpg)
Improved Discoverability of Digital Objects in Institutional Repositories Using Controlled Vocabularies
- 1. Improved Discoverability of Digital Objects in Institutional Repositories Using Controlled Vocabularies University of Zambia Lusaka, ZAMBIA Bertha Chipangila · Eric Liswaniso · Andrew Mawila Philomena Mwanza · Daisy Nawila · Robert M’sendo Mayumbo Nyirenda · Lighton Phiri ACM/IEEE Joint Conference on Digital Libraries (JCDL 2021) September 27–30, 2021
- 2. 2/30 September 27–30 , 2021 ACM/IEEE Joint Conference on Digital Libraries (JCDL 2021) About Us (1/2)
- 3. 3/30 September 27–30 , 2021 ACM/IEEE Joint Conference on Digital Libraries (JCDL 2021) About Us (2/2) ● The DataLab research group at The University of Zambia is composed of faculty staff and students—undergraduate and postgraduate—working in three main areas ○ Data Mining ○ Digital Libraries ○ Technology-Enhanced Learning http://datalab.unza.zm
- 4. 4/30 September 27–30 , 2021 ACM/IEEE Joint Conference on Digital Libraries (JCDL 2021) Outline ● Motivation ● Problem Statement ● Methodology ● Results and Discussion ● Conclusion and Future Work
- 5. 5/30 September 27–30 , 2021 ACM/IEEE Joint Conference on Digital Libraries (JCDL 2021) There is an Ever Increasing Amount of Scholarly Research Generated https://scholar.google.com
- 6. 6/30 September 27–30 , 2021 ACM/IEEE Joint Conference on Digital Libraries (JCDL 2021) There is an Ever Increasing Amount of Scholarly Research Generated https://academic.microsoft.com
- 7. 7/30 September 27–30 , 2021 ACM/IEEE Joint Conference on Digital Libraries (JCDL 2021) Discoverability Services Facilitate Findability of Scholarly Research in IRs http://open.uct.ac.za http://dspace.unza.zm
- 8. 8/30 September 27–30 , 2021 ACM/IEEE Joint Conference on Digital Libraries (JCDL 2021) Problem Statement ● There are numerous inconsistencies with digital object metadata elements used to describe subjects ○ Lack of use of controlled vocabularies and subject headings compromises effective searching and browsing of scholarly research
- 9. 9/30 September 27–30 , 2021 ACM/IEEE Joint Conference on Digital Libraries (JCDL 2021) Problem Statement ● There are numerous inconsistencies with digital object metadata elements used to describe subjects ○ Lack of use of controlled vocabularies and subject headings compromises effective searching and browsing of scholarly research
- 10. 10/30 September 27–30 , 2021 ACM/IEEE Joint Conference on Digital Libraries (JCDL 2021) Problem Statement ● There are numerous inconsistencies with digital object metadata elements used to describe subjects ○ Lack of use of controlled vocabularies and subject headings compromises effective searching and browsing of scholarly research
- 11. 11/30 September 27–30 , 2021 ACM/IEEE Joint Conference on Digital Libraries (JCDL 2021) Methodology ● Situational analysis to determine the implications of non-use of controlled vocabularies ● Identification of subject-specific controlled vocabularies for various disciplines ● Usability study of IRs integrated with controlled vocabularies when compared with IRs without controlled vocabularies ● Implementation of multi-label subject classification model for classifying ACM CCS concepts and arXiv subjects
- 12. 12/30 September 27–30 , 2021 ACM/IEEE Joint Conference on Digital Libraries (JCDL 2021) Methodology: Situational Analysis ● Dublin Core encoded metadata harvested from three repositories ○ NDLTD Union Catalog ○ University of Cape Town Computer Science Document Archive ○ University of Zambia Institutional Repository
- 13. 13/30 September 27–30 , 2021 ACM/IEEE Joint Conference on Digital Libraries (JCDL 2021) Methodology: Identification of Subjects and Usability Study ● 7 faculty interviewed to determine appropriate controlled vocabularies ● DSpace-powered IRs set-up to conduct controlled comparative study ○ IR #1: LCSH subjects ○ IR #2: No subjects ○ System Usability Scale used to assess usability
- 14. 14/30 September 27–30 , 2021 ACM/IEEE Joint Conference on Digital Libraries (JCDL 2021) Methodology: Identification of Subjects and Usability Study ● 7 faculty interviewed to determine appropriate controlled vocabularies ● DSpace-powered IRs set-up to conduct controlled comparative study ○ IR #1: LCSH subjects ○ IR #2: No subjects ○ System Usability Scale used to assess usability ● Within subject SUS study conducted with 50 participants
- 15. 15/30 September 27–30 , 2021 ACM/IEEE Joint Conference on Digital Libraries (JCDL 2021) Methodology: Multi-label Subject Classifier Title + Abstract F1 Score Hamming Loss Jaccard Score SGDClassifier TF-IDF 0.540 0.005 0.431 [...] [...] [...] [...] [...] ● Multi-label subject classification model implemented using arXiv CoRR dataset and validated using the UCT@ CS Document archive ○ Titles and Abstracts used as input features
- 16. 16/30 September 27–30 , 2021 ACM/IEEE Joint Conference on Digital Libraries (JCDL 2021) Results and Discussion: Situational Analysis (2/2) ● Analysis 1. Metadata preparation and ingestion workflow based on internal policy ● Analysis 2. Subject heading usage is sparing. 92.1% of tags only associated with one publication ● Analysis 3. Domain-specific subject headings are not used. Internally devised LCSH used
- 17. 17/30 September 27–30 , 2021 ACM/IEEE Joint Conference on Digital Libraries (JCDL 2021) Results and Discussion: Situational Analysis (1/2)
- 18. 18/30 September 27–30 , 2021 ACM/IEEE Joint Conference on Digital Libraries (JCDL 2021) Results and Discussion: Situational Analysis (1/2) ● Incidentally, the problem manifests itself in other repositories and downstream services
- 19. 19/30 September 27–30 , 2021 ACM/IEEE Joint Conference on Digital Libraries (JCDL 2021) Results and Discussion: Comparative Analysis (1/2)
- 20. 20/30 September 27–30 , 2021 ACM/IEEE Joint Conference on Digital Libraries (JCDL 2021) Results and Discussion: Comparative Analysis (1/2) ● SUS average scores ○ [66.2] Baseline ○ [68.9] Intervention
- 21. 21/30 September 27–30 , 2021 ACM/IEEE Joint Conference on Digital Libraries (JCDL 2021) Results and Discussion: Comparative Analysis (2/2)
- 22. 22/30 September 27–30 , 2021 ACM/IEEE Joint Conference on Digital Libraries (JCDL 2021) Results and Discussion: Comparative Analysis (2/2)
- 23. 23/30 September 27–30 , 2021 ACM/IEEE Joint Conference on Digital Libraries (JCDL 2021) Results and Discussion: Multi-label Subject Classification Model—Implementation Title + Abstract F1 Score Hamming Loss Jaccard Score SGDClassifier TF-IDF 0.540 0.005 0.431 [...] [...] [...] [...] [...] ● Approaches used: Binary Relevance, Classifier Chains and One-Versus-Rest ● Estimators: MultinomialNB vs SGDClassifier vs RandomForest ● Features: TF vs TF-IDF; Title vs Abstract vs Title + Abstract
- 24. 24/30 September 27–30 , 2021 ACM/IEEE Joint Conference on Digital Libraries (JCDL 2021) Results and Discussion: Multi-label Subject Classification Model—Validation ● Model evaluated using CS subject repository with self-archiving implemented
- 25. 25/30 September 27–30 , 2021 ACM/IEEE Joint Conference on Digital Libraries (JCDL 2021) Results and Discussion: Multi-label Subject Classification Model—Demonstration C.2.4 · D.2.11 · F.1.1 · H.3.4 · H.3.5 · H.5.2 Computer Science - Artificial Intelligence · Computer Science - Computation and Language · Computer Science - General Literature · Computer Science - Human-Computer Interaction ● Six (6) arXiv and four (4) ACM CCS subject predicted by model
- 26. 26/30 September 27–30 , 2021 ACM/IEEE Joint Conference on Digital Libraries (JCDL 2021) Conclusions and Future Work ● Integrating IRs with subject controlled vocabularies can potentially complement self-archiving and, additionally, has the benefit ensuring that IRs are usable and effective ● Potential future work and/or direction ○ Metadata cleaning, enhancement and augmentation of existing descriptive metadata ○ Implementation of subject classification models for other subject-specific controlled vocabularies ○ Automatic generation of subject classes for large-scale repositories such as the NDLTD Union Catalog
- 27. 27/30 September 27–30 , 2021 ACM/IEEE Joint Conference on Digital Libraries (JCDL 2021) Q & A Session ● Comments, concerns and complaints?
- 28. [1] Phiri, L. (2018). Research Visibility in the Global South: Towards Increased Online Visibility of Scholarly Research Output in Zambia. IEEE International Conference in Information and Communication Technologies. [2] Phiri, L. (2020). A Multi-Faceted Multi-Stakeholder Approach for Increased Visibility of ETDs in Zambia. Cadernos BAD, (1). DOI: 10.1017/S0269888910000032 [3] Phiri, L. (2020). Automatic classification of digital objects for improved metadata quality of electronic theses and dissertations in institutional repositories. International Journal of Metadata, Semantics and Ontologies, 14(3), 234-248. DOI: 10.1504/IJMSO.2020.112804 Bibliography
- 30. Improved Discoverability of Digital Objects in Institutional Repositories Using Controlled Vocabularies University of Zambia Lusaka, ZAMBIA Bertha Chipangila · Eric Liswaniso · Andrew Mawila Philomena Mwanza · Daisy Nawila · Robert M’sendo Mayumbo Nyirenda · Lighton Phiri ACM/IEEE Joint Conference on Digital Libraries (JCDL 2021) September 27–30, 2021