Opening and Integration of CASDD and Germplasm Data to AGRIS by Prof. Xuefu Zhang and Dr. Guojian Xian
Download as PPTX, PDF1 like882 views
The document discusses the integration and opening of Chinese Agricultural Sci-Tech Documents Database (CASDD) and germplasm data as RESTful APIs to enhance access to agricultural information. It details the features of these APIs, which include various querying methods and output formats, while emphasizing the extensive data coverage of CASDD and its role in supporting agricultural research. Additionally, it outlines the successful integration with AGRIS and the introduction of improved search functionalities.
Opening and Integration of CASDD and Germplasm Data to AGRIS by Prof. Xuefu Zhang and Dr. Guojian Xian
- 1. Opening and Integration of CASDD and Germplasm Data to AGRIS Prof. Xuefu Zhang & Dr. Guojian Xian Agricultural Information Institute of CAAS Research Data Alliance Fourth Plenary Meeting, 22-24 September, 2014, Amsterdam
- 2. Contents Open CASDD as Restful APIs Open Germplasm as Restful APIs Integration and Extension to AGRIS Fruitful Results 2
- 3. Main Materials • Chinese Agricultural Sci-tech Documents Database (CASDD) – 440,113 records • CGRIS Germplasm Data • AGROVOC – agrovoc_2013-12-17_core.rdf • Chinese Agricultural Thesaurus(CAT) • KOS Mapping Results: – AGROVOC_CAT.nt • AGRIS 2.0 – (Latest version: 20140427) 3
- 4. About CASDD • Chinese Agricultural Sci-tech Documents Database (CASDD), as agricultural bibliographic/abstracts database in China developed by CAAS, has the largest number of records and the longest time span of documents. • Covering over 1000 kinds of agricultural academic journals and other materials, over 6 million records, in the fields of agronomy, horticulture, plant protection, soil sciences, animal husbandry, veterinary, agricultural engineering, agricultural products processing, agricultural economic,etc. • It is the most comprehensive, reliable and accessible information resources of agricultural science and technology information from research institutions, education and related departments. 4
- 5. Refining and Analyzing CASDD 5 Tagging CAT and AGROVOC concepts to CASDD CASDD CAT AGROVOC RDF Core Mapping of CAT and AGROVOC Solr 4.7 Write&Read Tagging(URI,Preflabel) Indexing CASDD Index Virtuoso Triple Store Tagging(URI,Preflabel) Sparql query MMseg4J/ IKAnalyzer Java Application SQE Plugin
- 6. English Coverage Analysis of CASDD Records 6 Total Records: 440,113 Fields Records Percentage English Title 289,314 65.74% English Keywords 286,032 64.99% English Abstract 286,921 65.19%
- 7. 7 The CAT Concepts Coverage in CASDD
- 8. 8 The AGROVOC Concepts Coverage in CASDD TermCount TermFreq. Record Number Match Ratio TermFreq>=3 TermCount>=1 400,009 90.89% TermFreq>=3 TermCount>=2 320,472 72.82% TermFreq>=3 TermCount>=3 227,481 51.69% TermFreq>=3 TermCount>=5 83,992 19.08% TermFreq>=5 TermCount>=3 51,726 11.75%
- 9. CASDD Restful API (Architecture) Accessing & Linking CASDD Restful Web Service (API) Endpoint CASDD Database Tomcat (Jersey API) Reading Only Container Solr 4.7 (SQE Plugin) Third Part Application Container AGRIS agINFRA Index CAT + AGROVOC + Mapping
- 10. CASDD Restful API(Features) • Aims to provide a light-weight solution to expose the records of CASDD to the third party applications. • Providing several ways to access the records, such as query with keywords, ARN, PublicationDate, AGROVOC Concept URIs, Chinese Agricultural Thesaurus (CAT) URIs. • The results also supporting pagination and sorting. • The output formats include RDF/XML following the AGRIS AP standard and plain JSON. • Authentication and Detail Logging for evaluations 10
- 11. CASDD Restful API(Samples) 11 BrowsGinetg rreeccoorrddss wwiitthh ApGagRiOnaVtOioCn URI
- 12. Contents Open CASDD as Restful APIs Open Germplasm as Restful APIs Integration and Extension to AGRIS Fruitful Results 12
- 13. Germplasm Data of CGRIS • CGRIS germplasm database is a central repository for all type of plant genetic resources information in China. At present, there are over 4000 MB data on 200 kinds of crops, 410,000 accessions of germplasm stored in CGRIS. 13
- 14. The Germplasm Restful API (Architecture) Redirect to Detail CGRIS Germplasm Database Tomcat (Jersey API) CGRIS Website CGRIS Germplasm Restful API AGROVOC CAT Preflabel2URI Mapping Reading Only Accessing & Linking Container Third Part Application
- 15. The Germplasm Restful API (Features) • Aims to provide a light-weight solution to expose the records of CGRIS Germplasm to the third party applications. • Providing several ways to access the records, such as query with scientific name, vernacular name, catalogNumber, AGROVOC Concept URI, Chinese Agricultural Thesaurus (CAT) URI. • The output formats include RDF/XML following the darwincore-germplasm schema and plain JSON. • Authentication and Detail Logging for evaluations
- 16. The Germplasm Restful API (Samples) Get records with vernacular name Get reGcoetr drse cwoirtdhs s wciietnht AifGicR nOaVmOeC URI
- 17. Contents Open CASDD as Restful APIs Open Germplasm as Restful APIs Integration and Extension to AGRIS Fruitful Results 17
- 18. The Extended AGRIS in Chinese 18 The Extended AGRIS in Chinese QUERY STATISTICS (CASDD) SEARCH RESULT BROWSING Restful API SINGLE RECORD MASHUPS ( Germplasm) AGRIS SERVICES LAYER Read TOOLS LAYER CASDD Germplasm DATA LAYER JAVA APPLICATION Custom Modules CASDD New Page CASDD AGRIS CAT AGROVOC RDF Core Mapping of CAT and AGROVOC Chinese Query Solr 4.7 SQE Plugin CASDD Box Germplasm Other Resources
- 19. Enhanced Search in Chinese • Semantic Query Extension – Solr Query Expander (SQE)2.0 • Integrating and Linking CASDD API • Integrating and Linking Germplasm API • Other Improvements: – User Query Automatic Suggestion – Update AGRIS AP XML files Indexer to Solr 4.7 – Integrating Bing Cloud Dictionary 19
- 20. Improved and Updated SQE 2.0 • Totally be compliance with Solr 4.5. • Work with SKOS files with suffix .rdf (RDF/XML), .n3 (N3),.ttl (TURTLE) and .zip (ZIP) • Supports load more than one SKOS files at one time • Supports customized relationship types expansion, such as PREF, ALT, HIDDEN, BROADER, NARROWER, BROADERTRANSITIVE, RELATED. • Excellent performance with the improved version of IKAnalyzer2012FF (supports English phrase analysis and tagging based on English dictionary) 20
- 21. Semantic Expansion Search with SQE2.0 21
- 22. Integrating and Linking CASDD • AGRIS Search Results(CASDD Box) – The box displays the search results of CASDD (first five records) – Records include title, author, keywords, submission date, and abstract. – get more related records – get more (detail information) 22
- 23. Integrating and Linking CASDD • Detail information(Single Record information) – Title(ZH/EN), Keywords(ZH/EN), Authors, Submission Date, Abstract(ZH/EN), CAT keywords, AGROVOC keywords, Journal, ISSN • More Related Records – Display more related records – Browing records with pagination 23
- 24. Linking CGRIS Germplasm Resources • Germplasm Mashup – get more…(detail information) First five CGRIS Germplasm records information • Navigating to CGRIS Website – CGRIS website 24
- 25. Contents Open CASDD as Restful APIs Open Germplasm as Restful APIs Integration and Extension to AGRIS Fruitful Results 25
- 26. 26 Linking CASDD Records with Box http://agris.fao.org/agris-search/searchIndex.do?query=barley&x=-430&y=-58
- 27. Detail Info of a CASDD Record 27
- 28. More Related Records From CASDD 28
- 29. CGRIS Germplasm Mashup 29
- 30. Thanks for Your Listening! 30