Cross-domain data discovery and integration
Download as PPTX, PDF0 likes104 views
Simon Cox presented on cross-domain data discovery and integration. He discussed using cross-domain data catalogs and aligning metadata standards to facilitate finding data across different domains. Various metadata standards were shown including DCAT and EML that could be combined or "mashed up" to improve data discovery for multi-disciplinary applications.
Cross-domain data discovery and integration
- 1. Cross-domain data discovery and integration Simon J D Cox CSIRO Land and Water 7 November 2018
- 2. Finding data for x-disciplinary applications Use cross-domain data catalogs SCIDATACON 2018-11-07 | Cox et al. | Harmonising data description2 |
- 3. SCIDATACON 2018-11-07 | Cox | x-domain discovery and integration
- 4. SCIDATACON 2018-11-07 | Cox | x-domain discovery and integration
- 5. Generic dataset metadata SCIDATACON 2018-11-07 | Cox | x-domain discovery and integration
- 6. Metadata standards SCIDATACON 2018-11-07 | Cox | x-domain discovery and integration
- 7. Primarily domain usage Variable level Study level Generic applications SSN/SOSA DCAT 19115 EMLDQV FHIR HL7Q DATS DDI CERIF QB SCIDATACON 2018-11-07 | Cox | x-domain discovery and integration
- 8. Observation metadata SCIDATACON 2018-11-07 | Cox | x-domain discovery and integration SSN Observation - resultTime - madeBySensor - usedProcedure - hasFeatureOfInterest - hasResult - phenomenonTime - observedProperty
- 9. Lining up natural and social science concepts SCIDATACON 2018-11-07 | Cox | x-domain discovery and integration
- 10. Controlled vocabularies Provenance/context Observation properties Dataset statistics SCIDATACON 2018-11-07 | Cox | x-domain discovery and integration Metadata mashup
- 11. Is βmetadataβ the solution? β’ Discovery might be done other ways SCIDATACON 2018-11-07 | Cox | x-domain discovery and integration
- 12. Metadata property -> data source (type) Semantic Data Lake β Envisaged Architecture Decomposing User QuerySPARQL query Database XML File ?item gho:Country ?country . ?item gho:Disease ?disease . ... SELECT country, disease, ... FROM Observations Finding Relevant Data Sources + Queries Translation SQL XPathSQL MongoDB JSONPath SQL XML MongoDB Execution Plan SCIDATACON 2018-11-07 | Cox | x-domain discovery and integration Run-time integration?
- 13. Thank you Simon J D Cox Research Scientist CSIRO Land and Water simon.cox@csiro.au.au SCIDATACON 2018-11-07 | Cox | x-domain discovery and integration
- 14. Dataset catalog β schema.org SCIDATACON 2018-11-07 | Cox | x-domain discovery and integration
- 15. W3C Dataset catalog vocabulary - DCAT SCIDATACON 2018-11-07 | Cox | x-domain discovery and integration
- 16. SCIDATACON 2018-11-07 | Cox | x-domain discovery and integration CKAN β₯ DCAT
Editor's Notes
- #7: There β two dirty words in one title!
- #13: The Squerall project has proved initially promising results but needs further development. Performance is better than the traditional integration at ingestion time - i.e. mapping everything to a single unified data model - as you only ever query relevant data, not the whole lot, looking for things that arenβt there.