How to expose research data in EOSC
Download as PPTX, PDF1 like279 views
1. The document discusses the EOSC Dataset Minimum Information (EDMI) approach for exposing research data in the European Open Science Cloud (EOSC). 2. EDMI defines a set of 12 minimum metadata properties to facilitate finding and accessing datasets without being overly descriptive. 3. The approach was developed by engaging EOSC demonstrator data repositories and repositories to propose methods for exposing metadata in a simple and sustainable way.
How to expose research data in EOSC
- 1. How to expose research data in EOSC The “EOSC Dataset Minimum Information” (EDMI) approach EUDAT conference, January 22-25 Rafael C Jimenez ELIXIR & EOSCpilot
- 3. Nucleic Acids Research annual Database Issue and the NAR online Molecular Biology Database Collection in 2012. MY Galperin, GR Cochrane – Nucleic Acids Research, 2011 Data resources in life science 1800 • Diverse • Plentiful • Disperse databases
- 4. Challenges data resource interface datasets data resource interface datasets Many, different & variable … • Sustainability • Findability • Accessibility • Consistency • Interoperability • Completeness • Redundancy • Reusability • Integration • Replication • Compute • … users services +
- 5. To demonstrate how to ensure availability of scientific data and data-analysis services through a cloud infrastructure and design a stakeholder driven governance framework Lorenza Saracco and Carmela Asero | European Commission, DG Research & Innovation | Pisa, 14-15 September 2017 Pilot action for EOSC EOSC data services access governance <data> <data>
- 6. EOSC data services access governance <data> <data> To demonstrate how to ensure availability of scientific data and data-analysis services to users and services through a cloud infrastructure and design a stakeholder driven governance framework EOSCpilot data interoperability
- 7. Scope The work of this working group and its recommendations Defined by 13 guiding principles Reuse: Leverage the rich legacy of Research Infrastructures • Making data FAIR is the responsibility of the Research Infrastructures and their data repositories • We must rely on research infrastructure data catalogues • We must support an ecosystem of catalogues • We should provide metadata quality recommendations to feedback to RIs Least: The least possible metadata for the most benefit • Findability should come first • Common and minimum metadata • Focus on common data types: datasets and data repositories • Flexible metadata models to embrace domain specifics • Service requirements and operational metadata first class citizen Practical: Sustainable and pragmatic delivery • Engage EOSC demonstrator data repositories • Propose methods to expose metadata • Simple to implement, easy to sustain • Deliver guidelines and demonstrators Results from RDA-Bluebridge and OpenFair2017 workshops
- 8. 1. Making data FAIR is the responsibility of the Research Infrastructures and their data resources 2. We must rely on research infrastructure data resources and data catalogues 3. We must support an ecosystem of catalogues 4. We should provide recommendations to improve metadata quality Reuse: Leverage the rich legacy of Research Infrastructures
- 9. 1. Making data FAIR is the responsibility of the Research Infrastructures and their data repositories 2. We must rely on research infrastructure data catalogues 3. We must support an ecosystem of catalogues 4. We should provide recommendations to improve metadata quality Reuse: Leverage the rich legacy of Research Infrastructures
- 10. Some catalogues in life sciences Databases Datasets Publications Tools Training Events Ontologies Standards Samples OmicsDI Identifiers.org Bio.tools Biojs Biocontainers PSICQUIC Bio.tools STM
- 11. Ecosystem of catalogues Google re3data Generic Research Life Sciences Omics sciences OmicsDI +- metadata richness reuse … … … …
- 12. Ecosystem of catalogues EDMI EDMI EDMI EDMI EDMI EDMI … … EDMI …… import import import specificgeneric +- metadata submission and curation
- 13. Kimmo Koski, Managing Director, CSC – IT Center for Science & EUDAT Coordinator
- 14. 5. Findability and accessibility should come first 6. Common and minimum metadata for finding and accessing data 7. Reuse existing metadata models 8. Focus on common data types: datasets and data resources 9. Service requirements and operational metadata first class citizen Least: The least possible metadata for the most benefit
- 15. 5. Findability and accessibility should come first 6. Common and minimum metadata for finding and accessing data 7. Reuse existing metadata models 8. Focus on common data types: datasets and data resources 9. Service requirements and operational metadata first class citizen Least: The least possible metadata for the most benefit
- 16. Schema B Schema A Schema C Schema D • Common and minimum metadata for finding and accessing data • Not aiming to be descriptive • Reuse existing metadata models • Crosswalk across existing models
- 17. 5. Findability and accessibility should come first 6. Common and minimum metadata for finding and accessing data 7. Reuse existing metadata models 8. Focus on common data types: datasets and data resources 9. Service requirements and operational metadata first class citizen Least: The least possible metadata for the most benefit
- 18. Finding and Accessing datasets and data resources Data resource Dataset Records 1..* 1..* <data>
- 19. 5. Findability and accessibility should come first 6. Common and minimum metadata for finding and accessing data 7. Reuse existing metadata models 8. Focus on common data types: datasets and data resources 9. Service requirements and operational metadata first class citizen Least: The least possible metadata for the most benefit
- 20. Scientific Functional Researchers File Operational Services Scientific & Services metadata requirements for finding and accessing datasets
- 21. Practical: Sustainable and pragmatic delivery 10. Engage EOSC demonstrator data repositories 11. Propose methods to expose metadata 12. Simple to implement, easy to sustain 13. Deliver guidelines and demonstrators
- 22. guidelines • EOSC Dataset Minimum Information Guidelines • 12 metadata properties • Facilitating finding and accessing datasets • Not a data model, but a cross walk to existing dataset models • Focus on scientific and service requirements • Compatible with existing data models and interfaces • Provides pointers to the dataset records and references to their data standards and access interfaces • Not mandatory but minimum with the goal to improve status quo • One way to measure FAIRness of datasets and data resources EDMI
- 23. EDMI metadata properties Properties Description M/F M/O R/F R/O O/F O/O MINIMUM name A descriptive name for the dataset yes description A short summary describing a dataset yes identifier The identifier property represents any kind of identifier for any kind of dataset yes url The location of a page describing the dataset yes yes creator The creator/author of this dataset yes yes dateCreated The date on which the dataset was created yes yes license A license under which the dataset is distributed yes yes dataStandard The standard in which the content of the dataset is represented yes yes dateModified The date on which the dataset was most recently modified yes structure The description of the structure of the dataset yes accessUrl The link to download the dataset yes accessInterface The type of interface to present the dataset yes RECOMMENDED includedeIn A dataset or data catalog which contains the dataset yes yes measurementTechniqueA technique or technology used in a dataset corresponding to the method used for measuring the corresponding variables yes keywords Keywords or tags used to describe the dataset yes variablesMeasured The variables that are measured in the dataset yes format The format in which the content of the dataset is encoded to present the information, typically a MIME format yes scientificType Scientific domain or type of the information provided in the datataset yes includes A dataset or data catalog contained in the dataset yes contentType Type of content provided in the dataset based on its origin and type of processes (raw, processed, summarised) yes M/F: Minimum Functional, M/O: Minimum Operational R/F: Recommended Functional, R/O: Recommended Operational O/F: Optional Functional, O/O: Optional Operational
- 24. Simple architecture, simple adoption <data> data access metadata index metadata discovery metadata access Data resources Dataset metadata catalogues Services EDMIEDMI EDMI EOSC Datasets Minimum Information Metadata Guideline
- 25. Metadata catalogues RI data e-infrastructure services Compute Storage Transfer … Data resources … … … … … … … Datasets Scientific domain A Scientific domain B Scientific domain C Generic metadata catalogue Data entry … … … … … … … …
- 26. Metadata catalogues RI data e-infrastructure services Compute Storage Transfer … Data resources … … … … … … … Datasets Scientific domain A Scientific domain B Scientific domain C Data entry … … … … … … … … Generic metadata catalogue
- 27. Demonstrators • Evaluate findability and accessibility of datasets via EDMI functional and operational metadata • Discovery of compliant data resources and metadata catalogues • Research schemas for exposing dataset metadata • Description and guidelines per metadata property in collaboration with RDA MIG
- 28. Evaluate findability and accessibility of datasets via EDMI functional and operational metadata <data> data access metadata index metadata discovery metadata access Data resources Dataset metadata catalogues Services EDMIEDMI EDMI EOSC Datasets Minimum Information Guideline OmicsDI
- 29. Discovery of compliant data resources and metadata catalogues <data> data access metadata index metadata discovery metadata access Data resources Dataset metadata catalogues Services EDMIEDMI Data resources metadata catalogues EDMI metadata index metadata discovery metadata discovery EDMI EOSC Datasets Minimum Information Metadata Guideline metadata access OmicsDI
- 30. Research schemas for exposing dataset metadata
- 31. <div itemscope itemtype="http://schema.org/Recipe"> <h1 itemprop="name">Classic potato salad</h1> <div itemprop="nutrition” itemscope itemtype="http://schema.org/NutritionInformation"> Nutrition facts: <span itemprop="calories">144 kcal</span>, </div> Ingredients: - <span itemprop="recipeIngredient">800g small new potato</span> - <span itemprop="recipeIngredient">3 shallot</span> . . . Structured data markup for web pages RDFa JSON-LD Microdata With markup
- 32. Major data resource Small data resource Research schemas Research schemas Research schemas A community initiative built on top of Schemas.org to improve Findability and Accessibility in Research RegistrySearch engine Data aggregator
- 33. Evaluate findability and accessibility of datasets via EDMI functional and operational metadata <data> data access metadata index metadata discovery metadata access Data resources Dataset metadata catalogues Services EDMIEDMI EDMI EOSC Datasets Minimum Information Guideline OmicsDI
- 34. Description and guidelines per metadata property in collaboration with RDA MIG
- 35. Recommendations and guidelines per metadata property. Generic and domain specific with examples.
- 37. Structure Access interface Access URL Date modified
- 38. EOSCpilot data interoperability - summary Recommendations Architecture define validate Data int. demonstrators Adoption & testing Data resources discovery Research schemas markup RDA metadata elements Principles Scope | Direction 2017 2018 FAIR FAIR Goal Demonstrate how to ensure availability of scientific data to users and services through a cloud infrastructure Plan
- 39. Thanks for your attention Questions? EOSCpilot data interoperability EC Report Dec 2017 https://drive.google.com/file/d/1UmxB4YLp_LJmjcpZnWHqlu-52gcU7RON/view?usp=sharing
- 40. Data access interfaces Graphical User Interface (GUI) FTP access Database access Application Programmatic Interface (API) Web Services Reseachers Developers <data>
- 41. workspace clouds data tools workflows identities containers cloud