European Open Science Cloud: Concept, status and opportunities
- 1. EOSC-hub receives funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 777536. eosc-hub.eu @EOSC_eu European Open Science Cloud Concept, status and opportunities Gergely Sipos (EGI Foundation) ISGC 2019, Taipei, April 2019
- 2. 2 European Open Science Cloud (EOSC) – A short history EOSC-hub - 4 areas - 5 opportunities – with examples Implementing an Open Science story Next events Outline
- 3. 3 EOSC - Problem statement European Cloud Initiative by the European Commission (April 2016) 1. How to maximise the incentives for sharing data and to increase the capacity to exploit them? 2. How to ensure that data can be used as widely as possible, across scientific disciplines and between the public and the private sector? 3. How better to interconnect the existing and the new data infrastructures across Europe? 4. How best to coordinate the support available to European data infrastructures as they move towards exascale computing? “…a trusted, open environment for the scientific community for storing, sharing and re- using scientific data and results…” “…by 2020…”
- 4. 4 EC activities: - 1st EOSC High Level Expert Group (final report) - 2nd EOSC HLEG (final report) - EOSC summits (2017, 2018) - EOSC implementation Roadmap (April 2018) (link) -6 action lines (Archi., Data, Services, Access & Interf., Rules, Governance) - EOSC Portal launch (Nov 2018) - EOSC Executive Board (Nov 2018. Members) - EOSC Secretariat for the Executive board and Governance board (Jan 2019) H2020 projects: - EOSCpilot (2017-19) - EOSC-hub (2018-2020) - OpenAIRE-Advanced (2018-2020) - ESFRI Clusters (2019-2021) - INFRAEOSC-* (2019-2020/21) What happened since April 2016…
- 5. 5 The federated infrastructure and supporting initiative providing all researchers, innovators, companies and citizens with seamless access to an open-by-default, efficient and cross-disciplinary environment for storing, accessing, reusing data, tools, publications and other scientific outputs for research, innovation and educational purposes About the European Open Science Cloud Credits: EOSCpilot (https://eoscpilot.eu/)
- 6. Vision 100 partners 36 months 33 million Euro Researchers from all disciplines have easy, integrated and open access to the advanced digital services, scientific instruments, data, knowledge and expertise they need to collaborate to achieve excellence in science, research and innovation
- 7. Mission EGI Federation EUDAT INDIGO- DataCloud Research Infrastructur es The EOSC-hub project mobilises providers of European relevance offering services, software and data for advanced data-driven research and innovation. These resources are offered via the Hub – the integration and management system of the European Open Science Cloud, acting as a European-level entry point for all stakeholders.
- 8. 8 The Compute Platform: EGI Services for cloud, HTC, container computing and data staging Based on NGIs – National compute centres – coordinated by EGI Foundation 4.4 Billion CPU core wall time in 2018, 1 M cores, > 730 PB 1170 open access publications, +41 new international projects 31 large scale ESFRI projects/landmarks supported
- 9. 9 The Data Platform: EUDAT CDI (CDI - Collaborative Data Infrastructure)
- 10. 10 The project established ‘The Hub’ https://marketplace.eosc-portal.eu • Data • Applications & tools • Baseline services (storage, compute, connectivity)… • Training, consultants • Marketplace • AAI • Accounting • Monitoring • … Usage according to Rules of Participation From the consortium AND from external contributors • Lightweight certifi- cation of providers • SLA negotiation • Customer Relationship Management • … Based on FitSM • Security regulations, • Compliance to standards, • Terms of use, • FAIR implementation guidelines • …
- 11. 11 Area 1: Services e-Infra EGI Federation EUDAT CDI INDIGO- DataCloud Humanities Language and literature (CLARIN) Arts (DARIAH) Engineering Environmen tal engineering (sea vessels, LNEC) Civil Engineering (Disaster Mitigation) Medical and Health Sciences Biological Sciences (ELIXIR) Structural biology (WeNMR) Natural sciences PHYSICAL SCIENCES Astronomy (LOFAR) Fusion (ITER) High Energy Physics (CMS and VIRGO) Space Science (EISCAT-3D) EARTH SCIENCE EO Pillar GEO Climate Research (ENES) Seismology (ORFEUS, EPOS) BIOLOGICAL SCIENCES Marine and freshwater biology (IFREMER) Biodiversity conservation (LifeWatch) Ecology (ICOS) Generic services + a growing number of external contributions
- 12. 12 Generic services from e-Infrastructures: Support to the Research Data Lifecycle Processing & Analysis Data Management, Curation & Preservation Access, Deposition & Sharing ● B2FIND (data) ● Marketplace (Services) ● Applications on Demand ● Federated HTC & Cloud Compute IaaS & PaaS ● Processing of sensitive data ● Jupyter Notebook ● Application DB (software & VM) ● B2DROP (data) ● B2Note (data) ● B2SHARE (data) ● DataHub ● B2HANDLE ● B2SAFE ● European Certified Trusted Repository ● Thematic data analytics ● Scientific Workflow Management, Orchestration (DIRAC, PaaS Orchestrator) 1 2 3 4 Discover & Reuse
- 13. 13 Area 2: Federation services EOSC Portal website and Marketplace Authentication- Authorization Infrastructure Accounting Monitoring Helpdesk Service integration support and training Engagement with new service providers Making services discoverable and orderable Making services accessible Making services robust Making problems resolvable Making services usable
- 14. • IT service management - Focuses on the provision of high quality IT services that meet customers' and users’ expectations - Defines, documents and maintains service management processes with assigned roles and responsibilities • EOSC-hub Service Management: Based on FitSM standard - 85 requirements that should be fulfilled by an organisation (or federation) - 16 general requirements; 69 process-specific requirements - 14 processes 14 Area 3: Federated operation • Service portfolio management • Service level management • Incident management • Change management • Capacity management • Information security management • … Example – Service Portfolio Management requirements: PR1.1 A service portfolio shall be maintained. All services shall be specified as part of the service portfolio. PR1.2 Design and transition of new or changed services shall be planned. PR1.3 Plans for the design and transition of new or changed services shall consider timescales, responsibilities, new or changed technology, communication and service acceptance criteria. PR1.4 The organisational structure supporting the delivery of services shall be identified, including a potential federation structure as well as contact points for all parties involved. • Implementation in EOSC - EOSC-hub Federation services (will) implement all processes: First internal audit took place - Other services are encouraged and supported in implementation/alignment (FitSM training, consultancy, ‘implementation workshop’)
- 15. • Harmonising service-related policies and supporting providers in the implementation • Current focus - Acceptable use policy - Operations security policy - Conditions of use • Exploring new areas - Licenses - ‘FAIR’ services and data (Findable, Accessible, Interoperable, Reusable) 16 Area 4: Processes and policies
- 17. EOSC-hub Collaboration Areas: 5 opportunities EOSC Digital Innovation Hub Join as provider Use of existing service Networking Training, user support and best practices 18https://www.eosc-hub.eu/publications
- 18. 19 O1: Access services: https://eosc-portal.eu Some services accessible without authentication Some services work with automated login Some require manual approval of users ‘Simple’ requests: passed to individual service providers Complex requests: supported by EOSC-hub tech team
- 19. 21 O2: Join as provider - Benefits • Promote adoption of your services • Get a free online platform to manage the services request and interact with users (Marketplace) • Get statistics about access and customer feedbacks • Contribute to the definition and maintenance of the EOSC service provisioning policies and portfolio roadmap (providers’ board)
- 20. 22 O2: Join as provider - Requirements https://eosc-portal.eu/for-providers • The service is accessible by users outside the original community • The service is described via the common template (for publication on the EOSC portal) • At least one service instance is running in a production environment (software vs. service) • Published research data is ‘FAIR’ • Release notes and documentation are available • Helpdesk channels are available 57 Services published +28 in verification 52 service providers 85 Services until March 2019 Including iCOMCOT and WRF portals from AS
- 21. 24 O3: EOSC Digital Innovation Hub (DIH) • What are Digital Innovation Hubs? • An ecosystem of start-ups, SMEs, large industries, researchers, accelerators and investors that fosters the creation of partnerships to stimulate innovation. • 200+ DIHs registered comprising both regional and European ones that focus on a variety of sectors and topics • The EOSC DIH • To onboard industrial partnerships within the European Open Science Cloud • Focused on (not limited to) start-ups and SMEs • To persist beyond the life of the project • Main activity: Setup and run ‘Business pilots’ Join the DIH Propose/run a Business Pilot
- 22. Examples: Business pilots (initial 6) ACTION Seaport Space Weather Data for the DRACO Observatory Bot Mitigation Engine Sport Smart Video Analysis CyberHAB Furniture Enterprise Analytics Datafurn Improving safety and operational performance of seaports A Cloud Framework for State-of-the-art Space Weather Using data cloud Services to manage harmful algae blooms Solutions to secure online services from Botnets Attacks Analyzing Sport Performance through a Cloud-hosted platform Platform-as-a-Service Data Analytics for the Furniture Industry • All defined architecture and mapped to EOSC-hub generic services • Dedicated overview webpages for each pilot: https://eosc-hub.eu/digital-innovation-hub 25
- 23. EOSC-hub coordinates the development and delivery of EOSC related trainings - Service-specific training - Training to facilitate Open Science – E.g. RDM good practices; IT Service Management; … Training resources online (courses and content) - Innovation Management - Service Management - IT Security - Data management - EOSC services 26 O4: Training and support https://www.eosc-hub.eu/training-material Attend training Become training provider
- 24. 27 O5: Networking EOSC Implementation Projects PaNOSC ENVRI-FAIR EOSC-LIFE ESCAPE … INFRAEOSC-* National networks EGI (NGIs) GEANT (NRENs) OpenAIRE (NOADs) … Competence Centres ELIXIR Fusion Marine EISCAT_3D EPOS-ORFEUS ICOS - eLTER Disaster mitigation
- 25. 28 Competence Centers - Use cases Simplified use case Implemented by Application and data portal with compute and data management back-end • EISCAT_3D • ICOS-eLTER Data and application replication backbone for community centres • ELIXIR • EPOS-ORFEUS ‘Dropbox-like’ access to diverse and scattered data • Marine User workflows over distributed data and compute resources • Fusion • Radio astronomy • Disaster Mitigation
- 26. 29 Uptake of generic services in the Comp. Centres • CC leaders’ meetings • 2 face-to-face meetings • Monthly teleconference • WP10 requirements DB: • User stories • Use cases • Service requirements GREY - considered for adoption, but the integration is yet to start YELLOW - integration is ongoing BLUE - technology is integrated and assessment is yet to finish RED - technology was assessed and was found unsuitable GREEN - technology is integrated and positively evaluated
- 27. 30 An Open Science story we can already implement with EOSC GitHub Your repository EGI Notebooks services Zenodo Your laptop Upload ipynb file Add requirements.txt Execute Data repository MyBinder.org Re-execute Obtain GitHub project reference Provide GitHub project reference Discover Notebook (use DOI) Fellow researchers Journal paper DOI Notebooks tutorial on Tuesday
- 28. 31 An Open Science story we aim for with EOSC GitHub Your repository EGI Notebooks and Binder service Zenodo Your laptop Execute Obtain GitHub project reference Provide GitHub project reference Discover Notebook (use DOI) Fellow researchers Journal paper DOI Distributed big data DataHub B2DROP Etc. Data repository Upload ipynb file Add requirements.txt
- 29. EOSC-hub Week 2019 10-12 April, Prague Highlights: • Keynote by ESFRI Chair • Science discipline specific sessions • Plenary about ‘how to join’ • Service providers’ bootcamp • FAIR services workshop with RDA and OpenAIRE • Training (FitSM, Federated AAI, Data Management, Orchestration)
- 30. To engage with EGI service providers (generic services in EOSC) Conference (May 6-8) Design Your e-Infrastructure workshop (May 9): • Intro to EGI services & analysis of science community use cases 33 EGI Conference and Design Your e-Infratructure workshop
- 31. eosc-hub.eu @EOSC_eu Thank you for your attention! Questions?