Progress of the Helix Nebula Science Cloud PCP Project
- 2. Helix Nebula – The Science Cloud with Grant Agreement 687614 is a Pre-Commercial Procurement Action funded by H2020 Framework Programme Progress of the Helix Nebula Science Cloud PCP project 19 October 2016 Bob Jones CERN IT department 23/11/2016 This work is licensed under the Creative Commons Attribution-ShareAlike 4.0 International License. The content of this presentation is the sole responsibility of the authors and does not necessarily represent the views expressed by the European Commission or its services.
- 3. The Helix Nebula Science Cloud public-private partnership
- 4. D. Giordano HN GA8 21/09/2016 Series of short procurements of increasing size and complexity 4 Augmenting CERN’s scientific computing programme with commercial cloud services
- 5. D. Giordano WLCG Workshop 9/10/2016 CERN cloud procurements 2015-2016 5
- 6. 6 The Helix Nebula Initiative Brings together • research organisations, • data providers, • publicly funded e- infrastructures, • commercial cloud service providers In a hybrid cloud with procurement and governance approaches suitable for the dynamic cloud market In-house
- 7. Major challenges What if I get locked in? Are there relevant standards I should be looking into? What happens to my data? How do I get a good deal? What happens to my IT staff? How can I compare contracts & SLAs? What is PCP? What are the others doing? How can I allocate costs? What services do I need? 1. Cloud computing is disrupting the way IT resources are provisioned 2. In-house resources, publicly funded e-infrastructure and commercial cloud services are not integrated to provide a seamless environment 3. Current organisational and financial models are not appropriate 4. The new way of procuring cloud services is also a matter of skills and education 5. Legal impediments exist
- 8. Provides a landscape of cloud procurement in the European public research sector Makes pragmatic recommendations for the procurement of cloud services by PROs in Europe Provides a guide to cloud procurement, supported by best practices adopted worldwide Proposes actions within the pillar three of the Digital Single Market Strategy which focus on maximising the growth potential of the digital economy The PICSE Roadmap 4/5/2016 11 www.picse.eu/roadmap
- 9. HNSciCloud Joint Pre-Commercial Procurement Bob Jones, CERN 9 Procurers: CERN, CNRS, DESY, EMBL-EBI, ESRF, IFAE, INFN, KIT, STFC, SURFSara Experts: Trust-IT & EGI.eu The group of procurers have committed • Procurement funds • Manpower for testing/evaluation • Use-cases with applications & data • In-house IT resources Resulting services will be made available to end- users from many research communities Co-funded via H2020 Grant Agreement 687614 Total procurement budget >5M€
- 10. What will be procured A hybrid cloud platform for the European research community 11/23/2016 10 HNSciCloud PCP Source:CloudComputingforGovies,DLTSolutions, DavidBlankenhorn,VanRistauandCaronBeesley Combining services at the IaaS level to support science workflows The R&D services to be developed are to be integrated with Resources in data centres operated by the buyers group European-scale publicly funded e-Infrastructures
- 11. Challenges Innovative IaaS level cloud services integrated with procurers in-house resources and public e-infrastructure to support a range of scientific workloads Compute and Storage support a range of virtual machine and container configurations including HPC working with datasets in the petabyte range Network Connectivity and Federated Identity Management provide high-end network capacity via GEANT for the whole platform with common identity and access management Service Payment Models explore a range of purchasing options to determine those most appropriate for the scientific application workloads to be deployed Bob Jones, CERN 11
- 12. HNSciCloud project phases Preparation •Analysis of requirements, current market offers and relevant standards •Build stakeholder group •Develop tender material Implementation and sharing Jan’16 Dec’18 Each step is competitive - only contractors that successfully complete the previous step can bid in the next 4/5/2016 12 200+ downloads 70+ requests for clarifications 4 Designs 3 Prototypes 2 Pilots Call-off Feb’17 Call-off Oct’17 Tender Jul’16
- 13. Bob Jones, CERN 13 Research Infrastructures are facilities, resources or services of a unique nature identified by European research communities to conduct top-level research activities in all fields Interested Research Infrastructures: • EPOS, ESA, ESS • clusters: CORBEL, ASTERICS-OBELICS Will form an observer group
- 14. Launch event ICRI 2016, Cape Town - South Africa e-INFRASTRUCTURE Research Infrastructure as key nodes of e-Infrastructure for Research • Advanced e-Infrastructure of all Research Infrastructures • Optimal interfaces between RIs and the external e-Infrastructure (Networks, Cloud, HPC, HTC) • Data Quality assessment at RIs and setting quality standards for broad use • Data access to “enabling data” i.e. data completed with adequate metadata, traceable origin, FAIR • Long Term preservation of “useful data” • Key role of “public” institutions and interplay with commercial clouds/repositories Giorgio Rossi Chair
- 15. Helge Meinhard, CERN, 17 March 2016 • Foreseen users: - bioinformaticians who will do most of the large scale processing - less tech-savvy end-users will perform analysis • Data types: genotype & phenotype information. Data can be assumed to be anonymised but it is still sensitive
- 16. Helge Meinhard, CERN, 17 March 2016 Long tail of science • Foreseen users: individual researchers/small labs in the need of accessing highly performant IT resources to analyse their data on • Data types: Due to the nature of the use-cases, the exact type of datasets can’t be predicted upfront. However, the infrastructure will need to ensure datasets will be kept private to the single user, with the possibly to share them among other users / publicly provided sufficient authorisation is granted.
- 17. Helge Meinhard, CERN, 17 March 2016 • Foreseen users: the EuroBioImaging consortia through the representatives in EMBL • Data types: private and public datasets consisting of images coming from human cells, Drosophila and fungi, with a plan to further extend the coverage adding more datasets and cell types. No sensitive data are currently foreseen.
- 18. European Open Science Cloud 11/23/2016 18 https://bit.ly/cloudeu
- 19. Widening access (2/2): e-Infras as aggregators of demand 19 EOSC Scientific Users Commercial services e-Infrastructures EU H2020 funding € € Procurement Grants Augusto Burgueño Arjona, head of the Unit "eInfrastructure & Science Cloud“, DG CNECT, EC, Sept’16
- 20. Widening access (1/2): e-Infrastructures as service providers 20 EOSC Scientific Users Industry Public Sector e-Infrastructures Augusto Burgueño Arjona, head of the Unit "eInfrastructure & Science Cloud“, DG CNECT, EC, Sept’16 HNI 2.0: Building value chains with data intensive science
Editor's Notes
- #10: Research organisations from 7 countries have proposed to work together to develop a cross-border joint procurement of innovative cloud services. This is an important change: never before have research organisations across Europe pooled their funds and resources to procure cloud services to support their scientific programmes. eduGAIN federated identity mgmt. should include commercial identity providers taking into account multiple levels of assurance.