ATMOSPHERE at HPC2018 – Fogbow: Middleware for the Federation of IaaS Cloud Providers
0 likes120 views
The document discusses Fogbow, a middleware developed under the European Commission's Horizon 2020 project for federating IaaS cloud providers. It details different deployment models for IaaS, requirements for cloud federation, and Fogbow's architecture and functionalities, including its open-source nature and support for various cloud orchestrators. Additionally, it highlights success stories where Fogbow has been implemented in federating private clouds across various institutions.
ATMOSPHERE at HPC2018 – Fogbow: Middleware for the Federation of IaaS Cloud Providers
- 1. Co-funded by the European Commission Horizon 2020 - Grant #777154 Fogbow: a Middleware for the Federation of IaaS Cloud Providers Francisco Brasileiro fubica@computacao.ufcg.edu.br atmosphere-eubrazil.eu @AtmosphereEUBR
- 2. Agenda • IaaS deployment models – Motivation for the federation of IaaS providers • Key requirements for the implementation of cloud federation • Fogbow’s design • Use cases High Performance Computing Workshop, Cetraro, July 5th, 2018 2
- 3. IaaS deployment models • Public cloud providers – Shared multi-site infrastructures – Economy of scale leads to very efficient providers – Provides higher elasticity to users – Pay-as-you-go pricing scheme reduces upfront investments to users – Might raise security and privacy concerns 3High Performance Computing Workshop, Cetraro, July 5th, 2018
- 4. IaaS deployment models • Private cloud providers – Dedicated (typically) single-site infrastructures – Brings no new security and privacy concerns – Smaller than public providers, thus tend to be less efficient – Normally, less elastic than public providers • More restrictive usage quotas 4High Performance Computing Workshop, Cetraro, July 5th, 2018
- 5. Enhanced deployment models • Multi-clouds (brokers) – Allow multiple public clouds to be used, increasing the portfolio of services available to users – Allow resource allocation driven by some objective function (eg. minimize cost) – If implemented in-house, may mitigate vendor lock-in issue of public clouds 5High Performance Computing Workshop, Cetraro, July 5th, 2018
- 6. Enhanced deployment models • Hybrid clouds – Typically, extends the infrastructure of a private cloud by allowing some of the workload to be outsourced to a public cloud – Improves the efficiency of the private cloud – Provides access to multiple geographically distributed sites • Useful for fault tolerance and attend geo-sensitive workloads – Sensitive workload might need to run in the private part of the infrastructure – Business model defined by the public clouds 6High Performance Computing Workshop, Cetraro, July 5th, 2018
- 7. Enhanced deployment models • Federated/community clouds – Shared multi-site infrastructures • Fine control on how sharing is performed at each autonomous site (from the providers point of view) – Depending on how members trust each other, may not raise extra security and privacy concerns – May allow for higher elasticity – More flexibility in defining business models 7High Performance Computing Workshop, Cetraro, July 5th, 2018
- 8. Requirements for the federation of IaaS cloud providers • Key requirements for effective federation – Single sign-on – Single API • Additional requirements – Federation-wide services • Membership • Accounting • Catalogue of available services • Private networks spanning multiple providers • … 8High Performance Computing Workshop, Cetraro, July 5th, 2018
- 9. Approaches • Single IaaS Cloud Orchestrator that supports federation – Offers common (and extensive) API by construction – Relies on a federation-wide Identity Provider (IdP) • Multiple IaaS Cloud Orchestrators – Enhance the different cloud orchestrators to support: • A chosen common (and normally narrow) API (eg. OCCI) • Federation-wide IdP • Federation-wide auxiliary services (eg. image management, membership, etc.) – Relies on standards for conciliating different cloud orchestrators peculiarities • IaaS Cloud Federation middleware – Decouples federation functionalities from the orchestrator functionalities – Offers common (more easily extensible) API by construction – Uses adaptors (plug-ins) to provide interoperation with different orchestrators – Relies on standards for conciliating different orchestrators peculiarities 9High Performance Computing Workshop, Cetraro, July 5th, 2018
- 10. Using Fogbow to federate IaaS clouds • Fogbow is a suite of microservices to provide effective federation of IaaS cloud providers – An allocation management service that provides a standard API to manage resources anywhere in the infrastructure – A membership service to control which providers belong to the federation – Several auxiliary services to provide local and federated services atop the above two basic services • Open source project managed by the Federal University of Campina Grande – More details at http://fogbowcloud.org/ and http://github.com/fogbow 10High Performance Computing Workshop, Cetraro, July 5th, 2018
- 11. Fogbow’s architecture 11High Performance Computing Workshop, Cetraro, July 5th, 2018
- 12. Plugin-based approach 12 • Fogbow fully supports the OCCI API, and extends it to support: – Membership operations – Resource instantiation at remote locations • Interoperability plugins make it simple to support different cloud orchestrators • Behavioural plugins allow cloud administrators to customize the federation functionalities that they deploy in their local cloud High Performance Computing Workshop, Cetraro, July 5th, 2018
- 13. Multi-level A&A 13 • A&A is performed at multiple levels – A local request carries a federation access token that is A&A by customized A&A federation services – The allocation manager performs A&A of remote requests considering both the federation access token and the identity of the requesting allocation manager – A&A federation tokens are mapped to local access tokens that are used to access the local cloud – Mapping is autonomously defined by each cloud administrator High Performance Computing Workshop, Cetraro, July 5th, 2018
- 14. Implemented plugins • Interoperability – OpenStack, OpenNebula, CloudStack, Amazon EC2, Microsoft Azure, VMWare (under development) • Behavioural – Authentication of federation users • LDAP, X.509, Shibboleth – Authorization at the federation level • Allow-all, VOMS, OAuth – Federation to local cloud user mapping • All-to-one, VOMS-based, OAuth-based 14High Performance Computing Workshop, Cetraro, July 5th, 2018
- 15. Internet-friendly communication 15 • A messaging service allows membership and allocation managers to communicate with each other locally and remotely, without requiring them to have their endpoints publicly accessible • A reverse tunnelling service provides public IP access to all virtual machines created in the private clouds, even if the local cloud offers only private IPs to these virtual machines High Performance Computing Workshop, Cetraro, July 5th, 2018
- 16. Auxiliary services 16 Local microservices - Cloudbursting to other federation members - Barter-based business model for the opportunistic execution of Bag-of-Tasks applications - Accounting and billing Federated microservices - Federated private networks (vanilla version available, more sophisticated SDN-based versions under development) - Service catalogue (under development) High Performance Computing Workshop, Cetraro, July 5th, 2018
- 17. Success stories so far • Fogbow has been used to federate private clouds in a research project that involved Brazilian and European institutions (http://eubrazilcloudconnect.eu/) – VOMS was used to implement the federation level A&A – This federation had OpenStack and OpenNebula clouds • The Brazilian NREN (RNP) is currently experimenting the use of Fogbow to federate its private clouds and those of some of its clients – RNP’s federation of identity providers – CAFe – (Shibboleth-based) is used to implement authentication at the federation level – This federation has OpenStack, OpenNebula and CloudStack clouds • The ATMOSPHERE project (co-funded by RNP/Brazil and the EC) is enhancing Fogbow to support the creation of private networks spanning multiple sites in a federation of IaaS cloud providers 17High Performance Computing Workshop, Cetraro, July 5th, 2018
- 18. Thanks for your attention! 18 Check it out here! Get on board! Download and use it! High Performance Computing Workshop, Cetraro, July 5th, 2018