OpenNebula TechDay Boston 2015 - Bringing Private Cloud Computing to HPC and Science
2 likes1,664 views
This document discusses bringing private cloud computing to high-performance computing (HPC) and science. It outlines the challenges of using cloud infrastructure for HPC workloads, including performance penalties from virtualization and input/output overhead. It then describes OpenNebula, an open-source tool for managing private clouds that addresses these challenges. Finally, it presents several case studies of research institutions that have implemented private HPC clouds using OpenNebula to gain efficiencies while supporting a variety of applications and user groups.
OpenNebula TechDay Boston 2015 - Bringing Private Cloud Computing to HPC and Science
- 1. Bringing Private Cloud Computing to HPC and Science Ignacio M. Llorente OpenNebula Project Director © OpenNebula Project. Creative Commons Attribution-NonCommercial-ShareAlike License Cloud TechDay Boston, June 29th 2015
- 2. 2Bringing Private Cloud Computing to HPC and Science ! Contents Building Private Cloud Computing to HPC and Science This presentation is about: • The Private HPC Cloud Use Case • Main Challenges for Private HPC Cloud • Resource Provisioning Framework • Private HPC Cloud Case Studies
- 3. 3Bringing Private Cloud Computing to HPC and Science ! The Private HPC and Science Cloud Use Case Different Perspectives to Present Innovations in Cloud Computing! Demand Side (Consumption Model) Supply Side (Provisioning Model) HPC & Science Applications
- 4. 4Bringing Private Cloud Computing to HPC and Science ! The Private HPC and Science Cloud Use Case The Pre-cloud Era! LRMS (LSF, PBS, SGE…) Grid Middleware AccessProvision
- 5. 5Bringing Private Cloud Computing to HPC and Science ! The Private HPC and Science Cloud Use Case OpenNebula as an Infrastructure Tool – Enhanced Capabilities! Virtual Worker Nodes LRMS (LSF, PBS, SGE…) Grid Middleware AccessProvisionService • Common interfaces • Grid integration • Custom environments • Dynamic elasticity • Consolidation of WNs • Simplified management • Physical – Virtual WNs • Dynamic capacity partitioning • Faster upgrades Service/Provisioning Decoupling!
- 6. 6Bringing Private Cloud Computing to HPC and Science ! The Private HPC and Science Cloud Use Case OpenNebula as an Provisioning Tool – Enhanced Capabilities! Pilot Jobs, SSH… IaaS Interface AccessProvisionService • Simple Provisioning Interface • Raw/Appliance VMs • Dynamic scalable computing • Custom access to capacity • Not only batch workloads • Not only scientific workloads • Improve utilization • Reduced service management • Cost efficiency
- 7. 7Bringing Private Cloud Computing to HPC and Science ! Main Challenges for Private HPC Cloud Main Demands from Engineering, Research and Supercomputing ! Flexible Definition of Multi-tier Applications Resource Management Application Performance Provisioning Model
- 8. 8Bringing Private Cloud Computing to HPC and Science ! A Comprehensive Framework to Manage Complex Applications • Several tiers • Deployment dependencies between components • Each tier has its own cardinality and elasticity rules Main Challenges for Private HPC Cloud Execution of Multi-tiered Applications ! Front-end Worker Nodes { "name": ”Computing_Cluster", "deployment": "straight", "roles": [ { "name": "frontend", "vm_template": 0 }, { "name": "worker", "parents": frontend, "cardinality": 2, "vm_template": 3, "min_vms" : 1, "max_vms" : 5, "elasticity_policies" : { ”expressions" : ”CPU> 90%”, "type" : "CHANGE", "adjust" : 2, "period_number" : 3, "period" : 10}, …
- 9. 9Bringing Private Cloud Computing to HPC and Science ! Management of interconnected multi-VM applications: • Definition of application flows • Catalog with pre-defined applications • Sharing between users and groups • Management of persistent scientific data • Automatic elasticity Main Challenges for Private HPC Cloud Using the Cloud – Execution of Multi-tiered Applications !
- 10. 10Bringing Private Cloud Computing to HPC and Science ! Main Challenges for Private HPC Cloud Performance Penalty as a Small Tax You Have to Pay! Overhead in Virtualization • Single processor performance penalty between 1% and 5% • NASA has reported an overhead between 9% and 25% (HPCC and NPB)1 • Growing number of users demanding containers (OpenVZ and LXC) Need for Low-Latency High-Bandwidth Interconnection • Lower performance, 10 GigE typically, used in clouds has a significant negative (x2-x10, especially latency) impact on HPC applications1 • FermiCloud has reported MPI performance (HPL benchmark) on VMs and SR-IOV/Infiniband with only a 4% overhead2 • The Center for HPC at CSR has contributed the KVM SR-IOV Drivers for Infiniband3 (1) An Application-Based Performance Evaluation of Cloud Computing, NASA Ames, 2013 (2) FermiCloud Update, Keith Chadwick!, Fermilab, HePIX Spring Workshop 2013 (3) http://wiki.chpc.ac.za/acelab:opennebula_sr-iov_vmm_driver , 2013 Overhead in Input/Output • Growing number of Big Data apps • Support for multiple system datastores including automatic scheduling
- 11. 11Bringing Private Cloud Computing to HPC and Science ! Optimal Placement of Virtual Machines • Automatic placement of VM near input data • Striping policy to maximize the resources available to VMs Fair Share of Resources • Resource quota management to allocate, track and limit resource utilization Management of Different Hardware Profiles • Resource pools (physical clusters) with specific Hw and Sw profiles, or security levels for different workload profiles (HPC and HTC) Isolated Execution of Applications • Full Isolation of performance-sensitive applications Provide VOs with Isolated Cloud Environ • Automatic provision of Virtual Data Centers Hybrid Cloud Computing • Cloudbursting to address peak or fluctuating demands for no critical and HTC workloads Main Challenges for Private HPC Cloud Resource Management!
- 12. 12Bringing Private Cloud Computing to HPC and Science ! The Resource Provisioning Framework Challenges from the Organizational Perspective! Bio HTC Simulations HPC Simulations Big Data Analysis Comprehensive Framework to Manage User Groups • Several divisions, units, organizations… • Different workloads profiles • Different performance and security requirements • Dynamic groups that require admin privileges => From many private clusters to a single consolidated environment
- 13. 13Bringing Private Cloud Computing to HPC and Science ! Challenges from the Infrastructure Perspective! DC ESRIN DC ESACPublic Clouds Comprehensive Framework to Manage Infrastructure Resources • Scalability: Several DCs with multiple physical clusters • Outsourcing: Access to several clouds for cloudbursting • Heterogeneity: Different hardware for specific workload profiles The Resource Provisioning Framework
- 14. 14Bringing Private Cloud Computing to HPC and Science ! The Goal: Dynamic Allocation of Private and Public Resources to Groups of Users! DC West Coast DC EuropePublic Clouds Bio HTC Simulations HPC Simulations Big Data Analysis The Resource Provisioning Framework
- 15. 15Bringing Private Cloud Computing to HPC and Science ! Definition of vDCs! DC West Coast DC EuropePublic Clouds Bio HTC Simulations HPC Simulations Big Data Analysis The Resource Provisioning Framework
- 16. 16Bringing Private Cloud Computing to HPC and Science ! The Resource Provisioning Framework Admins in each Group Manage their Own Virtual Private Cloud ! !• Each Group has an admin • Delegation of management in the Group (and its VDC) • Only virtual resources, not the underlying physical infrastructure vDC Admin View
- 17. 17Bringing Private Cloud Computing to HPC and Science ! Users in each Group Access to its Own Virtual Private Cloud (VDC) ! DC West Coast DC EuropePublic Clouds Bio HTC Simulations HPC Simulations Big Data Analysis Cloud API The Resource Provisioning Framework
- 18. 18Bringing Private Cloud Computing to HPC and Science ! New Level of Provisioning: IaaS as a Service! DC West Coast DC EuropePublic Clouds Big Data Analysis CloudAdminsvDCAdminsConsumers HPC Simulations Bio HTC Simulations The Resource Provisioning Framework
- 19. 19Bringing Private Cloud Computing to HPC and Science ! Benefits! • Partition of cloud resources • Complete isolation of users, organizations or workloads • Allocation of Clusters with different levels of security, performance or high availability to different groups with different workload profiles • Containers for the execution of virtual appliances (SDDCs) • Way of hiding physical resources from Group members • Simple federation and scalability of cloud infrastructures beyond a single cloud instance and data center • … The Resource Provisioning Framework
- 20. 20Bringing Private Cloud Computing to HPC and Science ! Private HPC Cloud Case Studies One of Our Main User Communities! Supercomputing Centers Research Centers Distributed Computing Infrastructures Industry
- 21. 21Bringing Private Cloud Computing to HPC and Science ! Leibniz Supercomputing Center! Private HPC Cloud Case Studies Nodes KVM on 78 nodes (5 TB RAM – 624 cores) Network Open vSwitch Storage 100TB NAS with NFS AuthN LDAP Linux SLES11 Interface Sunstone Self-service and EC2 API App Profile Legacy, HTC and MPI HPC http://www.lrz.de/services/compute/cloud_en/
- 22. 22Bringing Private Cloud Computing to HPC and Science ! FermiCloud! Nodes KVM on 29 nodes (2 TB RAM – 608 cores) Koi Computer Network Gigabit and Infiniband Storage CLVM+GFS2 on shared 120TB NexSAN SataBeats AuthN X509 Linux Scientific Linux Interface Sunstone Self-service and EC2 API App Profile Legacy, HTC and MPI HPC http://www-fermicloud.fnal.gov/ Typical Workloads • Production VM-based batch system via the EC2 emulation => 1,000 VMs • Scientific stakeholders get access to on- demand VMs • Developers & integrators of new Grid applications Private HPC Cloud Case Studies
- 23. 23Bringing Private Cloud Computing to HPC and Science ! CESGA Cloud! Nodes KVM on 35 nodes (0.6 TB RAM – 280 cores) HP ProLiant Network 2 x Gigabit (1G and 10G) Storage ssh from remote EMC storage server AuthN X509 and core password Linux Scientific Linux 6.4 Interface Sunstone Self-service and OCCI App Profile Individual VMs and virtualised computing clusters Typical Workloads • 160 users • Genomic, rendering… • Grid services on production at CESGA • Node at FedCloud project • UMD middleware testing http://cloud.cesga.es/ Private HPC Cloud Case Studies
- 24. 24Bringing Private Cloud Computing to HPC and Science ! SARA Cloud! Nodes KVM on 23 HPC nodes (736 cores, 6 TB RAM, 76 TB SSD) KVM on 12 HPC nodes (192cores, 3 TB RAM) Network 2 x Gigabit (10G) with Arista switch Storage 800 TB central storage on a CEPH cluster (50 OSD nodes) AuthN Core password Linux CentOS Interface Sunstone and OCCI App Profile MPI clusters, windows clusters and independent VMs http://www.cloud.sara.nl Typical Workloads • Ad-hoc clusters with MPI and pilot jobs • Windows clusters for Windows-bound software • Single VMs, sometimes acting as web servers to disseminate results Private HPC Cloud Case Studies
- 25. 25Bringing Private Cloud Computing to HPC and Science ! SZTAKI Cloud! Nodes KVM on 8 nodes (2 TB RAM – 512 cores) DELL PowerEdge Network Redundant 10Gb Storage Dell storage servers: iSCSI ( 36TB ) and CEPH ( 288 TB ) AuthN X509 Linux CentOS 6.5 Interface Sunstone Self-service, EC2 and OCCI App Profile Individual VMs and virtualised computing cluster http://cloud.sztaki.hu/ . Typical Workloads • Run standard and grid services (e.g.: web servers, grid middleware…) • Development and testing of new codes • Research on performance and opportunistic computing Private HPC Cloud Case Studies
- 26. 26Bringing Private Cloud Computing to HPC and Science ! KTH Cloud! Nodes KVM on 768 cores (768 GB RAM) HP ProLiant Network Infiniband and Gigabit Storage NFS and LVM AuthN X509 and core password Linux Ubuntu Interface Sunstone self-service, OCCI and EC2 App Profile Individual VMs and virtualised computing cluster http://www.pdc.kth.se/ Typical Workloads • Mainly BIO • Hadoop, Spark, Galaxy, Cloud Bio Linux… Private HPC Cloud Case Studies
- 27. 27Bringing Private Cloud Computing to HPC and Science ! We Will Be Happy to Answer Your Questions ! Questions? OpenNebula.org @OpenNebula
- 28. 28Bringing Private Cloud Computing to HPC and Science ! 20 - 22 October 2015 in BARCELONA OpenNebula CONFERENCE 3rd edition!
- 29. one