Mixing bare-metal and virtualized workloads on OpenStack - 2014
1 like102 views
The document discusses managing heterogeneous cloud infrastructures, specifically focusing on OpenStack for hosting hybrid applications combining bare-metal and virtualized resources. It outlines the evolution and opportunities within cloud environments, the various challenges faced in integrating and managing these resources, and proposes solutions leveraging OpenStack's capabilities. Key points include a need for enhanced management and user experience, addressing existing gaps in OpenStack, and designing for security and isolation across hybrid environments.
Mixing bare-metal and virtualized workloads on OpenStack - 2014
- 1. 1 Managing Heterogeneous Cloud Infrastructure with OpenStack Example: Hosting ‘Hybrid’ Applications Comprising Bare-Metal and Virtualized Resources OpenStack Israel, June 2nd, 2014, Hertzliya IBM Research – Haifa Alex Glikson Ezra Silvera
- 2. 2 © 2014 IBM Corporation Outline Heterogeneous Clouds ‒ Background & Motivation ‒ Opportunities and Challenges Hosting ‘Hybrid’ (Bare-Metal + Virtualized) Applications ‒ Goals ‒ Scenarios ‒ Solution Approach ‒ Technical Challenges and Design Considerations ‒ Example Summary, Q&A
- 4. 4 © 2014 IBM Corporation Evolution of IaaS Usage Patterns and Offerings Large variety of configurations ‒ E.g., AWS now offers >60 instance configurations Rise of bare-metal providers ‒ IBM/SoftLayer, Internap, etc Small set of standardized (virtual) hardware configurations ‒ e.g., 1 instance type in AWS EC2 (2006) Commonly Offered Hardware Configurations Growing spectrum of workloads ‒ E.g., ‘legacy’ business applications, HPC, analytics, etc Dev & test Web applications (‘cloud-native’) Common Usage Patterns Agility, efficiency, elasticity, self-service, pay-per-useValue proposition NowEarly Days Trend: Increased Cloud Infrastructure Heterogeneity (including bare-metal) to Support Larger Variety of Applications with Special Requirements
- 5. 5 © 2014 IBM Corporation Embracing Infrastructure Heterogeneity for the Benefit of the Application: Example Virtualization Network Storage Compute CPU/mem/disk balance CPU-rich mem-rich disk-rich … CPUs GPGPUs Accelerators Local HDD SSD SAN NAS … …1 Gbit shared 10 Gbit shared 10 Gbit reserved … bare-metal KVM LXC VMware … Tier-1: web app Tier-2: 3D rendering Tier-4: Designs DB Tier-3: Hadoop balanced Example: Application for Collaborative 3D Design
- 6. 6 © 2014 IBM Corporation Embracing Infrastructure Heterogeneity for the Benefit of the Application: Example Virtualization Network Storage Compute CPU/mem/disk balance CPU-rich mem-rich disk-rich … CPUs GPGPUs Accelerators Local HDD SSD SAN NAS … …1 Gbit shared 10 Gbit shared 10 Gbit reserved … bare-metal KVM LXC VMware … Tier-1: web app Tier-2: 3D rendering Tier-4: Designs DB Tier-3: Hadoop balanced Example: Application for Collaborative 3D Design 4 4 4 4 3 3 3 3 2 2 2 2 2 3 41 1 1 1 1
- 8. 8 © 2014 IBM Corporation Opportunities and Challenges Business opportunities enabled by infrastructure heterogeneity ‒ Service/application providers: • Cloud adoption in ‘traditional’ market segments agility and efficiency – E.g., manufacturing, finance, scientific, etc – IDC: the (extended) HPC market alone is larger than IaaS ($15B vs $9B) ‒ Cloud/infrastructure providers: • Ability to better fit to specific classes of applications differentiation – Especially relevant given the overall commoditization of the IaaS market, recent price wars, etc
- 9. 9 © 2014 IBM Corporation Opportunities and Challenges Challenge: cloud infrastructure natively designed for: ‒ Underlying infrastructure heterogeneity ‒ Ability to seamlessly host applications with special requirements ‒ Still preserve the characteristics of the cloud model • Agility, efficiency, elasticity, etc
- 10. 10 © 2014 IBM Corporation Heterogeneous Cloud Requirements: Cloud User Cloud user perspective ‒ Access to a wider range of offered HW/SW configurations • Performance, security/isolation • Compute, storage, network ‒ Same or better user experience as with ‘regular’ cloud, e.g.: • Same: single/unified catalog of configuration ‘flavors’, images, etc • Same: seamless network connectivity, storage access, etc • Same: higher-level services (orchestration, LBaaS, DBaaS, etc) • Better: higher-level application abstraction, hiding the details of the underlying HW (critical with increased complexity of underlying HW) – Specify goals/requirements, rather than specific resource types/models etc
- 11. 11 © 2014 IBM Corporation Heterogeneous Cloud Requirements: Cloud Provider Cloud provider perspective ‒ Minimize management cost by unified processes and APIs • Provisioning, inventory, monitoring, life cycle operations, visualization, etc – Unified resource model ‒ Secure multi-tenancy across heterogeneous resources ‒ Elasticity & efficiency • Smart scheduling, to overcome resource fragmentation • Dynamic balancing between HW-compatible pools by “repurposing” (subject to compatibility) – Capacity monitoring & prediction – Automated evacuation & re-provisioning
- 12. 12 © 2014 IBM Corporation Heterogeneous Cloud: OpenStack Perspective OpenStack is following a similar pattern ‒ Started primarily with dev & test and cloud-native applications • Simplistic approach to flavors, scheduling, performance optimization, etc ‒ Extensive efforts to enable adoption by additional applications, e.g.: • Hadoop (Sahara) • DBaaS (Trove) • Bare-metal (Ironic, TripleO, Tuskar) • Scheduling (Gantt) • Reservations (Climate) • Policies (Congress) • Etc ‒ Growing community interest in supporting heterogeneous clouds, e.g.: • Multi-Tenant Bare Metal Provisioning (Mon, 11:15, B206; Tue, 2:50pm, B301) • Deploying OpenStack in a Multi-Hypervisor Environment (Wed, 11am, B101) • IBM, SoftLayer and OpenStack - Present and Future (Wed, 11:50am, B312) • Many more related sessions…
- 13. 13 © 2014 IBM Corporation Heterogeneous Cloud: OpenStack to the Rescue! Claim: OpenStack provides a good basis for building heterogeneous clouds • But few gaps yet to be addressed
- 14. 14 © 2014 IBM Corporation Hosting ‘Hybrid’ (Bare-Metal + Virtualized) Applications
- 15. 15 © 2014 IBM Corporation Hybrid Applications – Background and Goals Hybrid environment – includes both physical and virtual resources ‒ Pools: Resources are grouped into bare-metal and virtual pools Hybrid application – a composite (e.g., multi-tier) application that spans both virtual and physical server(s) Goals ‒ Admin: Minimize management complexity and overhead. Unified management across bare-metal and virtualized pools. ‒ User: Simplified ongoing management and application life cycle. Design guideline: If possible try to use OpenStack native solution ‒ E.g., no external coordinators, orchestration, management
- 16. 16 © 2014 IBM Corporation Pool 3: Virt Type 2 Pool 4: Bare metalPool 1: Spare (BM) Pool 2: Virt Type 1 Tenant ATenant B Hybrid Applications – Background and Goals
- 17. 17 © 2014 IBM Corporation Scenarios
- 18. 18 © 2014 IBM Corporation Scenario 1: Node Repurposing Support native policies that dynamically balance the virtual and physical pools (e.g., moving servers from one to another) Repurposing Virtualized node (Hypervisor) Physical node 1. Detect resource congestion in the physical pool 2. Identify a candidate host to be re-purposed into physical 3. Trigger “evacuate“ for all VMs on the selected host 4. Assign physical host to the BM pool ‒ Server is ready to be deployed as bare-metal application (later on) Bare metal pool Bare metal pool
- 19. 19 © 2014 IBM Corporation Scenario 2: Runtime Decision on Target Pool Scenario: decide on the server type (BM/VM) upon deployment E.g., according to performance requirements 1. User deploys an application without specifying the server type 2. The system automatically choose server type 3. Automatically choose/construct actual image for deployment ‒ Option 1: dynamic adaptation ‒ Option 2: Maintain multiple ‘versions’ 4. Deploy using corresponding provisioning mechanism ‒ leveraging Ironic ‒ Apply adaptation upon boot (e.g., leveraging Heat)
- 20. 20 © 2014 IBM Corporation Solution Approach
- 21. 21 © 2014 IBM Corporation Management Approaches Possible solutions range from 'share nothing to 'share everything' ‒ 'share nothing’ - dedicated services for each pool ‒ ‘share everything’ - Use single copy of each service ‒ Some of the services are shared (e.g., regions, cells) OpenStack can natively handle bare metal and virtual hosts, so why not combining them together in a single OpenStack? We focused on the 'shared everything' architecture ‒ Defined as 'Integrated Management‘
- 22. 22 © 2014 IBM Corporation Integrated Management: Basic Architecture Special resource types, mapped to specially designed resource pools Use multiple host aggregates, including ‘bare-metal’ ‒ Use aggregate filters for scheduling Networking ‒ Admin network for provisioning and management of compute nodes ‒ Separate management network for BM machines hosting user applications ‒ Data network combining both VMs and BM nodes Controller node mysql-server rabbit-server neutron-server glance Network node Neutron.*-plugin-agent Neutron-l3-agent Neutron-dhcp-agent Neutron.*-plugin-agent Neutron-l3-agent BM computeCompute node nova-compute Neutron.*-plugin-agent Compute node nova-compute Neutron.*-plugin-agent Compute node nova-compute Neutron.*-plugin-agent BM node BM node BM node External Data/Guests Management API Internet BM Mgt BM node BM node BM spare
- 23. 23 © 2014 IBM Corporation Advantages of Integrated Management approach Native and simple OpenStack-based solution ‒ Doesn’t require external orchestration ‒ No need to coordinate between services of different pools Simplified administration ‒ Controlling all pools/resources from one place in a unified manner ‒ Reduce configuration complexity and overhead ‒ Better diagnostics and root cause analysis Improved user experience and manageability ‒ Maintain/expose combined topology view for different server types ‒ Hide the underlying infrastructure details from the user (if needed)
- 24. 24 © 2014 IBM Corporation Technical Challenges and Design Considerations
- 25. 25 © 2014 IBM Corporation Challenges Examples OpenStack has some gaps when managing bare-metal and virtual machines simultaneously Networking: Simultaneously manage both virtual and physical networks ‒ Network isolation for tenants should combine both BM and VMs ‒ E.g., Use tunnels/overlays connecting physical nodes and VMs ‒ E.g., Use multiple plugins (e.g., OVS, openflow) simultaneously to gain advantages for each pool (if possible) Managing Bare-Metal workloads ‒ Security in multi-tenant environments (e.g., device firmware)
- 26. 26 © 2014 IBM Corporation Challenges Examples (Cont.) - Compute Enhanced scheduler to better support heterogeneity ‒ Use different placement policies for different pools (e.g., look at GPU consumption rather then CPU/memory) Support instance management for BM ‒ Actions for BM instances may differ from actions on VMs ‒ E.g., implement: Console, Explore usage of: Suspend, Resize Add hierarchical relations between BM instance and the compute node running on it ‒ E.g., Propagate BM delete command to the VM level Support aggregates for specific BM nodes
- 27. 27 © 2014 IBM Corporation Challenges Examples (Cont.) Image Management ‒ Enhance access control for images in Glance ‒ Support run-time image selection ‒ Support dynamic adaptation of images UI: extend UI to support Hybrid applications/instances ‒ Add bare-metal node management in Horizon ‒ Physical machines are shown as Hypervisors
- 28. 28 © 2014 IBM Corporation Hybrid Application example
- 29. 29 © 2014 IBM Corporation Example – Deploying Hybrid WordPress Apache+WordPress on VM and MySQL on physical node Setup and configuration ‒ A single HEAT template used for the two nodes ‒ Using host aggregates for scheduling ‒ Used diskimage-builder to build the baremetal images • cloud-init element installed
- 30. 30 © 2014 IBM Corporation Example – Deploying Hybrid WordPress Example gaps reflected in Horizon: ‒ BM nodes are shown as Hypervisors ‒ Deployment images are exposed to users
- 31. 31 © 2014 IBM Corporation Summary
- 32. 32 © 2014 IBM Corporation Summary Heterogeneous cloud environments are gaining momentum ‒ Critical in order to host a broad spectrum of applications OpenStack is a promising solution to manage hybrid clouds ‒ By using integrated management we can have • Simple native OpenStack solution • Simplified administration and enhanced user experience Need a careful design to ensure all requirements (e.g., security, isolation) are maintained across the hybrid environment Some gaps need to be addressed in OpenStack
- 33. 33 © 2014 IBM Corporation Q & A