Running OpenStack in Production - Barcamp Saigon 2016
1 like303 views
The document outlines a presentation on deploying OpenStack for mission-critical and production workloads, emphasizing the importance of high availability and the architecture of OpenStack components such as Ceph for storage and networking solutions. It provides technical specifications and best practices for configuring and sizing Ceph clusters, as well as considerations for compute resources and scalability. Additionally, it discusses challenges faced, areas for improvement, and the skill sets needed for successful implementation.
Running OpenStack in Production - Barcamp Saigon 2016
- 1. 1 Running in Production Thang Man Lead Cloud Engineer FimPlus Digital @Barcamp Saigon 2016
- 2. Mission-critical Business-critical 2 “Production” in our life and the IT world
- 4. Our Demand Workloads for Infrastructure 4
- 5. Solution Architecture Overview * Juno → Liberty * RHEL → Ubuntu * Swift → CEPH * Not used - Trove 5
- 6. HA for OpenStack Controllers ● MySQL Galera Cluster Active/Active ● HAProxy for Galera Active/Backup ● Stateless services HAProxy Active/Active ● RabbitMQ: Built-in ha-queues ● Virtual IP Pacemaker/Corosync 6
- 7. Networking: A Basic Block ● Aggregated throughput ● Fault tolerant ● Scalable ● Optimized for East-West traffic patterns ● White-box networking 7
- 9. Neutron vs. Nova-network Legacy networking (Nova-network) OpenStack networking (Neutron) Simple, single agent Complex, multiple agents More matured, established Newer, maturing Flat or VLAN Flat, VLAN, Overlays, L2-L3, SDN No plug-in support Plug-in support for 3rd parties Scales well Scaling requires 3rd party plug-ins No multi-tier topologies Multi-tier topologies 9
- 10. CEPH Storage: Network Model ● Cluster should have more 2X-4X capacity than Public network 10
- 12. Rules of Thumb Sizing ● Ceph OSD ○ Disks ■ 8-10 SAS HDDs per 1x 10Gbps ■ ~12 SATA HDDs per 1x 10Gbps ■ 1x SSD for write journal per 4-6 OSD drives ■ JBOD (no RAID configuration) ○ RAM ■ 1GB of RAM per 1 TB of OSD storage space ○ CPU ■ 0.5 CPU core/1 Ghz of a core per OSD disk (1-2 CPU cores for SSD drives) ● Ceph Monitor (min 3 per cluster) ○ 1 GB of RAM per daemon ○ 1 ceph-mon node per 15-20 OSD nodes 12
- 13. Configuration ● Ceph Storage Cluster ○ Multi-pools: SSD, SAS/SATA ○ Replicas of objects: min 2 ○ Data protection: Replicated vs. EC ○ Memory allocator: TCMalloc vs. JEMalloc 13 ● Ceph Block Device (RBD) ○ rbd_cache ● Ceph Object Gateway (RGW) ○ rgw_cache_lru_size ○ rgw_thread_pool_size ○ rgw_override_bucket_index_max_shards ○ LB: LVS (direct routing) vs. HAProxy ○ Web server: Civetweb vs. Nginx vs. Apache
- 14. Compute ● Specialized hardwares ○ GPU ○ SSD locality ○ Encryption keys in USB devices ● Scheduling on specific hosts (e.g. KVM, XenServer), hardwares ○ Host Aggregates ● Separating critical workloads from others ○ Availability Zones 14
- 15. Compute (cont.) ● High performance computing for VMs (*) ○ No overcommit (RAM, CPU) ○ CPU Pinning ○ NUMA Topology awareness ● High availability for VMs ○ Live/Cold Migration ○ Evacuation ○ resume_guests_state_on_host_boot=true ● Scalability for VMs ○ Resize (*) http://www.slideshare.net/vietstack/vietopenstack-meetup-7th-high-performance-vm 15
- 16. Multi-site 16
- 17. Hybrid Cloud 17
- 20. Improvement Needing ● Instance ○ Resize online ○ Auto-scaling ○ HA Compute ● Volume ○ Discard support ○ Ceph-based volume migration ● Network ○ QoS for egress traffic ● Do upgrade ○ Ceph (Jewel) ○ OpenStack 20
- 21. 21 Skilled people we need ● Linux Internals ● TCP/IP Networking ● Automation ● System Programming ● Performance Oriented
- 22. Image Reference (1) https://images.google.com (2) https://www.openstack.org (3) http://ceph.com (4) https://cumulusnetworks.com (5) https://www.redhat.com (6) https://www.mirantis.com (7) https://www.arista.com (8) http://www.sebastien-han.fr 22