Boyan Krosnov - Building a software-defined cloud - our experience
0 likes1,537 views
The document outlines the construction of a software-defined cloud infrastructure using Storpool, emphasizing its efficiency and deep integrations with platforms like OpenStack and CloudStack. It highlights the scalability and cost-effectiveness of hyper-converged systems for private cloud applications while contrasting various storage solutions and their performance metrics. The paper includes practical examples and benchmarks, showcasing Storpool’s capabilities in enhancing cloud service provider storage systems.
Boyan Krosnov - Building a software-defined cloud - our experience
- 1. Building a software-defined cloud with CloudStack 1
- 2. Boyan Krosnov ● Chief of Product ● Cloud architect ● "External technology" ● Previously service providers, networks, packet processing (SDN) ● Algorithms & data structures 2
- 3. StorPool ● Fast and efficient software-defined storage system ● Used by MSPs, Cloud service providers, Hosting companies and in private clouds ● Started in 2011 to solve storage for cloud service providers ● Clean slate design - scale-out, API-controlled, end-to-end data integrity, CoW 3
- 4. StorPool ● Most deployments are with KVM ○ some Xen, some VMWare, some Hyper-V ○ some bare metal / dedicated server ● Deep integrations into OpenStack, OpenNebula, OnApp ● New integration in CloudStack 4
- 5. Building a software-defined cloud our experience 5
- 6. Typical cloud infrastructure 1. Hyper-converged 2. Stand-alone storage system - rack-level or datacenter-level integration 3. Other
- 7. Modern server: 28-40 cores; 384GB+ RAM 5 - 15 % Hyper-converged: Efficiency StorPool takes 3 cores & 8 GB RAM < 10% of compute node resources 7
- 11. Hyper-converged: example Private cloud for production public-facing application (high-end VDI) ● Independent racks design. Each rack has… ● Top of rack switches, 2x 25GbE to each node ● 38 compute nodes ● Each with 2x 3.84 TB NVMe SSDs ● 88 TB usable space (3-way replication, checksums, CoW, safety)
- 12. Hyper-converged: example Private cloud for production public-facing application (high-end VDI) Strong case for all of ● SDS ● hyper-convergence ● scale-out storage architecture ● low latency and efficient storage software layer
- 13. Contrast requirements Best performance I can get at reasonable cost (private cloud for web/mobile application) vs. Good performance and low $/GB for desired performance level. Market leading, but not too expensive. (service providers) vs. Lowest $/GB (e.g. archival use-case)
- 15. Latency opspersecond best service lowest cost per delivered resource 15
- 16. Latency opspersecond best service lowest cost per delivered resource only pain 16
- 17. Latency opspersecond best service lowest cost per delivered resource only pain 17 benchmarks
- 18. When hyper-converged Hyperconverged: ● Green-field ● KVM virtualization ● Building pods / small availability zones Stand-alone storage system: ● Retrofit ● Other hypervisors: Xen, VMWare, Hyper-V ● Requirements for independence of infrastructure (I don't want an issue in my KVM environment to affect my VMWare environment) ● Requirement for administrative boundary - e.g. additional storage to dedicated server via iSCSI ● Building datacenter-scale infrastructure. Cost of high bandwidth leaf-spine network (3x) justified
- 20. StorPool-CloudStack StorPool for CloudStack/KVM ● Uses StorPool native driver in host OS ○ block devices in /dev/storpool/volname are raw disk images for qemu/KVM ○ data management - e.g. create, delete, clone, resize snapshot - API calls ● Implements a datastore driver for primary storage On other hypervisors use their native storage mechanisms - e.g. VMFS
- 22. Demo 3 nodes, each with 4-core CPU 16GB RAM dual-port 40G NIC 4x 1TB Intel P4500 NVMe SSD
- 23. Demo 50 ㎲ writes! $/IOPS - <$15 000 hardware = 1M IOPS $/GB - > 30TB capacity 3x NVMe approx $1.90/GB > 20 TB StorPool SSD-Hybrid approx $1.00/GB 84%16%
- 25. © 2018 StorPool. All rights reserved. Boyan Krosnov StorPool Storage bk@storpool.com www.storpool.com @storpool Thank you 25