StorPool Storage Оverview and Integration with CloudStack
0 likes932 views
The document provides a comprehensive overview of Storpool's storage solutions and their integration with Apache CloudStack, highlighting key benefits and use cases for modern cloud infrastructures. It showcases various case studies, including successful implementation examples, and emphasizes Storpool's recognition as the 'Software Defined Storage Vendor of the Year' in 2020. The document also details performance comparisons with major competitors, underscoring Storpool's efficiency and cost-effectiveness.
StorPool Storage Оverview and Integration with CloudStack
- 1. 1 StorPool Storage Overview and Integration with Apache CloudStack
- 2. 2 Alex Ivanov Product Lead StorPool Company Intro Products and Services Use Cases and Case Studies Venko Moyankov Solutions Lead StorPool Storage Intro CloudStack Integration Overview Key Technical Benefits Slavka Peleva CloudStack Tech Lead Quick Intro to the Speakers StorPool Integration with Apache CloudStack Plug-in Features and Benefits
- 3. 3 of Flash Storage Deployments Primary Workload Experts Years in Business Proven Track Record Countries of Deployment Global Software Company Team Members Stable and Profitable Partner Accelerating Human Activity with Faster Storage 100s 11 30+ 50+
- 4. 4 Software Defined Storage (SDS) Vendor of the Year at 2020 Storage Awards “The heated battle for the SDS vendor of the year award between the tech giant IBM and the storage market disruptor, StorPool, has ended with a well-deserved victory for the innovative scale-up company. IBM took the place of runner-up for the category and this showed us that modern business has become more accepting in the adoption of next-gen technologies.” Source: https://containerjournal.com/news/news-releases/storpool-named-software-defined-storage-sds-vendor-of-the-year-at-2020-storage-awards/
- 5. 5 Source: https://storpool.com/news/sdc-awards-2021-storpool-winner First place for the project “Dustin Group replaces VMware and Hyper-V with a New-Age IT Stack powered by StorPool” in the category “Storage Transformation Project of the Year”.
- 6. 6 StorPool Storage Products and Services Enablers of our Complete Storage Solution for CloudStack Environments StorPool Storage Converts customers’ servers into high-performance, linearly scalable primary storage systems. StorPool Analytics Collects millions of metrics daily for visualizations and deep insights into each StorPool storage system. StorPool Alerts & Reporting Uses StorPool Analytics’ metrics for per-node, per-host, per-volume, and whole-system alerts and reports. StorPool VolumeCare Periodic snapshotting and replication of incremental changes to local or remote lower storage tiers. StorPool Managed Service Our experts leverage the other StorPool products to proactively open tickets and deal with potential issues.
- 7. 7 We analyse and tune your StorPool storage system so that it runs optimally and reliably for your end-users’ workloads Design Deploy Fine-tune Monitor Maintain StorPool Helps Get Your Data In Order We help you select the ideal architecture for your cloud at the physical, network, and logical levels, using only standard hardware. We install StorPool Storage in your servers and connect your StorPool storage system to one or your CloudStack cluster. We monitor thousands of metrics per second to proactively open support tickets and deal with any issues that arise We ensure that your storage system always runs optimally by installing non-disruptive updates and helping with expansion when needed. Deploy Reliable, Agile, Speedy, and Cost-effective Clouds
- 8. 8 Typical Customers and Other Use Cases Typical Customers: ● Follow Modern Cloud Infrastructure Practices API-driven, Automated, DevOps, SREs, Cloud Management Platforms ● Offer Public Cloud Services IaaS, PaaS, hosted SaaS, virtual private servers, ecommerce, shared hosting, desktop as a service, BC/DR ● Offer Private Cloud Services Tailored solutions for SMBs and Enterprises - virtual private clouds, managed/hosted private clouds Primary Use Cases: ● Media: SATA/SAS/NVMe SSD-based storage served to workloads in bare-metal hosts, virtual machines, and containers ● Workloads: Databases, VM disks, VDI, large apps or SaaS, video game hosting, persistent K8s volumes, etc. ● Scale: 100TB to 10s of PB stored across multiple data centers ● Example Projects: Convert Legacy IT Stack to SDDC Design; Consolidate Мultiple IT systems to One Stack
- 9. 9 Case Study - Amito The Project: ● "Тhe most difficult part of the infrastructure stack for high availability virtualized servers is storage. Amito was using quite a few different solutions, from big-name vendors to in house developed HA SANs based on ZFS. None of these solutions brought the expected results." The goals: ● Implement a reliable, high-performance shared storage in multiple data centers - support a broad range of apps. The solution: ● “... our shortlist didn’t contain any of the vendors with older legacy solutions – the approach to storage has moved on, and they have not all kept up.”, Ed Butler, CEO, Amito. ● KVM, CloudStack, StorPool, supporting a large number of Windows and Linux virtual machines. Company Profile: ● UK-based Cloud and Managed Service Provider ● Founded in 2000 ● Presence in 28 data centers worldwide Case study: https://storpool.com/amito-cas e-study-storpool
- 10. 10 Use Cases / Customers / Workloads Summary Top Use Cases Selected Customers Workloads of Virtual Disks for VMs, Databases, VDI, K8s Large SaaS / Web Apps Test/Dev and Production Upgrade legacy IT systems to SDDC design
- 11. 11 Introduction to StorPool Storage
- 12. 12 StorPool Storage Technical Overview Software deployed on customer’s servers running standard Linux - CentOS, RHEL, Ubuntu, Debian. Converts off-the-shelf x86-64 servers into a high-performance primary storage system Distributed, shared-nothing architecture linearly scales to petabytes without performance bottlenecks. Decoupled from compute or hyper-converged Thin-provisioned volumes for virtual machines, containers, and bare-metal hosts. Fastest, Most Reliable Primary Storage Software on the Market
- 15. 15 ● Thin provisioning ● Efficient snapshots ● Clones ● Per volume QoS ● Storage tiering ● One volume per virtual disk ● RESTful JSON API Data Management
- 16. 16 Performance in Large-scale Environments StorPool is so efficient that 84% of end-to-end latency is caused by the laws of physics Measured with All-NVMe cluster using Micron 9300 Pro, 25Gbit/s Ethernet. (lower is better) Test using FIO on a raw block device, random read, 4k block size, QD=1
- 17. 17 Linear Scalability The Performance of StorPool Systems Scales with Usable Capacity
- 18. 18 ● Software upgrades ● Hardware replacement and upgrades ● Scale-out the storage ● Hardware refresh cycle ● Physical relocation Hardware Lifecycle Management
- 19. 19 The Fastest Cloud is Powered by StorPool https://storpool.com/wp-content/uploads/2022/01/StorPool-Public-Cloud-Performance-Measurement-Report_2022.pdf Unfit for Databases PostgreSQL Benchmark
- 20. 20 The Fastest Cloud is Powered by StorPool https://storpool.com/wp-content/uploads/2022/01/StorPool-Public-Cloud-Performance-Measurement-Report_2022.pdf PostgreSQL Benchmark 2-3x lower latency at fixed load
- 21. 21 The Fastest Cloud is Powered by StorPool https://storpool.com/wp-content/uploads/2022/01/StorPool-Public-Cloud-Performance-Measurement-Report_2022.pdf 2.6-4x transactions with same CPU, RAM PostgreSQL Benchmark
- 22. 22 StorPool Maintains a Complete Integration with Apache CloudStack CloudStack Integration
- 23. 23 StorPool Integration with Apache CloudStack
- 24. 24 CloudStack versions ● From CloudStack v4.11.3 to the main branch of CloudStack ● Included in the CloudStack v4.17.0 packages Separate jar ● On all versions on the managements’ host /usr/share/cloudstack-management/lib on the agents’ hosts /usr/share/cloudstack-agent/lib Prerequisites and requirements
- 25. 25 What’s the Impact for Service Providers? Direct Technical Benefits of the StorPool Plug-in for CloudStack Offloading Secondary Storage Creating snapshots and templates based on items in the CloudStack marketplace is done directly in StorPool, accelerating all operations. Offloading Hypervisor Network Since snapshots and templates are created in StorPool, the private and public compute node networks are used only for user traffic. Storage Usage Efficiency StorPool creates a volume for each virtual disk by cloning pre-existing snapshots. VM data is kept only on the primary storage and provisioned stored ratios are optimised – 2:1 to 6:1.
- 26. 26 StorPool plug-in features Simplified Cloud Architecture Both primary and secondary storage can be provided with a single StorPool storage system Reduce the usage of Secondary storage ● No need to download VM templates for VM deployments ● No snapshot transfers ● No volume migration
- 27. 27 The ideal foundation for large-scale clouds running diverse, mission-critical workloads Thanks Slavka, Venko, Alex
- 28. 28 Backup slides
- 29. 29
- 30. 30 Case Study #2 - Katapult The Project: ● Virtual IaaS platform built for extreme performance and simplicity ● Build the fastest public cloud using standard hardware The goals: ● Modern linearly scalable storage solution ● Unmatched performance ● High availability ● A high level of flexibility ● Guaranteed data integrity The solution: ● Tailored IaaS platform, focused on extreme performance ● Automation and API-first approach ● Extremely high level of data protection offered by StorPool’s end-to-end data integrity and 3x copies of data Company Profile: ● Katapult by Krystal Hosting ● One of the largest independent UK web hosting companies ● Founded in 2002 and steadily grown over the last 20 years ● Comprehensive portfolio of hosting, cloud and VPS services Case study: https://storpool.com/krystal-kat apult-case-study
- 31. 31 Comparing Cloud Performance and Cost How to Make Sure you are Comparing Apples to Apples Use Equivalent Technical Setup Work off the Same Cost Basis Run Tests Similar to Production Workloads Example from our tests: Storage – 1,024 GiB Memory – 16GB DDR4 vCPUs – 8 vCPUs If VMs have more memory, pre-allocate excess memory as a file to even the playing field. Experiment with vCPUs and memory to ensure they are not the bottleneck. Make sure to account for the total monthly cost of the VM. Example from our tests: Katapult – $274 / m AWS with io2 – $1,348 / m AWS with gp3 – $402 / m Azure – $358 / m GCP – $299 / m Remember to consider ingress/egress charges as they lead to cloud lock-in Pgbench – PostgreSQL benchmark, closest to application performance Sysbench – MySQL benchmark, control for Pgbench results fio – synthetic benchmarks that test the limits of the VM (latency, random read/write, sequential read/write) rsync – simulates rapid deployment and backups
- 32. 32 Cost/Performance Comparison $ / TPS / Month Small OLTP Database Large OLTP Database Sub-2ms Sub-3ms Sub-2ms Sub-3ms Sub-4ms Katapult $0.028 / TPS $0.024 / TPS $0.031 / TPS $0.029 / TPS $0.028 / TPS GCP SSD Persistent $0.687 / TPS $0.470 / TPS $0.620 / TPS $0.404 / TPS $0.413 / TPS Amazon EBS io2 Over Threshold $0.250 / TPS $0.880 / TPS $0.457 / TPS $0.314 / TPS Amazon EBS gp3 Over Threshold $0.151 / TPS Over Threshold $0.390 / TPS $0.083 / TPS Azure Premium SSD Over Threshold Over Threshold Over Threshold Over Threshold Over Threshold Katapult Delivers up to 20 Times Better Cost per TPS Running OLTP Databases in Katapult vs. Hyperscale Public Clouds
- 33. 33 Deployment Option 1: Decoupled Compute Storage Storage Network 10/25/50/100 GbE
- 34. 34 Deployment Option 2: Hyper-Converged (HCI) Compute Storage Network 10/25/50/100 GbE Storage