Benchmarking your cloud performance with top 4 global public clouds
0 likes166 views
The document discusses benchmarking cloud performance across the top four global public clouds, emphasizing the importance of application performance and cost efficiency in both public and private cloud scenarios. It provides insights into performance measurement, total cost of ownership (TCO), and practical benchmarks comparing various cloud storage and compute offerings. The conclusions stress the potential for achieving significantly higher application performance at lower costs compared to hyperscalers, highlighting the need for tailored optimization strategies.
![* - ramdisk used to reduce usable RAM to 16 GB
VMs and block storage
Provider Instance name region
monthly cost
(with 12 month
commitment)
vCPUs RAM free -m
AWS Compute optimized: c5.2xlarge us-east-2 $245 8 16GB 15,437
Google Cloud General purpose: n2-8vcpu-16gb us-central1 $197 8 32GB 32,116*
Microsoft Azure
Compute optimized:
Standard_F8s_v2 - 8 vcpus, 16
GiB memory
East US 2 $235 8 16GB 15,962
Digital Ocean CPU Optimized Droplet: 16GB sfo2 $160 8 16GB 16,039
Katapult ROCK-24 London $120 8 24GB 23,458*
Storage volume
Size of volume
[GiB]
IOPS limit Monthly cost
AWS - EBS gp2 1024 3,072.00 $102
Google Cloud - SSD persistent disk 1T 1024 15,000.00 $174
Microsoft Azure - Premium SSD 1T 1024 3,500.00 $123
DigitalOcean - Block Storage 1T 1024 10,000.00 $102
Katapult Shared disk NVMe (StorPool-based) 1024 unlimited $154](https://image.slidesharecdn.com/storpoolpptbenchmarking-your-cloud-performance-with-top-4-global-public-clouds-boyan-krosnov-210123105748/85/Benchmarking-your-cloud-performance-with-top-4-global-public-clouds-15-320.jpg)
![Results - FIO Storage type
FIO rand r/w
QD1 latency
[ms]
FIO QD1
random r/w
IOPS
FIO QD64
random r/w
IOPS
Katapult 1T ($153) StorPool-based 0.10 ms 10,101 IOPS 113,447 IOPS
AWS EBS gp2 1T ($102) 0.36 ms 2,762 IOPS 3,123 IOPS
Google Cloud SSD Persistent Disk 1T
($174)
0.72 ms 1,386 IOPS 15,436 IOPS
Azure Premium SSD 1T ($124) 8.18 ms 122 IOPS 5,100 IOPS
DO Block Storage 1T ($102) 3.34 ms 299 IOPS 1,044 IOPS](https://image.slidesharecdn.com/storpoolpptbenchmarking-your-cloud-performance-with-top-4-global-public-clouds-boyan-krosnov-210123105748/85/Benchmarking-your-cloud-performance-with-top-4-global-public-clouds-17-320.jpg)
Benchmarking your cloud performance with top 4 global public clouds
- 1. Benchmarking your cloud performance with top 4 global public clouds Boyan Krosnov data://disrupted 2020
- 2. ● Chief of Product & co-founder at StorPool ● 20+ years in ISPs, SDN, SDS ● IT Infrastructure with a focus on invention, performance & efficiency About me https://www.linkedin.com/in/krosnov/ bk@storpool.com
- 3. About StorPool ● NVMe software-defined storage for VMs and containers ● Scale-out, HA, API-controlled ● Since 2011, in commercial production use since 2013 ● Based in Sofia, Bulgaria ● Mostly virtual disks for KVM ● … and bare metal Linux hosts ● Also used with VMWare, Hyper-V, XenServer ● Integrations into OpenStack/Cinder, Kubernetes Persistent Volumes, CloudStack, OpenNebula, OnApp 3
- 4. Why performance ● Better application performance -- e.g. time to load a page, time to rebuild, time to execute specific query ● Happier customers in cloud and multi-tenant environments ● ROI, TCO - Lower cost per delivered resource (per VM) through higher density ● Public cloud - win customers over from your competitors ● Private cloud - do more with less; win applications / workloads / teams over from public cloud
- 5. 1. Understanding performance 2. Benchmarks of public clouds 3. How to measure measure and optimize your own cloud 4. What's in a TCO 5. Conclusion Agenda
- 7. Latency ops per second best service lowest cost per delivered resource 7
- 8. Latency ops per second best service lowest cost per delivered resource only pain 8
- 9. Latency ops per second best service lowest cost per delivered resource only pain 9 benchmarks
- 10. Benchmarks
- 11. Real load
- 12. ?
- 13. ?
- 14. 1. Understanding performance 2. Benchmarks of public clouds 3. How to measure and optimize your own cloud 4. What's in a TCO 5. Conclusion Agenda
- 15. * - ramdisk used to reduce usable RAM to 16 GB VMs and block storage Provider Instance name region monthly cost (with 12 month commitment) vCPUs RAM free -m AWS Compute optimized: c5.2xlarge us-east-2 $245 8 16GB 15,437 Google Cloud General purpose: n2-8vcpu-16gb us-central1 $197 8 32GB 32,116* Microsoft Azure Compute optimized: Standard_F8s_v2 - 8 vcpus, 16 GiB memory East US 2 $235 8 16GB 15,962 Digital Ocean CPU Optimized Droplet: 16GB sfo2 $160 8 16GB 16,039 Katapult ROCK-24 London $120 8 24GB 23,458* Storage volume Size of volume [GiB] IOPS limit Monthly cost AWS - EBS gp2 1024 3,072.00 $102 Google Cloud - SSD persistent disk 1T 1024 15,000.00 $174 Microsoft Azure - Premium SSD 1T 1024 3,500.00 $123 DigitalOcean - Block Storage 1T 1024 10,000.00 $102 Katapult Shared disk NVMe (StorPool-based) 1024 unlimited $154
- 16. ● Storage heavy, a little CPU ○ FIO, rsync ● Storage & CPU ○ pgbench, sysbench ● CPU, RAM* ○ coremark ● Network* * - future additions to our suite Tools used
- 17. Results - FIO Storage type FIO rand r/w QD1 latency [ms] FIO QD1 random r/w IOPS FIO QD64 random r/w IOPS Katapult 1T ($153) StorPool-based 0.10 ms 10,101 IOPS 113,447 IOPS AWS EBS gp2 1T ($102) 0.36 ms 2,762 IOPS 3,123 IOPS Google Cloud SSD Persistent Disk 1T ($174) 0.72 ms 1,386 IOPS 15,436 IOPS Azure Premium SSD 1T ($124) 8.18 ms 122 IOPS 5,100 IOPS DO Block Storage 1T ($102) 3.34 ms 299 IOPS 1,044 IOPS
- 18. Results - rsync storage type seconds to re-sync Katapult 1T ($153) StorPool-based 85 AWS EBS gp2 1T ($102) 176 Google Cloud SSD Persistent Disk 1T ($174) 281 Azure Premium SSD 1T ($124) 431 DO Block Storage 1T ($102) 1,303
- 20. Results - pgbench Unfit for databases!
- 22. Results - pgbench 2.3x transactions with same CPU, RAM / same cost
- 23. Results - pgbench 2.5-3x lower latency at fixed load
- 24. 1. Understanding performance 2. Benchmarks of public clouds 3. How to measure and optimize your own cloud 4. What's in a TCO 5. Conclusion Agenda
- 25. ● Design benchmarks which reflect your use-case and application ● Measure what matters. Examples: ○ developer productivity - simple SQL database for up to X users, so no need to pay for complexity of clusters; runs CI/tests in half the time ○ Efficiency - $ per user, $ per features ● If you can't measure what matters directly, find good proxies. Example: ○ "I can't run my entire stack as a benchmark, but I know it consists of a load balancer and a transaction-heavy database, so I'll use a load balancer and a DB benchmark" Benchmarks
- 26. Storage benchmarks Beware: lots of snake oil out there! ● performance numbers from hardware configurations totally unlike what you’d use in production ● synthetic tests with high iodepth - 10 nodes, 10 workloads * iodepth 256 each. (because why not) ● testing with ramdisk backend ● synthetic workloads don't approximate real world
- 27. ● Previous version of our tools and methodology: ○ https://storpool.com/storage-performance-and-resilience- testing ● We'll be releasing updated tools and method with the write-up in the next month ○ coremark, fio, rsync, pgbench, sysbench ● Until then drop us an email at info@storpool.com Benchmarks
- 28. 1. Your existing hardware can give you more a. See Venko's talk on KVM optimization (tomorrow 11am) b. fast networking (OVS-DPDK), fast storage (StorPool) 2. If you are building a new cloud - optimize for your use-case a. per-rack power limit b. per-core performance, per-core memory, per-core storage c. per-core cost Hardware
- 29. Hardware optimization example: Xeon scalable selection
- 30. ● Hardware ● Host OS and hypervisor (KVM) ● Virtual networking, service mesh ● Storage Optimization areas
- 31. 1. Understanding performance 2. Benchmarks of public clouds 3. How to measure and optimize your own cloud 4. What's in a TCO 5. Conclusion Agenda
- 32. ● Define minimum service level ● When comparing options use TCO tool (large spreadsheet) to find lowest cost per delivered unit of infrastructure (fixed-size VM/container with associated storage and networking) ● 100s of parameters ● Usable for both public and private scenarios TCO approach
- 34. 1. Datacenter - power, cooling, max power per rack, remote hands 2. Compute - servers, CPUs, RAM, minimum core performance, cloud orchestration, management cost 3. Storage - storage servers, drives, software, management cost 4. Network - virtual network, CPU/RAM allocation, software, management cost - public/wide area network, IP transit cost What to include
- 35. 1. Understanding performance 2. Benchmarks of public clouds 3. How to measure and optimize your own cloud 4. What's in a TCO 5. Conclusion Agenda
- 36. 1. You can't judge a VM by its vCPUs and vRAM 2. Measure what matters to you 3. If you are a public or private cloud 2x,3x, higher application performance (per $ !) than hyperscalers is within reach. Half price for the same workload! 4. On your next project work with partners who understand performance. You can gain a lot! Conclusions
- 37. Questions?
- 39. Talk to us: The virtual booth at Data://Disrupted info@storpool.com https://storpool.com Thank you!