Pushing the Limits: CloudStack at 25K Hosts
Download as PPTX, PDF0 likes207 views
Boris Stoyanov took a look at a load testing exercise conducted in the lab. Discovered how CloudStack performs with 25,000 hosts as we explore response times, performance challenges, and the code improvements needed to scale effectively -- The CloudStack European User Group 2025 took place on May 8th in Vienna, Austria. The event once again brought together open-source cloud professionals, contributors, developers, and users for a day of deep technical insights, knowledge sharing, and community connection.
Pushing the Limits: CloudStack at 25K Hosts
- 1. Pushing the Limits: Cloudstack at 25k hosts Boris Stoyanov, Shapeblue
- 2. Objective • Identify CloudStack’s performance boundaries under stress • Pinpoint bottlenecks and underperforming components • Lay the groundwork for a repeatable performance benchmark in future releases
- 3. Scope and specifics • Listing operations • No active workloads • Simulated resources • Virtual Environment
- 4. Test Environment Setup • Virtual Environment – dedicated pool • Simulated hosts, but not cloudstack- simulator • Single Management Server • 25k Hosts • 25k Instances • 50k Volumes • 25k Neworks • 25k Domains and Accounts
- 5. Management server • Oracle Linux 8 • mysql 8.0.36 • ACS 4.20 vanilla • 2xCPUs 1.5GHz each • 5GB RAM
- 6. Tools • Hosts • KVM based • Maintaining connection with management as regular cloudstack-agent • csbench – developed for Cloudstack • Provisioning resources • Benchmark testing
- 7. Load test • csbench –benchmark • Lists(accounts, domains, hosts, networks, virtualMachines and volumes) • 100x • Observations: • Response time • Avarages • Ratio between 5 and 95 percentile
- 8. Results listAccounts listDomains listHosts listNetworks listVirtualMachines listVolumes 0 20 40 60 80 100 120 22.94 2.4 5.28 24.11 9.75 4.81 98.48 7.5 12.9 96.92 17.28 14.4 49.63 4.6 7.6 43.57 11.79 9.24 Response time in seconds Avg Worst Best
- 9. 95th to 5th ratio listAccounts listDomains listHosts listNetworks listVirtualMachines listVolumes 0 20 40 60 80 100 120 3.68 1.6 1.3 2.9 2.8 1.8 22.94 2.4 5.28 24.11 9.75 4.81 98.48 7.5 12.9 96.92 17.28 14.4 Ratio between best and worst Worst Best Ratio
- 10. Takeaways • Expecting cracks, but we found headspace • Some APIs perform inconsistently • Unclear how it behaves in an active env
- 11. Next Steps • Emulate workload • Emulate continuous workload on the environment under test • Benchmark more List APIs • Benchmark workloads – CRUD APIs • Make Benchmarking part of regular release testing
- 12. Credits • csbench - https://github.com/apache/cloudst ack-csbench • Contributors: Rohit Yadav, Vishesh Jindal, Harikrishna Patnala • KVM based simulator – Abhishek Kumar • Testing – Vladimir Petrov
- 13. Q&A