Mirantis, Openstack, Ubuntu, and it's Performance on Commodity Hardware
0 likes800 views
The document outlines a testbed configuration for a 13-node OpenStack cluster using Mirantis Fuel 6.0 and various hardware specifications. It details performance testing for network and disk operations using iperf and bonnie++, revealing performance ranges and issues, particularly under different MTU settings and configurations. The conclusion notes that while network performance was solid, it could be improved with better hardware and a more optimized Ceph setup.
Mirantis, Openstack, Ubuntu, and it's Performance on Commodity Hardware
- 1. M2 Reference Architecture Performance (Mirantis, Openstack, and Commodity Hardware) Ubuntu 12.04 Fuel 6.0 Openstack Juno Myricom 10G Ethernet Ceph RBD Glance/Cinder 8 Disk Local Raid5 Ephemeral/Root
- 2. Ryan Aydelott - CELS Introduction The purpose of this exercise is to create a small testbed using the Mirantis “Fuel” deployment tool (version 6.0) to build a small 13 node cluster. This deployment consisted of: 3 Controller nodes, 10 compute nodes, and 13 Ceph nodes Each node class (controller/compute) was configured identically with 2TB of local disk for ephemeral storage and 1TB of local disk for Ceph per node, the remaining being used for local OS storage. The array holding this data consisted of a single 8 disk Raid5 array made up of 500GB drives. Additionally a small Raid1 array consisting of two 50GB SSD drives was used for Ceph journal storage. In testing, RBD, ephemeral volume, and inter-VM network performance will be measured. 2
- 3. Ryan Aydelott - CELS Testbed Hardware ‣ IBM System x3650 M3 ‣ 12 Cores @2.67Ghz ‣ x12 Dimms for 48GB @1.33Ghz ‣ ServeRAID M5015 x2 (x2 SSD for Ceph Journal, x8 Magnetic for Instance/ Ceph) ‣ x1 1G ethernet for provisioning, x1 10G ethernet for storage/vm tenant networking (MYRICOM Inc. Myri-10G Dual-Protocol NIC (rev 01)) ‣ Juniper ex4500-40f 3
- 4. Ryan Aydelott - CELS Testbed Software ‣ Deployment Node: Fuel 6.0 ‣ Controller Nodes: Ubuntu 12.04 ‣ Compute Nodes: Ubuntu 12.04 ‣ Openstack Release Juno ‣ Ceph version 0.80.7 (Firefly) with replica/3 4
- 5. Ryan Aydelott - CELS Testbed Configuration ‣ A flavor on Openstack was created that consumed an entire HV ‣ 10 total instances were created, each on it’s own dedicated hypervisor ‣ The guest OS was Ubuntu 14.04 for each instance ‣ iperf version 2.0.5 (08 Jul 2010) pthreads (networking) ‣ bonnie++ v1.97 (disk) ‣ Ephemeral storage ran on local hardware raid5 ‣ Ceph OSD ran on top of local hardware raid5 (not ideal, but simplest deployment with Fuel given hardware contraints) 5
- 6. Ryan Aydelott - CELS Local Network Performance ‣ iperf was utilized to test intervm network performance ‣ By default the HV and all backend network devices had an MTU set at 9000, the MTU on the default ubuntu 14.04 cloud image is set to 1500, we ran tests using both 1500 byte and 9000 byte MTU’s on the the guest. ‣ 1500 Byte MTU: 5Gigabit per instance/HV (25% of theoretical max) ‣ 9000 Byte MTU: 16Gigabit per instance/HV (80% of theoretical max) ‣ Latency ~.5ms average guest<—>guest ‣ Traffic was appropriately divided by node count, whereas one node would use 80% of the theoretical max, 10 nodes would each use 8% of the theoretical max, 5 nodes @ 16%, etc. 6
- 7. Ryan Aydelott - CELS Local Disk (ephemeral) Performance ‣ bonnie++ 1.97 was used to benchmark local disk Version 1.97 ------Sequential Output------ --Sequential Input- --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP ryantest-640 80000M 550 99 282290 80 134720 21 1272 98 152762 43 189.3 11 Latency 16469us 1550ms 2709ms 36363us 1195ms 1820ms Version 1.97 ------Sequential Create------ --------Random Create-------- ryantest-64011122-4 -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 13526 64 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ Latency 166ms 526us 461us 58307us 45us 132us 7
- 8. Ryan Aydelott - CELS Ceph RBD (Cinder) Performance ‣ bonnie++ 1.97 was used to benchmark Ceph RBD Version 1.97 ------Sequential Output------ --Sequential Input- --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP ryantest-5b7 80000M 593 99 95170 27 45953 10 956 75 124535 19 4283 125 Latency 15259us 6451ms 1568ms 820ms 999ms 31255us Version 1.97 ------Sequential Create------ --------Random Create-------- ryantest-5b7edbb6-2 -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 10107 80 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ Latency 110ms 597us 468us 11835us 581us 105us 8
- 9. Ryan Aydelott - CELS Conclusion While not amazing, a solid network performance using nova-network was achieved. Running under Neutron/OVS with VLAN segmentation was less then ideal and only achieved 10% of theoretical network performance under the best of circumstances. Furthering this, Neutron OVS also had varying latency from sub millisecond to upwards of 50ms for interVM latency. Local disk was as expected ~130-280mb/s with Ceph RBD fluttering ~50-125mb/s depending on the type of workload. Disk performance could certainly be improved with better local disks and a more optimal Ceph configuration, however as a generic deployment this configuration would work for most basic use cases. 9