Hp cloud performance_benchmark
1 like537 views
HP Cloud Services conducted performance testing on various VM configurations provided by OpenStack. Benchmark tests were run including byte-unixbench, mbw, iozone, iperf, pgbench and Hadoop wordcount. The results showed the larger VM configurations generally had better performance, but some defects were discovered in 7 out of 20 test VMs, indicating the defect rate was too high for production use. While defects were not directly related to OpenStack, the conclusions were that OpenStack still lacks functionality for production and building a full IaaS service is more complex than the software alone.
Hp cloud performance_benchmark
- 1. HP Cloud Services Performance Testing Qingye Jiang (John) Email: qjiang@ieee.org Weibo: @qyjohn_
- 2. Introduction • Virtual Machines • az-1.region-a.geo-1 • web-created • Ubuntu 11.04 64 bit • 3+ VM’s / model • total 20 VM’s • Benchmark Suite • byte-unixbench • mbw • iozone • iperf • pgbench • Hadoop wordcount Mediu XXLar XSmall Small Large XLarge m ge vCPU 1 2 2 4 4 8 • Data Filtering MEM (GB) 1 2 4 8 16 32 • best VM / model DISK (GB) 30 60 120 240 480 960 • average by 10 Price ($/hr) 0.04 0.08 0.16 0.32 0.64 1.28
- 3. byte-unixbench 4500 Si ngl e Thr ead 4000 M t i Thr ead ul 3500 3000 2500 2000 1500 1000 500 0 XSm l al Sm l al M um edi Lar ge XLar ge XXLar ge • byte-unixbench index measures overall system performance • in multi-thread testing, n-Thread = n-vCPU • system with the same amount of vCPU exhibits similar performance • memory size does not have much impact on performance • 2 x vCPU => 1.5 x performance
- 4. mbw 12000 10000 M CPY EM 8000 DM UP MCBLOCK 6000 4000 2000 0 XSm l al Sm l al M um edi Lar ge XLar ge XXLar ge • mbw 128 • MB/s • different systems exhibit similar memory performance
- 5. iozone – os disk 7000000 w i te r 6000000 r ew i t e r 5000000 r andom w i t e r r ead 4000000 r er ead r andom r ead 3000000 2000000 1000000 0 XSm l al Sm l al M um edi Lar ge XLar ge XXLar ge • iozone -Mcew -i0 -i1 -i2 -s4g -r256k -f /io.tmp • KB/s • difference systems exhibit similar write performance • L / XL / XXL systems exhibit much better read performance • cgroup blkio throttling? QEMU blk throttle? Different disk types?
- 6. iozone – data disk 6000000 w i te r 5000000 r ew i t e r r andom w i t e r 4000000 r ead r er ead 3000000 r andom r ead 2000000 1000000 0 XSm l al Sm l al M um edi Lar ge XLar ge XXLar ge • iozone -Mcew -i0 -i1 -i2 -s4g -r256k -f /mnt/io.tmp • KB/s • difference systems exhibit similar write performance • XL / XXL systems exhibit much better read performance • cgroup blkio throttling? QEMU blk throttle? Different disk types?
- 7. iperf XXLarg XSmall Small Medium Large XLarge e XSmall 25 25 25 25 25 25 Small 25 50 50 50 50 50 Medium 25 50 100 100 100 100 Large 25 50 100 200 200 200 XLarge 25 50 100 200 400 400 XXLarge 25 50 100 200 400 650 • (x, y) represents the bandwidth between two systems • Mbps • bandwidth limited by the system with lower configuration • cisco quantum plugin?
- 8. hadoop wordcount single no de 900 800 2GB 700 600 500 400 300 200 100 0 XSm l al Sm l al M um edi Lar ge XLar ge XXLar ge • hadoop wordcount application provided in official distribution • test directory with 3 files, total file size is 2 GB. • test result shows the time needed to finish the calculation (s)
- 9. hadoop wordcount multiple nod es 1000 900 800 700 600 500 400 300 200 100 0 1 x 2 x 3 x 4 x Sm l al XXLar ge Xsm l al XSm l al XSm l al XSm l al • dfs.replication = nNodes • test directory with 3 files, total file size is 2 GB. • test result shows the time needed to finish the calculation (s)
- 10. pgbench 1800 1600 Si ngl e Thr ead M t i Thr ead ul 1400 1200 1000 800 600 400 200 0 XSm l al Sm l al M um edi Lar ge XLar ge XXLar ge • postgresql-9.1, postgresql-contrib-9.1 • pgbench -i -s 16 pgbench • pgbench -t 2000 -c 16 –j n -U postgres pgbench • in multi-thread testing, n-Thread = n-vCPU
- 11. defects – pgbench single thr ead 1200 1000 800 N m or al D ect ef 600 400 200 0 XSm l al Sm l al M um edi Lar ge XLar ge XXLar ge • defects were observed in all VM models • test results were smooth on the same VM instance • the following test results were not affected on defected instances • mbw • iperf • byte-unixbench
- 12. defects – iozone write result s 300000 N m or al 250000 D ect ef 200000 150000 100000 50000 0 XSm l al Sm l al M um edi Lar ge XLar ge XXLar ge • test performed on OS disks only • write performance seems to be the major problem
- 13. defects – iozone read result s 7000000 N m or al 6000000 D ect ef 5000000 4000000 3000000 2000000 1000000 0 XSm l al Sm l al M um edi Lar ge XLar ge XXLar ge • test performed on OS disks only • read performance is similar for all instances in both cases
- 14. defect rate 7 = % 35 20 • 7 defected instances were found out of 20 total instances • defect rate too high for deploying production systems • need extra caution when VM’s are auto-generated by API’s
- 15. conclusion 先以欲勾牵，后令入佛智。 鸠摩罗什大师译 《维摩诘所说经 . 佛道品第八 》 • HP defects were not directly related to OpenStack • OpenStack still lacks key functionalities for production deployment • building IaaS service is more complicated than installing OpenStack • open source IaaS software => IaaS support and service => $$$
- 16. Thank You! Qingye Jiang (John) Email: qjiang@ieee.org Weibo: @qyjohn_ http://www.qyjohn.net/