![Matrix MultiplicationPerformance - 64 CPU coresSpeedup – Fixed matrix size (5184x5184)Implements Cannon’s Algorithm [1]Exchange large messagesMore susceptible to bandwidth than latencyAt least 14% reduction in speedup between bare-metal and 1-VM per node[1] S. Johnsson, T. Harris, and K. Mathur, “Matrix multiplication on the connection machine,” In Proceedings of the 1989 ACM/IEEE Conference on Supercomputing (Reno, Nevada, United States, November 12 - 17, 1989). Supercomputing '89. ACM, New York, NY, 326-332. DOI= http://doi.acm.org/10.1145/76263.76298](https://image.slidesharecdn.com/cloudcomp09presentation-091019212235-phpapp02/85/High-Performance-Parallel-Computing-with-Clouds-and-Cloud-Technologies-14-320.jpg)
![Conclusions and Future WorksCloud technologies works for most pleasingly parallel applicationsRuntimes such as MapReduce++ extends MapReduce to iterative MapReduce domainMPI applications experience moderate to high performance degradation (10% ~ 40%) in private cloudDr. Edward walker noticed (40% ~ 1000%) performance degradations in commercial clouds [1]Applications sensitive to latencies experience higher overheadsBandwidth does not seem to be an issue in private cloudsMore VMs per node => Higher overheadsIn-node communication support is crucialApplications such as MapReduce may perform well on VMs ?[1] Walker, E.: benchmarking Amazon EC2 for high-performance scientific computing, http://www.usenix.org/publications/login/2008-10/openpdfs/walker.pdf](https://image.slidesharecdn.com/cloudcomp09presentation-091019212235-phpapp02/85/High-Performance-Parallel-Computing-with-Clouds-and-Cloud-Technologies-19-320.jpg)
High Performance Parallel Computing with Clouds and Cloud Technologies
- 1. High Performance Parallel Computing with Clouds and Cloud TechnologiesCloudComp 09Munich, Germany11,2Jaliya Ekanayake, Geoffrey Fox{jekanaya,gcf}@indiana.eduSchool of Informatics and ComputingPervasive Technology InstituteIndiana University Bloomington12
- 2. Acknowledgements to:Joe Rinkovsky and Jenett Tillotson at IU UITSSALSA Team - Pervasive Technology Institution, Indiana UniversityScott BeasonXiaohong QiuThilina Gunarathne
- 3. Computing in CloudsEucalyptus(Open source)Commercial CloudsPrivate CloudsAmazon EC23TeraNimbusGoGridXenSome Benefits:On demand allocation of resources (pay per use)Customizable Virtual Machine (VM)s Any software configurationRoot/administrative privilegesProvisioning happens in minutes Compared to hours in traditional job queuesBetter resource utilizationNo need to allocated a whole 24 core machine to perform a single threaded R analysisAccessibility to a computation power is no longer a barrier.
- 4. Cloud Technologies/Parallel RuntimesCloud technologiesE.g. Apache Hadoop (MapReduce)Microsoft DryadLINQ MapReduce++ (earlier known as CGL-MapReduce)Moving computation to dataDistributed file systems (HDFS, GFS)Better quality of service (QoS) supportSimple communication topologiesMost HPC applications use MPIVariety of communication topologiesTypically use fast (or dedicated) network settings
- 5. Applications & Different Interconnection PatternsInputmapiterationsInputInputmapmapOutputPijreducereduceDomain of MapReduce and Iterative ExtensionsMPI
- 6. MapReduce++ (earlier known as CGL-MapReduce)In memory MapReduceStreaming based communicationAvoids file based communication mechanismsCacheable map/reduce tasksStatic data remains in memoryCombine phase to combine reductionsExtends the MapReduce programming model to iterative MapReduce applications
- 7. What I will present nextOur experience in applying cloud technologies to:EST (Expressed Sequence Tag) sequence assembly program -CAP3.HEP Processing large columns of physics data using ROOTK-means ClusteringMatrix MultiplicationPerformance analysis of MPI applications using a private cloud environment
- 8. Cluster ConfigurationsDryadLINQHadoop / MPI/ Eucalyptus
- 9. Pleasingly Parallel ApplicationsHigh Energy PhysicsCAP3Performance of CAP3Performance of HEP
- 10. Iterative ComputationsK-meansMatrix MultiplicationPerformance of K-Means Parallel Overhead Matrix Multiplication
- 11. Performance analysis of MPI applications using a private cloud environmentEucalyptus and Xen based private cloud infrastructure Eucalyptus version 1.4 and Xen version 3.0.3Deployed on 16 nodes each with 2 Quad Core Intel Xeon processors and 32 GB of memoryAll nodes are connected via a 1 giga-bit connectionsBare-metal and VMs use exactly the same software configurationsRed Hat Enterprise Linux Server release 5.2 (Tikanga) operating system. OpenMPI version 1.3.2 with gcc version 4.1.2.
- 12. Different Hardware/VM configurationsInvariant used in selecting the number of MPI processesNumber of MPI processes = Number of CPU cores used
- 14. Matrix MultiplicationPerformance - 64 CPU coresSpeedup – Fixed matrix size (5184x5184)Implements Cannon’s Algorithm [1]Exchange large messagesMore susceptible to bandwidth than latencyAt least 14% reduction in speedup between bare-metal and 1-VM per node[1] S. Johnsson, T. Harris, and K. Mathur, “Matrix multiplication on the connection machine,” In Proceedings of the 1989 ACM/IEEE Conference on Supercomputing (Reno, Nevada, United States, November 12 - 17, 1989). Supercomputing '89. ACM, New York, NY, 326-332. DOI= http://doi.acm.org/10.1145/76263.76298
- 15. Kmeans ClusteringPerformance – 128 CPU coresOverhead = (P * T(P) –T(1))/T(1)Up to 40 million 3D data pointsAmount of communication depends only on the number of cluster centersAmount of communication << Computation and the amount of data processedAt the highest granularity VMs show at least ~33% of total overheadExtremely large overheads for smaller grain sizes
- 16. Concurrent Wave Equation Solver Overhead = (P * T(P) –T(1))/T(1)Performance - 64 CPU coresClear difference in performance and overheads between VMs and bare-metalVery small messages (the message size in each MPI_Sendrecv() call is only 8 bytes)More susceptible to latencyAt 40560 data points, at least ~37% of total overhead in VMs
- 17. Higher latencies -11-VM per node 8 MPI processes inside the VM8-VMs per node 1 MPI process inside each VMdomUs (VMs that run on top of Xenpara-virtualization) are not capable of performing I/O operationsdom0 (privileged OS) schedules and execute I/O operations on behalf of domUsMore VMs per node => more scheduling => higher latencies
- 18. Higher latencies -2Kmeans ClusteringLack of support for in-node communication => “Sequentializing” parallel communicationBetter support for in-node communication in OpenMPIsm BTL (shared memory byte transfer layer)Both OpenMPI and LAM-MPI perform equally well in 8-VMs per node configuration
- 19. Conclusions and Future WorksCloud technologies works for most pleasingly parallel applicationsRuntimes such as MapReduce++ extends MapReduce to iterative MapReduce domainMPI applications experience moderate to high performance degradation (10% ~ 40%) in private cloudDr. Edward walker noticed (40% ~ 1000%) performance degradations in commercial clouds [1]Applications sensitive to latencies experience higher overheadsBandwidth does not seem to be an issue in private cloudsMore VMs per node => Higher overheadsIn-node communication support is crucialApplications such as MapReduce may perform well on VMs ?[1] Walker, E.: benchmarking Amazon EC2 for high-performance scientific computing, http://www.usenix.org/publications/login/2008-10/openpdfs/walker.pdf
- 20. Questions?
- 21. Thank You!