An adaptive distributed simulator for cloud andmap reduce algorithms and architectures
2 likes1,359 views
The document presents an adaptive distributed simulator for cloud and MapReduce algorithms, introduced at the IEEE/ACM UCC 2014 conference. It highlights the need for scalable and fault-tolerant simulations leveraging cloud environments, specifically through an enhanced version of the CloudSim framework. Key contributions include a distributed architecture, dynamic scaling algorithms, and performance evaluations comparing Hazelcast and Infinispan implementations.
An adaptive distributed simulator for cloud andmap reduce algorithms and architectures
- 1. AAnn AAddaappttiivvee DDiissttrriibbuutteedd SSiimmuullaattoorr ffoorr CClloouudd aanndd MMaappRReedduuccee AAllggoorriitthhmmss aanndd AArrcchhiitteeccttuurreess IEEE/ACM 7th International Conference on Utility and Cloud Computing – UCC 2014. Dec 8th – 11th, 2014. Pradeeban Kathiravelu Luis Veiga INESC-ID Lisboa Instituto Superior Técnico, Universidade de Lisboa PPoowweerrppooiinntt TTeemmppllaatteess 1
- 2. Agenda •Introduction •Background •Solution Architecture •Implementation •Evaluation •Conclusion Powerpoint Templates 2
- 3. Introduction •Computing systems becoming increasingly larger. •Simulations empower researches. •Cloud simulators are mostly sequential and executed from a single computer. –CloudSim (Calheiros et al. 2009; Buyya et al. 2009; Calheiros et al. 2011) –SimGrid (Casanova 2001; Legrand et al. 2003; Casanova et al. 2008) –GreenCloud (Kliazovich et al. 2012) Powerpoint Templates 3
- 4. Motivation •Large and complex simulations. •Distributed Execution Frameworks. – Illusion of a single large system. •Clusters in the research labs. Powerpoint Templates 4 •What if..?
- 5. Goals •An adaptive distributed cloud and MapReduce simulator. •Extending CloudSim Cloud Simulator – Leveraging in-memory data grids. • Hazelcast (Johns 2013) • Infinispan (Marchioni 2012) • ... Powerpoint Templates 5
- 6. Contributions •An adaptive distributed architecture – for cloud and MapReduce simulations. •A generic adaptive scaling algorithm. •A scalable middleware platform – elastic – multi-tenanted •Evaluation of MapReduce implementations. –Hazelcast vs Infinispan. Powerpoint Templates 6
- 7. Major Features of the Work •Simulations → Actual Technology. •Loosely coupled. •Fault-Tolerant. •Internal cycle-sharing. •Deployable over real clouds. Powerpoint Templates 7
- 9. Design and Deployment Storage, Execution, and Data Locality • Simulator–Initiator based Approach • Simulator–SimulatorSub based Approach •Multiple Simulator Instances Approach Powerpoint Templates 9
- 10. Cloud2Sim Execution Flow Powerpoint Templates 10
- 11. 1. Objects Initialization & Scheduling Powerpoint Templates 11
- 12. 2. Final Execution Powerpoint Templates 12
- 13. Cloud2Sim Execution Flow Powerpoint Templates 13
- 14. Powerpoint Templates 14 Cloud2Sim Software Architecture
- 15. Algorithms: Dynamic Scaling and Elasticity Powerpoint Templates 15
- 16. Algorithms: Dynamic Scaling and Elasticity •Auto Scaling •Adaptive Scaling Powerpoint Templates 16
- 17. Auto Scaling Powerpoint Templates 17
- 18. Adaptive Scaling Powerpoint Templates 18
- 20. Subscribing for Scaling Powerpoint Templates 20
- 21. High Load Powerpoint Templates 21
- 22. Updating the flag Powerpoint Templates 22
- 23. Open Access Powerpoint Templates 23
- 24. Scaling Out Powerpoint Templates 24
- 25. Spawning an Initiator Instance Powerpoint Templates 25
- 26. Waiting Period.. Powerpoint Templates 26
- 27. Waiting Period.. Powerpoint Templates 27
- 28. Monitor for Scale Ins Too.. Powerpoint Templates 28
- 29. After some time.. Powerpoint Templates 29
- 30. Scale Out Again.. Powerpoint Templates 30
- 31. One more Initiator.. Powerpoint Templates 31
- 32. After more scalings.. Powerpoint Templates 32
- 33. Scale In.. Powerpoint Templates 33
- 34. Shut down an Initiator Instance Powerpoint Templates 34
- 36. Parallel Simulations Powerpoint Templates 36
- 37. Multi-tenanted Deployments Powerpoint Templates 37
- 38. MapReduce Executions Powerpoint Templates 38
- 39. Implementation •CloudSim trunk forked •Hazelcast version 3.2 and Infinispan version 6.0.2. •Dependencies abstracted away. Powerpoint Templates 39
- 40. Evaluation •Setup: Cluster with 6 identical nodes –Intel® Core™ i7-2600K CPU @ 3.40GHz and 12 GB memory. •Varying number of parameters –Cloudlets: 100 → 400. –VMs: 100 → 200. –Nodes: 1 → 6. Powerpoint Templates 40
- 41. Simulation 1: CloudSim and Cloud2Sim •Round robin application scheduling with 200 VMs and 400 cloudlets. Execution Time Powerpoint Templates 41
- 42. Varying number of Cloudlets Powerpoint Templates 42
- 43. With Adaptive Scaling Powerpoint Templates 43
- 44. Simulation 2: Matchmaking-based Application Scheduling Execution Time Powerpoint Templates 44
- 45. Speed up Powerpoint Templates 45
- 46. Simulation 3: MapReduce Implementations Powerpoint Templates 46
- 47. Scalability Powerpoint Templates 47 Hazelcast Implementation Map() invocations = 3 Infinispan Implementation Reduce() invocations = 159,069
- 48. Conclusion •Summary – Distribution strategies and algorithms for cloud and MapReduce simulations. – Implementation of an Elastic Middleware platform. – Scale and perform with multiple nodes and larger simulations. Powerpoint Templates 48
- 49. Conclusion •Conclusions –Distributed architecture facilitates larger simulations. – Faster execution of time-consuming applications. Powerpoint Templates 49
- 50. Conclusion Powerpoint Templates 50 • Conclusions – Distributed architecture facilitates larger simulations. – Faster execution of time-consuming applications. • Future Work – State-aware Adaptive Scaling – Infinispan based Cloud Simulations. – Lighter objects. – Generic Elastic Middleware Platform-as-a- Service.
- 51. Conclusion Powerpoint Templates 51 • Conclusions – Distributed architecture facilitates larger simulations. – Faster execution of time-consuming applications. • Future Work – State-aware Adaptive Scaling – Infinispan based Cloud Simulations. – Lighter objects. – Generic Elastic Middleware Platform-as-a- Service. TThhaannkk yyoouu!! QQuueessttiioonnss??
- 52. References Buyya, R., R. Ranjan, & R. N. Calheiros (2009). Modeling and simulation of scalable cloud computing environments and the cloudsim toolkit: Challenges and opportunities. In High Performance Computing & Simulation, 2009. HPCS’09. International Conference on, pp. 1–11. IEEE. Calheiros, R. N., R. Ranjan, C. A. De Rose, & R. Buyya (2009). Cloudsim: A novel framework for modeling and simulation of cloud computing infrastructures and services. arXiv preprint arXiv:0903.2525 Calheiros, R. N., R. Ranjan, A. Beloglazov, C. A. De Rose, & R. Buyya (2011). Cloudsim: a toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms. Software: Practice and Experience 41 (1), 23–50. Casanova, H. (2001). Simgrid: A toolkit for the simulation of application scheduling. In Cluster Computing and the Grid, 2001. Proceedings. First IEEE/ACM International Symposium on, pp. 430–437. IEEE. Casanova, H., A. Legrand, & M. Quinson (2008). Simgrid: A generic framework for large-scale distributed experiments. In Computer Modeling and Simulation, 2008. UKSIM 2008. Tenth International Conference on, pp. 126–131. IEEE. Johns, M. (2013). Getting Started with Hazelcast. Packt Publishing Ltd. Kliazovich, D., P. Bouvry, & S. U. Khan (2012). Greencloud: a packet-level simulator of energy-aware cloud computing data centers. The Journal of Supercomputing 62 (3), 1263–1283. Legrand, A., L. Marchal, & H. Casanova (2003). Scheduling distributed applications: the simgrid simulation framework. In Cluster Computing and the Grid, 2003. Proceedings. CCGrid 2003. 3rd IEEE/ACM International Symposium on, pp. 138–145. IEEE. Marchioni, F. (2012). Infinispan Data Grid Platform. Packt Publishing Ltd. Powerpoint Templates 52