Towards the Realization of Multi-dimensional Elasticity for Distributed Cloud Systems
1 like273 views
The document discusses realizing multi-dimensional elasticity for distributed cloud systems. It proposes key concepts for multi-dimensional elasticity including elasticity zones, elasticity space, analysis and prediction, patterns, and adjustment functions. It analyzes current limitations in elasticity engineering and proposes approaches to address these, such as using formal models and algorithms. Future work involves further implementing these concepts for Internet of Things, network functions virtualization, and clouds.
Towards the Realization of Multi-dimensional Elasticity for Distributed Cloud Systems
- 1. Towards the Realization of Multi- dimensional Elasticity for Distributed Cloud Systems Hong-Linh Truong, Schahram Dustdar, Frank Leymann Distributed Systems Group, TU Wien & IAAS, University of Stuttgart truong@dsg.tuwien.ac.at http://dsg.tuwien.ac.at/staff/truong CloudForward 2016, Madrid, 19 Oct, 2016 1
- 2. Outline Toward “Complete Cloud Computing” Multi-dimensional elasticity – key concepts Current effort in some EU projects Realizing multi-dimensional elasticity Conclusions and future work CloudForward 2016, Madrid, 19 Oct, 2016 2
- 3. Toward „Complete Computing“ CloudForward 2016, Madrid, 19 Oct, 2016 3 Application example Near-Realtime Data Processing Sensors NoSQL BigData Sensor data Gateways EventHandling Web Service Load Balancer Message-oriented Middleware Sensor data IoT Cloud Systems – the software layer The cloud – cloud servicesThe edge – IoT units Large-scale Data Analytics Lightweighted Analytics and Control IoT Cloud Applications „Complete cloud system example“
- 4. What is multi-dimensional elasticity? Key concepts Not just auto-scaling computing resources (VMs or containers)! CloudForward 2016, Madrid, 19 Oct, 2016 4
- 5. Why are they important? Enable formal models, methods and tools for elasticity management, coordination and interoperability Few works in multi-cloud environments So far we have not entered into the edge and IoT systems CloudForward 2016, Madrid, 19 Oct, 2016 5 „High-level but complete view“
- 6. Current effort in some (finished) EU projects CloudForward 2016, Madrid, 19 Oct, 2016 6 Project Multi- cloud elasticit y Edge and Cloud Elasticity Elasticity Zone Elasticity Space Elasticity Prediction Elasticity Adjustment CELAR Partially Partially Partially Partially No Yes HARNESS No No Partially No Partially Yes MODAClouds Yes No Partially Partially Yes PaaSage Yes No Partially No Yes
- 7. Elasticity constraints surveyed by others CloudForward 2016, Madrid, 19 Oct, 2016 7 Source: A Review of Auto-scaling Techniques for Elastic Applications in Cloud Environments, Tania Lorido-Botran , Jose Miguel-Alonso, Jose A. Lozano, http://link.springer.com/article/10.1007%2Fs10723-014-9314-7
- 8. Realizing Multi-dimensional Elasticity – Elasticity zones Limitations Lack a “formal” model/specification Using rules fails to capture dynamics (e.g., time) No edge resources Elasticity Zones mainly defined by humans CloudForward 2016, Madrid, 19 Oct, 2016 8 Approach: Formal model: n-dimensional manifold More than just computing resources and prices Data sources, quality of data, uncertainties Dynamic Elasticity Zones: event-dependent Triggers: Human-in-the-loop, prediction functions, pre-defined events, etc.
- 9. Realizing Multi-dimensional Elasticity – Elasticity space Limitations Do not support changes of space dimensions Not enough monitoring data or too much data Lack of correlation among various layers: e.g., either applications or VM/containers Lack monitoring data from edge systems and NFV CloudForward 2016, Madrid, 19 Oct, 2016 9 Approach: Algorithmic models for Elasticity Space functions Generic functions for determining spaces (start, stop and why) and for different layers/topologies Operators on Elasticity Space, e.g., merging spaces for a composite topology of components
- 10. Realizing Multi-dimensional Elasticity – Analysis and prediction Limitations Quite traditional performance analysis (for the cloud) E.g., a lot of uncertainties and data quality metrics have not been considered Prediction is mainly for single dimensions CloudForward 2016, Madrid, 19 Oct, 2016 10 Approach: Elasticity dependency analysis Algorithms for elasticity prediction Understand which part of code cannot be elastilized and when the system might be “plastic” Prediction for common cloud patterns/basic building blocks
- 11. Realizing Multi-dimensional Elasticity – Patterns Limitations Mainly on scalability best practices and patterns No real elasticity patterns Lack of unified way to define and model “management function” (enabling elasticity) CloudForward 2016, Madrid, 19 Oct, 2016 11 Approach: Elasticity primitives: common notations for abstracting low-level APIs for elasticity management Common models for elasticity primitives for network functions and IoT/edge systems New primitives for data-aware elasticity Deduce elasticity patterns
- 12. Realizing Multi-dimensional Elasticity – Adjustment functions Limitations Several existing functions but no theoretical models to combine them to support multi-dimensional elasticity adjustment Focused mainly on centralized clouds CloudForward 2016, Madrid, 19 Oct, 2016 12 Approach: Deal with different level of abstractions and coordinated adjustment Fundamental steps Elasticity pattern selection Primitive operations selection Selecting/generating/configuring elasticity adjustment functions Elasticity operation management (e.g. incident management)
- 13. How far we are? Elasticity for IoT Cloud systems http://tuwiendsg.github.io/iCOMOT/ Elasticity partially considers uncertainties CloudCom 2015 Data elasticity ICSOC 2015 Coordination-aware elasticity UCC 2015 and the work in progress CloudForward 2016, Madrid, 19 Oct, 2016 13
- 14. Conclusions and future work Multi-dimensional elasticity Key concepts for “complete computing” atop IoT, edge systems and clouds Our work Analyzed current limitations of elasticity engineering in the clouds Proposed key concepts and suitable approaches: Elasticity Zone, Space, Prediction and Adjustment Future work Implementation is on progress for IoT, Network function virtualization, and clouds Check iCOMOT http://tuwiendsg.github.io/iCOMOT/ and SINC http://sincconcept.github.io CloudForward 2016, Madrid, 19 Oct, 2016 14
- 15. Thanks for your attention! Hong-Linh Truong Distributed Systems Group TU Wien dsg.tuwien.ac.at/staff/truong CloudForward 2016, Madrid, 19 Oct, 2016 15