Gray-Box Models for Performance Assessment of Spark Applications
0 likes32 views
The document discusses the development of gray-box models for assessing the performance of Spark applications, co-funded by the European Commission under Horizon 2020. It outlines the use of machine learning to optimize configurations and predict quality of service outcomes, utilizing various models and benchmarks in cloud computing environments. The findings indicate that while gray-box models are effective for identifying performance degradation, different machine learning techniques should be evaluated for specific scenarios.
Gray-Box Models for Performance Assessment of Spark Applications
- 1. Co-funded by the European Commission Horizon 2020 - Grant #777154 Gray-Box Models for Performance Assessment of Spark Applications atmosphere-eubrazil.eu @AtmosphereEUBR M. Lattuada, E. Gianniti, M. Hosseini, D. Ardagna, A. Maros, F. Murai, A. P. Couto da Silva and J.M. Almeida Politecnico di Milano, Italy Universidade Federal de Minas Gerais, Brazil
- 2. ● Develop models for: ○ Identify minimum cost configuration with a priori deadlines ○ Allow the adaptive actuation mechanisms to predict if QoS objective will be reached ○ Assessing (a posteriori) the main performance metrics in multi-tenancy environments ○ Evaluate if an application run was affected by resource contention ● Approach: ○ Gray box models based on Machine Learning (ML) ○ Open source ML library ○ Open source benchmarks 2 Contextualizing Performance Models The adoption of accurate models allows anticipating QoS violations and increasing cloud services trustworthiness by improving their performance
- 3. • Regression Models: l1-regularized Linear Regression, Neural Network, Decision Tree, Random Forests, and Support Vector Regression • Hyper-parameters optimization 3 ML Models overview
- 4. • Workloads: • TPC-DS - the industry benchmark (Query 26) for data warehouse systems • ML benchmarks (K-means) from the Sparkbench library • SparkDL developed on top of Sparkbench • Platforms: • Microsoft HDInsights on Azure - cloud computing service • IBM Power8 Cluster – dedicated cluster 4 ML Experiments Settings
- 5. • Sampling scheme analysis • Evaluation Metric: • Mean Absolute Percentage Error: 5 ML Experiments Settings
- 6. • Ernest Model by Spark inventors • Pure black-box approach: • Non Negative Least Square Regression (NNLS) • Features: • Ratio of data size to number of cores • Log of number of cores • Data size • Number of cores • Number of TensorFlow cores (SparkDL only) 6 Comparison with State of the Art Solutions S. Venkataraman, Z. Yang, M. Franklin, B. Recht, and I. Stoica. “Ernest: Efficient Performance Prediction for Large-Scale Advanced Analytics”. In: 13th USENIX Symposium on Networked Systems Design and Implementation. 2016, pp. 363–378.
- 7. 7 Gray-Box Model Features DAG structure constant across different runs
- 8. 8 Query 26 Results Interpolation Scenario Extrapolation Scenario
- 9. 9 K-Means & SparkDL Results Interpolation Scenario Extrapolation Scenario
- 10. • Gray-box models are effective for performance assessment to identify performance degradation (about 4-25% percentage error) • Work better when the application data size is fixed • Comparison with Ernest:in most cases, our best models improve Ernest considerably, especially when few profiling configurations are available in the training set and when workloads are less regular 10 ML Models Results Summary
- 11. • There is no ML technique which always outperforms the others, hence different techniques have to be evaluated in each scenario to choose the best model • Study the performance of Spark applications running on GPU- based clusters • Validate the models on production environments 11 Conclusions & Future work
- 12. 12 Thanks for your attention…