Coordinating CPU and Memory Elasticity Controllers toMeet Service Response Time Constraints

Coordinating CPU and Memory Elasticity Controllers to
Meet Service Response Time Constraints
Soodeh Farokhi1, Ewnetu Bayuh Lakew2, Cristian Klein3, Ivona Brandic1, and Erik Elmroth2
1 Vienna University of Technology, Austria
2 Umeå University, Sweden
3 SimScale GmbH, Germany
IEEE International Conference on Cloud and Autonomic Computing (ICCAC)
Cambridge, MA, USA
September 21-24, 2015

Soodeh Farokhi, IEEE International Conference on Cloud and Autonomic Computing (ICCAC’15), 21-24 Sep, 2015
Introduction: Elasticity on Cloud Computing
Horizontal Elasticity
 Course grained
 Slow (minutes)
 Needs application support
• State synchronization
• Load-balancing
Vertical Elasticity
 Fine grained
 Fast (sub-second)
 Little application support
• Multi-threaded
 Needs Hypervisor support
2
horizontal elasticity
verticalelasticity
[1] [E. Lakew, et. al. UCC 2014]
Fig. 1: vertical vs. horizontal elasticity [1]
Introduction Approach Evaluation

Vertical Elasticity
 key enabler for efficient resource utilization
 little support from the commercial cloud providers
• started to be supported by Hypervisors (e.g., Xen and KVM)
 few research efforts
• single resource => either CPU or memory (mostly CPU)
o need for multiple resources at different application execution stages
o need proper orchestration for using the existing approaches => inconsistent decisions!
• taking resource utilization as a decision making criterion
o capacity-based approaches
o response time as a key indicator => performance-based approaches
3

0
20
40
60
0 250 500 750 1000 1250
memutilization
[%]
Time [seconds] every 250 sec is an interval
100, 1 1500, 5 100, 0.1 100, 1 1000, 1
Motivation Scenario
4
Fig.3: Resource utilization of RUBiS (eBay-like e-commerce app.)
under different workload patterns.
0
10
20
30
40
50
60
CPUutilization
[%]
100 (cc) , 1 (tt) 1500, 5 100, 0.1 100, 1 1000, 1
1
2
3
4
Time [seconds] every 250 sec in an interval
 RUBiS as a benchmark application
• CPU-intensive behavior
• memory-intensive behavior
 Can an application be both?!
• based on the nature of the workload
 How to support CPU & Memory vertical elasticity?

Outline
 Goals & Challenges
 Autonomic Resource Controller
• CPU controller
• Memory controller
• Fuzzy controller (coordinator)
 Experimental Evaluation
 Conclusion
5Introduction Approach Evaluation

Research Goals & Challenges
 Goal: designing an autonomic resource elasticity controller
• Meet the application performance requirements
• Supporting vertical elasticity of both CPU & memory
 Challenges:
• Performance modeling for vertical scaling of both CPU & memory
• Elasticity reasoning for multiple resources under uncertainty
 F (average response time) -> CPU & memory capacity

Solution Overview: Autonomic Resource Controller
 following the MAPE loop
• Monitoring
• Analysis & Planning:
o Step 1: fuzzy controller infers the contributions degree of CPU & memory to app performance change
o Step 2: CPU & memory controllers determine the required CPU & memory capacity
• Execution
CPU
controller
memory
controller
Application
VM
# CPU
# MEM
CPU coefficient
fuzzy controller
mem coefficient
Hypervisor
Autonomic Resource Controller
App/VM Sensors
Fig.4: Solution overview
knowledge base

CPU & Memory Controllers
 Performance-based, adaptive & reactive controllers
…α and ß are system parameters
 CPU Controller [2]
• following an inverse model
• adopting RLS filtering to adaptively measure ß
o based on the past measurements
 Memory Controller [3]
• following a feedback control loop
• adopting LR to calculate α
8
Fig. 6. The feedback loop used for memory controller [3]
memory
controller
Application
VM
memory size
(ctli  memi)
workload
measured RT (rti)
+
-
desired RT (rt)~ ei = rt - rti
~
Fig. 5. CPU controller [2]
[2] [E. Lakew, et. al. UCC 2014]
[3] [S. Farokhi, et. al. ICAC 2015]

Fuzzy Controller
as a Coordination Technique
CPU
controller
memory
controller
Application
VM
# CPU
# MEM
CPU coefficient
fuzzy controller
mem coefficient
Hypervisor
Autonomic Resource Controller
App/VM Sensors
knowledge base

Fuzzy Controller Design
A fuzzy logic control approach is defined by:
• membership functions (MF): degree of truth with a value between 0 to 1
• fuzzy rules: a collection of “IF THEN ELSE” rules
 Design process
• Step 1: MF Construction
• Step 2: Fuzzy Rule Elicitation
• Step 3: Fuzzy Reasoning
 MIMO fuzzy logic system (FLS)
• three Input variables (next slide)
• two output variables
o CPU and memory coefficients, values between -1 (over-provisioning) to +1 (under-provisioning)
o -1 & +1 => fully allocate the resource ; -1 < value < +1 => partially allocate the resource

Step 1: MFs Construction
 Input variables
1. Response time
2. CPU utilization
3. Memory utilization
 linguistic terms represent the value of input variables
1. Slow (RT), Low (utilizations)
2. Medium (RT), Medium (utilizations)
3. Fast (RT), High (utilizations)
 We need to define a MF for each linguistic term
• 9 MFs in total, some triangular, some trapezoidal
11
Fig.8. MFs of Response time
Fig.9. MFs of CPU utilization

Step 2: Fuzzy Rule Elicitation
3 input variables => 33 combinations => 27 fuzzy rules
… using expert knowledge to extract them & then empirically update them at run time

Step 2: Output Variables (Hyper-surfaces)
 Output variables’ hyper-surfaces in accordance to input variables
13
Memory coefficient surfaceCPU coefficient surface

Step 3: Fuzzy Elasticity Reasoning
The process of elasticity reasoning (having “fuzzy knowledge-based”)
1. Measured values of input variables are fuzzified, using MFs
2. Fuzzy controller reasons & produces the output variables
o using the fuzzified input variables & fuzzy rules
3. Feeding the output variables into CPU & memory controller

Experimental
Evaluation
15

Experimental Setup
 Xen hypervisor
• focusing on the VM which hosts the BL tier (Apache Web Server)
 3 interactive benchmark applications
• Olio (Amazon-like, online bookstore)
• RUBBoS (Slashdot-like bulletin board)
• RUBiS (eBay-like e-commerce)
 Workload (synthetic traces)
• Open- & closed loop user model [4]
• Httpmon[5] load generator
 Control interval = 5sec
Xen hypervisor
server-side
VM1
workload
generator
PM (56 GB memory, 32 processors)
elastic RAM
Benchmark
Application
BL tier (Apache 2.0)
Autonomic
Resource
Controller
control-side
VM2
memory (fixed)
Benchmark
Application
DS tier (MySQL)
CPU (fixed) elastic CPU
App/VMsensors
client-side
[4] [Schroeder B., et al, NSDI, 2006] [5] https://github.com/cloud-control/httpmon

Evaluation Process
 The baseline approach
• Comparing fuzzy Controller (FC) with a Non-fuzzy Controller (NFC)
o NFC: an approach with two separated CPU & memory controllers
• better controller? => meets the desired RT without over-provisioning any of resources
 Evaluation metrics
• Control theoretical metrics: ISE and IAE (present control error)
• CPU usage (mean)
• Memory usage (mean)
• Response time (mean & SD)
 Process (run 10 different experiments for both FC & NFC)
• Time-series analysis
• Aggregated results analysis

Evaluation Results (1) - Olio (target RT: 1 sec)
response time
[sec]
CPU usage
[#core]
memory usage
[GB]

Evaluation Results (2) – Aggregated
control error =
desired RT – measured RT

Experimental Results (3) – Aggregated
 FC improvement in the following metrics compared to NFC
(1) allocated memory (2) allocated CPU (3) RT stability (standard deviation)
 Achieving almost better results under open loop user model in general
 For some applications (e.g., RUBBoS) the improvement is lower
60.76
20.08 18.98
47.85
3.97
15.44
2.86 0
56.51
7.30 2.45
64.78
26.23 29.68
79.08
9.38
22.12
0
20
40
60
80
RUBiS (0.5sec) RUBBoS (1sec) Olio (0.5sec) RUBiS (0.5sec) RUBBoS (1sec) Olio (0.5sec)
open system model closed system model
Improvement[%]
allocated memory
allocated CPU
stability (RT)

Summary of Results (4)
 Unpredictable system behavior without coordinating elasticity controllers (using NFC)
• due to conflicting decision
 under FC & NFC the application performance may be met
• at the expense of over-provision one of the resources (mostly memory)
• application crashing due to severe resource shortage as a result of conflicting decisions
 NFC uses more CPU and memory on average than FC
• under similar configurations
• with comparable mean RT
• more control error
 With careful coordination of auto-scaling controllers
• application performance is met with optimal amount of resources
• achieving high resource utilization & preventing both resource over- and under-provisioning

Conclusion & Future work
 A generic coordination technique for distributed controllers
• reasoning under uncertainty (using fuzzy logic)
 an autonomic resource controller consisting of three sub-controllers:
• Fuzzy controller, CPU controller, and Memory controller.
• Supporting both CPU & memory elasticity
 Observing the application RT as well as CPU & memory utilizations
• meeting the application response time constraints
• maintaining high utilization of resources
Ongoing work
• online learning for self-adaptation of the fuzzy rules and MFs
• using tail response time: 95th and 99th percentile
22

Coordinating CPU and Memory Elasticity Controllers to
Meet Service Response Time Constraints
23
soodeh.farokhi@tuwien.ac.at
Vienna University of Technology
at.linkedin.com/in/soodehfa
Soodeh Farokhi, Ewnetu Bayuh Lakew, Cristian Klein, Ivona Brandic, and Erik Elmroth

What is the State-of-the-art?
24
CPU Vertical Scaling
Capacity-based
[E. Kalyvianaki, et al. 2009]
[S. Spinner , et al. 2014]
Performance-based
[E. Lakew, at. al. 2014]
Memory Vertical Scaling
Capacity-based
[A. Baruchi, et al. 2011]
[G. Molto, et al. 2013]
[W. Wangusing, et al. 2014]
Performance-based
[S. Farokhi, et. al, 2015]
[S. Spinner, et. al, 2015]
CPU & Memory Vertical Scaling
Capacity-based
[Y. Diao, et al. 2002]
[L. Lu, et al. 2014]
Performance-based
?

Coordinating CPU and Memory Elasticity Controllers toMeet Service Response Time Constraints

More Related Content

What's hot (20)

Similar to Coordinating CPU and Memory Elasticity Controllers toMeet Service Response Time Constraints (20)

Recently uploaded (20)