Making Elasticity Testing of Cloud-Based Systems Reproducible
- 3. Cloud Computing Elasticity N. R. Herbst, S. Kounev, and R. Reussner, “Elasticity in Cloud Computing: What It Is, and What It Is Not,” ICAC, pp. 23–27, 2013. M. M. Bersani, D. Bianculli, S. Dustdar, A. Gambi, C. Ghezzi, and S. Krstić, “Towards the Formalization of Properties of Cloud-based Elastic Systems,” in Proceedings of the 6th International Workshop on Principles of Engineering Service-Oriented and Cloud Systems, New York, NY, USA, 2014, pp. 38–47.
- 4. Elastic Behavior : threshold-based elasticity Time Resource (CPUs) 2 1 0.3 0.8 0.6 1.2 Ready Resource Allocation Resource Demand Scale-out Threshold Scale-in Threshold
- 5. Elastic Behavior : threshold-based elasticity Ready Time Resource (CPUs) 2 1 scale-out reaction time 0.3 0.8 0.6 1.2 Resource Allocation Resource Demand Scale-out Threshold Scale-in Threshold
- 6. Elastic Behavior : threshold-based elasticity Time Resource (CPUs) 2 1 scale-out reaction time scale-out time 0.3 0.8 0.6 1.2 Ready Scaling-Out Resource Allocation Resource Demand Scale-out Threshold Scale-in Threshold
- 7. Elastic Behavior : threshold-based elasticity Time Resource (CPUs) 2 1 scale-out reaction time scale-out time 0.3 0.8 0.6 1.2 Ready Scaling-Out Resource Allocation Resource Demand Scale-out Threshold Scale-in Threshold
- 8. Elastic Behavior : threshold-based elasticity Time Resource (CPUs) 2 1 scale-out reaction time scale-out time 0.3 0.8 0.6 1.2 scale-in reaction time Ready Scaling-Out Resource Allocation Resource Demand Scale-out Threshold Scale-in Threshold
- 9. Elastic Behavior : threshold-based elasticity Time Resource (CPUs) 2 1 scale-out reaction time scale-out time scale-in time scale-in reaction time 0.3 0.8 0.6 1.2 Ready Scaling-Out Scaling-In Resource Allocation Resource Demand Scale-out Threshold Scale-in Threshold
- 10. Elastic Behavior : threshold-based elasticity Ready Scaling-Out Scaling-In Resource Allocation Resource Demand Scale-out Threshold Scale-in Threshold Time Resource (CPUs) 2 1 scale-out reaction time scale-out time scale-in reaction time scale-in time 0.3 0.8 0.6 1.2
- 12. Motivation : MongoDB’s bug 7974 “ A secondary server shuts down when it detects a replication error, ... “ ● ○ ○ ○ ■
- 13. Motivation : MongoDB’s bug 7974 Primary Ready
- 14. Motivation : MongoDB’s bug 7974 Primary Ready Scaling-out
- 15. Motivation : MongoDB’s bug 7974 Primary Secondary Secondary Ready Scaling-out
- 16. Motivation : MongoDB’s bug 7974 Primary Ready Secondary Scaling-In Secondary
- 17. Motivation : MongoDB’s bug 7974 Primary Ready Secondary Scaling-InCreate Index
- 18. Motivation : MongoDB’s bug 7974 Primary Ready Secondary Scaling-InCreate Index
- 19. Motivation : MongoDB’s bug 7974 Primary Ready Secondary Scaling-In Secondary Scaling-Out
- 20. Motivation : MongoDB’s bug 7974 Primary Ready Secondary Scaling-In Secondary Scaling-Out Add doc. with duplicated key
- 21. Motivation : MongoDB’s bug 7974 Primary Ready Secondary Scaling-In Secondary Scaling-Out Add doc. with duplicated key
- 22. Motivation : MongoDB’s bug 7974 Primary Ready Secondary Scaling-In Secondary Scaling-Out Add doc. with duplicated key
- 23. Motivation : MongoDB’s bug 7974 Primary Ready Secondary Scaling-In Secondary Scaling-Out Duplicate Key Error
- 24. Requirements for Elasticity Testing Reproduction ● ○ ● ○ ● ○
- 25. Elasticity Testing Reproduction Case Studies ●
- 28. Prototype’s Architecture Elasticity Controller Mock Event Scheduler EC = {ec1 =(es1 ,W1 ), …, ecn } SER = {(ec1 , ser1 ), …, (ecn , sern )} Workload Generator Cloud-Based System Cloud Monitor S={(ec1 , e1 ), …, (ecn , en )}
- 29. Prototype’s Architecture Elasticity Controller Mock Event Scheduler EC = {ec1 =(es1 ,W1 ), …, ecn } SER = {(ec1 , ser1 ), …, (ecn , sern )} S={(ec1 , e1 ), …, (ecn , en )} Workload Generator Wi Cloud-Based System Cloud Monitor
- 30. Elasticity Controller Mock Event Scheduler EC = {ec1 =(es1 ,W1 ), …, ecn } SER = {(ec1 , ser1 ), …, (ecn , sern )} S={(ec1 , e1 ), …, (ecn , en )} Workload Generator Wi Cloud-Based System Workload Cloud Monitor Prototype’s Architecture
- 31. Elasticity Controller Mock Event Scheduler EC = {ec1 =(es1 ,W1 ), …, ecn } SER = {(ec1 , ser1 ), …, (ecn , sern )} S={(ec1 , e1 ), …, (ecn , en )} Workload Generator Wi Cloud-Based System Workload Resource Variation Cloud Monitor Prototype’s Architecture
- 32. Elasticity Controller Mock Event Scheduler EC = {ec1 =(es1 ,W1 ), …, ecn } SER = {(ec1 , ser1 ), …, (ecn , sern )} S={(ec1 , e1 ), …, (ecn , en )} Workload Generator Wi Cloud-Based System Workload NewElasticityState Resource Variation Cloud Monitor Prototype’s Architecture
- 33. Elasticity Controller Mock Event Scheduler EC = {ec1 =(es1 ,W1 ), …, ecn } SER = {(ec1 , ser1 ), …, (ecn , sern )} S={(ec1 , e1 ), …, (ecn , en )} Workload Generator Wi Cloud-Based System Workload Event Execution Resource Variation Cloud Monitor NewElasticityState Prototype’s Architecture
- 34. Elasticity Controller Mock Event Scheduler EC = {ec1 =(es1 ,W1 ), …, ecn } SER = {(ec1 , ser1 ), …, (ecn , sern )} S={(ec1 , e1 ), …, (ecn , en )} Workload Generator Wi Cloud-Based System Workload Event Execution Resource Variation Cloud Monitor NewElasticityState Prototype’s Architecture
- 37. ● Reproduction of Bugs Bug Success of Reproduction Using Our Approach Success of Reproduction Without Our Approach MongoDB-7974 3 0 ZooKeeper-2164 3 1 ZooKeeper-2172 3 0
- 38. ● Reproduction of Bugs Bug Success of Reproduction Using Our Approach Success of Reproduction Without Our Approach MongoDB-7974 3 0 ZooKeeper-2164 3 1 ZooKeeper-2172 3 0 ● Comercial elastic controllers natively removes either the oldest (first added) or the newest (last added) resource; ● In our experiment, ZooKeeper node on the oldest resource (VM) is elected the leader; ● At one of the three executions, we configure elastic controller to remove the oldest node; ○ Then, the bug is reproduced. ● If the leader was the node in the middle, the bug would never be reproduced without our approach.