Estimating the Cost for Executing Business Processes in the Cloud
1 like679 views
The document discusses the estimation of costs for executing business processes in the cloud, focusing on the deployment of workflow management systems (WFMS) and the challenges of evaluating cloud cost models and suitable providers. It outlines a method for selecting virtual machines (VMs) based on workload requirements and performance metrics, involving steps like workload analysis, performance testing, and resource efficiency assessment. The document also highlights various examples of workload mixes and their complexities, emphasizing the need for robust validation of performance on cloud platforms.
![Vincenzo Ferme
15
4. Run Experiments
the BenchFlow framework
Instance
Database
DBMSFaban Drivers
ContainersServers
DATA
TRANSFORMERS
ANALYSERS
Performance
Metrics
Performance
KPIs
WfMS
TestExecutionAnalyses
Faban
+
Web
Service
Minio
harness
benchflow [BPM ’15] [ICPE ’16]
MONITOR
Adapters
COLLECTORS](https://image.slidesharecdn.com/estimatingthecostforexecutingbusinessprocessesinthecloudvincenzoferme-160920220800/85/Estimating-the-Cost-for-Executing-Business-Processes-in-the-Cloud-42-320.jpg)
![Vincenzo Ferme
29
Published Work
[BTW ’15]
C. Pautasso, V. Ferme, D. Roller, F. Leymann, and M. Skouradaki. Towards workflow
benchmarking: Open research challenges. In Proc. of the 16th conference on
Database Systems for Business,Technology, and Web, BTW 2015, pages 331–350, 2015.
[SSP ’14]
M. Skouradaki, D. H. Roller, F. Leymann, V. Ferme, and C. Pautasso. Technical open
challenges on benchmarking workflow management systems. In Proc. of the 2014
Symposium on Software Performance, SSP 2014, pages 105–112, 2014.
[ICPE ’15]
M. Skouradaki, D. H. Roller, L. Frank, V. Ferme, and C. Pautasso. On the Road to
Benchmarking BPMN 2.0 Workflow Engines. In Proc. of the 6th ACM/SPEC
International Conference on Performance Engineering, ICPE ’15, pages 301–304, 2015.](https://image.slidesharecdn.com/estimatingthecostforexecutingbusinessprocessesinthecloudvincenzoferme-160920220800/85/Estimating-the-Cost-for-Executing-Business-Processes-in-the-Cloud-70-320.jpg)
![Vincenzo Ferme
30
Published Work
[CLOSER ’15]
M. Skouradaki,V. Ferme, F. Leymann, C. Pautasso, and D. H. Roller. “BPELanon”: Protect
business processes on the cloud. In Proc. of the 5th International Conference on
Cloud Computing and Service Science, CLOSER 2015. SciTePress, 2015.
[SOSE ’15]
M. Skouradaki, K. Goerlach, M. Hahn, and F. Leymann. Application of Sub-Graph
Isomorphism to Extract Reoccurring Structures from BPMN 2.0 Process Models.
In Proc. of the 9th International IEEE Symposium on Service-Oriented System
Engineering, SOSE 2015, 2015.
[BPM ’15]
V. Ferme, A. Ivanchikj, C. Pautasso. A Framework for Benchmarking BPMN 2.0
Workflow Management Systems. In Proc. of the 13th International Conference on
Business Process Management, BPM ’15, pages 251-259, 2015.](https://image.slidesharecdn.com/estimatingthecostforexecutingbusinessprocessesinthecloudvincenzoferme-160920220800/85/Estimating-the-Cost-for-Executing-Business-Processes-in-the-Cloud-71-320.jpg)
![Vincenzo Ferme
31
Published Work
[BPMD ’15]
A. Ivanchikj,V. Ferme, C. Pautasso. BPMeter: Web Service and Application for Static
Analysis of BPMN 2.0 Collections. In Proc. of the 13th International Conference on
Business Process Management [Demo], BPM ’15, pages 30-34, 2015.
[ICPE ’16]
V. Ferme, and C. Pautasso. Integrating Faban with Docker for Performance
Benchmarking. In Proc. of the 7th ACM/SPEC International Conference on Performance
Engineering, ICPE ’16, 2016.
[CLOSER ’16]
V. Ferme, A. Ivanchikj, C. Pautasso., M. Skouradaki, F. Leymann. A Container-centric
Methodology for Benchmarking Workflow Management Systems. In Proc. of the
6th International Conference on Cloud Computing and Service Science, CLOSER 2016.
SciTePress, 2016.](https://image.slidesharecdn.com/estimatingthecostforexecutingbusinessprocessesinthecloudvincenzoferme-160920220800/85/Estimating-the-Cost-for-Executing-Business-Processes-in-the-Cloud-72-320.jpg)
![Vincenzo Ferme
[CAiSE ’16]
M. Skouradaki, V. Ferme, C. Pautasso, F. Leymann, A. van Hoorn. Micro-Benchmarking
BPMN 2.0 Workflow Management Systems with Workflow Patterns . In Proc. of
the 28th International Conference on Advanced Information Systems Engineering, CAiSE
’16, 2016.
32
Published Work
[ICWE ’16]
C. Jürgen,V. Ferme, H.C. Gall. Using Docker Containers to Improve Reproducibility
in Software and Web Engineering Research. In Proc. of the 16th International
Conference on Web Engineering, 2016.
[ICWS ’16]
M. Skouradaki,V.Andrikopoulos, F. Leymann. Representative BPMN 2.0 Process Model
Generation from Recurring Structures. In Proc. of the 23rd IEEE International
Conference on Web Services, ICWS '16, 2016.](https://image.slidesharecdn.com/estimatingthecostforexecutingbusinessprocessesinthecloudvincenzoferme-160920220800/85/Estimating-the-Cost-for-Executing-Business-Processes-in-the-Cloud-73-320.jpg)
![Vincenzo Ferme
33
Published Work
[SummerSOC ’16]
M. Skouradaki, T. Azad, U. Breitenbücher, O. Kopp, F. Leymann. A Decision Support
System for the Performance Benchmarking of Workflow Management Systems.
In Proc. of the 10th Symposium and Summer School On Service-Oriented Computing,
SummerSOC '16, 2016.
[OTM ’16]
M. Skouradaki, V. Andrikopoulos, O. Kopp, F. Leymann. RoSE: Reoccurring Structures
Detection in BPMN 2.0 Process Models Collections. In Proc. of On the Move to
Meaningful Internet Systems Conference, OTM '16, 2016. (to appear)
[BPM Forum ’16]
V. Ferme, A. Ivanchikj, C. Pautasso. Estimating the Cost for Executing Business
Processes in the Cloud. In Proc. of the 14th International Conference on Business
Process Management, BPM Forum ’16, 2016.](https://image.slidesharecdn.com/estimatingthecostforexecutingbusinessprocessesinthecloudvincenzoferme-160920220800/85/Estimating-the-Cost-for-Executing-Business-Processes-in-the-Cloud-74-320.jpg)
![Vincenzo Ferme
34
Docker Performance
[IBM ’14]
W. Felter, A. Ferreira, R. Rajamony, and J. Rubio. An updated performance comparison
of virtual machines and Linux containers. IBM Research Report, 2014.
Although containers themselves have almost no overhead,
Docker is not without performance gotchas. Docker
volumes have noticeably better performance than files stored in
AUFS. Docker’s NAT also introduces overhead for workloads
with high packet rates. These features represent a tradeoff
between ease of management and performance and should
be considered on a case-by-case basis.
”
“Our results show that containers result in equal or better
performance than VMs in almost all cases.
“
”
BenchFlow Configures Docker for Performance by Default](https://image.slidesharecdn.com/estimatingthecostforexecutingbusinessprocessesinthecloudvincenzoferme-160920220800/85/Estimating-the-Cost-for-Executing-Business-Processes-in-the-Cloud-75-320.jpg)
Estimating the Cost for Executing Business Processes in the Cloud
- 1. Vincenzo Ferme, Ana Ivanchikj, Cesare Pautasso Faculty of Informatics USI Lugano, Switzerland ESTIMATING THE COST FOR EXECUTING BUSINESS PROCESSES IN THE CLOUD
- 2. Vincenzo Ferme 2 Deploying WfMSs to the Cloud WfMS sweet spot in the performance vs. resource consumption trade-off
- 3. Vincenzo Ferme 2 Deploying WfMSs to the Cloud WfMS D A B C sweet spot in the performance vs. resource consumption trade-off
- 4. Vincenzo Ferme 2 Deploying WfMSs to the Cloud WfMS D A B C IT Resources sweet spot in the performance vs. resource consumption trade-off
- 5. Vincenzo Ferme 2 Deploying WfMSs to the Cloud WfMS D A B C $ IT Resources sweet spot in the performance vs. resource consumption trade-off
- 6. Vincenzo Ferme 2 Deploying WfMSs to the Cloud WfMS IaaS D A B C $ IT Resources sweet spot in the performance vs. resource consumption trade-off
- 7. Vincenzo Ferme 2 Deploying WfMSs to the Cloud WfMS IaaS D A B C P a a S Workload Users $ IT Resources sweet spot in the performance vs. resource consumption trade-off B P a a S
- 8. Vincenzo Ferme 2 Deploying WfMSs to the Cloud WfMS IaaS D A B C P a a S Workload Users $ $ IT Resources sweet spot in the performance vs. resource consumption trade-off B P a a S
- 9. Vincenzo Ferme 3 Challenges in Deploying WfMSs in the Cloud evaluate the Cloud cost models, determine the best suitable provider Evaluate the Cloud Offers
- 10. Vincenzo Ferme 3 Challenges in Deploying WfMSs in the Cloud evaluate the Cloud cost models, determine the best suitable provider Evaluate the Cloud Offers
- 11. Vincenzo Ferme 3 Challenges in Deploying WfMSs in the Cloud evaluate the Cloud cost models, determine the best suitable provider Evaluate the Cloud Offers Best Suitable Provider?
- 12. Vincenzo Ferme 3 Challenges in Deploying WfMSs in the Cloud evaluate the Cloud cost models, determine the best suitable provider Evaluate the Cloud Offers Best Suitable Provider? $ Performance Resources
- 13. Vincenzo Ferme 4 Goal select candidate VMs according to workload and wanted performance WfMS
- 14. Vincenzo Ferme 4 Goal select candidate VMs according to workload and wanted performance WfMS Throughput Resp. Time
- 15. Vincenzo Ferme 4 Goal select candidate VMs according to workload and wanted performance WfMS RAM USAGE CPU USAGE Throughput Resp. Time
- 16. Vincenzo Ferme 4 Goal select candidate VMs according to workload and wanted performance WfMS RAM USAGE CPU USAGE CPU + RAM Bound Throughput Resp. Time
- 17. Vincenzo Ferme 4 Goal select candidate VMs according to workload and wanted performance WfMS RAM USAGE CPU USAGE CPU + RAM Bound Throughput Resp. Time Matching Clouds VMs $$ $ $$ $
- 18. Vincenzo Ferme 5 The Steps of the Method from performance requirements to cheapest VMs 1. Workload Mix 80% C A B 20% D
- 19. Vincenzo Ferme 5 The Steps of the Method from performance requirements to cheapest VMs 1. Workload Mix 80% C A B 20% D 0 600 1200 0 1 2 3 4 5 6 7 8 9 10 2. Load Function
- 20. Vincenzo Ferme 5 The Steps of the Method from performance requirements to cheapest VMs 1. Workload Mix 80% C A B 20% D 0 600 1200 0 1 2 3 4 5 6 7 8 9 10 2. Load Function Metrics KPIs 3. Performance Req.
- 21. Vincenzo Ferme 5 The Steps of the Method from performance requirements to cheapest VMs 1. Workload Mix 80% C A B 20% D 0 600 1200 0 1 2 3 4 5 6 7 8 9 10 2. Load Function Metrics KPIs 3. Performance Req. 4. Run Experiments
- 22. Vincenzo Ferme 5 The Steps of the Method from performance requirements to cheapest VMs 1. Workload Mix 80% C A B 20% D 0 600 1200 0 1 2 3 4 5 6 7 8 9 10 2. Load Function Metrics KPIs 3. Performance Req. 4. Run Experiments5. Analyse Results Resources Perform ance
- 23. Vincenzo Ferme 5 The Steps of the Method from performance requirements to cheapest VMs 1. Workload Mix 80% C A B 20% D 0 600 1200 0 1 2 3 4 5 6 7 8 9 10 2. Load Function Metrics KPIs 3. Performance Req. 4. Run Experiments5. Analyse Results Resources Perform ance 6. Cloud Offers $
- 24. Vincenzo Ferme 5 The Steps of the Method from performance requirements to cheapest VMs 0 600 1200 0 1 2 3 4 5 6 7 8 9 10 2. Load Function Metrics KPIs 3. Performance Req. 4. Run Experiments5. Analyse Results Resources Perform ance 6. Cloud Offers $ 7. VMs Selection Res. Efficiency Cost $
- 25. Vincenzo Ferme 6 Assumptions RAM and CPU, only WfMS no DBMS, workload dependent CPU RAM Observed Resources
- 26. Vincenzo Ferme 6 Assumptions RAM and CPU, only WfMS no DBMS, workload dependent CPU RAM No Network, No I/O Observed Resources
- 27. Vincenzo Ferme 6 Assumptions RAM and CPU, only WfMS no DBMS, workload dependent CPU RAM No Network, No I/O Observed Resources WfMS DBMS Only WfMS
- 28. Vincenzo Ferme 6 Assumptions RAM and CPU, only WfMS no DBMS, workload dependent CPU RAM No Network, No I/O Observed Resources WfMS DBMS Only WfMS 80% C A B 20% D 0 600 1200 0 1 2 3 4 5 6 7 8 9 10 Workload Dependent
- 29. Vincenzo Ferme 7 An Example of Application applying the method for the Camunda WfMS 1. 80% C A B 20% D 0 600 1200 0 1 2 3 4 5 6 7 8 9 10 2. Metrics KPIs 3. 4. 5. Resources Perform ance 6. $ 7. Resources Efficiency Cost $
- 30. Vincenzo Ferme 8 1. The Workload Mix realistic, mix of 5 models with different complexity camunda_additional_approval Change necessary? Amend something (Empty Script) Do the next task (Empty Script) Do some approval (Decision Script) Approved? Do the next task (Empty Script) Counter instantiation no yes yes no 21 Elements (Nodes + Edges)
- 31. Vincenzo Ferme 9 1. The Workload Mix realistic, mix of 5 models with different complexity camunda_oopSignavio Order Received Determine Location (Decision Script) Exception detected? Escalation (Empty Script) Order amount? Order Canceled Approved? Send Rejection Email (Empty Script) Order Rejected Send Order to WH (Empty Script) Receive Dispatch Confirmation (Empty Script) Control Dispatch (Empty Script) Send Confirmation Email (Empty Script) Order Confirmed Order Approval (Decision Script) >200 <=200no yes no yes 32 Elements (Nodes + Edges)
- 32. Vincenzo Ferme 10 1. The Workload Mix realistic, mix of 5 models with different complexity camunda_invoice_collaboration Scan Invoice (Empty Script) Archive Original (Empty Script) Ask Approver Assignment (Empty Script) Approve Invoice (Decision Script) Assign Approver (Empty Script) Invoice Approved? Prepare Bank Transfer (Empty Script) Ask Invoice Review (Empty Script) Review successful? Review Invoice (Decision Script) Archive Invoice (Empty Script) noyes no yes 39 Elements (Nodes + Edges)
- 33. Vincenzo Ferme 11 1. The Workload Mix realistic, mix of 5 models with different complexityactiviti_modelSignavio In-depth Analysis Medical Analysis (Empty Script) Risk Assessment (Empty Script) Send Meeting Request (Empty Script) F2F Meeting (Empty Script) Calculate Score (Empty Script) Preliminary Judgement (Empty Script) Send Rejection Email (Empty Script) Approve Loan Application (Decision Script) Approved? Update Application With Calculation (Empty Script) Final Evaluation (Decision Script) yes not ok ok no 41 Elements (Nodes + Edges)
- 34. Vincenzo Ferme 12 1. The Workload Mix realistic, mix of 5 models with different complexity COUNTERPARTY Counterparty Requested Review Onboarding Request (Decision Script) Override requested? Review Override Request (Decision Script) Tax (Empty Script) Missing Tax Documents (Decision Script) Goto tax? Check Tax Rules (Decision Script) Override requested? Tax approved? Amend Tax Data (Decision Script) Override requested? High Risk Country (Empty Script) Missing Compliance Documents (Empty Script) World Check (Decision Script) Goto compliance? Check Compliance Rules (Decision Script) Override requested? Compliance approved? Amend Compliance Data (Decision Script) Override requested? Start SSI Review (Empty Script) Override approved? override rejected Ready to publish? Publish Counterparty (Empty Script) Cancel SSI Review (Empty Script) rejected published yes no yes no no maybe no yes no maybe noyes yes yes no yes no yes no no yes no yes yes 84 Elements (Nodes + Edges)
- 35. Vincenzo Ferme 13 2. The Load Functions realistic for different sized companies • 50 • 500 • 1000 users (usr)
- 36. Vincenzo Ferme 13 2. The Load Functions realistic for different sized companies Users 0 200 400 600 800 1000 Time (min:sec)0:000:301:002:003:004:005:006:007:008:0010:00 Example for usr=1000 • 50 • 500 • 1000 users (usr)
- 37. Vincenzo Ferme 13 2. The Load Functions realistic for different sized companies Users 0 200 400 600 800 1000 Time (min:sec)0:000:301:002:003:004:005:006:007:008:0010:00 Example for usr=1000 • 50 • 500 • 1000 users (usr) 1 second think time
- 38. Vincenzo Ferme 14 3. Performance Requirements determined with “unbounded” experiments “Unbounded” (U) Configuration: CPURAM 128 GB 64 Cores
- 39. Vincenzo Ferme 14 3. Performance Requirements determined with “unbounded” experiments “Unbounded” (U) Configuration: CPURAM 128 GB 64 Cores Target Performance
- 40. Vincenzo Ferme 14 3. Performance Requirements determined with “unbounded” experiments “Unbounded” (U) Configuration: CPURAM 128 GB 64 Cores Target Performance • Number of Client Requests per second (REQ/s) • Response Time (RT) • … Client-side Metrics:
- 41. Vincenzo Ferme 14 3. Performance Requirements determined with “unbounded” experiments “Unbounded” (U) Configuration: CPURAM 128 GB 64 Cores Target Performance • Number of Client Requests per second (REQ/s) • Response Time (RT) • … • Number of executed BP Instances (N) • BP Instance Duration (D) • Throughput (T) • … Client-side Metrics: Server-side Metrics:
- 42. Vincenzo Ferme 15 4. Run Experiments the BenchFlow framework Instance Database DBMSFaban Drivers ContainersServers DATA TRANSFORMERS ANALYSERS Performance Metrics Performance KPIs WfMS TestExecutionAnalyses Faban + Web Service Minio harness benchflow [BPM ’15] [ICPE ’16] MONITOR Adapters COLLECTORS
- 43. Vincenzo Ferme 16 5. Analyse Results define (determine) the wanted performance WfMS Configurations Containers
- 44. Vincenzo Ferme 16 5. Analyse Results define (determine) the wanted performance WfMS Configurations Containers 3 Servers 10 Gbit/s 10 Gbit/s WfMS, DBMS, Load Generator
- 45. Vincenzo Ferme 16 5. Analyse Results define (determine) the wanted performance Three Runs to Improve Results Reliability
- 46. Vincenzo Ferme 17 5. Analyse Results determine the right amount of needed resources Normalised performance over number of CPU cores with 500 Users
- 47. Vincenzo Ferme 17 5. Analyse Results determine the right amount of needed resources Normalised performance over number of CPU cores with 500 Users
- 48. Vincenzo Ferme 17 5. Analyse Results determine the right amount of needed resources Normalised performance over number of CPU cores with 500 Users
- 49. Vincenzo Ferme 18 5. Analyse Results validate the performance results Unbound vs Bound Performance Results
- 50. Vincenzo Ferme 18 5. Analyse Results validate the performance results Unbound vs Bound Performance Results
- 51. Vincenzo Ferme 19 6. Cloud Offers match resource requirements with Cloud offers $$ $ CPU RAM CPU + RAM Bound
- 52. Vincenzo Ferme 20 6. Cloud Offers match resource requirements with Cloud offers 0 0.2 0.4 0.6 0.8 1 1.2 50 users 500 users 1000 users Am Az p Am Gp 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 Price ($/hr) 500 users 1000 users Am Az Gc Am Gp Gc 0.6 0.8 1 1.2 1.4 1.6 1.8 2 Price ($/hr) 1000 users
- 53. Vincenzo Ferme 21 6. Cloud Offers match resource requirements with Cloud offers 0 0.2 0.4 0.6 0.8 1 1.2 50 users 500 users 1000 users Am Az p Am Gp 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 Price ($/hr) 500 users 1000 users Am Az Gc Am Gp Gc 0.6 0.8 1 1.2 1.4 1.6 1.8 2 Price ($/hr) 1000 users
- 54. Vincenzo Ferme 22 6. Cloud Offers match resource requirements with Cloud offers 0 0.2 0.4 0.6 0.8 1 1.2 50 users 500 users 1000 users Am Az p Am Gp 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 Price ($/hr) 500 users 1000 users Am Az Gc Am Gp Gc 0.6 0.8 1 1.2 1.4 1.6 1.8 2 Price ($/hr) 1000 users
- 55. Vincenzo Ferme 23 7. VMs Selection select the set of suitable VMs, and identify the cheapest ones Normalised Cores Utilisation vs Price of VMs
- 56. Vincenzo Ferme 23 7. VMs Selection select the set of suitable VMs, and identify the cheapest ones Normalised Cores Utilisation vs Price of VMs
- 57. Vincenzo Ferme 23 7. VMs Selection select the set of suitable VMs, and identify the cheapest ones Normalised Cores Utilisation vs Price of VMs
- 58. Vincenzo Ferme 24 Limitations and Future Work approximation of the actual performance on the Cloud Performance on the Cloud has more Variance
- 59. Vincenzo Ferme 25 Limitations and Future Work measure on the Cloud, validate the approach 1. Workload Mix 80% C A B 20% D
- 60. Vincenzo Ferme 25 Limitations and Future Work measure on the Cloud, validate the approach 0 600 1200 0 1 2 3 4 5 6 7 8 9 10 2. Load Function Metrics KPIs 3. Performance Req. 4. Run Experiments6. Cloud Offers $ 7. VMs Selection Res. Efficiency Cost $ 5. Analyse Results Resources Perform ance
- 61. Vincenzo Ferme 25 Limitations and Future Work measure on the Cloud, validate the approach Metrics KPIs 3. Performance Req. 4. Run Experiments6. Cloud Offers $ 7. VMs Selection Res. Efficiency Cost $ 8. Cloud Exp. 5. Analyse Results Resources Perform ance
- 62. Vincenzo Ferme 25 Limitations and Future Work measure on the Cloud, validate the approach Metrics KPIs 3. Performance Req. 4. Run Experiments6. Cloud Offers $ 7. VMs Selection Res. Efficiency Cost $ 8. Cloud Exp. Add Network, I/O 5. Analyse Results Resources Perform ance
- 63. Vincenzo Ferme 25 Limitations and Future Work measure on the Cloud, validate the approach Metrics KPIs 3. Performance Req. 4. Run Experiments6. Cloud Offers $ 7. VMs Selection Res. Efficiency Cost $ 8. Cloud Exp. Add Network, I/O 5. Analyse Results Resources Perform ance Consider DBMS
- 64. Highlights Deploying WfMSs to the Cloud Vincenzo Ferme 2 Deploying WfMSs to the Cloud WfMS IaaS D A B C P a a S Workload Users $ $ IT Resources sweet spot in the performance vs. resource consumption trade-off B P a a S
- 65. Highlights Deploying WfMSs to the Cloud Vincenzo Ferme 2 Deploying WfMSs to the Cloud WfMS IaaS D A B C P a a S Workload Users $ $ IT Resources sweet spot in the performance vs. resource consumption trade-off B P a a S Proposed Method Vincenzo Ferme 5 The Steps of the Method from performance requirements to cheapest VMs 1. Workload Mix 80% C A B 20% D 0 600 1200 0 1 2 3 4 5 6 7 8 9 10 2. Load Function Metrics KPIs 3. Performance Req. 4. Run Experiments5. Analyse Results Resources Perform ance 6. Cloud Offers $
- 66. Highlights Deploying WfMSs to the Cloud Vincenzo Ferme 2 Deploying WfMSs to the Cloud WfMS IaaS D A B C P a a S Workload Users $ $ IT Resources sweet spot in the performance vs. resource consumption trade-off B P a a S Proposed Method Vincenzo Ferme 5 The Steps of the Method from performance requirements to cheapest VMs 1. Workload Mix 80% C A B 20% D 0 600 1200 0 1 2 3 4 5 6 7 8 9 10 2. Load Function Metrics KPIs 3. Performance Req. 4. Run Experiments5. Analyse Results Resources Perform ance 6. Cloud Offers $ Application to Camunda Vincenzo Ferme 23 7. VMs Selection select the set of suitable VMs, and identify the cheapest ones Normalised Cores Utilisation vs Price of VMs
- 67. Highlights Deploying WfMSs to the Cloud Vincenzo Ferme 2 Deploying WfMSs to the Cloud WfMS IaaS D A B C P a a S Workload Users $ $ IT Resources sweet spot in the performance vs. resource consumption trade-off B P a a S Proposed Method Vincenzo Ferme 5 The Steps of the Method from performance requirements to cheapest VMs 1. Workload Mix 80% C A B 20% D 0 600 1200 0 1 2 3 4 5 6 7 8 9 10 2. Load Function Metrics KPIs 3. Performance Req. 4. Run Experiments5. Analyse Results Resources Perform ance 6. Cloud Offers $ Application to Camunda Vincenzo Ferme 23 7. VMs Selection select the set of suitable VMs, and identify the cheapest ones Normalised Cores Utilisation vs Price of VMs Limitations and Future Work Vincenzo Ferme 24 Limitations and Future Work approximation of the actual performance on the Cloud Performance on the Cloud has more Variance
- 68. benchflow benchflow vincenzo.ferme@usi.ch http://benchflow.inf.usi.ch ESTIMATING THE COST FOR EXECUTING BUSINESS PROCESSES IN THE CLOUD Vincenzo Ferme, Ana Ivanchikj, Cesare Pautasso Faculty of Informatics USI Lugano, Switzerland
- 70. Vincenzo Ferme 29 Published Work [BTW ’15] C. Pautasso, V. Ferme, D. Roller, F. Leymann, and M. Skouradaki. Towards workflow benchmarking: Open research challenges. In Proc. of the 16th conference on Database Systems for Business,Technology, and Web, BTW 2015, pages 331–350, 2015. [SSP ’14] M. Skouradaki, D. H. Roller, F. Leymann, V. Ferme, and C. Pautasso. Technical open challenges on benchmarking workflow management systems. In Proc. of the 2014 Symposium on Software Performance, SSP 2014, pages 105–112, 2014. [ICPE ’15] M. Skouradaki, D. H. Roller, L. Frank, V. Ferme, and C. Pautasso. On the Road to Benchmarking BPMN 2.0 Workflow Engines. In Proc. of the 6th ACM/SPEC International Conference on Performance Engineering, ICPE ’15, pages 301–304, 2015.
- 71. Vincenzo Ferme 30 Published Work [CLOSER ’15] M. Skouradaki,V. Ferme, F. Leymann, C. Pautasso, and D. H. Roller. “BPELanon”: Protect business processes on the cloud. In Proc. of the 5th International Conference on Cloud Computing and Service Science, CLOSER 2015. SciTePress, 2015. [SOSE ’15] M. Skouradaki, K. Goerlach, M. Hahn, and F. Leymann. Application of Sub-Graph Isomorphism to Extract Reoccurring Structures from BPMN 2.0 Process Models. In Proc. of the 9th International IEEE Symposium on Service-Oriented System Engineering, SOSE 2015, 2015. [BPM ’15] V. Ferme, A. Ivanchikj, C. Pautasso. A Framework for Benchmarking BPMN 2.0 Workflow Management Systems. In Proc. of the 13th International Conference on Business Process Management, BPM ’15, pages 251-259, 2015.
- 72. Vincenzo Ferme 31 Published Work [BPMD ’15] A. Ivanchikj,V. Ferme, C. Pautasso. BPMeter: Web Service and Application for Static Analysis of BPMN 2.0 Collections. In Proc. of the 13th International Conference on Business Process Management [Demo], BPM ’15, pages 30-34, 2015. [ICPE ’16] V. Ferme, and C. Pautasso. Integrating Faban with Docker for Performance Benchmarking. In Proc. of the 7th ACM/SPEC International Conference on Performance Engineering, ICPE ’16, 2016. [CLOSER ’16] V. Ferme, A. Ivanchikj, C. Pautasso., M. Skouradaki, F. Leymann. A Container-centric Methodology for Benchmarking Workflow Management Systems. In Proc. of the 6th International Conference on Cloud Computing and Service Science, CLOSER 2016. SciTePress, 2016.
- 73. Vincenzo Ferme [CAiSE ’16] M. Skouradaki, V. Ferme, C. Pautasso, F. Leymann, A. van Hoorn. Micro-Benchmarking BPMN 2.0 Workflow Management Systems with Workflow Patterns . In Proc. of the 28th International Conference on Advanced Information Systems Engineering, CAiSE ’16, 2016. 32 Published Work [ICWE ’16] C. Jürgen,V. Ferme, H.C. Gall. Using Docker Containers to Improve Reproducibility in Software and Web Engineering Research. In Proc. of the 16th International Conference on Web Engineering, 2016. [ICWS ’16] M. Skouradaki,V.Andrikopoulos, F. Leymann. Representative BPMN 2.0 Process Model Generation from Recurring Structures. In Proc. of the 23rd IEEE International Conference on Web Services, ICWS '16, 2016.
- 74. Vincenzo Ferme 33 Published Work [SummerSOC ’16] M. Skouradaki, T. Azad, U. Breitenbücher, O. Kopp, F. Leymann. A Decision Support System for the Performance Benchmarking of Workflow Management Systems. In Proc. of the 10th Symposium and Summer School On Service-Oriented Computing, SummerSOC '16, 2016. [OTM ’16] M. Skouradaki, V. Andrikopoulos, O. Kopp, F. Leymann. RoSE: Reoccurring Structures Detection in BPMN 2.0 Process Models Collections. In Proc. of On the Move to Meaningful Internet Systems Conference, OTM '16, 2016. (to appear) [BPM Forum ’16] V. Ferme, A. Ivanchikj, C. Pautasso. Estimating the Cost for Executing Business Processes in the Cloud. In Proc. of the 14th International Conference on Business Process Management, BPM Forum ’16, 2016.
- 75. Vincenzo Ferme 34 Docker Performance [IBM ’14] W. Felter, A. Ferreira, R. Rajamony, and J. Rubio. An updated performance comparison of virtual machines and Linux containers. IBM Research Report, 2014. Although containers themselves have almost no overhead, Docker is not without performance gotchas. Docker volumes have noticeably better performance than files stored in AUFS. Docker’s NAT also introduces overhead for workloads with high packet rates. These features represent a tradeoff between ease of management and performance and should be considered on a case-by-case basis. ” “Our results show that containers result in equal or better performance than VMs in almost all cases. “ ” BenchFlow Configures Docker for Performance by Default
- 76. Vincenzo Ferme 35 •We want to characterise the Workload Mix using Real-World process models • Share your executable BPMN 2.0 process models, even anonymised Process Models •We want to characterise the Load Functions using Real-World behaviours • Share your execution logs, even anonymised Execution Logs •We want to add more and more WfMSs to the benchmark • Contact us for collaboration, and BenchFlow framework support WfMSs Call for Collaboration WfMSs, process models, process logs
- 77. Vincenzo Ferme 36 Cloud WfMSs, business process optimisations, capacity planning State of the Art of WfMSs in the Cloud • Cloud WfMSs: emerging market • BPaaS: started to be evaluated • Business Process Execution Optimisations • Cost-aware WfMSs
- 78. Vincenzo Ferme 36 Cloud WfMSs, business process optimisations, capacity planning State of the Art of WfMSs in the Cloud • Cloud WfMSs: emerging market • BPaaS: started to be evaluated • Business Process Execution Optimisations • Cost-aware WfMSs We Focus on Capacity Planning with a Widely Used WfMSs
- 79. Vincenzo Ferme 37 BenchFlow Framework system under test Docker Engine
- 80. Vincenzo Ferme 37 BenchFlow Framework system under test Docker Engine Containers
- 81. Vincenzo Ferme 37 BenchFlow Framework system under test Docker Engine Docker Machine provides Containers
- 82. Vincenzo Ferme 37 BenchFlow Framework system under test Docker Swarm Docker Engine Docker Machine provides Containers
- 83. Vincenzo Ferme 37 BenchFlow Framework system under test Docker Swarm Docker Engine Docker Machine provides manages Containers Servers
- 84. Vincenzo Ferme 37 BenchFlow Framework system under test Docker Swarm Docker Engine Docker Compose Docker Machine provides manages Containers Servers
- 85. Vincenzo Ferme 37 BenchFlow Framework system under test Docker Swarm Docker Engine Docker Compose Docker Machine provides manages Containers Servers SUT’s Deployment Conf. deploys
- 86. Vincenzo Ferme 38 Server-side Data and Metrics Collection WfMS Start Workflow asynchronous execution of workflows
- 87. Vincenzo Ferme Users D A B C Start Application Server Web Service Load Driver DBMS WfMS Instance Database 38 Server-side Data and Metrics Collection WfMS Start Workflow asynchronous execution of workflows
- 88. Vincenzo Ferme Users D A B C Start Application Server Web Service Load Driver DBMS WfMS Instance Database 38 Server-side Data and Metrics Collection WfMS Start Workflow End asynchronous execution of workflows
- 89. Vincenzo Ferme 39 Server-side Data and Metrics Collection monitors DBMSFaban Drivers ContainersServers harness WfMS TestExecution Web Service
- 90. Vincenzo Ferme 39 Server-side Data and Metrics Collection monitors DBMSFaban Drivers ContainersServers harness WfMS TestExecution Web Service MONITOR
- 91. Vincenzo Ferme 40 Server-side Data and Metrics Collection monitors DBMSFaban Drivers ContainersServers harness WfMS TestExecution Web Service • CPU usage • Database state Examples of Monitors:Monitors’ Characteristics: • RESTful services • Lightweight (written in Go) • As less invasive on the SUT as possible
- 92. Vincenzo Ferme 40 Server-side Data and Metrics Collection monitors DBMSFaban Drivers ContainersServers harness WfMS TestExecution Web Service CPU Monitor DB Monitor • CPU usage • Database state Examples of Monitors:Monitors’ Characteristics: • RESTful services • Lightweight (written in Go) • As less invasive on the SUT as possible
- 93. Vincenzo Ferme 41 Server-side Data and Metrics Collection collect data DBMSFaban Drivers ContainersServers harness WfMS TestExecution Web Service
- 94. Vincenzo Ferme 41 Server-side Data and Metrics Collection collect data DBMSFaban Drivers ContainersServers harness WfMS TestExecution Web Service Minio Instance Database Analyses COLLECTORS
- 95. Vincenzo Ferme 42 Server-side Data and Metrics Collection collect data DBMSFaban Drivers ContainersServers harness WfMS TestExecution Web Service • Container’s Stats (e.g., CPU usage) • Database dump • Applications Logs Examples of Collectors:Collectors’ Characteristics: • RESTful services • Lightweight (written in Go) • Two types: online and offline • Buffer data locally
- 96. Vincenzo Ferme 42 Server-side Data and Metrics Collection collect data DBMSFaban Drivers ContainersServers harness WfMS TestExecution Web Service Stats Collector DB Collector • Container’s Stats (e.g., CPU usage) • Database dump • Applications Logs Examples of Collectors:Collectors’ Characteristics: • RESTful services • Lightweight (written in Go) • Two types: online and offline • Buffer data locally