![A workflow starter simply kicks off a workflow with some input via API
1
BEDCon 2016 | @KayLerch
{ "childPolicy": "string", "domain": "string", "executionStartToCloseTimeout": "string", "input": "string", "lambdaRole": "string", "tagList": [
"string" ], "taskList": { "name": "string" }, "taskPriority": "string", "taskStartToCloseTimeout": "string", "workflowId": "string", "workflowType":
{ "name": "string", "version": "string" } }
{ "runId": "string" }](https://image.slidesharecdn.com/20160916bedcon-tauscheesbgegenagileintegrationslsungindercloud-160916083455/85/BEDCon-2016-Kay-Lerch-on-Will-trade-an-ESB-for-an-agile-integration-solution-in-the-cloud-27-320.jpg)
![A decider owns the actual workflow logic. It is an application polling for
tasks via API and returns decisions to the SWF engine
3
BEDCon 2016 | @KayLerch
{ "domain": "string", "identity": "string", "maximumPageSize": number, "nextPageToken": "string", "reverseOrder": boolean, "taskList": { "name":
"string" } }
{ "decisions": [ { "cancelTimerDecisionAttributes": { … }, "cancelWorkflowExecutionDecisionAttributes": { … },
"completeWorkflowExecutionDecisionAttributes": { … }, "continueAsNewWorkflowExecutionDecisionAttributes": { … }, "recordMarkerDecisionAttributes": {
… }, "requestCancelActivityTaskDecisionAttributes": { … }, "requestCancelExternalWorkflowExecutionDecisionAttributes": { … },
"scheduleActivityTaskDecisionAttributes": { … }, "scheduleLambdaFunctionDecisionAttributes": { … },
"signalExternalWorkflowExecutionDecisionAttributes": { … }, "startChildWorkflowExecutionDecisionAttributes": { … }, "startTimerDecisionAttributes":
{ … } } ], "executionContext": "string", "taskToken": "string" }](https://image.slidesharecdn.com/20160916bedcon-tauscheesbgegenagileintegrationslsungindercloud-160916083455/85/BEDCon-2016-Kay-Lerch-on-Will-trade-an-ESB-for-an-agile-integration-solution-in-the-cloud-29-320.jpg)
BEDCon 2016 - Kay Lerch on "Will trade an ESB for an agile integration solution in the cloud"
- 1. Will trade an ESB for an agile Integration solution in the Cloud Berlin Expert Days, 16. September 2016, Urania Berlin @KayLerch | Engineering Manager | Immobilien Scout
- 2. Agenda BEDCon 2016 | @KayLerch Disruptive forces and what they do with enterprise IT An ideal integration platform AWS Simple Workflows (SWF) in a nutshell Demo time Leveraging SWF to get rid of a classical ESB solution Reclaim process ownership and end-2-end-autonomy Drawing the big picture of a hybrid integration solution
- 3. Competitive forces and what they do with Enterprise IT Competitive forcesMarket IT of an E-commerce business Fast-paced feature development (“Lean IT”) Core business systems and services (“Classical IT”) Innovation / Agility / Speed Agile teams Microservices / modularized DevOps Efficiency / compliance / reliability Project organization Legacy / monolithic Operations management Release-driven Deployments API-led Connectivity Proprietary Interfaces Boosting time to market CLD Continuous DeploymentProduct BEDCon 2016 | @KayLerch
- 4. “Two speed IT” parts heavily depend on each other not only for the good Competitive forcesMarket A digital business Fast-paced feature development (“Lean IT”) Core business systems and services (“Classical IT”) Innovation / Agility / Speed Agile teams Microservices / modularized DevOps Continuous Deployment Efficiency / compliance / reliability Project organization Legacy / monolithic Operations management Release-driven Deployments API-led Connectivity Proprietary Interfaces Boosting time to market CLD Organizational clash Technical clash „Two Speed IT“ Product BEDCon 2016 | @KayLerch
- 5. Overcome technical burden with Integration strategy (SOA?) Competitive forcesMarket A digital business Fast-paced feature development (“Lean IT”) Core business systems and services (“Classical IT”) Innovation / Agility / Speed Agile teams Microservices / modularized DevOps Continuous Deployment Efficiency / compliance / reliability Project organization Legacy / monolithic Operations management Release-driven Deployments API-led Connectivity Proprietary Interfaces Boosting time to market CLD „Two Speed IT“ Product BEDCon 2016 | @KayLerch Organizational clash Technical clash ICC ESB
- 6. Well, not … Integration projects tend to be traditional (often) for good reason Competitive forcesMarket A digital business Fast-paced feature development (“Lean IT”) Core business systems and services (“Classical IT”) Innovation / Agility / Speed Agile teams Microservices / modularized DevOps Continuous Deployment Efficiency / compliance / reliability Project organization Legacy / monolithic Operations management Release-driven Deployments API-led Connectivity Proprietary Interfaces Boosting time to market CLD „Two Speed IT“ Product BEDCon 2016 | @KayLerch Organizational clash Technical clash ICC ESB
- 7. An integration strategy should follow a “bimodal” approach where so- called hybrid integration platforms (HIP) strive for Self Service Integration Competitive forcesMarket A digital business Fast-paced feature development (“Lean IT”) Core business systems and services (“Classical IT”) Innovation / Agility / Speed Agile teams Microservices / modularized DevOps Continuous Deployment Efficiency / compliance / reliability Project organization Legacy / monolithic Operations management Release-driven Deployments API-led Connectivity Proprietary Interfaces Boosting time to market CLD „Two Speed IT“ Product ISG HIP BEDCon 2016 | @KayLerch
- 8. The digital ecosystem brings a lot of new potential but also threat Mobile IoT User Data Cloud Social Networks Digital marketplace Digital ecosystem discruptive BEDCon 2016 | @KayLerch
- 9. Businesses need to adopt and integrate these potential Mobile IoT User Data Cloud Social Networks Digital marketplace Digital ecosystem discruptive BEDCon 2016 | @KayLerch
- 10. An HIP got its name from being the gateway for those new stuff Mobile IoT User Data Cloud Social Networks Digital marketplace Digital ecosystem discruptive BEDCon 2016 | @KayLerch
- 11. HIP is a concept whereas iPaaS, iSaaS are (commercial) solutions in the cloud Mobile IoT User Data Cloud Social Networks Digital marketplace Digital ecosystem discruptive iPaaS iSaaS BEDCon 2016 | @KayLerch
- 12. Ok, you got your BINGO! Let’s move on in the real world … BEDCon 2016 | @KayLerch
- 13. Agenda BEDCon 2016 | @KayLerch Disruptive forces and what they do with enterprise IT An ideal integration platform AWS Simple Workflows (SWF) in a nutshell Demo time Leveraging SWF to get rid of a classical ESB solution Reclaim process ownership and end-2-end-autonomy Drawing the big picture of a hybrid integration solution
- 14. Given an ESB in charge of syncing business data across the enterprise ESB SINA Salesforce CRM Sales CC ERP AMGFinance Checkout Service ARTE Fraud Service SHIELD Customer Contract User Trust Open Payment Payment Profile Article Customer Contract User Trust Open Payment Payment Profile Article Performance Rec User Trust Customer Contract Article BEDCon 2016 | @KayLerch
- 15. This centralized hub is key for processes arching over multiple domains ESB SINA Salesforce CRM Sales CC ERP AMGFinance Checkout Service ARTE Fraud Service SHIELD Customer Contract User Trust Open Payment Payment Profile Article Customer Contract User Trust Open Payment Payment Profile Article Performance Rec User Trust Customer Contract Article BEDCon 2016 | @KayLerch
- 16. It encapsulates the burden of integrating system interfaces with brokers ESB SINA Salesforce CRM Sales CC ERP AMGFinance Checkout Service ARTE Fraud Service SHIELD Customer Contract User Trust Open Payment Payment Profile Article Customer Contract User Trust Open Payment Payment Profile Article Performance Rec User Trust Customer Contract Article { } { } { } { } BEDCon 2016 | @KayLerch
- 17. Teams depend on those brokers in order to change their interfaces ESB SINA Salesforce CRM Sales CC ERP AMGFinance Checkout Service ARTE Fraud Service SHIELD Customer Contract User Trust Open Payment Payment Profile Article Customer Contract User Trust Open Payment Payment Profile Article Performance Rec User Trust Customer Contract Article { } { } { } { } API API API API BEDCon 2016 | @KayLerch
- 18. It would be great to push responsibility (integration logic) to the edges ESB SINA Salesforce CRM Sales CC ERP AMGFinance Checkout Service ARTE Fraud Service SHIELD Customer Contract User Trust Open Payment Payment Profile Article Customer Contract User Trust Open Payment Payment Profile Article Performance Rec User Trust Customer Contract Article API API API API { } { } { } { } BEDCon 2016 | @KayLerch
- 19. It would also be great to delegate process ownership (orchestration logic) ESB SINA Salesforce CRM Sales CC ERP AMGFinance Checkout Service ARTE Fraud Service SHIELD Customer Contract User Trust Open Payment Payment Profile Article Customer Contract User Trust Open Payment Payment Profile Article Performance Rec User Trust Customer Contract Article API API API API { } { } { } { } BEDCon 2016 | @KayLerch
- 20. ESB left over for “dirty work” (messaging, tracking, governance, …) ESB SINA Salesforce CRM Sales CC ERP AMGFinance Checkout Service ARTE Fraud Service SHIELD Customer Contract User Trust Open Payment Payment Profile Article Customer Contract User Trust Open Payment Payment Profile Article Performance Rec User Trust Customer Contract Article API API API API { } { } { } { } BEDCon 2016 | @KayLerch
- 21. It should also be in the cloud to not be isolated from the digital ecosystem ESB SINA Salesforce CRM Sales CC ERP AMGFinance Checkout Service ARTE Fraud Service SHIELD Customer Contract User Trust Open Payment Payment Profile Article Customer Contract User Trust Open Payment Payment Profile Article Performance Rec User Trust Customer Contract Article API API API API { } { } { } { } BEDCon 2016 | @KayLerch
- 22. It should also be in the cloud to not be isolated from the digital ecosystem ESB SINA Salesforce CRM Sales CC ERP AMGFinance Checkout Service ARTE Fraud Service SHIELD Customer Contract User Trust Open Payment Payment Profile Article Customer Contract User Trust Open Payment Payment Profile Article Performance Rec User Trust Customer Contract Article API API API API { } { } { } { } BEDCon 2016 | @KayLerch
- 23. Why can’t this thing just be like an HIP? HIP SINA Salesforce CRM Sales CC ERP AMGFinance Checkout Service ARTE Fraud Service SHIELD Customer Contract User Trust Open Payment Payment Profile Article Customer Contract User Trust Open Payment Payment Profile Article Performance Rec User Trust Customer Contract Article API API API API { } { } { } { } BEDCon 2016 | @KayLerch
- 24. As our company moves to AWS anyway, let’s give SWF a try • There is PAYG • AWS ecosystem is huge, it serves all the hip stuff like serverless architecture (Lambda, API Gateway), IoT, Mobile integration, Messaging, Elastic computing, Container deployments and more • There‘s a big community around AWS • SWF (Simple Workflows) is used by NASA for processing data from the Mars-Rover on earth You got me at „Mars“… BEDCon 2016 | @KayLerch
- 25. Agenda BEDCon 2016 | @KayLerch Disruptive forces and what they do with enterprise IT An ideal integration platform AWS Simple Workflows (SWF) in a nutshell Demo time Leveraging SWF to get rid of a classical ESB solution Reclaim process ownership and end-2-end-autonomy
- 26. SWF is a workflow engine scheduling tasks for all the workflow participants BEDCon 2016 | @KayLerch
- 27. A workflow starter simply kicks off a workflow with some input via API 1 BEDCon 2016 | @KayLerch { "childPolicy": "string", "domain": "string", "executionStartToCloseTimeout": "string", "input": "string", "lambdaRole": "string", "tagList": [ "string" ], "taskList": { "name": "string" }, "taskPriority": "string", "taskStartToCloseTimeout": "string", "workflowId": "string", "workflowType": { "name": "string", "version": "string" } } { "runId": "string" }
- 28. SWF has no clue what comes next so it schedules a “decision task” 2 BEDCon 2016 | @KayLerch
- 29. A decider owns the actual workflow logic. It is an application polling for tasks via API and returns decisions to the SWF engine 3 BEDCon 2016 | @KayLerch { "domain": "string", "identity": "string", "maximumPageSize": number, "nextPageToken": "string", "reverseOrder": boolean, "taskList": { "name": "string" } } { "decisions": [ { "cancelTimerDecisionAttributes": { … }, "cancelWorkflowExecutionDecisionAttributes": { … }, "completeWorkflowExecutionDecisionAttributes": { … }, "continueAsNewWorkflowExecutionDecisionAttributes": { … }, "recordMarkerDecisionAttributes": { … }, "requestCancelActivityTaskDecisionAttributes": { … }, "requestCancelExternalWorkflowExecutionDecisionAttributes": { … }, "scheduleActivityTaskDecisionAttributes": { … }, "scheduleLambdaFunctionDecisionAttributes": { … }, "signalExternalWorkflowExecutionDecisionAttributes": { … }, "startChildWorkflowExecutionDecisionAttributes": { … }, "startTimerDecisionAttributes": { … } } ], "executionContext": "string", "taskToken": "string" }
- 30. SWF fulfills the remotely given decisions –> e.g. it schedules an activity task 4 BEDCon 2016 | @KayLerch
- 31. An activity worker polls for tasks from a task list, works on it and returns a result 5 BEDCon 2016 | @KayLerch { "domain": "string", "identity": "string", "taskList": { "name": "string" } } { "result": "string", "taskToken": "string" } { "details": "string", "reason": "string", "taskToken": "string" }
- 32. SWF receives the result – again has no clue how to go on – so it schedules another decision task 6 BEDCon 2016 | @KayLerch
- 33. The decider receives the task. It now contains a detailed protocol of what happened in this workflow-execution. Based on that it gives the next decision(s) 7 BEDCon 2016 | @KayLerch
- 34. SWF schedules the next task – according to the last decision by the decider. Activity workers can be serverless Lambda-functions as well 8 BEDCon 2016 | @KayLerch
- 35. This goes on and on until the decider aka workflow worker decides for completing the workflow. What again is executed by SWF itself. BEDCon 2016 | @KayLerch
- 36. You keep track of your workflow executions in AWS console BEDCon 2016 | @KayLerch
- 37. AWS CloudWatch provides performance metrics for your workflows BEDCon 2016 | @KayLerch
- 38. Agenda BEDCon 2016 | @KayLerch Disruptive forces and what they do with enterprise IT An ideal integration platform AWS Simple Workflows (SWF) in a nutshell Demo time Leveraging SWF to get rid of a classical ESB solution Reclaim process ownership and end-2-end-autonomy Drawing the big picture of a hybrid integration solution
- 39. SWF-powered IoT-Photobooth BEDCon 2016 | @KayLerch 2) Shoot a photo View progress on breadboard 3) View on Slack 1) Press a button
- 40. AWS Lambda AWS SWF Lambda Task Lambda Upload to S3 & Start Workflow Lambda Update Shadow Lambda Task Lambda Get S3 Photo and Post Message to Slack Lambda Task Lambda Task Lambda Task Lambda Shorten Url with Bitly Lambda Update Shadow Lambda Update Shadow AWS EC2 WorkflowWorker DeciderImpl Raspberry Pi AWS IoT Thing Shadow Python Script GPIO Rule AWS S3 Bucket PutFile StartWFInvoke MQTT Publish MQTT Receive Update Shadow Poll Tasks Shoot Impulse Impulse
- 41. Agenda BEDCon 2016 | @KayLerch Disruptive forces and what they do with enterprise IT An ideal integration platform AWS Simple Workflows (SWF) in a nutshell Demo time Leveraging SWF to get rid of a classical ESB solution Reclaim process ownership and end-2-end-autonomy Drawing the big picture of a hybrid integration solution
- 42. So let’s use SWF for replacing the ESB in our company. ESB SINA Salesforce CRM Sales CC ERP AMGFinance Checkout Service ARTE Fraud Service SHIELD Customer Contract User Trust Open Payment Payment Profile Article Customer Contract User Trust Open Payment Payment Profile Article Performance Rec User Trust Customer Contract Article { } { } { } { } API API API API BEDCon 2016 | @KayLerch
- 43. Client-side worker integrate their services by pulling tasks from SWF ESB SINA Salesforce CRM Sales CC ERP AMGFinance Checkout Service ARTE Fraud Service SHIELD Customer Contract User Trust Open Payment Payment Profile Article Customer Contract User Trust Open Payment Payment Profile Article Performance Rec User Trust Customer Contract Article API API API API wor ker wor ker wor ker wor ker Acti vity Acti vity Acti vityActi vity AWS SWF BEDCon 2016 | @KayLerch
- 44. Workers poll for tasks in task lists associated with declarative activity type ESB SINA Salesforce CRM Sales CC ERP AMGFinance Checkout Service ARTE Fraud Service SHIELD Customer Contract User Trust Open Payment Payment Profile Article Customer Contract User Trust Open Payment Payment Profile Article Performance Rec User Trust Customer Contract Article API API API API Acti vity Acti vity Acti vityActi vity Task list Task list Task list Task list HTTP REST HTTP REST HTTP REST HTTP REST AWS SWF wor ker wor ker wor ker wor ker BEDCon 2016 | @KayLerch
- 45. Workers can be Lambda functions as well in charge of the edge team ESB SINA Salesforce CRM Sales CC ERP AMGFinance Checkout Service ARTE Fraud Service SHIELD Customer Contract User Trust Open Payment Payment Profile Article Customer Contract User Trust Open Payment Payment Profile Article Performance Rec User Trust Customer Contract Article API API Acti vity Acti vity Acti vityActi vity Task list Task list HTTP REST HTTP REST AWS SWF wor ker wor ker lam bda AWS lam bda AWS Event Trigger Event Trigger BEDCon 2016 | @KayLerch
- 46. That’s nice, but there’s still a central orchestration controlled by one decider ESB SINA Salesforce CRM Sales CC ERP AMGFinance Checkout Service ARTE Fraud Service SHIELD Customer Contract User Trust Open Payment Payment Profile Article Customer Contract User Trust Open Payment Payment Profile Article Performance Rec User Trust Customer Contract Article API API Acti vity Acti vity Acti vityActi vity Task list Task list HTTP REST HTTP REST AWS SWF wor ker wor ker lam bda AWS lam bda AWS Event Trigger Event Trigger BEDCon 2016 | @KayLerch
- 47. Delegate parts of the orchestration in child workflows reclaims process ownership SINA Salesforce CRM Sales CC ERP AMGFinance Checkout Service ARTE Fraud Service SHIELD Customer Contract User Trust Open Payment Payment Profile Article Customer Contract User Trust Open Payment Payment Profile Article Performance Rec User Trust Customer Contract Article API API HTTP REST AWS SWF wor ker lam bda AWS Event TriggerEvent Trigger HTTP REST AWS Acti vity Task list wor ker Sub WF Acti vity BEDCon 2016 | @KayLerch Task list
- 48. Agenda BEDCon 2016 | @KayLerch Disruptive forces and what they do with enterprise IT An ideal integration platform AWS Simple Workflows (SWF) in a nutshell Demo time Leveraging SWF to get rid of a classical ESB solution Reclaim process ownership and end-2-end-autonomy Drawing the big picture of a hybrid integration solution
- 49. Reclaim process ownership leads to real end-2-end-autonomy of teams Sales BEDCon 2016 | @KayLerch
- 50. End-2-End autonomy along the whole lifecycle of a worker Sales BEDCon 2016 | @KayLerch
- 51. We call them Microworker as this is what they really are Sales BEDCon 2016 | @KayLerch
- 52. Agenda BEDCon 2016 | @KayLerch Disruptive forces and what they do with enterprise IT An ideal integration platform AWS Simple Workflows (SWF) in a nutshell Demo time Leveraging SWF to get rid of a classical ESB solution Reclaim process ownership and end-2-end-autonomy Drawing the big picture of a hybrid integration solution
- 56. Q & A BEDCon 2016 | @KayLerch It‘s time to ask questions and give feedback. Thanks for joining my session ...
- 57. Backup (SWF at JPL Datacenter of NASA) Sales BEDCon 2016 | @KayLerch
- 58. Backup (SWF-powered processing of images from Mars rover) Sales BEDCon 2016 | @KayLerch
- 59. Backup (SWF-powered contract data distribution at Scout24) Sales BEDCon 2016 | @KayLerch
- 60. Backup (SWF-powered contract data distribution at Scout24) Sales BEDCon 2016 | @KayLerch
Editor's Notes
- #4: Porter Vormachtstellung Lieferanten Vormachtstellung Kunden Neue Produkte (Ersatzprodukte) Konkurrenzkampf Neue Marktteilnehmer Klassische IT (Basis für das Kerngeschäft, oft Backend / Legacy) Wettbewerbsfähigkeit über schnelle Produkteinführung Verbraucherorienierte Produktentwicklung
- #5: Two Speed IT Suggeriert Gut-gegen-Böse Daseinsberechtigung beider „Fail Fast“-Kultur nicht überall anwendbar Besonders in großen Unternehmen Umfrage: Wer kennt das? In 2017 75% aller Unternehmen mit 2-Speed IT (Gartner) Ziel: Entkopplung von „customer-facing“ product dev und Legacy IT VW Digital Labs E-Post Gedankliche Hilfe (Frontend / Backend)
- #6: ICC = Integration Compentency Center ESB = Enterprise Service Bus SOA = Service orientierte Architektur Machte den Fehler sich im Core-Business breitmachen zu wollen Organisatorisches Umfeld passte nicht Die Entkopplung von Services braucht End-2-End-Autonomie / Verantwortlichkeit Microservices / DevOps / Container / CI / CD bringen all das „Microservices is SOA done right“
- #7: ESB / SOA Projekte nur ein „Wurmfortsatz“ der klassischen IT Erfahrungsbericht bei Scout
- #8: Bimodal: Mode 1: rechts Mode 2: links Streben nach Self Service Integration Integration Specialists Ad-Hoc Integrators (Integration Platform aas) Citizen Integrators (Integration Software assS) ISG ist interdisziplinär (mehr was für Konzerne) HIP nur ein Sammelbegriff für eine Integrationsplatform, die beide Sphären bedient
- #9: Durch Adaption neuer Technologien: Wettbewerbsvorteil mit zuweilen disruptiven Markteintritten Netflix vs. Blockbuster Amazon vs. Einzelhandel Apple vs. Nokia, Motorola … Digitale Transformation bedeutet Ausrichtung des gesamten Business (IT) auf diese Technologien
- #10: Sustaining (erhaltene) Innovation Disruptive Innovation