SAI - Serverless Integration Architectures - 09/2019
0 likes39 views
The document discusses serverless integration architectures, detailing various patterns and tools such as messaging, API gateways, and workflow adapters. It emphasizes the evolution from traditional enterprise integration methods to modern, serverless solutions that enhance scalability and efficiency. Key topics include event-driven paradigms, data interchange, governance, and potential use cases within this new integration landscape.
SAI - Serverless Integration Architectures - 09/2019
- 1. Serverless Integration Architectures b y S a m u e l Va n d e c a s t e e l e & S t e f a n D e Wa n d e l e i r 3. FaaS Tools and frameworks 7. (optional) iPaaS 2. Serverless Integration Patterns Messaging. API Gateway. Event Gateway. File transfer. Workflow. Adapters. 4. Conclusions 1. Introduction Enterprise Integration. Serverless. The new integration landscape. 6. (op.onal) Kubernetes ecosystem 5. (op.onal) Advanced use cases
- 2. E n t e r p r i s e I n t e g r a t i o n Ad hoc integration – PHASE 1 D i f f e r e n t d a t a m o d e l s D i f f e r e n t p r o t o c o l s D i f f e r e n t l i f e c y c l e D i f f e r e n t t i m i n g D e p e n d e n c y g r o w t h p r o b l e m s D a t a i n t e r c h a n g e s o l u t i o n s • D a t a i n t e r c h a n g e E n t e r p r i s e I n t e g r a t i o n Enterprise Integration Patterns (EIP) in a box – PHASE 2 I n c o r r e c t o w n e r s h i p L i m i t e d r e u s e p r o b l e m s C a n o n i c a l d a t a m o d e l A d a p t e r s O w n e r s h i p / G o v e r n a n c e s o l u t i o n s • D a t a i n t e r c h a n g e • T r a n s f o r m a t i o n • M e d i a t i o n • M e s s a g i n g
- 3. E n t e r p r i s e I n t e g r a t i o n Service Oriented Architecture – PHASE 3 S y s t e m f u n c t i o n s f o c u s p r o b l e m s O w n e r s h i p G o v e r n a n c e R e u s a b i l i t y s o l u t i o n s • D a t a i n t e r c h a n g e • Tr a n s f o r m a t i o n • M e d i a t i o n • M e s s a g i n g • R e u s a b l e S e r v i c e s • G o v e r n a n c e ( l a y e r i n g / l i f e c y c l e / p o l i c i e s ) • D o m a i n m o d e l E n t e r p r i s e I n t e g r a t i o n API Management – PHASE 4 B r o a d d o m a i n L e s s f o c u s o n g o v e r n a n c e p r o b l e m s O w n e r s h i p R e u s a b i l i t y C l i e n t - f o c u s s o l u t i o n s • D a t a i n t e r c h a n g e • Tr a n s f o r m a t i o n • M e d i a t i o n • M e s s a g i n g • R e u s a b l e S e r v i c e s • G o v e r n a n c e ( l a y e r i n g / l i f e c y c l e / p o l i c i e s ) • D o m a i n m o d e l • C l i e n t f o c u s
- 4. Enterprise Integration Data interchange Transformation Mediation Messaging Reusable Services Governance (layering/lifecycle/policies) Domain model Client focus Serverless is still looking for a definition that the industry as a whole can agree on ! b u t i t ’ s a b o u t h a n d i n g o v e r r e s p o n s i b i l i t i e s
- 6. Serverless is a new abstraction layer No server/container management Automatic scaling & high available Micro billing per usage Event-driven E v e n t - d r i v e n , S t a t e l e s s , E p h e m e r a l c o m p u t e M e s s a g i n g D a t a b a s e s O b j e c t s t o r e … Serverless = FaaS + BaaS
- 7. Slide from
- 8. 3. FaaS Tools and frameworks 7. (optional) iPaaS 2. Serverless Integration Patterns Messaging. API Gateway. Event Gateway. File transfer. Workflow. Adapters. 4. Conclusions 1. Introduction Enterprise Integration. Serverless. The new integration landscape. 6. (optional) Kubernetes ecosystem 5. (op.onal) Advanced use cases
- 9. D e c o u p l i n g s y s t e m s MESSAGING A n “ h o n e s t ” a r c h i t e c t u ra l s ty l e t h a t d o e s n o t t r y t o d e ny t h e l i m i t a t i o n s o f t h e u n d e r l y i n g m e d i u m - G r e g o r H o h p e E n t e r p r i s e I n t e g r a t i o n Pa t t e r n Messaging by providing a bridge across systems locaRons (cloud & on premises) technologies Rme Load balancing Queuing Circuit breaking by providing Decoupling systems by reversing dependencies
- 10. A message is a packet of data. An event is a message that notifies about a change or an action that has taken place. Classic Messaging bus Cloud Event Router Distributed Commit Log The ‘kafka’ model
- 11. Classic Messaging bus Cloud Event Router Distributed Commit Log The ‘kafka’ model
- 12. M e s s a g i n g b u s Typical async integration flow Canonical data model Content enrichment Message translation Pub-sub channel Fan-out Event-driven consumers Queue channel Polling consumer M e s s a g i n g b u s Example Queue based load levelling
- 13. M e s s a g i n g b u s Example Request/Response messaging Classic Messaging bus Cloud Event Router Distributed Commit Log The ‘kafka’ model
- 14. M e s s a g i n g The Distributed Commit Log ( a k a t h e K a f k a m o d e l ) “Dump pipes, smart endpoints” Classic Messaging bus Cloud Event Router Distributed Commit Log The ‘kafka’ model
- 15. Message Translation Message Routing Content-based routing Message Filter Error handling Retry policies Dead letter queue Publish-Subscribe channel Event-driven consumer * Out of the box SaaS Adapters R M e s s a g i n g The ‘new’ Cloud Event Router hAps://docs.microsoB.com/en-us/azure/event-grid/overview RT E T E M e s s a g i n g The ‘new’ Cloud Event Router
- 16. M e s s a g i n g The ‘new’ Cloud Event Router for SaaS Integration https://cloudevents.io/ https://github.com/cloudevents/spec Specification for describing event data in common formats to provide interoperability across services, platforms and systems
- 17. E x p o s i n g f u n c t i o n a l i t y a n d d a t a i n a s e c u r e a n d s c a l a b l e w a y API Gateway API AS A PRODUCT PARTNER INTEGRATION UNLOCK DATA
- 18. A P I G a t e w a y p a t t e r n API Management “API Management is the process of publishing, promoting and overseeing APIs in a secure and scalable environment” Provision Publish API Management A P I G a t e w a y p a t t e r n API Management - toolset API Gateway Security Transform Quota Protocol Switching Caching Metrics Collection Analy<cs API Endpoints µService Collect APP Java AS Creates Registers Uses Invokes Manages Manage User Developer API Manager API Portal API Manager fn(..)
- 20. APIGatewaypattern MicroserviceArchitectureExample externalizing Security Quality of Service Monitoring Billing
- 21. C o m b i n i n g A P I M a n a g e m e n t w i t h t h e e v e n t d r i v e n w o r l d Event Gateway A s y n c A P I M a n a g e m e n t The sync layer
- 22. A s y n c A P I M a n a g e m e n t Event ‘AsyncAPI’ Management M e s s a g i n g The ‘new’ Cloud Event Router for SaaS Integration
- 23. https://cloudevents.io/ https://github.com/cloudevents/spec Specification for describing event data in common formats to provide interoperability across services, platforms and systems The new industry standard for defining asynchronous APIs?
- 24. Event Portal Documentation Registration Analysis Community Event Broker Security Persistence Protocol TranslaRon RouRng
- 25. S i m p l e & w i d e l y u s e d File based Integration Batches are scheduled at night when impact on ESB performance is ‘allowed’ ESB relies on polling to receive new files
- 26. Event enabled serverless object store A W S e x a m p l e u s i n g S 3 , Tr a n s f e r f o r S F T P a n d S N S
- 27. = > Eve nt d r i ve n p ro c e s s i n g of inco ming f iles = > S e r v e r l e s s b e n e f i t s ( s t o r a g e , s c a l e , o p s , … ) = > F a s t e r d a t a s c h e m a v a l i d a t i o n f e e d b a c k Workflows L o n g r u n n i n g / A s y n c / S t a t e H u m a n i n t e r a c t i o n
- 28. Workflow - pattern 72 w o r k f l o w Example - Azure Logic Apps
- 29. Adapters C i t i z e n d e v e l o p e r s L i f e c y c l e m a n a g e m e n t Abstraction of functionalities => Less in-depth knowledge needed => Target for citizen developers (?)
- 30. Management of target lifecycle => Less in-depth knowledge needed => No more LC management needed (?) 3. FaaS Tools and frameworks 7. (optional) iPaaS 2. Serverless Integration Patterns Messaging. API Gateway. Event Gateway. File transfer. Workflow. Adapters. 4. Conclusions 1. Introduction Enterprise Integration. Serverless. The new integration landscape. 6. (optional) Kubernetes ecosystem 5. (optional) Advanced use cases
- 31. SERVERLESS LANGUAGE LANDSCAPE h t t p s : / / s e r v e r l e s s . c o m / b l o g / s e r v e r l e s s - b y - t h e - n u m b e r s - 2 0 1 8 - d a t a - r e p o r t / M i d d l e w a r e f r a m e w o r k s Middly
- 32. M i d d l e w a r e f r a m e w o r k s TIBCO Flogo http://www.flogo.io/ https://integration.cloud.tibco.com/docs/flogo_connectors/index.html
- 33. Disable outbound internet connectivity Disable read/write on the /tmp/ directory Disable child process execuQon Disable read access to the function's handler and prevent source code leakage FunctionShield Hybrid Cloud Architecture ( 1 ) V P N / D i r e c t C o n n e c t i o n N E T W O R K L E V E L A P P L I C A T I O N L E V E L ( 2 ) O n - p r e m A p p l i c a t i o n i n i t i a t e s ( 3 ) C l o u d a g e n t s o n - p r e m i n i t i a t e s A P I L E V E L ( 4 ) E x p o s i n g A P I s o n - p r e m ( M a t u r e o n - p r e m A P I M a n a g e m e n t e n v i r o n m e n t )
- 34. 3. FaaS Tools and frameworks 7. (optional) iPaaS 2. Serverless Integration Patterns Messaging. API Gateway. Event Gateway. File transfer. Workflow. Adapters. 4. Conclusions 1. Introduction Enterprise Integration. Serverless. The new integration landscape. 6. (op.onal) Kubernetes ecosystem 5. (op.onal) Advanced use cases P r e v i o u s l y w e h a d a n E S B c o n t a i n i n g m e s s a g i n g , s t o r a g e , c a c h i n g , a p i g a t e w a y, s c h e d u l i n g , a d a p t e r s , m a p p i n g s , … Almost all capabilities and many more are today available using Serverless Integration Services
- 35. F r o m E S B t o C l o u d - n a t i v e i n t e g r a t i o n The composite integration application https://www.integration-playbook.io/docs/composite-integration-applications F r o m E S B t o C l o u d - n a t i v e i n t e g r a t i o n The composite integration application • Architect without compromises • Complete set of integraRon capabiliRes • Event driven paradigm • iPaaS capabiliRes are being incorporated
- 36. 3. FaaS Tools and frameworks 7. (optional) iPaaS 2. Serverless Integration Patterns Messaging. API Gateway. Event Gateway. File transfer. Workflow. Adapters. 4. Conclusions 1. Introduction Enterprise Integration. Serverless. The new integration landscape. 6. (optional) Kubernetes ecosystem 5. (optional) Advanced use cases
- 38. I n t e g r a t i o n A r c h i t e c t u r e Service hub Governance exposed data Backend decoupling Leverage serverless capabilities “Fit for purpose”-data Extra functionality 3. FaaS Tools and frameworks 7. (op.onal) iPaaS 2. Serverless IntegraRon Pacerns Messaging. API Gateway. Event Gateway. File transfer. Workflow. Adapters. 4. Conclusions 1. Introduction Enterprise Integration. Serverless. The new integration landscape. 6. (optional) Kubernetes ecosystem 5. (optional) Advanced use cases