Upleveling Brownfield Integration
- 1. Associate Technical Lead, WSO2 Up-leveling Brownfield Integration Asanka Abeyweera
- 2. ● What is Brownfield Integration? ● Why do we need Brownfield Integration? ● What are the common problems we face in doing Brownfield Integration? ● How can you do Brownfield Integration with WSO2 Enterprise Integrator? Agenda
- 3. Greenfield Project In software development, a greenfield project could be one of developing a system for a totally new environment, without concern for integrating with other systems. - Wikipedia
- 4. Brownfield Project https://www.designingbuildings.co.uk/wiki/Building_an_extension
- 5. Why is “Brownfield” Integration Important ? ● Cost of replacing/rewriting ● Reuse of existing systems ● Cutdown the time it takes for digital transformation ● Not all existing systems are replaceable
- 6. Types of Systems in Brownfield ● Systems based on standard protocols ● Systems based on non standard or proprietary protocols ● File based systems ● Systems with very limited integration points ○ Databases SQL Computer System Input Output Storage
- 7. Key Functionality Requirements ● Transformation ● Routing ● Orchestration ● Protocol Switching ● Parallel Processing
- 9. Routing New System Brown-Field Existing System A Existing System B
- 10. Orchestration New System Brown-Field Existing System A Existing System B Existing System C 1 2 3
- 11. Protocol Switching New System Brown-Field Existing System HTTP JMS
- 12. Parallel Processing New System Brown-Field Existing System A Existing System B
- 16. WSO2 Enterprise Integrator Single integrated packaging of ● ESB and DSS as a single runtime ● Business Process Server ● Message Broker ● EI Analytics (primarily ESB analytics) ● Micro-Integrator (New!)
- 17. Service Integration - Integrator Runtime A lightweight, high performance service bus ● Comprehensive REST, SOAP, and WS-* support ● SAP, FIX, and HL7 - Domain specific solutions ● Configuration driven ● Extensible and scalable ● 100% coverage of all EIPs
- 18. Data Integration - Integrator Runtime Building service abstractions on top of disparate data sources SQL SOAP/REST/OData XML/JSON
- 19. Business Processes - BPS Runtime ● Define and execute business processes - WS-BPEL 2.0, BPEL4WS 1.1, BPMN 2.0, long running stateful processes ● Define workflows interacting with people - BPMN user tasks, WS-HumanTask 1.1, BPEL4People 1.1 ● Graphical process modeling
- 20. Message broker runtime ● Lightweight, easy-to-use, distributed message-brokering server ● Used to connect, persist and reliably distribute data and event information generated from multiple systems and applications ● Supports JMS and AMQP ● RDBMS for persistence
- 21. Micro-Integrator ● Startup time reduced ● Distribution size reduced ● Supports all key features ● Removed features that are not useful for MSA ● Container-friendly
- 22. Integrator vs Micro-Integrator Profile ESB MicroESB Startup Time 40s 5s Distribution size 814 MB 379 MB Mediation (ESB) Features ✅ ✅ Data Integration Features ✅ ✅ Built-in Clustering ✅ ❌ Tooling ✅ ✅ Management Console ✅ ❌ Dep-Sync ✅ ❌ Multitenancy ✅ ❌ CAR Deployment ✅ ✅ Hot deployment ✅ ❌ Micro-IntegratorIntegrator
- 24. Implementation of Key Functionality Requirements ● Transformation ● Routing ● Orchestration ● Protocol Switching
- 25. Transformation New System Brown-Field Existing System ● Ability to transform any format to any format ● Data mapping ● Enrich payload ● XSLT Data Mapper, Enrich, Payload Factory, XSLT Mediators
- 26. Transformation - Visual Data Mapping
- 27. Routing Based on ● Content ● Headers ● A property of the message ● Environment Routing New System Brown-Field Existing System A Existing System B Switch, Filter Mediators
- 28. Orchestration New System Brown-Field Existing System A Existing System B Existing System C 1 2 3 Synchronous service invocations with underlying Asynchronous non-blocking thread model Call Mediator
- 29. Protocol Switching New System Brown-Field Existing SystemHTTP JMS Receive over any protocol and deliver over any protocol
- 30. Brownfield Integration with Different Types of Systems ● Integration with systems that talk in standard protocols ● Integration with systems that talk in non-standard/ proprietary protocols ● Integration with file-based systems ● Integration with systems that use databases
- 31. Integration with Systems that Talk in Standard Protocols Support for a wide range of standard protocols ● HTTP/HTTPS ● JMS ● AMQP ● FTP/SFTP/FTPS/SMB ● SMTP/IMAP ● MQTT ● KAFKA ● And many more SMTP HTTP JMS FTP Websocket
- 32. Integration with Systems that Talk in Standard Protocols HTTP/API Public Private Organization Existing System FTP
- 33. Integration with Systems that Talk in Non-standard/ Proprietary Protocols ● Built in support for well-known proprietary protocols E.g. SAP ● Easy to extend the functionality to introduce new protocols
- 34. Integration with Systems that Talk in Non-standard/ Proprietary Protocols HTTP/API Public Private Organization Existing System Non Standard Protocol Connector
- 35. Extending the Functionality ● Extension points are available in each layer of the integrator architecture ● Extensions can be written using Java and plugged into the runtime Carbon Platform Log Send Enrich XSLT Drop ... Security SOAP POX Text ... Registry ManagementUI ScheduledTasks Message Stores Message Processors Sequences Mediators Endpoints ProxyMain Inbound EndpointAPI QoS Manage Builder Manage Formatter HTTP TCP Mail JMS ... Transport System Engine Request Response
- 36. Extension Points ● Script mediator ● Connectors ● Custom/Class mediators ● API handlers ● Custom tasks ● Custom inbound endpoints ● Custom message stores/processors ● Custom XPath functions ● Synapse handlers ● Axis2 handlers ● Axis2 modules ● Message builders and formatters ● Transport listeners and senders
- 37. Connectors ● Add-ons for Integrator runtime ● Can build your own connector easily - Using SDKs - Using web APIs ● Over 150 ready-made connectors available in WSO2 store
- 38. Integration with Files-based Systems ● A lot of legacy systems still use files as the medium to integrate with other systems ● WSO2 Enterprise Integrator supports a wide range of file transferring protocols such as FTP, SFTP, FTPS, and SMB
- 39. File Handling Use Cases Listen to a location, pick files, and invoke a web API using the content of the files File HTTP Web API Invocation Existing Legacy System
- 40. File Handling Use Cases Expose a file-based system as an API over HTTP protocol FileHTTP Existing Legacy System
- 41. Integration with Systems that Use Databases Expose databases or other data sources as services HTTP
- 42. Conclusion We discussed ● What is Brownfield Integration? ● Why do we need Brownfield Integration? ● What are the common problems we face when doing Brownfield Integration? ● WSO2 Enterprise Integrator ● How you can do Brownfield Integration with WSO2 Enterprise Integrator