Architecting for cloud native data: Data Microservices done right using Spring Cloud
3 likes832 views
The document discusses a new platform for building data microservices using Spring and Pivotal Cloud Foundry. It describes how microservices can break apart a monolithic integration system into independent, scalable services. Example microservices are shown for order management, inventory, and other functions. The platform enables building a pipeline of microservices to process and analyze data from various sources and systems in real time.
Architecting for cloud native data: Data Microservices done right using Spring Cloud
- 1. A NEW PLATFORM FOR A NEW ERA
- 2. © Copyright 2014 Pivotal. All rights reserved.© Copyright 2014 Pivotal. All rights reserved. Data Microservices with Spring 2 Breaking the enterprise integration monolith Fred Melo @fredmelo_br
- 3. © Copyright 2014 Pivotal. All rights reserved.© Copyright 2014 Pivotal. All rights reserved. 3 About Me @fredmelo_br Fred Melo Mountain View, CA https://www.linkedin.com/in/ melofred
- 4. © Copyright 2014 Pivotal. All rights reserved.
- 5. © Copyright 2014 Pivotal. All rights reserved.
- 6. © Copyright 2014 Pivotal. All rights reserved. User Mgmt Order Mgmt Shipping Inventory Payment Payment Order Mgmt Inventory User Mgmt Shipping Contract Contract Contract Contract Contract Conway’s Law and Microservices Going Agile - Breaking the monolith Dev Team
- 7. © Copyright 2014 Pivotal. All rights reserved. Deploy >cf push Cloud Controller BBS (message bus) Router App Deployed Order Mgmt Contract Automating through the Platform
- 8. © Copyright 2014 Pivotal. All rights reserved. Scale Re-deploy Migrate Distribute Upgrade Update Order Mgmt Contract User Mgmt Contract Pivotal Cloud Foundry
- 9. © Copyright 2014 Pivotal. All rights reserved. Integration Bus / ESB System A System BUser Mgmt Order Mgmt Shipping Inventory Payment
- 10. © Copyright 2014 Pivotal. All rights reserved. Integration Bus / ESB System A System BOrder Mgmt Contract Inventory Contract Shipping ContractUser Mgmt Contract
- 11. © Copyright 2014 Pivotal. All rights reserved. Integration Bus / ESB System A System B E T L Data WarehouseOrder Mgmt Contract Inventory Contract Shipping ContractUser Mgmt Contract
- 12. © Copyright 2014 Pivotal. All rights reserved. Integration Bus / ESB System A System BOrder Mgmt Contract Inventory Contract Shipping ContractUser Mgmt Contract E T L Data Warehouse ???
- 13. © Copyright 2014 Pivotal. All rights reserved. Anatomy of a typical Integration Pipeline Source Processing Step Processing Step Processing Step Processing Step Data Data Monolith Destination
- 14. © Copyright 2014 Pivotal. All rights reserved. Integration Pipeline Scale Re-deploy Upgrade Migrate Distribute Update Source Destination Processing Step Processing Step Processing Step Processing Step
- 15. © Copyright 2014 Pivotal. All rights reserved. Integration Pipeline Source Destination Processing Step Processing Step Processing Step Processing Step SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract
- 16. © Copyright 2014 Pivotal. All rights reserved. Binding Abstraction Layer Transport Options Integration Pipeline Data Messaging Microservices Spring Cloud Stream Data Data SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract
- 17. © Copyright 2014 Pivotal. All rights reserved. Backing Services Transport Options Pivotal Cloud Foundry Auto Scaling Auto Healing Aggregated Logging Integrated Metrics Transport Transparency Infrastructure Transparency SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract Data Pipeline Visual Design Integrated Monitoring
- 18. © Copyright 2014 Pivotal. All rights reserved. Backing Services Transport Options Pivotal Cloud Foundry Auto Scaling Auto Healing Aggregated Logging Integrated Metrics Transport Transparency Infrastructure Transparency SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract Data Pipeline Visual Design Integrated Monitoring
- 19. © Copyright 2014 Pivotal. All rights reserved. JSON Filter Transform Enrich Custom HTTP Sample pipeline Deploy SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract Pivotal Cloud Foundry Spring Cloud Data Flow
- 20. © Copyright 2014 Pivotal. All rights reserved. DEMO “Fast Data" Machine Learning Normal High Risk
- 21. © Copyright 2014 Pivotal. All rights reserved. Rest / HTTP Transactions Browser Machine Learning ? GemFire Greenplum Analytics
- 22. © Copyright 2014 Pivotal. All rights reserved. Machine Learning (Clustering) GemFire Greenplum Analytics Train
- 23. © Copyright 2014 Pivotal. All rights reserved. Machine Learning (Clustering) GemFire Greenplum Analytics Train distance from home location transactionvalue Medium RiskLow Risk High Risk
- 24. © Copyright 2014 Pivotal. All rights reserved. Machine Learning (Clustering) GemFire Greenplum Analytics Train PMML + Distance from home ML Evaluation high risk transactions Suspects
- 25. © Copyright 2014 Pivotal. All rights reserved. Rest / HTTP Transactions Browser High Risk Alerts GemFire Greenplum + Distance from hom
- 26. © Copyright 2014 Pivotal. All rights reserved. Order Mgmt Contract Inventory Contract Shipping Contract Payments Contract Pivotal Cloud Foundry SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract Data Warehouse PMML Model External services Actionable insights Recommendation Engine Business Microservices Data Microservices
- 27. A NEW PLATFORM FOR A NEW ERA