DevOps, CD and [Data] Microservices
5 likes1,183 views
The document outlines a new era of technology platforms focused on DevOps, microservices, and data management, emphasizing breaking away from monolithic data structures. It discusses the architecture and deployment of applications using Pivotal Cloud Foundry, along with concepts such as polyglot persistence and data pipelines. Key features include scalability, automation, and the integration of various data services.
![© Copyright 2014 Pivotal. All rights reserved.© Copyright 2014 Pivotal. All rights reserved.
DevOps, CD & [Data] Microservices
2
Breaking the data monolith
Fred Melo
@fredmelo_br](https://image.slidesharecdn.com/qconsp-devopsfordata-160405174944/85/DevOps-CD-and-Data-Microservices-2-320.jpg)
DevOps, CD and [Data] Microservices
- 1. A NEW PLATFORM FOR A NEW ERA
- 2. © Copyright 2014 Pivotal. All rights reserved.© Copyright 2014 Pivotal. All rights reserved. DevOps, CD & [Data] Microservices 2 Breaking the data monolith Fred Melo @fredmelo_br
- 3. © Copyright 2014 Pivotal. All rights reserved. $ Business DevelopmentQA Operations Customer WHY?
- 4. © Copyright 2014 Pivotal. All rights reserved. $ WHY? Concept Cash
- 5. © Copyright 2014 Pivotal. All rights reserved. How?
- 6. © Copyright 2014 Pivotal. All rights reserved.
- 7. © Copyright 2014 Pivotal. All rights reserved. User Mgmt Order Mgmt Shipping Inventory Payment Payment Order Mgmt Inventory User Mgmt Shipping Contract Contract Contract Contract Contract Conway’s Law and Microservices Architecture x Development teams
- 8. © Copyright 2014 Pivotal. All rights reserved. Deploy >cf push Cloud Controller BBS (message bus) Router App Deployed Order Mgmt Contract
- 9. © Copyright 2014 Pivotal. All rights reserved. App App App App App App Scale AppApp App Auto-heal Migrate Pivotal Cloud Foundry
- 10. © Copyright 2014 Pivotal. All rights reserved. App App App App App App Scale AppApp App Auto-heal Migrate Logging Metrics Platform Services Pivotal Cloud Foundry
- 11. © Copyright 2014 Pivotal. All rights reserved. App Stateless Ephemeral disk Short-lived context “12 Factor" App App App App App AppAppApp App Pivotal Cloud Foundry
- 12. © Copyright 2014 Pivotal. All rights reserved. Statefull Disk-based storage Long-lived context Data Service ??????? App App App App App AppAppApp App Pivotal Cloud Foundry
- 13. © Copyright 2014 Pivotal. All rights reserved. Scale Re-deploy Migrate Distribute Upgrade Update Pivotal Cloud Foundry
- 14. © Copyright 2014 Pivotal. All rights reserved. User Mgmt Order Mgmt Shipping Inventory Payment Payment Order Mgmt Inventory User Mgmt Shipping Contract Contract Contract Contract Contract Polyglot Persistence "RDBMs is the king" “The right tool for each job" One Size Fits All
- 15. © Copyright 2014 Pivotal. All rights reserved. Polyglot Persistence RDBMs only IMDGs, K-V Stores, Object stores, … One Size Fits All Scale-up only Grid-based HADisk-based HA Scale-out Specific hardware Containerized
- 16. © Copyright 2014 Pivotal. All rights reserved. Scale Re-deploy Migrate Distribute Upgrade Update Order Mgmt Contract User Mgmt Contract Pivotal Cloud Foundry
- 17. © Copyright 2014 Pivotal. All rights reserved. Order Mgmt Contract User Mgmt Contract Inventory Contract Payments Contract Shipping Contract SQL? Analytics? BI? Reporting?
- 18. © Copyright 2014 Pivotal. All rights reserved. Order Mgmt Contract User Mgmt Contract Inventory Contract Payments Contract Shipping Contract Analytical System DeliveryFrequency Degreeofautomation Manual & Tooling Transactions Analytics Data Scientist MDM Analytical Team
- 19. © Copyright 2014 Pivotal. All rights reserved. How about Data Pipelines? Anatomy of a Data Pipeline Source Destination Processing Step Processing Step Processing Step Processing Step Data Data Monolith
- 20. © Copyright 2014 Pivotal. All rights reserved. Data Pipeline Scale Re-deploy Upgrade Migrate Distribute Update Source Destination Processing Step Processing Step Processing Step Processing Step
- 21. © Copyright 2014 Pivotal. All rights reserved. SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract Binding Abstraction Layer Transport Options Data Pipeline Spring Cloud Stream Breaking the Data Pipeline Monolith
- 22. © Copyright 2014 Pivotal. All rights reserved. SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract Transport Options Spring Cloud Stream Apache YARN Breaking the Data Pipeline Monolith
- 23. © Copyright 2014 Pivotal. All rights reserved. From Microservices to Data Microservices SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract Cloud Services Transport Choices Spring Cloud Stream Pivotal Cloud Foundry Scale Auto-heal Logging Metrics Registry and Discovery Circuit Breaker Distributed Config
- 24. © Copyright 2014 Pivotal. All rights reserved. SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract Pivotal Cloud Foundry Services Data Microservices Spring Cloud Data Flow Data Microservices Orchestration Pivotal Cloud Foundry
- 25. © Copyright 2014 Pivotal. All rights reserved. SpringBoot app Contract SpringBoot app Contract Spring Cloud Data Flow JSON Filter Transform Aggregate Custom HTTP Sample pipeline Pivotal Cloud Foundry SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract SpringBoot app Contract Deploy
- 26. © Copyright 2014 Pivotal. All rights reserved. Your own infrastructure Pivotal Cloud Foundry Microservices Spring Cloud Services GemFire Data Microservices Spring Cloud Data Flow Redis MySQL Greenplum Polyglot persistence PostgreSQL HTAP
- 27. A NEW PLATFORM FOR A NEW ERA