![1 Copyright © 2020, Oracle and/or its affiliates | Confidential: Internal/Restricted/Highly Restricted
[Date]
Should I be building Microservices?
Paul Parkinson
Cloud Platform Dev
Lead
Microservices
Sanjay Goil
VP Product
Management
Database Server Tech
(microservices)](https://image.slidesharecdn.com/microserviceswithconvergeddboracle-200512221043/85/dbhouseparty-Should-I-be-building-Microservices-1-320.jpg)
![2 Copyright © 2020, Oracle and/or its affiliates | Confidential: Internal/Restricted/Highly Restricted
[Date]
Sanjay Goil
VP Product Management
At Oracle for 2+ years
Messaging & Microservices; Previously Event
Hub Cloud Service (Kafka); AI Service
Past 20 years:
AdTech Startup: Social Media & TV Ads Platform
HP/Autonomy : Big Data platform (IDOL,
Vertica, Logger, and Hadoop)
Sun & Intel : Built HPC Systems and Developer
Tools for Converged Parallelism
About your presenters:
Paul Parkinson
Cloud Platform Dev Lead
Dev lead for Transaction Processing in the mid-
tier for 19 years at Oracle
Helidon Microservices Platform Dev Lead](https://image.slidesharecdn.com/microserviceswithconvergeddboracle-200512221043/85/dbhouseparty-Should-I-be-building-Microservices-2-320.jpg)
![Sagas | For Transactions across Microservices
Order
Service
Inventory
Service
Delivery
Service
Kafka or JMS
client API
Kafka or JMS
in DB
VeggieDash
App
HTTP
Begin Saga
1. Place order
(local commit)
2. Check Inventory
(local commit)
3. Delivery Service
(rollback)
4. If all 3 succeed
Commit Saga [Id];
Record overall
booking
Else
Abort Saga [Id]
(Compensation)
End Saga
Transparent TEQ message propagation cross PDBs and CDB
Saga pattern requires compensation code (~80%) | Can we make it simpler ?](https://image.slidesharecdn.com/microserviceswithconvergeddboracle-200512221043/85/dbhouseparty-Should-I-be-building-Microservices-27-320.jpg)
#dbhouseparty - Should I be building Microservices?
- 1. 1 Copyright © 2020, Oracle and/or its affiliates | Confidential: Internal/Restricted/Highly Restricted [Date] Should I be building Microservices? Paul Parkinson Cloud Platform Dev Lead Microservices Sanjay Goil VP Product Management Database Server Tech (microservices)
- 2. 2 Copyright © 2020, Oracle and/or its affiliates | Confidential: Internal/Restricted/Highly Restricted [Date] Sanjay Goil VP Product Management At Oracle for 2+ years Messaging & Microservices; Previously Event Hub Cloud Service (Kafka); AI Service Past 20 years: AdTech Startup: Social Media & TV Ads Platform HP/Autonomy : Big Data platform (IDOL, Vertica, Logger, and Hadoop) Sun & Intel : Built HPC Systems and Developer Tools for Converged Parallelism About your presenters: Paul Parkinson Cloud Platform Dev Lead Dev lead for Transaction Processing in the mid- tier for 19 years at Oracle Helidon Microservices Platform Dev Lead
- 3. Two incredible voices of Experience! Copyright © 2019 Oracle and/or its affiliates.
- 4. Copyright © 2019 Oracle and/or its affiliates. Sanjay Paul Rolando Guido WeWelcome to you all from around theWorld.
- 5. 5 6: Should I Build Microservices? Sanjay Goil Paul Parkinson Copyright © 2019 Oracle and/or its affiliates. Vice President, Product Management Tx Processing Dev Lead Database Messaging and Microservices Microservices Cloud Platform With Insights From GUIDO And ROLANDO, Oracle Aces
- 6. Agenda • Introductions – 5 mins (All) • What are Microservices – 5 mins (Sanjay) • Recommendations from Guido – 10 mins • Recommendations from Rolando – 10 mins • Converged Database Simplicity (Sanjay) – 5 mins • Building a Microservice Example (Paul) – 20 mins • Quick walkthrough of the LiveLabs for Microservices • Summary + Q&A – 5 mins Copyright © 2019 Oracle and/or its affiliates.
- 7. Can we make Microservices SIMPLE ? Help you build Microservices with open interfaces • Simplify data platform • Simplify the code • Simplify integration Copyright © 2019 Oracle and/or its affiliates. Data-driven Applications at Scale
- 8. What are Microservices? “loosely coupled service oriented architecture with bounded contexts.” • Adrian Cockroft, while at Netflix Copyright © 2019 Oracle and/or its affiliates. If every service has to be updated at the same time it’s not loosely coupled If you know too much about the surrounding services you don’t have a bounded context Source: 12factor.net
- 9. Complexity of Lambda and Kappa • Many technologies • Multiple data stores • Copy contagion – Data copied many times • Complex and difficult to maintain • Skill set is fragmented 9 Apache Kafka Web servers Order service API/Brokers Mobile IoT Producers Databases Oracle NoSQL MySQL GG Kafka Connector Mobile App1 Consumers Search Portal Ops Dashboards Realtime Analytics, Alerts ML Model Training Analytic Reports Adhoc ExplorationHadoop data lake Vertica/Hive Event and data streams Pub/Sub AWS S3 Mobile Platform ELK Platform Apache Flink Transactions Hadoop Stream Everything A move from DB + DW Special purpose databases ; Event hub ; Streaming data
- 10. How do we think about Microservices? • Application Platform • Data Platform • Simplicity of Development, Deployment, Upgrades, Maintenance, Scale, Cost effectiveness,Agility, … Oracle Restricted
- 11. Some (not so) simple questions first ! • Is a separate database per (micro)service recommended? • How about REST APIs vs Event sourcing for Microservices? • Does expertise on a team benefit from a Converged approach? Copyright © 2019 Oracle and/or its affiliates.
- 12. Microservice Approach Tightly Scoped behind interfaces & highly decoupled Polyglot Programming/Persistence => Skills Own their data != absolutely need their own database instance! Communication between services is complex 3 Mechanisms Service can interact: Command & Query but also Event Smart Endpoints and Dumb Pipes Finding the right granularity of a service is challenging Really something new?Or just SOA done right?
- 13. Service 1 Service 2 Service 3 Service 4 Smart Endpoints and Dumb Pipes! – no complex Integration in the middle! Integration X Service Bus Service Orchestration Service 1 Service 2 Service 3 Service 4 X X X X
- 14. Modernizing SOA • SOA is a similar paradigm as Microservices • Reasons to move from SOA to Microservices • 7 suggestions to get started Copyright © 2019 Oracle and/or its affiliates.
- 15. First & foremost. SOA is a discipline and the Microservices architecture shares some of its principles • The first thing to understand is that if you are really doing SOA & you are looking to implement microservices, then the path is pretty much open for you • Services Design Principles such as Abstraction, Loosely Coupling, Autonomy, Composability, Standardized Service Contract; applies perfectly for your Microservices • As the title says, SOA is a discipline is not a technology. • If you are really designing services, then you are in very good position Copyright © 2019 Oracle and/or its affiliates. Microservices SOA
- 16. But…what about Oracle SOA Suite? • Now we are talking about technology • Oracle SOA Suite is a great platform, no doubt about it. Personally, I’ve been using it since the very beginning (Oracle BPEL in 2003) • Sometimes people uses it as a Services platform and sometimes as an Integration Platform • But, if you are going into a direction where some of your services running on top of Oracle Suite, needs to be moved to a microservices architecture, then we can give you some suggestions.That architecture can be deployed on Oracle Kubernetes Engine, for example. Copyright © 2019 Oracle and/or its affiliates.
- 17. Reasons of doing this type of change 1. Business drivers and needs 2. Different consumers generation different dynamics for your services and you need to respond accordingly with agility 3. If you are looking to isolate your services and reduce the dependencies (the metadata DB for example) 4. Autoscaling for your services 5. Service Isolation 6. Service Performance 7. Service Containerization 8. Service isolation needs Copyright © 2019 Oracle and/or its affiliates. https://innovationmanagement.se/imtool-articles/improvement-is-not-innovation/
- 18. Seven (7) suggestions for you 1. This is a progressive change. Do not try to make a big bang plan 2. Identify services that really need microservices dynamics. If they are OSB based (Proxy, Pipelines, Business), even better. Migrate your pipeline code 3. Reuse your contracts, if you really have applied Standardized Service Contracts 4. Do not try to migrate dependencies, that is exactly what you are trying to avoid. Example: database dehydration store 5. If you are using JMS for communication among your services, you can use similar approaches such as Oracle AQ/TEQ 6. Business Rules engine is a good candidate, since most of the times the rules are stand alone and they do not depend on others 7. Cross cutting functionality, such as OWSM policies, can be expressed like service proxies or sidecars, and depending on the dynamics you can implement the service mesh pattern Copyright © 2019 Oracle and/or its affiliates.
- 19. Multitenant for Efficient,Agile DatabaseClouds In-Memory for DatabaseAcceleration Sharding for Hyperscale and Geo Distribution Native JSON for Document Data In-Memory Ingest for Fastest IoT Cloud SQL for integratingObject Store Data Lake AutoML for simple integrated Machine Learning Persistent Memory Store for Lowest Latency BlockchainTables for Preventing Fraud Spatial and Graph for Mapping and Social Networks Events forTransactional Event-driven Microservices And many more … Oracle Autonomous Database Converged Features In-Memory Analytics Hyperscale Machine Learning Blockchain Cloud Integration { } JSON In-Memory IoT Spatial Graph Multitenant Persistent Memory Events
- 20. Web servers Order service API/Brokers Mobile IoT Producers Consumers Mobile App1 Search Portal Ops Dashboards Realtime Analytics, Alerts ML Model Training Analytic Reports Adhoc Exploration Converged database with Event Queues + Managed + Cloud Scale Transactions 20 • Microservices support • Open interfaces • Easy app building blocks & APIs • Less to learn, manage, backup, upgrade, secure • Self-managing with Autonomous Database Database Convergence Simplicity OCI Object Store Converged DB JSON
- 21. Converged Database Architecture • Event Queues supports transactional messaging - microservice state and events can be persisted by the same local transaction (not 2 phase commit) • Simplify development of fault-tolerant microservices • Error recovery logic is typically 90% of the code. And this code is often poorly tested Confidential – Gartner SAS Day 7/18/2019 compatible Multimodel PDBs withTransactional EventQueues Order Inventory Delivery JSON JSON JSON
- 22. A Modern Mobile Food Delivery Application Copyright © 2019 Oracle and/or its affiliates. order inventory delivery supplier Transactions Data AI/ML Source: Doordash
- 23. LiveLabs to get hands-on experience Copyright © 2019 Oracle and/or its affiliates. Lab3 Multitenant TEQ Multitenant Lab1 Lab4 APP PLATFORM Lab5 DATA PLATFORM Lab6 Polyglot Coding in Java, Python, node.js Lab2 OCI Registry
- 24. LiveLabs for Microservices • https://oracle.github.io/learning-library/developer- library/microservices-with-converged-db/#Let'sGetStarted! Copyright © 2019 Oracle and/or its affiliates.
- 25. Quick Walkthrough of Microservices on Converged DB with Helidon Copyright © 2019 Oracle and/or its affiliates.
- 26. Copyright © 2019 Oracle and/or its affiliates. Food order App – Microservices and message flow Mobile App Order μService Helidon MP Data type: JSON Delivery μService WebLogic Data type: Spatial Inventory μService Helidon MP Data type: relational ATP PDBS with propagated AQ queues add inventory call over gRPC Supplier μService Helidon SE Data type: relational routing and visualization API over REST place order and show order call over REST Streaming Order μService Helidon SE Data type: JSON insert order (SODA API) + send event over AQ OCI Streaming Service produce/stream orders over Kakfa consume/stream orders over Kakfa receive inventory status and other events over AQ receive order event over AQ send inventory status event over AQ stream orders call over gRPC OKE cluster OCI Monitoring Service update inventory (SQL – commutative operation over reactive) OCIR OCI Object Storage Event sourcing and choreography saga
- 27. Sagas | For Transactions across Microservices Order Service Inventory Service Delivery Service Kafka or JMS client API Kafka or JMS in DB VeggieDash App HTTP Begin Saga 1. Place order (local commit) 2. Check Inventory (local commit) 3. Delivery Service (rollback) 4. If all 3 succeed Commit Saga [Id]; Record overall booking Else Abort Saga [Id] (Compensation) End Saga Transparent TEQ message propagation cross PDBs and CDB Saga pattern requires compensation code (~80%) | Can we make it simpler ?
- 28. Copyright © 2019 Oracle and/or its affiliates. No Matter Where you Are? You can touch a Microservice? build
- 29. Future Labs Ideas – Collaborate with us! • Spatial Optimization for order delivery team. (Spatial data) Step 1: Multiple routes planning for a deliverer picking up 5 orders Step 2: Plot the routes on a map Step 3: Optimize for all delivery persons for optimal delivery time • Social Graph Recommendations (Graph data) Step 1: Social graph for recommendations of restaurants Step 2: Provide Top 3 recommendations for Thai food • Ops Analytics (Analytics dashboards) Step 1: Order analysis and prediction of delivery times Step 2: Delivery Ops dashboard on spatial map in real time Step 3: Compare predicted delivery times vs actual times with feedback learning • Machine Learning (Machine Learning) Step 1: Dynamic pricing algorithms based on demand and supply Step 2: Profitability predictions with surge pricing Step 3: Predicting models for bringing surge pricing to normal • Performance, HA, and Scaling of Microservices Platform (Performance/HA) Step 1: Adding capacity to the system Step 2: 100 orders to 10,000 orders per minute horizontal scaling of platform Step 3: App, Messaging, Data layer cost scaling Step 4: HA best practices for always on service experience Copyright © 2019 Oracle and/or its affiliates.
- 30. Copyright © 2019 Oracle and/or its affiliates. For Microservices Build the App and the Data Platform together! FINAL WORD DATA PLATFORM APP
- 31. Questions? Guido.Schmutz@trivadis.com | Trivadis rcarrasco@spsolutions.com.mx | S&P Solutions Paul.Parkinson@oracle.com | Oracle Sanjay.Goil@oracle.com | Oracle Copyright © 2019 Oracle and/or its affiliates.
Editor's Notes
- #8: We see a proliferation of data streams being generated by applications, devices, web, processes, etc. which need to be processed, stored, analyzed securely and at scale. There are many technologies to address these event streams, but the key is to recognize that event-driven action is the end goal of these event streams. Once can think of event-driven workflows in ERP, CRM, HCM that have event flows, traditionally using JMS messaging or streaming news, sensor data, social media posts to a varied set of subscribers with Apache Kafka. Often there are tough architecture choices for setting up the infrastructure, operating it, tuning it, upgrading it, scaling it. In this talk we present the simplicity of setting up Transactional Event Queues in the Oracle database which brings the best of JMS messaging and Kafka produce/consume with transactions that combine data and events delivered with notifications, ideal to build scalable microservices with a multi-model, multi-tenant architecture