The Playful Bond Between REST And Data Streams
0 likes66 views
1. REST APIs have proliferated as a way to integrate microservices but don't meet all integration needs and can result in tight coupling between systems. 2. Using streaming data platforms like Kafka can help reduce the number of integration lines needed between systems and provides stronger delivery guarantees compared to REST APIs. 3. While REST APIs are good for synchronous requests and responses, a data streaming platform that includes both REST and streaming data capabilities can help integrate application and data systems using the best approach for different use cases and requirements.
The Playful Bond Between REST And Data Streams
- 1. The Playful Bond Between REST And Data Streams Warren Vella | Solutions Engineer, Melbourne | Confluent
- 2. About me.
- 3. “Hey, I am a microservice, come and talk to me.” REST API
- 4. “I need to share my data with an external application.” REST API
- 5. “I need to modernise my database with a facade.” REST API
- 6. API ≠ HTTP-based RESTful APIs
- 7. Operational Databases Analytic Systems Azure Synapse How do we integrate these systems together? SaaS Applications Applications Microservices Monoliths LEGACY INTEGRATION REST APIs only work here
- 8. REST APIs 8 1. …have proliferated as a way to integrate microservices. 2.…are often the default when exchanging data between organisations. 3.…don’t meet all our integration needs.
- 9. Lets go on a journey… The Stream Bean Cafe ☕
- 10. Early 2000s
- 12. 2011ish
- 13. Microservices. Trend away from monolithic applications.
- 14. Distributed application architecture POS Payment Gateway DataWarehouse Realtime Analytics CRM
- 15. How many lines to integrate 4 systems?
- 16. n2-n
- 17. App 2 App 3 App 4 App 1 42-4 = 12 lines
- 18. How many lines to integrate 2,000 systems?
- 19. n2-n
- 22. The microservices death star…
- 23. 1. Physical 2. Data Link 3. Network 4. Transport 5. Session 6. Presentation 7. Application REST API POS DataWarehouse
- 24. Synchronous request-response REST APIS are addictive. Like an addiction, they give us a false sense of comfort of being always-available. The more we build, the more we want but we often overlook the exponential costs associated with building failure mechanisms for never ending point- to-point integrations. We end up relapsing to REST APIs.
- 25. Payment Gateway Request Response POS DataWarehouse Realtime Analytics CRM Asynchronous Synchronous
- 26. REST APIs 26 1. …have proliferated as a way to integrate microservices. 2.…are often the default when exchanging data between organisations. 3.…don’t meet all our integration needs. But we have ended up with a death star. But are not suited for high-throughput. So we have lots of different integration tools.
- 27. Let’s meet
- 28. Govern Process Connect Store Kafka decouples the producers and consumers Data Topic
- 29. Side note… Why is Kafka not a HTTP protocol like REST?
- 30. Old World New World Event Processor Stream Processor REST API
- 31. REST API • Great for Synchronous Responses • Tight-coupling • No delivery guarantees • Receiving systems can be overwhelmed • Emphasis on Requests/Responses • No Replay • True Decoupling • Stronger delivery guarantees. Kafka is a buffer. • Receiving systems process at a rate they can handle. • Emphasis on events. • Rebuild state by replay.
- 32. Payment Gateway Data Topic Can REST and Kafka co-exist? DataWarehouse Realtime Analytics CRM Inside Boundary POS Microservices Outside Boundary Request / Response
- 33. 1. Physical 2. Data Link 3. Network 4. Transport 5. Session 6. Presentation 7. Application POS Realtime Analytics
- 36. Operational Databases SaaS Applications Analytic Systems Azure Synaps e | DATA STREAMING PLATFORM Today, a fourth estate is emerging that provides a new foundation for your data strategy
- 37. “Hey, I am a microservice, come and talk to me.” REST API
- 38. “I need to share my data with an external application.” REST API
- 39. “I need to modernise my database with a facade.”
- 40. The future of integration.
- 41. Key Takeaways 41 1. ✅ Reduce the amount of lines between systems with Kafka. 2.✅ REST APIs are great for Request- Response but also consider Kafka as a pattern for exchanging data between organisations. 3.✅ A Data Streaming Platform integrates both application and data estates.