API Days Australia
0 likes263 views
This document discusses how enterprises are moving from monolithic applications to microservices architectures and the challenges of using messaging queues and REST APIs for communication between microservices. It introduces Apache Kafka as an event streaming platform that can be used to build an event-driven architecture and overcome these challenges. Kafka allows for completely decoupled microservices that communicate asynchronously through events. It provides a single standard for inter-communication, maintains version compatibility, and enables state to be persisted in a distributed and highly scalable manner. The document argues that Kafka is well-suited for microservices as it reduces dependencies between services and allows for reactive, asynchronous execution.
API Days Australia
- 1. Building an Enterprise Eventing Platform using Apache Kafka From REST APIs and Message Queues, to Event-Driven Streaming Gnanaguru (Guru) Sattanathan, Solutions Engineer - Confluent Kevin Barton, Senior Consultant, Solution Design - National Australia Bank Mathew Chai, Senior Consultant, Support - National Australia Bank
- 2. Enterprises (especially Banks) are moving from software users, to becoming software Faster application development & time to market Enable teams to build independently Reduce infrastructure cost and complexity Eliminate dependencies in feature delivery Scale services with smaller, agile teams “In many ways, we see ourselves as a technology company with a banking license.” — Michael Corbat, Citi CEO
- 3. Evolution of Monolith APP APP APP APP Monolith Large self contained application Complex app with highly interdependent parts Poor developer experience and productivity Slow feature delivery Difficult to deploy, impacting other systems Requires replication of entire application Microservices Multiple smaller, single function apps Independently deployable and upgradable Built around solving business capabilities Can be built w/ different programming languages Scalable and agile = faster feature delivery Leverages newer development platforms To Microservices
- 4. Existing approaches to developing microservices Messaging Queues (rabbitMQ, ActiveMQ) ● Message brokers act as a centralized messaging service through which all of the microservices communicate ● Brokers handle messaging queueing, HA, reliable communication between services ● Messages received in a queue rather than dropped and processed later Communication via HTTP REST API ● Benefits: ○ Simple set up ○ Efficient message delivery ● Synchronous communication ○ Client sends request and waits for response
- 5. Challenges with REST based microservices Fulfillme nt service Stock service Order service Return service Payment service UI service GUI ● Difficult to enforce standards across services ● Does not scale if servers are synchronous ● Risky inter-service dependencies ● Services required to maintain state ● Complex management and version compatibility—slows development ● Requires load balancing
- 6. Challenges with legacy messaging queues Microservice Producer Legacy Messaging Queue Queue 1 Queue 2 Microservice A Microservice B ● Extremely difficult to scale ● Lacks message persistence—if message delivery fails, its unavailable for replay ● Low throughput and high latency ● Need prior knowledge of consumers ● Expensive MQ and mainframe technology costs ●
- 7. Fulfillment service Stock service Order service Return service Payment service UI service GUI Why build with Confluent ● Completely decoupled microservices ● Single standard for inter-communication ● Maintains version compatibility ● Asynchronous services development ● State persistent on single platform for replay ● Distributed and highly scalable ● Process data in flight and real-time ● Deployment flexibility -- on prem, in cloud, or hybrid Confluent enables a new class of event driven microservices
- 8. Why does it make sense? Microservices Distributed Semi-loose with API versioning Requires a consistent distributed system Synchronous Event Driven Microservices Central, synchronized through events Completely decoupled (Fire and forget) The streaming platform reduces the dependencies to external system Reactive (asynchronous) State Coupling E2E Testing Execution Model
- 9. Fireside Chat Kevin C Barton National Australia Bank Mathew Chai National Australia Bank Gnanaguru(Guru) Sattanathan Confluent
- 10. Workshop happening today ! Kafka streaming in minutes on Confluent Cloud: Create a complete stack of fully managed services in Confluent Cloud September 16, 1:50pm-2:40pm