Demystifying Event-Driven Architectures with Apache Kafka | Bogdan Sucaciu, Pluralsight
1 like938 views
The document discusses event-driven architectures (EDA) with a focus on Apache Kafka, highlighting the transition from monolithic to microservices architectures. It addresses common challenges in scalability, data access, and consistency, while presenting solutions and the importance of understanding eventual consistency. The use of event streams, producers, and consumers is emphasized, showcasing code snippets for implementing Kafka producers and consumers.
Demystifying Event-Driven Architectures with Apache Kafka | Bogdan Sucaciu, Pluralsight
- 1. Bogdan Sucaciu Deloitte Digital | Pluralsight Demystifying Event-Driven Architectures with Apache Kafka bsucaciu.com @bsucaciu linkedin.com/bogdan-sucaciu
- 2. Bogdan Sucaciu Deloitte Digital | Pluralsight Demystifying Event-Driven Architectures with Apache Kafka bsucaciu.com @bsucaciu linkedin.com/bogdan-sucaciu
- 3. Bogdan Sucaciu Deloitte Digital | Pluralsight Demystifying Event-Driven Architectures with Apache Kafka bsucaciu.com @bsucaciu linkedin.com/bogdan-sucaciu
- 4. The Monolith
- 5. The Monolith A lot of code goes in here
- 7. The Monolith
- 8. SOA
- 12. Challenge -> Solution -> New Challenge GOOD BAD Data Access Scalability
- 14. Event-Driven Architecture Software architecture paradigm promoting production, consumption of, and reaction of events - Wikipedia
- 15. EDA
- 17. EDA
- 18. EDA Producer Consumer Event Stream Event Stream Event Stream Consumer Producer / Consumer Producer / Consumer
- 19. EDA Producer Consumer Event Stream Event Stream Event Stream Consumer Producer / Consumer Producer / Consumer
- 20. EDA Producer Consumer Event Stream Event Stream Event Stream Consumer Producer / Consumer Producer / Consumer
- 22. EDA Producer Consumer Event Stream Event Stream Event Stream Consumer Producer / Consumer Producer / Consumer
- 23. EDA with Apache Kafka Kafka Broker Topics
- 24. EDA with Apache Kafka Kafka Broker Inversion of Control Fan-out 1 P-type / Event Stream Producers don’t know Consumers Topics
- 26. Challenge -> Solution GOOD BAD Data Access Scalability
- 27. Challenge -> Solution -> New Challenge GOOD BAD Data Access Consistency Scalability
- 28. The key to building successful EDA is understanding and accepting Eventual Consistency
- 29. Past Present Future Query Command Event A B Message A B A ? A B A ?B A B
- 31. Event Producer Event Consumer private static String TOPIC = "event-notification"; … Properties props = new Properties(); props.put(BOOTSTRAP_SERVERS_CONFIG, "broker:9092"); props.put(KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName()); props.put(VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName()); KafkaProducer<String, String> producer = new KafkaProducer<>(props); ProducerRecord<String, String> producerRecord = new ProducerRecord<>(TOPIC, key, value); producer.send(producerRecord); private static String TOPIC = "event-notification"; … Properties props = new Properties(); props.put(GROUP_ID_CONFIG, "event.consumer"); props.put(BOOTSTRAP_SERVERS_CONFIG, "broker:9092"); props.put(KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class); props.put(VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class); KafkaConsumer<String, String> consumer = new KafkaConsumer<>(props); consumer.subscribe(Collections.singletonList(TOPIC)); while(true) { ConsumerRecords<String, String> records = consumer.poll(ofMillis(100)); records.forEach(record -> processRecord(record)); }
- 32. Producer Consumer 3 Consumer 2 Consumer 1
- 33. Producer Consumer 3 Consumer 2 Consumer 1
- 35. Address Changed Event P Kafka Broker C1 C2 C3 Only new address? Old & new address? User info? ?
- 36. Address Changed Event P Only new address C1 C2 C3 Kafka Broker
- 37. Address Changed Event P C1 C2 C3 Kafka Broker
- 38. Address Changed Event P C1 C2 C3 Kafka Broker
- 39. Address Changed Event P Tax C2 C3 Kafka Broker
- 40. Avoid querying back the source system when using events!
- 41. Avoid querying back the source system when using events! Events still make sense after 10 years in a datalake!
- 42. Distributed Source of Truth
- 43. Event Sourcing
- 44. Event Sourcing Initial Amount 0 Add 500 Add 200 Subtract 300 0 Current Amount
- 45. Event Sourcing Initial Amount 0 Add 500 Add 200 Subtract 300 Current Amount 500
- 46. Event Sourcing Initial Amount 0 Add 500 Add 200 Subtract 300 Current Amount 700
- 47. Event Sourcing Initial Amount 0 Add 500 Add 200 Subtract 300 Current Amount 400
- 48. GIT