Webinar patterns anti patterns
Download as PPTX, PDF1 like11,277 views
1) Apache Kafka is a distributed event streaming platform that is highly scalable, durable, fault-tolerant and provides publish-subscribe capabilities. 2) When deploying Kafka, it is important to consider data durability and tune configuration parameters like acks and min.insync.replicas to achieve the appropriate tradeoff between durability, availability and latency. 3) Many common issues can arise from not properly handling errors, exceptions and failures throughout the Kafka ecosystem. Proper monitoring, configuration, and exception handling is required.
Webinar patterns anti patterns
- 1. Confluent & Apache Kafka Patterns / Anti-patterns Marcelo Manta and Jean Louis Boudart
- 2. Problem ?
- 3. With
- 4. How
- 5. Store &ETL Process Publish & Subscribe In short
- 6. From a simple idea
- 7. From a simple idea
- 8. with great properties ! • Scalability • Retention • Durability • Replication • Security • Resiliency • Throughput • Ordering • Exactly Once Semantic • Transaction • Idempotency • Immutability • … 8
- 9. So goooooood
- 10. What could potentially go wrong ?
- 12. …which is true for any data systems
- 13. Not thinking about Durability
- 14. Data durability 14 If you didn’t think about it… it’s not durable!
- 18. And you might have lost data!
- 19. Data durability 1 9 Kafka is notwaiting for a disk flush by default. Durability is achieved through replication.
- 24. Data durability 24 It depends on your configuration...
- 29. Data durability 29 acks=1 (default value) good for latency acks=all good for durability
- 31. acks=all 3 1 The leader will wait for the full setof in-sync replicas to acknowledge the record.
- 35. min.insync.replicas 35 minimum number of replicas that must acknowledge. Default is 1.
- 37. Data Durability while Producing ? Tune it with the parameters acks and min.insync.replicas
- 38. defaults 38 The default values are optimized for availability & latency. If durability is more important, tune it!
- 39. Deploying on multi datacenters ?
- 40. r ·-::i- _• ;.!!.c_:,;--... . , L.L:.-_::;;. .--. •c - ?: . '-:" ?"% -:- ':.. ¥.. c. " .; '< -.. ".:-,:,'<',,,,"'= ,...,,../-.-,<.a!.J V(fl..?---..,..'.. '?, 8i''{.. ·g ----,-,,.>- *cf" ,., '--. z _ _,.,....,-,;,,:_). :; ?li-·-:-... _J C re, .Dubli- ,',.7_ I(-<'---_ i {..'t ..J · •--·/4· .• "' /4, _; . ,.c.,,; -"->. '·r , .& ( r ('.. .-, ..._- r,.I, •:;-, .c ..:;, •.: .. r ( ... . ) "'IJ J_J ,,v ,/ > ;o i.; , .. •' .-.. -·
- 41. Multi-dc 41 It’s quite complicated... It’s easy to makeit wrong on many levels. It could be a 3htalk.
- 42. Multi-dc Disaster recovery for multi datacenter 42
- 43. What about the consumers ?
- 44. consumers 44 Consumer can read only committed data.
- 46. Think about data durability and decide of the best trade-off for you
- 47. Throughput, latency, durability, availability Optimizing your Apache Kafka deployment 47
- 48. Focusing only on the happy path
- 52. retries 52 It will cause theclient to resend any record whose send fails with a potentially transient error. Default value : 0
- 54. retries 54 Use built in retries ! Bump it from0 to infinity!
- 55. retries 55 But you are exposed to a different kind of issue…
- 57. enable.idempotence 57 When set to 'true', the producer will ensure that exactly one copy of each message is written. Default value: false
- 60. Use built in idempotency!
- 61. But it does not save you from - Managing exception and failure - Developing Idempotent consumer
- 64. At least once (default) At most once Exactly Once
- 71. commit 7 1 Manually committing aggressively... Add a huge workload on Apache Kafka
- 73. commit 73 Manually committing aggressively... Does not provide exactly once semantic
- 74. Embrace at least once
- 75. Rely on Kafka Streams with Exactly Once !
- 78. Future<RecordMetadata> send(ProducerRecord<K, V> record);
- 79. Future<RecordMetadata> send(ProducerRecord<K, V> record, Callback callback);
- 80. producer.send(record, (metadata, exception) -> { });
- 81. error handling 81 We don’t expect the unexpected until the unexpected is expected.
- 83. error handling 83 A message can not be processed
- 84. error handling 84 A message can not be processed A message doesn’t have the expected schema
- 85. Retry
- 87. Infinite retry
- 89. Write to a dead letter queue and continue
- 93. No silver bullet
- 99. governance 99 Changes in producers might impact consumers
- 103. Share Schemas
- 104. Let bad citizens wander around
- 106. Leverage Security, ACL and Quota Security Authorization and ACLs Enforcing Client Quotas
- 107. Installing prod on Sunday night
- 109. configuration 109 If you use thedefault configuration… You will have issues!
- 110. Please read the doc Running Kafka in Production Running ZooKeeper in Production
- 111. Not configuring your OS
- 113. os 11 3 Tune at least your open file descriptors and mmap count.
- 114. Configure your os Running Kafka in Production
- 117. No monitoring
- 118. Too much partitions
- 120. Partition key choice
- 121. Topics vs Partitions
- 122. Call external services in Kafka Streams
- 123. Questions