NAB Tech Talk
0 likes169 views
The document discusses a tech talk about Kafka applications and stream processing, highlighting testing methods such as using Testcontainers and the TopologyTestDriver. It covers various aspects of Kafka, including its producer/consumer API, Kafka Streams, and ksqlDB for real-time applications. The talk emphasizes the importance of testing in production and provides resources for further learning and community engagement.
NAB Tech Talk
- 1. @gAmUssA | #techtalk | @confluentinc I don’t always test my streams, but when I do I do it in production June 16th / NAB Techtalk
- 2. @gAmUssA | #techtalk | @confluentinc https://gamov.dev/nab-tech-talk-june2020
- 3. @gAmUssA | #techtalk | @confluentinc Agenda 01 Kafka Apps What are those? 02 Testing basics Run Kafka and CP locally 03 Can we better? Test with TestContainers 04 Testing Streams TopologyTestDriver vs TestContainers 05 Testing ksqlDB Ksql-test-runner vs TestContainers 06 Conclusion Always Be Testing
- 4. @gAmUssA | #techtalk | @confluentinc Preface
- 5. @gAmUssA | #techtalk | @confluentinc Senior Developer: - Where did you get this code? Junior Developer: - From Stackoverflow Senior Developer: - from question or answer?
- 6. @gAmUssA | #techtalk | @confluentinc An example from my life: Kafka Tutorials website
- 7. @gAmUssA | #techtalk | @confluentinc Literate Programming… Do you speak it… https://en.wikipedia.org/wiki/Literate_programming
- 8. @gAmUssA | #techtalk | @confluentinc
- 9. @gAmUssA | #techtalk | @confluentinc
- 10. @gAmUssA | #techtalk | @confluentinc
- 11. @gAmUssA | #techtalk | @confluentinc
- 12. @gAmUssA | #techtalk | @confluentinc
- 13. @gAmUssA | #techtalk | @confluentinc Kafka Apps aka streams processing
- 14. @gAmUssA | #techtalk | @confluentinc Stream Processing is the toolset for dealing with events as they move!
- 15. @gAmUssA | #techtalk | @confluentinc What’s Kafka Apps? My App
- 16. @gAmUssA | #techtalk | @confluentinc Producers Consumers Kafka
- 17. @gAmUssA | #techtalk | @confluentinc Schema, WUT?
- 18. @gAmUssA | #techtalk | @confluentinc Let’s see some code
- 19. @gAmUssA | #techtalk | @confluentinc Kafka Streams for stream processing
- 20. @gAmUssA | #techtalk | @confluentinc the KAFKA STREAMS API is a JAVA API to BUILD REAL-TIME APPLICATIONS
- 21. @gAmUssA | #techtalk | @confluentinc Things Kafka Streams Does Runs everywhere Clustering done for you Exactly-once processing Event-time processing Integrated database Joins, windowing, aggregation S/M/L/XL/XXL/XXXL sizes
- 22. @gAmUssA | #techtalk | @confluentinc https://kafka.apache.org/documentation/streams/developer-guide/testing.html
- 23. @gAmUssA | #techtalk | @confluentinc Topology Test driver
- 24. @gAmUssA | #techtalk | @confluentinc • TopologyTestDriver • Unit test your Kafka Streams Topology • Doesn’t Require Kafka infra to run • Convenience methods to define input and output • Convenience methods to provide fixtures and validate the results
- 25. @gAmUssA | #techtalk | @confluentinc Let’s see some code
- 26. @gAmUssA | #techtalk | @confluentinc
- 27. @gAmUssA | #techtalk | @confluentinc
- 28. @gAmUssA | #techtalk | @confluentinc When TTD doesn’t help? https://habr.com/ru/company/ jugru/blog/499408/
- 29. @gAmUssA | #techtalk | @confluentinc When TTD doesn’t help? https://gamov.dev/ livestreams-s01e05
- 30. @gAmUssA | #techtalk | @confluentinc Ksqldb for Stream processing
- 31. @gAmUssA | #techtalk | @confluentinc
- 32. @gAmUssA | #techtalk | @confluentinc
- 33. @gAmUssA | #techtalk | @confluentinc Let’s see some code
- 34. @gAmUssA | #techtalk | @confluentinc Conclusion
- 35. @gAmUssA | #techtalk | @confluentinc What did you learn today? •What’s Kafka apps apps and how to test them •Producer / Consumer API •Kafka Streams •ksqlDB •How containers and TestContainer help a lot
- 36. @gAmUssA | #techtalk | @confluentinc
- 37. @gAmUssA | #techtalk | @confluentinc Want to learn more?
- 38. developer.confluent.io Learn Kafka. Start building with Apache Kafka at Confluent Developer.
- 39. @gAmUssA | #techtalk | @confluentinc ALL UPCOMING MEETUPS NEW EVENT EMAIL ALERTS THE CONFLUENT MEETUP HUB CNFL.IO/MEETUP-HUB VIDEOS OF PAST MEETUPS SLIDES FROM THE TALKS
- 40. @gAmUssA | #techtalk | @confluentinc Confluent Community Slack A vibrant community of over 16,000 members Come along and discuss Apache Kafka and Confluent Platform on dedicated channels including #ksqlDB, #connect, #clients, and more http://cnfl.io/slack
- 41. @gAmUssA | #techtalk | @confluentinc Free eBooks Designing Event-Driven Systems Ben Stopford Kafka: The Definitive Guide Neha Narkhede, Gwen Shapira, Todd Palino Making Sense of Stream Processing Martin Kleppmann I ❤ Logs Jay Kreps http://cnfl.io/book-bundle
- 42. @gAmUssA | #techtalk | @confluentinc