I Don’t Always Test My Streams, But When I Do, I Do it in Production (Viktor Gamov, Confluent) Kafka Summit 2020
0 likes1,417 views
The document presents a presentation from Kafka Summit 2020, focusing on Kafka applications and testing methodologies. It covers topics such as stream processing, testing with Testcontainers, and using the Kafka Streams API, while emphasizing the importance of continuous testing. Additionally, it encourages further learning and engagement with the Kafka community.
I Don’t Always Test My Streams, But When I Do, I Do it in Production (Viktor Gamov, Confluent) Kafka Summit 2020
- 1. Your projector is working…
- 2. Your projector is working…
- 3. Your projector is working… screenshare
- 4. @gamussa | #kafkasummit | @confluentinc August 2020 / Kafka Summit I don’t always test my streams, but when I do I do it in production
- 5. @gamussa | #kafkasummit | @confluentinc
- 6. @gamussa | #kafkasummit | @confluentinc Special thanks @inponomarev @bsideup
- 7. @gamussa | #kafkasummit | @confluentinc https://gamov.dev/kafkasummit2020
- 8. @gamussa | #kafkasummit | @confluentinc Agenda
- 9. @gamussa | #kafkasummit | @confluentinc Agenda 01 Kafka Apps What are those?
- 10. @gamussa | #kafkasummit | @confluentinc Agenda 01 Kafka Apps What are those? 02 Testing basics Run Kafka and CP locally
- 11. @gamussa | #kafkasummit | @confluentinc Agenda 01 Kafka Apps What are those? 02 Testing basics Run Kafka and CP locally 03 Can we better? Test with TestContainers
- 12. @gamussa | #kafkasummit | @confluentinc Agenda 01 Kafka Apps What are those? 02 Testing basics Run Kafka and CP locally 03 Can we better? Test with TestContainers 04 Testing Streams TopologyTestDriver vs TestContainers
- 13. @gamussa | #kafkasummit | @confluentinc Agenda 01 Kafka Apps What are those? 02 Testing basics Run Kafka and CP locally 03 Can we better? Test with TestContainers 04 Testing Streams TopologyTestDriver vs TestContainers 05 Testing ksqlDB Ksql-test-runner vs TestContainers
- 14. @gamussa | #kafkasummit | @confluentinc Agenda 01 Kafka Apps What are those? 02 Testing basics Run Kafka and CP locally 03 Can we better? Test with TestContainers 04 Testing Streams TopologyTestDriver vs TestContainers 05 Testing ksqlDB Ksql-test-runner vs TestContainers 06 Conclusion Always Be Testing
- 15. @gamussa | #kafkasummit | @confluentinc Preface
- 16. @gamussa | #kafkasummit | @confluentinc Senior Developer: - Where did you get this code? Junior Developer: - From Stackoverflow Senior Developer: - from question or answer?
- 17. @gamussa | #kafkasummit | @confluentinc An example from my life: Kafka Tutorials website
- 18. @gamussa | #kafkasummit | @confluentinc Literate Programming… Do you speak it… https://en.wikipedia.org/wiki/Literate_programming
- 19. @gamussa | #kafkasummit | @confluentinc
- 20. @gamussa | #kafkasummit | @confluentinc
- 21. @gamussa | #kafkasummit | @confluentinc
- 22. @gamussa | #kafkasummit | @confluentinc
- 23. @gamussa | #kafkasummit | @confluentinc
- 24. @gamussa | #kafkasummit | @confluentinc
- 25. @gamussa | #kafkasummit | @confluentinc Kafka Apps aka streams processing
- 26. @gamussa | #kafkasummit | @confluentinc Stream Processing is the toolset for dealing with events as they move!
- 27. @gamussa | #kafkasummit | @confluentinc What’s Kafka Apps? My App
- 28. @gamussa | #kafkasummit | @confluentinc Producers Consumers Kafka
- 29. @gamussa | #kafkasummit | @confluentinc Schema, WUT?
- 30. @gamussa | #kafkasummit | @confluentinc Let’s see some code
- 31. @gamussa | #kafkasummit | @confluentinc Kafka Streams for stream processing
- 32. @gamussa | #kafkasummit | @confluentinc the KAFKA STREAMS API is a JAVA API to BUILD REAL-TIME APPLICATIONS
- 33. @gamussa | #kafkasummit | @confluentinc https://kafka.apache.org/documentation/streams/developer-guide/testing.html
- 34. @gamussa | #kafkasummit | @confluentinc Topology Test driver
- 35. @gamussa | #kafkasummit | @confluentinc
- 36. @gamussa | #kafkasummit | @confluentinc • TopologyTestDriver
- 37. @gamussa | #kafkasummit | @confluentinc • TopologyTestDriver • Unit test your Kafka Streams Topology
- 38. @gamussa | #kafkasummit | @confluentinc • TopologyTestDriver • Unit test your Kafka Streams Topology • Doesn’t Require Kafka infra to run
- 39. @gamussa | #kafkasummit | @confluentinc • TopologyTestDriver • Unit test your Kafka Streams Topology • Doesn’t Require Kafka infra to run • Convenience methods to define input and output
- 40. @gamussa | #kafkasummit | @confluentinc • TopologyTestDriver • Unit test your Kafka Streams Topology • Doesn’t Require Kafka infra to run • Convenience methods to define input and output • Convenience methods to provide fixtures and validate the results
- 41. @gamussa | #kafkasummit | @confluentinc Let’s see some code
- 42. @gamussa | #kafkasummit | @confluentinc
- 43. @gamussa | #kafkasummit | @confluentinc
- 44. @gamussa | #kafkasummit | @confluentinc When TTD doesn’t help? https://gamov.dev/ livestreams-s01e05
- 45. @gamussa | #kafkasummit | @confluentinc Ksqldb for Stream processing
- 46. @gamussa | #kafkasummit | @confluentinc
- 47. @gamussa | #kafkasummit | @confluentinc
- 48. @gamussa | #kafkasummit | @confluentinc Let’s see some code
- 49. @gamussa | #kafkasummit | @confluentinc Conclusion
- 50. @gamussa | #kafkasummit | @confluentinc What did you learn today?
- 51. @gamussa | #kafkasummit | @confluentinc What did you learn today?
- 52. @gamussa | #kafkasummit | @confluentinc What did you learn today? •What’s Kafka apps apps and how to test them
- 53. @gamussa | #kafkasummit | @confluentinc What did you learn today? •What’s Kafka apps apps and how to test them •Producer / Consumer API
- 54. @gamussa | #kafkasummit | @confluentinc What did you learn today? •What’s Kafka apps apps and how to test them •Producer / Consumer API •Kafka Streams
- 55. @gamussa | #kafkasummit | @confluentinc What did you learn today? •What’s Kafka apps apps and how to test them •Producer / Consumer API •Kafka Streams •ksqlDB
- 56. @gamussa | #kafkasummit | @confluentinc What did you learn today? •What’s Kafka apps apps and how to test them •Producer / Consumer API •Kafka Streams •ksqlDB •How containers and TestContainer help a lot
- 57. @gamussa | #kafkasummit | @confluentinc
- 58. @gamussa | #kafkasummit | @confluentinc Want to learn more?
- 59. developer.confluent.io Learn Kafka. Start building with Apache Kafka at Confluent Developer.
- 60. @gamussa | #kafkasummit | @confluentinc ALL UPCOMING MEETUPS NEW EVENT EMAIL ALERTS THE CONFLUENT MEETUP HUB CNFL.IO/MEETUP-HUB VIDEOS OF PAST MEETUPS SLIDES FROM THE TALKS
- 61. @gamussa | #kafkasummit | @confluentinc Confluent Community Slack A vibrant community of over 16,000 members Come along and discuss Apache Kafka and Confluent Platform on dedicated channels including #ksqlDB, #connect, #clients, and more http://cnfl.io/slack
- 62. @gamussa | #kafkasummit | @confluentinc