3 Kafka patterns to deliver Streaming Machine Learning models with Andrea Spina | Kafka Summit London 2022
0 likes1,158 views
The document discusses three key Kafka patterns for delivering streaming machine learning models presented at Kafka Summit 2022 by Andrea Spina. It highlights the benefits of online learning, including low computational requirements and robustness to concept drift, as well as the use of River ML and Seldon Core for dynamic model serving and async scoring capabilities. The importance of feedback mechanisms in streaming ML via Kafka is also emphasized, detailing the data structure and user interactions necessary for effective model predictions.
![Synonyms: online, incremental, sequential.
Many algorithms already present online
behavior.
Benefits: low computational requirements
(green AI [2]
), robust to concept drift
Introducing Streaming ML
Machine learning models that are able to update one sample at a
time[1]
. Traditionally, machine learning has been a batch task.](https://image.slidesharecdn.com/lt05andreaspinaradicalbit-220503182804/85/3-Kafka-patterns-to-deliver-Streaming-Machine-Learning-models-with-Andrea-Spina-Kafka-Summit-London-2022-4-320.jpg)
![Async scoring unfolds to
3x scoring rate[3]
Kafka REST
#KS replicas
to hit max.
throughput
GKE, 1 node, e2-standard8
Kafka streams job,
single-threaded
DummyModel on 1-replica
SeldonDeployment
max throughput 450 req/sec 1
3
2](https://image.slidesharecdn.com/lt05andreaspinaradicalbit-220503182804/85/3-Kafka-patterns-to-deliver-Streaming-Machine-Learning-models-with-Andrea-Spina-Kafka-Summit-London-2022-15-320.jpg)
![references
[1] - Online Machine Learning
[2] - Green AI
[3] - https://github.com/radicalbit/kafkasummit2022.git](https://image.slidesharecdn.com/lt05andreaspinaradicalbit-220503182804/85/3-Kafka-patterns-to-deliver-Streaming-Machine-Learning-models-with-Andrea-Spina-Kafka-Summit-London-2022-23-320.jpg)
3 Kafka patterns to deliver Streaming Machine Learning models with Andrea Spina | Kafka Summit London 2022
- 1. 3 Kafka Patterns to Deliver Streaming Machine Learning Models Kafka Summit 2022, Europe Andrea Spina, CTO @Radicalbit
- 4. Synonyms: online, incremental, sequential. Many algorithms already present online behavior. Benefits: low computational requirements (green AI [2] ), robust to concept drift Introducing Streaming ML Machine learning models that are able to update one sample at a time[1] . Traditionally, machine learning has been a batch task.
- 5. River ML The “sklearn project” for streaming machine learning Seldon Core Enables k8s-based MLOps
- 7. 1. Dynamic Models Serving i.e. runtime adding, removing, replacing ML models
- 8. join S C . model_ID . location . schema
- 9. Performing the wrong regular join
- 10. Performing the right (map-side) join global Ktable
- 11. 2. Scoring API with seldon-core Made with Kafka
- 12. Scoring API with Seldon Core
- 13. Seldon Core enables async scoring
- 14. Seldon offers out-of-the-box Kafka support k8s pod Pod belonging to SeldonDeployment instance seldon sidecar seldon-container-engine It proxies all the in/out communication
- 15. Async scoring unfolds to 3x scoring rate[3] Kafka REST #KS replicas to hit max. throughput GKE, 1 node, e2-standard8 Kafka streams job, single-threaded DummyModel on 1-replica SeldonDeployment max throughput 450 req/sec 1 3 2
- 16. 3. Feedback API, explained
- 17. DATA STRUCTURE ● feature vector ● true class FEEDBACK SOURCES ● applications (browser, backends) ● from File How do we handle feedback with Kafka?
- 18. Online predictor operator From the user perspective, it’s about a binary operator taking: ● The to be predicted stream and ● a feedback stream
- 21. Streaming ML with Kafka async scoring
- 23. references [1] - Online Machine Learning [2] - Green AI [3] - https://github.com/radicalbit/kafkasummit2022.git