Kafka at the Edge: an IoT scenario with OpenShift Streams for Apache Kafka | DevNation Tech Talk
2 likes2,438 views
This document discusses Apache Kafka and Red Hat OpenShift Streams for Apache Kafka. It begins with an overview of what Apache Kafka is and its common use cases. It then demonstrates how Red Hat OpenShift Streams provides a managed Apache Kafka cluster as a service, including a dedicated cluster, configuration management, metrics, monitoring and other features to provide a streamlined developer experience. It concludes with information on trying OpenShift Streams for Apache Kafka and additional resources.
Kafka at the Edge: an IoT scenario with OpenShift Streams for Apache Kafka | DevNation Tech Talk
- 1. Kafka at the Edge: An IoT scenario with OpenShift Streams for Apache Kafka Bernard Tison Red Hat 1
- 2. Agenda 2 Agenda â–¸ What is Apache Kafka, and how are we using it? â–¸ IoT use case â–¸ Demo â–¸ Questions
- 3. 3 Apache Kafka is a distributed system designed for streams. It is built to be an high-availability, horizontally-scalable, fault-tolerant, commit log, and allows distributed data streams and stream processing applications. Known use cases are: â–¸ High-performance data pipelines â–¸ Streaming analytics â–¸ Data integration â–¸ Event bus in Event Driven Architecture What is Kafka? What is Apache Kafka? Open-source distributed event streaming platform Learn more on the Apache Kafka community page: https://kafka.apache.org/
- 4. Source: https://kafka.apache.org/ Brokers, topics, partitions, producers, and consumers! What is Kafka? 4 What is Apache Kafka? Kafka Cluster Broker A Broker B Broker C Consumer Group Topic: Clicks Topic: Purchases P1 P2 P3 P1 P2 P3 Producers
- 5. Source: https://kafka.apache.org/intro What is Kafka? 5 APIs What is Apache Kafka? â–¸ The Admin API to manage and inspect topics, brokers, and other Kafka objects. â–¸ The Producer API to publish (write) a stream of events to one or more Kafka topics. â–¸ The Consumer API to subscribe to (read) one or more topics and to process the stream of events produced to them. â–¸ The Kafka Streams API to implement stream processing applications and microservices. â–¸ The Kafka Connect API to build and run reusable data import/export connectors that consume (read) or produce (write) streams of events from and to external systems and applications so they can integrate with Kafka.
- 6. High-throughput, high-availability, and decoupled data pipeline. What is Kafka? 6 Hub and Spoke Architecture 10s of thousands of events Producers/Data Sources Consumers/Data Sinks
- 7. OpenShift Streams for Apache Kafka 7 https://red.ht/TryKafka Development Preview trial available for free at:
- 8. Red Hat OpenShift Streams for Apache Kafka Complete solution for stream-based applications OpenShift Streams for Apache Kafka 8 Metrics & monitoring Configuration mgmt Hosted & managed (99.95% SLA) Kafka cluster Broker(s) Topic(s) Streamlined developer experience: a curated solution with a developer-first, consistent experience Delivered as a service, managed by Red Hat SRE - 24x7 global support and a 99.95% service-level agreement (SLA) Real-time, streaming data broker - Dedicated Apache Kafka cluster deliver as a service in the cloud and location of choice â–¸ Access to Kafka brokers, topics, and partitions â–¸ Configuration management â–¸ Metrics and monitoring â–¸ UI / CLI / API / service bindings â–¸ Integrated identity & access management STREAMLINED DEVELOPER EXPERIENCE UI API CLI Service Binding https://red.ht/TryKafka
- 9. 9 â–¸ Based on the Strimzi community project that provides an optimized version of Apache Kafka on Kubernetes â–¸ Access to customer dedicated Kafka instances. Each instance provides Kafka core capabilities â–¸ Kafka instances are configured based on opinionated recommendations made by Red Hat experts OpenShift Streams for Apache Kafka A dedicated, fully managed, Apache Kafka cluster An opinionated experience for your Kafka service Apache Kafka implementation optimized for OpenShift https://red.ht/TryKafka
- 10. 10 There is More â–¸ AMQ Broker: MQTT broker â–¸ Apache Camel: bridge MQTT to Kafka â–¸ Debezium: captures row-level changes to database tables and passes corresponding change events to Kafka â–¸ GraphQL: query language and server-side runtime for APIs that prioritizes giving clients exactly the data they request and no more.
- 11. 11
- 13. 13 Product Overview Try it out! https://red.ht/TryKafka
- 14. 14 Resources â–¸ GitHub repo: https://github.com/rh-bu-cs-rhosak-iot (WIP) â–¸ Try Kafka: https://red.ht/TryKafka â–¸ Streams for Apache Kafka YouTube playlist: https://www.youtube.com/playlist?list=PLf3vm0UK6HKqZ3Vi7h1Ynfbi0TpdXUr25 â–¸ Streams for Apache Kafka getting started blog post: https://developers.redhat.com/articles/2021/07/07/getting-started-red-hat-openshift-str eams-apache-kafka
- 15. linkedin.com/company/red-hat youtube.com/user/RedHatVideos facebook.com/redhatinc twitter.com/RedHat 15 Red Hat is the world’s leading provider of enterprise open source software solutions. Award-winning support, training, and consulting services make Red Hat a trusted adviser to the Fortune 500. Thank you