Apache kafka
1 like1,061 views
Apache Kafka is a fast, scalable, and durable distributed publish-subscribe messaging system. It uses a distributed commit log architecture to allow for publishing and subscribing to streams of records over a cluster of servers. Many large companies use Apache Kafka as the backbone for their real-time data pipelines due to its ability to handle large volumes of data across multiple systems like Hadoop and Storm. While powerful, Kafka does have some downsides like requiring ZooKeeper for coordination and its complex consumer processes.
Related topics:1 of 21
Download to read offline
More Related Content
What's hot (20)
Ad
Viewers also liked (11)
Ad
Similar to Apache kafka (20)
More from the100rabh (6)
Apache kafka
- 1. Apache Kafka Your step 1 to Big Data
- 2. Who am I Saurabh Minni also on the web as @the100rabh Yet another developer in Bangalore I just love tinkering with different technologies Currently working as Technical Architect at Near Been part of planning for Barcamp Bangalore since 2007 Author of Apache Kafka Cookbook - https://www. packtpub.com/big-data-and-business-intelligence/apache- kafka-cookbook
- 3. Features of Kafka ➤ Fast ➤ Scalable ➤ Durable ➤ Distributed by Design
- 6. Architecture of an era Node Processor
- 7. Architecture of an scaling era - II Front End Node Hadoop Front End Node Front End Node
- 8. Things got a little more complicated Front End Node Hadoop Front End Node Front End Node Realtime Processing System
- 9. Apache Kafka to the rescue Front End Node Apache Kafka Front End Node Front End Node Hadoop Apache Storm
- 10. We need more Front End Node Apache Kafka Cluster Front End Node Front End Node Hadoop Apache Storm Apache Kafka Node Apache Kafka Node Apache Kafka Node Apache Kafka Node
- 11. Some more details Kafka Cluster Apache Kafka Node Apache Kafka Node Apache Kafka Node Apache Kafka Node Producer Kafka Topic Producer Kafka Topic Producer Consumer (Hadoop) Consumer (Apache Storm)
- 12. What is Apache Kafka Apache Kafka is publish-subscribe messaging rethought as a distributed commit log.
- 17. Monitoring Kafka
- 18. Which frameworks think Kafka is good
- 19. Which companies use Kafka Probably a better question would be which significantly large companies don’t use Kafka
- 20. Where Apache Kafka gives trouble? ➤ Requires ZooKeeper ➤ Complexity in making a consumer ➤ A lot of manual processes ➤ Management tools leave a lot to be desired