Big Data Day LA 2016/ Hadoop/ Spark/ Kafka track - Introduction to Kafka - Jesse Anderson, CEO, Smoking Hand
1 like293 views
Kafka is a distributed publish-subscribe system that uses a commit log to track changes. It was originally created at LinkedIn and open sourced in 2011. Kafka decouples systems and is commonly used in enterprise data flows. The document then demonstrates how Kafka works using Legos and discusses key Kafka concepts like topics, partitioning, and the commit log. It also provides examples of how to create Kafka producers and consumers using the Java API.
![while (true) {
// Poll for ConsumerRecords for a certain amount of time
ConsumerRecords<String, String> records = consumer.poll(100);
// Process the ConsumerRecords, if any, that came back
for (ConsumerRecord<String, String> record : records) {
String key = record.key();
String value = record.value();
// Do something with message
}
}
}
public void close() {
consumer.close();
}
public static void main(String[] args) {
MyConsumer consumer = new MyConsumer();
consumer.createConsumer();
consumer.close();
}
}
Creating a Consumer (2/2)](https://image.slidesharecdn.com/rxxrppglrak8axcece1p-signature-f79912d1da0371c7c24aade80c94712e49a4a2faae0d4cecc36ea1d84796d304-poli-160812183140/85/Big-Data-Day-LA-2016-Hadoop-Spark-Kafka-track-Introduction-to-Kafka-Jesse-Anderson-CEO-Smoking-Hand-17-320.jpg)
Big Data Day LA 2016/ Hadoop/ Spark/ Kafka track - Introduction to Kafka - Jesse Anderson, CEO, Smoking Hand
- 2. Kafka
- 3. About Kafka
- 13. Kafka API