SlideShare a Scribd company logo
Developer’s Guide to
Contributing Code to Kafka
Mickael Maison


Tom Bentley
Who are we?
Mickael


Contributor since 2016


Committer since 2019


PMC member since 2020


Tom


Contributor since 2017


Committer since 2021


PMC member since 2021
Apache Kafka: An Open Source Project
https://github.com/apache/kafka/blob/trunk/CONTRIBUTING.md
Become a contributor!
Collaboration
Dev Mailing List: https://kafka.apache.org/contact
Wiki: https://cwiki.apache.org/confluence/display/KAFKA/Index
Bug tracker: https://issues.apache.org/jira/projects/KAFKA/issues
Code: https://github.com/apache/kafka
“I just need to fix
this one bug”
“I just want to
contribute”
Finding something to work on
Finding something to work on
Long term
involvement
First


Contribution
Small


Changes
Large
Changes
Finding something to work on
Long term
involvement
First


Contribution
Small


Changes
Large
Changes
Contributing Code Changes
Issue Minor?
Contributing Code Changes
Issue PR
Minor?
Contributing Code Changes
Issue
JIRA
PR
Minor?
Same APIs?
Writing code and tests
https://github.com/apache/kafka/blob/trunk/README.md
Gradle Tasks
Compile: assemble
	
	
	
	
2 minutes


Build + run test: build
	
	
	
20-30 minutes


Run test on specific project: clients:test
	
variable


Run specific tests: core:test --tests "*PartitionTest"
Opening a Pull Request
Reviewers: https://kafka.apache.org/committers
Making a Change
Issue
JIRA
PR
Minor?
Same
APIs?
LGTM? Commit
Kafka Improvement Proposals
Issue
JIRA
PR
Minor?
Same
APIs?
LGTM? Commit
KIP
Developer’s guide to contributing code to Kafka with Mickael Maison and Tom Bentley | Kafka Summit London 2022
Developer’s guide to contributing code to Kafka with Mickael Maison and Tom Bentley | Kafka Summit London 2022
Idea & JIRA
Draft &
PoC
Idea & JIRA
Draft &
PoC
Discussion
Idea & JIRA
Draft &
PoC
Discussion
LGTM?
Idea & JIRA
Draft &
PoC
Discussion
LGTM? Voting
Idea & JIRA
Draft &
PoC
Discussion
LGTM? Voting
PR
Idea & JIRA
Accepted?
Anatomy of a KIP
Draft &
PoC
KIP Index lists all the KIPs, and their current state


Standard sections:


Status


Motivation


Public interfaces


Proposed changes


Compatibility, Deprecation, Migration


Test Plan


Rejected alternatives
Anatomy of a KIP - Status section
Draft &
PoC
Anatomy of a KIP - Motivation section
Example:


KIP-730: Producer ID generation in KRaft mode
Draft &
PoC
Anatomy of a KIP - Public interfaces
Example:


KIP-664 Provide tooling to detect and abort hanging transactions
Draft &
PoC
Anatomy of a KIP - Proposed changes
Example:


KIP-98: Exactly Once Delivery and Transactional Messaging
Draft &
PoC
Anatomy of a KIP - Compatibility, Deprecation, Migration
Example:


KIP-802 Validation Support for Kafka Connect SMT and
Converter Options
Draft &
PoC
Anatomy of a KIP - Test plan
Example:


None! This is new


Draft &
PoC
Anatomy of a KIP - Rejected alternatives
Example:


KIP-618: Exactly Once support for Source Connectors
Draft &
PoC
To: dev@kafka.apache.org


Subject: [DISCUSS] KIP-XXX: My great idea
Discussion
To: dev@kafka.apache.org


Subject: [VOTE] KIP-XXX: My great idea
Voting
Developer’s guide to contributing code to Kafka with Mickael Maison and Tom Bentley | Kafka Summit London 2022
Interminable discussion
Real problem?


Clear + strong motivation?


Too narrow?


Too broad?


Listened to feedback?
Contributing regularly
Thank you!
Key Resources:


https://kafka.apache.org/contributing


https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals


https://kafka.apache.org/committers


https://github.com/apache/kafka

More Related Content

PPTX
Hadoop Hive Tutorial | Hive Fundamentals | Hive Architecture
PDF
Apache Spark Data Source V2 with Wenchen Fan and Gengliang Wang
PDF
Understanding Apache Kafka® Latency at Scale
PDF
How to Extend Apache Spark with Customized Optimizations
PDF
InfluxDB IOx Tech Talks: Query Processing in InfluxDB IOx
PDF
Apache Iceberg - A Table Format for Hige Analytic Datasets
PPTX
Dynamic Rule-based Real-time Market Data Alerts
PPTX
Apache Kafka
Hadoop Hive Tutorial | Hive Fundamentals | Hive Architecture
Apache Spark Data Source V2 with Wenchen Fan and Gengliang Wang
Understanding Apache Kafka® Latency at Scale
How to Extend Apache Spark with Customized Optimizations
InfluxDB IOx Tech Talks: Query Processing in InfluxDB IOx
Apache Iceberg - A Table Format for Hige Analytic Datasets
Dynamic Rule-based Real-time Market Data Alerts
Apache Kafka

What's hot (20)

PPTX
The openCypher Project - An Open Graph Query Language
PPTX
Kafka 101
PDF
Cloud-Native Apache Spark Scheduling with YuniKorn Scheduler
PDF
Iceberg: A modern table format for big data (Strata NY 2018)
PDF
Kafka 101 and Developer Best Practices
PDF
Top 5 Event Streaming Use Cases for 2021 with Apache Kafka
PDF
Improving Data Locality for Spark Jobs on Kubernetes Using Alluxio
PDF
Introducing the Apache Flink Kubernetes Operator
PDF
Scaling Apache Pulsar to 10 PB/day
PDF
From Zero to Hero with Kafka Connect
PDF
Apache Spark At Scale in the Cloud
PDF
InfluxDB IOx Tech Talks: Intro to the InfluxDB IOx Read Buffer - A Read-Optim...
PDF
Can Apache Kafka Replace a Database?
PDF
2022-06-23 Apache Arrow and DataFusion_ Changing the Game for implementing Da...
PPTX
Migrating your clusters and workloads from Hadoop 2 to Hadoop 3
PPTX
Introduction to Apache Flink
PPTX
Managing 2000 Node Cluster with Ambari
PDF
Introduction to Presto at Treasure Data
PPTX
Exactly-Once Financial Data Processing at Scale with Flink and Pinot
PDF
A Deep Dive into Query Execution Engine of Spark SQL
The openCypher Project - An Open Graph Query Language
Kafka 101
Cloud-Native Apache Spark Scheduling with YuniKorn Scheduler
Iceberg: A modern table format for big data (Strata NY 2018)
Kafka 101 and Developer Best Practices
Top 5 Event Streaming Use Cases for 2021 with Apache Kafka
Improving Data Locality for Spark Jobs on Kubernetes Using Alluxio
Introducing the Apache Flink Kubernetes Operator
Scaling Apache Pulsar to 10 PB/day
From Zero to Hero with Kafka Connect
Apache Spark At Scale in the Cloud
InfluxDB IOx Tech Talks: Intro to the InfluxDB IOx Read Buffer - A Read-Optim...
Can Apache Kafka Replace a Database?
2022-06-23 Apache Arrow and DataFusion_ Changing the Game for implementing Da...
Migrating your clusters and workloads from Hadoop 2 to Hadoop 3
Introduction to Apache Flink
Managing 2000 Node Cluster with Ambari
Introduction to Presto at Treasure Data
Exactly-Once Financial Data Processing at Scale with Flink and Pinot
A Deep Dive into Query Execution Engine of Spark SQL
Ad

Similar to Developer’s guide to contributing code to Kafka with Mickael Maison and Tom Bentley | Kafka Summit London 2022 (11)

PDF
Welcome to Kafka; We’re Glad You’re Here (Dave Klein, Centene) Kafka Summit 2020
PDF
Virtual Bash! A Lunchtime Introduction to Kafka
PDF
Big Data Day LA 2016/ Hadoop/ Spark/ Kafka track - Introduction to Kafka - Je...
PPTX
Introduction Apache Kafka
PDF
Fundamentals of Apache Kafka
PDF
Introduction à kafka
PPTX
Being Ready for Apache Kafka - Apache: Big Data Europe 2015
PDF
Best Apache Kafka Training in Bangalore. Join myTectra
PPTX
kafka_session_updated.pptx
PPTX
Apache kafka
DOCX
Apache kafka configuration-guide
Welcome to Kafka; We’re Glad You’re Here (Dave Klein, Centene) Kafka Summit 2020
Virtual Bash! A Lunchtime Introduction to Kafka
Big Data Day LA 2016/ Hadoop/ Spark/ Kafka track - Introduction to Kafka - Je...
Introduction Apache Kafka
Fundamentals of Apache Kafka
Introduction à kafka
Being Ready for Apache Kafka - Apache: Big Data Europe 2015
Best Apache Kafka Training in Bangalore. Join myTectra
kafka_session_updated.pptx
Apache kafka
Apache kafka configuration-guide
Ad

More from HostedbyConfluent (20)

PDF
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
PDF
Renaming a Kafka Topic | Kafka Summit London
PDF
Evolution of NRT Data Ingestion Pipeline at Trendyol
PDF
Ensuring Kafka Service Resilience: A Dive into Health-Checking Techniques
PDF
Exactly-once Stream Processing with Arroyo and Kafka
PDF
Fish Plays Pokemon | Kafka Summit London
PDF
Tiered Storage 101 | Kafla Summit London
PDF
Building a Self-Service Stream Processing Portal: How And Why
PDF
From the Trenches: Improving Kafka Connect Source Connector Ingestion from 7 ...
PDF
Future with Zero Down-Time: End-to-end Resiliency with Chaos Engineering and ...
PDF
Navigating Private Network Connectivity Options for Kafka Clusters
PDF
Apache Flink: Building a Company-wide Self-service Streaming Data Platform
PDF
Explaining How Real-Time GenAI Works in a Noisy Pub
PDF
TL;DR Kafka Metrics | Kafka Summit London
PDF
A Window Into Your Kafka Streams Tasks | KSL
PDF
Mastering Kafka Producer Configs: A Guide to Optimizing Performance
PDF
Data Contracts Management: Schema Registry and Beyond
PDF
Code-First Approach: Crafting Efficient Flink Apps
PDF
Debezium vs. the World: An Overview of the CDC Ecosystem
PDF
Beyond Tiered Storage: Serverless Kafka with No Local Disks
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Renaming a Kafka Topic | Kafka Summit London
Evolution of NRT Data Ingestion Pipeline at Trendyol
Ensuring Kafka Service Resilience: A Dive into Health-Checking Techniques
Exactly-once Stream Processing with Arroyo and Kafka
Fish Plays Pokemon | Kafka Summit London
Tiered Storage 101 | Kafla Summit London
Building a Self-Service Stream Processing Portal: How And Why
From the Trenches: Improving Kafka Connect Source Connector Ingestion from 7 ...
Future with Zero Down-Time: End-to-end Resiliency with Chaos Engineering and ...
Navigating Private Network Connectivity Options for Kafka Clusters
Apache Flink: Building a Company-wide Self-service Streaming Data Platform
Explaining How Real-Time GenAI Works in a Noisy Pub
TL;DR Kafka Metrics | Kafka Summit London
A Window Into Your Kafka Streams Tasks | KSL
Mastering Kafka Producer Configs: A Guide to Optimizing Performance
Data Contracts Management: Schema Registry and Beyond
Code-First Approach: Crafting Efficient Flink Apps
Debezium vs. the World: An Overview of the CDC Ecosystem
Beyond Tiered Storage: Serverless Kafka with No Local Disks

Recently uploaded (20)

PDF
Per capita expenditure prediction using model stacking based on satellite ima...
PDF
TokAI - TikTok AI Agent : The First AI Application That Analyzes 10,000+ Vira...
PDF
A comparative analysis of optical character recognition models for extracting...
PDF
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
PPTX
ACSFv1EN-58255 AWS Academy Cloud Security Foundations.pptx
PDF
Peak of Data & AI Encore- AI for Metadata and Smarter Workflows
PDF
cuic standard and advanced reporting.pdf
DOCX
The AUB Centre for AI in Media Proposal.docx
PPTX
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
PPT
Teaching material agriculture food technology
PDF
Agricultural_Statistics_at_a_Glance_2022_0.pdf
PDF
Building Integrated photovoltaic BIPV_UPV.pdf
PDF
MIND Revenue Release Quarter 2 2025 Press Release
PDF
Encapsulation theory and applications.pdf
PDF
Encapsulation_ Review paper, used for researhc scholars
PDF
Unlocking AI with Model Context Protocol (MCP)
PPTX
sap open course for s4hana steps from ECC to s4
PDF
Optimiser vos workloads AI/ML sur Amazon EC2 et AWS Graviton
PDF
Spectral efficient network and resource selection model in 5G networks
PDF
Profit Center Accounting in SAP S/4HANA, S4F28 Col11
Per capita expenditure prediction using model stacking based on satellite ima...
TokAI - TikTok AI Agent : The First AI Application That Analyzes 10,000+ Vira...
A comparative analysis of optical character recognition models for extracting...
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
ACSFv1EN-58255 AWS Academy Cloud Security Foundations.pptx
Peak of Data & AI Encore- AI for Metadata and Smarter Workflows
cuic standard and advanced reporting.pdf
The AUB Centre for AI in Media Proposal.docx
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
Teaching material agriculture food technology
Agricultural_Statistics_at_a_Glance_2022_0.pdf
Building Integrated photovoltaic BIPV_UPV.pdf
MIND Revenue Release Quarter 2 2025 Press Release
Encapsulation theory and applications.pdf
Encapsulation_ Review paper, used for researhc scholars
Unlocking AI with Model Context Protocol (MCP)
sap open course for s4hana steps from ECC to s4
Optimiser vos workloads AI/ML sur Amazon EC2 et AWS Graviton
Spectral efficient network and resource selection model in 5G networks
Profit Center Accounting in SAP S/4HANA, S4F28 Col11

Developer’s guide to contributing code to Kafka with Mickael Maison and Tom Bentley | Kafka Summit London 2022