Messaging Elixir
1 like412 views
The document provides a detailed guide on setting up and using Apache Kafka with Elixir, specifically utilizing the Kafka Ex library. It includes instructions on configuring Docker, cloning the necessary repositories, and running a sample Elixir application for producing and streaming messages through Kafka. Additionally, it features links to resources and tools related to Kafka and Elixir development.
![KafkaEx
|> Elixir Kafka
defmodule DummyProducer.Mixfile do
#…
#Run "mix help compile.app" to learn about applications.
def application do
[
extra_applications: [:kafka_ex]
]
end
# Run "mix help deps" to learn about dependencies.
defp deps do
[{:kafka_ex, "~> 0.7.0”}]
end
end
$ mix deps.get](https://image.slidesharecdn.com/messagingex-170824061438/85/Messaging-Elixir-12-320.jpg)
![KafkaEx on Docker
# docker-compose.yml
version: '2'
services:
# ... #
elixir-app:
build: elixir_app
depends_on:
- kafka-stack
# Dockerfile
FROM elixir
ADD . /app
WORKDIR /app
RUN yes | mix deps.get
# config/config.exs
config :kafka_ex,
brokers: [
{"kafka-stack", 9092} # {"hostname", port}
],
consumer_group: :no_consumer_group,
use_ssl: false](https://image.slidesharecdn.com/messagingex-170824061438/85/Messaging-Elixir-13-320.jpg)
Messaging Elixir
- 2. @kohei_kimura Ɠ IDC IoT DC @kohei_kimura kohei-kimura kohei-kimura.github.io ♥ Visual Studio Code -> ->
- 3. IDCF Cloud - One coin IaaS
- 8. Apache Kafka |> LinkedIn |> |> PubSub |> Pull |> |> |> API
- 9. |> Kafka |> Kafka Topics UI, Kafka Connect UI |> Landoop/fast-data-connect-cluster |> https://fast-data-dev.demo.landoop.com/ Landoop
- 11. Getting Started $ git clone https://github.com/Landoop/fast-data-connect-cluster $ cd fast-data-connect-cluster $ sudo docker-compose up -d
- 12. KafkaEx |> Elixir Kafka defmodule DummyProducer.Mixfile do #… #Run "mix help compile.app" to learn about applications. def application do [ extra_applications: [:kafka_ex] ] end # Run "mix help deps" to learn about dependencies. defp deps do [{:kafka_ex, "~> 0.7.0”}] end end $ mix deps.get
- 13. KafkaEx on Docker # docker-compose.yml version: '2' services: # ... # elixir-app: build: elixir_app depends_on: - kafka-stack # Dockerfile FROM elixir ADD . /app WORKDIR /app RUN yes | mix deps.get # config/config.exs config :kafka_ex, brokers: [ {"kafka-stack", 9092} # {"hostname", port} ], consumer_group: :no_consumer_group, use_ssl: false
- 14. How to use KafkaEx # metadata iex> KafkaEx.metadata # produce iex> KafkaEx.produce("foo", 0, “hey”) # stream iex> KafkaEx.stream("foo", 0, offset: 0, auto_commit: false) |> Enum.take(2) $ docker-compose up -d elixir-app $ docker exec -it $ID /bin/bash # $ID: elixir-app ID # iex -S mix
- 15. |> Apache Kafka: |> KafkaEx: Kafka |> produce, offset, metadata |> Landoop/fast-data-connect-cluster |> Kafka UI docker-compose
- 16. Enjoy your Elixir programming! Please follow me! ->