A prototype of utilizing Apache Kafka and Lightweight M2M protocol as the backbone for cloud/edge IoT integration
1 like242 views
The document discusses a prototype utilizing Apache Kafka and the Lightweight M2M (LwM2M) protocol for cloud/edge IoT integration, emphasizing Kafka's scalability, fault-tolerance, and real-time processing capabilities. It highlights Kafka's advantages for edge computing, such as local message storage and data replication, while also outlining LwM2M's support for resource-constrained devices and its open standard nature. The document provides resources for further exploration, including a blog post and a GitHub project for practical experimentation.
A prototype of utilizing Apache Kafka and Lightweight M2M protocol as the backbone for cloud/edge IoT integration
- 1. A prototype of utilizing Apache Kafka and Lightweight M2M protocol as the backbone for cloud/edge IoT integration Christos Vasilakis Consultant Twitter: @cvasilak
- 3. Why Kafka? ● Entered mature phase by providing a rich ecosystem of frameworks to build upon. ● Easy for programmers to get started by simple programming constructs and semantics. ● Supporting next gen. Real Time Streaming and Event Driven architectures. ● First class support by many real-time analytical frameworks e.g. Kafka Streams, KSQL, Spark or Flink. ● Scalability and fault-tolerance built-in in the components with excessive list of knobs to adjust for specific requirements and use cases.
- 4. Why Kafka on the edge? ● Support of replication of data between brokers in the form of Apache MirrorMaker or Confluent’s own Replicator. ● Eliminating another dependency to a database to store the messages since they are stored to Kafka itself (single source of truth). ● Disconnected and offline operation of edge gateways since messages are stored locally at the edge and replicated to the cloud when connection resumes. ● Ability to deploy Kafka Stream analytics at the edge for real-time processing of incoming messages. ● A running Kafka broker at the edge allows us to develop new event-driven microservices to support new requirements, with familiar programming model.
- 5. Why LightweightM2M (LWM2M) ? ● Supports embedded hardware with limited resources and limited bandwidth connectivity. Both Zephyr and Contiki-ng operating systems supports it and can be deployed to many types of hardware devices. ● Designed by telecom providers to support real-world scenarios in managing million of devices at the field, so it’s usage and limitations are well understood and documented. ● The specification is an open standard and actively developed incorporating newest developments such as support for LORA and NB-IoT. ● Availability of robust implementations for a variety of languages which makes it easy for the developer to get started. ● The protocol provides enough flexibility in it’s design that allows routing of many existing protocols such as Modbus or OPC UA. Check leshan-modbus-adapter github page for more details
- 6. Resources ● Blog post describing the architecture https://bit.ly/kafka-lwm2m ● Github project with a detailed setup guide to experiment https://github.com/zeelos/zeelos
- 7. Demo
- 8. Thank you