Open Source IoT Project Flogo - Building a Custom Apache Kafka Connector
1 like1,121 views
This document provides a guide for developing custom Flogo connectors, particularly using Apache Kafka as an example. It emphasizes the ease and speed of development, as well as the importance of utilizing existing Flogo connectors as templates and contributing to the open-source community. Key steps include installing a corresponding Go library, creating and testing the connector, and sharing it on GitHub.
Open Source IoT Project Flogo - Building a Custom Apache Kafka Connector
- 2. © Copyright 2000-2016 TIBCO Software Inc. This is not an introduction to Open Source IoT Project Flogo www.flogo.io https://community.tibco.com/wiki/flogo- introduction-slides-and-videos
- 3. © Copyright 2000-2016 TIBCO Software Inc. • is very easy, fast and lightweight • can be done without much experience in Go programming language • is best done by using other Flogo connectors (and unit tests) as template • should be contributed to the Flogo open source community (not a must, of course!) Key Takeaways Building custom Flogo connectors…
- 4. Flogo Concepts Trigger Flow Activity Error Handler Flogo App One or more Flows Flows Activities + Transitions + Optional error handler Triggers Starts those flows
- 5. Flogo Repositories on Github flogo-lib Core libraries. flogo-services Backing services required by Flogo for flow and state management. flogo-contrib Flogo extensions available out of the box, like triggers or activities. flogo-cli A command line tool to build Flogo apps. https://github.com/TIBCOSoftware/flogo (very permissive open source BSD license) This projects includes Triggers and Activities go get github.com/TIBCOSoftware/flogo-contrib/...
- 6. © Copyright 2000-2016 TIBCO Software Inc. Apache Kafka https://kafka.apache.org/ “Kafka is used for building real-time data pipelines and streaming apps. It is horizontally scalable, fault-tolerant, wicked fast, and runs in production in thousands of companies.”
- 7. © Copyright 2000-2016 TIBCO Software Inc. • Get Access to a technology which you want to integrate • For Kafka, this could be a locally installed broker, or a remote connection • Find, install and use a corresponding Go library • Various Golang APIs are available for Apache Kafka • I chose the Kafka API from optiopay because it is very simply and easy to use – depending on requirements regarding functionality, performance or maturity, other options might be better (I did not do more research) • Install the library into your Go workspace (command ‘go install’) • Run an “hello world“ example (optiopay delivers a very nice simple demo where you send and consume messages from command line) • Duplicate an existing Flogo Activity • Choose a similar Flogo Activity (e.g. Kafka is similar to MQTT, you connect to a broker and send messages) • The folder includes implementation, test, runtime config and UI config • Use the existing folder as template and re-write / update / replace / enhance the code + config + test • Run “go test” to validate correctness of the new connector • In my Kafka test, I see that the test sends a new message to Kafka (verified in a Kafka consumer running on the command line) • Deploy and test the new connector in the Flogo Web UI • The web UI allows to install a new activity (or trigger) via URL Link (you need to publish your connector on Github) • Github Pull Request to contribute your code back to the Flogo community • (optional step, but highly recommended J) Steps to develop a custom connector
- 8. © Copyright 2000-2016 TIBCO Software Inc. Live Demo Development of a Flogo Activity for Apache Kafka
- 9. © Copyright 2000-2016 TIBCO Software Inc. • is very easy, fast and lightweight • can be done without much experience in Go programming language • is best done by using other Flogo connectors (and unit tests) as template • should be contributed to the Flogo open source community (not a must, of course!) Key Takeaways Building custom Flogo connectors…
- 10. Questions? Please contact me! Kai Wähner Technology Evangelist kontakt@kai-waehner.de @KaiWaehner www.kai-waehner.de LinkedIn Ask questions on the Flogo community: https://community.tibco.com/products/project-flogo