Timothy Spann [StreamNative] | Using FLaNK with InfluxDB for EdgeAI IoT at Scale | InfluxDays NA 2021
0 likes264 views
The document discusses the use of Apache Flink and Apache Pulsar with InfluxDB for scalable edge AI and IoT applications. It highlights StreamNative's cloud-native event streaming platform, which supports real-time messaging and data integration across various cloud environments. Additionally, it provides technical instructions for setting up and managing InfluxDB sinks and general functionalities of the associated technologies.
![Timothy Spann [StreamNative] | Using FLaNK with InfluxDB for EdgeAI IoT at Scale | InfluxDays NA 2021](https://image.slidesharecdn.com/timothyspannusingflipwithinfluxdbforedgeaiiotatscale-211022193410/85/Timothy-Spann-StreamNative-Using-FLaNK-with-InfluxDB-for-EdgeAI-IoT-at-Scale-InfluxDays-NA-2021-1-320.jpg)
Timothy Spann [StreamNative] | Using FLaNK with InfluxDB for EdgeAI IoT at Scale | InfluxDays NA 2021
- 2. Using FLaNK with InfluxDB for EdgeAI IoT at Scale Tim Spann | Developer Advocate
- 3. streamnative.io Tim Spann, Developer Advocate DZone Zone Leader and Big Data MVB Data DJay
- 4. streamnative.io Founded by the original developers of Apache Pulsar and Apache BookKeeper, StreamNative builds a cloud-native event streaming platform that enables enterprises to easily access data as real-time event streams.
- 5. streamnative.io Apache is an open source, cloud-native distributed messaging and streaming platform.
- 6. streamnative.io Powered by Apache Pulsar, StreamNative provides a cloud-native, real-time messaging and streaming platform to support multi-cloud and hybrid cloud strategies. Built for Containers Cloud Native Flink SQL StreamNative Cloud
- 7. © 2021 InfluxData Inc. All Rights Reserved. ● Apache Flink ● Apache Pulsar ● StreamNative's Flink Connector for Pulsar ● Apache NiFi ● Apache +++ FLiP(N) Stack
- 8. © 2021 InfluxData Inc. All Rights Reserved. Apache Pulsar InfluxDB Sink https:/ /pulsar.apache.org/docs/en/io-influxdb-sink/
- 9. © 2021 InfluxData Inc. All Rights Reserved. Apache Pulsar InfluxDB Sink bin/pulsar-admin sink stop --name influxdb-sink-jetson --namespace default --tenant public bin/pulsar-admin sinks delete --tenant public --namespace default --name influxdb-sink-jetson bin/pulsar-admin sinks create --archive ./connectors/pulsar-io-influxdb-2.8.0.nar --tenant public --namespace default --name influxdb-sink-jetson --sink-config-file conf/influxcloud.yml --inputs jetsoninflux --parallelism 1 bin/pulsar-admin sinks get --tenant public --namespace default --name influxdb-sink-jetson bin/pulsar-admin sinks status --tenant public --namespace default --name influxdb-sink-jetson https:/ /pulsar.apache.org/docs/en/io-influxdb-sink/
- 10. © 2021 InfluxData Inc. All Rights Reserved. InfluxDB Telegraf MQTT https:/ /www.influxdata.com/integration/mqtt-monitoring/ https:/ /github.com/streamnative/mop MQTT on Pulsar (MoP)
- 11. © 2021 InfluxData Inc. All Rights Reserved. InfluxDB Telegraf RabbitMQ https:/ /www.influxdata.com/integration/mqtt-monitoring/ https:/ /github.com/streamnative/aop AMQP on Pulsar (AoP) https:/ /hub.streamnative.io/connectors/amqp-1-0-sink/ https:/ /hub.streamnative.io/connectors/amqp-1-0-source
- 12. streamnative.io • Guaranteed delivery • Data buffering - Backpressure - Pressure release • Prioritized queuing • Flow specific QoS - Latency vs. throughput - Loss tolerance • Data provenance • Supports push and pull models • Hundreds of processors • Visual command and control • Over a 300 sources • Flow templates • Pluggable/multi-role security • Designed for extension • Clustering • Version Control Why Apache NiFi?
- 13. © 2021 InfluxData Inc. All Rights Reserved. InfluxDB via Apache NiFi https:/ /www.influxdata.com/integration/mqtt-monitoring/ https:/ /github.com/streamnative/mop https://github.com/influxdata/nifi-influxdb-bundle https://www.influxdata.com/blog/building-a-data-stream-for-iot-with-nifi-and-influxdb/
- 14. streamnative.io ● Unified computing engine ● Batch processing is a special case of stream processing ● Stateful processing ● Massive Scalability ● Flink SQL for queries, inserts against Pulsar Topics ● Streaming Analytics ● Continuous SQL ● Continuous ETL ● Complex Event Processing ● Standard SQL Powered by Apache Calcite Why Apache Flink?
- 16. StreamNative Hub StreamNative Cloud Unified Batch and Stream COMPUTING Batch (Batch + Stream) Unified Batch and Stream STORAGE Offload (Queuing + Streaming) Apache Flink - Apache Pulsar - Apache NiFi <-> Devices <-> influxdb Tiered Storage Pulsar --- KoP --- MoP --- Websocket --- HTTP Pulsar Sink Pulsar Sink Streaming Edge Gateway Protocols End-to-End Streaming Edge App
- 18. NVIDIA Device
- 19. Apache NiFi
- 20. Apache NiFi Data to Enhance IoT Stream
- 21. Apache NiFi
- 22. Apache NiFi Influxdb properties Controller properties
- 23. Apache NiFi
- 24. Apache NiFi
- 25. Apache NiFi
- 26. Apache Pulsar
- 43. ● https://github.com/tspannhw/FLiP-InfluxDB ● https://pulsar.apache.org/docs/en/io-influxdb-sink/ ● https://pulsar.apache.org/docs/en/io-overview/ ● https://github.com/tspannhw/Flip-solr ● https://github.com/tspannhw/Flip-transit ● https://github.com/tspannhw/Flip-iot ● https://www.datainmotion.dev/2020/10/flank-strea ming-edgeai-on-new-nvidia.html ● https://github.com/tspannhw/minifi-jetson-nano ● https://github.com/tspannhw/SpeakerProfile/ ● https://hub.streamnative.io/connectors/influxdb-sin k/2.5.1/ ● https://github.com/tspannhw/minifi-xaviernx/
- 44. Let’s Keep in Touch! Speaker Name Speaker title @PaasDev https://www.linkedin.com/in/timothyspann https://github.com/tspannhw
- 45. Questions?
- 46. Thank You