FIWARE Global Summit - DRACO: Managing the Stream of Context Information History Using Apache NIFI
0 likes279 views
The document describes FIWARE Draco, a data ingestion and persistence system based on Apache NiFi, designed for processing and distributing data efficiently. It supports NGSI v2 for seamless integration with the Orion context broker and offers multiple persistent backends like MySQL, MongoDB, and HDFS. Future developments include new data persistence modes, enhanced support for various databases, and a more secure Docker image.
FIWARE Global Summit - DRACO: Managing the Stream of Context Information History Using Apache NIFI
- 1. FIWARE Draco Joaquin Salvachua (jsalvachua@dit.upm.es) Andres Muñoz (jamunoz@dit.upm.es) Sonsoles López Pernas (slopez@dit.upm.es) Gabriel Huecas (ghuecas@dit.upm.es) es) Context Management (Core) Chapter Universidad Politécnica de Madrid
- 2. Draco ● The Draco Generic Enabler takes care of the data ingestion and persistence. Is a is an easy to use, powerful, and reliable system for processing and distributing data. Internally, Draco is based on Apache NiFi. ● NiFi is a dataflow system based on the concepts of flow-based programming. It supports powerful and scalable directed graphs of data routing, transformation, and system mediation logic. It was built to automate the flow of data between systems. 2
- 4. Fiware Draco MiniNiFI (low profile version) 4
- 5. Draco ● Based on Apache NiFi. ● NGSI 2 Support both for ingestion and serialization to have full integration with the Orion Context Broker. ● Several persistent backends : ● MySQL, the well-know relational database manager. ● MongoDB, the NoSQL document-oriented database. ● PostgreSQL, the well-know relational database manager. ● HDFS, Hadoop distributed file system. ● Templates for some common scenarios ● Rest API 5
- 6. 6
- 7. Features ● Currently ○ Based on Apache NiFi. ○ NGSIv2 Support both for ingestion and serialization to have full integration with the Orion Context bróker. ○ Several persistant backend : ■ MySQL, the well-know relational database manager. ■ MongoDB, the NoSQL document-oriented database. ■ PostgreSQL, the well-know relational database manager. ■ HDFS Hadoop file system ○ Templates for some common scenarios ○ Rest API ● Getting started : https://fiware-draco.readthedocs.io/en/latest/quick_start_guide/index.html 7
- 8. Draco Roadmap next release ● Working on new persistence mode (column) models for each available processors ● Ongoing research of new data models in order to work with NGSI-LD ● Working on developing new processors for distributed databases, cloud based databases and geo databases : ● NGSIToDynamoDb ● NGSIToCartoDB ● NGSIToCassandra ● Ongoing research for working with time-series and graph-oriented databases ● Advance features for data persistence (batching) for current processors ● Working on develop a new docker image based on alpine for reducing the vulnerabilities presented in the official docker image of NIFI 9
- 12. Thank you!