DDS-to-JSON and DDS Real-time Data Storage with MongoDB
4 likes7,152 views
The document discusses using MongoDB to store time-series sensor data transmitted using the Data Distribution Service (DDS) standard. DDS is used to acquire real-time data from sensors and devices, which is then converted to JSON format and stored as documents in MongoDB collections. MongoDB is suitable for this application because it can manage flexible data structures and scale to large volumes of sensor data. The system provides benefits like replay capability, querying, analysis and integration testing using the recorded sensor information.
DDS-to-JSON and DDS Real-time Data Storage with MongoDB
- 1. DDS-to-JSON and DDS Real-time Data Storage with MongoDB Mil-DDS IoT Suite Abdullah OZTURK,Technical Lead
- 2. • The world is becoming more and more instrumented, interconnected and intelligent, resulting in newly generated data. • Among all the types of Big Data, data from sensors is the most widespread and is referred to as time-series data. • With storage costs coming down significantly, companies now want to leverage this sensor/device generated data for conducting analysis.
- 3. JSON, or JavaScript Object Notation, is an open- standard format for human-readable text. • It often consists of key-value pairs in a hierarchical structure. • JSON has become a de facto standard format for exchanging machine-generated data.
- 4. • Multi-structured data types typical in IoT applications can be modeled more efficiently using JSON documents, rather than SQL tables. • JSON is better adapted than XML to devices with limited capabilities such as smart things. • JSON has become a nice connector to bridge the physical (IoT) and the digital worlds (Web Services).
- 5. • The flood of machine-generated data from IoT is one of the primary drivers behind the accelerating adoption of non- relational systems. • RDBMS in its natural format was not designed for storing or managing time-series data. • NoSQL databases are both flexible and scalable enough to keep pace with the explosion in connected devices. • JSON is emerging as a preferred format in web-centric, so- called NoSQL databases.
- 6. How DDS Recording Service Works? • Acquire large volumes of real-time data from sensors and devices with Data Distribution Service (DDS), • convert to JSON format and, • store it into a time series document in a MongoDB collection.
- 7. DDS Recording Service • Recorded data is immediately available for • replay, • querying, • conversion to commonly accepted formats, • data analysis. • It becomes useful in project development, testing and system integration as well as in deployed systems. • Replay can be configured • domain, topics, replay speed and time interval.
- 8. • MongoDB as a NoSQL database can manage data of any structure, no matter how often it changes. • New DDS topic types can be added to the system without redesigning the database. • MongoDB provides high-performance data persistence. • Robust authentication, authorization, auditing and encryption controls. • MongoDB can run complex ad-hoc or reporting analytics in-place against sensor data.
- 9. • MongoDB is built to scale out by its automatic sharding, which distributes data across a cluster of servers, with application transparency. • Replica sets provide redundancy and high availability, and are the basis for all production deployments.
- 10. DDS is the standard that addresses most of the messaging requirements of IoT systems with its performance, reliability, fault-tolerance, security, data-model evolution and eventual consistency for heterogeneous networks. Mil-DDS provides additional solutions to enhance its DDS core for mobile, embedded, web, enterprise, and cloud applications of IoT systems.
- 11. Thank you. Mil-DDS IoT Suite Abdullah OZTURK,Technical Lead