On time-series databases
- 1. Time Series Databases Dmitry Namiot Lomonosov Moscow State University dnamiot@gmail.com
- 2. Time Series • Time series is just a sequence of data elements. • The typical use case is a set of measurements made over a time interval. • Much of the data generated by sensors, in a machine to machine communication, in Internet of Things area • Survey of data persistence for time series
- 3. SQL & Time Series CREATE TABLE TS AS ( ts_time TIMESTAMP NOT NULL PRIMARY KEY, ts_id INT, ts_value FLOAT )
- 4. Specific of processing • INSERT prevails • UPDATE (DELETE, MODIFY) are uncommon • High-frequent INSERT operations (M2M, IoT applications) • Sliding window for processing
- 7. SQL extensions SELECT item, slice_time, ts_first_value(price, 'const') price FROM ts_test WHERE price_time BETWEEN timestamp '2015-04-14 09:00' AND timestamp '2015-04-14 09:25' TIMESERIES slice_time AS '1 minute' OVER (PARTITION BY item ORDER BY price_time) ORDER BY item, slice_time, price; SELECT id_sensor, name_sensor, temperature, avg(temperature) OVER (ORDER BY id_sensor ROWS BETWEEN 1 PRECEDING AND 1 FOLLOWING) FROM temperature_table;
- 8. NoSQL solutions • The simplest model for storing time series data is creating a wide row of data for each measurement: • SensorID, {timestamp1, value1}, {timestamp2, value2} … {timestampN, valueN} • Cassandra can store up to 2 billion columns per row. • For high frequency measured data, we can add a shard interval to a row key.
- 9. Open TSDB
- 10. SenML & MQTT
- 12. More NoSQL systems • Geras DB uses SenML • SciDB’s native multi-dimensional array data model is designed for ordered, highly dimensional, multifaceted data • BlinkDB supports a slightly constrained set of SQL-style declarative queries • InfluxDB is open-source, distributed, time series database with no external dependencies
- 13. SAP HANA • series property aspect of tables; • built-in Special SQL functions for working with series data; • analytic functions: special SQL functions for analyzing series data; • storage support: advanced techniques for storing equidistant data using dictionary encoding