Power Your Predictive Analytics with InfluxDB

| © Copyright 2023, InﬂuxData
Predictive Analytics
Using a Time Series
Database
Anais Dotis-Georgiou
July 2023

2
Anais Dotis-Georgiou
Developer Advocate
LinkedIn

Agenda
● What is Predictive Analytics?
● Time Series Databases
● Use Cases
● Using Quix, Hugging Face and InﬂuxDB 3.0 for forecasting
and anomaly detection
● Resources

The age of instrumentation
Instrumentation
of the virtual world
(DevOps)
Sensors
in the physical world
(IoT)

Characteristics of
the data
• Time-stamped
• Generated in regular
(metric) and
irregular (event) time
periods
• Huge volumes
• Real time and time
sensitive

Time series in every application
Infrastructure & data sources
Consumer & Industrial IoT Software Infrastructure
Renewable
&
alternative
energy
systems
Manufact
uring &
industrial
platforms
Fleet
management
& telematics
Real-time Applications
Developer
Tools
& APIs
Kubernetes
(K8s)
DevOps
Monitoring
Gaming
Applications
Fintech
Applications
Network
Monitoring
TIME SERIES DATA

Rise of time series as a category
TIME SERIES
RELATIONAL DOCUMENT SEARCH
• Distributed
search
• Logs
• Geo
• High
throughput
• Large
document
• Orders
• Customers
• Records
• Events, metrics, time stamped
• for IoT, analytics, cloud native
Time series is fastest growing
data category by far
Time series
All others
source: DB Engines

Characteristics
of the data
• Time-stamped
• Generated in regular (metric) and
irregular (event) time periods
• Huge volumes
• Real time and time sensitive

InﬂuxDB is 3 things
API &
Toolset
POWERFUL
for real-time apps
HIGH PERFORMANCE MASSIVE
for real-time data
workloads
of cloud & open source
developers
1 2 3
Time Series
Engine
Community &
Ecosystem

InﬂuxData Reference Architecture

InﬂuxDB’s new storage engine is built on
● Rust
● Apache Arrow
● Apache Parquet
● Arrow Flight
● DataFusion

Release Details
14

SQL Support
15

Power Your Predictive Analytics with InfluxDB

Using Quix, Hugging Face and
InﬂuxDB 3.0 for forecasting
and anomaly detection

The Dataset

The Quix Pipeline

23

Source Project
24

import influxdb_client_3 as InfluxDBClient3
client = InfluxDBClient3.InfluxDBClient3(token=os.environ["INFLUX_TOKEN"],
host=os.environ["INFLUX_HOST"],
org=os.environ["INFLUX_ORG"],
database=os.environ["INFLUX_DATABASE"])
25

def get_data():
# Query InfluxDB Cloud 3.0
while run:
try:
query = "SHOW TAG VALUES WITH KEY = "machineID""
table = client.query(query=query, language="influxql")
machines = table["value"].to_pylist()
for machine in machines:
# this will end up loop when all data has been sent
table = client.query(query=f"SELECT vibration, machineID FROM
machine_data WHERE time >= now() - 5m AND machineID = '{machine}'",
language="influxql")
df = table.to_pandas()
print(df)
if df.empty:
break
# If there are rows to write to the stream at this time
stream_producer.timeseries.buffer.publish(df)
sleep(int(os.environ["task_interval"]))
except:
print("query failed")
sleep(int(os.environ["task_interval"]))
26

Event Detection Project
27

Training the Model
28

29

from huggingface_hub import from_pretrained_keras
# Quix injects credentials automatically to the client.
# Alternatively, you can always pass an SDK token manually as an
argument.
client = qx.QuixStreamingClient()
print("Opening input and output topics")
consumer_topic = client.get_topic_consumer(os.environ["input"],
"default-consumer-group")
producer_topic = client.get_topic_producer(os.environ["output"])
model = from_pretrained_keras(os.environ["model"])
30

Forecasting Transformation Project
31

df = df.set_index('timestamp')
data = df.drop(columns=['iox::measurement', 'machineID'])
fit = Holt(data,damped_trend=True,initialization_method="estimated").fit(optimized=True)
fcast = fit.forecast(10).rename("Multiplicative damped trend")
fcast = fcast.reset_index().rename(columns={'index': 'timestamp'})
32

Write Project
33

import influxdb_client_3 as InfluxDBClient3
client = qx.QuixStreamingClient()
# get the topic consumer for a specific consumer group
topic_consumer = client.get_topic_consumer(topic_id_or_name = os.environ["input"],
consumer_group = "empty-destination")
client = InfluxDBClient3.InfluxDBClient3(token=os.environ["INFLUX_TOKEN"],
host=os.environ["INFLUX_HOST"],
org=os.environ["INFLUX_ORG"],
database=os.environ["INFLUX_DATABASE"])
def on_dataframe_received_handler(stream_consumer: qx.StreamConsumer, df:
pd.DataFrame):
# do something with the data here
df = df.rename(columns={'timestamp': 'time'})
df = df.set_index('time')
client.write(df, data_frame_measurement_name='mlresult',
data_frame_tag_columns=['machineID'])
print("Write successful")
34

Get Started
36
inﬂuxdata.com/inﬂuxdb-engine-beta/

InfluxDB
Community
Slack workspace
Please join us in the InfluxDB
Community Slack at
www.influxdata.com/slack.
To participate in conversations,
join the #influxdb_iox channel.

38
inﬂuxdata.com https://github.com/InﬂuxCommunity

39
Try it yourself
https://www.inﬂuxdata.com https://github.com/InﬂuxCommunity
Get Started

Get Help + Resources!
40
Forums: community.influxdata.com
Slack: influxcommunity.slack.com
GH: github.com/InfluxCommunity
Docs: docs.influxdata.com
Blogs: influxdata.com/blog
InfluxDB University: influxdata.com/university

T H A N K Y O U

Power Your Predictive Analytics with InfluxDB

More Related Content

What's hot (20)

Similar to Power Your Predictive Analytics with InfluxDB (20)

More from InfluxData (20)

Recently uploaded (20)

Power Your Predictive Analytics with InfluxDB