Influx data basic

z
Time Series data processing and analysis.
InfluxData - Time Series Platform.
z
www.influxdata.com
GitHub

z
Serhii Savaryn
z
 Python Engineer
 > 3 years of experience

z
Agenda
z
 InfluxData general information.
 Open Source TICK Stack.
 InfluxDB, what’s inside?
 Telegraf, Chronograf, Kapacitor.
 Demo.
 Pros & Cons.

z
The modern engine for Metrics and Events
The Leading Time Series Platform
z
 DevOps
Observability
 IoT Analytics
 Real-Time
Analytics

z The InfluxData Platform is a complete platform for handling all time series data, from humans,
sensors, or machines—seamlessly collecting, storing, visualizing, and turning insight into action.
With both fast deployment and fast performance, InfluxData delivers real value in real time.
InfluxData has three major product offerings: InfluxCloud (fully managed and hosted service
offering), InfluxEnterprise (software that can run on-premises or on any cloud provider), and an
open source Time Series Platform.
Functional Architecture

z
Components of the TICK Stack
InfluxData provides a Modern Time Series Platform, designed from
the ground up to handle metrics and events. InfluxData’s
products are based on an open source core. This open source core
consists of the projects—Telegraf, InfluxDB, Chronograf,
and Kapacitor; collectively called the TICK Stack.

z
InfluxDB is the Time Series Database in
the TICK Stack
InfluxDB is used as a data store for any use case involving large
amounts of time-stamped data, including DevOps monitoring, log
data, application metrics, IoT sensor data, and real-time analytics.
Conserve space on your machine by configuring InfluxDB to keep
data for a defined length of time, automatically expiring & deleting
any unneeded data from the system. InfluxDB also offers a SQL-
like query language for interacting with data.

z
TSDB unique combination
 Ingest: thousands to millions of points per second
 Store: fast accumulating, append-mostly data, lots of repetition,
often with time-to-live
 Query: analytic queries with fast filtering, windowing
 Scale: availability, storage, query

z
InfluxDB: Adding data (1)
POST ’http://localhost:8086/write?db=mydb' --data- binary
'cpu_load_short,host=server01,region=us-west value=0.64
1434055562000000000’

z
InfluxDB: Adding data (2)
 fsync( ) batch to WAL
 Add to in- memory cache & Snapshot cache to TSM
 Add to index

z
InfluxDB: on-disk (filesystem)
CREATE RETENTION POLICY <retention_policy_name> ON
<database_name> DURATION <duration> REPLICATION <n> [SHARD
DURATION <duration>] [DEFAULT]
Database directory /db
Retention Policy directory /db/rp
Shard Group (time bounded) (Logical)
Shard directory (db/rp/Id#)
TSM0001.tsm (data file)
TSM0002.tsm (data file)

z
InfluxDB: Adding data (index)
 Measurement name -> field keys
 Measurement name -> series
 Measurement name -> tag keys -> tag value -> series
 Series -> shards
 (Also sketches of series and measurements for fast cardinality
estimation)

z
InfluxDB: TSI
 Roaring-bitmaps to short- cut
series creation on insert
 Iterators for index mappings
 Index is per-shard; series id file
is per-database
 Partitioned for lock-splitting

z
Telegraf is the Agent for Collecting and
Reporting Metrics & Data
Telegraf is part of the TICK Stack and is a plugin-driven server
agent for collecting and reporting metrics. Telegraf
has integrations to source a variety of metrics, events, and logs
directly from the containers and systems it’s running on, pull metrics
from third-party APIs, or even listen for metrics via a StatsD and
Kafka consumer services. It also has output plugins to send metrics
to a variety of other datastores, services, and message queues,
including InfluxDB, Graphite, OpenTSDB, Datadog, Librato, Kafka,
MQTT, NSQ, and many others.

z
Telegraf metrics
Telegraf metrics are the internal representation used to model data during processing. These
metrics are closely based on InfluxDB’s data model and contain four main components:
 Measurement name: Description and namespace for the metric.
 Tags: Key/Value string pairs and usually used to identify the metric.
 Fields: Key/Value pairs that are typed and usually contain the metric data.
 Timestamp: Date and time associated with the fields.
This metric type exists only in memory and must be converted to a concrete representation in
order to be transmitted or viewed. Telegraf provides output data formats (also known
as serializers) for these conversions. Telegraf’s default serializer converts to InfluxDB Line
Protocol, which provides a high performance and one-to-one direct mapping from Telegraf
metrics.

z
Telegraf aggregator and processor plugins

z
Kapacitor is a Real-Time Streaming Data
Processing Engine
Kapacitor is a native data processing engine in the TICK Stack. It
can process both stream and batch data from InfluxDB. It lets you
plug in your own custom logic or user-defined functions to process
alerts with dynamic thresholds, match metrics for patterns, compute
statistical anomalies, and perform specific actions based on these
alerts like dynamic load rebalancing. It also integrates with HipChat,
OpsGenie, Alerta, Sensu, PagerDuty, Slack, and more.

z
Basic examples
An elementary stream → from() pipeline
The simple script in Example 2 can be used to create a task with the default Telegraf database.
The task, sf_task, will simply cache the latest cpu datapoint as JSON to the HTTP REST
endpoint(e.g http://localhost:9092/kapacitor/v1/tasks/sf_task/dump).

z
Chronograf is the Complete Interface for the
InfluxData Platform
Chronograf is the user interface component of InfluxData’s TICK
Stack. It allows you to quickly see the data that you have stored in
InfluxDB so you can build robust queries and alerts. It is simple to
use and includes templates and libraries to allow you to rapidly
build dashboards with real-time visualizations of your data.

Influx data basic

More Related Content

What's hot (20)

Similar to Influx data basic (20)

Recently uploaded (20)

Influx data basic