Low-Cost Adaptive Monitoring Techniques for the Internet of Things

Low-Cost Adaptive Monitoring
Techniques for the Internet of Things
The AdaM Framework
Demetris Trihinas
University of Cyprus
trihinas@cs.ucy.ac.cy
Talk at TU Berlin, 15 mar. 2015

This talk is based on…
AdaM: an Adaptive Monitoring Framework for Sampling and Filtering on IoT Devices, D. Trihinas
and G. Pallis and M. D. Dikaiakos, 2015 IEEE International Conference on Big Data (IEEE BigData
2015), Santa Clara, CA, USA Pages: 717–726, 2015.
Low-Cost Adaptive Monitoring Techniques for the Internet of Things, D. Trihinas and G. Pallis and
M. D. Dikaiakos, Transactions on Big Data, 2016, (In Review).
Demetris Trihinas Talk at TU Berlin, 15 mar. 2015 2

Demetris Trihinas 3
IoT was initially devices sensing and exchanging data streams
with humans or other network-enabled devices
Init
Sense
Transmit
Sleep

Edge-Mining
Demetris Trihinas 4
A term coined to reflect data processing and decision-making on
“smart” devices that sit at the edge of IoT networks
Buy more
detergent
…our devices just got a little bit more “smarter”…
Init
Sense
Transmit
Sleep
Receive
Actuation

The “Big Data” in IoT
Demetris Trihinas 5
[IDC, Big Data in IoT, 2014]
[Cisco, IBSG, Apr 2011]
IoT Monitoring Data
2% of digital universe in
2012
Projections for >10% in
2020
[Gartner, 2015]

Big Data Challenges Transitioning to IoT
Demetris Trihinas 6
• Taming data volume and data velocity with
limited processing and network capabilities
• IoT devices are usually battery-powered which means intense
processing results in increased energy consumption (less battery-life)
Zhang et al., Usenix HotCloud, 2015

Low-Cost Adaptive
Monitoring Techniques
Adaptive Sampling and Filtering
Demetris Trihinas 7Talk at TU Berlin, 15 mar. 2015

Metric Stream
Demetris Trihinas 8
• A metric stream 𝑀 = {𝑠𝑖}𝑖=0
𝑛
published from a monitoring source is
a large sequence of samples, denoted as 𝑠𝑖 , where 𝑖 = 0, 1, … , 𝑛
and 𝑛 → ∞
• Each sample 𝑠𝑖 is a tuple 𝑡𝑖, 𝑣𝑖 described by a timestamp 𝑡𝑖 and a
value 𝑣𝑖
𝑠𝑖
𝑠𝑖+1
Metric Stream M

Periodic Sampling
Demetris Trihinas 9
• The process of triggering the collection mechanism of a monitored source every
𝑇 time units such that the 𝑖 𝑡ℎ sample is collected at time 𝑡𝑖 = 𝑖 ∙ 𝑇
𝑠𝑖
𝑠𝑖+1
Metric Stream M sampled every T = 1s
Compute resources and energy are wasted while generating large data volumes at a high velocity
Metric Stream M sampled every T = 10s
Sudden events and significant insights are missed

Adaptive Sampling
Demetris Trihinas 10
• Dynamically adjust the sampling period 𝑇𝑖 based on some function 𝜌(𝑀),
containing information of the metric stream evolution
𝑠𝑖
𝑠𝑖+1
𝑇𝑖+1
Metric Stream 𝑀′ dynamically sampledMetric Stream 𝑀 with 𝑇 = 1𝑠
• Find the max 𝑇 ∈ [𝑇 𝑚𝑖𝑛, 𝑇 𝑚𝑎𝑥] to collect sample 𝑠𝑖+1 based on an estimation
of the evolution of the metric stream 𝜌(𝑀), such that 𝑀′ differs from 𝑀 less
than a user-defined imprecision value 𝛾 (𝑑𝑖𝑠𝑡 < 𝛾)

Metric Filtering
• The process of suppressing metric value dissemination when
consecutive values do not “change” (e.g. differ less than a
range of values)
• Goal: Reduce data volume and network overhead in favour of
exact precision
• How much “change” is required, depends on the type of filter
applied
• The receiver-side (e.g., base station, monitoring server) assumes
that the values of any unreported metrics remain unchanged

Metric Filtering
• Fixed Range Metric Filters
Metric Stream M with fixed filter range 𝑅 = 1%
if (curValue ∈ [prevValue – R, prevValue + R ])
filter(curValue)
No samples are filtered
Can we know this R
beforehand?
Should it keep the
same value forever?
Is it the same for each
and every metric?

Adaptive Filtering
• Dynamically adjusting the filter range 𝑅 based on the current variability
of the metric stream, denoted as 𝑞(𝑀)
Metric Stream 𝑀′ with adaptive RangeMetric Stream 𝑀
𝑠𝑖
𝑠𝑖+1
𝑅𝑖+1
𝑅𝑖
• Find the max filter range 𝑅𝑖+1 ∈ [𝑅 𝑚𝑖𝑛, 𝑅 𝑚𝑎𝑥] for sample 𝑠𝑖+1 such
that 𝑀′ differs from 𝑀 less than a user-defined imprecision value 𝛾
based on the variability of the metric stream 𝑞(𝑀)

The Adaptive Monitoring Framework
• Software library with no external dependencies embeddable on IoT devices
• Reduces processing, network traffic and energy consumption by adapting the
monitoring intensity based on the metric stream evolution and variability
• Achieves a balance between efficiency and accuracy
It’s open-source! Give it a try!
https://github.com/dtrihinas/AdaM
Raspberry Pi
Arduino
Beacons
Sensing
Unit
Processing
Unit
Dissemination
Unit
Adaptive
Sampling
Knowledge
Base
AdaM IoT Device
𝑠𝑖
(𝑇𝑖+1, 𝑐𝑖)
(𝑅𝑖+1, 𝐹𝑖)
𝑠𝑖
Adaptive
Filtering
A
P
I
𝑠𝑖
The AdaM Framework

AdaM’s
Algorithms
Adaptive Sampling & Filtering

Adaptive Sampling Algorithm
• Step 1: Compute the distance 𝛿𝑖 between the current two consecutive
sample values
Why use the distance instead of the current value?
𝛿𝑖 = |𝑣𝑖 − 𝑣𝑖−1|
Threshold-base techniques increase sampling rate while sample values
approach a user-defined threshold
• Good for anomaly detection (e.g. DDoS attacks)
• But, what about events away from threshold? (e.g. low rate DDoS attacks)
[Meng, Trans on Computers, 2013]

• Step 2: Compute metric stream evolution based on a Probabilistic
Exponential Weighted Moving Average (PEWMA) to estimate 𝛿𝑖+1
and the standard deviation 𝜎𝑖+1:
𝛿𝑖+1 = 𝜇𝑖 = 𝑎 ∙ 𝜇𝑖−1 + 1 − 𝑎 𝛿𝑖
Looks like an exponential moving average, right?
But weighting is probabilistically applied!
𝑎 = 𝛼 1 − 𝛽P𝑖
Moving standard deviation
with only previous value
knowledge

Why use a Probabilistic EWMA?
Simple EWMA is volatile to abrupt transient changes
• Slow to acknowledge spike after “stable” periods
• If “stable” phases follow sudden spikes, then subsequent
values are overestimated
Sounds a lot like a job for
a Gaussian distribution!

• Step 3: Compute actual standard deviation 𝜎𝑖
• Step 4: Compute the current confidence (𝑐𝑖 ≤ 1) of our approach based
on the actual and estimated standard deviation
𝑐𝑖 = 1 −
| 𝜎𝑖 − 𝜎𝑖|
𝜎𝑖
• Step 5: Compute sampling period 𝑇𝑖+1 based on the current confidence
and the user-defined imprecision γ (e.g. 10% tolerance to errors)
… the more “confident” the algorithm is, the larger the
outputted sampling period 𝑇𝑖+1 can be…
O(1) complexity as all steps only use their previous values!
λ is an
aggressiveness
multiplier
(default λ=1)

Adaptive Filtering Algorithm
• Step 1: Compute the current variability of the metric stream using a moving
Fano Factor 𝐹𝑖 based on exponentially weighted average 𝜇𝑖 and standard
deviation 𝜎𝑖 already computed from adaptive sampling
𝐹𝑖 =
𝜎𝑖
2
𝜇𝑖
𝑅𝑖 = 𝑅𝑖−1 ± 0.01 ∙ 𝑅𝑖−1
Even for a 1% adjustment critical values can be filtered out in biosignal monitoring
…a low 𝐹𝑖 (due to 𝜎𝑖) indicates a currently in-dispersed data
stream which means low variability in the metric stream…
Why use variability and not follow a stepwise approach?

Adaptive Filtering Algorithm
• Step 2: Compute the new filter range 𝑅𝑖+1 based on 𝐹𝑖 and the user-
defined imprecision 𝛾
O(1) complexity as all steps only use their previous values!
𝑠𝑖
𝑠𝑖+1
𝑅𝑖+1
𝑠𝑖+2
𝑠𝑖+3
𝐹𝑖𝑙𝑡𝑒𝑟𝑒𝑑!
𝑅𝑖+2
𝑅𝑖
𝐹 < 𝛾
𝐹 > 𝛾
𝐹𝑘−1 > 𝐹𝑘
𝑠 𝑘

Evaluation

AdaM vs State-of-the-Art
• i-EWMA [1]: an EWMA adapting sampling period by 1 time unit when
the estimated error (ε) is under/over user-defined imprecision
• L-SIP [2]: a linear algorithm using a double EWMA to produce
estimates of current data distribution based on rate values change
• Slow to react to highly transient and abrupt fluctuations in the metric stream
• FAST [3]: an (aggressive) framework using a PID controller to compute
(large) sampling periods accompanied by a Kalman Filter to predict
values at non sampling points
[1] Trihinas et al., CCGrid, 2014 [2] Gaura et al., IEEE Sensors Journal, 2013 [3] Fan et al., ACM CIKM, 2012

Datasets
Memory Trace
Java Sorting Benchmark
CPU Trace
Carnegie Mellon RainMon Project
Disk I/O Trace
TCP Port Monitoring Trace
Cyber Defence SANS Tech Institute
Step Trace
Fitbit Charge HR Wearable
Heart Trace

Our Small “Big Data” Testbeds
Raspberry Pi 1st Gen. Model B
512MB RAM, Single Core ARM 700MHz
Daemon on OS emulates traces while feeding
samples to each algorithm
Android Emulator + SensorSimulator
128MB RAM, Single Core ARM 32MHz
SensorSimulator script emulates traces by
feeding samples to Android Wear emulator for
Step and Heartrate processing
Daemon

Evaluation Metrics
• Estimation Accuracy – Mean Absolute Percentage Error (MAPE)
• CPU Cycles
• Outgoing Network Traffic
• Edge Device Energy Consumption*
*Xiao et al., ACM DAC, 2010
**Brooks, ACM SIGARCH, 2000
perf
nethogs
Powerstat and
Other than error evaluation no other study goes through an overhead study!
Wattch**

Error Comparison
Memory Trace CPU Trace Disk I/O Trace
TCP Port Monitoring Trace Fitbit Step Trace
For all traces AdaM has the smallest inaccuracy (<10%)
Even in a more aggressive configuration (λ=2) AdaM is still comparable to L-SIP
AdaM AdaM AdaM
AdaM AdaM
User-defined accepted imprecision set to γ = 0.1
Fitbit Heartrate Trace
AdaM

AdaM γ-Parameter Evaluation
For a wide range of γ-parameter values we compute AdaM’s
MAPE per trace
Even for extreme imprecision values (>0.3) AdaM can still take correct
decisions signifying the importance of the confidence metric

So How Well Does AdaM Perform?
Memory Trace
Java Sorting Benchmark
CPU Trace
Disk I/O Trace
TCP Port Monitoring Trace
Cyber Defence SANS Tech Institute
Step Trace
Heartrate Trace

Overhead Comparison (1)
Network Traffic
Energy Consumption
CPU Cycles
Filtering adds an overhead <2% to AdaM but
benefits network traffic reduction by at least
32% and energy consumption by 12%

Network Traffic
Energy Consumption
CPU Cycles
Data volume is reduced by 74%, energy
consumption by 71%, while accuracy is always
above 89% and 83% with filtering enabled

Network Traffic Energy Consumption
Error (MAPE)
FAST’s aggressiveness results in slightly lower
energy consumption and network traffic
But, this does not come for free…
FAST features a large inaccuracy
AdaM achieves a balance between
efficiency and accuracy

Energy Consumption
In the case of heartrate monitoring where signal
analysis on AC wavelets reflected on wrist arteries is
needed, AdaM reduces energy consumption by 86%
Calorie Counting is based on human body indicators
(age, weight, height) and heartrate monitoring
Error (MAPE)
AdaM’s MAPE grows from 6.42% in heartrate monitoring to 9.07% in calorie counting
in contrast to FAST with 13.61% and 21.83% respectively

+ AdaM
3 - 5 days
7-8 days

Scalability Evaluation (1)
• Integrated AdaM to a data streaming
system
• JCatascopia Cloud Monitoring System
• JCatascopia Agents (data sources) use AdaM
to adapt monitoring intensity
• Archiving time is measured at JCatascopia
Server to evaluate data velocity
• Compare AdaM over Periodic Sampling
https://github.com/dtrihinas/JCatascopia
DB
Agent
Agent
Agent
.
.
.
JCatascopia
AdaM
AdaM
metrics
Data velocity is reduced
to a linear growth
TALK AT TU BERLIN, 15 MAR. 2015
JCatascopia: Monitoring Elastically Adaptive Applications in the Cloud. Trihinas, D.; Pallis, G.; and Dikaiakos, M. D. In the 14th IEEE/ACM
International Symposium on Cluster, Cloud and Grid Computing (CCGRID 2014), pages 226-235, May, Chicago, IL, USA, 2014.
AdaM
Agent

• Dublin Smart City Intelligent Transportation Service (Dublin ITS)
.
.
.
.
.
.
1000 Buses* with GPS tracking
sending updates to ITS with 16
params (e.g. busID, location,
current route delay)
*Real data from Jan. 2014
Apache Kafka queuing
service on x-large VM
(16VCPU, 16GB RAM,
100GB Disk)
Apache Spark cluster with 5 workers on large VMs
(8VCPU, 8GB RAM, 40GB Disk)
Alerts ITS operators when more than 10 buses in a
Dublin city area, per 5min window, are reporting delays
over 1 standard deviation from their weekly average

• Spark total delay (processing + scheduling) for T=1, 5, 10 intervals and
AdaM with max imprecision γ = 0.15
After 572 buses deployment
becomes unstable
After 746 buses deployment
becomes unstable
Deployment remains stable
barely rising above threshold
at 971 buses
Deployment remains stable
but rises above threshold at
817 buses

AdaM achieves an 83% average accuracy with >85% in all major Dublin areas
compared to 46% for T=5s and 17% for T=10s
High variety of sampling rates used throughout the hours of the day showing the
need of adaptive sampling

Conclusions
• Edge-mining on IoT devices is both resource and
energy intensive
• Big data streaming engines struggle to cope as the
volume and velocity of IoT-generated data keep
increasing
• Adapts the monitoring intensity based on current metric
evolution and variability
• Reduces processing, network traffic and energy
consumption on IoT devices and the IoT network
• Achieves a balance between efficiency and accuracy
The AdaM Framework

Acknowledgements
co-funded by the
European Commission

Laboratory for Internet Computing
Department of Computer Science
University of Cyprus
http://linc.ucy.ac.cy

Low-Cost Adaptive Monitoring Techniques for the Internet of Things

More Related Content

Similar to Low-Cost Adaptive Monitoring Techniques for the Internet of Things (20)

More from Demetris Trihinas (16)

Recently uploaded (20)

Low-Cost Adaptive Monitoring Techniques for the Internet of Things

Editor's Notes