SlideShare a Scribd company logo
Exploring the feature space of large time series collections 1
Exploring the
feature space of
large collections
of time series
Rob J Hyndman
Exploring the feature space of large time series collections 1
Exploring the
feature space of
large collections
of time series
Rob J Hyndman
with Earo Wang, Nikolay Laptev
Yanfei Kang, Kate Smith-Miles
Outline
1 M3 competition data
2 Yahoo web traffic
3 What next?
Exploring the feature space of large time series collections M3 competition data 2
M3 forecasting competition
Exploring the feature space of large time series collections M3 competition data 3
M3 forecasting competition
Exploring the feature space of large time series collections M3 competition data 3
M3 forecasting competition
“The M3-Competition is a final attempt by the authors to
settle the accuracy issue of various time series methods. . .
The extension involves the inclusion of more methods/
researchers (in particular in the areas of neural networks
and expert systems) and more series.”
Makridakis & Hibon, IJF 2000
3003 series
All data from business, demography, finance and
economics.
Series length between 14 and 126.
Either non-seasonal, monthly or quarterly.
All time series positive.
Exploring the feature space of large time series collections M3 competition data 4
M3 forecasting competition
Exploring the feature space of large time series collections M3 competition data 5
q
Key idea
Examples for time series
lag correlation
size and direction of trend
strength of seasonality
timing of peak seasonality
spectral entropy
Called “features” or “characteristics” in the
machine learning literature.
Exploring the feature space of large time series collections M3 competition data 6
John W Tukey
Cognostics
Computer-produced diagnostics
(Tukey and Tukey, 1985).
Key idea
Examples for time series
lag correlation
size and direction of trend
strength of seasonality
timing of peak seasonality
spectral entropy
Called “features” or “characteristics” in the
machine learning literature.
Exploring the feature space of large time series collections M3 competition data 6
John W Tukey
Cognostics
Computer-produced diagnostics
(Tukey and Tukey, 1985).
Key idea
Examples for time series
lag correlation
size and direction of trend
strength of seasonality
timing of peak seasonality
spectral entropy
Called “features” or “characteristics” in the
machine learning literature.
Exploring the feature space of large time series collections M3 competition data 6
John W Tukey
Cognostics
Computer-produced diagnostics
(Tukey and Tukey, 1985).
An STL decomposition
Quarterly visitor nights: Mornington Peninsula
Yt = St + Tt + Rt St is periodic with mean 0
5.06.07.0
data
−0.50.5
seasonal
5.86.16.4
trend
−0.40.0
2000 2005 2010
remainder
timeExploring the feature space of large time series collections M3 competition data 7
Candidate features
STL decomposition
Yt = St + Tt + Rt
Seasonal period
Strength of seasonality: 1 − Var(Rt)
Var(Yt−Tt)
Strength of trend: 1 − Var(Rt)
Var(Yt−St)
Spectral entropy: H = −
π
−π fy(λ) log fy(λ)dλ,
where fy(λ) is spectral density of Yt.
Low values of H suggest a time series that is
easier to forecast (more signal).
Autocorrelations: r1, r2, r3, . . .
Optimal Box-Cox transformation parameter λ
Exploring the feature space of large time series collections M3 competition data 8
Candidate features
STL decomposition
Yt = St + Tt + Rt
Seasonal period
Strength of seasonality: 1 − Var(Rt)
Var(Yt−Tt)
Strength of trend: 1 − Var(Rt)
Var(Yt−St)
Spectral entropy: H = −
π
−π fy(λ) log fy(λ)dλ,
where fy(λ) is spectral density of Yt.
Low values of H suggest a time series that is
easier to forecast (more signal).
Autocorrelations: r1, r2, r3, . . .
Optimal Box-Cox transformation parameter λ
Exploring the feature space of large time series collections M3 competition data 8
Candidate features
STL decomposition
Yt = St + Tt + Rt
Seasonal period
Strength of seasonality: 1 − Var(Rt)
Var(Yt−Tt)
Strength of trend: 1 − Var(Rt)
Var(Yt−St)
Spectral entropy: H = −
π
−π fy(λ) log fy(λ)dλ,
where fy(λ) is spectral density of Yt.
Low values of H suggest a time series that is
easier to forecast (more signal).
Autocorrelations: r1, r2, r3, . . .
Optimal Box-Cox transformation parameter λ
Exploring the feature space of large time series collections M3 competition data 8
Candidate features
STL decomposition
Yt = St + Tt + Rt
Seasonal period
Strength of seasonality: 1 − Var(Rt)
Var(Yt−Tt)
Strength of trend: 1 − Var(Rt)
Var(Yt−St)
Spectral entropy: H = −
π
−π fy(λ) log fy(λ)dλ,
where fy(λ) is spectral density of Yt.
Low values of H suggest a time series that is
easier to forecast (more signal).
Autocorrelations: r1, r2, r3, . . .
Optimal Box-Cox transformation parameter λ
Exploring the feature space of large time series collections M3 competition data 8
Candidate features
STL decomposition
Yt = St + Tt + Rt
Seasonal period
Strength of seasonality: 1 − Var(Rt)
Var(Yt−Tt)
Strength of trend: 1 − Var(Rt)
Var(Yt−St)
Spectral entropy: H = −
π
−π fy(λ) log fy(λ)dλ,
where fy(λ) is spectral density of Yt.
Low values of H suggest a time series that is
easier to forecast (more signal).
Autocorrelations: r1, r2, r3, . . .
Optimal Box-Cox transformation parameter λ
Exploring the feature space of large time series collections M3 competition data 8
Candidate features
STL decomposition
Yt = St + Tt + Rt
Seasonal period
Strength of seasonality: 1 − Var(Rt)
Var(Yt−Tt)
Strength of trend: 1 − Var(Rt)
Var(Yt−St)
Spectral entropy: H = −
π
−π fy(λ) log fy(λ)dλ,
where fy(λ) is spectral density of Yt.
Low values of H suggest a time series that is
easier to forecast (more signal).
Autocorrelations: r1, r2, r3, . . .
Optimal Box-Cox transformation parameter λ
Exploring the feature space of large time series collections M3 competition data 8
Candidate features
STL decomposition
Yt = St + Tt + Rt
Seasonal period
Strength of seasonality: 1 − Var(Rt)
Var(Yt−Tt)
Strength of trend: 1 − Var(Rt)
Var(Yt−St)
Spectral entropy: H = −
π
−π fy(λ) log fy(λ)dλ,
where fy(λ) is spectral density of Yt.
Low values of H suggest a time series that is
easier to forecast (more signal).
Autocorrelations: r1, r2, r3, . . .
Optimal Box-Cox transformation parameter λ
Exploring the feature space of large time series collections M3 competition data 8
Dimension reduction for time series
Exploring the feature space of large time series collections M3 competition data 9
q
Dimension reduction for time series
Exploring the feature space of large time series collections M3 competition data 9
q
SpecEntr
0.0 0.4 0.8 2 6 10 0.0 0.4 0.8
0.50.9
0.00.6
Trend
Season
0.00.6
28
Freq
ACF
−0.40.6
0.5 0.7 0.9
0.00.6
0.0 0.4 0.8 −0.4 0.2 0.8
Lambda
Feature
calculation
Dimension reduction for time series
Exploring the feature space of large time series collections M3 competition data 9
q
SpecEntr
0.0 0.4 0.8 2 6 10 0.0 0.4 0.8
0.50.9
0.00.6
Trend
Season
0.00.6
28
Freq
ACF
−0.40.6
0.5 0.7 0.9
0.00.6
0.0 0.4 0.8 −0.4 0.2 0.8
Lambda
q q
q
q
q
q
qq qq
q q
q
q
q
q
q
q
q
q
q
q
q
qq q
qq
q
qqq
q
q
qq q
q
qq q
qq qq
q qqq
q
q
q
q
qq
qqqq
q
q
qq
q
q
qq
q
q
q
q
q
q
qq
q
qq
q q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
qq
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q q
q
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
qq
qqq
q q
q
q
q
qq
q
qqq
qqqq
q
qq
q
q
q
q
qq
q
qq
qq
qqqq
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
qq
q
q
q
q
q
qq
qq
q
q
q
q
q
q q
q
qqq
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
qqqq
q
qqq
q
qqq
qq
q
qq
qq
qqqqqqqqqq
q
q
q
q q
q
qq
q
qq
q qq
q q
q
q
qq
q
q
qqq
qq
qqq
q q
qqqqqq q
q
q
q
qq
qq qqqq
q
q
qq
q
qq
q
q
qqqq q
q
q
q
qqq
qq
q
q
qqqqq qqq
qq
q
q
q
q
q
q
q q
q
q
q
qq
q
qqq
q
qqq
qqq
q
qq
q
q
q
q
q
q
qqqqq
q
q
qq
q
q
qq
q
q qq
q
q
q
q
q
q
q q
q q
qq
q
q
q
qqqqq
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
qqq
qqqq
q
q
q
qqq
q
q q
qq
q
q
q
q
qqqqq
q
qqqq
q q
q
qq
q
q
qq
qq
q
q
qq
q q
q
q
qqqq
q
q
q
q
q
q
q
q
q
q
qq
q qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q qq
q
q
qq
q
q
q
q
qqq q
q
q
qq qq
q
q
q
q
q
qq
qqq
q
qq
q
q
qqq q
q
q
qq
q
q
qq
qqqqqqq
qq
q
q
q
q
qqqqqq
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q q qqq
q
q
q
q
q
qq qq
q
q
q
q
q
q
q
qq
q
q
q q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q q
q
q
q q
q
q qq
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qq q
q
q
qq
q
q
q
qqq
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
qq
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qqq
q
qq
q
q
qq
q
qq
q
q
qq
q
qqq q
q q
q
q
q
q
q
q
q
q qq q
q
qq
q
q
q
q
qq q
q
q q
qq
q
q
qq q
q
qq
q
q
q
qq
qqq q
qq
q
q
qq
q
q
q
q
qqqq
q
qq
q
q
q qqqq
qq
q
q
q
qqq qqqq
q
q qqq
q
qqq
q
q
qq q
qqq
q q
qq
q
q qqq
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
qqqq
q
qqq
q
qqq
q
qqq q
q
q
q
qq q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q q
q
q
q
q
q
q
qq
q
qq
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
qq qq
q
q
qqqq
q
q
qqq
q
q
q
q
q
q
q
q
qq
q
qqqq
q
qq qq
q
q
q
qq
qq
q
q
q
q q
q
q
qq
q
q
q
q
q
q
qq
q
q
q q
q
q
qq
q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q qq
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
qq
q
q
q
q q
qq
q
q
q
q
q
q
q
q
qq
qq q
q
qq
q
q
qq
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
qq
qqq
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
qq
qq
qq
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q qq q
q
q
q
q qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qqq
q
q q
q q
q
q q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q qq
qq
q
q qq
q
q
q
q
q qq q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
qq
q
q
q qq
q
q
q
q
qq
q
q
q
q
q q
q
q
q
q
q
q
q q
q
q
q
q
qq
q
q
q q
q
q
q
q
q
q
q
q
qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
q q
q
q
qq
q
qq
q
q
q
q
qq
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
qq qqq
q
qq
q q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q q
qq
q
qq
q
q q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qq
qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qq
q qq
q q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
q
q
q
q
qqqq
q
q
q
qqqq
q
q
qq
qq q
q
q
q
q
q
q qqqq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q
qq
q
q
q
q
qq qqq qqqq qqqq
q
qq
qqqqqq
q
qqqq
q
q
q
q
q q
q
q
q
q
qqq
q
q
q
q
q
q
q q
q
q
qqqq q
q
q
q
q
q
q
qq
q
q
qq
q
q
qq
q
qq
q
q
q
q
qq
qqq
q
q
q
q q
q q
q
qq
q
qq
q
q
q
qqq
q q
q q
q
q
qq
q
q
q
q
q
q
q qq
qqq
q
q
qq
q
q
q
q
q
qq
q qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qqqq
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
qq
q
q
qq
qq
q
q q
q
qq
q
q
qqqq
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q qq
q
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
qq
q
qq
qq q qq
q
qq
q
q
qqq
q
qq
qq
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
qqq
q
qq
q
q
qq
qq
q
q
qqqq
q
q
q
q
q
q
q q
q
q
qqq
q
q
q
q
q
qqqq qqqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
qq
q
q
q
q
qq
q
q q
q qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q qq qq
q
q
q
qq
q
q qq
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
qq
q
q
qq
q
q
qqq
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q q
q
q
q
q qq q q
q
qq q
q
qq
q
q
q
q
q
qq
qq
qqq
qqq
q
q
q
q
q
q qqqqqqqq
q
q
qqqqqqqqqqqqqqqqq
q qq
q
q
q
q
qqqqq
qqq
q
q
qqqqq
q
q
q
qqqq
q
q
q
q
q
q
q
qq
q
q
q
qqqqq
q
qqqqq
q
qq
qqq
qqqq
q
qq
q
qq
q
q
q
qq
q
qq
−3
−2
−1
0
1
2
3
−2 0 2 4
PC1
PC2
Feature
calculation
Principal
component
decomposition
Feature space of M3 data
Exploring the feature space of large time series collections M3 competition data 10
q q
q
q
q
q
qq qq
q q
q
q
q
q
q
q
q
q
q
q
q
qq q
qq
q
qqq
q
q
qq q
q
qq q
qq qq
q qqq
q
q
q
q
qq
qqqq
q
q
qq
q
q
qq
q
q
q
q
q
q
qq
q
qq
q q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
qq
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q q
q
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
qq
qqq
q q
q
q
q
qq
q
qqq
qqqq
q
qq
q
q
q
q
qq
q
qq
qq
qqqq
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
qq
q
q
q
q
q
qq
qq
q
q
q
q
q
qq
q
qqq
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
qqqq
q
qqq
q
qqq
qq
q
qq
qq
qqqqqqqqqq
q
q
q
q q
q
qq
q
qq
q qq
q q
q
q
qq
q
q
qqq
qq
qqq
q q
qqqqqq q
q
q
q
qq
qq qqqq
q
q
qq
q
qq
q
q
qqq
q q
q
q
q
qqq
qq
q
q
qqqqq qqq
qq
q
q
q
q
q
q
q q
q
q
q
qq
q
qqq
q
qqq
qqq
q
qq
q
q
q
q
q
q
qqqqq
q
q
qq
q
q
qq
q
q qq
q
q
q
q
q
q
q q
q q
qq
q
q
q
qqqqq
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
qqq
qqqq
q
q
q
qqq
q
q q
qq
q
q
q
q
qqqqq
q
qqqq
q q
q
qq
q
q
qq
qq
q
q
qq
q q
q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
qq
q qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q qq
q
q
qq
q
q
q
q
qqq q
q
q
qq qq
q
q
q
q
q
qq
qqq
q
qq
q
q
qqq q
q
q
qq
q
q
qq
qqqqqq
q
qq
q
q
q
q
qqqqqq
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q q qqq
q
q
q
q
q
qq qq
q
q
q
q
q
q
q
qq
q
q
q q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q q
q
q
q q
q
q qq
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qqq
q
q
qq
q
q
q
qqq
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
qq
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qqq
q
qq
q
q
qq
q
qq
q
q
qq
q
qqq q
q q
q
q
q
q
q
q
q
q qq q
q
qq
q
q
q
q
qq q
q
q q
qq
q
q
qq q
q
qq
q
q
q
qq
qqq q
qq
q
q
qq
q
q
q
q
qqqq
q
qq
q
q
q qqqq
qq
q
q
q
qqq qqqq
q
q qqq
q
qqq
q
q
qq q
qqq
q q
qq
q
q qqq
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
qqqq
q
qqq
q
qqq
q
qqq q
q
q
q
qq q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q q
q
q
q
q
q
q
qq
q
qq
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
qq qq
q
q
qqqq
q
q
qqq
q
q
q
q
q
q
q
q
qq
q
qqqq
q
qq qq
q
q
q
qq
qq
q
q
q
q q
q
q
qq
q
q
q
q
q
q
qq
q
q
q q
q
q
qq
q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q qq
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
qq
q
q
q
q q
qq
q
q
q
q
q
q
q
q
qq
qq q
q
qq
q
q
qq
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
qq
qqq
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
qq
qq
qq
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q qq q
q
q
q
q qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qqq
q
q q
q q
q
q q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q qq
qq
q
q qq
q
q
q
q
q qq q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
qq
q
q
q qq
q
q
q
q
qq
q
q
q
q
q q
q
q
q
q
q
q
q q
q
q
q
q
qq
q
q
q q
q
q
q
q
q
q
q
q
qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
q q
q
q
qq
q
qq
q
q
q
q
qq
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
qq qqq
q
qq
q q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q q
qq
q
qq
q
q q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qq
qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qq
q qq
q q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
q
q
q
q
qqqq
q
q
q
qqqq
q
q
qq
qq q
q
q
q
q
q
q qqqq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q
qq
q
q
q
q
qq qqq qqqq qqqq
q
qq
qqqqqq
q
qqqq
q
q
q
q
q q
q
q
q
q
qqq
q
q
q
q
q
q
q q
q
q
qqqq q
q
q
q
q
q
q
qq
q
q
qq
q
q
qq
q
qq
q
q
q
q
qq
qqq
q
q
q
q q
q q
q
qq
q
qq
q
q
q
qqq
q q
q q
q
q
qq
q
q
q
q
q
q
q qq
qqq
q
q
qq
q
q
q
q
q
qq
q qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qqqq
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
qq
q
q
qq
qq
q
q q
q
qq
q
q
qqqq
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q qq
q
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
qq
q
qq
qq q qq
q
qq
q
q
qq
q
q
qq
qq
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
qqq
q
qq
q
q
qq
qq
q
q
qqqq
q
q
q
q
q
q
q q
q
q
qqq
q
q
q
q
q
qqqq qqqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
qq
q
q
q
q
qq
q
q q
q qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q qq qq
q
q
q
qq
q
q qq
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
qq
q
q
qq
q
q
qqq
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q q
q
q
q
q qq q q
q
qq q
q
qq
q
q
q
q
q
qq
qq
qqq
qqq
q
q
q
q
q
q qqqqqqqq
q
q
qqqqqqqqqqqqqqqqq
q qq
q
q
q
q
qqqqq
qqq
q
q
qqqqq
q
q
q
qqqq
q
q
q
q
q
q
q
qq
q
q
q
qqqqq
q
qqqqq
q
qq
qqq
qqqq
q
qq
q
qq
q
q
q
qq
q
qq
−3
−2
−1
0
1
2
3
−2 0 2 4
PC1
PC2
First two PCs explain 68% of variation.
Feature space of M3 data
Exploring the feature space of large time series collections M3 competition data 10
q q
q
q
q
q
qq qq
q q
q
q
q
q
q
q
q
q
q
q
q
qqq
qq
q
qqq
q
q
qq q
q
qq q
qq qq
q qqq
q
q
q
q
qq
qqqq
q
q
qq
q
q
qq
q
q
q
q
q
q
qq
q
qq
q q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
qq
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
qqq
q q
q
q
q
qq
q
qqq
qqqq
q
qq
q
q
q
q
qq
q
qq
qq
qqqq
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
qq
q
q
q
q
q
qq
qq
q
q
q
q
q
qq
q
qqq
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
qqqq
q
qqq
q
qqq
qq
q
qq
qq
qqqqqqqqqq
q
q
q
qq
q
qq
q
qq
q qq
q q
q
q
qq
q
q
qqq
qq
qqq
q q
qqqqqq q
q
q
q
qq
qq qqqq
q
q
qq
q
qq
q
q
qqq
q q
q
q
q
qqq
qq
q
q
qqqqq qqq
qq
q
q
q
q
q
q
q q
q
q
q
qq
q
qqq
q
qqq
qqq
q
qq
q
q
q
q
q
q
qqqqq
q
q
qq
q
q
qq
q
q qq
q
q
q
q
q
q
q q
q q
qq
q
q
q
qqqqq
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
qqq
qqqq
q
q
q
qqq
q
q q
qq
q
q
q
q
qqqqq
q
qqqq
q q
q
qq
q
q
qq
qq
q
q
qq
qq
q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
qq
q qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q qq
q
q
qq
q
q
q
q
qqqq
q
q
qq qq
q
q
q
q
q
qq
qqq
q
qq
q
q
qqq q
q
q
qq
q
q
qq
qqqqqq
q
qq
q
q
q
q
qqqqqq
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q q qqq
q
q
q
q
q
qq qq
q
q
q
q
q
q
q
qq
q
q
q q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q q
q
q
q q
q
q qq
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qqq
q
q
qq
q
q
q
qqq
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
qq
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qqq
q
qq
q
q
qq
q
qq
q
q
qq
q
qqq q
q q
q
q
q
q
q
q
q
q qq q
q
qq
q
q
q
q
qq q
q
q q
qq
q
q
qq q
q
qq
q
q
q
qq
qqqq
qq
q
q
qq
q
q
q
q
qqqq
q
qq
q
q
q qqqq
qq
q
q
q
qqq qqqq
q
q qqq
q
qqq
q
q
qq q
qqq
q q
qq
q
q qqq
q
q
q
q
q q
q
qq
q
q
q
q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
qqqq
q
qqq
q
qqq
q
qqq q
q
q
q
qqq
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q q
q
q
q
q
q
q
qq
q
qq
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
qq qq
q
q
qqqq
q
q
qqq
q
q
q
q
q
q
q
q
qq
q
qqqq
q
qq qq
q
q
q
qq
qq
q
q
q
q q
q
q
qq
q
q
q
q
q
q
qq
q
q
qq
q
q
qq
q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q qq
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
qq
q
q
q
q q
qq
q
q
q
q
q
q
q
q
qq
qq q
q
qq
q
q
qq
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
qq
qqq
q
q
q
q q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
qq
qq
qq
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q qq q
q
q
q
q qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q q
qq
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qqq
q
q q
q q
q
q q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
qqq
qq
q
q qq
q
q
q
q
q qq q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
qq
q
q
q qq
q
q
q
q
qq
q
q
q
q
q q
q
q
q
q
q
q
q q
q
q
q
q
qq
q
q
q q
q
q
q
q
q
q
q
q
qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
q q
q
q
qq
q
qq
q
q
q
q
qq
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
qq qqq
q
qq
q q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q q
qq
q
qq
q
q q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qq
qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qq
qqq
q q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
q
q
q
q
qqqq
q
q
q
qqqq
q
q
qq
qq q
q
q
q
q
q
q qqqq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q
qq
q
q
q
q
qq qqq qqqqqqqq
q
qq
qqqqqq
q
qqqq
q
q
q
q
q q
q
q
q
q
qqq
q
q
q
q
q
q
q q
q
q
qqqq q
q
q
q
q
q
q
qq
q
q
qq
q
q
qq
q
qq
q
q
q
q
qq
qqq
q
q
q
q q
q q
q
qq
q
qq
q
q
q
qqq
qq
q q
q
q
qq
q
q
q
q
q
q
q qq
qqq
q
q
qq
q
q
q
q
q
qq
q qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qqqq
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
qq
q
q
qq
qq
q
q q
q
qq
q
q
qqqq
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q qq
q
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
qq
q
qq
qq q qq
q
qq
q
q
qqq
q
qq
qq
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
qqq
q
qq
q
q
qq
qq
q
q
qqqq
q
q
q
q
q
q
q q
q
q
qqq
q
q
q
q
q
qqqq qqqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
qq
q
q
q
q
qq
q
q q
q qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
qqq qq
q
q
q
qq
q
q qq
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
qq
q
q
qq
q
q
qqq
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q q
q
q
q
q qq q q
q
qq q
q
qq
q
q
q
q
q
qq
qq
qqq
qqq
q
q
q
q
q
qqqqqqqqq
q
q
qqqqqqqqqqqqqqqqq
qqq
q
q
q
q
qqqqq
qqq
q
q
qqqqq
q
q
q
qqqq
q
q
q
q
q
q
q
qq
q
q
q
qqqqq
q
qqqqq
q
qq
qqq
qqqq
q
qq
q
qq
q
q
q
qq
q
qq
−3
−2
−1
0
1
2
3
−2 0 2 4
PC1
PC2
3
6
9
12
value
Freq
Feature space of M3 data
Exploring the feature space of large time series collections M3 competition data 10
q q
q
q
q
q
qq qq
q q
q
q
q
q
q
q
q
q
q
q
q
qqq
qq
q
qqq
q
q
qq q
q
qq q
qq qq
q qqq
q
q
q
q
qq
qqqq
q
q
qq
q
q
qq
q
q
q
q
q
q
qq
q
qq
q q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
qq
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
qqq
q q
q
q
q
qq
q
qqq
qqqq
q
qq
q
q
q
q
qq
q
qq
qq
qqqq
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
qq
q
q
q
q
q
qq
qq
q
q
q
q
q
qq
q
qqq
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
qqqq
q
qqq
q
qqq
qq
q
qq
qq
qqqqqqqqqq
q
q
q
qq
q
qq
q
qq
q qq
q q
q
q
qq
q
q
qqq
qq
qqq
q q
qqqqqq q
q
q
q
qq
qqqqqq
q
q
qq
q
qq
q
q
qqq
q q
q
q
q
qqq
qq
q
q
qqqqq qqq
qq
q
q
q
q
q
q
q q
q
q
q
qq
q
qqq
q
qqq
qqq
q
qq
q
q
q
q
q
q
qqqqq
q
q
qq
q
q
qq
q
q qq
q
q
q
q
q
q
q q
q q
qq
q
q
q
qqqqq
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
qqq
qqqq
q
q
q
qqq
q
q q
qq
q
q
q
q
qqqqq
q
qqqq
q q
q
qq
q
q
qq
qq
q
q
qq
qq
q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
qq
q qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q qq
q
q
qq
q
q
q
q
qqqq
q
q
qq qq
q
q
q
q
q
qq
qqq
q
qq
q
q
qqq q
q
q
qq
q
q
qq
qqqqqq
q
qq
q
q
q
q
qqqqqq
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q q qqq
q
q
q
q
q
qq qq
q
q
q
q
q
q
q
qq
q
q
q q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q q
q
q
q q
q
q qq
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qqq
q
q
qq
q
q
q
qqq
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
qq
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qqq
q
qq
q
q
qq
q
qq
q
q
qq
q
qqq q
q q
q
q
q
q
q
q
q
q qq q
q
qq
q
q
q
q
qq q
q
q q
qq
q
q
qq q
q
qq
q
q
q
qq
qqqq
qq
q
q
qq
q
q
q
q
qqqq
q
qq
q
q
q qqqq
qq
q
q
q
qqq qqqq
q
q qqq
q
qqq
q
q
qq q
qqq
q q
qq
q
q qqq
q
q
q
q
q q
q
qq
q
q
q
q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
qqqq
q
qqq
q
qqq
q
qqq q
q
q
q
qqq
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q q
q
q
q
q
q
q
qq
q
qq
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
qq qq
q
q
qqqq
q
q
qqq
q
q
q
q
q
q
q
q
qq
q
qqqq
q
qq qq
q
q
q
qq
qq
q
q
q
q q
q
q
qq
q
q
q
q
q
q
qq
q
q
qq
q
q
qq
q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q qq
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
qq
q
q
q
q q
qq
q
q
q
q
q
q
q
q
qq
qq q
q
qq
q
q
qq
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
qq
qqq
q
q
q
q q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
qq
qq
qq
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q qq q
q
q
q
q qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q q
qq
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qqq
q
q q
q q
q
q q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
qqq
qq
q
q qq
q
q
q
q
q qq q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
qq
q
q
q qq
q
q
q
q
qq
q
q
q
q
q q
q
q
q
q
q
q
q q
q
q
q
q
qq
q
q
q q
q
q
q
q
q
q
q
q
qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
q q
q
q
qq
q
qq
q
q
q
q
qq
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
qq qqq
q
qq
q q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q q
qq
q
qq
q
q q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qq
qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qq
qqq
q q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
q
q
q
q
qqqq
q
q
q
qqqq
q
q
qq
qqq
q
q
q
q
q
q qqqq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q
qq
q
q
q
q
qq qqq qqqqqqqq
q
qq
qqqqqq
q
qqqq
q
q
q
q
q q
q
q
q
q
qqq
q
q
q
q
q
q
q q
q
q
qqqq q
q
q
q
q
q
q
qq
q
q
qq
q
q
qq
q
qq
q
q
q
q
qq
qqq
q
q
q
q q
q q
q
qq
q
qq
q
q
q
qqq
qq
q q
q
q
qq
q
q
q
q
q
q
q qq
qqq
q
q
qq
q
q
q
q
q
qq
q qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qqqq
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
qq
q
q
qq
qq
q
q q
q
qq
q
q
qqqq
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q qq
q
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
qq
q
qq
qq q qq
q
qq
q
q
qqq
q
qq
qq
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
qqq
q
qq
q
q
qq
qq
q
q
qqqq
q
q
q
q
q
q
q q
q
q
qqq
q
q
q
q
q
qqqq qqqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
qq
q
q
q
q
qq
q
q q
q qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
qqqqq
q
q
q
qq
q
q qq
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
qq
q
q
qq
q
q
qqq
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q q
q
q
q
q qq q q
q
qq q
q
qq
q
q
q
q
q
qq
qq
qqq
qqq
q
q
q
q
q
qqqqqqqqq
q
q
qqqqqqqqqqqqqqqqq
qqq
q
q
q
q
qqqqq
qqq
q
q
qqqqq
q
q
q
qqqq
q
q
q
q
q
q
q
qq
q
q
q
qqqqq
q
qqqqq
q
qq
qqq
qqqq
q
qq
q
qq
q
q
q
qq
q
qq
−3
−2
−1
0
1
2
3
−2 0 2 4
PC1
PC2
0.00
0.25
0.50
0.75
value
Season
Feature space of M3 data
Exploring the feature space of large time series collections M3 competition data 10
q q
q
q
q
q
qq qq
q q
q
q
q
q
q
q
q
q
q
q
q
qqq
qq
q
qqq
q
q
qq q
q
qq q
qq qq
q qqq
q
q
q
q
qq
qqqq
q
q
qq
q
q
qq
q
q
q
q
q
q
qq
q
qq
q q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
qq
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
qqq
q q
q
q
q
qq
q
qqq
qqqq
q
qq
q
q
q
q
qq
q
qq
qq
qqqq
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
qq
q
q
q
q
q
qq
qq
q
q
q
q
q
qq
q
qqq
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
qqqq
q
qqq
q
qqq
qq
q
qq
qq
qqqqqqqqqq
q
q
q
qq
q
qq
q
qq
q qq
q q
q
q
qq
q
q
qqq
qq
qqq
q q
qqqqqq q
q
q
q
qq
qqqqqq
q
q
qq
q
qq
q
q
qqq
q q
q
q
q
qqq
qq
q
q
qqqqq qqq
qq
q
q
q
q
q
q
q q
q
q
q
qq
q
qqq
q
qqq
qqq
q
qq
q
q
q
q
q
q
qqqqq
q
q
qq
q
q
qq
q
q qq
q
q
q
q
q
q
q q
q q
qq
q
q
q
qqqqq
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
qqq
qqqq
q
q
q
qqq
q
q q
qq
q
q
q
q
qqqqq
q
qqqq
q q
q
qq
q
q
qq
qq
q
q
qq
qq
q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
qq
q qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q qq
q
q
qq
q
q
q
q
qqqq
q
q
qq qq
q
q
q
q
q
qq
qqq
q
qq
q
q
qqq q
q
q
qq
q
q
qq
qqqqqq
q
qq
q
q
q
q
qqqqqq
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q q qqq
q
q
q
q
q
qq qq
q
q
q
q
q
q
q
qq
q
q
q q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q q
q
q
q q
q
q qq
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qqq
q
q
qq
q
q
q
qqq
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
qq
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qqq
q
qq
q
q
qq
q
qq
q
q
qq
q
qqq q
q q
q
q
q
q
q
q
q
q qq q
q
qq
q
q
q
q
qq q
q
q q
qq
q
q
qq q
q
qq
q
q
q
qq
qqqq
qq
q
q
qq
q
q
q
q
qqqq
q
qq
q
q
q qqqq
qq
q
q
q
qqq qqqq
q
q qqq
q
qqq
q
q
qq q
qqq
q q
qq
q
q qqq
q
q
q
q
q q
q
qq
q
q
q
q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
qqqq
q
qqq
q
qqq
q
qqq q
q
q
q
qqq
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q q
q
q
q
q
q
q
qq
q
qq
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
qq qq
q
q
qqqq
q
q
qqq
q
q
q
q
q
q
q
q
qq
q
qqqq
q
qq qq
q
q
q
qq
qq
q
q
q
q q
q
q
qq
q
q
q
q
q
q
qq
q
q
qq
q
q
qq
q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q qq
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
qq
q
q
q
q q
qq
q
q
q
q
q
q
q
q
qq
qq q
q
qq
q
q
qq
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
qq
qqq
q
q
q
q q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
qq
qq
qq
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q qq q
q
q
q
q qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q q
qq
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qqq
q
q q
q q
q
q q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
qqq
qq
q
q qq
q
q
q
q
q qq q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
qq
q
q
q qq
q
q
q
q
qq
q
q
q
q
q q
q
q
q
q
q
q
q q
q
q
q
q
qq
q
q
q q
q
q
q
q
q
q
q
q
qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
q q
q
q
qq
q
qq
q
q
q
q
qq
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
qq qqq
q
qq
q q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q q
qq
q
qq
q
q q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qq
qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qq
qqq
q q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
q
q
q
q
qqqq
q
q
q
qqqq
q
q
qq
qqq
q
q
q
q
q
q qqqq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q
qq
q
q
q
q
qq qqq qqqqqqqq
q
qq
qqqqqq
q
qqqq
q
q
q
q
q q
q
q
q
q
qqq
q
q
q
q
q
q
q q
q
q
qqqq q
q
q
q
q
q
q
qq
q
q
qq
q
q
qq
q
qq
q
q
q
q
qq
qqq
q
q
q
q q
q q
q
qq
q
qq
q
q
q
qqq
qq
q q
q
q
qq
q
q
q
q
q
q
q qq
qqq
q
q
qq
q
q
q
q
q
qq
q qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qqqq
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
qq
q
q
qq
qq
q
q q
q
qq
q
q
qqqq
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q qq
q
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
qq
q
qq
qq q qq
q
qq
q
q
qqq
q
qq
qq
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
qqq
q
qq
q
q
qq
qq
q
q
qqqq
q
q
q
q
q
q
q q
q
q
qqq
q
q
q
q
q
qqqq qqqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
qq
q
q
q
q
qq
q
q q
q qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
qqqqq
q
q
q
qq
q
q qq
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
qq
q
q
qq
q
q
qqq
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q q
q
q
q
q qq q q
q
qq q
q
qq
q
q
q
q
q
qq
qq
qqq
qqq
q
q
q
q
q
qqqqqqqqq
q
q
qqqqqqqqqqqqqqqqq
qqq
q
q
q
q
qqqqq
qqq
q
q
qqqqq
q
q
q
qqqq
q
q
q
q
q
q
q
qq
q
q
q
qqqqq
q
qqqqq
q
qq
qqq
qqqq
q
qq
q
qq
q
q
q
qq
q
qq
−3
−2
−1
0
1
2
3
−2 0 2 4
PC1
PC2
0.25
0.50
0.75
value
Trend
Feature space of M3 data
Exploring the feature space of large time series collections M3 competition data 10
q q
q
q
q
q
qq qq
q q
q
q
q
q
q
q
q
q
q
q
q
qqq
qq
q
qqq
q
q
qq q
q
qq q
qq qq
q qqq
q
q
q
q
qq
qqqq
q
q
qq
q
q
qq
q
q
q
q
q
q
qq
q
qq
q q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
qq
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
qqq
q q
q
q
q
qq
q
qqq
qqqq
q
qq
q
q
q
q
qq
q
qq
qq
qqqq
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
qq
q
q
q
q
q
qq
qq
q
q
q
q
q
qq
q
qqq
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
qqqq
q
qqq
q
qqq
qq
q
qq
qq
qqqqqqqqqq
q
q
q
qq
q
qq
q
qq
q qq
q q
q
q
qq
q
q
qqq
qq
qqq
q q
qqqqqq q
q
q
q
qq
qq qqqq
q
q
qq
q
qq
q
q
qqq
q q
q
q
q
qqq
qq
q
q
qqqqq qqq
qq
q
q
q
q
q
q
q q
q
q
q
qq
q
qqq
q
qqq
qqq
q
qq
q
q
q
q
q
q
qqqqq
q
q
qq
q
q
qq
q
q qq
q
q
q
q
q
q
q q
q q
qq
q
q
q
qqqqq
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
qqq
qqqq
q
q
q
qqq
q
q q
qq
q
q
q
q
qqqqq
q
qqqq
q q
q
qq
q
q
qq
qq
q
q
qq
qq
q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
qq
q qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q qq
q
q
qq
q
q
q
q
qqqq
q
q
qq qq
q
q
q
q
q
qq
qqq
q
qq
q
q
qqq q
q
q
qq
q
q
qq
qqqqqq
q
qq
q
q
q
q
qqqqqq
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q q qqq
q
q
q
q
q
qq qq
q
q
q
q
q
q
q
qq
q
q
q q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q q
q
q
q q
q
q qq
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qqq
q
q
qq
q
q
q
qqq
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
qq
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qqq
q
qq
q
q
qq
q
qq
q
q
qq
q
qqq q
q q
q
q
q
q
q
q
q
q qq q
q
qq
q
q
q
q
qq q
q
q q
qq
q
q
qq q
q
qq
q
q
q
qq
qqqq
qq
q
q
qq
q
q
q
q
qqqq
q
qq
q
q
q qqqq
qq
q
q
q
qqq qqqq
q
q qqq
q
qqq
q
q
qq q
qqq
q q
qq
q
q qqq
q
q
q
q
q q
q
qq
q
q
q
q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
qqqq
q
qqq
q
qqq
q
qqq q
q
q
q
qqq
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q q
q
q
q
q
q
q
qq
q
qq
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
qq qq
q
q
qqqq
q
q
qqq
q
q
q
q
q
q
q
q
qq
q
qqqq
q
qq qq
q
q
q
qq
qq
q
q
q
q q
q
q
qq
q
q
q
q
q
q
qq
q
q
qq
q
q
qq
q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q qq
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
qq
q
q
q
q q
qq
q
q
q
q
q
q
q
q
qq
qq q
q
qq
q
q
qq
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
qq
qqq
q
q
q
q q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
qq
qq
qq
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q qq q
q
q
q
q qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q q
qq
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qqq
q
q q
q q
q
q q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
qqq
qq
q
q qq
q
q
q
q
q qq q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
qq
q
q
q qq
q
q
q
q
qq
q
q
q
q
q q
q
q
q
q
q
q
q q
q
q
q
q
qq
q
q
q q
q
q
q
q
q
q
q
q
qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
q q
q
q
qq
q
qq
q
q
q
q
qq
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
qq qqq
q
qq
q q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q q
qq
q
qq
q
q q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qq
qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qq
qqq
q q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
q
q
q
q
qqqq
q
q
q
qqqq
q
q
qq
qq q
q
q
q
q
q
q qqqq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q
qq
q
q
q
q
qq qqq qqqqqqqq
q
qq
qqqqqq
q
qqqq
q
q
q
q
q q
q
q
q
q
qqq
q
q
q
q
q
q
q q
q
q
qqqq q
q
q
q
q
q
q
qq
q
q
qq
q
q
qq
q
qq
q
q
q
q
qq
qqq
q
q
q
q q
q q
q
qq
q
qq
q
q
q
qqq
qq
q q
q
q
qq
q
q
q
q
q
q
q qq
qqq
q
q
qq
q
q
q
q
q
qq
q qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qqqq
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
qq
q
q
qq
qq
q
q q
q
qq
q
q
qqqq
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q qq
q
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
qq
q
qq
qq q qq
q
qq
q
q
qqq
q
qq
qq
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
qqq
q
qq
q
q
qq
qq
q
q
qqqq
q
q
q
q
q
q
q q
q
q
qqq
q
q
q
q
q
qqqq qqqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
qq
q
q
q
q
qq
q
q q
q qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
qqq qq
q
q
q
qq
q
q qq
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
qq
q
q
qq
q
q
qqq
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q q
q
q
q
q qq q q
q
qq q
q
qq
q
q
q
q
q
qq
qq
qqq
qqq
q
q
q
q
q
qqqqqqqqq
q
q
qqqqqqqqqqqqqqqqq
qqq
q
q
q
q
qqqqq
qqq
q
q
qqqqq
q
q
q
qqqq
q
q
q
q
q
q
q
qq
q
q
q
qqqqq
q
qqqqq
q
qq
qqq
qqqq
q
qq
q
qq
q
q
q
qq
q
qq
−3
−2
−1
0
1
2
3
−2 0 2 4
PC1
PC2
0.0
0.5
value
ACF
Feature space of M3 data
Exploring the feature space of large time series collections M3 competition data 10
q q
q
q
q
q
qq qq
q q
q
q
q
q
q
q
q
q
q
q
q
qqq
qq
q
qqq
q
q
qq q
q
qq q
qq qq
q qqq
q
q
q
q
qq
qqqq
q
q
qq
q
q
qq
q
q
q
q
q
q
qq
q
qq
q q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
qq
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
qqq
q q
q
q
q
qq
q
qqq
qqqq
q
qq
q
q
q
q
qq
q
qq
qq
qqqq
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
qq
q
q
q
q
q
qq
qq
q
q
q
q
q
qq
q
qqq
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
qqqq
q
qqq
q
qqq
qq
q
qq
qq
qqqqqqqqqq
q
q
q
qq
q
qq
q
qq
q qq
q q
q
q
qq
q
q
qqq
qq
qqq
q q
qqqqqq q
q
q
q
qq
qq qqqq
q
q
qq
q
qq
q
q
qqq
q q
q
q
q
qqq
qq
q
q
qqqqq qqq
qq
q
q
q
q
q
q
q q
q
q
q
qq
q
qqq
q
qqq
qqq
q
qq
q
q
q
q
q
q
qqqqq
q
q
qq
q
q
qq
q
q qq
q
q
q
q
q
q
q q
q q
qq
q
q
q
qqqqq
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
qqq
qqqq
q
q
q
qqq
q
q q
qq
q
q
q
q
qqqqq
q
qqqq
q q
q
qq
q
q
qq
qq
q
q
qq
qq
q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
qq
q qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q qq
q
q
qq
q
q
q
q
qqqq
q
q
qq qq
q
q
q
q
q
qq
qqq
q
qq
q
q
qqq q
q
q
qq
q
q
qq
qqqqqq
q
qq
q
q
q
q
qqqqqq
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q q qqq
q
q
q
q
q
qq qq
q
q
q
q
q
q
q
qq
q
q
q q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q q
q
q
q q
q
q qq
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qqq
q
q
qq
q
q
q
qqq
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
qq
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qqq
q
qq
q
q
qq
q
qq
q
q
qq
q
qqq q
q q
q
q
q
q
q
q
q
q qq q
q
qq
q
q
q
q
qq q
q
q q
qq
q
q
qq q
q
qq
q
q
q
qq
qqqq
qq
q
q
qq
q
q
q
q
qqqq
q
qq
q
q
q qqqq
qq
q
q
q
qqq qqqq
q
q qqq
q
qqq
q
q
qq q
qqq
q q
qq
q
q qqq
q
q
q
q
q q
q
qq
q
q
q
q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
qqqq
q
qqq
q
qqq
q
qqq q
q
q
q
qqq
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q q
q
q
q
q
q
q
qq
q
qq
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
qq qq
q
q
qqqq
q
q
qqq
q
q
q
q
q
q
q
q
qq
q
qqqq
q
qq qq
q
q
q
qq
qq
q
q
q
q q
q
q
qq
q
q
q
q
q
q
qq
q
q
qq
q
q
qq
q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q qq
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
qq
q
q
q
q q
qq
q
q
q
q
q
q
q
q
qq
qq q
q
qq
q
q
qq
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
qq
qqq
q
q
q
q q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
qq
qq
qq
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q qq q
q
q
q
q qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q q
qq
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qqq
q
q q
q q
q
q q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
qqq
qq
q
q qq
q
q
q
q
q qq q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
qq
q
q
q qq
q
q
q
q
qq
q
q
q
q
q q
q
q
q
q
q
q
q q
q
q
q
q
qq
q
q
q q
q
q
q
q
q
q
q
q
qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
q q
q
q
qq
q
qq
q
q
q
q
qq
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
qq qqq
q
qq
q q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q q
qq
q
qq
q
q q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qq
qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qq
qqq
q q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
q
q
q
q
qqqq
q
q
q
qqqq
q
q
qq
qq q
q
q
q
q
q
q qqqq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q
qq
q
q
q
q
qq qqq qqqqqqqq
q
qq
qqqqqq
q
qqqq
q
q
q
q
q q
q
q
q
q
qqq
q
q
q
q
q
q
q q
q
q
qqqq q
q
q
q
q
q
q
qq
q
q
qq
q
q
qq
q
qq
q
q
q
q
qq
qqq
q
q
q
q q
q q
q
qq
q
qq
q
q
q
qqq
qq
q q
q
q
qq
q
q
q
q
q
q
q qq
qqq
q
q
qq
q
q
q
q
q
qq
q qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qqqq
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
qq
q
q
qq
qq
q
q q
q
qq
q
q
qqqq
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q qq
q
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
qq
q
qq
qq q qq
q
qq
q
q
qqq
q
qq
qq
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
qqq
q
qq
q
q
qq
qq
q
q
qqqq
q
q
q
q
q
q
q q
q
q
qqq
q
q
q
q
q
qqqq qqqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
qq
q
q
q
q
qq
q
q q
q qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
qqq qq
q
q
q
qq
q
q qq
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
qq
q
q
qq
q
q
qqq
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q q
q
q
q
q qq q q
q
qq q
q
qq
q
q
q
q
q
qq
qq
qqq
qqq
q
q
q
q
q
qqqqqqqqq
q
q
qqqqqqqqqqqqqqqqq
qqq
q
q
q
q
qqqqq
qqq
q
q
qqqqq
q
q
q
qqqq
q
q
q
q
q
q
q
qq
q
q
q
qqqqq
q
qqqqq
q
qq
qqq
qqqq
q
qq
q
qq
q
q
q
qq
q
qq
−3
−2
−1
0
1
2
3
−2 0 2 4
PC1
PC2
0.5
0.6
0.7
0.8
0.9
value
SpecEntr
Feature space of M3 data
Exploring the feature space of large time series collections M3 competition data 10
q q
q
q
q
q
qq qq
q q
q
q
q
q
q
q
q
q
q
q
q
qqq
qq
q
qqq
q
q
qq q
q
qq q
qq qq
q qqq
q
q
q
q
qq
qqqq
q
q
qq
q
q
qq
q
q
q
q
q
q
qq
q
qq
q q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
qq
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
qqq
q q
q
q
q
qq
q
qqq
qqqq
q
qq
q
q
q
q
qq
q
qq
qq
qqqq
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
qq
q
q
q
q
q
qq
qq
q
q
q
q
q
qq
q
qqq
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
qqqq
q
qqq
q
qqq
qq
q
qq
qq
qqqqqqqqqq
q
q
q
qq
q
qq
q
qq
q qq
q q
q
q
qq
q
q
qqq
qq
qqq
q q
qqqqqq q
q
q
q
qq
qqqqqq
q
q
qq
q
qq
q
q
qqq
q q
q
q
q
qqq
qq
q
q
qqqqq qqq
qq
q
q
q
q
q
q
q q
q
q
q
qq
q
qqq
q
qqq
qqq
q
qq
q
q
q
q
q
q
qqqqq
q
q
qq
q
q
qq
q
q qq
q
q
q
q
q
q
q q
q q
qq
q
q
q
qqqqq
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
qqq
qqqq
q
q
q
qqq
q
q q
qq
q
q
q
q
qqqqq
q
qqqq
q q
q
qq
q
q
qq
qq
q
q
qq
qq
q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
qq
q qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q qq
q
q
qq
q
q
q
q
qqqq
q
q
qq qq
q
q
q
q
q
qq
qqq
q
qq
q
q
qqq q
q
q
qq
q
q
qq
qqqqqq
q
qq
q
q
q
q
qqqqqq
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q q qqq
q
q
q
q
q
qq qq
q
q
q
q
q
q
q
qq
q
q
q q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q q
q
q
q q
q
q qq
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qqq
q
q
qq
q
q
q
qqq
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
qq
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qqq
q
qq
q
q
qq
q
qq
q
q
qq
q
qqq q
q q
q
q
q
q
q
q
q
q qq q
q
qq
q
q
q
q
qq q
q
q q
qq
q
q
qq q
q
qq
q
q
q
qq
qqqq
qq
q
q
qq
q
q
q
q
qqqq
q
qq
q
q
q qqqq
qq
q
q
q
qqq qqqq
q
q qqq
q
qqq
q
q
qq q
qqq
q q
qq
q
q qqq
q
q
q
q
q q
q
qq
q
q
q
q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
qqqq
q
qqq
q
qqq
q
qqq q
q
q
q
qqq
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q q
q
q
q
q
q
q
qq
q
qq
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
qq qq
q
q
qqqq
q
q
qqq
q
q
q
q
q
q
q
q
qq
q
qqqq
q
qq qq
q
q
q
qq
qq
q
q
q
q q
q
q
qq
q
q
q
q
q
q
qq
q
q
qq
q
q
qq
q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q qq
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
qq
q
q
q
q q
qq
q
q
q
q
q
q
q
q
qq
qq q
q
qq
q
q
qq
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
qq
qqq
q
q
q
q q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
qq
qq
qq
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q qq q
q
q
q
q qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q q
qq
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qqq
q
q q
q q
q
q q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
qqq
qq
q
q qq
q
q
q
q
q qq q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
qq
q
q
q qq
q
q
q
q
qq
q
q
q
q
q q
q
q
q
q
q
q
q q
q
q
q
q
qq
q
q
q q
q
q
q
q
q
q
q
q
qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
q q
q
q
qq
q
qq
q
q
q
q
qq
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
qq qqq
q
qq
q q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q q
qq
q
qq
q
q q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qq
qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qq
qqq
q q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
q
q
q
q
qqqq
q
q
q
qqqq
q
q
qq
qqq
q
q
q
q
q
q qqqq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q
qq
q
q
q
q
qq qqq qqqqqqqq
q
qq
qqqqqq
q
qqqq
q
q
q
q
q q
q
q
q
q
qqq
q
q
q
q
q
q
q q
q
q
qqqq q
q
q
q
q
q
q
qq
q
q
qq
q
q
qq
q
qq
q
q
q
q
qq
qqq
q
q
q
q q
q q
q
qq
q
qq
q
q
q
qqq
qq
q q
q
q
qq
q
q
q
q
q
q
q qq
qqq
q
q
qq
q
q
q
q
q
qq
q qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qqqq
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
qq
q
q
qq
qq
q
q q
q
qq
q
q
qqqq
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q qq
q
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
qq
q
qq
qq q qq
q
qq
q
q
qqq
q
qq
qq
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
qqq
q
qq
q
q
qq
qq
q
q
qqqq
q
q
q
q
q
q
q q
q
q
qqq
q
q
q
q
q
qqqq qqqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
qq
q
q
q
q
qq
q
q q
q qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
qqqqq
q
q
q
qq
q
q qq
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
qq
q
q
qq
q
q
qqq
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q q
q
q
q
q qq q q
q
qq q
q
qq
q
q
q
q
q
qq
qq
qqq
qqq
q
q
q
q
q
qqqqqqqqq
q
q
qqqqqqqqqqqqqqqqq
qqq
q
q
q
q
qqqqq
qqq
q
q
qqqqq
q
q
q
qqqq
q
q
q
q
q
q
q
qq
q
q
q
qqqqq
q
qqqqq
q
qq
qqq
qqqq
q
qq
q
qq
q
q
q
qq
q
qq
−3
−2
−1
0
1
2
3
−2 0 2 4
PC1
PC2
0.00
0.25
0.50
0.75
1.00
value
Lambda
Feature space of M3 data
Exploring the feature space of large time series collections M3 competition data 11
q q
q
q
q
q
qq qq
q q
q
q
q
q
q
q
q
q
q
q
q
qq q
qq
q
qqq
q
q
qq q
q
qq q
qq qq
q qqq
q
q
q
q
qq
qqqq
q
q
qq
q
q
qq
q
q
q
q
q
q
qq
q
qq
q q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
qq
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q q
q
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
qq
qqq
q q
q
q
q
qq
q
qqq
qqqq
q
qq
q
q
q
q
qq
q
qq
qq
qqqq
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
qq
q
q
q
q
q
qq
qq
q
q
q
q
q
qq
q
qqq
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
qqqq
q
qqq
q
qqq
qq
q
qq
qq
qqqqqqqqqq
q
q
q
q q
q
qq
q
qq
q q
q
q q
q
q
qq
q
q
qqq
qq
qqq
q q
qqqqqq q
q
q
q
qq
qq qqqq
q
q
qq
q
qq
q
q
qqq
q q
q
q
q
qqq
qq
q
q
qqqqq
qqq
qq
q
q
q
q
q
q
q q
q
q
q
qq
q
qqq
q
qqq
qqq
q
qq
q
q
q
q
q
q
qqqqq
q
q
qq
q
q
qq
q
q qq
q
q
q
q
q
q
q q
q q
qq
q
q
q
qqqqq
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
qqq
qqqq
q
q
q
qqq
q
q q
qq
q
q
q
q
qqqqq
q
qqqq
q q
q
qq
q
q
qq
qq
q
q
qq
q q
q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
qq
q qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q qq
q
q
qq
q
q
q
q
qqq q
q
q
qq qq
q
q
q
q
q
qq
qqq
q
q
q
q
q
qqq q
q
q
qq
q
q
qq
qqqqqq
q
qq
q
q
q
q
qqqqqq
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q q qqq
q
q
q
q
q
qq qq
q
q
q
q
q
q
q
qq
q
q
q q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q q
q
q
q q
q
q qq
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qqq
q
q
qq
q
q
q
qqq
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
qq
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qqq
q
qq
q
q
qq
q
qq
q
q
qq
q
qqq q
q q
q
q
q
q
q
q
q
q qq q
q
qq
q
q
q
q
qq q
q
q q
qq
q
q
qq q
q
qq
q
q
q
qq
qqq q
q
q
q
q
qq
q
q
q
q
qqqq
q
qq
q
q
q qqqq
qq
q
q
q
qqq qqqq
q
q qqq
q
qqq
q
q
q
q q
qqq
q q
qq
q
q
qqq
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
qqqq
q
qqq
q
qqq
q
qqq q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q q
q
q
q
q
q
q
qq
q
qq
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
qq qq
q
q
q
qqq
q
q
qqq
q
q
q
q
q
q
q
q
qq
q
qqqq
q
qq qq
q
q
q
qq
qq
q
q
q
q q
q
q
qq
q
q
q
q
q
q
qq
q
q
q q
q
q
qq
q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q qq
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
qq
q
q
q
q q
qq
q
q
q
q
q
q
q
q
qq
qq q
q
qq
q
q
qq
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
qq
qqq
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
qq
qq
qq
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q qq q
q
q
q
q qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q q
q q
q
q q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q qq
qq
q
q qq
q
q
q
q
q qq q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
qq
q
q
q qq
q
q
q
q
qq
q
q
q
q
q q
q
q
q
q
q
q
q q
q
q
q
q
qq
q
q
q q
q
q
q
q
q
q
q
q
qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
q q
q
q
qq
q
qq
q
q
q
q
qq
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
qq qqq
q
qq
q q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q q
qq
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qq
qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qq
q qq
q q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
q
q
q
q
qqqq
q
q
q
qqqq
q
q
qq
qq q
q
q
q
q
q
q qqqq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q
qq
q
q
q
q
qq qqq qqqq qq
qq
q
qq
qqqqqq
q
q
qqq
q
q
q
q
q q
q
q
q
q
qqq
q
q
q
q
q
q
q q
q
q
qqq
q q
q
q
q
q
q
q
qq
q
q
qq
q
q
qq
q
qq
q
q
q
q
qq
qqq
q
q
q
q q
q q
q
qq
q
qq
q
q
q
qqq
q
q
q q
q
q
qq
q
q
q
q
q
q
q qq
qqq
q
q
qq
q
q
q
q
q
qq
q qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qqqq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
qq
q
q
qq
qq
q
q q
q
qq
q
q
qqqq
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q qq
q
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
qq
q
qq
qq q qq
q
qq
q
q
qqq
q
qq
qq
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
qqq
q
qq
q
q
qq
qq
q
q
qqqq
q
q
q
q
q
q
q q
q
q
qqq
q
q
q
q
q
qqqq qqqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
qq
q
q
q
q
qq
q
q q
q qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q qq qq
q
q
q
qq
q
q qq
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
qq
q
q
qq
q
q
qqq
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q q
q
q
q
q qq q q
q
qq q
q
qq
q
q
q
q
q
qq
qq
qqq
qqq
q
q
q
q
q
q qqqqqqqq
q
q
qqqqqqqqqqqqqqqqq
qqq
q
q
q
q
qqqqq
qqq
q
q
qqqqq
q
q
q
qqqq
q
q
q
q
q
q
q
qq
q
q
q
qqqqq
q
qqqqq
q
qq
qqq
qqqq
q
qq
q
qq
q
q
q
qq
q
qq
−2 0 2 4
−3−2−10123
PC1
PC2
q
q
q
q
Feature space of M3 data
Exploring the feature space of large time series collections M3 competition data 11
N1627
1990 1991 1992 1993 1994
N0636
1920 1925 1930 1935 1940 1945
N2981
0 10 20 30 40 50 60
N2699
1984 1986 1988 1990 1992
Feature space of M3 data
Exploring the feature space of large time series collections M3 competition data 11
q q
q
q
q
q
qq qq
q q
q
q
q
q
q
q
q
q
q
q
q
qq q
qq
q
qqq
q
q
qq q
q
qq q
qq qq
q qqq
q
q
q
q
qq
qqqq
q
q
qq
q
q
qq
q
q
q
q
q
q
qq
q
qq
q q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
qq
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q q
q
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
qq
qqq
q q
q
q
q
qq
q
qqq
qqqq
q
qq
q
q
q
q
qq
q
qq
qq
qqqq
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
qq
q
q
q
q
q
qq
qq
q
q
q
q
q
qq
q
qqq
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
qqqq
q
qqq
q
qqq
qq
q
qq
qq
qqqqqqqqqq
q
q
q
q q
q
qq
q
qq
q q
q
q q
q
q
qq
q
q
qqq
qq
qqq
q q
qqqqqq q
q
q
q
qq
qq qqqq
q
q
qq
q
qq
q
q
qqq
q q
q
q
q
qqq
qq
q
q
qqqqq
qqq
qq
q
q
q
q
q
q
q q
q
q
q
qq
q
qqq
q
qqq
qqq
q
qq
q
q
q
q
q
q
qqqqq
q
q
qq
q
q
qq
q
q qq
q
q
q
q
q
q
q q
q q
qq
q
q
q
qqqqq
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
qqq
qqqq
q
q
q
qqq
q
q q
qq
q
q
q
q
qqqqq
q
qqqq
q q
q
qq
q
q
qq
qq
q
q
qq
q q
q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
qq
q qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q qq
q
q
qq
q
q
q
q
qqq q
q
q
qq qq
q
q
q
q
q
qq
qqq
q
q
q
q
q
qqq q
q
q
qq
q
q
qq
qqqqqq
q
qq
q
q
q
q
qqqqqq
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q q qqq
q
q
q
q
q
qq qq
q
q
q
q
q
q
q
qq
q
q
q q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q q
q
q
q q
q
q qq
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qqq
q
q
qq
q
q
q
qqq
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
qq
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qqq
q
qq
q
q
qq
q
qq
q
q
qq
q
qqq q
q q
q
q
q
q
q
q
q
q qq q
q
qq
q
q
q
q
qq q
q
q q
qq
q
q
qq q
q
qq
q
q
q
qq
qqq q
q
q
q
q
qq
q
q
q
q
qqqq
q
qq
q
q
q qqqq
qq
q
q
q
qqq qqqq
q
q qqq
q
qqq
q
q
q
q q
qqq
q q
qq
q
q
qqq
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
qqqq
q
qqq
q
qqq
q
qqq q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q q
q
q
q
q
q
q
qq
q
qq
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
qq qq
q
q
q
qqq
q
q
qqq
q
q
q
q
q
q
q
q
qq
q
qqqq
q
qq qq
q
q
q
qq
qq
q
q
q
q q
q
q
qq
q
q
q
q
q
q
qq
q
q
q q
q
q
qq
q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q qq
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
qq
q
q
q
q q
qq
q
q
q
q
q
q
q
q
qq
qq q
q
qq
q
q
qq
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
qq
qqq
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
qq
qq
qq
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q qq q
q
q
q
q qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q q
q q
q
q q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q qq
qq
q
q qq
q
q
q
q
q qq q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
qq
q
q
q qq
q
q
q
q
qq
q
q
q
q
q q
q
q
q
q
q
q
q q
q
q
q
q
qq
q
q
q q
q
q
q
q
q
q
q
q
qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
q q
q
q
qq
q
qq
q
q
q
q
qq
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
qq qqq
q
qq
q q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q q
qq
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qq
qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qq
q qq
q q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
q
q
q
q
qqqq
q
q
q
qqqq
q
q
qq
qq q
q
q
q
q
q
q qqqq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q
qq
q
q
q
q
qq qqq qqqq qq
qq
q
qq
qqqqqq
q
q
qqq
q
q
q
q
q q
q
q
q
q
qqq
q
q
q
q
q
q
q q
q
q
qqq
q q
q
q
q
q
q
q
qq
q
q
qq
q
q
qq
q
qq
q
q
q
q
qq
qqq
q
q
q
q q
q q
q
qq
q
qq
q
q
q
qqq
q
q
q q
q
q
qq
q
q
q
q
q
q
q qq
qqq
q
q
qq
q
q
q
q
q
qq
q qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qqqq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
qq
q
q
qq
qq
q
q q
q
qq
q
q
qqqq
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q qq
q
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
qq
q
qq
qq q qq
q
qq
q
q
qqq
q
qq
qq
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
qqq
q
qq
q
q
qq
qq
q
q
qqqq
q
q
q
q
q
q
q q
q
q
qqq
q
q
q
q
q
qqqq qqqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
qq
q
q
q
q
qq
q
q q
q qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q qq qq
q
q
q
qq
q
q qq
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
qq
q
q
qq
q
q
qqq
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q q
q
q
q
q qq q q
q
qq q
q
qq
q
q
q
q
q
qq
qq
qqq
qqq
q
q
q
q
q
q qqqqqqqq
q
q
qqqqqqqqqqqqqqqqq
qqq
q
q
q
q
qqqqq
qqq
q
q
qqqqq
q
q
q
qqqq
q
q
q
q
q
q
q
qq
q
q
q
qqqqq
q
qqqqq
q
qq
qqq
qqqq
q
qq
q
qq
q
q
q
qq
q
qq
−2 0 2 4
−3−2−10123
PC1
PC2
Yahoo log returns series
Feature space of M3 data
Exploring the feature space of large time series collections M3 competition data 11
q q
q
q
q
q
qq qq
q q
q
q
q
q
q
q
q
q
q
q
q
qq q
qq
q
qqq
q
q
qq q
q
qq q
qq qq
q qqq
q
q
q
q
qq
qqqq
q
q
qq
q
q
qq
q
q
q
q
q
q
qq
q
qq
q q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
qq
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q q
q
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
qq
qqq
q q
q
q
q
qq
q
qqq
qqqq
q
qq
q
q
q
q
qq
q
qq
qq
qqqq
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
qq
q
q
q
q
q
qq
qq
q
q
q
q
q
qq
q
qqq
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
qqqq
q
qqq
q
qqq
qq
q
qq
qq
qqqqqqqqqq
q
q
q
q q
q
qq
q
qq
q q
q
q q
q
q
qq
q
q
qqq
qq
qqq
q q
qqqqqq q
q
q
q
qq
qq qqqq
q
q
qq
q
qq
q
q
qqq
q q
q
q
q
qqq
qq
q
q
qqqqq
qqq
qq
q
q
q
q
q
q
q q
q
q
q
qq
q
qqq
q
qqq
qqq
q
qq
q
q
q
q
q
q
qqqqq
q
q
qq
q
q
qq
q
q qq
q
q
q
q
q
q
q q
q q
qq
q
q
q
qqqqq
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
qqq
qqqq
q
q
q
qqq
q
q q
qq
q
q
q
q
qqqqq
q
qqqq
q q
q
qq
q
q
qq
qq
q
q
qq
q q
q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
qq
q qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q qq
q
q
qq
q
q
q
q
qqq q
q
q
qq qq
q
q
q
q
q
qq
qqq
q
q
q
q
q
qqq q
q
q
qq
q
q
qq
qqqqqq
q
qq
q
q
q
q
qqqqqq
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q q qqq
q
q
q
q
q
qq qq
q
q
q
q
q
q
q
qq
q
q
q q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q q
q
q
q q
q
q qq
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qqq
q
q
qq
q
q
q
qqq
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
qq
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qqq
q
qq
q
q
qq
q
qq
q
q
qq
q
qqq q
q q
q
q
q
q
q
q
q
q qq q
q
qq
q
q
q
q
qq q
q
q q
qq
q
q
qq q
q
qq
q
q
q
qq
qqq q
q
q
q
q
qq
q
q
q
q
qqqq
q
qq
q
q
q qqqq
qq
q
q
q
qqq qqqq
q
q qqq
q
qqq
q
q
q
q q
qqq
q q
qq
q
q
qqq
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
qqqq
q
qqq
q
qqq
q
qqq q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q q
q
q
q
q
q
q
qq
q
qq
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
qq qq
q
q
q
qqq
q
q
qqq
q
q
q
q
q
q
q
q
qq
q
qqqq
q
qq qq
q
q
q
qq
qq
q
q
q
q q
q
q
qq
q
q
q
q
q
q
qq
q
q
q q
q
q
qq
q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q qq
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
qq
q
q
q
q q
qq
q
q
q
q
q
q
q
q
qq
qq q
q
qq
q
q
qq
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
qq
qqq
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
qq
qq
qq
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q qq q
q
q
q
q qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q q
q q
q
q q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q qq
qq
q
q qq
q
q
q
q
q qq q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
qq
q
q
q qq
q
q
q
q
qq
q
q
q
q
q q
q
q
q
q
q
q
q q
q
q
q
q
qq
q
q
q q
q
q
q
q
q
q
q
q
qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
q q
q
q
qq
q
qq
q
q
q
q
qq
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
qq qqq
q
qq
q q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q q
qq
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qq
qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qq
q qq
q q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
q
q
q
q
qqqq
q
q
q
qqqq
q
q
qq
qq q
q
q
q
q
q
q qqqq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q
qq
q
q
q
q
qq qqq qqqq qq
qq
q
qq
qqqqqq
q
q
qqq
q
q
q
q
q q
q
q
q
q
qqq
q
q
q
q
q
q
q q
q
q
qqq
q q
q
q
q
q
q
q
qq
q
q
qq
q
q
qq
q
qq
q
q
q
q
qq
qqq
q
q
q
q q
q q
q
qq
q
qq
q
q
q
qqq
q
q
q q
q
q
qq
q
q
q
q
q
q
q qq
qqq
q
q
qq
q
q
q
q
q
qq
q qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qqqq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
qq
q
q
qq
qq
q
q q
q
qq
q
q
qqqq
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q qq
q
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
qq
q
qq
qq q qq
q
qq
q
q
qqq
q
qq
qq
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
qqq
q
qq
q
q
qq
qq
q
q
qqqq
q
q
q
q
q
q
q q
q
q
qqq
q
q
q
q
q
qqqq qqqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
qq
q
q
q
q
qq
q
q q
q qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q qq qq
q
q
q
qq
q
q qq
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
qq
q
q
qq
q
q
qqq
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q q
q
q
q
q qq q q
q
qq q
q
qq
q
q
q
q
q
qq
qq
qqq
qqq
q
q
q
q
q
q qqqqqqqq
q
q
qqqqqqqqqqqqqqqqq
qqq
q
q
q
q
qqqqq
qqq
q
q
qqqqq
q
q
q
qqqq
q
q
q
q
q
q
q
qq
q
q
q
qqqqq
q
qqqqq
q
qq
qqq
qqqq
q
qq
q
qq
q
q
q
qq
q
qq
−2 0 2 4
−3−2−10123
PC1
PC2
qYahoo log returns series
Feature space of M3 data
Exploring the feature space of large time series collections M3 competition data 12
q q
q
q
q
q
qq qq
q q
q
q
q
q
q
q
q
q
q
q
q
qq q
qq
q
qqq
q
q
qq q
q
qq q
qq qq
q qqq
q
q
q
q
qq
qqqq
q
q
qq
q
q
qq
q
q
q
q
q
q
qq
q
qq
q q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
qq
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q q
q
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
qq
qqq
q q
q
q
q
qq
q
qqq
qqqq
q
qq
q
q
q
q
qq
q
qq
qq
qqqq
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
qq
q
q
q
q
q
qq
qq
q
q
q
q
q
qq
q
qqq
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
qqqq
q
qqq
q
qqq
qq
q
qq
qq
qqqqqqqqqq
q
q
q
q q
q
qq
q
qq
q q
q
q q
q
q
qq
q
q
qqq
qq
qqq
q q
qqqqqq q
q
q
q
qq
qq qqqq
q
q
qq
q
qq
q
q
qqq
q q
q
q
q
qqq
qq
q
q
qqqqq
qqq
qq
q
q
q
q
q
q
q q
q
q
q
qq
q
qqq
q
qqq
qqq
q
qq
q
q
q
q
q
q
qqqqq
q
q
qq
q
q
qq
q
q qq
q
q
q
q
q
q
q q
q q
qq
q
q
q
qqqqq
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
qqq
qqqq
q
q
q
qqq
q
q q
qq
q
q
q
q
qqqqq
q
qqqq
q q
q
qq
q
q
qq
qq
q
q
qq
q q
q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
qq
q qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q qq
q
q
qq
q
q
q
q
qqq q
q
q
qq qq
q
q
q
q
q
qq
qqq
q
q
q
q
q
qqq q
q
q
qq
q
q
qq
qqqqqq
q
qq
q
q
q
q
qqqqqq
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q q qqq
q
q
q
q
q
qq qq
q
q
q
q
q
q
q
qq
q
q
q q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q q
q
q
q q
q
q qq
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qqq
q
q
qq
q
q
q
qqq
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
qq
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qqq
q
qq
q
q
qq
q
qq
q
q
qq
q
qqq q
q q
q
q
q
q
q
q
q
q qq q
q
qq
q
q
q
q
qq q
q
q q
qq
q
q
qq q
q
qq
q
q
q
qq
qqq q
q
q
q
q
qq
q
q
q
q
qqqq
q
qq
q
q
q qqqq
qq
q
q
q
qqq qqqq
q
q qqq
q
qqq
q
q
q
q q
qqq
q q
qq
q
q
qqq
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
qqqq
q
qqq
q
qqq
q
qqq q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q q
q
q
q
q
q
q
qq
q
qq
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
qq qq
q
q
q
qqq
q
q
qqq
q
q
q
q
q
q
q
q
qq
q
qqqq
q
qq qq
q
q
q
qq
qq
q
q
q
q q
q
q
qq
q
q
q
q
q
q
qq
q
q
q q
q
q
qq
q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q qq
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
qq
q
q
q
q q
qq
q
q
q
q
q
q
q
q
qq
qq q
q
qq
q
q
qq
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
qq
qqq
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
qq
qq
qq
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q qq q
q
q
q
q qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q q
q q
q
q q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q qq
qq
q
q qq
q
q
q
q
q qq q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
qq
q
q
q qq
q
q
q
q
qq
q
q
q
q
q q
q
q
q
q
q
q
q q
q
q
q
q
qq
q
q
q q
q
q
q
q
q
q
q
q
qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
q q
q
q
qq
q
qq
q
q
q
q
qq
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
qq qqq
q
qq
q q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q q
qq
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qq
qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qq
q qq
q q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
q
q
q
q
qqqq
q
q
q
qqqq
q
q
qq
qq q
q
q
q
q
q
q qqqq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q
qq
q
q
q
q
qq qqq qqqq qq
qq
q
qq
qqqqqq
q
q
qqq
q
q
q
q
q q
q
q
q
q
qqq
q
q
q
q
q
q
q q
q
q
qqq
q q
q
q
q
q
q
q
qq
q
q
qq
q
q
qq
q
qq
q
q
q
q
qq
qqq
q
q
q
q q
q q
q
qq
q
qq
q
q
q
qqq
q
q
q q
q
q
qq
q
q
q
q
q
q
q qq
qqq
q
q
qq
q
q
q
q
q
qq
q qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qqqq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
qq
q
q
qq
qq
q
q q
q
qq
q
q
qqqq
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q qq
q
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
qq
q
qq
qq q qq
q
qq
q
q
qqq
q
qq
qq
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
qqq
q
qq
q
q
qq
qq
q
q
qqqq
q
q
q
q
q
q
q q
q
q
qqq
q
q
q
q
q
qqqq qqqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
qq
q
q
q
q
qq
q
q q
q qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q qq qq
q
q
q
qq
q
q qq
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
qq
q
q
qq
q
q
qqq
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q q
q
q
q
q qq q q
q
qq q
q
qq
q
q
q
q
q
qq
qq
qqq
qqq
q
q
q
q
q
q qqqqqqqq
q
q
qqqqqqqqqqqqqqqqq
qqq
q
q
q
q
qqqqq
qqq
q
q
qqqqq
q
q
q
qqqq
q
q
q
q
q
q
q
qq
q
q
q
qqqqq
q
qqqqq
q
qq
qqq
qqqq
q
qq
q
qq
q
q
q
qq
q
qq
−2 0 2 4
−3−2−10123
PC1
PC2
White noise series
Feature space of M3 data
Exploring the feature space of large time series collections M3 competition data 13
q q
q
q
q
q
qq qq
q q
q
q
q
q
q
q
q
q
q
q
q
qq q
qq
q
qqq
q
q
qq q
q
qq q
qq qq
q qqq
q
q
q
q
qq
qqqq
q
q
qq
q
q
qq
q
q
q
q
q
q
qq
q
qq
q q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
qq
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q q
q
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
qq
qqq
q q
q
q
q
qq
q
qqq
qqqq
q
qq
q
q
q
q
qq
q
qq
qq
qqqq
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
qq
q
q
q
q
q
qq
qq
q
q
q
q
q
qq
q
qqq
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
qqqq
q
qqq
q
qqq
qq
q
qq
qq
qqqqqqqqqq
q
q
q
q q
q
qq
q
qq
q q
q
q q
q
q
qq
q
q
qqq
qq
qqq
q q
qqqqqq q
q
q
q
qq
qq qqqq
q
q
qq
q
qq
q
q
qqq
q q
q
q
q
qqq
qq
q
q
qqqqq
qqq
qq
q
q
q
q
q
q
q q
q
q
q
qq
q
qqq
q
qqq
qqq
q
qq
q
q
q
q
q
q
qqqqq
q
q
qq
q
q
qq
q
q qq
q
q
q
q
q
q
q q
q q
qq
q
q
q
qqqqq
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
qqq
qqqq
q
q
q
qqq
q
q q
qq
q
q
q
q
qqqqq
q
qqqq
q q
q
qq
q
q
qq
qq
q
q
qq
q q
q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
qq
q qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q qq
q
q
qq
q
q
q
q
qqq q
q
q
qq qq
q
q
q
q
q
qq
qqq
q
q
q
q
q
qqq q
q
q
qq
q
q
qq
qqqqqq
q
qq
q
q
q
q
qqqqqq
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q q qqq
q
q
q
q
q
qq qq
q
q
q
q
q
q
q
qq
q
q
q q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q q
q
q
q q
q
q qq
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qqq
q
q
qq
q
q
q
qqq
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
qq
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qqq
q
qq
q
q
qq
q
qq
q
q
qq
q
qqq q
q q
q
q
q
q
q
q
q
q qq q
q
qq
q
q
q
q
qq q
q
q q
qq
q
q
qq q
q
qq
q
q
q
qq
qqq q
q
q
q
q
qq
q
q
q
q
qqqq
q
qq
q
q
q qqqq
qq
q
q
q
qqq qqqq
q
q qqq
q
qqq
q
q
q
q q
qqq
q q
qq
q
q
qqq
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
qqqq
q
qqq
q
qqq
q
qqq q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q q
q
q
q
q
q
q
qq
q
qq
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
qq qq
q
q
q
qqq
q
q
qqq
q
q
q
q
q
q
q
q
qq
q
qqqq
q
qq qq
q
q
q
qq
qq
q
q
q
q q
q
q
qq
q
q
q
q
q
q
qq
q
q
q q
q
q
qq
q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q qq
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
qq
q
q
q
q q
qq
q
q
q
q
q
q
q
q
qq
qq q
q
qq
q
q
qq
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
qq
qqq
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
qq
qq
qq
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q qq q
q
q
q
q qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q q
q q
q
q q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q qq
qq
q
q qq
q
q
q
q
q qq q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
qq
q
q
q qq
q
q
q
q
qq
q
q
q
q
q q
q
q
q
q
q
q
q q
q
q
q
q
qq
q
q
q q
q
q
q
q
q
q
q
q
qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
q q
q
q
qq
q
qq
q
q
q
q
qq
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
qq qqq
q
qq
q q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q q
qq
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qq
qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qq
q qq
q q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
q
q
q
q
qqqq
q
q
q
qqqq
q
q
qq
qq q
q
q
q
q
q
q qqqq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q
qq
q
q
q
q
qq qqq qqqq qq
qq
q
qq
qqqqqq
q
q
qqq
q
q
q
q
q q
q
q
q
q
qqq
q
q
q
q
q
q
q q
q
q
qqq
q q
q
q
q
q
q
q
qq
q
q
qq
q
q
qq
q
qq
q
q
q
q
qq
qqq
q
q
q
q q
q q
q
qq
q
qq
q
q
q
qqq
q
q
q q
q
q
qq
q
q
q
q
q
q
q qq
qqq
q
q
qq
q
q
q
q
q
qq
q qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qqqq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
qq
q
q
qq
qq
q
q q
q
qq
q
q
qqqq
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q qq
q
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
qq
q
qq
qq q qq
q
qq
q
q
qqq
q
qq
qq
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
qqq
q
qq
q
q
qq
qq
q
q
qqqq
q
q
q
q
q
q
q q
q
q
qqq
q
q
q
q
q
qqqq qqqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
qq
q
q
q
q
qq
q
q q
q qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q qq qq
q
q
q
qq
q
q qq
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
qq
q
q
qq
q
q
qqq
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q q
q
q
q
q qq q q
q
qq q
q
qq
q
q
q
q
q
qq
qq
qqq
qqq
q
q
q
q
q
q qqqqqqqq
q
q
qqqqqqqqqqqqqqqqq
qqq
q
q
q
q
qqqqq
qqq
q
q
qqqqq
q
q
q
qqqq
q
q
q
q
q
q
q
qq
q
q
q
qqqqq
q
qqqqq
q
qq
qqq
qqqq
q
qq
q
qq
q
q
q
qq
q
qq
−2 0 2 4
−3−2−10123
PC1
PC2
Random walk series
Feature space of M3 data
Exploring the feature space of large time series collections M3 competition data 14
q q
q
q
q
q
qq qq
q q
q
q
q
q
q
q
q
q
q
q
q
qq q
qq
q
qqq
q
q
qq q
q
qq q
qq qq
q qqq
q
q
q
q
qq
qqqq
q
q
qq
q
q
qq
q
q
q
q
q
q
qq
q
qq
q q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
qq
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q q
q
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
qq
qqq
q q
q
q
q
qq
q
qqq
qqqq
q
qq
q
q
q
q
qq
q
qq
qq
qqqq
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
qq
q
q
q
q
q
qq
qq
q
q
q
q
q
qq
q
qqq
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
qqqq
q
qqq
q
qqq
qq
q
qq
qq
qqqqqqqqqq
q
q
q
q q
q
qq
q
qq
q q
q
q q
q
q
qq
q
q
qqq
qq
qqq
q q
qqqqqq q
q
q
q
qq
qq qqqq
q
q
qq
q
qq
q
q
qqq
q q
q
q
q
qqq
qq
q
q
qqqqq
qqq
qq
q
q
q
q
q
q
q q
q
q
q
qq
q
qqq
q
qqq
qqq
q
qq
q
q
q
q
q
q
qqqqq
q
q
qq
q
q
qq
q
q qq
q
q
q
q
q
q
q q
q q
qq
q
q
q
qqqqq
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
qqq
qqqq
q
q
q
qqq
q
q q
qq
q
q
q
q
qqqqq
q
qqqq
q q
q
qq
q
q
qq
qq
q
q
qq
q q
q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
qq
q qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q qq
q
q
qq
q
q
q
q
qqq q
q
q
qq qq
q
q
q
q
q
qq
qqq
q
q
q
q
q
qqq q
q
q
qq
q
q
qq
qqqqqq
q
qq
q
q
q
q
qqqqqq
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q q qqq
q
q
q
q
q
qq qq
q
q
q
q
q
q
q
qq
q
q
q q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q q
q
q
q q
q
q qq
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qqq
q
q
qq
q
q
q
qqq
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
qq
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qqq
q
qq
q
q
qq
q
qq
q
q
qq
q
qqq q
q q
q
q
q
q
q
q
q
q qq q
q
qq
q
q
q
q
qq q
q
q q
qq
q
q
qq q
q
qq
q
q
q
qq
qqq q
q
q
q
q
qq
q
q
q
q
qqqq
q
qq
q
q
q qqqq
qq
q
q
q
qqq qqqq
q
q qqq
q
qqq
q
q
q
q q
qqq
q q
qq
q
q
qqq
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
qqqq
q
qqq
q
qqq
q
qqq q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q q
q
q
q
q
q
q
qq
q
qq
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
qq qq
q
q
q
qqq
q
q
qqq
q
q
q
q
q
q
q
q
qq
q
qqqq
q
qq qq
q
q
q
qq
qq
q
q
q
q q
q
q
qq
q
q
q
q
q
q
qq
q
q
q q
q
q
qq
q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q qq
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
qq
q
q
q
q q
qq
q
q
q
q
q
q
q
q
qq
qq q
q
qq
q
q
qq
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
qq
qqq
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
qq
qq
qq
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q qq q
q
q
q
q qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q q
q q
q
q q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q qq
qq
q
q qq
q
q
q
q
q qq q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
qq
q
q
q qq
q
q
q
q
qq
q
q
q
q
q q
q
q
q
q
q
q
q q
q
q
q
q
qq
q
q
q q
q
q
q
q
q
q
q
q
qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
q q
q
q
qq
q
qq
q
q
q
q
qq
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
qq qqq
q
qq
q q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q q
qq
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qq
qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qq
q qq
q q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
q
q
q
q
qqqq
q
q
q
qqqq
q
q
qq
qq q
q
q
q
q
q
q qqqq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q
qq
q
q
q
q
qq qqq qqqq qq
qq
q
qq
qqqqqq
q
q
qqq
q
q
q
q
q q
q
q
q
q
qqq
q
q
q
q
q
q
q q
q
q
qqq
q q
q
q
q
q
q
q
qq
q
q
qq
q
q
qq
q
qq
q
q
q
q
qq
qqq
q
q
q
q q
q q
q
qq
q
qq
q
q
q
qqq
q
q
q q
q
q
qq
q
q
q
q
q
q
q qq
qqq
q
q
qq
q
q
q
q
q
qq
q qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qqqq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
qq
q
q
qq
qq
q
q q
q
qq
q
q
qqqq
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q qq
q
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
qq
q
qq
qq q qq
q
qq
q
q
qqq
q
qq
qq
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
qqq
q
qq
q
q
qq
qq
q
q
qqqq
q
q
q
q
q
q
q q
q
q
qqq
q
q
q
q
q
qqqq qqqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
qq
q
q
q
q
qq
q
q q
q qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q qq qq
q
q
q
qq
q
q qq
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
qq
q
q
qq
q
q
qqq
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q q
q
q
q
q qq q q
q
qq q
q
qq
q
q
q
q
q
qq
qq
qqq
qqq
q
q
q
q
q
q qqqqqqqq
q
q
qqqqqqqqqqqqqqqqq
qqq
q
q
q
q
qqqqq
qqq
q
q
qqqqq
q
q
q
qqqq
q
q
q
q
q
q
q
qq
q
q
q
qqqqq
q
qqqqq
q
qq
qqq
qqqq
q
qq
q
qq
q
q
q
qq
q
qq
−2 0 2 4
−3−2−10123
PC1
PC2
q
q
Feature space of M3 data
Exploring the feature space of large time series collections M3 competition data 14
Time
5 10 15
Outline
1 M3 competition data
2 Yahoo web traffic
3 What next?
Exploring the feature space of large time series collections Yahoo web traffic 15
Yahoo web-traffic
Tens of thousands of time series collected at
one-hour intervals over one month.
Consisting of several server metrics (e.g. CPU usage
and paging views) from many server farms globally.
Aim: find unusual (anomalous) time series.
Exploring the feature space of large time series collections Yahoo web traffic 16
Yahoo web-traffic
busy233busy271busy50busy200busy369
2014−11−09
2014−11−10
2014−11−11
2014−11−12
2014−11−13
2014−11−14
2014−11−15
2014−11−16
2014−11−17
2014−11−18
2014−11−19
2014−11−20
2014−11−21
2014−11−22
2014−11−23
2014−11−24
2014−11−25
2014−11−26
2014−11−27
2014−11−28
2014−11−29
2014−11−30
2014−12−01
2014−12−02
2014−12−03
2014−12−04
2014−12−05
2014−12−06
2014−12−07
2014−12−08
2014−12−09
2014−12−10
2014−12−11
2014−12−12
date
25
30
35
40
45
20
25
30
35
40
50
60
70
10
15
20
25
50
60
memory460memory429memory147memory413memory484
2014−11−09
2014−11−10
2014−11−11
2014−11−12
2014−11−13
2014−11−14
2014−11−15
2014−11−16
2014−11−17
2014−11−18
2014−11−19
2014−11−20
2014−11−21
2014−11−22
2014−11−23
2014−11−24
2014−11−25
2014−11−26
2014−11−27
2014−11−28
2014−11−29
2014−11−30
2014−12−01
2014−12−02
2014−12−03
2014−12−04
2014−12−05
2014−12−06
2014−12−07
2014−12−08
2014−12−09
2014−12−10
2014−12−11
2014−12−12
date
value
0
5000
10000
15000
20000
200
400
600
0
5000
10000
15000
20000
500
1000
0
5000
10000
15000
20000
25000
2014−11−09
2014−11−10
2014−11−11
2014−11−12
2014−11−13
2014−11−14
2014−11−15
2014−11−16
2014−11−17
2014−11−18
2014−11−19
2014−11−20
2014−11−21
2014−11−22
2014−11−23
2014−11−24
2014−11−25
2014−11−26
2014−11−27
2014−11−28
2014−11−29
2014−11−30
2014−12−01
2014−12−02
2014−12−03
2014−12−04
2014−12−05
2014−12−06
2014−12−07
2014−12−08
2014−12−09
2014−12−10
date
value
Exploring the feature space of large time series collections Yahoo web traffic 17
Feature space
ACF1: first order autocorrelation = Corr(Yt, Yt−1)
Strength of trend and seasonality based on STL
Trend linearity and curvature
Size of seasonal peak and trough
Spectral entropy
Lumpiness: variance of block variances (block size 24).
Spikiness: variances of leave-one-out variances of STL remainders.
Level shift: Maximum difference in trimmed means of consecutive
moving windows of size 24.
Variance change: Max difference in variances of consecutive
moving windows of size 24.
Flat spots: Discretize sample space into 10 equal-sized intervals.
Find max run length in any interval.
Number of crossing points of mean line.
Kullback-Leibler score: Maximum of
DKL(P Q) = P(x) ln P(x)/Q(x)dx where P and Q are estimated by
kernel density estimators applied to consecutive windows of size 48.
Change index: Time of maximum KL score
Exploring the feature space of large time series collections Yahoo web traffic 18
Principal component analysis
q
q
qq q
q
q q
q
q
q
qq
q
q
q
q
qq
q
qq
q
q
q
q
q
q
qq
q
q
q
q
qqq
q qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
qqq
q
q
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q q
q
qq
q
q
q
q q
q
q q
q qq
q
q
qqq
q
q
qq
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q q
q
q
q
q
q
q q
q
q qq q qq
q
q
q
q
q
q qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
qq
qqqqq q
q
qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq
qq
qq q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qqq
q
qqqqq q
q
q
q
q
q
q
qqq
q
q
q
q
q
qq
qq
q
qq
qqqq
qq
q
q
q
q q
q
q
q qq
q
qqq
qqqq
qqq
q
q
q
qq
q
q
q
q
q
q
q
qq
qq q
q
q
q
q
q
q
q
q
qq
q qqq
q
q q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q qq
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q qq
q
q q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q qq
q q
q q
q
q
q qq
q qq
q
q
q
q
q q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q q
q q
q
qq
q
q
q
q
q
q
q
q
q
qq
qq
q
q
q
q qq
q
q
qq
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qqqq
q q
q
q
q
q
q
q
qq
qq
q
qqq
q
q
q
qq
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
qq qq
q
q
q
q
qqq
q
q
q
q q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
q
qq
q q
qq
q
q
qq
q
q
q q
q
q
q
q
q
qq
q
qqq
q
q qq
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q q q
q
q
q
q
q
q
qq
q q
q
q
q
qqq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qqq
q
q
qq
q
q q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q
q
q
qq
q
q q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q q q
q
q
q
q
q
q
q
qq
q
qq
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
q q
qq q
q q
qq
q
q
q
qqq
q
q
q
q
q
q q
q q
qqq
qq
q
qq
q
q
qq
q
qq qqqq
qqq
q
q
qqq
q
qq q
q qq
q q
q
q
q
q
q
q
q q
q
q
q q qq
q
q q
q qq qq
q
q
q
q
qq qq
qq
q q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqqq
q
q
q
q q
qq
q
q
qq qq
qq
q
q
qq qq
q
qq q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq qqq q
qq
qq
q
q
q
q
qq
qq
q
q
qq
q
qq
q
q
q
q q
q q q
q
qq
q q
q
q
q
q
q
q
q
qq q
q
q
q
q q
qq
q
qq qq
qq q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q q
q
qq q
qq
q
q qq q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q q
q
q q q
qq
q
qq
q
q qqqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq qqq
q
q
q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q
qq q
qq
q
qq
q
q
q
q
q qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qqq
q
qq
q
q
q
q
q q
q
qqq
q
q
q
q q
q
q
q
q qq
q
qq
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
qq qq
q
q
qq
q
q q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q q
q
qq
q
q
q
q qqqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
qq
q
qq
q q
q
q
q q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
qq q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
qq
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q
qq
q qq
q
q
qq
q
q
q
q
q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q qq
q
q
q
q
q
q
q
q
qq
qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q q
q
q
q
q
q
q
q qq q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
qq
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
qqq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
qq
q
q
q
q
q
q q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
qq
q
q
q q
qq
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q qqq
q
qq
q qq
qq qq
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q
q
q
qq
q
q
q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qq
q
q
q
q
qq q
q
qq qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
qq
q
q
q
q
qq
q
q
q
q
q
q
q
q
q q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q q
q
qq
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
qq
q
q
q
q
q
q
q
q
qq
qq
q q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qq q
q
qq
q
q
q
q q
q
q
q
q
q
q q
qq
q
qq
q
q
q
q
q
q
q
q
qq
q q
q
q
q
q
q
q
q
q q
q
q
q
q qq q
q
q
q
q q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq qq
q
q
q
q
q
q q
q
q
q
q q qq
q
q
q
q q
q q
q
q
q
q qq
q
q
q
q
q
qq
q
q
qqq
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q q
q
qq
q
q
qq q
q
q
q q
q
q
q
q
q
q qqq
q
q
q
q
q
q
q
qq
q
q
q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
qq
q
q
q
q q
q
q q
q
q q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qqq
q
q
q qq q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q qq
q
q
q
qqqq
qq
q
q
q
qq
q
q
q
q
q
qq
qq
q
q
q
q qq
q
q
q
q
q
q
qq
q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
qq
qq
qqq
q
q q
qq
q
q
q
q
q
q
q
q q
q
q
qq
q
q
q
q q
qq
q
q
q
qqq
q
q
q
q
q
q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q qq
q
q
q
qq
q q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
qqq
q
qq
q
q
q
q
q
q q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q q
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq
qq
q
q
q q
qq
q
q
q
q q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q q
q
q
q
q
qq
q
qq
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q q
q
q
q q
q
q
q
q q
q
q
q
q qqqq
qq
q q
qq
q
q
qq
q
q q
qq
q
q
qq
q
q
qq
q q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q
q q
q
q qq
q
q
q
q
q
qq
q
q
q
q
qq
q
qq qq qq q
qq
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q qq
qq
q
qqq
q
q qqqqq
q
q
q
q
qq
q
q
qq
qq
q
q
q
q
q
q
qqq q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
qq
q
q
q q q
qq
q
q
q
q
q
q
qq
qqqq qq
q
q
q
q
qq
q
q q
q
q
q
q
q
q
q
q
q
qq
qq qqqqqq
q q
q
qq
q q
q
q
qq
q
qq
qqq
q
qqqq
q q
q
q
q
q
q
q
q
q
q
q qq
q
q
q
q
q
q
q
qq
q
qq
q
qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q qq
q
q qq
q
q
q
q
q
q
qq qq
q
q
q
qq
q
q
q
q
qq
q
q
q
q q
q q
q
q
q
q
qq
qq
q q
q
q q
q
q qqq
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q q
q
qq
q
q
q
q
qq q
q
qq
q
q
q
q
q q
qq
q
q
q q
q
q
q
q
q
q
q
qq
q
q
q
qq
qqq
qq
q
q
qq
q
q
qqq
q
q
qq
q
qq q
q
q qqq
q
q q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q q
qqq q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q qqq
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
qq q
q
qq
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q q
q
q
q
q
q q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qqq
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q q
q
q
qqq
q
q
q
q
q
q
q
q
qq
qq
q q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
qq
qq
q
q
q
q
q
q
q
q
qq qq q
q
q
q q q
q
q
qq
q
q
qq q
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
q
q
q q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q q
q
qq q
qq
q
q
q
qq
q
q
q
qq q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
q
q
qq
q
q q
q q
q
qq
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
qqqqq
qqq
qqqqq qq
q
q qqq
q
q qq
q
qq
q
q
qq
qq qq
q
q
q q
q q
q
q
qq
q
q
q qqq
q
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q qq
q
q
q q
q
qq
q
q
q
q
q
q
q
q
q
q qq
q
q
qq
q
q
q
qq
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
q
qq q
q
qq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q qq
q
q
qq
q
q
q
q
q
qq
q q
q
q
q qq
q
q
q
qq
qq
q
q
q
q
q
q
q
q q
q
q
q
q
qq
q
q
q
q
q
q
qqq
qq
q
q
q
q
qq
q
qqq
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q q
q
qq
qq
q
q q
q
q
q
q
q
qq
q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
qq
q
q qq
q
qq
q
q
q
q
qq q
q
q
q q
q
q
qq
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq qqq qq
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
qq q
q
qqq
q
qqq
q
q
qq
q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
qq
q
qq
q
q qq
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq q
qq
q
q
q
qq
q
q
q
q
q
qqq
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q qq
q
q
q
qq
q
q
q
q qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q q
q
q qq
q
q
q
q qq q
q
qq q
q
q
q
q
q qq
q
q
q qq
q
q
q
q qqq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
qq
q
q
q
q
qq
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq q
qq qq q
qq
qq
q q
q
q
qq
q
q
q
q q
q
qq
q
q
q
q
qq
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q q
q
q
q
q
qq
q
q
q
q
q q
q q
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
qq
qq
q
q
q
qqqq
qq
q
q q
qqqq
q
q
q
q
q
q
qqq
q
q
q
q
q
qqq qqq
q
q
q
q
q
q q
q
q
q
q
q
qq
q
q
qq
qq
q
qq
q q
q
q
qq
q
q q
q
qq
q
qq
q
q
q
q
q
q q q
q q
q
q
q
q
q
q
q
q
q
q
q
qq
qq
qq
qq
q
q
q
q
qq
qq
q
q
q
q
q
q
qq
q
q
q q
q
q
q
qq
q
q
qq
q q
qqq
q
q
q
qq
q
q
qq
qq
q
q
q q
q
q
q
q
q
q
qq
q
q
qq
qqq
q
q
q qq qq
q
q qq
q q
q
q
q
q q
q
q
q q
q
q q
qq
q
q qqq
q
q
q
q
q q
q
q
q q
q q
q
qq
q q
q
q
qq
qq
q
q
q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
qqqqq
q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq qq
q
q
q q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
qqq
q
qqq
q
qq
q
q
q
q
q q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
qq
q
q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q
qqq
q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q qq
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
qq
q
q q
q
q
q
q
qqq
q
q q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
q
q
q q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
qq
qq
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
qq
qq
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q q
q
q
q
q
q
qq
q
q
q
q q
q
q
q
qq
q
q
q
q
q
q
qq
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q q q
q
q
q
q
q
q
q
q
q
qq
qq
q
q
q q
q
q q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qq
q
q
q
qq
qq
qq
q qq qq
q
qqq
q
qq
q
qq q
q
q
qq
q
q
qq
q
qq
q
q
q q q
q qqq
qqq q
qqq
q
qq
q
q
q
q
q
qq qq qqqq q
qq
q qq
q
q q
q
q
q
qq
qq
q
q
q
q
qq
q
q q
q
q
q
q qqqq
qqqqqqq
q
q
q
q
qq
q
q
qq
q
qq
q
qq
qq
qq
q
qqq
q
q
q
q q
q
q
q
q
q q
qqqqqq
qq
q
q
q
qq q
q
q
q q
qqq
q
qq
q
q
q
qq
q
qq
q
q
qqq qq q
qq
q
q
q
q
q
q
q
q
q qq
q
qq
q
q q
q
q q
q
q
qqq
q
q
q
q
q q
q
q
qq
qqq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
qq
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
qqq
q
qq
q q
qq
q
q
q
q
q
q
q
q
q
q
qq
qq
q
qq q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q qq
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q q
q
q
q
q
q
q
q
qq
q
q q
q
q q
q
q q
q
qq
q
q
q
qq
q
q
q
q qq
qq
qq
q
q
q
q
q
q qq
q q
q
q
q
q
qq
q q
q
q
q
q
q
q
qqq
q
qq
qqq
qq q q
q
qq
q
q
q
q
q
q qqq
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
q q
q
q
q q
q
q
q q q
q
q
q
q
q
q
q
q
q
qq
q
q
q q
q q
q
q
q qq q
qq
q
qq
q
qq qqq
q
q
qq
qqq
q
q
q
qq
q
q q
q q qq
qq
q
qqq
qqqq
qqq q q
qqq qq qq q
q
q
q
qq q
q
q
q
q
q
q
q q
q
q
qq
q q
qq
q
q
q
q
qq qq
q
q
qq
q
q
q
q
q
q
q
qq
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq q
q
q
q
q
q
q
qq
q
q
q qq
q
q q
q qqq
q
q
q
q
qq q
q
q
qq
q
qq
q
q
q q
q
q
q
q
q
q
q
q
q qq
q
q
q q
q
q
q
q
q
q
q
qq
q
q
q
qq
qq
qq
q
qqq
q
q q
q qqqq
qqqqq q
q
qq
q
q
qq qq
q qq
q
q
q q
q q
q
qq q
q
qq
q
q
q
qqq
q
q
q
q q
qq
q
q
q
q
q q
q
q
q
q
q
q
q q
q
q q
q
q
qq q
q
q
q
q
q q
q
q q
q
q
q
q
q
q q
q
q
q
qq
q
q
q
qq
q
q
qq
q
q
q
q
qq
q
q
qqq
q
qq
q
q
q
q qq
q
q
qq
q
q
q
q
q
q
qq
qq
q
q q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q q
qq
q
q
q
q
qq qq
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
qq
qq
q
q
q q
q
q
q
q
q
qq
q
q qq
q
qq
q
q
qqqq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
q
qq
q qq
q
qq
q
qq
qqqq
qq
qqqqq
q
q
qq
q
q
q
q qq
qq
q
q
qq
q qq
q
q
qq q
q
q
q
q
qqq q
q
q
q q
q
q
q
qq
q
q
qqqq
q
q
qq
q
qq
qq
q
q
q
q
q
q
qq
q
q
q
q
q
qq
qq q
q
qq
q
qqq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q q
q q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qq
q
q
q q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q q
q
q
qq
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
qq
q
qq
qq
qqq
q
q
q
q q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
qq
qq
q
q
q
q
q
q
q q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qq
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q qqq
q
q
qqq
q
q
q
q
q
qq
q
q
q
qq
qqq
q
q
q qq qq
q
q
q
q
q
q
q
qq qq
q
q
q
q
q
qq
q
q
q
q
qqqq
qqq
qq
q
qq q
q q
q
q
qq
q
qq
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
qq
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqqq
q
q qq qqqqq
q
q
q
qq
q
q
q
q
q
qq qq
qqq
q
q
q qq q
q
q
q
q qq
qqq qqq
q
qqqqq qqqqqq qq
q
q
q
q
qq
q
q
q
qq
qq
q
qqqq
q q
qqq
q
qqq
q q
qqqq
qq
q
q
q
q
q
q q
q
qq
q
q
q
q
qq qq
qqq
qq qqqqq
q
q
qq q
q
q
qq qqq q
q
q
q q
q
qq qq q
q
qq
qqq
q
q qq q
q q
q
qq
q
q
q
q q
q
q
q
q
q qq
qq
q
q
qq qq
q q
q
qqq
qq
q
qqq
qq qqqq
q
q
q
q
q
q
qq
q
qq
qq qqq q
q
qq
q qqq qq
qqqqqq
q
q
qq q
qqqqq qqq
q qqq
q
qq
qq
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q q
q
q
q
qq
q
q
q
qq
qqq
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
qqqq
qq
q
qq qq
q
q
q
q
q
q
q
q
q
q
q qq
qqq q
q
qq
qq
q q
q
q q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
q
qq
qq q
q q
q
q
q
qq
q
q
q
q
q
q
qqqq
q
q
q
q
q qq
q
q
q
q
q
q
q
q
q
q
q qq qq
q
q
q q
q
q
q
q
q
q q
q q
q q
q
q
q
q
q
q
q
qq qq
q
q
q
q
qqq
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
qqqq
q
q
q
q q
qqqqq
q
q q
q q
q
q
q
q
q
q
q
q
qq
q
q
q
qqq
q
q
qqqq
qqq
q
q
q
q
q
qq
q
q
q q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qqq
q
q
q
q
q
qq
q q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
qq
q q
q
q
q
q
q
q
q
qq
q
q
qq
q
q q
q
q
q
q
q
q
qq
q
q q
q
q
q
q q
q
q
q
qq q
q
qq
q
q
qq
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
qq
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q q
q
q
qqq
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qq
qq
q
q q
q
q
q q
q
q
q
q q
q
q
q
qqq
qq
q
qq q
q
qq
q
q
q
q
q
q q
q
q
qq
q
qq
q q
q q
q
qq
qq
q
qq q
q
q
q
q
qq
q
q
q
q
q
q
q q
q
q
qq
q
q
q
qq
qq
q
q qqq
q q
q
qq qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
qqqq qq
q
q
qq
q
qqq
q
q
q
q
qq qq
q
q
q
q
q q
q
qq
q
q
q
q
q
qq
qqq
qq qq
q
q
q
qq
qq
q
q
q
qq q
q q
qqq
q
q
qq
q
q
q
q
q
q
q
qqqqq
q
q
q
q qq
q
q
q
qq
q
q
q
q q
q
qqqq q
q
qq
q
qq
q
q
q
q
q
qq
q
qq
qq
qq
q
q
q
q
qq
qq
q q
qq qq
q
q
q
q
q
q
q
q
qq qqq
q
q
q
qq
qqq
qq
q
q q q
qq
q
q
q q
qq
q
q
q
q
q
qq
q
qq
q
q
qq
q
q
q
q
q
qq
q
q
qq
q q
q q
q
q
q q
q
q q
q
q
qq
q q
q
q
q
qqq
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qq
q
q
q
q
q
qqq
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqqq
q
q
qq qq
q
q
qq
q
qq q
q
q
q
qq
q
q
q
qq
q
q
qq
q q
q
q
q
qq
q
q
q
q
qq
q
q q
qq
q
q
q q qq
q
q
q
qqq
q
q
q
q
qq
qqq qqq
q
q
qq q
q
q
qq
q
qq
qqq
q
q
q
q q
qq
q
q
qq
qq
q
q qq
q
q
q
q
qqqqq
qq
q qqqq
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
qq
q
qqq
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q q
q
q
qq
q
q
q
q q
q q
q
qq
qq qq
qq
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
qq
q
qq q
q
q
qqq
qqqq
q
q
q
q
q
q
q
q
qqqq
qqq
q
qq
q
q
qqq
q
q
q
q
q
q
qqq
q
q
q
q
qqq
q
q
q
q q
qq
q
q
qq
q
q
qq
q
q
qq
q
q
q
q
q
qqq
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q qq
q
qq
q q
q
q
q
q
q
q
q
q
q
qq
q
q q
q
q
qq q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
qq
q
qq
q
q qq
q
q
q
q q
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
qq
q
q
q
q
qqq q
q
q
q
qq
qq
q
q
q
q
q
q
qqqq q
q
q
q
qq
qqq q
q
q
qq
q
q
q
qqq
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
ACF1
lumpiness
entropy
lshiftvchange
cpoints
fspots
trend
linearity
curvature
spikiness
season
peaktrough
klscore
change.idx
−4
−2
0
2
−2.5 0.0 2.5
standardized PC1 (28.7% explained var.)
standardizedPC2(17.3%explainedvar.)
Exploring the feature space of large time series collections Yahoo web traffic 19
What is “anomalous”
q
q
qq q
q
q q
q
q
q
qq
q
q
q
q
qq
q
qq
q
q
q
q
q
q
qq
q
q
q
q
qqq
q qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
qqq
q
q
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q q
q
qq
q
q
q
q q
q
q q
q qq
q
q
qqq
q
q
qq
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q q
q
q
q
q
q
q q
q
q qq q qq
q
q
q
q
q
q qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
qq
qqqqq q
q
qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq
qq
qq q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qqq
q
qqqqq q
q
q
q
q
q
q
qqq
q
q
q
q
q
qq
qq
q
qq
qqqq
qq
q
q
q
q q
q
q
q qq
q
qqq
qqqq
qqq
q
q
q
qq
q
q
q
q
q
q
q
qq
qq q
q
q
q
q
q
q
q
q
qq
q qqq
q
q q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q qq
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q qq
q
q q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q qq
q q
q q
q
q
q qq
q qq
q
q
q
q
q q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q q
q q
q
qq
q
q
q
q
q
q
q
q
q
qq
qq
q
q
q
q qq
q
q
qq
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qqqq
q q
q
q
q
q
q
q
qq
qq
q
qqq
q
q
q
qq
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
qq qq
q
q
q
q
qqq
q
q
q
q q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
q
qq
q q
qq
q
q
qq
q
q
q q
q
q
q
q
q
qq
q
qqq
q
q qq
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q q q
q
q
q
q
q
q
qq
q q
q
q
q
qqq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qqq
q
q
qq
q
q q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q
q
q
qq
q
q q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q q q
q
q
q
q
q
q
q
qq
q
qq
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
q q
qq q
q q
qq
q
q
q
qqq
q
q
q
q
q
q q
q q
qqq
qq
q
qq
q
q
qq
q
qq qqqq
qqq
q
q
qqq
q
qq q
q qq
q q
q
q
q
q
q
q
q q
q
q
q q qq
q
q q
q qq qq
q
q
q
q
qq qq
qq
q q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqqq
q
q
q
q q
qq
q
q
qq qq
qq
q
q
qq qq
q
qq q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq qqq q
qq
qq
q
q
q
q
qq
qq
q
q
qq
q
qq
q
q
q
q q
q q q
q
qq
q q
q
q
q
q
q
q
q
qq q
q
q
q
q q
qq
q
qq qq
qq q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q q
q
qq q
qq
q
q qq q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q q
q
q q q
qq
q
qq
q
q qqqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq qqq
q
q
q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q
qq q
qq
q
qq
q
q
q
q
q qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qqq
q
qq
q
q
q
q
q q
q
qqq
q
q
q
q q
q
q
q
q qq
q
qq
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
qq qq
q
q
qq
q
q q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q q
q
qq
q
q
q
q qqqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
qq
q
qq
q q
q
q
q q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
qq q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
qq
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q
qq
q qq
q
q
qq
q
q
q
q
q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q qq
q
q
q
q
q
q
q
q
qq
qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q q
q
q
q
q
q
q
q qq q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
qq
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
qqq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
qq
q
q
q
q
q
q q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
qq
q
q
q q
qq
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q qqq
q
qq
q qq
qq qq
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q
q
q
qq
q
q
q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qq
q
q
q
q
qq q
q
qq qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
qq
q
q
q
q
qq
q
q
q
q
q
q
q
q
q q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q q
q
qq
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
qq
q
q
q
q
q
q
q
q
qq
qq
q q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qq q
q
qq
q
q
q
q q
q
q
q
q
q
q q
qq
q
qq
q
q
q
q
q
q
q
q
qq
q q
q
q
q
q
q
q
q
q q
q
q
q
q qq q
q
q
q
q q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq qq
q
q
q
q
q
q q
q
q
q
q q qq
q
q
q
q q
q q
q
q
q
q qq
q
q
q
q
q
qq
q
q
qqq
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q q
q
qq
q
q
qq q
q
q
q q
q
q
q
q
q
q qqq
q
q
q
q
q
q
q
qq
q
q
q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
qq
q
q
q
q q
q
q q
q
q q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qqq
q
q
q qq q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q qq
q
q
q
qqqq
qq
q
q
q
qq
q
q
q
q
q
qq
qq
q
q
q
q qq
q
q
q
q
q
q
qq
q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
qq
qq
qqq
q
q q
qq
q
q
q
q
q
q
q
q q
q
q
qq
q
q
q
q q
qq
q
q
q
qqq
q
q
q
q
q
q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q qq
q
q
q
qq
q q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
qqq
q
qq
q
q
q
q
q
q q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q q
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq
qq
q
q
q q
qq
q
q
q
q q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q q
q
q
q
q
qq
q
qq
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q q
q
q
q q
q
q
q
q q
q
q
q
q qqqq
qq
q q
qq
q
q
qq
q
q q
qq
q
q
qq
q
q
qq
q q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q
q q
q
q qq
q
q
q
q
q
qq
q
q
q
q
qq
q
qq qq qq q
qq
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q qq
qq
q
qqq
q
q qqqqq
q
q
q
q
qq
q
q
qq
qq
q
q
q
q
q
q
qqq q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
qq
q
q
q q q
qq
q
q
q
q
q
q
qq
qqqq qq
q
q
q
q
qq
q
q q
q
q
q
q
q
q
q
q
q
qq
qq qqqqqq
q q
q
qq
q q
q
q
qq
q
qq
qqq
q
qqqq
q q
q
q
q
q
q
q
q
q
q
q qq
q
q
q
q
q
q
q
qq
q
qq
q
qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q qq
q
q qq
q
q
q
q
q
q
qq qq
q
q
q
qq
q
q
q
q
qq
q
q
q
q q
q q
q
q
q
q
qq
qq
q q
q
q q
q
q qqq
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q q
q
qq
q
q
q
q
qq q
q
qq
q
q
q
q
q q
qq
q
q
q q
q
q
q
q
q
q
q
qq
q
q
q
qq
qqq
qq
q
q
qq
q
q
qqq
q
q
qq
q
qq q
q
q qqq
q
q q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q q
qqq q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q qqq
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
qq q
q
qq
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q q
q
q
q
q
q q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qqq
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q q
q
q
qqq
q
q
q
q
q
q
q
q
qq
qq
q q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
qq
qq
q
q
q
q
q
q
q
q
qq qq q
q
q
q q q
q
q
qq
q
q
qq q
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
q
q
q q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q q
q
qq q
qq
q
q
q
qq
q
q
q
qq q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
q
q
qq
q
q q
q q
q
qq
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
qqqqq
qqq
qqqqq qq
q
q qqq
q
q qq
q
qq
q
q
qq
qq qq
q
q
q q
q q
q
q
qq
q
q
q qqq
q
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q qq
q
q
q q
q
qq
q
q
q
q
q
q
q
q
q
q qq
q
q
qq
q
q
q
qq
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
q
qq q
q
qq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q qq
q
q
qq
q
q
q
q
q
qq
q q
q
q
q qq
q
q
q
qq
qq
q
q
q
q
q
q
q
q q
q
q
q
q
qq
q
q
q
q
q
q
qqq
qq
q
q
q
q
qq
q
qqq
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q q
q
qq
qq
q
q q
q
q
q
q
q
qq
q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
qq
q
q qq
q
qq
q
q
q
q
qq q
q
q
q q
q
q
qq
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq qqq qq
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
qq q
q
qqq
q
qqq
q
q
qq
q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
qq
q
qq
q
q qq
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq q
qq
q
q
q
qq
q
q
q
q
q
qqq
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q qq
q
q
q
qq
q
q
q
q qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q q
q
q qq
q
q
q
q qq q
q
qq q
q
q
q
q
q qq
q
q
q qq
q
q
q
q qqq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
qq
q
q
q
q
qq
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq q
qq qq q
qq
qq
q q
q
q
qq
q
q
q
q q
q
qq
q
q
q
q
qq
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q q
q
q
q
q
qq
q
q
q
q
q q
q q
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
qq
qq
q
q
q
qqqq
qq
q
q q
qqqq
q
q
q
q
q
q
qqq
q
q
q
q
q
qqq qqq
q
q
q
q
q
q q
q
q
q
q
q
qq
q
q
qq
qq
q
qq
q q
q
q
qq
q
q q
q
qq
q
qq
q
q
q
q
q
q q q
q q
q
q
q
q
q
q
q
q
q
q
q
qq
qq
qq
qq
q
q
q
q
qq
qq
q
q
q
q
q
q
qq
q
q
q q
q
q
q
qq
q
q
qq
q q
qqq
q
q
q
qq
q
q
qq
qq
q
q
q q
q
q
q
q
q
q
qq
q
q
qq
qqq
q
q
q qq qq
q
q qq
q q
q
q
q
q q
q
q
q q
q
q q
qq
q
q qqq
q
q
q
q
q q
q
q
q q
q q
q
qq
q q
q
q
qq
qq
q
q
q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
qqqqq
q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq qq
q
q
q q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
qqq
q
qqq
q
qq
q
q
q
q
q q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
qq
q
q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qqq
q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q qq
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
qq
q
q q
q
q
q
q
qqq
q
q q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
q
q
q q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
qq
qq
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
qq
qq
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q q
q
q
q
q
q
qq
q
q
q
q q
q
q
q
qq
q
q
q
q
q
q
qq
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q q q
q
q
q
q
q
q
q
q
q
qq
qq
q
q
q q
q
q q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qq
q
q
q
qq
qq
qq
q qq qq
q
qqq
q
qq
q
qq q
q
q
qq
q
q
qq
q
qq
q
q
q q q
q qqq
qqq q
qqq
q
qq
q
q
q
q
q
qq qq qqqq q
qq
q qq
q
q q
q
q
q
qq
qq
q
q
q
q
qq
q
q q
q
q
q
q qqqq
qqqqqqq
q
q
q
q
qq
q
q
qq
q
qq
q
qq
qq
qq
q
qqq
q
q
q
q q
q
q
q
q
q q
qqqqqq
qq
q
q
q
qq q
q
q
q q
qqq
q
qq
q
q
q
qq
q
qq
q
q
qqq qq q
qq
q
q
q
q
q
q
q
q
q qq
q
qq
q
q q
q
q q
q
q
qqq
q
q
q
q
q q
q
q
qq
qqq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
qq
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
qqq
q
qq
q q
qq
q
q
q
q
q
q
q
q
q
q
qq
qq
q
qq q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q qq
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q q
q
q
q
q
q
q
q
qq
q
q q
q
q q
q
q q
q
qq
q
q
q
qq
q
q
q
q qq
qq
qq
q
q
q
q
q
q qq
q q
q
q
q
q
qq
q q
q
q
q
q
q
q
qqq
q
qq
qqq
qq q q
q
qq
q
q
q
q
q
q qqq
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
q q
q
q
q q
q
q
q q q
q
q
q
q
q
q
q
q
q
qq
q
q
q q
q q
q
q
q qq q
qq
q
qq
q
qq qqq
q
q
qq
qqq
q
q
q
qq
q
q q
q q qq
qq
q
qqq
qqqq
qqq q q
qqq qq qq q
q
q
q
qq q
q
q
q
q
q
q
q q
q
q
qq
q q
qq
q
q
q
q
qq qq
q
q
qq
q
q
q
q
q
q
q
qq
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq q
q
q
q
q
q
q
qq
q
q
q qq
q
q q
q qq
q
q
q
q
q
qq q
q
q
qq
q
qq
q
q
q q
q
q
q
q
q
q
q
q
q qq
q
q
q q
q
q
q
q
q
q
q
qq
q
q
q
qq
qq
qq
q
qqq
q
q q
q qqqq
qqqqq q
q
qq
q
q
qq qq
q qq
q
q
q q
q q
q
qq q
q
qq
q
q
q
qqq
q
q
q
q q
qq
q
q
q
q
q q
q
q
q
q
q
q
q q
q
q q
q
q
qq q
q
q
q
q
q q
q
q q
q
q
q
q
q
q q
q
q
q
qq
q
q
q
qq
q
q
qq
q
q
q
q
qq
q
q
qqq
q
qq
q
q
q
q qq
q
q
qq
q
q
q
q
q
q
qq
qq
q
q q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q q
qq
q
q
q
q
qq qq
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
qq
qq
q
q
q q
q
q
q
q
q
qq
q
q qq
q
qq
q
q
qqqq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
q
qq
q qq
q
qq
q
qq
qqqq
qq
qqqqq
q
q
qq
q
q
q
q qq
qq
q
q
qq
q qq
q
q
qq q
q
q
q
q
qqq q
q
q
q q
q
q
q
qq
q
q
qqqq
q
q
qq
q
qq
qq
q
q
q
q
q
q
qq
q
q
q
q
q
qq
qq q
q
qq
q
qqq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q q
q q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qq
q
q
q q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q q
q
q
qq
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
qq
q
qq
qq
qqq
q
q
q
q q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
qq
qq
q
q
q
q
q
q
q q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qq
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q qqq
q
q
qqq
q
q
q
q
q
qq
q
q
q
qq
qqq
q
q
q qq qq
q
q
q
q
q
q
q
qq qq
q
q
q
q
q
qq
q
q
q
q
qqqq
qqq
qq
q
qq q
q q
q
q
qq
q
qq
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
qq
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqqq
q
q qq qqqqq
q
q
q
qq
q
q
q
q
q
qq qq
qqq
q
q
q qq q
q
q
q
q qq
qqq qqq
q
qqqqq qqqqqq qq
q
q
q
q
qq
q
q
q
qq
qq
q
qqqq
q q
qqq
q
qqq
q q
qqqq
qq
q
q
q
q
q
q q
q
qq
q
q
q
q
qq qq
qqq
qq qqqqq
q
q
qq q
q
q
qq qqq q
q
q
q q
q
qq qq q
q
qq
qqq
q
q qq q
q q
q
qq
q
q
q
q q
q
q
q
q
q qq
qq
q
q
qq qq
q q
q
qqq
qq
q
qqq
qq qqqq
q
q
q
q
q
q
qq
q
qq
qq qqq q
q
qq
q qqq qq
qqqqqq
q
q
qq q
qqqqq qqq
q qqq
q
qq
qq
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q q
q
q
q
qq
q
q
q
qq
qqq
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
qqqq
qq
q
qq qq
q
q
q
q
q
q
q
q
q
q
q qq
qqq q
q
qq
qq
q q
q
q q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
q
qq
qq q
q q
q
q
q
qq
q
q
q
q
q
q
qqqq
q
q
q
q
q qq
q
q
q
q
q
q
q
q
q
q
q qq qq
q
q
q q
q
q
q
q
q
q q
q q
q q
q
q
q
q
q
q
q
qq qq
q
q
q
q
qqq
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
qqqq
q
q
q
q q
qqqqq
q
q q
q q
q
q
q
q
q
q
q
q
qq
q
q
q
qqq
q
q
qqqq
qqq
q
q
q
q
q
qq
q
q
q q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qqq
q
q
q
q
q
qq
q q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
qq
q q
q
q
q
q
q
q
q
qq
q
q
qq
q
q q
q
q
q
q
q
q
qq
q
q q
q
q
q
q q
q
q
q
qq q
q
qq
q
q
qq
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
qq
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q q
q
q
qqq
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qq
qq
q
q q
q
q
q q
q
q
q
q q
q
q
q
qqq
qq
q
qq q
q
qq
q
q
q
q
q
q q
q
q
qq
q
qq
q q
q q
q
qq
qq
q
qq q
q
q
q
q
qq
q
q
q
q
q
q
q q
q
q
qq
q
q
q
qq
qq
q
q qqq
q q
q
qq qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
qqqq qq
q
q
qq
q
qqq
q
q
q
q
qq qq
q
q
q
q
q q
q
qq
q
q
q
q
q
qq
qqq
qq qq
q
q
q
qq
qq
q
q
q
qq q
q q
qqq
q
q
qq
q
q
q
q
q
q
q
qqqqq
q
q
q
q qq
q
q
q
qq
q
q
q
q q
q
qqqq q
q
qq
q
qq
q
q
q
q
q
qq
q
qq
qq
qq
q
q
q
q
qq
qq
q q
qq qq
q
q
q
q
q
q
q
q
qq qqq
q
q
q
qq
qqq
qq
q
q q q
qq
q
q
q q
qq
q
q
q
q
q
qq
q
qq
q
q
qq
q
q
q
q
q
qq
q
q
qq
q q
q q
q
q
q q
q
q q
q
q
qq
q q
q
q
q
qqq
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qq
q
q
q
q
q
qqq
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqqq
q
q
qq qq
q
q
qq
q
qq q
q
q
q
qq
q
q
q
qq
q
q
qq
q q
q
q
q
qq
q
q
q
q
qq
q
q q
qq
q
q
q q qq
q
q
q
qqq
q
q
q
q
qq
qqq qqq
q
q
qq q
q
q
qq
q
qq
qqq
q
q
q
q q
qq
q
q
qq
qq
q
q qq
q
q
q
q
qqqqq
qq
q qqqq
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
qq
q
qqq
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q q
q
q
qq
q
q
q
q q
q q
q
qq
qq qq
qq
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
qq
q
qq q
q
q
qqq
qqqq
q
q
q
q
q
q
q
q
qqqq
qqq
q
qq
q
q
qqq
q
q
q
q
q
q
qqq
q
q
q
q
qqq
q
q
q
q q
qq
q
q
qq
q
q
qq
q
q
qq
q
q
q
q
q
qqq
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q qq
q
qq
q q
q
q
q
q
q
q
q
q
q
qq
q
q q
q
q
qq q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
qq
q
qq
q
q qq
q
q
q
q q
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
qq
q
q
q
q
qqq q
q
q
q
qq
qq
q
q
q
q
q
q
qqqq q
q
q
q
qq
qqq q
q
q
qq
q
q
q
qqq
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
ACF1
lumpiness
entropy
lshiftvchange
cpoints
fspots
trend
linearity
curvature
spikiness
season
peaktrough
klscore
change.idx
−4
−2
0
2
−2.5 0.0 2.5
standardized PC1 (28.7% explained var.)
standardizedPC2(17.3%explainedvar.)
We need a measure of the “anomalousness” of a time
series.
1 Rank points based on their local density.
2 Rank points based on whether they are within
α-convex hulls of different radius.
Exploring the feature space of large time series collections Yahoo web traffic 20
What is “anomalous”
q
q
qq q
q
q q
q
q
q
qq
q
q
q
q
qq
q
qq
q
q
q
q
q
q
qq
q
q
q
q
qqq
q qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
qqq
q
q
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q q
q
qq
q
q
q
q q
q
q q
q qq
q
q
qqq
q
q
qq
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q q
q
q
q
q
q
q q
q
q qq q qq
q
q
q
q
q
q qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
qq
qqqqq q
q
qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq
qq
qq q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qqq
q
qqqqq q
q
q
q
q
q
q
qqq
q
q
q
q
q
qq
qq
q
qq
qqqq
qq
q
q
q
q q
q
q
q qq
q
qqq
qqqq
qqq
q
q
q
qq
q
q
q
q
q
q
q
qq
qq q
q
q
q
q
q
q
q
q
qq
q qqq
q
q q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q qq
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q qq
q
q q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q qq
q q
q q
q
q
q qq
q qq
q
q
q
q
q q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q q
q q
q
qq
q
q
q
q
q
q
q
q
q
qq
qq
q
q
q
q qq
q
q
qq
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qqqq
q q
q
q
q
q
q
q
qq
qq
q
qqq
q
q
q
qq
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
qq qq
q
q
q
q
qqq
q
q
q
q q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
q
qq
q q
qq
q
q
qq
q
q
q q
q
q
q
q
q
qq
q
qqq
q
q qq
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q q q
q
q
q
q
q
q
qq
q q
q
q
q
qqq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qqq
q
q
qq
q
q q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q
q
q
qq
q
q q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q q q
q
q
q
q
q
q
q
qq
q
qq
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
q q
qq q
q q
qq
q
q
q
qqq
q
q
q
q
q
q q
q q
qqq
qq
q
qq
q
q
qq
q
qq qqqq
qqq
q
q
qqq
q
qq q
q qq
q q
q
q
q
q
q
q
q q
q
q
q q qq
q
q q
q qq qq
q
q
q
q
qq qq
qq
q q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqqq
q
q
q
q q
qq
q
q
qq qq
qq
q
q
qq qq
q
qq q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq qqq q
qq
qq
q
q
q
q
qq
qq
q
q
qq
q
qq
q
q
q
q q
q q q
q
qq
q q
q
q
q
q
q
q
q
qq q
q
q
q
q q
qq
q
qq qq
qq q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q q
q
qq q
qq
q
q qq q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q q
q
q q q
qq
q
qq
q
q qqqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq qqq
q
q
q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q
qq q
qq
q
qq
q
q
q
q
q qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qqq
q
qq
q
q
q
q
q q
q
qqq
q
q
q
q q
q
q
q
q qq
q
qq
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
qq qq
q
q
qq
q
q q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q q
q
qq
q
q
q
q qqqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
qq
q
qq
q q
q
q
q q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
qq q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
qq
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q
qq
q qq
q
q
qq
q
q
q
q
q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q qq
q
q
q
q
q
q
q
q
qq
qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q q
q
q
q
q
q
q
q qq q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
qq
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
qqq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
qq
q
q
q
q
q
q q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
qq
q
q
q q
qq
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q qqq
q
qq
q qq
qq qq
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q
q
q
qq
q
q
q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qq
q
q
q
q
qq q
q
qq qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
qq
q
q
q
q
qq
q
q
q
q
q
q
q
q
q q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q q
q
qq
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
qq
q
q
q
q
q
q
q
q
qq
qq
q q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qq q
q
qq
q
q
q
q q
q
q
q
q
q
q q
qq
q
qq
q
q
q
q
q
q
q
q
qq
q q
q
q
q
q
q
q
q
q q
q
q
q
q qq q
q
q
q
q q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq qq
q
q
q
q
q
q q
q
q
q
q q qq
q
q
q
q q
q q
q
q
q
q qq
q
q
q
q
q
qq
q
q
qqq
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q q
q
qq
q
q
qq q
q
q
q q
q
q
q
q
q
q qqq
q
q
q
q
q
q
q
qq
q
q
q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
qq
q
q
q
q q
q
q q
q
q q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qqq
q
q
q qq q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q qq
q
q
q
qqqq
qq
q
q
q
qq
q
q
q
q
q
qq
qq
q
q
q
q qq
q
q
q
q
q
q
qq
q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
qq
qq
qqq
q
q q
qq
q
q
q
q
q
q
q
q q
q
q
qq
q
q
q
q q
qq
q
q
q
qqq
q
q
q
q
q
q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q qq
q
q
q
qq
q q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
qqq
q
qq
q
q
q
q
q
q q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q q
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq
qq
q
q
q q
qq
q
q
q
q q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q q
q
q
q
q
qq
q
qq
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q q
q
q
q q
q
q
q
q q
q
q
q
q qqqq
qq
q q
qq
q
q
qq
q
q q
qq
q
q
qq
q
q
qq
q q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q
q q
q
q qq
q
q
q
q
q
qq
q
q
q
q
qq
q
qq qq qq q
qq
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q qq
qq
q
qqq
q
q qqqqq
q
q
q
q
qq
q
q
qq
qq
q
q
q
q
q
q
qqq q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
qq
q
q
q q q
qq
q
q
q
q
q
q
qq
qqqq qq
q
q
q
q
qq
q
q q
q
q
q
q
q
q
q
q
q
qq
qq qqqqqq
q q
q
qq
q q
q
q
qq
q
qq
qqq
q
qqqq
q q
q
q
q
q
q
q
q
q
q
q qq
q
q
q
q
q
q
q
qq
q
qq
q
qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q qq
q
q qq
q
q
q
q
q
q
qq qq
q
q
q
qq
q
q
q
q
qq
q
q
q
q q
q q
q
q
q
q
qq
qq
q q
q
q q
q
q qqq
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q q
q
qq
q
q
q
q
qq q
q
qq
q
q
q
q
q q
qq
q
q
q q
q
q
q
q
q
q
q
qq
q
q
q
qq
qqq
qq
q
q
qq
q
q
qqq
q
q
qq
q
qq q
q
q qqq
q
q q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q q
qqq q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q qqq
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
qq q
q
qq
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q q
q
q
q
q
q q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qqq
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q q
q
q
qqq
q
q
q
q
q
q
q
q
qq
qq
q q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
qq
qq
q
q
q
q
q
q
q
q
qq qq q
q
q
q q q
q
q
qq
q
q
qq q
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
q
q
q q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q q
q
qq q
qq
q
q
q
qq
q
q
q
qq q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
q
q
qq
q
q q
q q
q
qq
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
qqqqq
qqq
qqqqq qq
q
q qqq
q
q qq
q
qq
q
q
qq
qq qq
q
q
q q
q q
q
q
qq
q
q
q qqq
q
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q qq
q
q
q q
q
qq
q
q
q
q
q
q
q
q
q
q qq
q
q
qq
q
q
q
qq
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
q
qq q
q
qq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q qq
q
q
qq
q
q
q
q
q
qq
q q
q
q
q qq
q
q
q
qq
qq
q
q
q
q
q
q
q
q q
q
q
q
q
qq
q
q
q
q
q
q
qqq
qq
q
q
q
q
qq
q
qqq
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q q
q
qq
qq
q
q q
q
q
q
q
q
qq
q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
qq
q
q qq
q
qq
q
q
q
q
qq q
q
q
q q
q
q
qq
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq qqq qq
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
qq q
q
qqq
q
qqq
q
q
qq
q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
qq
q
qq
q
q qq
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq q
qq
q
q
q
qq
q
q
q
q
q
qqq
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q qq
q
q
q
qq
q
q
q
q qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q q
q
q qq
q
q
q
q qq q
q
qq q
q
q
q
q
q qq
q
q
q qq
q
q
q
q qqq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
qq
q
q
q
q
qq
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq q
qq qq q
qq
qq
q q
q
q
qq
q
q
q
q q
q
qq
q
q
q
q
qq
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q q
q
q
q
q
qq
q
q
q
q
q q
q q
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
qq
qq
q
q
q
qqqq
qq
q
q q
qqqq
q
q
q
q
q
q
qqq
q
q
q
q
q
qqq qqq
q
q
q
q
q
q q
q
q
q
q
q
qq
q
q
qq
qq
q
qq
q q
q
q
qq
q
q q
q
qq
q
qq
q
q
q
q
q
q q q
q q
q
q
q
q
q
q
q
q
q
q
q
qq
qq
qq
qq
q
q
q
q
qq
qq
q
q
q
q
q
q
qq
q
q
q q
q
q
q
qq
q
q
qq
q q
qqq
q
q
q
qq
q
q
qq
qq
q
q
q q
q
q
q
q
q
q
qq
q
q
qq
qqq
q
q
q qq qq
q
q qq
q q
q
q
q
q q
q
q
q q
q
q q
qq
q
q qqq
q
q
q
q
q q
q
q
q q
q q
q
qq
q q
q
q
qq
qq
q
q
q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
qqqqq
q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq qq
q
q
q q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
qqq
q
qqq
q
qq
q
q
q
q
q q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
qq
q
q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qqq
q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q qq
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
qq
q
q q
q
q
q
q
qqq
q
q q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
q
q
q q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
qq
qq
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
qq
qq
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q q
q
q
q
q
q
qq
q
q
q
q q
q
q
q
qq
q
q
q
q
q
q
qq
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q q q
q
q
q
q
q
q
q
q
q
qq
qq
q
q
q q
q
q q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qq
q
q
q
qq
qq
qq
q qq qq
q
qqq
q
qq
q
qq q
q
q
qq
q
q
qq
q
qq
q
q
q q q
q qqq
qqq q
qqq
q
qq
q
q
q
q
q
qq qq qqqq q
qq
q qq
q
q q
q
q
q
qq
qq
q
q
q
q
qq
q
q q
q
q
q
q qqqq
qqqqqqq
q
q
q
q
qq
q
q
qq
q
qq
q
qq
qq
qq
q
qqq
q
q
q
q q
q
q
q
q
q q
qqqqqq
qq
q
q
q
qq q
q
q
q q
qqq
q
qq
q
q
q
qq
q
qq
q
q
qqq qq q
qq
q
q
q
q
q
q
q
q
q qq
q
qq
q
q q
q
q q
q
q
qqq
q
q
q
q
q q
q
q
qq
qqq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
qq
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
qqq
q
qq
q q
qq
q
q
q
q
q
q
q
q
q
q
qq
qq
q
qq q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q qq
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q q
q
q
q
q
q
q
q
qq
q
q q
q
q q
q
q q
q
qq
q
q
q
qq
q
q
q
q qq
qq
qq
q
q
q
q
q
q qq
q q
q
q
q
q
qq
q q
q
q
q
q
q
q
qqq
q
qq
qqq
qq q q
q
qq
q
q
q
q
q
q qqq
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
q q
q
q
q q
q
q
q q q
q
q
q
q
q
q
q
q
q
qq
q
q
q q
q q
q
q
q qq q
qq
q
qq
q
qq qqq
q
q
qq
qqq
q
q
q
qq
q
q q
q q qq
qq
q
qqq
qqqq
qqq q q
qqq qq qq q
q
q
q
qq q
q
q
q
q
q
q
q q
q
q
qq
q q
qq
q
q
q
q
qq qq
q
q
qq
q
q
q
q
q
q
q
qq
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq q
q
q
q
q
q
q
qq
q
q
q qq
q
q q
q qq
q
q
q
q
q
qq q
q
q
qq
q
qq
q
q
q q
q
q
q
q
q
q
q
q
q qq
q
q
q q
q
q
q
q
q
q
q
qq
q
q
q
qq
qq
qq
q
qqq
q
q q
q qqqq
qqqqq q
q
qq
q
q
qq qq
q qq
q
q
q q
q q
q
qq q
q
qq
q
q
q
qqq
q
q
q
q q
qq
q
q
q
q
q q
q
q
q
q
q
q
q q
q
q q
q
q
qq q
q
q
q
q
q q
q
q q
q
q
q
q
q
q q
q
q
q
qq
q
q
q
qq
q
q
qq
q
q
q
q
qq
q
q
qqq
q
qq
q
q
q
q qq
q
q
qq
q
q
q
q
q
q
qq
qq
q
q q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q q
qq
q
q
q
q
qq qq
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
qq
qq
q
q
q q
q
q
q
q
q
qq
q
q qq
q
qq
q
q
qqqq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
q
qq
q qq
q
qq
q
qq
qqqq
qq
qqqqq
q
q
qq
q
q
q
q qq
qq
q
q
qq
q qq
q
q
qq q
q
q
q
q
qqq q
q
q
q q
q
q
q
qq
q
q
qqqq
q
q
qq
q
qq
qq
q
q
q
q
q
q
qq
q
q
q
q
q
qq
qq q
q
qq
q
qqq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q q
q q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qq
q
q
q q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q q
q
q
qq
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
qq
q
qq
qq
qqq
q
q
q
q q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
qq
qq
q
q
q
q
q
q
q q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qq
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q qqq
q
q
qqq
q
q
q
q
q
qq
q
q
q
qq
qqq
q
q
q qq qq
q
q
q
q
q
q
q
qq qq
q
q
q
q
q
qq
q
q
q
q
qqqq
qqq
qq
q
qq q
q q
q
q
qq
q
qq
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
qq
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqqq
q
q qq qqqqq
q
q
q
qq
q
q
q
q
q
qq qq
qqq
q
q
q qq q
q
q
q
q qq
qqq qqq
q
qqqqq qqqqqq qq
q
q
q
q
qq
q
q
q
qq
qq
q
qqqq
q q
qqq
q
qqq
q q
qqqq
qq
q
q
q
q
q
q q
q
qq
q
q
q
q
qq qq
qqq
qq qqqqq
q
q
qq q
q
q
qq qqq q
q
q
q q
q
qq qq q
q
qq
qqq
q
q qq q
q q
q
qq
q
q
q
q q
q
q
q
q
q qq
qq
q
q
qq qq
q q
q
qqq
qq
q
qqq
qq qqqq
q
q
q
q
q
q
qq
q
qq
qq qqq q
q
qq
q qqq qq
qqqqqq
q
q
qq q
qqqqq qqq
q qqq
q
qq
qq
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q q
q
q
q
qq
q
q
q
qq
qqq
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
qqqq
qq
q
qq qq
q
q
q
q
q
q
q
q
q
q
q qq
qqq q
q
qq
qq
q q
q
q q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
q
qq
qq q
q q
q
q
q
qq
q
q
q
q
q
q
qqqq
q
q
q
q
q qq
q
q
q
q
q
q
q
q
q
q
q qq qq
q
q
q q
q
q
q
q
q
q q
q q
q q
q
q
q
q
q
q
q
qq qq
q
q
q
q
qqq
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
qqqq
q
q
q
q q
qqqqq
q
q q
q q
q
q
q
q
q
q
q
q
qq
q
q
q
qqq
q
q
qqqq
qqq
q
q
q
q
q
qq
q
q
q q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qqq
q
q
q
q
q
qq
q q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
qq
q q
q
q
q
q
q
q
q
qq
q
q
qq
q
q q
q
q
q
q
q
q
qq
q
q q
q
q
q
q q
q
q
q
qq q
q
qq
q
q
qq
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
qq
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q q
q
q
qqq
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qq
qq
q
q q
q
q
q q
q
q
q
q q
q
q
q
qqq
qq
q
qq q
q
qq
q
q
q
q
q
q q
q
q
qq
q
qq
q q
q q
q
qq
qq
q
qq q
q
q
q
q
qq
q
q
q
q
q
q
q q
q
q
qq
q
q
q
qq
qq
q
q qqq
q q
q
qq qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
qqqq qq
q
q
qq
q
qqq
q
q
q
q
qq qq
q
q
q
q
q q
q
qq
q
q
q
q
q
qq
qqq
qq qq
q
q
q
qq
qq
q
q
q
qq q
q q
qqq
q
q
qq
q
q
q
q
q
q
q
qqqqq
q
q
q
q qq
q
q
q
qq
q
q
q
q q
q
qqqq q
q
qq
q
qq
q
q
q
q
q
qq
q
qq
qq
qq
q
q
q
q
qq
qq
q q
qq qq
q
q
q
q
q
q
q
q
qq qqq
q
q
q
qq
qqq
qq
q
q q q
qq
q
q
q q
qq
q
q
q
q
q
qq
q
qq
q
q
qq
q
q
q
q
q
qq
q
q
qq
q q
q q
q
q
q q
q
q q
q
q
qq
q q
q
q
q
qqq
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qq
q
q
q
q
q
qqq
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqqq
q
q
qq qq
q
q
qq
q
qq q
q
q
q
qq
q
q
q
qq
q
q
qq
q q
q
q
q
qq
q
q
q
q
qq
q
q q
qq
q
q
q q qq
q
q
q
qqq
q
q
q
q
qq
qqq qqq
q
q
qq q
q
q
qq
q
qq
qqq
q
q
q
q q
qq
q
q
qq
qq
q
q qq
q
q
q
q
qqqqq
qq
q qqqq
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
qq
q
qqq
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q q
q
q
qq
q
q
q
q q
q q
q
qq
qq qq
qq
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
qq
q
qq q
q
q
qqq
qqqq
q
q
q
q
q
q
q
q
qqqq
qqq
q
qq
q
q
qqq
q
q
q
q
q
q
qqq
q
q
q
q
qqq
q
q
q
q q
qq
q
q
qq
q
q
qq
q
q
qq
q
q
q
q
q
qqq
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q qq
q
qq
q q
q
q
q
q
q
q
q
q
q
qq
q
q q
q
q
qq q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
qq
q
qq
q
q qq
q
q
q
q q
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
qq
q
q
q
q
qqq q
q
q
q
qq
qq
q
q
q
q
q
q
qqqq q
q
q
q
qq
qqq q
q
q
qq
q
q
q
qqq
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
ACF1
lumpiness
entropy
lshiftvchange
cpoints
fspots
trend
linearity
curvature
spikiness
season
peaktrough
klscore
change.idx
−4
−2
0
2
−2.5 0.0 2.5
standardized PC1 (28.7% explained var.)
standardizedPC2(17.3%explainedvar.)
We need a measure of the “anomalousness” of a time
series.
1 Rank points based on their local density.
2 Rank points based on whether they are within
α-convex hulls of different radius.
Exploring the feature space of large time series collections Yahoo web traffic 20
What is “anomalous”
q
q
qq q
q
q q
q
q
q
qq
q
q
q
q
qq
q
qq
q
q
q
q
q
q
qq
q
q
q
q
qqq
q qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
qqq
q
q
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q q
q
qq
q
q
q
q q
q
q q
q qq
q
q
qqq
q
q
qq
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q q
q
q
q
q
q
q q
q
q qq q qq
q
q
q
q
q
q qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
qq
qqqqq q
q
qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq
qq
qq q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qqq
q
qqqqq q
q
q
q
q
q
q
qqq
q
q
q
q
q
qq
qq
q
qq
qqqq
qq
q
q
q
q q
q
q
q qq
q
qqq
qqqq
qqq
q
q
q
qq
q
q
q
q
q
q
q
qq
qq q
q
q
q
q
q
q
q
q
qq
q qqq
q
q q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q qq
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q qq
q
q q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q qq
q q
q q
q
q
q qq
q qq
q
q
q
q
q q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q q
q q
q
qq
q
q
q
q
q
q
q
q
q
qq
qq
q
q
q
q qq
q
q
qq
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qqqq
q q
q
q
q
q
q
q
qq
qq
q
qqq
q
q
q
qq
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
qq qq
q
q
q
q
qqq
q
q
q
q q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
q
qq
q q
qq
q
q
qq
q
q
q q
q
q
q
q
q
qq
q
qqq
q
q qq
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q q q
q
q
q
q
q
q
qq
q q
q
q
q
qqq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qqq
q
q
qq
q
q q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q
q
q
qq
q
q q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q q q
q
q
q
q
q
q
q
qq
q
qq
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
q q
qq q
q q
qq
q
q
q
qqq
q
q
q
q
q
q q
q q
qqq
qq
q
qq
q
q
qq
q
qq qqqq
qqq
q
q
qqq
q
qq q
q qq
q q
q
q
q
q
q
q
q q
q
q
q q qq
q
q q
q qq qq
q
q
q
q
qq qq
qq
q q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqqq
q
q
q
q q
qq
q
q
qq qq
qq
q
q
qq qq
q
qq q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq qqq q
qq
qq
q
q
q
q
qq
qq
q
q
qq
q
qq
q
q
q
q q
q q q
q
qq
q q
q
q
q
q
q
q
q
qq q
q
q
q
q q
qq
q
qq qq
qq q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q q
q
qq q
qq
q
q qq q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q q
q
q q q
qq
q
qq
q
q qqqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq qqq
q
q
q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q
qq q
qq
q
qq
q
q
q
q
q qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qqq
q
qq
q
q
q
q
q q
q
qqq
q
q
q
q q
q
q
q
q qq
q
qq
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
qq qq
q
q
qq
q
q q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q q
q
qq
q
q
q
q qqqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
qq
q
qq
q q
q
q
q q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
qq q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
qq
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q
qq
q qq
q
q
qq
q
q
q
q
q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q qq
q
q
q
q
q
q
q
q
qq
qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q q
q
q
q
q
q
q
q qq q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
qq
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
qqq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
qq
q
q
q
q
q
q q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
qq
q
q
q q
qq
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q qqq
q
qq
q qq
qq qq
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q
q
q
qq
q
q
q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qq
q
q
q
q
qq q
q
qq qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
qq
q
q
q
q
qq
q
q
q
q
q
q
q
q
q q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q q
q
qq
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
qq
q
q
q
q
q
q
q
q
qq
qq
q q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qq q
q
qq
q
q
q
q q
q
q
q
q
q
q q
qq
q
qq
q
q
q
q
q
q
q
q
qq
q q
q
q
q
q
q
q
q
q q
q
q
q
q qq q
q
q
q
q q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq qq
q
q
q
q
q
q q
q
q
q
q q qq
q
q
q
q q
q q
q
q
q
q qq
q
q
q
q
q
qq
q
q
qqq
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q q
q
qq
q
q
qq q
q
q
q q
q
q
q
q
q
q qqq
q
q
q
q
q
q
q
qq
q
q
q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
qq
q
q
q
q q
q
q q
q
q q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qqq
q
q
q qq q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q qq
q
q
q
qqqq
qq
q
q
q
qq
q
q
q
q
q
qq
qq
q
q
q
q qq
q
q
q
q
q
q
qq
q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
qq
qq
qqq
q
q q
qq
q
q
q
q
q
q
q
q q
q
q
qq
q
q
q
q q
qq
q
q
q
qqq
q
q
q
q
q
q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q qq
q
q
q
qq
q q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
qqq
q
qq
q
q
q
q
q
q q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q q
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq
qq
q
q
q q
qq
q
q
q
q q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q q
q
q
q
q
qq
q
qq
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q q
q
q
q q
q
q
q
q q
q
q
q
q qqqq
qq
q q
qq
q
q
qq
q
q q
qq
q
q
qq
q
q
qq
q q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q
q q
q
q qq
q
q
q
q
q
qq
q
q
q
q
qq
q
qq qq qq q
qq
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q qq
qq
q
qqq
q
q qqqqq
q
q
q
q
qq
q
q
qq
qq
q
q
q
q
q
q
qqq q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
qq
q
q
q q q
qq
q
q
q
q
q
q
qq
qqqq qq
q
q
q
q
qq
q
q q
q
q
q
q
q
q
q
q
q
qq
qq qqqqqq
q q
q
qq
q q
q
q
qq
q
qq
qqq
q
qqqq
q q
q
q
q
q
q
q
q
q
q
q qq
q
q
q
q
q
q
q
qq
q
qq
q
qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q qq
q
q qq
q
q
q
q
q
q
qq qq
q
q
q
qq
q
q
q
q
qq
q
q
q
q q
q q
q
q
q
q
qq
qq
q q
q
q q
q
q qqq
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q q
q
qq
q
q
q
q
qq q
q
qq
q
q
q
q
q q
qq
q
q
q q
q
q
q
q
q
q
q
qq
q
q
q
qq
qqq
qq
q
q
qq
q
q
qqq
q
q
qq
q
qq q
q
q qqq
q
q q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q q
qqq q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q qqq
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
qq q
q
qq
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q q
q
q
q
q
q q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qqq
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q q
q
q
qqq
q
q
q
q
q
q
q
q
qq
qq
q q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
qq
qq
q
q
q
q
q
q
q
q
qq qq q
q
q
q q q
q
q
qq
q
q
qq q
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
q
q
q q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q q
q
qq q
qq
q
q
q
qq
q
q
q
qq q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
q
q
qq
q
q q
q q
q
qq
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
qqqqq
qqq
qqqqq qq
q
q qqq
q
q qq
q
qq
q
q
qq
qq qq
q
q
q q
q q
q
q
qq
q
q
q qqq
q
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q qq
q
q
q q
q
qq
q
q
q
q
q
q
q
q
q
q qq
q
q
qq
q
q
q
qq
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
q
qq q
q
qq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q qq
q
q
qq
q
q
q
q
q
qq
q q
q
q
q qq
q
q
q
qq
qq
q
q
q
q
q
q
q
q q
q
q
q
q
qq
q
q
q
q
q
q
qqq
qq
q
q
q
q
qq
q
qqq
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q q
q
qq
qq
q
q q
q
q
q
q
q
qq
q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
qq
q
q qq
q
qq
q
q
q
q
qq q
q
q
q q
q
q
qq
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq qqq qq
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
qq q
q
qqq
q
qqq
q
q
qq
q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
qq
q
qq
q
q qq
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq q
qq
q
q
q
qq
q
q
q
q
q
qqq
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q qq
q
q
q
qq
q
q
q
q qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q q
q
q qq
q
q
q
q qq q
q
qq q
q
q
q
q
q qq
q
q
q qq
q
q
q
q qqq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
qq
q
q
q
q
qq
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq q
qq qq q
qq
qq
q q
q
q
qq
q
q
q
q q
q
qq
q
q
q
q
qq
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q q
q
q
q
q
qq
q
q
q
q
q q
q q
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
qq
qq
q
q
q
qqqq
qq
q
q q
qqqq
q
q
q
q
q
q
qqq
q
q
q
q
q
qqq qqq
q
q
q
q
q
q q
q
q
q
q
q
qq
q
q
qq
qq
q
qq
q q
q
q
qq
q
q q
q
qq
q
qq
q
q
q
q
q
q q q
q q
q
q
q
q
q
q
q
q
q
q
q
qq
qq
qq
qq
q
q
q
q
qq
qq
q
q
q
q
q
q
qq
q
q
q q
q
q
q
qq
q
q
qq
q q
qqq
q
q
q
qq
q
q
qq
qq
q
q
q q
q
q
q
q
q
q
qq
q
q
qq
qqq
q
q
q qq qq
q
q qq
q q
q
q
q
q q
q
q
q q
q
q q
qq
q
q qqq
q
q
q
q
q q
q
q
q q
q q
q
qq
q q
q
q
qq
qq
q
q
q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
qqqqq
q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq qq
q
q
q q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
qqq
q
qqq
q
qq
q
q
q
q
q q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
qq
q
q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qqq
q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q qq
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
qq
q
q q
q
q
q
q
qqq
q
q q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
q
q
q q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
qq
qq
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
qq
qq
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q q
q
q
q
q
q
qq
q
q
q
q q
q
q
q
qq
q
q
q
q
q
q
qq
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q q q
q
q
q
q
q
q
q
q
q
qq
qq
q
q
q q
q
q q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qq
q
q
q
qq
qq
qq
q qq qq
q
qqq
q
qq
q
qq q
q
q
qq
q
q
qq
q
qq
q
q
q q q
q qqq
qqq q
qqq
q
qq
q
q
q
q
q
qq qq qqqq q
qq
q qq
q
q q
q
q
q
qq
qq
q
q
q
q
qq
q
q q
q
q
q
q qqqq
qqqqqqq
q
q
q
q
qq
q
q
qq
q
qq
q
qq
qq
qq
q
qqq
q
q
q
q q
q
q
q
q
q q
qqqqqq
qq
q
q
q
qq q
q
q
q q
qqq
q
qq
q
q
q
qq
q
qq
q
q
qqq qq q
qq
q
q
q
q
q
q
q
q
q qq
q
qq
q
q q
q
q q
q
q
qqq
q
q
q
q
q q
q
q
qq
qqq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
qq
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
qqq
q
qq
q q
qq
q
q
q
q
q
q
q
q
q
q
qq
qq
q
qq q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q qq
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q q
q
q
q
q
q
q
q
qq
q
q q
q
q q
q
q q
q
qq
q
q
q
qq
q
q
q
q qq
qq
qq
q
q
q
q
q
q qq
q q
q
q
q
q
qq
q q
q
q
q
q
q
q
qqq
q
qq
qqq
qq q q
q
qq
q
q
q
q
q
q qqq
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
q q
q
q
q q
q
q
q q q
q
q
q
q
q
q
q
q
q
qq
q
q
q q
q q
q
q
q qq q
qq
q
qq
q
qq qqq
q
q
qq
qqq
q
q
q
qq
q
q q
q q qq
qq
q
qqq
qqqq
qqq q q
qqq qq qq q
q
q
q
qq q
q
q
q
q
q
q
q q
q
q
qq
q q
qq
q
q
q
q
qq qq
q
q
qq
q
q
q
q
q
q
q
qq
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq q
q
q
q
q
q
q
qq
q
q
q qq
q
q q
q qq
q
q
q
q
q
qq q
q
q
qq
q
qq
q
q
q q
q
q
q
q
q
q
q
q
q qq
q
q
q q
q
q
q
q
q
q
q
qq
q
q
q
qq
qq
qq
q
qqq
q
q q
q qqqq
qqqqq q
q
qq
q
q
qq qq
q qq
q
q
q q
q q
q
qq q
q
qq
q
q
q
qqq
q
q
q
q q
qq
q
q
q
q
q q
q
q
q
q
q
q
q q
q
q q
q
q
qq q
q
q
q
q
q q
q
q q
q
q
q
q
q
q q
q
q
q
qq
q
q
q
qq
q
q
qq
q
q
q
q
qq
q
q
qqq
q
qq
q
q
q
q qq
q
q
qq
q
q
q
q
q
q
qq
qq
q
q q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q q
qq
q
q
q
q
qq qq
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
qq
qq
q
q
q q
q
q
q
q
q
qq
q
q qq
q
qq
q
q
qqqq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
q
qq
q qq
q
qq
q
qq
qqqq
qq
qqqqq
q
q
qq
q
q
q
q qq
qq
q
q
qq
q qq
q
q
qq q
q
q
q
q
qqq q
q
q
q q
q
q
q
qq
q
q
qqqq
q
q
qq
q
qq
qq
q
q
q
q
q
q
qq
q
q
q
q
q
qq
qq q
q
qq
q
qqq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q q
q q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qq
q
q
q q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q q
q
q
qq
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
qq
q
qq
qq
qqq
q
q
q
q q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
qq
qq
q
q
q
q
q
q
q q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qq
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q qqq
q
q
qqq
q
q
q
q
q
qq
q
q
q
qq
qqq
q
q
q qq qq
q
q
q
q
q
q
q
qq qq
q
q
q
q
q
qq
q
q
q
q
qqqq
qqq
qq
q
qq q
q q
q
q
qq
q
qq
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
qq
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqqq
q
q qq qqqqq
q
q
q
qq
q
q
q
q
q
qq qq
qqq
q
q
q qq q
q
q
q
q qq
qqq qqq
q
qqqqq qqqqqq qq
q
q
q
q
qq
q
q
q
qq
qq
q
qqqq
q q
qqq
q
qqq
q q
qqqq
qq
q
q
q
q
q
q q
q
qq
q
q
q
q
qq qq
qqq
qq qqqqq
q
q
qq q
q
q
qq qqq q
q
q
q q
q
qq qq q
q
qq
qqq
q
q qq q
q q
q
qq
q
q
q
q q
q
q
q
q
q qq
qq
q
q
qq qq
q q
q
qqq
qq
q
qqq
qq qqqq
q
q
q
q
q
q
qq
q
qq
qq qqq q
q
qq
q qqq qq
qqqqqq
q
q
qq q
qqqqq qqq
q qqq
q
qq
qq
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q q
q
q
q
qq
q
q
q
qq
qqq
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
qqqq
qq
q
qq qq
q
q
q
q
q
q
q
q
q
q
q qq
qqq q
q
qq
qq
q q
q
q q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
q
qq
qq q
q q
q
q
q
qq
q
q
q
q
q
q
qqqq
q
q
q
q
q qq
q
q
q
q
q
q
q
q
q
q
q qq qq
q
q
q q
q
q
q
q
q
q q
q q
q q
q
q
q
q
q
q
q
qq qq
q
q
q
q
qqq
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
qqqq
q
q
q
q q
qqqqq
q
q q
q q
q
q
q
q
q
q
q
q
qq
q
q
q
qqq
q
q
qqqq
qqq
q
q
q
q
q
qq
q
q
q q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qqq
q
q
q
q
q
qq
q q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
qq
q q
q
q
q
q
q
q
q
qq
q
q
qq
q
q q
q
q
q
q
q
q
qq
q
q q
q
q
q
q q
q
q
q
qq q
q
qq
q
q
qq
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
qq
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q q
q
q
qqq
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qq
qq
q
q q
q
q
q q
q
q
q
q q
q
q
q
qqq
qq
q
qq q
q
qq
q
q
q
q
q
q q
q
q
qq
q
qq
q q
q q
q
qq
qq
q
qq q
q
q
q
q
qq
q
q
q
q
q
q
q q
q
q
qq
q
q
q
qq
qq
q
q qqq
q q
q
qq qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
qqqq qq
q
q
qq
q
qqq
q
q
q
q
qq qq
q
q
q
q
q q
q
qq
q
q
q
q
q
qq
qqq
qq qq
q
q
q
qq
qq
q
q
q
qq q
q q
qqq
q
q
qq
q
q
q
q
q
q
q
qqqqq
q
q
q
q qq
q
q
q
qq
q
q
q
q q
q
qqqq q
q
qq
q
qq
q
q
q
q
q
qq
q
qq
qq
qq
q
q
q
q
qq
qq
q q
qq qq
q
q
q
q
q
q
q
q
qq qqq
q
q
q
qq
qqq
qq
q
q q q
qq
q
q
q q
qq
q
q
q
q
q
qq
q
qq
q
q
qq
q
q
q
q
q
qq
q
q
qq
q q
q q
q
q
q q
q
q q
q
q
qq
q q
q
q
q
qqq
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qq
q
q
q
q
q
qqq
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqqq
q
q
qq qq
q
q
qq
q
qq q
q
q
q
qq
q
q
q
qq
q
q
qq
q q
q
q
q
qq
q
q
q
q
qq
q
q q
qq
q
q
q q qq
q
q
q
qqq
q
q
q
q
qq
qqq qqq
q
q
qq q
q
q
qq
q
qq
qqq
q
q
q
q q
qq
q
q
qq
qq
q
q qq
q
q
q
q
qqqqq
qq
q qqqq
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
qq
q
qqq
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q q
q
q
qq
q
q
q
q q
q q
q
qq
qq qq
qq
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
qq
q
qq q
q
q
qqq
qqqq
q
q
q
q
q
q
q
q
qqqq
qqq
q
qq
q
q
qqq
q
q
q
q
q
q
qqq
q
q
q
q
qqq
q
q
q
q q
qq
q
q
qq
q
q
qq
q
q
qq
q
q
q
q
q
qqq
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q qq
q
qq
q q
q
q
q
q
q
q
q
q
q
qq
q
q q
q
q
qq q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
qq
q
qq
q
q qq
q
q
q
q q
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
qq
q
q
q
q
qqq q
q
q
q
qq
qq
q
q
q
q
q
q
qqqq q
q
q
q
qq
qqq q
q
q
qq
q
q
q
qqq
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
ACF1
lumpiness
entropy
lshiftvchange
cpoints
fspots
trend
linearity
curvature
spikiness
season
peaktrough
klscore
change.idx
−4
−2
0
2
−2.5 0.0 2.5
standardized PC1 (28.7% explained var.)
standardizedPC2(17.3%explainedvar.)
We need a measure of the “anomalousness” of a time
series.
1 Rank points based on their local density.
2 Rank points based on whether they are within
α-convex hulls of different radius.
Exploring the feature space of large time series collections Yahoo web traffic 20
Bivariate kernel density
ˆf(x; H) =
1
n
n
i=1
KH(x − Xi)
Xi ∈ a bivariate random sample {X1, X2, . . . , Xn}
KH(x) is the standard normal kernel function
H estimated by minimizing the sum of AMISE
Rank points based on ˆf values in 2d PCA space.
Exploring the feature space of large time series collections Yahoo web traffic 21
Bivariate kernel density
ˆf(x; H) =
1
n
n
i=1
KH(x − Xi)
Xi ∈ a bivariate random sample {X1, X2, . . . , Xn}
KH(x) is the standard normal kernel function
H estimated by minimizing the sum of AMISE
Rank points based on ˆf values in 2d PCA space.
Exploring the feature space of large time series collections Yahoo web traffic 21
Bivariate density ranking
Exploring the feature space of large time series collections Yahoo web traffic 22
−5 0 5
−8−6−4−20246
pc1
pc2
1
2
3
4
5
Bivariate density ranking
Exploring the feature space of large time series collections Yahoo web traffic 22
0
10000
20000
30000
40000
0
2000
4000
6000
0
10000
20000
30000
40000
50000
0
10000
20000
30000
40000
0
1000
2000
3000
4000
5000
S7793S8494S10464S7833S1715
2015−02−28
2015−03−01
2015−03−02
2015−03−03
2015−03−04
2015−03−05
2015−03−06
2015−03−07
2015−03−08
2015−03−09
2015−03−10
2015−03−11
2015−03−12
2015−03−13
2015−03−14
2015−03−15
2015−03−16
2015−03−17
2015−03−18
2015−03−19
2015−03−20
2015−03−21
2015−03−22
2015−03−23
2015−03−24
2015−03−25
2015−03−26
2015−03−27
2015−03−28
2015−03−29
2015−03−30
2015−03−31
2015−04−01
date
value
α-convex hulls
The space generated by point pairs that can be
touched by an empty disc of radius α.
α → ∞ gives a convex hull.
Points can become isolated when α is small.
We rank points based on the value of α when
they become isolated.
Exploring the feature space of large time series collections Yahoo web traffic 23
α-convex hulls
The space generated by point pairs that can be
touched by an empty disc of radius α.
α → ∞ gives a convex hull.
Points can become isolated when α is small.
We rank points based on the value of α when
they become isolated.
Exploring the feature space of large time series collections Yahoo web traffic 23
α-convex hulls
The space generated by point pairs that can be
touched by an empty disc of radius α.
α → ∞ gives a convex hull.
Points can become isolated when α is small.
We rank points based on the value of α when
they become isolated.
Exploring the feature space of large time series collections Yahoo web traffic 23
α-convex hulls
The space generated by point pairs that can be
touched by an empty disc of radius α.
α → ∞ gives a convex hull.
Points can become isolated when α is small.
We rank points based on the value of α when
they become isolated.
Exploring the feature space of large time series collections Yahoo web traffic 23
α-convex hull
Exploring the feature space of large time series collections Yahoo web traffic 24
−5 0 5
−8−6−4−20246
q
q
qq q
q
q q
q
q
q
qq
q
q
q
q
qq
q
qq
q
q
q
q
q
q
qq
q
q
q
q
qqqq qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
qqq
q
q
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q q
q
qq
q
q
q
q q
q q q
q qq
q
q
qqq
q
q
qq
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
q
q
q
qq
q
q q
q
q
q
q
q
q q
q
q qq q qq
q
q
q
q
q
q qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q
q
qqq
q
qq
q
qq
qqqqq q
q
qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq
qq
qq q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qqq
q
qqqqq q
q
q
q
q
q
q
qqq
q
q
q
q
q
qq
qq
q
qq
qqqq
qq
q
q
q
q q
q
q
q qq
q
qqq
qqqq qqq
q
q
q
qq
q
q
q
q
q
q
q
qq
qqq
q
q
q
q
q
q
q
q
qq
q qqq
q
q q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q qqq
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q qq
q
q q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q qq
q q
q q
q
q
q qq
q qq
q
q
q
q
q q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q q
q q
q
qq
q
q
q
q
q
q
q
q
q
qq
qq
q
q
q
q qq
q
q
qq
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qqqq
q q
q
q
q
q
q
q
qq
qq
q
qqq
q
q
q
qq
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
qq qq
q
q
q
q
qqq
q
q
q
q q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
q
qq
q q
qq
q
q
qq
q
q
q q
q
q
q
q
q
qq
q
qqq
q
q qq
q
q
q
q q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q q q
q
q
q
q
q
q
qq
q q
q
q
q
qqq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qqq
q
q
qq
q
q q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q
q
q
qq
q
q q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q q
q
q
q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q qq
q
q
q
q
q
q
q
qq
q
qq
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
qq
q
q q
qq q
q q
qq
q
q
q
qqq
q
q
q
q
q
q q
q q
qqq
qq
q
qq
q
q
qq
q
qq qqqq
qqq
q
q
qqq
q
qq q
q qq
q q
q
q
q
q
q
q
q q
q
q
q q qq
q
q qq qqqq
q
q
q
q
qq qq
qq
q q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqqq
q
q
q
q q
qq
q
q
qq qq
qq
q
q
qq qq
q
qq q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq qqq q
qq
qq
q
q
q
q
qq
qq
q
q
qq
q
qq
q
q
q
q q
q q q
q
qq
q q
q
q
q
q
q
q
q
qq q
q
q
q
q q
qq
q
qq qq
qq q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q q
q
qq q
qq
q
q qq q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q q
q
q q q
qq
q
qq
q
q qqqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q
q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq qqq
q
q
q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q
qq q
qq
q
qq
q
q
q
q
q qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qqq
q
qq
q
q
q
q
q q
q
qqq
q
q
q
q q
q
q
q
q qq
q
qq
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
qq qq
q q
qq
q
q q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q q
q
qq
q
q
q
q qqqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
qq
q
qq
q q
q
q
q q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq qq q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
qq
q
q
qq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q
qq
q qq
q
q
qq
q
q
q
q
q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q qq
q
q
q
q
q
q
q
q
qq
qq
q
q
q
q
q
q
q
q
q q
q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q q
q
q
q
q
q
q
q qq q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
qq
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
qqq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
qq
q
q
q
q
q
q q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
qq
q
q
q q
qq
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q qqq
q
qq
q qq
qq qq
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q
q
q
qq
q
q
q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qq
q
q
q
q
qq q
q
qq qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
qq
q
q
q
q
qq
q
q
q
q
q
q
q
q
q q
q
q
qq q
q
q
q
qq
q
q
q
q
q
q
q q
q
q
q
q
q
q q
q
qq
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
qq
q
q
q
q
q
q
q
q
qq
qq
q q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qq q
q
qq
qq
q
q q
q
q
q
q
q
q q
qq
q
qq
q
q
q
q
q
q
q
q
qq
q q
q
q
q
q
q
q
q
q q
q
q
q
q qq q
q
q
q
q q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq qq
q
q
q
q
q
q q
q
q
q
q q qq
q
q
q
q qq q
q
q
q
q qq
q
q
q
q
q
qq
q
q
qqq
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq q
q
qq
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q
q qq
q
q
qq
q
q
q
q q
q
qq
q
q
qq q
q
q
q q
q
q
q
q
q
q qqq
q
q
q
q
q
q
q
qq
q
q
q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
qq
q q
q
q q
q
q q
q
q q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qq
q
q
q
q qq q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q qq
q
q
q
qqqq
qq
q
q
q
qq
q
q
q
q
q
qq
qq
q
q
q
q qq
q
q
q
q
q
q
qq
q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
qq
qq
qqq
q
q q
qq
q
q
q
q
q
q
q
q q
q
q
qq
q
q
q
q q
qq
q
q
q
qqq
q
q
q
q
q
q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
qq q
q qq
q
q
q
qq
qq
q
q
qq
q
q
q
q
q
q
q
q
q
qq
qqq
q
qq
q
q
q
q
q
q q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q q q
qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq
qq
q
q
qq
qq
q
q
q
q q
q
qq
qq
q
q
q
q
q
qq
q q
q
q
q
qq
q
q
q
q q
q
q
q
q
qq
q
qq
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
qq
q
q
q q
q
q
q
q q
q
q
q
q qqqq
qq
q q
qq
q
q
qq
q
q q
qq
q
q
qq
q
q
qq
q q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q
q q
q
q qq
q
q
q
qq
qq
q
q
q
qqq
q
q
q q
q q
q
q
qq
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q qq
qq
q
qqq
q
q qqqqq
q
q
q
q
qq
q
qqq
qq
q
q
q
q
q
q
qqq q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
qq
q
q
q q q
qq
q
q
q q
q
q
qq
qqqq qq
q
q
q
q
qq
q
q q
q
q
q
q
q
q
q
q
q
qq
qq q qqqqq
q q
qqq
q q
q
q
qq
q
qq qqqq
qqqq
q q
q
q
q
q
q
q
q
q
q
q qq
q
q
q
q
q
q
q
qq
q
qq
q
qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q qq
q
q qq
q
q
q
q
q
q
qq qq
q
q
q
qq
q
q
q
q
qq
q
q
q
q q
q q
q
q
q
q
qq
qq
q q
q
q q
q
q qqq
q
q
q
q
q
q
q
qq
qq q
q
q
q
q q
q
qq
q
q
q
q
qq q
q
qq
q
q
q
q
q q
qq
q
q
q q
q
q
q q
q
q
qqq
q
q
q
qq
qqq
qq
q
q
qq
q
q
qqq
q
q
qq
q
qq q
q
q qqq
q
qq
q
q
qq
q
q
q
q
qqq
q
q
q
q
q
q
q
q
q q
qqq q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qqq
q
q
q
q q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q qqq
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
qq q
q
qq
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q q
q
q
qq
q q
q
q
q
q
q
q
qq
q
q
q
qq
q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qqq
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
q q
q
q
qqq
q
q
q
q
q
q
q
q
qq
qq
q q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
qq
qq
q
q
q
q
q
q
q
q
qq qq q
q
q
q q q
q
q
qq
q
q
qq q
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
q
q
q q
q
q
q
q
q
q
q
q
qqq
q
q
q
q q
q
qq q
qq
q
q
q
qq
q
q
q
qq q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
q
q
qq
q
q q
q q
q
qq
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
qqqqq
qqq
qqqqq qq
q
q qqq
q
q qqq
qq
q
q
qq
qq qq
q
q
q q
q q
q
q
qq
q
q
q qqq
q
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q qq
q
q
q q
q
qq
q
q
q
q
q
q
q
q
q
q qq
q
q
qq
q
q
q
qq
q
q
qq
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
q
qqq
q
qq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q qq
q
q
qq
q
q
q
q
q
qq
q q
q
q
q qq
q
q
q
qq
qq
q
q
q
q
q
q
q
q q
q
q
q
q
qq
q
q
q
q
q
q qqq
qq
qq
q
q
qq
q
qqq
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q q
q
qq
qq
q
q q
q
q
q
q
q
qq
q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
qq
q
q qq
q
qq
q
q
q
q
qq q
q
q
q q
q
q
qq
q
q
q
q
q
q
qq q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
qq qqq qq
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
qq q
q
qqq
q
qqq
q
q
qq
q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
qq
q
qq
q
q qq
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq q
qq
q
q
q
qq
q
q
q
q
q
qqq
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qqq
q
q
q
q qq
q
q
q
qq
q
q
q
q qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q q
q
q qq
q
q
q
q qq q
q
qq qq
q
q
q
q qq
q
q
q qq
q
q
q
q qqq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
qq
q
q
q
q
qq
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q q
q qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq q
qq qq q
qq
qq
q q
q
q
qq
q
q
q
q q
q
qq
q
q
qq
qq
q qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q q
q
q
q
q
qq
q
q
q
q
q q
q q
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
qq
qq
q
q
q
qqqq
qq
q
q q
qqqq
q
q
q
q
q
q
qqq
q
q
q
q
q
qqq qqq
q
q
q
q
q
q q
q
q
q q
q
qq
q
q
qq
qq
q
qq
q q
q
q
qq
q
q q
q
qq
q
qq
qq
q
q
q
q q q
q q
q
q
q
q
q
q
q
q
q
q
q
qq
qq
qq
qq
q
q
q
q
qq
qq
q
q
q
q
q
q
qq
q
q
q q
q
q
q
qq
q
q
qq
q qqqq
q
q
q
qq
q
q
qq
qq
q
q
q q
q
q
q
q
q
q
qq
q
q
qq
qqq
q
q
q qq qq
qq qq
q q q
q q
q
q
q
q
q q
q
q q
qq
q
q qqq
q
q
q
q
q q
q
q
q q
q q
q
qq
q q
q
q
qqqq
q
q
q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q q
qqq
q
q
q
q
q
q
q
qq
q
q
q
qqqqq
q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq qq
q
q
q q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
qqq
q
qqq
q
qq
q
q
q
q
q q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q
qq
q
q
q
q
qq
q
q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q
qqq
q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q q
q
q
q
q
q
q qqq q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
qq
q
q q
q
q
q
q
qqq
q
q q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
q
q
q q
q
q
q
q
q
q
q q
q
q
q
q
q
qq
qq
qq
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
qq
qq
q q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
qqq q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q q
q
q
q
q
q
qq
q
q
q
q q
q
q
q
qq
q
q
q
q
q
q
qq
q
qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq qq
q
qq
q
q
q
q
q
q
q
q
q
q
q q q
q
q
q
q
q
q
q
q
q
qq
qq
q
q
q q
q
q q
q
q
q
q
q
qqqq
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qq
q
q
q
qq
qq
qq
q qq qq
q
qqq
q
qq
q
qq q
q
q
qq
q
q
qq
q
qq
q
q
q q q
q qqq
qqq q
qqq
q
qq
q
q
q
q
q
qq qq qqqqq
qq
q qq
q
q q
q
q
q
qq
qq
q
q
q
q
qq
q
q q
q
q
qq qqqq
qqq
qqqq
q
q
q
q
qq
q
q
qq
q
qq
q
qq
qq
qq
q
qqq
q
q
q
q q
q
q
q
q
q q
qqqqqq
qq
q
q
q
qq q
q
q
qq
qqq
q
qq
q
q
q
qq
q
qq
q
qqqq qq q
qq
q
q
q
q
q
q
q
q
q qq
q
qq
q
q q
q
q q
q
q
qqq
q
q
q
q
q q
q
q
qq
qqq
q
q
q
q
q
q
q
q
q q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
qq
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
qqq
q
qq
q q
qq
q
q
q
q
q
q
q
q
q
q
qq
qq
q
qq q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q qq
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q q
q
q
q
q
q
q
q
qq
q
q qq
q q
q
q q
q
qq
q
q
q
qq
q
q
q
q qq
qq
qq
q
q
q q
q
q qq
q q
q
q
q
q
qq
q qq
q
q
q
q
q
qqq
q
q
q
qqq
qq q q
q qq
q
q
q
q
q
q qqq
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
q q
q
q
q q
q
q
q q q
q
q
q
q
qq
q
q
q
qq
q
q
q q
q q q
q
q qq q
qqq
qq
q
qq qqq
q
q
qq
qqqq
q
q
qq
q
q qq q qq
qq
q
qqq
qqqq
qqq q q
qqq qq qq q
q
q
q
qq q
q
q
q
q
q
q
q q
q
q
qq
q q
qq
q
qq
q
qq qq
q
q
qq q
q
q
q
q
q
q
qq
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq q
q
q q
q
q
q
qq
q
q
q qq
q
q q
q qqq
q
q
q
q
qq q
q
q
qq
q
qq
q
q
q q q
q
q
q
q
q
q
q
q qq
q
q
q q
q
q
q
q
q
q
q
qq
q
q
q
qq
qq
qq
q
qqq
q
q q
q qqqq
qqqqq q
q
qq
q
q
qq qq
q qq
q
q
q q
q q
q
qq q
q
qq q
q
q
qqq
q
q
q
q q
qq
q
q
q
q
q q
q
q
q
q
q
q
q q
q
q q
q
q
qq q
q
q
q
q
q q
q
q q
q
q
q
q
q
q q
q
q
q
qq
q
q
q
qq
q
q
qq
q
q
q
q
qq
q
qqqq
q
qq
q
q
q
q qq
q
q
qq
q
q
q
q
q
q
qq
qq
q
q q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q q
qq
q
q
q
q
qq qq
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q q
q
q
qq
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
qq
qq
q
q
q q
q
q
q
q
q
qq
q
q qq
q
qq
q
q
qqqq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
q
qq
q qq
q
qq
q
qq
qqqq
qq
qqqqq
qq
qq
q
q
q
q qq
qq
q
q
qq
q qq
qq
qq q
q
q
q
q
qqq q
qq
q q
q
q
q
qq
q
q
qqqq
q
q
qq
q
qq
qq
q
q
q
q
q
q
qq
q
q
q
q
q
qq
qq q
q
qq
q
qqq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q q
q q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qq
q
q
q q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q q
q
q
qq
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
qq
q
qq
qq
qqq
q
q
q
q q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
qq
qq
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q qq
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q qqq
q
q
qqq
q
q
q
q
q
qq
q
q
q
qq
qqq
q
q
q qq qq
q
q
q
q
q
q
q
qq qq
q
q
q
q
q
qq
q
q
q
qqqqq
qqq
qq
q
qq q
q q
q
q
qq
q
qq
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
qq
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqqq
q
q qqqqqqq
q
q
q
qq
q
q
q
q
q
qq qq
qqq
q
q
q qq q
q
q
q
q qq
qqq qqqq
qqqqq qqqqqq qq
q
qq
q
qq
q
q
q
qq
qq
q
qqqq
q q
qqq
q
qqq
q q
qqqq
qq
q
q
q
q
q
q q
q
qq
qq
q
q
qq qq
qqq
qq qqqqq
q
q
qq q
q
q
qq qqq q
q
q
q q
qqq qq q
qqq
qqq
q
qqq q
q q
q
qq
q
q
q
q q
q
q
q
q
q qq
qq
q
q
qq qq
q q
q
qqq
qq
q
qqq
qq qqqq
q
q
q
q
q
q
qq
q
qq
qq qqq q
q
qq
q qqq qq
qqqqqq
q
q
qq q
qqqqq qqq
q qqq
qqq
qq
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q q
q
q
q
qq
q
q
q
qq
qqq
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
qqqq
qq
q
qq qq
q
q
q
q
q
q
q
q
q
q
q qqqqq q
q
qq
qq
q q
q
q q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
q
qq
qq q
q q
qq
q
qq
q
q
q
q
q
q
qqqq
q
q
q
q
q qq
q
q
q
q
q
q
q
q
q
q
q qq qq
q
q
q q
q
q
q
q
q
q q
q q
q qq
q
q
q
q
q
q
qq qq
q
q
q
q
qqq
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
qqqq
q
q
q
q q
qqqqq
q
q qqq
q
q
q
q
q
q
q
q
qq q
q
q
qqq
q
q
qqqq
qqq
q
q
q
q
qqq
q
q
q q
q
q
q
q q
q
q q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qqq
q
q
q
q
q
qq
q q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
qq
q q
q
q
q
q
q
q
q
qq
q
q
qq
q
q q
q
q
q
q
q
q
qq
q
q q
q
q
q
q q
q
q
q
qq q
q
qq
q
qqq
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
qq
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q q
q
q
q q
q
q
qqq
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qq
qq
q
q q
q
q
q q
q
q
q
q q
q
q
q
qqq
qq
q
qq q
q
qq
q
q
q
q
q
q q
q
q
qq
q qq
q q
q qq
qq
qq
q
qq q
q
q
q
q
qq
q
q
q
q
q
q
q q
q
q
qq
q
q
q
qq
qq
q
q qqq
q qq
qq qq
q
q
q
q
q
q
q
q
q q
qq
q
q
q
q
q q
qq
q
qqqq qq
q
q
qq
q
qqq
q
q
q
q
qq qq
q
q
q
q
q q
q
qq
q
q
q
q
q
qq
qqq
qq qq
q
q
q
qq
qq
q
q
q
qq q
q q
qqq
q
q
qqq
q
q
q
q
q
q
qqqqq
q
q
q
q qq
q
q
q
qq
q
q
q
q q
q
qqqq q
q
qq
q
qq
q
q
q
q
q
qq
q
qq
qq
qq
q
q
q
q
qq qq
q q
qq qq
q
q
q
q
q
q
q
q
qq qqq
q
q
q
qq
qqq
qq
q
qq qqq
q
q
q q
qq
q
q
q
q
q
qq
q
qq
q
q
qq
q q
q
q
q
qq
q
q
qq
q q
q q
q
q
q q
q
q q
q
q
qq
q q
q
q
q
qqq
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
q
q
q qq
q
q
q
q
q
qqq
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
qqqq
q
q
qq qq
q
q
qq
q
qq q
q
q
q
qq
q
q
q
qq
q
q
qq
q q
q
q
q
qq
q
q
q
q
qq
q
q q
qq
q
q
qq qq
q
q
q
qqq q
q
q
q
qq
qqq qqq
q
q
qq q
q
q
qq
q
qq
qqq
q
q
q
q q
qq
q
q
qq
qq
q
q qq
q
q
q
q qqqqqqq
q qqqq
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
qq
q
qqq
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q q
q
q
qq
q
q
q
q q
q q
q
qq
qq qq
qq
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
qq
q
qq q
q
q
qqq
qqqq
q
q
q
q
q
q q
q
qqqq
qqq
q
qq
q
q
qqq
q
q
q
q
q
q
qqq
q
q
q
q
qqq
q
q
q
q q
qq
q
q
qq
q
q
qq
q
q
qq
q
q
q
q
q
qqq
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q qq
q
qq
q q
q
q
q
q
q
q
q
q
q
qq
q
q q q
q
qq q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
qq
q
qq
q
q qqq
q
q
q q
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
qq
q
q
q
q
qqq q
q
q
q
qq
qq
q
q
q
q
q
q
qqqq q
q
q
q
qq
qqq q
q
q
qq
q
q q
qqq
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
α-convex hull ranking
Exploring the feature space of large time series collections Yahoo web traffic 25
−5 0 5
−8−6−4−20246
q
q
qq q
q
q q
q
q
q
qq
q
q
q
q
qq
q
qq
q
q
q
q
q
q
qq
q
q
q
q
qqqq qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
qqq
q
q
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q q
q
qq
q
q
q
q q
q q q
q qq
q
q
qqq
q
q
qq
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
q
q
q
qq
q
q q
q
q
q
q
q
q q
q
q qq q qq
q
q
q
q
q
q qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q
q
qqq
q
qq
q
qq
qqqqq q
q
qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq
qq
qq q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qqq
q
qqqqq q
q
q
q
q
q
q qqq
q
q
q
q
q
qq
qq
q
qq
qqqq
qq
q
q
q
q q
q
q
q qq
q
qqq
qqqq qqq
q
q
q
qq
q
q
q
q
q
q
q
qq
qqq
q
q
q
q
q
q
q
q
qq
q qqq
q
q q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q qqq
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q qq
q
q q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q qq
q q
q q
q
q
q qq
q qq
q
q
q
q
q q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q q
q q
q
qq
q
q
q
q
q
q
q
q
q
qq
qq
q
q
q
q qq
q
q
qq
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qqqq
q q
q
q
q
q
q
q
qq
qq
q
qqq
q
q
q
qq
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
qq qq
q
q
q
q
qqq
q
q
q
q q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
q
qq
q q
qq
q
q
qq
q
q
q q
q
q
q
q
q
qq
q
qqq
q
q qq
q
q
q
q q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q q q
q
q
q
q
q
q
qq
q q
q
q
q
qqq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qqq
q
q
qq
q
q q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q
q
q
qq
q
q q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q q
q
q
q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q qq
q
q
q
q
q
q
q
qq
q
qq
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
qq
q
q q
qq q
q q
qq
q
q
q
qqq
q
q
q
q
q
q q
q q
qqq
qq
q
qq
q
q
qq
q
qq qqqq
qqq
q
q
qqq
q
qq q
q qq
q q
q
q
q
q
q
q
q q
q
q
q q qq
q
q qq qqqq
q
q
q
q
qq qq
qq
q q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqqq
q
q
q
q q
qq
q
q
qq qq
qq
q
q
qq qq
q
qq q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq qqq q
qq
qq
q
q
q
q
qq
qq
q
q
qq
q
qq
q
q
q
q q
q q q
q
qq
q q
q
q
q
q
q
q
q
qq q
q
q
q
q q
qq
q
qq qq
qq q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q q
q
qq q
qq
q
q qq q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q q
q
q q q
qq
q
qq
q
q qqqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q
q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq qqq
q
q
q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q
qq q
qq
q
qq
q
q
q
q
q qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qqq
q
qq
q
q
q
q
q q
q
qqq
q
q
q
q q
q
q
q
q qq
q
qq
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
qq qq
q q
qq
q
q q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q q
q
qq
q
q
q
q qqqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
qq
q
qq
q q
q
q
q q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq qq q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
qq
q
q
qq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q
qq
q qq
q
q
qq
q
q
q
q
q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q qq
q
q
q
q
q
q
q
q
qq
qq
q
q
q
q
q
q
q
q
q q
q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q q
q
q
q
q
q
q
q qq q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
qq
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
qqq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
qq
q
q
q
q
q
q q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
qq
q
q
q q
qq
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q qqq
q
qq
q qq
qq qq
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q
q
q
qq
q
q
q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qq
q
q
q
q
qq q
q
qq qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
qq
q
q
q
q
qq
q
q
q
q
q
q
q
q
q q
q
q
qq q
q
q
q
qq
q
q
q
q
q
q
q q
q
q
q
q
q
q q
q
qq
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
qq
q
q
q
q
q
q
q
q
qq
qq
q q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qq q
q
qq
qq
q
q q
q
q
q
q
q
q q
qq
q
qq
q
q
q
q
q
q
q
q
qq
q q
q
q
q
q
q
q
q
q q
q
q
q
q qq q
q
q
q
q q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq qq
q
q
q
q
q
q q
q
q
q
q q qq
q
q
q
q qq q
q
q
q
q qq
q
q
q
q
q
qq
q
q
qqq
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq q
q
qq
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q
q qq
q
q
qq
q
q
q
q q
q
qq
q
q
qq q
q
q
q q
q
q
q
q
q
q qqq
q
q
q
q
q
q
q
qq
q
q
q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
qq
q q
q
q q
q
q q
q
q q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qq
q
q
q
q qq q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q qq
q
q
q
qqqq
qq
q
q
q
qq
q
q
q
q
q
qq
qq
q
q
q
q qq
q
q
q
q
q
q
qq
q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
qq
qq
qqq
q
q q
qq
q
q
q
q
q
q
q
q q
q
q
qq
q
q
q
q q
qq
q
q
q
qqq
q
q
q
q
q
q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
qq q
q qq
q
q
q
qq
qq
q
q
qq
q
q
q
q
q
q
q
q
q
qq
qqq
q
qq
q
q
q
q
q
q q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q q q
qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq
qq
q
q
qq
qq
q
q
q
q q
q
qq
qq
q
q
q
q
q
qq
q q
q
q
q
qq
q
q
q
q q
q
q
q
q
qq
q
qq
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
qq
q
q
q q
q
q
q
q q
q
q
q
q qqqq
qq
q q
qq
q
q
qq
q
q q
qq
q
q
qq
q
q
qq
q q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q
q q
q
q qq
q
q
q
qq
qq
q
q
q
qqq
q
q
q q
q q
q
q
qq
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q qq
qq
q
qqq
q
q qqqqq
q
q
q
q
qq
q
qqq
qq
q
q
q
q
q
q
qqq q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
qq
q
q
q q q
qq
q
q
q q
q
q
qq
qqqq qq
q
q
q
q
qq
q
q q
q
q
q
q
q
q
q
q
q
qq
qq q qqqqq
q q
qqq
q q
q
q
qq
q
qq qqqq
qqqq
q q
q
q
q
q
q
q
q
q
q
q qq
q
q
q
q
q
q
q
qq
q
qq
q
qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q qq
q
q qq
q
q
q
q
q
q
qq qq
q
q
q
qq
q
q
q
q
qq
q
q
q
q q
q q
q
q
q
q
qq
qq
q q
q
q q
q
q qqq
q
q
q
q
q
q
q
qq
qq q
q
q
q
q q
q
qq
q
q
q
q
qq q
q
qq
q
q
q
q
q q
qq
q
q
q q
q
q
q q
q
q
qqq
q
q
q
qq
qqq
qq
q
q
qq
q
q
qqq
q
q
qq
q
qq q
q
q qqq
q
qq
q
q
qq
q
q
q
q
qqq
q
q
q
q
q
q
q
q
q q
qqq q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qqq
q
q
q
q q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q qqq
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
qq q
q
qq
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q q
q
q
qq
q q
q
q
q
q
q
q
qq
q
q
q
qq
q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qqq
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
q q
q
q
qqq
q
q
q
q
q
q
q
q
qq
qq
q q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
qq
qq
q
q
q
q
q
q
q
q
qq qq q
q
q
q q q
q
q
qq
q
q
qq q
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
q
q
q q
q
q
q
q
q
q
q
q
qqq
q
q
q
q q
q
qq q
qq
q
q
q
qq
q
q
q
qq q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
q
q
qq
q
q q
q q
q
qq
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
qqqqq
qqq
qqqqq qq
q
q qqq
q
q qqq
qq
q
q
qq
qq qq
q
q
q q
q q
q
q
qq
q
q
q qqq
q
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q qq
q
q
q q
q
qq
q
q
q
q
q
q
q
q
q
q qq
q
q
qq
q
q
q
qq
q
q
qq
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
q
qqq
q
qq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q qq
q
q
qq
q
q
q
q
q
qq
q q
q
q
q qq
q
q
q
qq
qq
q
q
q
q
q
q
q
q q
q
q
q
q
qq
q
q
q
q
q
q qqq
qq
qq
q
q
qq
q
qqq
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q q
q
qq
qq
q
q q
q
q
q
q
q
qq
q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
qq
q
q qq
q
qq
q
q
q
q
qq q
q
q
q q
q
q
qq
q
q
q
q
q
q
qq q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
qq qqq qq
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
qq q
q
qqq
q
qqq
q
q
qq
q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
qq
q
qq
q
q qq
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq q
qq
q
q
q
qq
q
q
q
q
q
qqq
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qqq
q
q
q
q qq
q
q
q
qq
q
q
q
q qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q q
q
q qq
q
q
q
q qq q
q
qq qq
q
q
q
q qq
q
q
q qq
q
q
q
q qqq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
qq
q
q
q
q
qq
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q q
q qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq q
qq qq q
qq
qq
q q
q
q
qq
q
q
q
q q
q
qq
q
q
qq
qq
q qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q q
q
q
q
q
qq
q
q
q
q
q q
q q
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
qq
qq
q
q
q
qqqq
qq
q
q q
qqqq
q
q
q
q
q
q
qqq
q
q
q
q
q
qqq qqq
q
q
q
q
q
q q
q
q
q q
q
qq
q
q
qq
qq
q
qq
q q
q
q
qq
q
q q
q
qq
q
qq
qq
q
q
q
q q q
q q
q
q
q
q
q
q
q
q
q
q
q
qq
qq
qq
qq
q
q
q
q
qq
qq
q
q
q
q
q
q
qq
q
q
q q
q
q
q
qq
q
q
qq
q qqqq
q
q
q
qq
q
q
qq
qq
q
q
q q
q
q
q
q
q
q
qq
q
q
qq
qqq
q
q
q qq qq
qq qq
q q q
q q
q
q
q
q
q q
q
q q
qq
q
q qqq
q
q
q
q
q q
q
q
q q
q q
q
qq
q q
q
q
qqqq
q
q
q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q q
qqq
q
q
q
q
q
q
q
qq
q
q
q
qqqqq
q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq qq
q
q
q q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
qqq
q
qqq
q
qq
q
q
q
q
q q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q
qq
q
q
q
q
qq
q
q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q
qqq
q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q q
q
q
q
q
q
q qqq q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
qq
q
q q
q
q
q
q
qqq
q
q q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
q
q
q q
q
q
q
q
q
q
q q
q
q
q
q
q
qq
qq
qq
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
qq
qq
q q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
qqq q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q q
q
q
q
q
q
qq
q
q
q
q q
q
q
q
qq
q
q
q
q
q
q
qq
q
qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq qq
q
qq
q
q
q
q
q
q
q
q
q
q
q q q
q
q
q
q
q
q
q
q
q
qq
qq
q
q
q q
q
q q
q
q
q
q
q
qqqq
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qq
q
q
q
qq
qq
qq
q qq qq
q
qqq
q
qq
q
qq q
q
q
qq
q
q
qq
q
qq
q
q
q q q
q qqq
qqq q
qqq
q
qq
q
q
q
q
q
qq qq qqqqq
qq
q qq
q
q q
q
q
q
qq
qq
q
q
q
q
qq
q
q q
q
q
qq qqqq
qqq
qqqq
q
q
q
q
qq
q
q
qq
q
qq
q
qq
qq
qq
q
qqq
q
q
q
q q
q
q
q
q
q q
qqqqqq
qq
q
q
q
qq q
q
q
qq
qqq
q
qq
q
q
q
qq
q
qq
q
qqqq qq q
qq
q
q
q
q
q
q
q
q
q qq
q
qq
q
q q
q
q q
q
q
qqq
q
q
q
q
q q
q
q
qq
qqq
q
q
q
q
q
q
q
q
q q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
qq
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
qqq
q
qq
q q
qq
q
q
q
q
q
q
q
q
q
q
qq
qq
q
qq q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q qq
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q q
q
q
q
q
q
q
q
qq
q
q qq
q q
q
q q
q
qq
q
q
q
qq
q
q
q
q qq
qq
qq
q
q
q q
q
q qq
q q
q
q
q
q
qq
q qq
q
q
q
q
q
qqq
q
q
q
qqq
qq q q
q qq
q
q
q
q
q
q qqq
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
q q
q
q
q q
q
q
q q q
q
q
q
q
qq
q
q
q
qq
q
q
q q
q q q
q
q qq q
qqq
qq
q
qq qqq
q
q
qq
qqqq
q
q
qq
q
q qq q qq
qq
q
qqq
qqqq
qqq q q
qqq qq qq q
q
q
q
qq q
q
q
q
q
q
q
q q
q
q
qq
q q
qq
q
qq
q
qq qq
q
q
qq q
q
q
q
q
q
q
qq
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq q
q
q q
q
q
q
qq
q
q
q qq
q
q q
q qqq
q
q
q
q
qq q
q
q
qq
q
qq
q
q
q q q
q
q
q
q
q
q
q
q qq
q
q
q q
q
q
q
q
q
q
q
qq
q
q
q
qq
qq
qq
q
qqq
q
q q
q qqqq
qqqqq q
q
qq
q
q
qq qq
q qq
q
q
q q
q q
q
qq q
q
qq q
q
q
qqq
q
q
q
q q
qq
q
q
q
q
q q
q
q
q
q
q
q
q q
q
q q
q
q
qq q
q
q
q
q
q q
q
q q
q
q
q
q
q
q q
q
q
q
qq
q
q
q
qq
q
q
qq
q
q
q
q
qq
q
qqqq
q
qq
q
q
q
q qq
q
q
qq
q
q
q
q
q
q
qq
qq
q
q q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q q
qq
q
q
q
q
qq qq
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q q
q
q
qq
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
qq
qq
q
q
q q
q
q
q
q
q
qq
q
q qq
q
qq
q
q
qqqq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
q
qq
q qq
q
qq
q
qq
qqqq
qq
qqqqq
qq
qq
q
q
q
q qq
qq
q
q
qq
q qq
qq
qq q
q
q
q
q
qqq q
qq
q q
q
q
q
qq
q
q
qqqq
q
q
qq
q
qq
qq
q
q
q
q
q
q
qq
q
q
q
q
q
qq
qq q
q
qq
q
qqq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q q
q q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qq
q
q
q q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q q
q
q
qq
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
qq
q
qq
qq
qqq
q
q
q
q q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
qq
qq
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q qq
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q qqq
q
q
qqq
q
q
q
q
q
qq
q
q
q
qq
qqq
q
q
q qq qq
q
q
q
q
q
q
q
qq qq
q
q
q
q
q
qq
q
q
q
qqqqq
qqq
qq
q
qq q
q q
q
q
qq
q
qq
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
qq
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqqq
q
q qqqqqqq
q
q
q
qq
q
q
q
q
q
qq qq
qqq
q
q
q qq q
q
q
q
q qq
qqq qqqq
qqqqq qqqqqq qq
q
qq
q
qq
q
q
q
qq
qq
q
qqqq
q q
qqq
q
qqq
q q
qqqq
qq
q
q
q
q
q
q q
q
qq
qq
q
q
qq qq
qqq
qq qqqqq
q
q
qq q
q
q
qq qqq q
q
q
q q
qqq qq q
qqq
qqq
q
qqq q
q q
q
qq
q
q
q
q q
q
q
q
q
q qq
qq
q
q
qq qq
q q
q
qqq
qq
q
qqq
qq qqqq
q
q
q
q
q
q
qq
q
qq
qq qqq q
q
qq
q qqq qq
qqqqqq
q
q
qq q
qqqqq qqq
q qqq
qqq
qq
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q q
q
q
q
qq
q
q
q
qq
qqq
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
qqqq
qq
q
qq qq
q
q
q
q
q
q
q
q
q
q
q qqqqq q
q
qq
qq
q q
q
q q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
q
qq
qq q
q q
qq
q
qq
q
q
q
q
q
q
qqqq
q
q
q
q
q qq
q
q
q
q
q
q
q
q
q
q
q qq qq
q
q
q q
q
q
q
q
q
q q
q q
q qq
q
q
q
q
q
q
qq qq
q
q
q
q
qqq
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
qqqq
q
q
q
q q
qqqqq
q
q qqq
q
q
q
q
q
q
q
q
qq q
q
q
qqq
q
q
qqqq
qqq
q
q
q
q
qqq
q
q
q q
q
q
q
q q
q
q q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qqq
q
q
q
q
q
qq
q q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
qq
q q
q
q
q
q
q
q
q
qq
q
q
qq
q
q q
q
q
q
q
q
q
qq
q
q q
q
q
q
q q
q
q
q
qq q
q
qq
q
qqq
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
qq
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q q
q
q
q q
q
q
qqq
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qq
qq
q
q q
q
q
q q
q
q
q
q q
q
q
q
qqq
qq
q
qq q
q
qq
q
q
q
q
q
q q
q
q
qq
q qq
q q
q qq
qq
qq
q
qq q
q
q
q
q
qq
q
q
q
q
q
q
q q
q
q
qq
q
q
q
qq
qq
q
q qqq
q qq
qq qq
q
q
q
q
q
q
q
q
q q
qq
q
q
q
q
q q
qq
q
qqqq qq
q
q
qq
q
qqq
q
q
q
q
qq qq
q
q
q
q
q q
q
qq
q
q
q
q
q
qq
qqq
qq qq
q
q
q
qq
qq
q
q
q
qq q
q q
qqq
q
q
qqq
q
q
q
q
q
q
qqqqq
q
q
q
q qq
q
q
q
qq
q
q
q
q q
q
qqqq q
q
qq
q
qq
q
q
q
q
q
qq
q
qq
qq
qq
q
q
q
q
qq qq
q q
qq qq
q
q
q
q
q
q
q
q
qq qqq
q
q
q
qq
qqq
qq
q
qq qqq
q
q
q q
qq
q
q
q
q
q
qq
q
qq
q
q
qq
q q
q
q
q
qq
q
q
qq
q q
q q
q
q
q q
q
q q
q
q
qq
q q
q
q
q
qqq
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
q
q
q qq
q
q
q
q
q
qqq
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
qqqq
q
q
qq qq
q
q
qq
q
qq q
q
q
q
qq
q
q
q
qq
q
q
qq
q q
q
q
q
qq
q
q
q
q
qq
q
q q
qq
q
q
qq qq
q
q
q
qqq q
q
q
q
qq
qqq qqq
q
q
qq q
q
q
qq
q
qq
qqq
q
q
q
q q
qq
q
q
qq
qq
q
q qq
q
q
q
q qqqqqqq
q qqqq
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
qq
q
qqq
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q q
q
q
qq
q
q
q
q q
q q
q
qq
qq qq
qq
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
qq
q
qq q
q
q
qqq
qqqq
q
q
q
q
q
q q
q
qqqq
qqq
q
qq
q
q
qqq
q
q
q
q
q
q
qqq
q
q
q
q
qqq
q
q
q
q q
qq
q
q
qq
q
q
qq
q
q
qq
q
q
q
q
q
qqq
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q qq
q
qq
q q
q
q
q
q
q
q
q
q
q
qq
q
q q q
q
qq q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
qq
q
qq
q
q qqq
q
q
q q
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
qq
q
q
q
q
qqq q
q
q
q
qq
qq
q
q
q
q
q
q
qqqq q
q
q
q
qq
qqq q
q
q
qq
q
q q
qqq
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
1
2
3
4
5
α-convex hull ranking
Exploring the feature space of large time series collections Yahoo web traffic 25
0
10000
20000
30000
40000
50000
0
10000
20000
30000
40000
0
2000
4000
6000
0
1000
2000
3000
4000
5000
0
10000
20000
30000
S10464S7793S8494S1715S7826
2015−02−28
2015−03−01
2015−03−02
2015−03−03
2015−03−04
2015−03−05
2015−03−06
2015−03−07
2015−03−08
2015−03−09
2015−03−10
2015−03−11
2015−03−12
2015−03−13
2015−03−14
2015−03−15
2015−03−16
2015−03−17
2015−03−18
2015−03−19
2015−03−20
2015−03−21
2015−03−22
2015−03−23
2015−03−24
2015−03−25
2015−03−26
2015−03−27
2015−03−28
2015−03−29
2015−03−30
2015−03−31
2015−04−01
date
value
HDR versus α-convex hull
HDR boxplot
−5 0 5
−8−6−4−20246
pc1
pc2
1
2
3
4
5
α-convex hull
−5 0 5
−8−6−4−20246
q
q
qq q
q
q q
q
q
q
qq
q
q
q
q
qq
q
qq
q
q
q
q
q
q
qq
q
q
q
q
qqqq qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
qqq
q
q
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q q
q
qq
q
q
q
q q
q q q
q qq
q
q
qqq
q
q
qq
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
q
q
q
qq
q
q q
q
q
q
q
q
q q
q
q qq q qq
q
q
q
q
q
q qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q
q
qqq
q
qq
q
qq
qqqqq q
q
qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq
qq
qq q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qqq
q
qqqqq q
q
q
q
q
q
q qqq
q
q
q
q
q
qq
qq
q
qq
qqqq
qq
q
q
q
q q
q
q
q qq
q
qqq
qqqq qqq
q
q
q
qq
q
q
q
q
q
q
q
qq
qqq
q
q
q
q
q
q
q
q
qq
q qqq
q
q q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q qqq
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q qq
q
q q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q qq
q q
q q
q
q
q qq
q qq
q
q
q
q
q q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q q
q q
q
qq
q
q
q
q
q
q
q
q
q
qq
qq
q
q
q
q qq
q
q
qq
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qqqq
q q
q
q
q
q
q
q
qq
qq
q
qqq
q
q
q
qq
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
qq qq
q
q
q
q
qqq
q
q
q
q q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
q
qq
q q
qq
q
q
qq
q
q
q q
q
q
q
q
q
qq
q
qqq
q
q qq
q
q
q
q q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q q q
q
q
q
q
q
q
qq
q q
q
q
q
qqq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qqq
q
q
qq
q
q q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q
q
q
qq
q
q q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q q
q
q
q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q qq
q
q
q
q
q
q
q
qq
q
qqqq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
qq
q
q q
qq q
q q
qq
q
q
q
qqq
q
q
q
q
q
q q
q q
qqq
qq
q
qq
q
q
qq
q
qq qqqq
qqq
q
q
qqq
q
qq q
q qq
q q
q
q
q
q
q
q
q q
q
q
q q qq
q
q qq qqqq
q
q
q
q
qq qq
qq
q q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqqq
q
q
q
q q
qq
q
q
qq qq
qq
q
q
qq qq
q
qq q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq qqq q
qq
qq
q
q
q
q
qq
qq
q
q
qq
q
qq
q
q
q
q q
q q q
q
qq
q q
q
q
q
q
q
q
q
qq q
q
q
q
q q
qq
q
qq qq
qq q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q q
q
qq q
qq
q
q qq q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q q
q
q q q
qq
q
qq
q
q qqqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q
q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq qqq
q
q
q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q
qq q
qq
q
qq
q
q
q
q
q qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qqq
q
qq
q
q
q
q
q q
q
qqq
q
q
q
q q
q
q
q
q qq
q
qq
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
qq qq
q q
qq
q
q q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q q
q
qq
q
q
q
q qqqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
qq
q
qq
q q
q
q
q q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq qq q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
qq
q
q
qq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q
qq
q qq
q
q
qq
q
q
q
q
q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q qq
q
q
q
q
q
q
q
q
qq
qq
q
q
q
q
q
q
q
q
q q
q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q q
q
q
q
q
q
q
q qq q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
qq
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
qqq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
qq
q
q
q
q
q
q q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
qq
q
q
q q
qq
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q qqq
q
qq
q qq
qq qq
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q
q
q
qq
q
q
q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qq
q
q
q
q
qq q
q
qq qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
qq
q
q
q
q
qq
q
q
q
q
q
q
q
q
q q
q
q
qq q
q
q
q
qq
q
q
q
q
q
q
q q
q
q
q
q
q
q q
q
qq
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
qq
q
q
q
q
q
q
q
q
qq
qq
q q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qq q
q
qq
qq
q
q q
q
q
q
q
q
q q
qq
q
qq
q
q
q
q
q
q
q
q
qq
q q
q
q
q
q
q
q
q
q q
q
q
q
q qq q
q
q
q
q q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq qq
q
q
q
q
q
q q
q
q
q
q q qq
q
q
q
q qq q
q
q
q
q qq
q
q
q
q
q
qq
q
q
qqq
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq q
q
qq
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q
q qq
q
q
qq
q
q
q
q q
q
qq
q
q
qq q
q
q
q q
q
q
q
q
q
q qqq
q
q
q
q
q
q
q
qq
q
q
q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
qq
q q
q
q q
q
q q
q
q q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qq
q
q
q
q qq q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q qq
q
q
q
qqqq
qq
q
q
q
qq
q
q
q
q
q
qq
qq
q
q
q
q qq
q
q
q
q
q
q
qq
q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
qq
qq
qqq
q
q q
qq
q
q
q
q
q
q
q
q q
q
q
qq
q
q
q
q q
qq
q
q
q
qqq
q
q
q
q
q
q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
qq q
q qq
q
q
q
qq
qq
q
q
qq
q
q
q
q
q
q
q
q
q
qq
qqq
q
qq
q
q
q
q
q
q q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q q q
qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq
qq
q
q
qq
qq
q
q
q
q q
q
qq
qq
q
q
q
q
q
qq
q q
q
q
q
qq
q
q
q
q q
q
q
q
q
qq
q
qq
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
qq
q
q
q q
q
q
q
q q
q
q
q
q qqqq
qq
q q
qq
q
q
qq
q
q q
qq
q
q
qq
q
q
qq
q q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q
q q
q
q qq
q
q
q
qq
qq
q
q
q
qqq
q
q
q q
q q
q
q
qq
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q qq
qq
q
qqq
q
q qqqqq
q
q
q
q
qq
q
qqq
qq
q
q
q
q
q
q
qqq q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
qq
q
q
q q q
qq
q
q
q q
q
q
qq
qqqq qq
q
q
q
q
qq
q
q q
q
q
q
q
q
q
q
q
q
qq
qq q qqqqq
q q
qqq
q q
q
q
qq
q
qq qqqq
qqqq
q q
q
q
q
q
q
q
q
q
q
q qq
q
q
q
q
q
q
q
qq
q
qq
q
qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q qq
q
q qq
q
q
q
q
q
q
qq qq
q
q
q
qq
q
q
q
q
qq
q
q
q
q q
q q
q
q
q
q
qq
qq
q q
q
q q
q
q qqq
q
q
q
q
q
q
q
qq
qq q
q
q
q
q q
q
qq
q
q
q
q
qq q
q
qq
q
q
q
q
q q
qq
q
q
q q
q
q
q q
q
q
qqq
q
q
q
qq
qqq
qq
q
q
qq
q
q
qqq
q
q
qq
q
qq q
q
q qqq
q
qq
q
q
qq
q
q
q
q
qqq
q
q
q
q
q
q
q
q
q q
qqq q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qqq
q
q
q
q q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q qqq
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
qq q
q
qq
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q q
q
q
qq
q q
q
q
q
q
q
q
qq
q
q
q
qq
q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qqq
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
q q
q
q
qqq
q
q
q
q
q
q
q
q
qq
qq
q q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
qq
qq
q
q
q
q
q
q
q
q
qq qq q
q
q
q q q
q
q
qq
q
q
qq q
q
q
q
q
q
qq
q
q
q
q
q
q
qq
q
q
q
q q
q
q
q
q
q
q
q
q
qqq
q
q
q
q q
q
qq q
qq
q
q
q
qq
q
q
q
qq q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
qq
q
q
q
q
qq
q
q q
q q
q
qq
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
qqqqq
qqq
qqqqq qq
q
q qqq
q
q qqq
qq
q
q
qq
qq qq
q
q
q q
q q
q
q
qq
q
q
q qqq
q
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q qq
q
q
q q
q
qq
q
q
q
q
q
q
q
q
q
q qq
q
q
qq
q
q
q
qq
q
q
qq
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q q
q q
q
q
q
q
q
q
q
q
qqq
q
qq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q qq
q
q
qq
q
q
q
q
q
qq
q q
q
q
q qq
q
q
q
qq
qq
q
q
q
q
q
q
q
q q
q
q
q
q
qq
q
q
q
q
q
q qqq
qq
qq
q
q
qq
q
qqq
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q q
q
qq
qq
q
q q
q
q
q
q
q
qq
q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
qq
q
q qq
q
qq
q
q
q
q
qq q
q
q
q q
q
q
qq
q
q
q
q
q
q
qq q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
qq qqq qq
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
qq q
q
qqq
q
qqq
q
q
qq
q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
qq
q
qqq
q qq
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq q
qq
q
q
q
qq
q
q
q
q
q
qqq
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qqq
q
q
q
q qq
q
q
q
qq
q
q
q
q qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q q
q
q qq
q
q
q
q qq q
q
qq qq
q
q
q
q qq
q
q
q qq
q
q
q
q qqq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
qq
q
q
q
q
qq
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q q
q qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq q
qq qq q
qq
qq
q q
q
q
qq
q
q
q
q q
q
qq
q
q
qq
qq
q qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q q
q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q q
q
q
q
q
qq
q
q
q
q
q q
q q
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q
qq
qq
q
q
q
qqqq
qq
q
q q
qqqq
q
q
q
q
q
q
qqq
q
q
q
q
q
qqq qqq
q
q
q
q
q
q q
q
q
q q
q
qq
q
q
qq
qq
q
qq
q q
q
q
qq
q
q q
q
qq
q
qq
qq
q
q
q
q q q
q q
q
q
q
q
q
q
q
q
q
q
q
qq
qq
qq
qq
q
q
q
q
qq
qq
q
q
q
q
q
q
qq
q
q
q q
q
q
q
qq
q
q
qq
q qqqq
q
q
q
qq
q
q
qq
qq
q
q
q q
q
q
q
q
q
q
qq
q
q
qq
qqq
q
q
q qq qq
qq qq
q q q
q q
q
q
q
q
q q
q
q q
qq
q
q qqq
q
q
q
q
q q
q
q
q q
q q
q
qq
q q
q
q
qqqq
q
q
q
q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q q
qqq
q
q
q
q
q
q
q
qq
q
q
q
qqqqq
q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq qq
q
q
q q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
qqqq
qqq
q
qq
q
q
q
q
q q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q
qq
q
q
q
q
qq
q
q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
qq
q
qq
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qqq
q
qqq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q q
q
q
q
q
q
q qqq q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
qq
q
q q
q
q
q
q
qqq
q
q q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
q
q
q q
q
q
q
q
q
q
q q
q
q
q
q
q
qq
qq
qq
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
qq
qq
q q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
qqq
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q q
q
q
q
q
q
qq
q
q
q
q q
q
q
q
qq
q
q
q
q
q
q
qq
q
qq
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
qq qq
q
qq
q
q
q
q
q
q
q
q
q
q
q q q
q
q
q
q
q
q
q
q
q
qq
qq
q
q
q q
q
q q
q
q
q
q
q
qqqq
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qq
q
q
q
qq
qq
qq
q qq qq
q
qqq
q
qq
q
qq q
q
q
qq
q
q
qq
q
qq
q
q
q q q
q qqq
qqq q
qqq
q
qq
q
q
q
q
q
qq qq qqqqq
qq
q qq
q
q q
q
q
q
qq
qq
q
q
q
q
qq
q
q q
q
q
qq qqqq
qqqqqqq
q
q
q
q
qq
q
q
qq
q
qq
q
qq
qq
qq
q
qqq
q
q
q
q q
q
q
q
q
q q
qqqqqq
qq
q
q
q
qq q
q
q
qq
qqq
q
qq
q
q
q
qq
q
qq
q
qqqq qq q
qq
q
q
q
q
q
q
q
q
q qq
q
qq
q
q q
q
q q
q
q
qqq
q
q
q
q
q q
q
q
qq
qqq
q
q
q
q
q
q
q
q
q q
q
q
qq
q
q
q
q
q
q
q
q q
q
q
qq
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
qqq
q
qq
q q
qq
q
q
q
q
q
q
q
q
q
q
qq
qq
q
qq q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q qq
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq q
q
q
q
q q
q
q
q
q
q
q
q
qq
q
q qq
q q
q
q q
q
qq
q
q
q
qq
q
q
q
q qq
qq
qq
q
q
q q
q
q qq
q q
q
q
q
q
qq
q qq
q
q
q
q
q
qqq
q
q
q
qqq
qq q q
q qq
q
q
q
q
q
q qqq
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q
q q
q
q
q q
q
q
q q q
q
q
q
q
qq
q
q
q
qq
q
q
q q
q q q
q
q qq q
qqq
qq
q
qq qqq
q
q
qq
qqqq
q
q
qq
q
q qq q qq
qq
q
qqq
qqqq
qqq q q
qqq qq qq q
q
q
q
qq q
q
q
q
q
q
q
q q
q
q
qq
q q
qq
q
qq
q
qq qq
q
q
qq q
q
q
q
q
q
q
qq
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq q
q
q q
q
q
q
qq
q
q
q qq
q
q q
q qq
q
q
q
q
q
qq q
q
q
qq
q
qq
q
q
q q q
q
q
q
q
q
q
q
q qq
q
q
q q
q
q
q
q
q
q
q
qq
q
q
q
qq
qq
qq
q
qqq
q
q q
q qqqq
qqqqq q
q
qq
q
q
qq qq
q qq
q
q
q q
q q
q
qq q
q
qq q
q
q
qqq
q
q
q
q q
qq
q
q
q
q
q q
q
q
q
q
q
q
q q
q
q q
q
q
qq q
q
q
q
q
q q
q
q q
q
q
q
q
q
q q
q
q
q
qq
q
q
q
qq
q
q
qq
q
q
q
q
qq
q
qqqq
q
qq
q
q
q
q qq
q
q
qq
q
q
q
q
q
q
qq
qq
q
q q
q
q
q q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q
q q
qq
q
q
q
q
qq qq
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q q
q
q
qq
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
qq
qq
q
q
q q
q
q
q
q
q
qq
q
q qq
q
qq
q
q
qqqq
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
qq
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q q
q
q
q
q
q
q q
q
q
q
q q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
q
qq
q qq
q
qq
q
qq
qqqq
qq
qqqqq
qq
qq
q
q
q
q qq
qq
q
q
qq
q qq
qq
qq q
q
q
q
q
qqq q
qq
q q
q
q
q
qq
q
q
qqqq
q
q
qq
q
qq
qq
q
q
q
q
q
q
qq
q
q
q
q
q
qq
qq q
q
qq
q
qqq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
qq
q
q
q q
q q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qq
q
q
q q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q q
q
q
qq
q
q
q
q
q
q
q
q
q q
q q
q
q
q
q
qq
q
qq
qq
qqq
q
q
q
q q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
qq
qq
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q q q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q qq
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q qqq
q
q
qqq
q
q
q
q
q
qq
q
q
q
qq
qqq
q
q
q qq qq
q
q
q
q
q
q
q
qq qq
q
q
q
q
q
qq
q
q
q
qqqqq
qqq
qq
q
qq q
q q
q
q
qq
q
qq
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
qq
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqqq
q
q qqqqqqq
q
q
q
qq
q
q
q
q
q
qq qq
qqq
q
q
q qq q
q
q
q
q qq
qqq qqqq
qqqqq qqqqqq qq
q
qq
q
qq
q
q
q
qq
qq
q
qqqq
q q
qqq
q
qqq
q q
qqqq
qq
q
q
q
q
q
q q
q
qq
qq
q
q
qq qq
qqq
qq qqqqq
q
q
qq q
q
q
qq qqq q
q
q
q q
qqq qq q
qqq
qqq
q
qqq q
q q
q
qq
q
q
q
q q
q
q
q
q
q qq
qq
q
q
qq qq
q q
q
qqq
qq
q
qqqqq qqqq
q
q
q
q
q
q
qq
q
qq
qq qqq q
q
qq
q qqq qq
qqqqqq
q
q
qq q
qqqqq qqq
q qqq
qqq
qq
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q q
q
q
q
qq
q
q
q
qq
qqq
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
qqqq
qq
q
qq qq
q
q
q
q
q
q
q
q
q
q
q qqqqq q
q
qq
qq
q q
q
q q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
q
qq
qq q
q q
qq
q
qq
q
q
q
q
q
q
qqqq
q
q
q
q
q qq
q
q
q
q
q
q
q
q
q
q
q qq qq
q
q
q q
q
q
q
q
q
q q
q q
q qq
q
q
q
q
q
q
qq qq
q
q
q
q
qqq
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
qqqq
q
q
q
q q
qqqqq
q
q qqq
q
q
q
q
q
q
q
q
qq q
q
q
qqq
q
q
qq
qq
qqq
q
q
q
q
qqq
q
q
q q
q
q
q
q q
q
q q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qqq
q
q
q
q
q
qq
q q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
qq
q q
q
q
q
q
q
q
q
qq
q
q
qq
q
q q
q
q
q
q
q
q
qq
q
q q
q
q
q
q q
q
q
q
qq q
q
qq
q
qqq
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
qq
q qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q q
q
q
q q
q
q
qqq
q
q
q
q
q
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q qq
qq
q
q q
q
q
q q
q
q
q
q q
q
q
q
qqq
qq
q
qq q
q
qq
q
q
q
q
q
q q
q
q
qq
q qq
q q
q qq
qq
qq
q
qq q
q
q
q
q
qq
q
q
q
q
q
q
q q
q
q
qq
q
q
q
qq
qq
q
q qqq
q qq
qq qq
q
q
q
q
q
q
q
q
q q
qq
q
q
q
q
q q
qq
q
qqqq qq
q
q
qq
q
qqq
q
q
q
q
qq qq
q
q
q
q
q q
q
qq
q
q
q
q
q
qq
qqq
qq qq
q
q
q
qq
qq
q
q
q
qq q
q q
qqq
q
q
qqq
q
q
q
q
q
q
qqqqq
q
q
q
q qq
q
q
q
qq
q
q
q
q q
q
qqqq q
q
qq
q
qq
q
q
q
q
q
qq
q
qq
qq
qq
q
q
q
q
qq qq
q q
qq qq
q
q
q
q
q
q
q
q
qq qqq
q
q
q
qq
qqq
qq
q
qq qqq
q
q
q q
qq
q
q
q
q
q
qq
q
qq
q
q
qq
q q
q
q
q
qq
q
q
qq
q q
q q
q
q
q q
q
q q
q
q
qq
q q
q
q
q
qqq
qq
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qqq
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
qq
q
q
q
q qq
q
q
q
q
q
qqq
q
q
q
q
q
q
qq
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
qqqq
q
q
qq qq
q
q
qq
q
qq q
q
q
q
qq
q
q
q
qq
q
q
qq
q q
q
q
q
qq
q
q
q
q
qq
q
q q
qq
q
q
qq qq
q
q
q
qqq q
q
q
q
qq
qqq qqq
q
q
qq q
q
q
qq
q
qq
qqq
q
q
q
q q
qq
q
q
qq
qq
q
q qq
q
q
q
q qqqqqqq
q qqqq
q
q
q
q
qq
q
q
q
q
q
qq
q
q
q
qq
q
qqq
q
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q q
q
q
qq
q
q
q
q q
q q
q
qq
qq qq
qq
q
q
q
q
q
q
q
qq
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
qq
q
q
q
qq
q
q
q
q
qq
q
qq q
q
q
qqq
qqqq
q
q
q
q
q
q q
q
qqqq
qqq
q
qq
q
q
qqq
q
q
q
q
q
q
qqq
q
q
q
q
qqq
q
q
q
q q
qq
q
q
qq
q
q
qq
q
q
qq
q
q
q
q
q
qqq
q
q
q
q
q
qqq
q
q
q
q
q
q
q
q qq
q
qq
q q
q
q
q
q
q
q
q
q
q
qq
q
q q q
q
qq q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
qq
q
qq
q
q qqq
q
q
q q
q
q
q
qq q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
qq
q
q
q
q
qqq q
q
q
q
qq
qq
q
q
q
q
q
q
qqqq q
q
q
q
qq
qqq q
q
q
qq
q
q q
qqq
q
q
q
q
qq
q
q
q
q
q
q
q
qq
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
q
1
2
3
4
5
Exploring the feature space of large time series collections Yahoo web traffic 26
Top 5 anomalous time series
HDR
0
10000
20000
30000
40000
0
2000
4000
6000
0
10000
20000
30000
40000
50000
0
10000
20000
30000
40000
0
1000
2000
3000
4000
5000
S7793S8494S10464S7833S1715
2015−02−28
2015−03−01
2015−03−02
2015−03−03
2015−03−04
2015−03−05
2015−03−06
2015−03−07
2015−03−08
2015−03−09
2015−03−10
2015−03−11
2015−03−12
2015−03−13
2015−03−14
2015−03−15
2015−03−16
2015−03−17
2015−03−18
2015−03−19
2015−03−20
2015−03−21
2015−03−22
2015−03−23
2015−03−24
2015−03−25
2015−03−26
2015−03−27
2015−03−28
2015−03−29
2015−03−30
2015−03−31
2015−04−01
date
value
α-convex hull
0
10000
20000
30000
40000
50000
0
10000
20000
30000
40000
0
2000
4000
6000
0
1000
2000
3000
4000
5000
0
10000
20000
30000
S10464S7793S8494S1715S7826
2015−02−28
2015−03−01
2015−03−02
2015−03−03
2015−03−04
2015−03−05
2015−03−06
2015−03−07
2015−03−08
2015−03−09
2015−03−10
2015−03−11
2015−03−12
2015−03−13
2015−03−14
2015−03−15
2015−03−16
2015−03−17
2015−03−18
2015−03−19
2015−03−20
2015−03−21
2015−03−22
2015−03−23
2015−03−24
2015−03−25
2015−03−26
2015−03−27
2015−03−28
2015−03−29
2015−03−30
2015−03−31
2015−04−01
date
value
Exploring the feature space of large time series collections Yahoo web traffic 27
Outline
1 M3 competition data
2 Yahoo web traffic
3 What next?
Exploring the feature space of large time series collections What next? 28
What next?
Develop a more comprehensive set of features
that are reliable measures and fast to compute.
e.g., for finance data.
Consider application to functional data from
other contexts (not time series).
Is PCA the right approach? Perhaps we should
use multidimensional scaling? Or something
else?
Should we use more than 2 PC dimensions?
Develop dynamic and interactive visualization
tools.
Make methods available in an R package.
Some of the methods are already available in the
anomalous package for R on github.
Exploring the feature space of large time series collections What next? 29
What next?
Develop a more comprehensive set of features
that are reliable measures and fast to compute.
e.g., for finance data.
Consider application to functional data from
other contexts (not time series).
Is PCA the right approach? Perhaps we should
use multidimensional scaling? Or something
else?
Should we use more than 2 PC dimensions?
Develop dynamic and interactive visualization
tools.
Make methods available in an R package.
Some of the methods are already available in the
anomalous package for R on github.
Exploring the feature space of large time series collections What next? 29
What next?
Develop a more comprehensive set of features
that are reliable measures and fast to compute.
e.g., for finance data.
Consider application to functional data from
other contexts (not time series).
Is PCA the right approach? Perhaps we should
use multidimensional scaling? Or something
else?
Should we use more than 2 PC dimensions?
Develop dynamic and interactive visualization
tools.
Make methods available in an R package.
Some of the methods are already available in the
anomalous package for R on github.
Exploring the feature space of large time series collections What next? 29
What next?
Develop a more comprehensive set of features
that are reliable measures and fast to compute.
e.g., for finance data.
Consider application to functional data from
other contexts (not time series).
Is PCA the right approach? Perhaps we should
use multidimensional scaling? Or something
else?
Should we use more than 2 PC dimensions?
Develop dynamic and interactive visualization
tools.
Make methods available in an R package.
Some of the methods are already available in the
anomalous package for R on github.
Exploring the feature space of large time series collections What next? 29
What next?
Develop a more comprehensive set of features
that are reliable measures and fast to compute.
e.g., for finance data.
Consider application to functional data from
other contexts (not time series).
Is PCA the right approach? Perhaps we should
use multidimensional scaling? Or something
else?
Should we use more than 2 PC dimensions?
Develop dynamic and interactive visualization
tools.
Make methods available in an R package.
Some of the methods are already available in the
anomalous package for R on github.
Exploring the feature space of large time series collections What next? 29
What next?
Develop a more comprehensive set of features
that are reliable measures and fast to compute.
e.g., for finance data.
Consider application to functional data from
other contexts (not time series).
Is PCA the right approach? Perhaps we should
use multidimensional scaling? Or something
else?
Should we use more than 2 PC dimensions?
Develop dynamic and interactive visualization
tools.
Make methods available in an R package.
Some of the methods are already available in the
anomalous package for R on github.
Exploring the feature space of large time series collections What next? 29
What next?
Develop a more comprehensive set of features
that are reliable measures and fast to compute.
e.g., for finance data.
Consider application to functional data from
other contexts (not time series).
Is PCA the right approach? Perhaps we should
use multidimensional scaling? Or something
else?
Should we use more than 2 PC dimensions?
Develop dynamic and interactive visualization
tools.
Make methods available in an R package.
Some of the methods are already available in the
anomalous package for R on github.
Exploring the feature space of large time series collections What next? 29
Further information
Exploring the feature space of large time series collections What next? 30
¯ Papers: robjhyndman.com
¯ Code: github.com/robjhyndman
¯ Email: Rob.Hyndman@monash.edu

More Related Content

PDF
Coherent mortality forecasting using functional time series models
PDF
Demographic forecasting
PDF
[PDF] Visualizing and discovering non trivial patterns in large time-series d...
PDF
Algebra 2 Section 4-2
PDF
R tools for hierarchical time series
PDF
Anomaly Detection in Sequences of Short Text Using Iterative Language Models
PDF
A walk through the intersection between machine learning and mechanistic mode...
PDF
Spillover Dynamics for Systemic Risk Measurement Using Spatial Financial Time...
Coherent mortality forecasting using functional time series models
Demographic forecasting
[PDF] Visualizing and discovering non trivial patterns in large time-series d...
Algebra 2 Section 4-2
R tools for hierarchical time series
Anomaly Detection in Sequences of Short Text Using Iterative Language Models
A walk through the intersection between machine learning and mechanistic mode...
Spillover Dynamics for Systemic Risk Measurement Using Spatial Financial Time...

Similar to Exploring the feature space of large collections of time series (20)

PDF
Master's Thesis defence presentation
PDF
Cambridge 2014 Complexity, tails and trends
PDF
Exploring temporal graph data with Python: 
a study on tensor decomposition o...
PDF
Time-Frequency Representation of Microseismic Signals using the SST
PDF
Contextual Bandit Survey
ODP
An Application of Uncertainty Quantification to MPM
PDF
Visualization of big time series data
PDF
Slides_Neural Networks for Time Series Prediction
PDF
Funciones básicas, representación y Operaciones de convolucion
PPT
Wavelet Multi-resolution Analysis of High Frequency FX Rates
PDF
multiscale_tutorial.pdf
PPT
PDF
residual-spectra
PDF
PDF
Module v sp
PDF
Beyond function approximators for batch mode reinforcement learning: rebuildi...
PPT
Hprec7.3
PDF
2013.06.18 Time Series Analysis Workshop ..Applications in Physiology, Climat...
PDF
2013.06.17 Time Series Analysis Workshop ..Applications in Physiology, Climat...
Master's Thesis defence presentation
Cambridge 2014 Complexity, tails and trends
Exploring temporal graph data with Python: 
a study on tensor decomposition o...
Time-Frequency Representation of Microseismic Signals using the SST
Contextual Bandit Survey
An Application of Uncertainty Quantification to MPM
Visualization of big time series data
Slides_Neural Networks for Time Series Prediction
Funciones básicas, representación y Operaciones de convolucion
Wavelet Multi-resolution Analysis of High Frequency FX Rates
multiscale_tutorial.pdf
residual-spectra
Module v sp
Beyond function approximators for batch mode reinforcement learning: rebuildi...
Hprec7.3
2013.06.18 Time Series Analysis Workshop ..Applications in Physiology, Climat...
2013.06.17 Time Series Analysis Workshop ..Applications in Physiology, Climat...
Ad

More from Rob Hyndman (16)

PDF
Automatic algorithms for time series forecasting
PDF
Exploring the boundaries of predictability
PDF
MEFM: An R package for long-term probabilistic forecasting of electricity demand
PDF
Probabilistic forecasting of long-term peak electricity demand
PDF
Visualization and forecasting of big time series data
PDF
Academia sinica jan-2015
PDF
Automatic time series forecasting
PDF
Automatic time series forecasting
PDF
Forecasting Hierarchical Time Series
PDF
Forecasting using R
PDF
SimpleR: tips, tricks & tools
PDF
Forecasting without forecasters
PDF
Ysc2013
PDF
Forecasting electricity demand distributions using a semiparametric additive ...
PDF
Advances in automatic time series forecasting
PDF
FPP 1. Getting started
Automatic algorithms for time series forecasting
Exploring the boundaries of predictability
MEFM: An R package for long-term probabilistic forecasting of electricity demand
Probabilistic forecasting of long-term peak electricity demand
Visualization and forecasting of big time series data
Academia sinica jan-2015
Automatic time series forecasting
Automatic time series forecasting
Forecasting Hierarchical Time Series
Forecasting using R
SimpleR: tips, tricks & tools
Forecasting without forecasters
Ysc2013
Forecasting electricity demand distributions using a semiparametric additive ...
Advances in automatic time series forecasting
FPP 1. Getting started
Ad

Recently uploaded (20)

PPTX
DISORDERS OF THE LIVER, GALLBLADDER AND PANCREASE (1).pptx
PDF
Clinical guidelines as a resource for EBP(1).pdf
PDF
Foundation of Data Science unit number two notes
PPTX
Introduction to machine learning and Linear Models
PPTX
ALIMENTARY AND BILIARY CONDITIONS 3-1.pptx
PPTX
MODULE 8 - DISASTER risk PREPAREDNESS.pptx
PPTX
Introduction to Firewall Analytics - Interfirewall and Transfirewall.pptx
PPTX
Microsoft-Fabric-Unifying-Analytics-for-the-Modern-Enterprise Solution.pptx
PDF
168300704-gasification-ppt.pdfhghhhsjsjhsuxush
PPTX
Database Infoormation System (DBIS).pptx
PPTX
01_intro xxxxxxxxxxfffffffffffaaaaaaaaaaafg
PDF
Fluorescence-microscope_Botany_detailed content
PDF
.pdf is not working space design for the following data for the following dat...
PPT
ISS -ESG Data flows What is ESG and HowHow
PPTX
advance b rammar.pptxfdgdfgdfsgdfgsdgfdfgdfgsdfgdfgdfg
PPTX
mbdjdhjjodule 5-1 rhfhhfjtjjhafbrhfnfbbfnb
PDF
BF and FI - Blockchain, fintech and Financial Innovation Lesson 2.pdf
PPTX
1_Introduction to advance data techniques.pptx
PPTX
Acceptance and paychological effects of mandatory extra coach I classes.pptx
DISORDERS OF THE LIVER, GALLBLADDER AND PANCREASE (1).pptx
Clinical guidelines as a resource for EBP(1).pdf
Foundation of Data Science unit number two notes
Introduction to machine learning and Linear Models
ALIMENTARY AND BILIARY CONDITIONS 3-1.pptx
MODULE 8 - DISASTER risk PREPAREDNESS.pptx
Introduction to Firewall Analytics - Interfirewall and Transfirewall.pptx
Microsoft-Fabric-Unifying-Analytics-for-the-Modern-Enterprise Solution.pptx
168300704-gasification-ppt.pdfhghhhsjsjhsuxush
Database Infoormation System (DBIS).pptx
01_intro xxxxxxxxxxfffffffffffaaaaaaaaaaafg
Fluorescence-microscope_Botany_detailed content
.pdf is not working space design for the following data for the following dat...
ISS -ESG Data flows What is ESG and HowHow
advance b rammar.pptxfdgdfgdfsgdfgsdgfdfgdfgsdfgdfgdfg
mbdjdhjjodule 5-1 rhfhhfjtjjhafbrhfnfbbfnb
BF and FI - Blockchain, fintech and Financial Innovation Lesson 2.pdf
1_Introduction to advance data techniques.pptx
Acceptance and paychological effects of mandatory extra coach I classes.pptx

Exploring the feature space of large collections of time series

  • 1. Exploring the feature space of large time series collections 1 Exploring the feature space of large collections of time series Rob J Hyndman
  • 2. Exploring the feature space of large time series collections 1 Exploring the feature space of large collections of time series Rob J Hyndman with Earo Wang, Nikolay Laptev Yanfei Kang, Kate Smith-Miles
  • 3. Outline 1 M3 competition data 2 Yahoo web traffic 3 What next? Exploring the feature space of large time series collections M3 competition data 2
  • 4. M3 forecasting competition Exploring the feature space of large time series collections M3 competition data 3
  • 5. M3 forecasting competition Exploring the feature space of large time series collections M3 competition data 3
  • 6. M3 forecasting competition “The M3-Competition is a final attempt by the authors to settle the accuracy issue of various time series methods. . . The extension involves the inclusion of more methods/ researchers (in particular in the areas of neural networks and expert systems) and more series.” Makridakis & Hibon, IJF 2000 3003 series All data from business, demography, finance and economics. Series length between 14 and 126. Either non-seasonal, monthly or quarterly. All time series positive. Exploring the feature space of large time series collections M3 competition data 4
  • 7. M3 forecasting competition Exploring the feature space of large time series collections M3 competition data 5 q
  • 8. Key idea Examples for time series lag correlation size and direction of trend strength of seasonality timing of peak seasonality spectral entropy Called “features” or “characteristics” in the machine learning literature. Exploring the feature space of large time series collections M3 competition data 6 John W Tukey Cognostics Computer-produced diagnostics (Tukey and Tukey, 1985).
  • 9. Key idea Examples for time series lag correlation size and direction of trend strength of seasonality timing of peak seasonality spectral entropy Called “features” or “characteristics” in the machine learning literature. Exploring the feature space of large time series collections M3 competition data 6 John W Tukey Cognostics Computer-produced diagnostics (Tukey and Tukey, 1985).
  • 10. Key idea Examples for time series lag correlation size and direction of trend strength of seasonality timing of peak seasonality spectral entropy Called “features” or “characteristics” in the machine learning literature. Exploring the feature space of large time series collections M3 competition data 6 John W Tukey Cognostics Computer-produced diagnostics (Tukey and Tukey, 1985).
  • 11. An STL decomposition Quarterly visitor nights: Mornington Peninsula Yt = St + Tt + Rt St is periodic with mean 0 5.06.07.0 data −0.50.5 seasonal 5.86.16.4 trend −0.40.0 2000 2005 2010 remainder timeExploring the feature space of large time series collections M3 competition data 7
  • 12. Candidate features STL decomposition Yt = St + Tt + Rt Seasonal period Strength of seasonality: 1 − Var(Rt) Var(Yt−Tt) Strength of trend: 1 − Var(Rt) Var(Yt−St) Spectral entropy: H = − π −π fy(λ) log fy(λ)dλ, where fy(λ) is spectral density of Yt. Low values of H suggest a time series that is easier to forecast (more signal). Autocorrelations: r1, r2, r3, . . . Optimal Box-Cox transformation parameter λ Exploring the feature space of large time series collections M3 competition data 8
  • 13. Candidate features STL decomposition Yt = St + Tt + Rt Seasonal period Strength of seasonality: 1 − Var(Rt) Var(Yt−Tt) Strength of trend: 1 − Var(Rt) Var(Yt−St) Spectral entropy: H = − π −π fy(λ) log fy(λ)dλ, where fy(λ) is spectral density of Yt. Low values of H suggest a time series that is easier to forecast (more signal). Autocorrelations: r1, r2, r3, . . . Optimal Box-Cox transformation parameter λ Exploring the feature space of large time series collections M3 competition data 8
  • 14. Candidate features STL decomposition Yt = St + Tt + Rt Seasonal period Strength of seasonality: 1 − Var(Rt) Var(Yt−Tt) Strength of trend: 1 − Var(Rt) Var(Yt−St) Spectral entropy: H = − π −π fy(λ) log fy(λ)dλ, where fy(λ) is spectral density of Yt. Low values of H suggest a time series that is easier to forecast (more signal). Autocorrelations: r1, r2, r3, . . . Optimal Box-Cox transformation parameter λ Exploring the feature space of large time series collections M3 competition data 8
  • 15. Candidate features STL decomposition Yt = St + Tt + Rt Seasonal period Strength of seasonality: 1 − Var(Rt) Var(Yt−Tt) Strength of trend: 1 − Var(Rt) Var(Yt−St) Spectral entropy: H = − π −π fy(λ) log fy(λ)dλ, where fy(λ) is spectral density of Yt. Low values of H suggest a time series that is easier to forecast (more signal). Autocorrelations: r1, r2, r3, . . . Optimal Box-Cox transformation parameter λ Exploring the feature space of large time series collections M3 competition data 8
  • 16. Candidate features STL decomposition Yt = St + Tt + Rt Seasonal period Strength of seasonality: 1 − Var(Rt) Var(Yt−Tt) Strength of trend: 1 − Var(Rt) Var(Yt−St) Spectral entropy: H = − π −π fy(λ) log fy(λ)dλ, where fy(λ) is spectral density of Yt. Low values of H suggest a time series that is easier to forecast (more signal). Autocorrelations: r1, r2, r3, . . . Optimal Box-Cox transformation parameter λ Exploring the feature space of large time series collections M3 competition data 8
  • 17. Candidate features STL decomposition Yt = St + Tt + Rt Seasonal period Strength of seasonality: 1 − Var(Rt) Var(Yt−Tt) Strength of trend: 1 − Var(Rt) Var(Yt−St) Spectral entropy: H = − π −π fy(λ) log fy(λ)dλ, where fy(λ) is spectral density of Yt. Low values of H suggest a time series that is easier to forecast (more signal). Autocorrelations: r1, r2, r3, . . . Optimal Box-Cox transformation parameter λ Exploring the feature space of large time series collections M3 competition data 8
  • 18. Candidate features STL decomposition Yt = St + Tt + Rt Seasonal period Strength of seasonality: 1 − Var(Rt) Var(Yt−Tt) Strength of trend: 1 − Var(Rt) Var(Yt−St) Spectral entropy: H = − π −π fy(λ) log fy(λ)dλ, where fy(λ) is spectral density of Yt. Low values of H suggest a time series that is easier to forecast (more signal). Autocorrelations: r1, r2, r3, . . . Optimal Box-Cox transformation parameter λ Exploring the feature space of large time series collections M3 competition data 8
  • 19. Dimension reduction for time series Exploring the feature space of large time series collections M3 competition data 9 q
  • 20. Dimension reduction for time series Exploring the feature space of large time series collections M3 competition data 9 q SpecEntr 0.0 0.4 0.8 2 6 10 0.0 0.4 0.8 0.50.9 0.00.6 Trend Season 0.00.6 28 Freq ACF −0.40.6 0.5 0.7 0.9 0.00.6 0.0 0.4 0.8 −0.4 0.2 0.8 Lambda Feature calculation
  • 21. Dimension reduction for time series Exploring the feature space of large time series collections M3 competition data 9 q SpecEntr 0.0 0.4 0.8 2 6 10 0.0 0.4 0.8 0.50.9 0.00.6 Trend Season 0.00.6 28 Freq ACF −0.40.6 0.5 0.7 0.9 0.00.6 0.0 0.4 0.8 −0.4 0.2 0.8 Lambda q q q q q q qq qq q q q q q q q q q q q q q qq q qq q qqq q q qq q q qq q qq qq q qqq q q q q qq qqqq q q qq q q qq q q q q q q qq q qq q q q q q q q q q q qq q q q q q qq q q qq qq q q qq q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q q q qq qqq q q q q q qq q qqq qqqq q qq q q q q qq q qq qq qqqq q q q q q q q q q qqq q q q q q q qq q q q q q qq qq q q q q q q q q qqq q q q q q q q q q q q q q q q q q q q qq q qq q q q qqqq q qqq q qqq qq q qq qq qqqqqqqqqq q q q q q q qq q qq q qq q q q q qq q q qqq qq qqq q q qqqqqq q q q q qq qq qqqq q q qq q qq q q qqqq q q q q qqq qq q q qqqqq qqq qq q q q q q q q q q q q qq q qqq q qqq qqq q qq q q q q q q qqqqq q q qq q q qq q q qq q q q q q q q q q q qq q q q qqqqq q q q q q q q q q q q q q q q q q q qq q q q q qq q q qqq qqqq q q q qqq q q q qq q q q q qqqqq q qqqq q q q qq q q qq qq q q qq q q q q qqqq q q q q q q q q q q qq q qq q q q qq q q q q q q q q q q q qq q q qq q q q q qqq q q q qq qq q q q q q qq qqq q qq q q qqq q q q qq q q qq qqqqqqq qq q q q q qqqqqq qqq q q q q q q q q q q q q q q qqq q q q q q qq qq q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q qq q q q qq q q q qqq q q q qq q q q q q q q qq q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q qq q q qq q qq q q qq q qqq q q q q q q q q q q q qq q q qq q q q q qq q q q q qq q q qq q q qq q q q qq qqq q qq q q qq q q q q qqqq q qq q q q qqqq qq q q q qqq qqqq q q qqq q qqq q q qq q qqq q q qq q q qqq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q q q qqqq q qqq q qqq q qqq q q q q qq q qq q q q q q q q qq q q q q q q q qq q qq q q q q q q q q q q q q qq q qq qq q q q q q q q q q q q q q q q qq qq q q qqqq q q qqq q q q q q q q q qq q qqqq q qq qq q q q qq qq q q q q q q q qq q q q q q q qq q q q q q q qq q qqq q q q q q q q q q q q q q q q q qq q qq q q q q q q q qq q qq q q q q q qq q q q q q qq q q q q q q q q qq qq q q qq q q qq q q q q q qq q q qq q q q q q q q q q q q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq qqq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq qq qq q q q q q qq q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q qqq q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq qq q q qq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q qq q qq q q q qq q q q q qq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q qq q q q q q q q qq q qq q q q q qq q q q q qq q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q qq q q q q q q q q q q q qq qqq q qq q q q q q q q qq q q q q qq q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q qq q qq q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q q qqq q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q qq q q q q q q q qq q q q q qq q q q q q q qqqq q q q qqqq q q qq qq q q q q q q q qqqq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q q q q qq qqq qqqq qqqq q qq qqqqqq q qqqq q q q q q q q q q q qqq q q q q q q q q q q qqqq q q q q q q q qq q q qq q q qq q qq q q q q qq qqq q q q q q q q q qq q qq q q q qqq q q q q q q qq q q q q q q q qq qqq q q qq q q q q q qq q qq q q q q q q q qq q q q q q q q q q qqq q q q q q q q q qq q q q q q q q q q q qq q q q q q q q q qqqq q q q q q qq q q qq q q q q q q q qq q q q q q q q q q q q q qq q q q q qq q q q q q q q q q q q q q q q q q qq q q qq qq q q q q qq q q qqqq q q q q q q q q q q q q q q q q q q q q q qqq q qq q q q q q q q q q q q q q q q q q q q q qq q qq qq q qq q qq q q qqq q qq qq q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q qq q q qq q q qqq q qq q q qq qq q q qqqq q q q q q q q q q q qqq q q q q q qqqq qqqq q q q q q q q q q q q q q q q q qq q q q q q q qq q qq q q q q qq q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q qq qq q q q qq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q qq q qq q q qq q q qqq q q q q q qq q qq q q q q q q q q q q q qq q q q qq q q qq q q q q q qq qq qqq qqq q q q q q q qqqqqqqq q q qqqqqqqqqqqqqqqqq q qq q q q q qqqqq qqq q q qqqqq q q q qqqq q q q q q q q qq q q q qqqqq q qqqqq q qq qqq qqqq q qq q qq q q q qq q qq −3 −2 −1 0 1 2 3 −2 0 2 4 PC1 PC2 Feature calculation Principal component decomposition
  • 22. Feature space of M3 data Exploring the feature space of large time series collections M3 competition data 10 q q q q q q qq qq q q q q q q q q q q q q q qq q qq q qqq q q qq q q qq q qq qq q qqq q q q q qq qqqq q q qq q q qq q q q q q q qq q qq q q q q q q q q q q qq q q q q q qq q q qq qq q q qq q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q q q qq qqq q q q q q qq q qqq qqqq q qq q q q q qq q qq qq qqqq q q q q q q q q q qqq q q q q q q qq q q q q q qq qq q q q q q qq q qqq q q q q q q q q q q q q q q q q q q q qq q qq q q q qqqq q qqq q qqq qq q qq qq qqqqqqqqqq q q q q q q qq q qq q qq q q q q qq q q qqq qq qqq q q qqqqqq q q q q qq qq qqqq q q qq q qq q q qqq q q q q q qqq qq q q qqqqq qqq qq q q q q q q q q q q q qq q qqq q qqq qqq q qq q q q q q q qqqqq q q qq q q qq q q qq q q q q q q q q q q qq q q q qqqqq q q q q q q q q q q q q q q q q q q qq q q q q qq q q qqq qqqq q q q qqq q q q qq q q q q qqqqq q qqqq q q q qq q q qq qq q q qq q q q q qq qq q q q q q q q q q q qq q qq q q q qq q q q q q q q q q q q qq q q qq q q q q qqq q q q qq qq q q q q q qq qqq q qq q q qqq q q q qq q q qq qqqqqq q qq q q q q qqqqqq qqq q q q q q q q q q q q q q q qqq q q q q q qq qq q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q qqq q q qq q q q qqq q q q qq q q q q q q q qq q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q qq q q qq q qq q q qq q qqq q q q q q q q q q q q qq q q qq q q q q qq q q q q qq q q qq q q qq q q q qq qqq q qq q q qq q q q q qqqq q qq q q q qqqq qq q q q qqq qqqq q q qqq q qqq q q qq q qqq q q qq q q qqq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q q q qqqq q qqq q qqq q qqq q q q q qq q qq q q q q q q q qq q q q q q q q qq q qq q q q q q q q q q q q q qq q qq qq q q q q q q q q q q q q q q q qq qq q q qqqq q q qqq q q q q q q q q qq q qqqq q qq qq q q q qq qq q q q q q q q qq q q q q q q qq q q q q q q qq q qqq q q q q q q q q q q q q q q q q qq q qq q q q q q q q qq q qq q q q q q qq q q q q q qq q q q q q q q q qq qq q q qq q q qq q q q q q qq q q qq q q q q q q q q q q q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq qqq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq qq qq q q q q q qq q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q qqq q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq qq q q qq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q qq q qq q q q qq q q q q qq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q qq q q q q q q q qq q qq q q q q qq q q q q qq q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q qq q q q q q q q q q q q qq qqq q qq q q q q q q q qq q q q q qq q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q qq q qq q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q q qqq q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q qq q q q q q q q qq q q q q qq q q q q q q qqqq q q q qqqq q q qq qq q q q q q q q qqqq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q q q q qq qqq qqqq qqqq q qq qqqqqq q qqqq q q q q q q q q q q qqq q q q q q q q q q q qqqq q q q q q q q qq q q qq q q qq q qq q q q q qq qqq q q q q q q q q qq q qq q q q qqq q q q q q q qq q q q q q q q qq qqq q q qq q q q q q qq q qq q q q q q q q qq q q q q q q q q q qqq q q q q q q q q qq q q q q q q q q q q qq q q q q q q q q qqqq q q q q q qq q q qq q q q q q q q qq q q q q q q q q q q q q qq q q q q qq q q q q q q q q q q q q q q q q q qq q q qq qq q q q q qq q q qqqq q q q q q q q q q q q q q q q q q q q q q qqq q qq q q q q q q q q q q q q q q q q q q q q qq q qq qq q qq q qq q q qq q q qq qq q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q qq q q qq q q qqq q qq q q qq qq q q qqqq q q q q q q q q q q qqq q q q q q qqqq qqqq q q q q q q q q q q q q q q q q qq q q q q q q qq q qq q q q q qq q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q qq qq q q q qq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q qq q qq q q qq q q qqq q q q q q qq q qq q q q q q q q q q q q qq q q q qq q q qq q q q q q qq qq qqq qqq q q q q q q qqqqqqqq q q qqqqqqqqqqqqqqqqq q qq q q q q qqqqq qqq q q qqqqq q q q qqqq q q q q q q q qq q q q qqqqq q qqqqq q qq qqq qqqq q qq q qq q q q qq q qq −3 −2 −1 0 1 2 3 −2 0 2 4 PC1 PC2 First two PCs explain 68% of variation.
  • 23. Feature space of M3 data Exploring the feature space of large time series collections M3 competition data 10 q q q q q q qq qq q q q q q q q q q q q q q qqq qq q qqq q q qq q q qq q qq qq q qqq q q q q qq qqqq q q qq q q qq q q q q q q qq q qq q q q q q q q q q q qq q q q q q qq q q qq qq q q qq q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q q q qq q q q q q qq qqq q q q q q qq q qqq qqqq q qq q q q q qq q qq qq qqqq q q q q q q q q q qqq q q q q q q qq q q q q q qq qq q q q q q qq q qqq q q q q q q q q q q q q q q q q q q q qq q qq q q q qqqq q qqq q qqq qq q qq qq qqqqqqqqqq q q q qq q qq q qq q qq q q q q qq q q qqq qq qqq q q qqqqqq q q q q qq qq qqqq q q qq q qq q q qqq q q q q q qqq qq q q qqqqq qqq qq q q q q q q q q q q q qq q qqq q qqq qqq q qq q q q q q q qqqqq q q qq q q qq q q qq q q q q q q q q q q qq q q q qqqqq q q q q q q q q q q q q q q q q q q qq q q q q qq q q qqq qqqq q q q qqq q q q qq q q q q qqqqq q qqqq q q q qq q q qq qq q q qq qq q q qq qq q q q q q q q q q q qq q qq q q q qq q q q q q q q q q q q qq q q qq q q q q qqqq q q qq qq q q q q q qq qqq q qq q q qqq q q q qq q q qq qqqqqq q qq q q q q qqqqqq qqq q q q q q q q q q q q q q q qqq q q q q q qq qq q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q qqq q q qq q q q qqq q q q qq q q q q q q q qq q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q qq q q qq q qq q q qq q qqq q q q q q q q q q q q qq q q qq q q q q qq q q q q qq q q qq q q qq q q q qq qqqq qq q q qq q q q q qqqq q qq q q q qqqq qq q q q qqq qqqq q q qqq q qqq q q qq q qqq q q qq q q qqq q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q q q qqqq q qqq q qqq q qqq q q q q qqq qq q q q q q q q qq q q q q q q q qq q qq q q q q q q q q q q q q qq q qq qq q q q q q q q q q q q q q q q qq qq q q qqqq q q qqq q q q q q q q q qq q qqqq q qq qq q q q qq qq q q q q q q q qq q q q q q q qq q q qq q q qq q qqq q q q q q q q q q q q q q q q q qq q qq q q q q q q q qq q qq q q q q q qq q q q q q qq q q q q q q q q qq qq q q qq q q qq q q q q q qq q q qq q q q q q q q q q q q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq qqq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq qq qq q q q q q qq q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q qq q q q q qq q q q q q q q q q q q q q q q qq q q qqq q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qqq qq q q qq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q qq q qq q q q qq q q q q qq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q qq q q q q q q q qq q qq q q q q qq q q q q qq q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q qq q q q q q q q q q q q qq qqq q qq q q q q q q q qq q q q q qq q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q qq q qq q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q q qqq q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q qq q q q q q q q q q q q q q q q q q qq qqq q q q q q q q qq q q q q qq q q q q q q qqqq q q q qqqq q q qq qq q q q q q q q qqqq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q q q q qq qqq qqqqqqqq q qq qqqqqq q qqqq q q q q q q q q q q qqq q q q q q q q q q q qqqq q q q q q q q qq q q qq q q qq q qq q q q q qq qqq q q q q q q q q qq q qq q q q qqq qq q q q q qq q q q q q q q qq qqq q q qq q q q q q qq q qq q q q q q q q qq q q q q q q q q q qqq q q q q q q q q qq q q q q q q q q q q qq q q q q q q q q qqqq q q q q q qq q q qq q q q q q q q qq q q q q q q q q q q q q qq q q q q qq q q q q q q q q q q q q q q q q q qq q q qq qq q q q q qq q q qqqq q q q q q q q q q q q q q q q q q q q q q qqq q qq q q q q q q q q q q q q q q q q q q q q qq q qq qq q qq q qq q q qqq q qq qq q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q qq q q qq q q qqq q qq q q qq qq q q qqqq q q q q q q q q q q qqq q q q q q qqqq qqqq q q q q q q q q q q q q q q q q qq q q q q q q qq q qq q q q q qq q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q qqq qq q q q qq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q qq q qq q q qq q q qqq q q q q q qq q qq q q q q q q q q q q q qq q q q qq q q qq q q q q q qq qq qqq qqq q q q q q qqqqqqqqq q q qqqqqqqqqqqqqqqqq qqq q q q q qqqqq qqq q q qqqqq q q q qqqq q q q q q q q qq q q q qqqqq q qqqqq q qq qqq qqqq q qq q qq q q q qq q qq −3 −2 −1 0 1 2 3 −2 0 2 4 PC1 PC2 3 6 9 12 value Freq
  • 24. Feature space of M3 data Exploring the feature space of large time series collections M3 competition data 10 q q q q q q qq qq q q q q q q q q q q q q q qqq qq q qqq q q qq q q qq q qq qq q qqq q q q q qq qqqq q q qq q q qq q q q q q q qq q qq q q q q q q q q q q qq q q q q q qq q q qq qq q q qq q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q q q qq q q q q q qq qqq q q q q q qq q qqq qqqq q qq q q q q qq q qq qq qqqq q q q q q q q q q qqq q q q q q q qq q q q q q qq qq q q q q q qq q qqq q q q q q q q q q q q q q q q q q q q qq q qq q q q qqqq q qqq q qqq qq q qq qq qqqqqqqqqq q q q qq q qq q qq q qq q q q q qq q q qqq qq qqq q q qqqqqq q q q q qq qqqqqq q q qq q qq q q qqq q q q q q qqq qq q q qqqqq qqq qq q q q q q q q q q q q qq q qqq q qqq qqq q qq q q q q q q qqqqq q q qq q q qq q q qq q q q q q q q q q q qq q q q qqqqq q q q q q q q q q q q q q q q q q q qq q q q q qq q q qqq qqqq q q q qqq q q q qq q q q q qqqqq q qqqq q q q qq q q qq qq q q qq qq q q qq qq q q q q q q q q q q qq q qq q q q qq q q q q q q q q q q q qq q q qq q q q q qqqq q q qq qq q q q q q qq qqq q qq q q qqq q q q qq q q qq qqqqqq q qq q q q q qqqqqq qqq q q q q q q q q q q q q q q qqq q q q q q qq qq q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q qqq q q qq q q q qqq q q q qq q q q q q q q qq q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q qq q q qq q qq q q qq q qqq q q q q q q q q q q q qq q q qq q q q q qq q q q q qq q q qq q q qq q q q qq qqqq qq q q qq q q q q qqqq q qq q q q qqqq qq q q q qqq qqqq q q qqq q qqq q q qq q qqq q q qq q q qqq q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q q q qqqq q qqq q qqq q qqq q q q q qqq qq q q q q q q q qq q q q q q q q qq q qq q q q q q q q q q q q q qq q qq qq q q q q q q q q q q q q q q q qq qq q q qqqq q q qqq q q q q q q q q qq q qqqq q qq qq q q q qq qq q q q q q q q qq q q q q q q qq q q qq q q qq q qqq q q q q q q q q q q q q q q q q qq q qq q q q q q q q qq q qq q q q q q qq q q q q q qq q q q q q q q q qq qq q q qq q q qq q q q q q qq q q qq q q q q q q q q q q q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq qqq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq qq qq q q q q q qq q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q qq q q q q qq q q q q q q q q q q q q q q q qq q q qqq q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qqq qq q q qq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q qq q qq q q q qq q q q q qq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q qq q q q q q q q qq q qq q q q q qq q q q q qq q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q qq q q q q q q q q q q q qq qqq q qq q q q q q q q qq q q q q qq q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q qq q qq q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q q qqq q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q qq q q q q q q q q q q q q q q q q q qq qqq q q q q q q q qq q q q q qq q q q q q q qqqq q q q qqqq q q qq qqq q q q q q q qqqq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q q q q qq qqq qqqqqqqq q qq qqqqqq q qqqq q q q q q q q q q q qqq q q q q q q q q q q qqqq q q q q q q q qq q q qq q q qq q qq q q q q qq qqq q q q q q q q q qq q qq q q q qqq qq q q q q qq q q q q q q q qq qqq q q qq q q q q q qq q qq q q q q q q q qq q q q q q q q q q qqq q q q q q q q q qq q q q q q q q q q q qq q q q q q q q q qqqq q q q q q qq q q qq q q q q q q q qq q q q q q q q q q q q q qq q q q q qq q q q q q q q q q q q q q q q q q qq q q qq qq q q q q qq q q qqqq q q q q q q q q q q q q q q q q q q q q q qqq q qq q q q q q q q q q q q q q q q q q q q q qq q qq qq q qq q qq q q qqq q qq qq q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q qq q q qq q q qqq q qq q q qq qq q q qqqq q q q q q q q q q q qqq q q q q q qqqq qqqq q q q q q q q q q q q q q q q q qq q q q q q q qq q qq q q q q qq q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q qqqqq q q q qq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q qq q qq q q qq q q qqq q q q q q qq q qq q q q q q q q q q q q qq q q q qq q q qq q q q q q qq qq qqq qqq q q q q q qqqqqqqqq q q qqqqqqqqqqqqqqqqq qqq q q q q qqqqq qqq q q qqqqq q q q qqqq q q q q q q q qq q q q qqqqq q qqqqq q qq qqq qqqq q qq q qq q q q qq q qq −3 −2 −1 0 1 2 3 −2 0 2 4 PC1 PC2 0.00 0.25 0.50 0.75 value Season
  • 25. Feature space of M3 data Exploring the feature space of large time series collections M3 competition data 10 q q q q q q qq qq q q q q q q q q q q q q q qqq qq q qqq q q qq q q qq q qq qq q qqq q q q q qq qqqq q q qq q q qq q q q q q q qq q qq q q q q q q q q q q qq q q q q q qq q q qq qq q q qq q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q q q qq q q q q q qq qqq q q q q q qq q qqq qqqq q qq q q q q qq q qq qq qqqq q q q q q q q q q qqq q q q q q q qq q q q q q qq qq q q q q q qq q qqq q q q q q q q q q q q q q q q q q q q qq q qq q q q qqqq q qqq q qqq qq q qq qq qqqqqqqqqq q q q qq q qq q qq q qq q q q q qq q q qqq qq qqq q q qqqqqq q q q q qq qqqqqq q q qq q qq q q qqq q q q q q qqq qq q q qqqqq qqq qq q q q q q q q q q q q qq q qqq q qqq qqq q qq q q q q q q qqqqq q q qq q q qq q q qq q q q q q q q q q q qq q q q qqqqq q q q q q q q q q q q q q q q q q q qq q q q q qq q q qqq qqqq q q q qqq q q q qq q q q q qqqqq q qqqq q q q qq q q qq qq q q qq qq q q qq qq q q q q q q q q q q qq q qq q q q qq q q q q q q q q q q q qq q q qq q q q q qqqq q q qq qq q q q q q qq qqq q qq q q qqq q q q qq q q qq qqqqqq q qq q q q q qqqqqq qqq q q q q q q q q q q q q q q qqq q q q q q qq qq q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q qqq q q qq q q q qqq q q q qq q q q q q q q qq q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q qq q q qq q qq q q qq q qqq q q q q q q q q q q q qq q q qq q q q q qq q q q q qq q q qq q q qq q q q qq qqqq qq q q qq q q q q qqqq q qq q q q qqqq qq q q q qqq qqqq q q qqq q qqq q q qq q qqq q q qq q q qqq q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q q q qqqq q qqq q qqq q qqq q q q q qqq qq q q q q q q q qq q q q q q q q qq q qq q q q q q q q q q q q q qq q qq qq q q q q q q q q q q q q q q q qq qq q q qqqq q q qqq q q q q q q q q qq q qqqq q qq qq q q q qq qq q q q q q q q qq q q q q q q qq q q qq q q qq q qqq q q q q q q q q q q q q q q q q qq q qq q q q q q q q qq q qq q q q q q qq q q q q q qq q q q q q q q q qq qq q q qq q q qq q q q q q qq q q qq q q q q q q q q q q q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq qqq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq qq qq q q q q q qq q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q qq q q q q qq q q q q q q q q q q q q q q q qq q q qqq q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qqq qq q q qq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q qq q qq q q q qq q q q q qq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q qq q q q q q q q qq q qq q q q q qq q q q q qq q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q qq q q q q q q q q q q q qq qqq q qq q q q q q q q qq q q q q qq q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q qq q qq q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q q qqq q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q qq q q q q q q q q q q q q q q q q q qq qqq q q q q q q q qq q q q q qq q q q q q q qqqq q q q qqqq q q qq qqq q q q q q q qqqq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q q q q qq qqq qqqqqqqq q qq qqqqqq q qqqq q q q q q q q q q q qqq q q q q q q q q q q qqqq q q q q q q q qq q q qq q q qq q qq q q q q qq qqq q q q q q q q q qq q qq q q q qqq qq q q q q qq q q q q q q q qq qqq q q qq q q q q q qq q qq q q q q q q q qq q q q q q q q q q qqq q q q q q q q q qq q q q q q q q q q q qq q q q q q q q q qqqq q q q q q qq q q qq q q q q q q q qq q q q q q q q q q q q q qq q q q q qq q q q q q q q q q q q q q q q q q qq q q qq qq q q q q qq q q qqqq q q q q q q q q q q q q q q q q q q q q q qqq q qq q q q q q q q q q q q q q q q q q q q q qq q qq qq q qq q qq q q qqq q qq qq q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q qq q q qq q q qqq q qq q q qq qq q q qqqq q q q q q q q q q q qqq q q q q q qqqq qqqq q q q q q q q q q q q q q q q q qq q q q q q q qq q qq q q q q qq q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q qqqqq q q q qq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q qq q qq q q qq q q qqq q q q q q qq q qq q q q q q q q q q q q qq q q q qq q q qq q q q q q qq qq qqq qqq q q q q q qqqqqqqqq q q qqqqqqqqqqqqqqqqq qqq q q q q qqqqq qqq q q qqqqq q q q qqqq q q q q q q q qq q q q qqqqq q qqqqq q qq qqq qqqq q qq q qq q q q qq q qq −3 −2 −1 0 1 2 3 −2 0 2 4 PC1 PC2 0.25 0.50 0.75 value Trend
  • 26. Feature space of M3 data Exploring the feature space of large time series collections M3 competition data 10 q q q q q q qq qq q q q q q q q q q q q q q qqq qq q qqq q q qq q q qq q qq qq q qqq q q q q qq qqqq q q qq q q qq q q q q q q qq q qq q q q q q q q q q q qq q q q q q qq q q qq qq q q qq q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q q q qq q q q q q qq qqq q q q q q qq q qqq qqqq q qq q q q q qq q qq qq qqqq q q q q q q q q q qqq q q q q q q qq q q q q q qq qq q q q q q qq q qqq q q q q q q q q q q q q q q q q q q q qq q qq q q q qqqq q qqq q qqq qq q qq qq qqqqqqqqqq q q q qq q qq q qq q qq q q q q qq q q qqq qq qqq q q qqqqqq q q q q qq qq qqqq q q qq q qq q q qqq q q q q q qqq qq q q qqqqq qqq qq q q q q q q q q q q q qq q qqq q qqq qqq q qq q q q q q q qqqqq q q qq q q qq q q qq q q q q q q q q q q qq q q q qqqqq q q q q q q q q q q q q q q q q q q qq q q q q qq q q qqq qqqq q q q qqq q q q qq q q q q qqqqq q qqqq q q q qq q q qq qq q q qq qq q q qq qq q q q q q q q q q q qq q qq q q q qq q q q q q q q q q q q qq q q qq q q q q qqqq q q qq qq q q q q q qq qqq q qq q q qqq q q q qq q q qq qqqqqq q qq q q q q qqqqqq qqq q q q q q q q q q q q q q q qqq q q q q q qq qq q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q qqq q q qq q q q qqq q q q qq q q q q q q q qq q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q qq q q qq q qq q q qq q qqq q q q q q q q q q q q qq q q qq q q q q qq q q q q qq q q qq q q qq q q q qq qqqq qq q q qq q q q q qqqq q qq q q q qqqq qq q q q qqq qqqq q q qqq q qqq q q qq q qqq q q qq q q qqq q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q q q qqqq q qqq q qqq q qqq q q q q qqq qq q q q q q q q qq q q q q q q q qq q qq q q q q q q q q q q q q qq q qq qq q q q q q q q q q q q q q q q qq qq q q qqqq q q qqq q q q q q q q q qq q qqqq q qq qq q q q qq qq q q q q q q q qq q q q q q q qq q q qq q q qq q qqq q q q q q q q q q q q q q q q q qq q qq q q q q q q q qq q qq q q q q q qq q q q q q qq q q q q q q q q qq qq q q qq q q qq q q q q q qq q q qq q q q q q q q q q q q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq qqq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq qq qq q q q q q qq q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q qq q q q q qq q q q q q q q q q q q q q q q qq q q qqq q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qqq qq q q qq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q qq q qq q q q qq q q q q qq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q qq q q q q q q q qq q qq q q q q qq q q q q qq q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q qq q q q q q q q q q q q qq qqq q qq q q q q q q q qq q q q q qq q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q qq q qq q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q q qqq q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q qq q q q q q q q q q q q q q q q q q qq qqq q q q q q q q qq q q q q qq q q q q q q qqqq q q q qqqq q q qq qq q q q q q q q qqqq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q q q q qq qqq qqqqqqqq q qq qqqqqq q qqqq q q q q q q q q q q qqq q q q q q q q q q q qqqq q q q q q q q qq q q qq q q qq q qq q q q q qq qqq q q q q q q q q qq q qq q q q qqq qq q q q q qq q q q q q q q qq qqq q q qq q q q q q qq q qq q q q q q q q qq q q q q q q q q q qqq q q q q q q q q qq q q q q q q q q q q qq q q q q q q q q qqqq q q q q q qq q q qq q q q q q q q qq q q q q q q q q q q q q qq q q q q qq q q q q q q q q q q q q q q q q q qq q q qq qq q q q q qq q q qqqq q q q q q q q q q q q q q q q q q q q q q qqq q qq q q q q q q q q q q q q q q q q q q q q qq q qq qq q qq q qq q q qqq q qq qq q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q qq q q qq q q qqq q qq q q qq qq q q qqqq q q q q q q q q q q qqq q q q q q qqqq qqqq q q q q q q q q q q q q q q q q qq q q q q q q qq q qq q q q q qq q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q qqq qq q q q qq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q qq q qq q q qq q q qqq q q q q q qq q qq q q q q q q q q q q q qq q q q qq q q qq q q q q q qq qq qqq qqq q q q q q qqqqqqqqq q q qqqqqqqqqqqqqqqqq qqq q q q q qqqqq qqq q q qqqqq q q q qqqq q q q q q q q qq q q q qqqqq q qqqqq q qq qqq qqqq q qq q qq q q q qq q qq −3 −2 −1 0 1 2 3 −2 0 2 4 PC1 PC2 0.0 0.5 value ACF
  • 27. Feature space of M3 data Exploring the feature space of large time series collections M3 competition data 10 q q q q q q qq qq q q q q q q q q q q q q q qqq qq q qqq q q qq q q qq q qq qq q qqq q q q q qq qqqq q q qq q q qq q q q q q q qq q qq q q q q q q q q q q qq q q q q q qq q q qq qq q q qq q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q q q qq q q q q q qq qqq q q q q q qq q qqq qqqq q qq q q q q qq q qq qq qqqq q q q q q q q q q qqq q q q q q q qq q q q q q qq qq q q q q q qq q qqq q q q q q q q q q q q q q q q q q q q qq q qq q q q qqqq q qqq q qqq qq q qq qq qqqqqqqqqq q q q qq q qq q qq q qq q q q q qq q q qqq qq qqq q q qqqqqq q q q q qq qq qqqq q q qq q qq q q qqq q q q q q qqq qq q q qqqqq qqq qq q q q q q q q q q q q qq q qqq q qqq qqq q qq q q q q q q qqqqq q q qq q q qq q q qq q q q q q q q q q q qq q q q qqqqq q q q q q q q q q q q q q q q q q q qq q q q q qq q q qqq qqqq q q q qqq q q q qq q q q q qqqqq q qqqq q q q qq q q qq qq q q qq qq q q qq qq q q q q q q q q q q qq q qq q q q qq q q q q q q q q q q q qq q q qq q q q q qqqq q q qq qq q q q q q qq qqq q qq q q qqq q q q qq q q qq qqqqqq q qq q q q q qqqqqq qqq q q q q q q q q q q q q q q qqq q q q q q qq qq q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q qqq q q qq q q q qqq q q q qq q q q q q q q qq q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q qq q q qq q qq q q qq q qqq q q q q q q q q q q q qq q q qq q q q q qq q q q q qq q q qq q q qq q q q qq qqqq qq q q qq q q q q qqqq q qq q q q qqqq qq q q q qqq qqqq q q qqq q qqq q q qq q qqq q q qq q q qqq q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q q q qqqq q qqq q qqq q qqq q q q q qqq qq q q q q q q q qq q q q q q q q qq q qq q q q q q q q q q q q q qq q qq qq q q q q q q q q q q q q q q q qq qq q q qqqq q q qqq q q q q q q q q qq q qqqq q qq qq q q q qq qq q q q q q q q qq q q q q q q qq q q qq q q qq q qqq q q q q q q q q q q q q q q q q qq q qq q q q q q q q qq q qq q q q q q qq q q q q q qq q q q q q q q q qq qq q q qq q q qq q q q q q qq q q qq q q q q q q q q q q q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq qqq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq qq qq q q q q q qq q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q qq q q q q qq q q q q q q q q q q q q q q q qq q q qqq q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qqq qq q q qq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q qq q qq q q q qq q q q q qq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q qq q q q q q q q qq q qq q q q q qq q q q q qq q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q qq q q q q q q q q q q q qq qqq q qq q q q q q q q qq q q q q qq q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q qq q qq q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q q qqq q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q qq q q q q q q q q q q q q q q q q q qq qqq q q q q q q q qq q q q q qq q q q q q q qqqq q q q qqqq q q qq qq q q q q q q q qqqq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q q q q qq qqq qqqqqqqq q qq qqqqqq q qqqq q q q q q q q q q q qqq q q q q q q q q q q qqqq q q q q q q q qq q q qq q q qq q qq q q q q qq qqq q q q q q q q q qq q qq q q q qqq qq q q q q qq q q q q q q q qq qqq q q qq q q q q q qq q qq q q q q q q q qq q q q q q q q q q qqq q q q q q q q q qq q q q q q q q q q q qq q q q q q q q q qqqq q q q q q qq q q qq q q q q q q q qq q q q q q q q q q q q q qq q q q q qq q q q q q q q q q q q q q q q q q qq q q qq qq q q q q qq q q qqqq q q q q q q q q q q q q q q q q q q q q q qqq q qq q q q q q q q q q q q q q q q q q q q q qq q qq qq q qq q qq q q qqq q qq qq q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q qq q q qq q q qqq q qq q q qq qq q q qqqq q q q q q q q q q q qqq q q q q q qqqq qqqq q q q q q q q q q q q q q q q q qq q q q q q q qq q qq q q q q qq q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q qqq qq q q q qq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q qq q qq q q qq q q qqq q q q q q qq q qq q q q q q q q q q q q qq q q q qq q q qq q q q q q qq qq qqq qqq q q q q q qqqqqqqqq q q qqqqqqqqqqqqqqqqq qqq q q q q qqqqq qqq q q qqqqq q q q qqqq q q q q q q q qq q q q qqqqq q qqqqq q qq qqq qqqq q qq q qq q q q qq q qq −3 −2 −1 0 1 2 3 −2 0 2 4 PC1 PC2 0.5 0.6 0.7 0.8 0.9 value SpecEntr
  • 28. Feature space of M3 data Exploring the feature space of large time series collections M3 competition data 10 q q q q q q qq qq q q q q q q q q q q q q q qqq qq q qqq q q qq q q qq q qq qq q qqq q q q q qq qqqq q q qq q q qq q q q q q q qq q qq q q q q q q q q q q qq q q q q q qq q q qq qq q q qq q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q q q qq q q q q q qq qqq q q q q q qq q qqq qqqq q qq q q q q qq q qq qq qqqq q q q q q q q q q qqq q q q q q q qq q q q q q qq qq q q q q q qq q qqq q q q q q q q q q q q q q q q q q q q qq q qq q q q qqqq q qqq q qqq qq q qq qq qqqqqqqqqq q q q qq q qq q qq q qq q q q q qq q q qqq qq qqq q q qqqqqq q q q q qq qqqqqq q q qq q qq q q qqq q q q q q qqq qq q q qqqqq qqq qq q q q q q q q q q q q qq q qqq q qqq qqq q qq q q q q q q qqqqq q q qq q q qq q q qq q q q q q q q q q q qq q q q qqqqq q q q q q q q q q q q q q q q q q q qq q q q q qq q q qqq qqqq q q q qqq q q q qq q q q q qqqqq q qqqq q q q qq q q qq qq q q qq qq q q qq qq q q q q q q q q q q qq q qq q q q qq q q q q q q q q q q q qq q q qq q q q q qqqq q q qq qq q q q q q qq qqq q qq q q qqq q q q qq q q qq qqqqqq q qq q q q q qqqqqq qqq q q q q q q q q q q q q q q qqq q q q q q qq qq q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q qqq q q qq q q q qqq q q q qq q q q q q q q qq q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q qq q q qq q qq q q qq q qqq q q q q q q q q q q q qq q q qq q q q q qq q q q q qq q q qq q q qq q q q qq qqqq qq q q qq q q q q qqqq q qq q q q qqqq qq q q q qqq qqqq q q qqq q qqq q q qq q qqq q q qq q q qqq q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q q q qqqq q qqq q qqq q qqq q q q q qqq qq q q q q q q q qq q q q q q q q qq q qq q q q q q q q q q q q q qq q qq qq q q q q q q q q q q q q q q q qq qq q q qqqq q q qqq q q q q q q q q qq q qqqq q qq qq q q q qq qq q q q q q q q qq q q q q q q qq q q qq q q qq q qqq q q q q q q q q q q q q q q q q qq q qq q q q q q q q qq q qq q q q q q qq q q q q q qq q q q q q q q q qq qq q q qq q q qq q q q q q qq q q qq q q q q q q q q q q q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq qqq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq qq qq q q q q q qq q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q qq q q q q qq q q q q q q q q q q q q q q q qq q q qqq q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qqq qq q q qq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q qq q qq q q q qq q q q q qq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q qq q q q q q q q qq q qq q q q q qq q q q q qq q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q qq q q q q q q q q q q q qq qqq q qq q q q q q q q qq q q q q qq q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q qq q qq q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q q qqq q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q qq q q q q q q q q q q q q q q q q q qq qqq q q q q q q q qq q q q q qq q q q q q q qqqq q q q qqqq q q qq qqq q q q q q q qqqq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q q q q qq qqq qqqqqqqq q qq qqqqqq q qqqq q q q q q q q q q q qqq q q q q q q q q q q qqqq q q q q q q q qq q q qq q q qq q qq q q q q qq qqq q q q q q q q q qq q qq q q q qqq qq q q q q qq q q q q q q q qq qqq q q qq q q q q q qq q qq q q q q q q q qq q q q q q q q q q qqq q q q q q q q q qq q q q q q q q q q q qq q q q q q q q q qqqq q q q q q qq q q qq q q q q q q q qq q q q q q q q q q q q q qq q q q q qq q q q q q q q q q q q q q q q q q qq q q qq qq q q q q qq q q qqqq q q q q q q q q q q q q q q q q q q q q q qqq q qq q q q q q q q q q q q q q q q q q q q q qq q qq qq q qq q qq q q qqq q qq qq q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q qq q q qq q q qqq q qq q q qq qq q q qqqq q q q q q q q q q q qqq q q q q q qqqq qqqq q q q q q q q q q q q q q q q q qq q q q q q q qq q qq q q q q qq q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q qqqqq q q q qq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q qq q qq q q qq q q qqq q q q q q qq q qq q q q q q q q q q q q qq q q q qq q q qq q q q q q qq qq qqq qqq q q q q q qqqqqqqqq q q qqqqqqqqqqqqqqqqq qqq q q q q qqqqq qqq q q qqqqq q q q qqqq q q q q q q q qq q q q qqqqq q qqqqq q qq qqq qqqq q qq q qq q q q qq q qq −3 −2 −1 0 1 2 3 −2 0 2 4 PC1 PC2 0.00 0.25 0.50 0.75 1.00 value Lambda
  • 29. Feature space of M3 data Exploring the feature space of large time series collections M3 competition data 11 q q q q q q qq qq q q q q q q q q q q q q q qq q qq q qqq q q qq q q qq q qq qq q qqq q q q q qq qqqq q q qq q q qq q q q q q q qq q qq q q q q q q q q q q qq q q q q q qq q q qq qq q q qq q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q q q qq qqq q q q q q qq q qqq qqqq q qq q q q q qq q qq qq qqqq q q q q q q q q q qqq q q q q q q qq q q q q q qq qq q q q q q qq q qqq q q q q q q q q q q q q q q q q q q q qq q qq q q q qqqq q qqq q qqq qq q qq qq qqqqqqqqqq q q q q q q qq q qq q q q q q q q qq q q qqq qq qqq q q qqqqqq q q q q qq qq qqqq q q qq q qq q q qqq q q q q q qqq qq q q qqqqq qqq qq q q q q q q q q q q q qq q qqq q qqq qqq q qq q q q q q q qqqqq q q qq q q qq q q qq q q q q q q q q q q qq q q q qqqqq q q q q q q q q q q q q q q q q q q qq q q q q qq q q qqq qqqq q q q qqq q q q qq q q q q qqqqq q qqqq q q q qq q q qq qq q q qq q q q q qq qq q q q q q q q q q q qq q qq q q q qq q q q q q q q q q q q qq q q qq q q q q qqq q q q qq qq q q q q q qq qqq q q q q q qqq q q q qq q q qq qqqqqq q qq q q q q qqqqqq qqq q q q q q q q q q q q q q q qqq q q q q q qq qq q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q qqq q q qq q q q qqq q q q qq q q q q q q q qq q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q qq q q qq q qq q q qq q qqq q q q q q q q q q q q qq q q qq q q q q qq q q q q qq q q qq q q qq q q q qq qqq q q q q q qq q q q q qqqq q qq q q q qqqq qq q q q qqq qqqq q q qqq q qqq q q q q q qqq q q qq q q qqq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q q q qqqq q qqq q qqq q qqq q q q q qq q qq q q q q q q q qq q q q q q q q qq q qq q q q q q q q q q q q q qq q qq qq q q q q q q q q q q q q q q q qq qq q q q qqq q q qqq q q q q q q q q qq q qqqq q qq qq q q q qq qq q q q q q q q qq q q q q q q qq q q q q q q qq q qqq q q q q q q q q q q q q q q q q qq q qq q q q q q q q qq q qq q q q q q qq q q q q q qq q q q q q q q q qq qq q q qq q q qq q q q q q qq q q qq q q q q q q q q q q q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq qqq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq qq qq q q q q q qq q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq qq q q qq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q qq q qq q q q qq q q q q qq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q qq q q q q q q q qq q qq q q q q qq q q q q qq q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q qq q q q q q q q q q q q qq qqq q qq q q q q q q q qq q q q q qq q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q qq q qq q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q q qqq q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q qq q q q q q q q qq q q q q qq q q q q q q qqqq q q q qqqq q q qq qq q q q q q q q qqqq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q q q q qq qqq qqqq qq qq q qq qqqqqq q q qqq q q q q q q q q q q qqq q q q q q q q q q q qqq q q q q q q q q qq q q qq q q qq q qq q q q q qq qqq q q q q q q q q qq q qq q q q qqq q q q q q q qq q q q q q q q qq qqq q q qq q q q q q qq q qq q q q q q q q qq q q q q q q q q q qqq q q q q q q q q qq q q q q q q q q q q qq q q q q q q q q qqqq q q q q q qq q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q qq q q q q q q q q q q q q q q q q q qq q q qq qq q q q q qq q q qqqq q q q q q q q q q q q q q q q q q q q q q qqq q qq q q q q q q q q q q q q q q q q q q q q qq q qq qq q qq q qq q q qqq q qq qq q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q qq q q qq q q qqq q qq q q qq qq q q qqqq q q q q q q q q q q qqq q q q q q qqqq qqqq q q q q q q q q q q q q q q q q qq q q q q q q qq q qq q q q q qq q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q qq qq q q q qq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q qq q qq q q qq q q qqq q q q q q qq q qq q q q q q q q q q q q qq q q q qq q q qq q q q q q qq qq qqq qqq q q q q q q qqqqqqqq q q qqqqqqqqqqqqqqqqq qqq q q q q qqqqq qqq q q qqqqq q q q qqqq q q q q q q q qq q q q qqqqq q qqqqq q qq qqq qqqq q qq q qq q q q qq q qq −2 0 2 4 −3−2−10123 PC1 PC2 q q q q
  • 30. Feature space of M3 data Exploring the feature space of large time series collections M3 competition data 11 N1627 1990 1991 1992 1993 1994 N0636 1920 1925 1930 1935 1940 1945 N2981 0 10 20 30 40 50 60 N2699 1984 1986 1988 1990 1992
  • 31. Feature space of M3 data Exploring the feature space of large time series collections M3 competition data 11 q q q q q q qq qq q q q q q q q q q q q q q qq q qq q qqq q q qq q q qq q qq qq q qqq q q q q qq qqqq q q qq q q qq q q q q q q qq q qq q q q q q q q q q q qq q q q q q qq q q qq qq q q qq q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q q q qq qqq q q q q q qq q qqq qqqq q qq q q q q qq q qq qq qqqq q q q q q q q q q qqq q q q q q q qq q q q q q qq qq q q q q q qq q qqq q q q q q q q q q q q q q q q q q q q qq q qq q q q qqqq q qqq q qqq qq q qq qq qqqqqqqqqq q q q q q q qq q qq q q q q q q q qq q q qqq qq qqq q q qqqqqq q q q q qq qq qqqq q q qq q qq q q qqq q q q q q qqq qq q q qqqqq qqq qq q q q q q q q q q q q qq q qqq q qqq qqq q qq q q q q q q qqqqq q q qq q q qq q q qq q q q q q q q q q q qq q q q qqqqq q q q q q q q q q q q q q q q q q q qq q q q q qq q q qqq qqqq q q q qqq q q q qq q q q q qqqqq q qqqq q q q qq q q qq qq q q qq q q q q qq qq q q q q q q q q q q qq q qq q q q qq q q q q q q q q q q q qq q q qq q q q q qqq q q q qq qq q q q q q qq qqq q q q q q qqq q q q qq q q qq qqqqqq q qq q q q q qqqqqq qqq q q q q q q q q q q q q q q qqq q q q q q qq qq q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q qqq q q qq q q q qqq q q q qq q q q q q q q qq q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q qq q q qq q qq q q qq q qqq q q q q q q q q q q q qq q q qq q q q q qq q q q q qq q q qq q q qq q q q qq qqq q q q q q qq q q q q qqqq q qq q q q qqqq qq q q q qqq qqqq q q qqq q qqq q q q q q qqq q q qq q q qqq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q q q qqqq q qqq q qqq q qqq q q q q qq q qq q q q q q q q qq q q q q q q q qq q qq q q q q q q q q q q q q qq q qq qq q q q q q q q q q q q q q q q qq qq q q q qqq q q qqq q q q q q q q q qq q qqqq q qq qq q q q qq qq q q q q q q q qq q q q q q q qq q q q q q q qq q qqq q q q q q q q q q q q q q q q q qq q qq q q q q q q q qq q qq q q q q q qq q q q q q qq q q q q q q q q qq qq q q qq q q qq q q q q q qq q q qq q q q q q q q q q q q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq qqq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq qq qq q q q q q qq q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq qq q q qq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q qq q qq q q q qq q q q q qq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q qq q q q q q q q qq q qq q q q q qq q q q q qq q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q qq q q q q q q q q q q q qq qqq q qq q q q q q q q qq q q q q qq q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q qq q qq q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q q qqq q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q qq q q q q q q q qq q q q q qq q q q q q q qqqq q q q qqqq q q qq qq q q q q q q q qqqq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q q q q qq qqq qqqq qq qq q qq qqqqqq q q qqq q q q q q q q q q q qqq q q q q q q q q q q qqq q q q q q q q q qq q q qq q q qq q qq q q q q qq qqq q q q q q q q q qq q qq q q q qqq q q q q q q qq q q q q q q q qq qqq q q qq q q q q q qq q qq q q q q q q q qq q q q q q q q q q qqq q q q q q q q q qq q q q q q q q q q q qq q q q q q q q q qqqq q q q q q qq q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q qq q q q q q q q q q q q q q q q q q qq q q qq qq q q q q qq q q qqqq q q q q q q q q q q q q q q q q q q q q q qqq q qq q q q q q q q q q q q q q q q q q q q q qq q qq qq q qq q qq q q qqq q qq qq q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q qq q q qq q q qqq q qq q q qq qq q q qqqq q q q q q q q q q q qqq q q q q q qqqq qqqq q q q q q q q q q q q q q q q q qq q q q q q q qq q qq q q q q qq q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q qq qq q q q qq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q qq q qq q q qq q q qqq q q q q q qq q qq q q q q q q q q q q q qq q q q qq q q qq q q q q q qq qq qqq qqq q q q q q q qqqqqqqq q q qqqqqqqqqqqqqqqqq qqq q q q q qqqqq qqq q q qqqqq q q q qqqq q q q q q q q qq q q q qqqqq q qqqqq q qq qqq qqqq q qq q qq q q q qq q qq −2 0 2 4 −3−2−10123 PC1 PC2 Yahoo log returns series
  • 32. Feature space of M3 data Exploring the feature space of large time series collections M3 competition data 11 q q q q q q qq qq q q q q q q q q q q q q q qq q qq q qqq q q qq q q qq q qq qq q qqq q q q q qq qqqq q q qq q q qq q q q q q q qq q qq q q q q q q q q q q qq q q q q q qq q q qq qq q q qq q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q q q qq qqq q q q q q qq q qqq qqqq q qq q q q q qq q qq qq qqqq q q q q q q q q q qqq q q q q q q qq q q q q q qq qq q q q q q qq q qqq q q q q q q q q q q q q q q q q q q q qq q qq q q q qqqq q qqq q qqq qq q qq qq qqqqqqqqqq q q q q q q qq q qq q q q q q q q qq q q qqq qq qqq q q qqqqqq q q q q qq qq qqqq q q qq q qq q q qqq q q q q q qqq qq q q qqqqq qqq qq q q q q q q q q q q q qq q qqq q qqq qqq q qq q q q q q q qqqqq q q qq q q qq q q qq q q q q q q q q q q qq q q q qqqqq q q q q q q q q q q q q q q q q q q qq q q q q qq q q qqq qqqq q q q qqq q q q qq q q q q qqqqq q qqqq q q q qq q q qq qq q q qq q q q q qq qq q q q q q q q q q q qq q qq q q q qq q q q q q q q q q q q qq q q qq q q q q qqq q q q qq qq q q q q q qq qqq q q q q q qqq q q q qq q q qq qqqqqq q qq q q q q qqqqqq qqq q q q q q q q q q q q q q q qqq q q q q q qq qq q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q qqq q q qq q q q qqq q q q qq q q q q q q q qq q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q qq q q qq q qq q q qq q qqq q q q q q q q q q q q qq q q qq q q q q qq q q q q qq q q qq q q qq q q q qq qqq q q q q q qq q q q q qqqq q qq q q q qqqq qq q q q qqq qqqq q q qqq q qqq q q q q q qqq q q qq q q qqq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q q q qqqq q qqq q qqq q qqq q q q q qq q qq q q q q q q q qq q q q q q q q qq q qq q q q q q q q q q q q q qq q qq qq q q q q q q q q q q q q q q q qq qq q q q qqq q q qqq q q q q q q q q qq q qqqq q qq qq q q q qq qq q q q q q q q qq q q q q q q qq q q q q q q qq q qqq q q q q q q q q q q q q q q q q qq q qq q q q q q q q qq q qq q q q q q qq q q q q q qq q q q q q q q q qq qq q q qq q q qq q q q q q qq q q qq q q q q q q q q q q q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq qqq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq qq qq q q q q q qq q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq qq q q qq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q qq q qq q q q qq q q q q qq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q qq q q q q q q q qq q qq q q q q qq q q q q qq q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q qq q q q q q q q q q q q qq qqq q qq q q q q q q q qq q q q q qq q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q qq q qq q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q q qqq q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q qq q q q q q q q qq q q q q qq q q q q q q qqqq q q q qqqq q q qq qq q q q q q q q qqqq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q q q q qq qqq qqqq qq qq q qq qqqqqq q q qqq q q q q q q q q q q qqq q q q q q q q q q q qqq q q q q q q q q qq q q qq q q qq q qq q q q q qq qqq q q q q q q q q qq q qq q q q qqq q q q q q q qq q q q q q q q qq qqq q q qq q q q q q qq q qq q q q q q q q qq q q q q q q q q q qqq q q q q q q q q qq q q q q q q q q q q qq q q q q q q q q qqqq q q q q q qq q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q qq q q q q q q q q q q q q q q q q q qq q q qq qq q q q q qq q q qqqq q q q q q q q q q q q q q q q q q q q q q qqq q qq q q q q q q q q q q q q q q q q q q q q qq q qq qq q qq q qq q q qqq q qq qq q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q qq q q qq q q qqq q qq q q qq qq q q qqqq q q q q q q q q q q qqq q q q q q qqqq qqqq q q q q q q q q q q q q q q q q qq q q q q q q qq q qq q q q q qq q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q qq qq q q q qq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q qq q qq q q qq q q qqq q q q q q qq q qq q q q q q q q q q q q qq q q q qq q q qq q q q q q qq qq qqq qqq q q q q q q qqqqqqqq q q qqqqqqqqqqqqqqqqq qqq q q q q qqqqq qqq q q qqqqq q q q qqqq q q q q q q q qq q q q qqqqq q qqqqq q qq qqq qqqq q qq q qq q q q qq q qq −2 0 2 4 −3−2−10123 PC1 PC2 qYahoo log returns series
  • 33. Feature space of M3 data Exploring the feature space of large time series collections M3 competition data 12 q q q q q q qq qq q q q q q q q q q q q q q qq q qq q qqq q q qq q q qq q qq qq q qqq q q q q qq qqqq q q qq q q qq q q q q q q qq q qq q q q q q q q q q q qq q q q q q qq q q qq qq q q qq q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q q q qq qqq q q q q q qq q qqq qqqq q qq q q q q qq q qq qq qqqq q q q q q q q q q qqq q q q q q q qq q q q q q qq qq q q q q q qq q qqq q q q q q q q q q q q q q q q q q q q qq q qq q q q qqqq q qqq q qqq qq q qq qq qqqqqqqqqq q q q q q q qq q qq q q q q q q q qq q q qqq qq qqq q q qqqqqq q q q q qq qq qqqq q q qq q qq q q qqq q q q q q qqq qq q q qqqqq qqq qq q q q q q q q q q q q qq q qqq q qqq qqq q qq q q q q q q qqqqq q q qq q q qq q q qq q q q q q q q q q q qq q q q qqqqq q q q q q q q q q q q q q q q q q q qq q q q q qq q q qqq qqqq q q q qqq q q q qq q q q q qqqqq q qqqq q q q qq q q qq qq q q qq q q q q qq qq q q q q q q q q q q qq q qq q q q qq q q q q q q q q q q q qq q q qq q q q q qqq q q q qq qq q q q q q qq qqq q q q q q qqq q q q qq q q qq qqqqqq q qq q q q q qqqqqq qqq q q q q q q q q q q q q q q qqq q q q q q qq qq q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q qqq q q qq q q q qqq q q q qq q q q q q q q qq q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q qq q q qq q qq q q qq q qqq q q q q q q q q q q q qq q q qq q q q q qq q q q q qq q q qq q q qq q q q qq qqq q q q q q qq q q q q qqqq q qq q q q qqqq qq q q q qqq qqqq q q qqq q qqq q q q q q qqq q q qq q q qqq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q q q qqqq q qqq q qqq q qqq q q q q qq q qq q q q q q q q qq q q q q q q q qq q qq q q q q q q q q q q q q qq q qq qq q q q q q q q q q q q q q q q qq qq q q q qqq q q qqq q q q q q q q q qq q qqqq q qq qq q q q qq qq q q q q q q q qq q q q q q q qq q q q q q q qq q qqq q q q q q q q q q q q q q q q q qq q qq q q q q q q q qq q qq q q q q q qq q q q q q qq q q q q q q q q qq qq q q qq q q qq q q q q q qq q q qq q q q q q q q q q q q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq qqq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq qq qq q q q q q qq q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq qq q q qq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q qq q qq q q q qq q q q q qq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q qq q q q q q q q qq q qq q q q q qq q q q q qq q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q qq q q q q q q q q q q q qq qqq q qq q q q q q q q qq q q q q qq q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q qq q qq q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q q qqq q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q qq q q q q q q q qq q q q q qq q q q q q q qqqq q q q qqqq q q qq qq q q q q q q q qqqq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q q q q qq qqq qqqq qq qq q qq qqqqqq q q qqq q q q q q q q q q q qqq q q q q q q q q q q qqq q q q q q q q q qq q q qq q q qq q qq q q q q qq qqq q q q q q q q q qq q qq q q q qqq q q q q q q qq q q q q q q q qq qqq q q qq q q q q q qq q qq q q q q q q q qq q q q q q q q q q qqq q q q q q q q q qq q q q q q q q q q q qq q q q q q q q q qqqq q q q q q qq q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q qq q q q q q q q q q q q q q q q q q qq q q qq qq q q q q qq q q qqqq q q q q q q q q q q q q q q q q q q q q q qqq q qq q q q q q q q q q q q q q q q q q q q q qq q qq qq q qq q qq q q qqq q qq qq q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q qq q q qq q q qqq q qq q q qq qq q q qqqq q q q q q q q q q q qqq q q q q q qqqq qqqq q q q q q q q q q q q q q q q q qq q q q q q q qq q qq q q q q qq q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q qq qq q q q qq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q qq q qq q q qq q q qqq q q q q q qq q qq q q q q q q q q q q q qq q q q qq q q qq q q q q q qq qq qqq qqq q q q q q q qqqqqqqq q q qqqqqqqqqqqqqqqqq qqq q q q q qqqqq qqq q q qqqqq q q q qqqq q q q q q q q qq q q q qqqqq q qqqqq q qq qqq qqqq q qq q qq q q q qq q qq −2 0 2 4 −3−2−10123 PC1 PC2 White noise series
  • 34. Feature space of M3 data Exploring the feature space of large time series collections M3 competition data 13 q q q q q q qq qq q q q q q q q q q q q q q qq q qq q qqq q q qq q q qq q qq qq q qqq q q q q qq qqqq q q qq q q qq q q q q q q qq q qq q q q q q q q q q q qq q q q q q qq q q qq qq q q qq q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q q q qq qqq q q q q q qq q qqq qqqq q qq q q q q qq q qq qq qqqq q q q q q q q q q qqq q q q q q q qq q q q q q qq qq q q q q q qq q qqq q q q q q q q q q q q q q q q q q q q qq q qq q q q qqqq q qqq q qqq qq q qq qq qqqqqqqqqq q q q q q q qq q qq q q q q q q q qq q q qqq qq qqq q q qqqqqq q q q q qq qq qqqq q q qq q qq q q qqq q q q q q qqq qq q q qqqqq qqq qq q q q q q q q q q q q qq q qqq q qqq qqq q qq q q q q q q qqqqq q q qq q q qq q q qq q q q q q q q q q q qq q q q qqqqq q q q q q q q q q q q q q q q q q q qq q q q q qq q q qqq qqqq q q q qqq q q q qq q q q q qqqqq q qqqq q q q qq q q qq qq q q qq q q q q qq qq q q q q q q q q q q qq q qq q q q qq q q q q q q q q q q q qq q q qq q q q q qqq q q q qq qq q q q q q qq qqq q q q q q qqq q q q qq q q qq qqqqqq q qq q q q q qqqqqq qqq q q q q q q q q q q q q q q qqq q q q q q qq qq q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q qqq q q qq q q q qqq q q q qq q q q q q q q qq q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q qq q q qq q qq q q qq q qqq q q q q q q q q q q q qq q q qq q q q q qq q q q q qq q q qq q q qq q q q qq qqq q q q q q qq q q q q qqqq q qq q q q qqqq qq q q q qqq qqqq q q qqq q qqq q q q q q qqq q q qq q q qqq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q q q qqqq q qqq q qqq q qqq q q q q qq q qq q q q q q q q qq q q q q q q q qq q qq q q q q q q q q q q q q qq q qq qq q q q q q q q q q q q q q q q qq qq q q q qqq q q qqq q q q q q q q q qq q qqqq q qq qq q q q qq qq q q q q q q q qq q q q q q q qq q q q q q q qq q qqq q q q q q q q q q q q q q q q q qq q qq q q q q q q q qq q qq q q q q q qq q q q q q qq q q q q q q q q qq qq q q qq q q qq q q q q q qq q q qq q q q q q q q q q q q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq qqq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq qq qq q q q q q qq q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq qq q q qq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q qq q qq q q q qq q q q q qq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q qq q q q q q q q qq q qq q q q q qq q q q q qq q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q qq q q q q q q q q q q q qq qqq q qq q q q q q q q qq q q q q qq q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q qq q qq q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q q qqq q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q qq q q q q q q q qq q q q q qq q q q q q q qqqq q q q qqqq q q qq qq q q q q q q q qqqq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q q q q qq qqq qqqq qq qq q qq qqqqqq q q qqq q q q q q q q q q q qqq q q q q q q q q q q qqq q q q q q q q q qq q q qq q q qq q qq q q q q qq qqq q q q q q q q q qq q qq q q q qqq q q q q q q qq q q q q q q q qq qqq q q qq q q q q q qq q qq q q q q q q q qq q q q q q q q q q qqq q q q q q q q q qq q q q q q q q q q q qq q q q q q q q q qqqq q q q q q qq q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q qq q q q q q q q q q q q q q q q q q qq q q qq qq q q q q qq q q qqqq q q q q q q q q q q q q q q q q q q q q q qqq q qq q q q q q q q q q q q q q q q q q q q q qq q qq qq q qq q qq q q qqq q qq qq q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q qq q q qq q q qqq q qq q q qq qq q q qqqq q q q q q q q q q q qqq q q q q q qqqq qqqq q q q q q q q q q q q q q q q q qq q q q q q q qq q qq q q q q qq q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q qq qq q q q qq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q qq q qq q q qq q q qqq q q q q q qq q qq q q q q q q q q q q q qq q q q qq q q qq q q q q q qq qq qqq qqq q q q q q q qqqqqqqq q q qqqqqqqqqqqqqqqqq qqq q q q q qqqqq qqq q q qqqqq q q q qqqq q q q q q q q qq q q q qqqqq q qqqqq q qq qqq qqqq q qq q qq q q q qq q qq −2 0 2 4 −3−2−10123 PC1 PC2 Random walk series
  • 35. Feature space of M3 data Exploring the feature space of large time series collections M3 competition data 14 q q q q q q qq qq q q q q q q q q q q q q q qq q qq q qqq q q qq q q qq q qq qq q qqq q q q q qq qqqq q q qq q q qq q q q q q q qq q qq q q q q q q q q q q qq q q q q q qq q q qq qq q q qq q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q q q qq qqq q q q q q qq q qqq qqqq q qq q q q q qq q qq qq qqqq q q q q q q q q q qqq q q q q q q qq q q q q q qq qq q q q q q qq q qqq q q q q q q q q q q q q q q q q q q q qq q qq q q q qqqq q qqq q qqq qq q qq qq qqqqqqqqqq q q q q q q qq q qq q q q q q q q qq q q qqq qq qqq q q qqqqqq q q q q qq qq qqqq q q qq q qq q q qqq q q q q q qqq qq q q qqqqq qqq qq q q q q q q q q q q q qq q qqq q qqq qqq q qq q q q q q q qqqqq q q qq q q qq q q qq q q q q q q q q q q qq q q q qqqqq q q q q q q q q q q q q q q q q q q qq q q q q qq q q qqq qqqq q q q qqq q q q qq q q q q qqqqq q qqqq q q q qq q q qq qq q q qq q q q q qq qq q q q q q q q q q q qq q qq q q q qq q q q q q q q q q q q qq q q qq q q q q qqq q q q qq qq q q q q q qq qqq q q q q q qqq q q q qq q q qq qqqqqq q qq q q q q qqqqqq qqq q q q q q q q q q q q q q q qqq q q q q q qq qq q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q qqq q q qq q q q qqq q q q qq q q q q q q q qq q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q qq q q qq q qq q q qq q qqq q q q q q q q q q q q qq q q qq q q q q qq q q q q qq q q qq q q qq q q q qq qqq q q q q q qq q q q q qqqq q qq q q q qqqq qq q q q qqq qqqq q q qqq q qqq q q q q q qqq q q qq q q qqq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q q q qqqq q qqq q qqq q qqq q q q q qq q qq q q q q q q q qq q q q q q q q qq q qq q q q q q q q q q q q q qq q qq qq q q q q q q q q q q q q q q q qq qq q q q qqq q q qqq q q q q q q q q qq q qqqq q qq qq q q q qq qq q q q q q q q qq q q q q q q qq q q q q q q qq q qqq q q q q q q q q q q q q q q q q qq q qq q q q q q q q qq q qq q q q q q qq q q q q q qq q q q q q q q q qq qq q q qq q q qq q q q q q qq q q qq q q q q q q q q q q q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq qqq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq qq qq q q q q q qq q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq qq q q qq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q qq q qq q q q qq q q q q qq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q qq q q q q q q q qq q qq q q q q qq q q q q qq q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q qq q q q q q q q q q q q qq qqq q qq q q q q q q q qq q q q q qq q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q qq q qq q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q q qqq q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q qq q q q q q q q qq q q q q qq q q q q q q qqqq q q q qqqq q q qq qq q q q q q q q qqqq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q q q q qq qqq qqqq qq qq q qq qqqqqq q q qqq q q q q q q q q q q qqq q q q q q q q q q q qqq q q q q q q q q qq q q qq q q qq q qq q q q q qq qqq q q q q q q q q qq q qq q q q qqq q q q q q q qq q q q q q q q qq qqq q q qq q q q q q qq q qq q q q q q q q qq q q q q q q q q q qqq q q q q q q q q qq q q q q q q q q q q qq q q q q q q q q qqqq q q q q q qq q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q qq q q q q q q q q q q q q q q q q q qq q q qq qq q q q q qq q q qqqq q q q q q q q q q q q q q q q q q q q q q qqq q qq q q q q q q q q q q q q q q q q q q q q qq q qq qq q qq q qq q q qqq q qq qq q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q qq q q qq q q qqq q qq q q qq qq q q qqqq q q q q q q q q q q qqq q q q q q qqqq qqqq q q q q q q q q q q q q q q q q qq q q q q q q qq q qq q q q q qq q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q qq qq q q q qq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q qq q qq q q qq q q qqq q q q q q qq q qq q q q q q q q q q q q qq q q q qq q q qq q q q q q qq qq qqq qqq q q q q q q qqqqqqqq q q qqqqqqqqqqqqqqqqq qqq q q q q qqqqq qqq q q qqqqq q q q qqqq q q q q q q q qq q q q qqqqq q qqqqq q qq qqq qqqq q qq q qq q q q qq q qq −2 0 2 4 −3−2−10123 PC1 PC2 q q
  • 36. Feature space of M3 data Exploring the feature space of large time series collections M3 competition data 14 Time 5 10 15
  • 37. Outline 1 M3 competition data 2 Yahoo web traffic 3 What next? Exploring the feature space of large time series collections Yahoo web traffic 15
  • 38. Yahoo web-traffic Tens of thousands of time series collected at one-hour intervals over one month. Consisting of several server metrics (e.g. CPU usage and paging views) from many server farms globally. Aim: find unusual (anomalous) time series. Exploring the feature space of large time series collections Yahoo web traffic 16
  • 39. Yahoo web-traffic busy233busy271busy50busy200busy369 2014−11−09 2014−11−10 2014−11−11 2014−11−12 2014−11−13 2014−11−14 2014−11−15 2014−11−16 2014−11−17 2014−11−18 2014−11−19 2014−11−20 2014−11−21 2014−11−22 2014−11−23 2014−11−24 2014−11−25 2014−11−26 2014−11−27 2014−11−28 2014−11−29 2014−11−30 2014−12−01 2014−12−02 2014−12−03 2014−12−04 2014−12−05 2014−12−06 2014−12−07 2014−12−08 2014−12−09 2014−12−10 2014−12−11 2014−12−12 date 25 30 35 40 45 20 25 30 35 40 50 60 70 10 15 20 25 50 60 memory460memory429memory147memory413memory484 2014−11−09 2014−11−10 2014−11−11 2014−11−12 2014−11−13 2014−11−14 2014−11−15 2014−11−16 2014−11−17 2014−11−18 2014−11−19 2014−11−20 2014−11−21 2014−11−22 2014−11−23 2014−11−24 2014−11−25 2014−11−26 2014−11−27 2014−11−28 2014−11−29 2014−11−30 2014−12−01 2014−12−02 2014−12−03 2014−12−04 2014−12−05 2014−12−06 2014−12−07 2014−12−08 2014−12−09 2014−12−10 2014−12−11 2014−12−12 date value 0 5000 10000 15000 20000 200 400 600 0 5000 10000 15000 20000 500 1000 0 5000 10000 15000 20000 25000 2014−11−09 2014−11−10 2014−11−11 2014−11−12 2014−11−13 2014−11−14 2014−11−15 2014−11−16 2014−11−17 2014−11−18 2014−11−19 2014−11−20 2014−11−21 2014−11−22 2014−11−23 2014−11−24 2014−11−25 2014−11−26 2014−11−27 2014−11−28 2014−11−29 2014−11−30 2014−12−01 2014−12−02 2014−12−03 2014−12−04 2014−12−05 2014−12−06 2014−12−07 2014−12−08 2014−12−09 2014−12−10 date value Exploring the feature space of large time series collections Yahoo web traffic 17
  • 40. Feature space ACF1: first order autocorrelation = Corr(Yt, Yt−1) Strength of trend and seasonality based on STL Trend linearity and curvature Size of seasonal peak and trough Spectral entropy Lumpiness: variance of block variances (block size 24). Spikiness: variances of leave-one-out variances of STL remainders. Level shift: Maximum difference in trimmed means of consecutive moving windows of size 24. Variance change: Max difference in variances of consecutive moving windows of size 24. Flat spots: Discretize sample space into 10 equal-sized intervals. Find max run length in any interval. Number of crossing points of mean line. Kullback-Leibler score: Maximum of DKL(P Q) = P(x) ln P(x)/Q(x)dx where P and Q are estimated by kernel density estimators applied to consecutive windows of size 48. Change index: Time of maximum KL score Exploring the feature space of large time series collections Yahoo web traffic 18
  • 41. Principal component analysis q q qq q q q q q q q qq q q q q qq q qq q q q q q q qq q q q q qqq q qq q q q q q q q q q q q q q q q q q q q qqq q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q qqq q q qq q q q q qq q q q q q qq q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q qq q qq q q q q q q qq q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qqq q q q q qq qqqqq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q qq qq qq q q q q q q q qq q q q q q q q q q qqq q qqqqq q q q q q q q qqq q q q q q qq qq q qq qqqq qq q q q q q q q q qq q qqq qqqq qqq q q q qq q q q q q q q qq qq q q q q q q q q q qq q qqq q q q q q q q q q q q q q q q q q q q qq q q q q qq q q q q q qq q q q q q q q qq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q qq q q qq q q q q q q q q q q q q q q qq q q q q q q q q q qq qq q q q q qq q q qq q q q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q qqqq q q q q q q q q qq qq q qqq q q q qq q q q q q q q q q q q q q q q q qqq q q qq qq q q q q qqq q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q qq q q q q q q q qq q q q q qq q q q qq q q qq q q qq q q q q q q q q q qq q qqq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q qqq q q q q q q qq q q q q q q q q q qqq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q q q q q q q q q q qq q q q q qq qq q q q q q q q q q q q q qqq q q q q q q q q qq q q q q q q q q q q q q q q qq q qq q q q q q q q q q q q q q qq q qq qq q q q q q q q q q q q q qq q q q q q q q q q qq q q q qq q q q qq q q q qqq q q q q q q q q q qqq qq q qq q q qq q qq qqqq qqq q q qqq q qq q q qq q q q q q q q q q q q q q q qq q q q q qq qq q q q q qq qq qq q q q q q q q q q q q q q q q q q q qqqq q q q q q qq q q qq qq qq q q qq qq q qq q q q q q q q q q q q q q q q q q q q qq qqq q qq qq q q q q qq qq q q qq q qq q q q q q q q q q qq q q q q q q q q q qq q q q q q q qq q qq qq qq q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q qqq q q q q q q qq q qq q q qq q q q q qq q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q qq q qq q q qqqq q q q q q q q q q q q q q q q q q q q qq q q q q qq q q q q q q q q q q q q qq q q q qq qqq q q q q q q q q q q q qq q qq q q q q q q qq q qq q qq q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q qqq q qq q q q q q q q qqq q q q q q q q q q qq q qq q q q q q q q q q q q q q q q q qq qq q q qq q q q q q q q q q q q qq q q q q q q q q q qq q q qq q q q q q q qq q q q q qqqq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q q q q qqq qq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q qq q qq q q q q q q qq q qq q q qq q q q q q q qq qq q q q q q q q q q q q qq q q q q q q qq q q q q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q qq q q q q qqq q q qq q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q qq q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q qq q q q q q q q q q q q q q q q q q q qq q q q q qq q q q q q q q qq q q qq q q q q q q q q q q q q qqq q qq q qq qq qq q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q qq q q q q qq q q q qqq q q q q q q q q q q q q q q q q q q q q q q qq q q q q qq q q qq qq q q q q q q q q q q q q q q q q qq q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq qq q q q q q q q q qq qq q q q q q q q q qq q q q q q q qq q q qq q q q q q q q q q q q q qq q qq q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q q q qq q q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q qq q q q q q q q q q q q qq q q q q q qq q q qqq q q q q q q q q q q q q q q q q q q q q q q qq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q qq q q q q q q qq q q qq q q q q q q q q q q q qqq q q q q q q q qq q q q q qq q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q qq q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q q q qq q q q q q q q q q qq q qq q q q q q q q q qq q q q qq q q q q q q q q qq q q q qqqq qq q q q qq q q q q q qq qq q q q q qq q q q q q q qq q q q q qq q qq q q q q q q q q q q q q qq q qq qq qqq q q q qq q q q q q q q q q q q qq q q q q q qq q q q qqq q q q q q q q qq q q q q q q q q q q q q q q q qq q q qq q q q qq q q q q qq q q q q q q q q q qq qqq q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q q qq qq q q q q qq q q q q q q q q qq q q q q q qq q q q q q qq q q q q q q q q q qq q qq q q q q q q q q q q q q q q q q qq q q qq q q q q q q q q q q q q q q q qqqq qq q q qq q q qq q q q qq q q qq q q qq q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q qqq q q q q q q q q q q q q qq q q q q q qq q q q q qq q qq qq qq q qq q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q qq qq q qqq q q qqqqq q q q q qq q q qq qq q q q q q q qqq q q q q q qq q q qq q q q q q qq q q q q q qq q q q q q q qq qqqq qq q q q q qq q q q q q q q q q q q q qq qq qqqqqq q q q qq q q q q qq q qq qqq q qqqq q q q q q q q q q q q q qq q q q q q q q qq q qq q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q qq q q qq q q q q q q qq qq q q q qq q q q q qq q q q q q q q q q q q qq qq q q q q q q q qqq q q q q q q q q q qq q q q q q q q qq q q q q qq q q qq q q q q q q qq q q q q q q q q q q q qq q q q qq qqq qq q q qq q q qqq q q qq q qq q q q qqq q q q q q qq q q q q q qq q q q q q q q q q q qqq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q qqq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q q q q q q q q q q q q q q qq q q q q q qqq q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q qq q q qq q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q q q q qqq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q qqq q q q q q q q q q q q q q q qq q q qq q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qqq q q q q q q q q qq qq q q q q qq q q q q q q q q q q q qq q q q qq q q q q q q q q q q qq qq qq q q q q q q q q qq qq q q q q q q q q qq q q qq q q q q q q qq q q q q q q qq q q q q q q q q q q q q q q qq q q q q q q qq q qq q q q qq q q q qq q q q q q q q q q qq q q q q q qqq q q q q q q q q q q qq q q q q qq q q q q qq q q q q q q qq q q q q q q qq q qq q q q q q q q q qq q q q qqqqq qqq qqqqq qq q q qqq q q qq q qq q q qq qq qq q q q q q q q q qq q q q qqq q q qq q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q qq q qq q q q q q qq q q q q q q q q q q qq q q qq q q q qq q q q q qq q q qq q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q qq q qq q q q q q q q q q q q q q q q q q qq q q qq q q q q q q qq q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q q q q qq q q q qq q q qq q q q q q qq q q q q q qq q q q qq qq q q q q q q q q q q q q q qq q q q q q q qqq qq q q q q qq q qqq qq q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q qq q q q qq qq q q q q q q q q qq q q qq q q q q q q q q q q q q q q qq q q qq q q q q q qq q q qq q qq q q q q qq q q q q q q q qq q q q q q q qq q q q q q q q q q q q q q q qq q q qq qqq qq q q q q q q qq q q q qq q q q q q q q q q q qq qq q q qqq q qqq q q qq q q q q qq q qq q q q q q q q q q q q q q qq q q q q q q q q q qq q qq q qq q q qq qq q q q q q q q q q q q q qq q qq q q q qq q q q q q qqq qq q q qq q q q q q q q q q q q q q q q q q q q q qqq q q q q qq q q q qq q q q q qq q q qq q q q q q q q q q q qq q q qq q q q q q q q q qq q q q q qq q q qq q q q q q q qq q q q qq q q q q qqq q q q qq q q q q q q q q q q q q q q q qq q q q q qq qq q q qq q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q qq q qq qq q qq qq q q q q qq q q q q q q qq q q q q qq q qq q q q q q q q q q q q q q q qq q q q q q q q qq q q q q q q q q q q q qq q qq q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q qq qq q q q qqqq qq q q q qqqq q q q q q q qqq q q q q q qqq qqq q q q q q q q q q q q q qq q q qq qq q qq q q q q qq q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q qq qq qq qq q q q q qq qq q q q q q q qq q q q q q q q qq q q qq q q qqq q q q qq q q qq qq q q q q q q q q q q qq q q qq qqq q q q qq qq q q qq q q q q q q q q q q q q q q qq q q qqq q q q q q q q q q q q q q qq q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q qqq q q q q q q q q q q q q qqqqq q q qq q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q qqq q qqq q qq q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q qq q q q q qq q q qqq q q q q q q q q q q q q q q q q q q q q q qq q qq q q q q qq q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q qqq q qqq q q q q q q q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q qq q q q q q q q qqq q q q q q q q q q q qq q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q qq qq qq q q q q q q q q q q q q q q q q q qqq q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q qq q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q q q qq q q qq q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q qqq q q q q q q q q q q q q q q qq q q q q q q q q qq q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q qqq q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q qq qq qq q qq qq q qqq q qq q qq q q q qq q q qq q qq q q q q q q qqq qqq q qqq q qq q q q q q qq qq qqqq q qq q qq q q q q q q qq qq q q q q qq q q q q q q q qqqq qqqqqqq q q q q qq q q qq q qq q qq qq qq q qqq q q q q q q q q q q q qqqqqq qq q q q qq q q q q q qqq q qq q q q qq q qq q q qqq qq q qq q q q q q q q q q qq q qq q q q q q q q q qqq q q q q q q q q qq qqq q q q q q q q q q q q q qq q q q q q q q q q q q qq qq q q q q q qq q q q q q q q qqq q qq q q qq q q q q q q q q q q qq qq q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q q q q q q qq q q q qq q q q q qq qq qq q q q q q q qq q q q q q q qq q q q q q q q q qqq q qq qqq qq q q q qq q q q q q q qqq q qq q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q qq q qq q qq qqq q q qq qqq q q q qq q q q q q qq qq q qqq qqqq qqq q q qqq qq qq q q q q qq q q q q q q q q q q q qq q q qq q q q q qq qq q q qq q q q q q q q qq qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q qq q q q q q q q qq q q q qq q q q q qqq q q q q qq q q q qq q qq q q q q q q q q q q q q q qq q q q q q q q q q q q qq q q q qq qq qq q qqq q q q q qqqq qqqqq q q qq q q qq qq q qq q q q q q q q qq q q qq q q q qqq q q q q q qq q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q qq q q q qq q q qq q q q q qq q q qqq q qq q q q q qq q q qq q q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q qqq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q qq q q q q qq qq qq q q q q q q q qq q q q q q q q q q q q q qq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q q q q q qq qq q q q q q q q q q qq q q qq q qq q q qqqq q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q qq q qq q qq q qq q qq qqqq qq qqqqq q q qq q q q q qq qq q q qq q qq q q qq q q q q q qqq q q q q q q q q qq q q qqqq q q qq q qq qq q q q q q q qq q q q q q qq qq q q qq q qqq q q q q q qq q q q q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q qq q qq qq qqq q q q q q q q q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q qq q q q q q q q q q q q q q q qq q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qqq q q qqq q q q q q qq q q q qq qqq q q q qq qq q q q q q q q qq qq q q q q q qq q q q q qqqq qqq qq q qq q q q q q qq q qq q q q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq qq q q q q qq q q q q q q q q q q q q q q qqqq q q qq qqqqq q q q qq q q q q q qq qq qqq q q q qq q q q q q qq qqq qqq q qqqqq qqqqqq qq q q q q qq q q q qq qq q qqqq q q qqq q qqq q q qqqq qq q q q q q q q q qq q q q q qq qq qqq qq qqqqq q q qq q q q qq qqq q q q q q q qq qq q q qq qqq q q qq q q q q qq q q q q q q q q q q qq qq q q qq qq q q q qqq qq q qqq qq qqqq q q q q q q qq q qq qq qqq q q qq q qqq qq qqqqqq q q qq q qqqqq qqq q qqq q qq qq q q q q q q q q q q q q q q q q q q q q qq q q q qq qqq q q q q q q q q q q q q q q q q q q q qq q qqqq qq q qq qq q q q q q q q q q q q qq qqq q q qq qq q q q q q q qq q q q qq q q q q q q q q q q q q q qq qq q q q q q q qq q q q q q q qqqq q q q q q qq q q q q q q q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q qq qq q q q q qqq q q q q qq q q q q q qq q q q qqqq q q q q q qqqqq q q q q q q q q q q q q q qq q q q qqq q q qqqq qqq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q qq q q q q q q q q q q q q q q q q qqq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q qq q q q q q q q q q qq q q qq q q q q q q q q q qq q q q q q q q q q q q qq q q qq q q qq q qq q qq q q q q q q q q q q q q q q qqq q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qqq q q q q q q q q qq q q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q q q qqq qq q qq q q qq q q q q q q q q q qq q qq q q q q q qq qq q qq q q q q q qq q q q q q q q q q q qq q q q qq qq q q qqq q q q qq qq q q q q q q q q q q qq q q q q q q qq q qqqq qq q q qq q qqq q q q q qq qq q q q q q q q qq q q q q q qq qqq qq qq q q q qq qq q q q qq q q q qqq q q qq q q q q q q q qqqqq q q q q qq q q q qq q q q q q q qqqq q q qq q qq q q q q q qq q qq qq qq q q q q qq qq q q qq qq q q q q q q q q qq qqq q q q qq qqq qq q q q q qq q q q q qq q q q q q qq q qq q q qq q q q q q qq q q qq q q q q q q q q q q q q q qq q q q q q qqq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q qqq q q q q q qq q q q q q q q q q q q q q q q qq q q q q q qqq q q q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqqq q q qq qq q q qq q qq q q q q qq q q q qq q q qq q q q q q qq q q q q qq q q q qq q q q q qq q q q qqq q q q q qq qqq qqq q q qq q q q qq q qq qqq q q q q q qq q q qq qq q q qq q q q q qqqqq qq q qqqq q q q q qq q q q q q qq q q q qq q qqq q q q q q q q q qq q q qq q q q q q q q q q q q qq q q q q q q q q qq qq qq qq q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q qq q qq q q q qqq qqqq q q q q q q q q qqqq qqq q qq q q qqq q q q q q q qqq q q q q qqq q q q q q qq q q qq q q qq q q qq q q q q q qqq q q q q q qqq q q q q q q q q qq q qq q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q qq q qq q q qq q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q q qqq q q q q qq qq q q q q q q qqqq q q q q qq qqq q q q qq q q q qqq q q q q qq q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q ACF1 lumpiness entropy lshiftvchange cpoints fspots trend linearity curvature spikiness season peaktrough klscore change.idx −4 −2 0 2 −2.5 0.0 2.5 standardized PC1 (28.7% explained var.) standardizedPC2(17.3%explainedvar.) Exploring the feature space of large time series collections Yahoo web traffic 19
  • 42. What is “anomalous” q q qq q q q q q q q qq q q q q qq q qq q q q q q q qq q q q q qqq q qq q q q q q q q q q q q q q q q q q q q qqq q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q qqq q q qq q q q q qq q q q q q qq q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q qq q qq q q q q q q qq q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qqq q q q q qq qqqqq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q qq qq qq q q q q q q q qq q q q q q q q q q qqq q qqqqq q q q q q q q qqq q q q q q qq qq q qq qqqq qq q q q q q q q q qq q qqq qqqq qqq q q q qq q q q q q q q qq qq q q q q q q q q q qq q qqq q q q q q q q q q q q q q q q q q q q qq q q q q qq q q q q q qq q q q q q q q qq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q qq q q qq q q q q q q q q q q q q q q qq q q q q q q q q q qq qq q q q q qq q q qq q q q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q qqqq q q q q q q q q qq qq q qqq q q q qq q q q q q q q q q q q q q q q q qqq q q qq qq q q q q qqq q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q qq q q q q q q q qq q q q q qq q q q qq q q qq q q qq q q q q q q q q q qq q qqq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q qqq q q q q q q qq q q q q q q q q q qqq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q q q q q q q q q q qq q q q q qq qq q q q q q q q q q q q q qqq q q q q q q q q qq q q q q q q q q q q q q q q qq q qq q q q q q q q q q q q q q qq q qq qq q q q q q q q q q q q q qq q q q q q q q q q qq q q q qq q q q qq q q q qqq q q q q q q q q q qqq qq q qq q q qq q qq qqqq qqq q q qqq q qq q q qq q q q q q q q q q q q q q q qq q q q q qq qq q q q q qq qq qq q q q q q q q q q q q q q q q q q q qqqq q q q q q qq q q qq qq qq q q qq qq q qq q q q q q q q q q q q q q q q q q q q qq qqq q qq qq q q q q qq qq q q qq q qq q q q q q q q q q qq q q q q q q q q q qq q q q q q q qq q qq qq qq q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q qqq q q q q q q qq q qq q q qq q q q q qq q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q qq q qq q q qqqq q q q q q q q q q q q q q q q q q q q qq q q q q qq q q q q q q q q q q q q qq q q q qq qqq q q q q q q q q q q q qq q qq q q q q q q qq q qq q qq q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q qqq q qq q q q q q q q qqq q q q q q q q q q qq q qq q q q q q q q q q q q q q q q q qq qq q q qq q q q q q q q q q q q qq q q q q q q q q q qq q q qq q q q q q q qq q q q q qqqq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q q q q qqq qq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q qq q qq q q q q q q qq q qq q q qq q q q q q q qq qq q q q q q q q q q q q qq q q q q q q qq q q q q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q qq q q q q qqq q q qq q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q qq q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q qq q q q q q q q q q q q q q q q q q q qq q q q q qq q q q q q q q qq q q qq q q q q q q q q q q q q qqq q qq q qq qq qq q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q qq q q q q qq q q q qqq q q q q q q q q q q q q q q q q q q q q q q qq q q q q qq q q qq qq q q q q q q q q q q q q q q q q qq q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq qq q q q q q q q q qq qq q q q q q q q q qq q q q q q q qq q q qq q q q q q q q q q q q q qq q qq q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q q q qq q q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q qq q q q q q q q q q q q qq q q q q q qq q q qqq q q q q q q q q q q q q q q q q q q q q q q qq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q qq q q q q q q qq q q qq q q q q q q q q q q q qqq q q q q q q q qq q q q q qq q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q qq q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q q q qq q q q q q q q q q qq q qq q q q q q q q q qq q q q qq q q q q q q q q qq q q q qqqq qq q q q qq q q q q q qq qq q q q q qq q q q q q q qq q q q q qq q qq q q q q q q q q q q q q qq q qq qq qqq q q q qq q q q q q q q q q q q qq q q q q q qq q q q qqq q q q q q q q qq q q q q q q q q q q q q q q q qq q q qq q q q qq q q q q qq q q q q q q q q q qq qqq q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q q qq qq q q q q qq q q q q q q q q qq q q q q q qq q q q q q qq q q q q q q q q q qq q qq q q q q q q q q q q q q q q q q qq q q qq q q q q q q q q q q q q q q q qqqq qq q q qq q q qq q q q qq q q qq q q qq q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q qqq q q q q q q q q q q q q qq q q q q q qq q q q q qq q qq qq qq q qq q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q qq qq q qqq q q qqqqq q q q q qq q q qq qq q q q q q q qqq q q q q q qq q q qq q q q q q qq q q q q q qq q q q q q q qq qqqq qq q q q q qq q q q q q q q q q q q q qq qq qqqqqq q q q qq q q q q qq q qq qqq q qqqq q q q q q q q q q q q q qq q q q q q q q qq q qq q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q qq q q qq q q q q q q qq qq q q q qq q q q q qq q q q q q q q q q q q qq qq q q q q q q q qqq q q q q q q q q q qq q q q q q q q qq q q q q qq q q qq q q q q q q qq q q q q q q q q q q q qq q q q qq qqq qq q q qq q q qqq q q qq q qq q q q qqq q q q q q qq q q q q q qq q q q q q q q q q q qqq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q qqq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q q q q q q q q q q q q q q qq q q q q q qqq q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q qq q q qq q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q q q q qqq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q qqq q q q q q q q q q q q q q q qq q q qq q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qqq q q q q q q q q qq qq q q q q qq q q q q q q q q q q q qq q q q qq q q q q q q q q q q qq qq qq q q q q q q q q qq qq q q q q q q q q qq q q qq q q q q q q qq q q q q q q qq q q q q q q q q q q q q q q qq q q q q q q qq q qq q q q qq q q q qq q q q q q q q q q qq q q q q q qqq q q q q q q q q q q qq q q q q qq q q q q qq q q q q q q qq q q q q q q qq q qq q q q q q q q q qq q q q qqqqq qqq qqqqq qq q q qqq q q qq q qq q q qq qq qq q q q q q q q q qq q q q qqq q q qq q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q qq q qq q q q q q qq q q q q q q q q q q qq q q qq q q q qq q q q q qq q q qq q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q qq q qq q q q q q q q q q q q q q q q q q qq q q qq q q q q q q qq q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q q q q qq q q q qq q q qq q q q q q qq q q q q q qq q q q qq qq q q q q q q q q q q q q q qq q q q q q q qqq qq q q q q qq q qqq qq q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q qq q q q qq qq q q q q q q q q qq q q qq q q q q q q q q q q q q q q qq q q qq q q q q q qq q q qq q qq q q q q qq q q q q q q q qq q q q q q q qq q q q q q q q q q q q q q q qq q q qq qqq qq q q q q q q qq q q q qq q q q q q q q q q q qq qq q q qqq q qqq q q qq q q q q qq q qq q q q q q q q q q q q q q qq q q q q q q q q q qq q qq q qq q q qq qq q q q q q q q q q q q q qq q qq q q q qq q q q q q qqq qq q q qq q q q q q q q q q q q q q q q q q q q q qqq q q q q qq q q q qq q q q q qq q q qq q q q q q q q q q q qq q q qq q q q q q q q q qq q q q q qq q q qq q q q q q q qq q q q qq q q q q qqq q q q qq q q q q q q q q q q q q q q q qq q q q q qq qq q q qq q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q qq q qq qq q qq qq q q q q qq q q q q q q qq q q q q qq q qq q q q q q q q q q q q q q q qq q q q q q q q qq q q q q q q q q q q q qq q qq q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q qq qq q q q qqqq qq q q q qqqq q q q q q q qqq q q q q q qqq qqq q q q q q q q q q q q q qq q q qq qq q qq q q q q qq q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q qq qq qq qq q q q q qq qq q q q q q q qq q q q q q q q qq q q qq q q qqq q q q qq q q qq qq q q q q q q q q q q qq q q qq qqq q q q qq qq q q qq q q q q q q q q q q q q q q qq q q qqq q q q q q q q q q q q q q qq q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q qqq q q q q q q q q q q q q qqqqq q q qq q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q qqq q qqq q qq q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q qq q q q q qq q q qqq q q q q q q q q q q q q q q q q q q q q q qq q qq q q q q qq q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q qqq q qqq q q q q q q q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q qq q q q q q q q qqq q q q q q q q q q q qq q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q qq qq qq q q q q q q q q q q q q q q q q q qqq q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q qq q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q q q qq q q qq q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q qqq q q q q q q q q q q q q q q qq q q q q q q q q qq q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q qqq q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q qq qq qq q qq qq q qqq q qq q qq q q q qq q q qq q qq q q q q q q qqq qqq q qqq q qq q q q q q qq qq qqqq q qq q qq q q q q q q qq qq q q q q qq q q q q q q q qqqq qqqqqqq q q q q qq q q qq q qq q qq qq qq q qqq q q q q q q q q q q q qqqqqq qq q q q qq q q q q q qqq q qq q q q qq q qq q q qqq qq q qq q q q q q q q q q qq q qq q q q q q q q q qqq q q q q q q q q qq qqq q q q q q q q q q q q q qq q q q q q q q q q q q qq qq q q q q q qq q q q q q q q qqq q qq q q qq q q q q q q q q q q qq qq q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q q q q q q qq q q q qq q q q q qq qq qq q q q q q q qq q q q q q q qq q q q q q q q q qqq q qq qqq qq q q q qq q q q q q q qqq q qq q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q qq q qq q qq qqq q q qq qqq q q q qq q q q q q qq qq q qqq qqqq qqq q q qqq qq qq q q q q qq q q q q q q q q q q q qq q q qq q q q q qq qq q q qq q q q q q q q qq qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q qq q q q q q q q qq q q q qq q q q q qq q q q q q qq q q q qq q qq q q q q q q q q q q q q q qq q q q q q q q q q q q qq q q q qq qq qq q qqq q q q q qqqq qqqqq q q qq q q qq qq q qq q q q q q q q qq q q qq q q q qqq q q q q q qq q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q qq q q q qq q q qq q q q q qq q q qqq q qq q q q q qq q q qq q q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q qqq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q qq q q q q qq qq qq q q q q q q q qq q q q q q q q q q q q q qq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q q q q q qq qq q q q q q q q q q qq q q qq q qq q q qqqq q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q qq q qq q qq q qq q qq qqqq qq qqqqq q q qq q q q q qq qq q q qq q qq q q qq q q q q q qqq q q q q q q q q qq q q qqqq q q qq q qq qq q q q q q q qq q q q q q qq qq q q qq q qqq q q q q q qq q q q q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q qq q qq qq qqq q q q q q q q q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q qq q q q q q q q q q q q q q q qq q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qqq q q qqq q q q q q qq q q q qq qqq q q q qq qq q q q q q q q qq qq q q q q q qq q q q q qqqq qqq qq q qq q q q q q qq q qq q q q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq qq q q q q qq q q q q q q q q q q q q q q qqqq q q qq qqqqq q q q qq q q q q q qq qq qqq q q q qq q q q q q qq qqq qqq q qqqqq qqqqqq qq q q q q qq q q q qq qq q qqqq q q qqq q qqq q q qqqq qq q q q q q q q q qq q q q q qq qq qqq qq qqqqq q q qq q q q qq qqq q q q q q q qq qq q q qq qqq q q qq q q q q qq q q q q q q q q q q qq qq q q qq qq q q q qqq qq q qqq qq qqqq q q q q q q qq q qq qq qqq q q qq q qqq qq qqqqqq q q qq q qqqqq qqq q qqq q qq qq q q q q q q q q q q q q q q q q q q q q qq q q q qq qqq q q q q q q q q q q q q q q q q q q q qq q qqqq qq q qq qq q q q q q q q q q q q qq qqq q q qq qq q q q q q q qq q q q qq q q q q q q q q q q q q q qq qq q q q q q q qq q q q q q q qqqq q q q q q qq q q q q q q q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q qq qq q q q q qqq q q q q qq q q q q q qq q q q qqqq q q q q q qqqqq q q q q q q q q q q q q q qq q q q qqq q q qqqq qqq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q qq q q q q q q q q q q q q q q q q qqq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q qq q q q q q q q q q qq q q qq q q q q q q q q q qq q q q q q q q q q q q qq q q qq q q qq q qq q qq q q q q q q q q q q q q q q qqq q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qqq q q q q q q q q qq q q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q q q qqq qq q qq q q qq q q q q q q q q q qq q qq q q q q q qq qq q qq q q q q q qq q q q q q q q q q q qq q q q qq qq q q qqq q q q qq qq q q q q q q q q q q qq q q q q q q qq q qqqq qq q q qq q qqq q q q q qq qq q q q q q q q qq q q q q q qq qqq qq qq q q q qq qq q q q qq q q q qqq q q qq q q q q q q q qqqqq q q q q qq q q q qq q q q q q q qqqq q q qq q qq q q q q q qq q qq qq qq q q q q qq qq q q qq qq q q q q q q q q qq qqq q q q qq qqq qq q q q q qq q q q q qq q q q q q qq q qq q q qq q q q q q qq q q qq q q q q q q q q q q q q q qq q q q q q qqq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q qqq q q q q q qq q q q q q q q q q q q q q q q qq q q q q q qqq q q q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqqq q q qq qq q q qq q qq q q q q qq q q q qq q q qq q q q q q qq q q q q qq q q q qq q q q q qq q q q qqq q q q q qq qqq qqq q q qq q q q qq q qq qqq q q q q q qq q q qq qq q q qq q q q q qqqqq qq q qqqq q q q q qq q q q q q qq q q q qq q qqq q q q q q q q q qq q q qq q q q q q q q q q q q qq q q q q q q q q qq qq qq qq q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q qq q qq q q q qqq qqqq q q q q q q q q qqqq qqq q qq q q qqq q q q q q q qqq q q q q qqq q q q q q qq q q qq q q qq q q qq q q q q q qqq q q q q q qqq q q q q q q q q qq q qq q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q qq q qq q q qq q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q q qqq q q q q qq qq q q q q q q qqqq q q q q qq qqq q q q qq q q q qqq q q q q qq q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q ACF1 lumpiness entropy lshiftvchange cpoints fspots trend linearity curvature spikiness season peaktrough klscore change.idx −4 −2 0 2 −2.5 0.0 2.5 standardized PC1 (28.7% explained var.) standardizedPC2(17.3%explainedvar.) We need a measure of the “anomalousness” of a time series. 1 Rank points based on their local density. 2 Rank points based on whether they are within α-convex hulls of different radius. Exploring the feature space of large time series collections Yahoo web traffic 20
  • 43. What is “anomalous” q q qq q q q q q q q qq q q q q qq q qq q q q q q q qq q q q q qqq q qq q q q q q q q q q q q q q q q q q q q qqq q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q qqq q q qq q q q q qq q q q q q qq q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q qq q qq q q q q q q qq q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qqq q q q q qq qqqqq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q qq qq qq q q q q q q q qq q q q q q q q q q qqq q qqqqq q q q q q q q qqq q q q q q qq qq q qq qqqq qq q q q q q q q q qq q qqq qqqq qqq q q q qq q q q q q q q qq qq q q q q q q q q q qq q qqq q q q q q q q q q q q q q q q q q q q qq q q q q qq q q q q q qq q q q q q q q qq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q qq q q qq q q q q q q q q q q q q q q qq q q q q q q q q q qq qq q q q q qq q q qq q q q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q qqqq q q q q q q q q qq qq q qqq q q q qq q q q q q q q q q q q q q q q q qqq q q qq qq q q q q qqq q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q qq q q q q q q q qq q q q q qq q q q qq q q qq q q qq q q q q q q q q q qq q qqq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q qqq q q q q q q qq q q q q q q q q q qqq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q q q q q q q q q q qq q q q q qq qq q q q q q q q q q q q q qqq q q q q q q q q qq q q q q q q q q q q q q q q qq q qq q q q q q q q q q q q q q qq q qq qq q q q q q q q q q q q q qq q q q q q q q q q qq q q q qq q q q qq q q q qqq q q q q q q q q q qqq qq q qq q q qq q qq qqqq qqq q q qqq q qq q q qq q q q q q q q q q q q q q q qq q q q q qq qq q q q q qq qq qq q q q q q q q q q q q q q q q q q q qqqq q q q q q qq q q qq qq qq q q qq qq q qq q q q q q q q q q q q q q q q q q q q qq qqq q qq qq q q q q qq qq q q qq q qq q q q q q q q q q qq q q q q q q q q q qq q q q q q q qq q qq qq qq q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q qqq q q q q q q qq q qq q q qq q q q q qq q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q qq q qq q q qqqq q q q q q q q q q q q q q q q q q q q qq q q q q qq q q q q q q q q q q q q qq q q q qq qqq q q q q q q q q q q q qq q qq q q q q q q qq q qq q qq q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q qqq q qq q q q q q q q qqq q q q q q q q q q qq q qq q q q q q q q q q q q q q q q q qq qq q q qq q q q q q q q q q q q qq q q q q q q q q q qq q q qq q q q q q q qq q q q q qqqq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q q q q qqq qq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q qq q qq q q q q q q qq q qq q q qq q q q q q q qq qq q q q q q q q q q q q qq q q q q q q qq q q q q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q qq q q q q qqq q q qq q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q qq q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q qq q q q q q q q q q q q q q q q q q q qq q q q q qq q q q q q q q qq q q qq q q q q q q q q q q q q qqq q qq q qq qq qq q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q qq q q q q qq q q q qqq q q q q q q q q q q q q q q q q q q q q q q qq q q q q qq q q qq qq q q q q q q q q q q q q q q q q qq q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq qq q q q q q q q q qq qq q q q q q q q q qq q q q q q q qq q q qq q q q q q q q q q q q q qq q qq q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q q q qq q q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q qq q q q q q q q q q q q qq q q q q q qq q q qqq q q q q q q q q q q q q q q q q q q q q q q qq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q qq q q q q q q qq q q qq q q q q q q q q q q q qqq q q q q q q q qq q q q q qq q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q qq q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q q q qq q q q q q q q q q qq q qq q q q q q q q q qq q q q qq q q q q q q q q qq q q q qqqq qq q q q qq q q q q q qq qq q q q q qq q q q q q q qq q q q q qq q qq q q q q q q q q q q q q qq q qq qq qqq q q q qq q q q q q q q q q q q qq q q q q q qq q q q qqq q q q q q q q qq q q q q q q q q q q q q q q q qq q q qq q q q qq q q q q qq q q q q q q q q q qq qqq q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q q qq qq q q q q qq q q q q q q q q qq q q q q q qq q q q q q qq q q q q q q q q q qq q qq q q q q q q q q q q q q q q q q qq q q qq q q q q q q q q q q q q q q q qqqq qq q q qq q q qq q q q qq q q qq q q qq q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q qqq q q q q q q q q q q q q qq q q q q q qq q q q q qq q qq qq qq q qq q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q qq qq q qqq q q qqqqq q q q q qq q q qq qq q q q q q q qqq q q q q q qq q q qq q q q q q qq q q q q q qq q q q q q q qq qqqq qq q q q q qq q q q q q q q q q q q q qq qq qqqqqq q q q qq q q q q qq q qq qqq q qqqq q q q q q q q q q q q q qq q q q q q q q qq q qq q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q qq q q qq q q q q q q qq qq q q q qq q q q q qq q q q q q q q q q q q qq qq q q q q q q q qqq q q q q q q q q q qq q q q q q q q qq q q q q qq q q qq q q q q q q qq q q q q q q q q q q q qq q q q qq qqq qq q q qq q q qqq q q qq q qq q q q qqq q q q q q qq q q q q q qq q q q q q q q q q q qqq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q qqq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q q q q q q q q q q q q q q qq q q q q q qqq q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q qq q q qq q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q q q q qqq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q qqq q q q q q q q q q q q q q q qq q q qq q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qqq q q q q q q q q qq qq q q q q qq q q q q q q q q q q q qq q q q qq q q q q q q q q q q qq qq qq q q q q q q q q qq qq q q q q q q q q qq q q qq q q q q q q qq q q q q q q qq q q q q q q q q q q q q q q qq q q q q q q qq q qq q q q qq q q q qq q q q q q q q q q qq q q q q q qqq q q q q q q q q q q qq q q q q qq q q q q qq q q q q q q qq q q q q q q qq q qq q q q q q q q q qq q q q qqqqq qqq qqqqq qq q q qqq q q qq q qq q q qq qq qq q q q q q q q q qq q q q qqq q q qq q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q qq q qq q q q q q qq q q q q q q q q q q qq q q qq q q q qq q q q q qq q q qq q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q qq q qq q q q q q q q q q q q q q q q q q qq q q qq q q q q q q qq q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q q q q qq q q q qq q q qq q q q q q qq q q q q q qq q q q qq qq q q q q q q q q q q q q q qq q q q q q q qqq qq q q q q qq q qqq qq q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q qq q q q qq qq q q q q q q q q qq q q qq q q q q q q q q q q q q q q qq q q qq q q q q q qq q q qq q qq q q q q qq q q q q q q q qq q q q q q q qq q q q q q q q q q q q q q q qq q q qq qqq qq q q q q q q qq q q q qq q q q q q q q q q q qq qq q q qqq q qqq q q qq q q q q qq q qq q q q q q q q q q q q q q qq q q q q q q q q q qq q qq q qq q q qq qq q q q q q q q q q q q q qq q qq q q q qq q q q q q qqq qq q q qq q q q q q q q q q q q q q q q q q q q q qqq q q q q qq q q q qq q q q q qq q q qq q q q q q q q q q q qq q q qq q q q q q q q q qq q q q q qq q q qq q q q q q q qq q q q qq q q q q qqq q q q qq q q q q q q q q q q q q q q q qq q q q q qq qq q q qq q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q qq q qq qq q qq qq q q q q qq q q q q q q qq q q q q qq q qq q q q q q q q q q q q q q q qq q q q q q q q qq q q q q q q q q q q q qq q qq q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q qq qq q q q qqqq qq q q q qqqq q q q q q q qqq q q q q q qqq qqq q q q q q q q q q q q q qq q q qq qq q qq q q q q qq q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q qq qq qq qq q q q q qq qq q q q q q q qq q q q q q q q qq q q qq q q qqq q q q qq q q qq qq q q q q q q q q q q qq q q qq qqq q q q qq qq q q qq q q q q q q q q q q q q q q qq q q qqq q q q q q q q q q q q q q qq q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q qqq q q q q q q q q q q q q qqqqq q q qq q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q qqq q qqq q qq q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q qq q q q q qq q q qqq q q q q q q q q q q q q q q q q q q q q q qq q qq q q q q qq q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q qqq q qqq q q q q q q q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q qq q q q q q q q qqq q q q q q q q q q q qq q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q qq qq qq q q q q q q q q q q q q q q q q q qqq q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q qq q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q q q qq q q qq q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q qqq q q q q q q q q q q q q q q qq q q q q q q q q qq q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q qqq q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q qq qq qq q qq qq q qqq q qq q qq q q q qq q q qq q qq q q q q q q qqq qqq q qqq q qq q q q q q qq qq qqqq q qq q qq q q q q q q qq qq q q q q qq q q q q q q q qqqq qqqqqqq q q q q qq q q qq q qq q qq qq qq q qqq q q q q q q q q q q q qqqqqq qq q q q qq q q q q q qqq q qq q q q qq q qq q q qqq qq q qq q q q q q q q q q qq q qq q q q q q q q q qqq q q q q q q q q qq qqq q q q q q q q q q q q q qq q q q q q q q q q q q qq qq q q q q q qq q q q q q q q qqq q qq q q qq q q q q q q q q q q qq qq q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q q q q q q qq q q q qq q q q q qq qq qq q q q q q q qq q q q q q q qq q q q q q q q q qqq q qq qqq qq q q q qq q q q q q q qqq q qq q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q qq q qq q qq qqq q q qq qqq q q q qq q q q q q qq qq q qqq qqqq qqq q q qqq qq qq q q q q qq q q q q q q q q q q q qq q q qq q q q q qq qq q q qq q q q q q q q qq qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q qq q q q q q q q qq q q q qq q q q q qq q q q q q qq q q q qq q qq q q q q q q q q q q q q q qq q q q q q q q q q q q qq q q q qq qq qq q qqq q q q q qqqq qqqqq q q qq q q qq qq q qq q q q q q q q qq q q qq q q q qqq q q q q q qq q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q qq q q q qq q q qq q q q q qq q q qqq q qq q q q q qq q q qq q q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q qqq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q qq q q q q qq qq qq q q q q q q q qq q q q q q q q q q q q q qq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q q q q q qq qq q q q q q q q q q qq q q qq q qq q q qqqq q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q qq q qq q qq q qq q qq qqqq qq qqqqq q q qq q q q q qq qq q q qq q qq q q qq q q q q q qqq q q q q q q q q qq q q qqqq q q qq q qq qq q q q q q q qq q q q q q qq qq q q qq q qqq q q q q q qq q q q q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q qq q qq qq qqq q q q q q q q q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q qq q q q q q q q q q q q q q q qq q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qqq q q qqq q q q q q qq q q q qq qqq q q q qq qq q q q q q q q qq qq q q q q q qq q q q q qqqq qqq qq q qq q q q q q qq q qq q q q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq qq q q q q qq q q q q q q q q q q q q q q qqqq q q qq qqqqq q q q qq q q q q q qq qq qqq q q q qq q q q q q qq qqq qqq q qqqqq qqqqqq qq q q q q qq q q q qq qq q qqqq q q qqq q qqq q q qqqq qq q q q q q q q q qq q q q q qq qq qqq qq qqqqq q q qq q q q qq qqq q q q q q q qq qq q q qq qqq q q qq q q q q qq q q q q q q q q q q qq qq q q qq qq q q q qqq qq q qqq qq qqqq q q q q q q qq q qq qq qqq q q qq q qqq qq qqqqqq q q qq q qqqqq qqq q qqq q qq qq q q q q q q q q q q q q q q q q q q q q qq q q q qq qqq q q q q q q q q q q q q q q q q q q q qq q qqqq qq q qq qq q q q q q q q q q q q qq qqq q q qq qq q q q q q q qq q q q qq q q q q q q q q q q q q q qq qq q q q q q q qq q q q q q q qqqq q q q q q qq q q q q q q q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q qq qq q q q q qqq q q q q qq q q q q q qq q q q qqqq q q q q q qqqqq q q q q q q q q q q q q q qq q q q qqq q q qqqq qqq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q qq q q q q q q q q q q q q q q q q qqq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q qq q q q q q q q q q qq q q qq q q q q q q q q q qq q q q q q q q q q q q qq q q qq q q qq q qq q qq q q q q q q q q q q q q q q qqq q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qqq q q q q q q q q qq q q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q q q qqq qq q qq q q qq q q q q q q q q q qq q qq q q q q q qq qq q qq q q q q q qq q q q q q q q q q q qq q q q qq qq q q qqq q q q qq qq q q q q q q q q q q qq q q q q q q qq q qqqq qq q q qq q qqq q q q q qq qq q q q q q q q qq q q q q q qq qqq qq qq q q q qq qq q q q qq q q q qqq q q qq q q q q q q q qqqqq q q q q qq q q q qq q q q q q q qqqq q q qq q qq q q q q q qq q qq qq qq q q q q qq qq q q qq qq q q q q q q q q qq qqq q q q qq qqq qq q q q q qq q q q q qq q q q q q qq q qq q q qq q q q q q qq q q qq q q q q q q q q q q q q q qq q q q q q qqq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q qqq q q q q q qq q q q q q q q q q q q q q q q qq q q q q q qqq q q q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqqq q q qq qq q q qq q qq q q q q qq q q q qq q q qq q q q q q qq q q q q qq q q q qq q q q q qq q q q qqq q q q q qq qqq qqq q q qq q q q qq q qq qqq q q q q q qq q q qq qq q q qq q q q q qqqqq qq q qqqq q q q q qq q q q q q qq q q q qq q qqq q q q q q q q q qq q q qq q q q q q q q q q q q qq q q q q q q q q qq qq qq qq q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q qq q qq q q q qqq qqqq q q q q q q q q qqqq qqq q qq q q qqq q q q q q q qqq q q q q qqq q q q q q qq q q qq q q qq q q qq q q q q q qqq q q q q q qqq q q q q q q q q qq q qq q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q qq q qq q q qq q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q q qqq q q q q qq qq q q q q q q qqqq q q q q qq qqq q q q qq q q q qqq q q q q qq q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q ACF1 lumpiness entropy lshiftvchange cpoints fspots trend linearity curvature spikiness season peaktrough klscore change.idx −4 −2 0 2 −2.5 0.0 2.5 standardized PC1 (28.7% explained var.) standardizedPC2(17.3%explainedvar.) We need a measure of the “anomalousness” of a time series. 1 Rank points based on their local density. 2 Rank points based on whether they are within α-convex hulls of different radius. Exploring the feature space of large time series collections Yahoo web traffic 20
  • 44. What is “anomalous” q q qq q q q q q q q qq q q q q qq q qq q q q q q q qq q q q q qqq q qq q q q q q q q q q q q q q q q q q q q qqq q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q qqq q q qq q q q q qq q q q q q qq q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q qq q qq q q q q q q qq q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qqq q q q q qq qqqqq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q qq qq qq q q q q q q q qq q q q q q q q q q qqq q qqqqq q q q q q q q qqq q q q q q qq qq q qq qqqq qq q q q q q q q q qq q qqq qqqq qqq q q q qq q q q q q q q qq qq q q q q q q q q q qq q qqq q q q q q q q q q q q q q q q q q q q qq q q q q qq q q q q q qq q q q q q q q qq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q qq q q qq q q q q q q q q q q q q q q qq q q q q q q q q q qq qq q q q q qq q q qq q q q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q qqqq q q q q q q q q qq qq q qqq q q q qq q q q q q q q q q q q q q q q q qqq q q qq qq q q q q qqq q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q qq q q q q q q q qq q q q q qq q q q qq q q qq q q qq q q q q q q q q q qq q qqq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q qqq q q q q q q qq q q q q q q q q q qqq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q q q q q q q q q q qq q q q q qq qq q q q q q q q q q q q q qqq q q q q q q q q qq q q q q q q q q q q q q q q qq q qq q q q q q q q q q q q q q qq q qq qq q q q q q q q q q q q q qq q q q q q q q q q qq q q q qq q q q qq q q q qqq q q q q q q q q q qqq qq q qq q q qq q qq qqqq qqq q q qqq q qq q q qq q q q q q q q q q q q q q q qq q q q q qq qq q q q q qq qq qq q q q q q q q q q q q q q q q q q q qqqq q q q q q qq q q qq qq qq q q qq qq q qq q q q q q q q q q q q q q q q q q q q qq qqq q qq qq q q q q qq qq q q qq q qq q q q q q q q q q qq q q q q q q q q q qq q q q q q q qq q qq qq qq q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q qqq q q q q q q qq q qq q q qq q q q q qq q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q qq q qq q q qqqq q q q q q q q q q q q q q q q q q q q qq q q q q qq q q q q q q q q q q q q qq q q q qq qqq q q q q q q q q q q q qq q qq q q q q q q qq q qq q qq q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q qqq q qq q q q q q q q qqq q q q q q q q q q qq q qq q q q q q q q q q q q q q q q q qq qq q q qq q q q q q q q q q q q qq q q q q q q q q q qq q q qq q q q q q q qq q q q q qqqq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q q q q qqq qq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q qq q qq q q q q q q qq q qq q q qq q q q q q q qq qq q q q q q q q q q q q qq q q q q q q qq q q q q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q qq q q q q qqq q q qq q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q qq q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q qq q q q q q q q q q q q q q q q q q q qq q q q q qq q q q q q q q qq q q qq q q q q q q q q q q q q qqq q qq q qq qq qq q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q qq q q q q qq q q q qqq q q q q q q q q q q q q q q q q q q q q q q qq q q q q qq q q qq qq q q q q q q q q q q q q q q q q qq q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq qq q q q q q q q q qq qq q q q q q q q q qq q q q q q q qq q q qq q q q q q q q q q q q q qq q qq q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q q q qq q q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q qq q q q q q q q q q q q qq q q q q q qq q q qqq q q q q q q q q q q q q q q q q q q q q q q qq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q qq q q q q q q qq q q qq q q q q q q q q q q q qqq q q q q q q q qq q q q q qq q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q qq q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q q q qq q q q q q q q q q qq q qq q q q q q q q q qq q q q qq q q q q q q q q qq q q q qqqq qq q q q qq q q q q q qq qq q q q q qq q q q q q q qq q q q q qq q qq q q q q q q q q q q q q qq q qq qq qqq q q q qq q q q q q q q q q q q qq q q q q q qq q q q qqq q q q q q q q qq q q q q q q q q q q q q q q q qq q q qq q q q qq q q q q qq q q q q q q q q q qq qqq q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q q qq qq q q q q qq q q q q q q q q qq q q q q q qq q q q q q qq q q q q q q q q q qq q qq q q q q q q q q q q q q q q q q qq q q qq q q q q q q q q q q q q q q q qqqq qq q q qq q q qq q q q qq q q qq q q qq q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q qqq q q q q q q q q q q q q qq q q q q q qq q q q q qq q qq qq qq q qq q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q qq qq q qqq q q qqqqq q q q q qq q q qq qq q q q q q q qqq q q q q q qq q q qq q q q q q qq q q q q q qq q q q q q q qq qqqq qq q q q q qq q q q q q q q q q q q q qq qq qqqqqq q q q qq q q q q qq q qq qqq q qqqq q q q q q q q q q q q q qq q q q q q q q qq q qq q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q qq q q qq q q q q q q qq qq q q q qq q q q q qq q q q q q q q q q q q qq qq q q q q q q q qqq q q q q q q q q q qq q q q q q q q qq q q q q qq q q qq q q q q q q qq q q q q q q q q q q q qq q q q qq qqq qq q q qq q q qqq q q qq q qq q q q qqq q q q q q qq q q q q q qq q q q q q q q q q q qqq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q qqq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q q q q q q q q q q q q q q qq q q q q q qqq q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q qq q q qq q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q q q q qqq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q qqq q q q q q q q q q q q q q q qq q q qq q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qqq q q q q q q q q qq qq q q q q qq q q q q q q q q q q q qq q q q qq q q q q q q q q q q qq qq qq q q q q q q q q qq qq q q q q q q q q qq q q qq q q q q q q qq q q q q q q qq q q q q q q q q q q q q q q qq q q q q q q qq q qq q q q qq q q q qq q q q q q q q q q qq q q q q q qqq q q q q q q q q q q qq q q q q qq q q q q qq q q q q q q qq q q q q q q qq q qq q q q q q q q q qq q q q qqqqq qqq qqqqq qq q q qqq q q qq q qq q q qq qq qq q q q q q q q q qq q q q qqq q q qq q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q qq q qq q q q q q qq q q q q q q q q q q qq q q qq q q q qq q q q q qq q q qq q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q qq q qq q q q q q q q q q q q q q q q q q qq q q qq q q q q q q qq q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q q q q qq q q q qq q q qq q q q q q qq q q q q q qq q q q qq qq q q q q q q q q q q q q q qq q q q q q q qqq qq q q q q qq q qqq qq q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q qq q q q qq qq q q q q q q q q qq q q qq q q q q q q q q q q q q q q qq q q qq q q q q q qq q q qq q qq q q q q qq q q q q q q q qq q q q q q q qq q q q q q q q q q q q q q q qq q q qq qqq qq q q q q q q qq q q q qq q q q q q q q q q q qq qq q q qqq q qqq q q qq q q q q qq q qq q q q q q q q q q q q q q qq q q q q q q q q q qq q qq q qq q q qq qq q q q q q q q q q q q q qq q qq q q q qq q q q q q qqq qq q q qq q q q q q q q q q q q q q q q q q q q q qqq q q q q qq q q q qq q q q q qq q q qq q q q q q q q q q q qq q q qq q q q q q q q q qq q q q q qq q q qq q q q q q q qq q q q qq q q q q qqq q q q qq q q q q q q q q q q q q q q q qq q q q q qq qq q q qq q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q qq q qq qq q qq qq q q q q qq q q q q q q qq q q q q qq q qq q q q q q q q q q q q q q q qq q q q q q q q qq q q q q q q q q q q q qq q qq q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q qq qq q q q qqqq qq q q q qqqq q q q q q q qqq q q q q q qqq qqq q q q q q q q q q q q q qq q q qq qq q qq q q q q qq q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q qq qq qq qq q q q q qq qq q q q q q q qq q q q q q q q qq q q qq q q qqq q q q qq q q qq qq q q q q q q q q q q qq q q qq qqq q q q qq qq q q qq q q q q q q q q q q q q q q qq q q qqq q q q q q q q q q q q q q qq q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q qqq q q q q q q q q q q q q qqqqq q q qq q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q qqq q qqq q qq q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q qq q q q q qq q q qqq q q q q q q q q q q q q q q q q q q q q q qq q qq q q q q qq q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q qqq q qqq q q q q q q q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q qq q q q q q q q qqq q q q q q q q q q q qq q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q qq qq qq q q q q q q q q q q q q q q q q q qqq q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q qq q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q q q qq q q qq q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q qqq q q q q q q q q q q q q q q qq q q q q q q q q qq q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q qqq q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q qq qq qq q qq qq q qqq q qq q qq q q q qq q q qq q qq q q q q q q qqq qqq q qqq q qq q q q q q qq qq qqqq q qq q qq q q q q q q qq qq q q q q qq q q q q q q q qqqq qqqqqqq q q q q qq q q qq q qq q qq qq qq q qqq q q q q q q q q q q q qqqqqq qq q q q qq q q q q q qqq q qq q q q qq q qq q q qqq qq q qq q q q q q q q q q qq q qq q q q q q q q q qqq q q q q q q q q qq qqq q q q q q q q q q q q q qq q q q q q q q q q q q qq qq q q q q q qq q q q q q q q qqq q qq q q qq q q q q q q q q q q qq qq q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q q q q q q qq q q q qq q q q q qq qq qq q q q q q q qq q q q q q q qq q q q q q q q q qqq q qq qqq qq q q q qq q q q q q q qqq q qq q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q qq q qq q qq qqq q q qq qqq q q q qq q q q q q qq qq q qqq qqqq qqq q q qqq qq qq q q q q qq q q q q q q q q q q q qq q q qq q q q q qq qq q q qq q q q q q q q qq qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q qq q q q q q q q qq q q q qq q q q q qq q q q q q qq q q q qq q qq q q q q q q q q q q q q q qq q q q q q q q q q q q qq q q q qq qq qq q qqq q q q q qqqq qqqqq q q qq q q qq qq q qq q q q q q q q qq q q qq q q q qqq q q q q q qq q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q qq q q q qq q q qq q q q q qq q q qqq q qq q q q q qq q q qq q q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q qqq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q qq q q q q qq qq qq q q q q q q q qq q q q q q q q q q q q q qq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q q q q q qq qq q q q q q q q q q qq q q qq q qq q q qqqq q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q qq q qq q qq q qq q qq qqqq qq qqqqq q q qq q q q q qq qq q q qq q qq q q qq q q q q q qqq q q q q q q q q qq q q qqqq q q qq q qq qq q q q q q q qq q q q q q qq qq q q qq q qqq q q q q q qq q q q q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q qq q qq qq qqq q q q q q q q q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q qq q q q q q q q q q q q q q q qq q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qqq q q qqq q q q q q qq q q q qq qqq q q q qq qq q q q q q q q qq qq q q q q q qq q q q q qqqq qqq qq q qq q q q q q qq q qq q q q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq qq q q q q qq q q q q q q q q q q q q q q qqqq q q qq qqqqq q q q qq q q q q q qq qq qqq q q q qq q q q q q qq qqq qqq q qqqqq qqqqqq qq q q q q qq q q q qq qq q qqqq q q qqq q qqq q q qqqq qq q q q q q q q q qq q q q q qq qq qqq qq qqqqq q q qq q q q qq qqq q q q q q q qq qq q q qq qqq q q qq q q q q qq q q q q q q q q q q qq qq q q qq qq q q q qqq qq q qqq qq qqqq q q q q q q qq q qq qq qqq q q qq q qqq qq qqqqqq q q qq q qqqqq qqq q qqq q qq qq q q q q q q q q q q q q q q q q q q q q qq q q q qq qqq q q q q q q q q q q q q q q q q q q q qq q qqqq qq q qq qq q q q q q q q q q q q qq qqq q q qq qq q q q q q q qq q q q qq q q q q q q q q q q q q q qq qq q q q q q q qq q q q q q q qqqq q q q q q qq q q q q q q q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q qq qq q q q q qqq q q q q qq q q q q q qq q q q qqqq q q q q q qqqqq q q q q q q q q q q q q q qq q q q qqq q q qqqq qqq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q qq q q q q q q q q q q q q q q q q qqq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q qq q q q q q q q q q qq q q qq q q q q q q q q q qq q q q q q q q q q q q qq q q qq q q qq q qq q qq q q q q q q q q q q q q q q qqq q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qqq q q q q q q q q qq q q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q q q qqq qq q qq q q qq q q q q q q q q q qq q qq q q q q q qq qq q qq q q q q q qq q q q q q q q q q q qq q q q qq qq q q qqq q q q qq qq q q q q q q q q q q qq q q q q q q qq q qqqq qq q q qq q qqq q q q q qq qq q q q q q q q qq q q q q q qq qqq qq qq q q q qq qq q q q qq q q q qqq q q qq q q q q q q q qqqqq q q q q qq q q q qq q q q q q q qqqq q q qq q qq q q q q q qq q qq qq qq q q q q qq qq q q qq qq q q q q q q q q qq qqq q q q qq qqq qq q q q q qq q q q q qq q q q q q qq q qq q q qq q q q q q qq q q qq q q q q q q q q q q q q q qq q q q q q qqq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q qqq q q q q q qq q q q q q q q q q q q q q q q qq q q q q q qqq q q q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqqq q q qq qq q q qq q qq q q q q qq q q q qq q q qq q q q q q qq q q q q qq q q q qq q q q q qq q q q qqq q q q q qq qqq qqq q q qq q q q qq q qq qqq q q q q q qq q q qq qq q q qq q q q q qqqqq qq q qqqq q q q q qq q q q q q qq q q q qq q qqq q q q q q q q q qq q q qq q q q q q q q q q q q qq q q q q q q q q qq qq qq qq q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q qq q qq q q q qqq qqqq q q q q q q q q qqqq qqq q qq q q qqq q q q q q q qqq q q q q qqq q q q q q qq q q qq q q qq q q qq q q q q q qqq q q q q q qqq q q q q q q q q qq q qq q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q qq q qq q q qq q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q q qqq q q q q qq qq q q q q q q qqqq q q q q qq qqq q q q qq q q q qqq q q q q qq q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q ACF1 lumpiness entropy lshiftvchange cpoints fspots trend linearity curvature spikiness season peaktrough klscore change.idx −4 −2 0 2 −2.5 0.0 2.5 standardized PC1 (28.7% explained var.) standardizedPC2(17.3%explainedvar.) We need a measure of the “anomalousness” of a time series. 1 Rank points based on their local density. 2 Rank points based on whether they are within α-convex hulls of different radius. Exploring the feature space of large time series collections Yahoo web traffic 20
  • 45. Bivariate kernel density ˆf(x; H) = 1 n n i=1 KH(x − Xi) Xi ∈ a bivariate random sample {X1, X2, . . . , Xn} KH(x) is the standard normal kernel function H estimated by minimizing the sum of AMISE Rank points based on ˆf values in 2d PCA space. Exploring the feature space of large time series collections Yahoo web traffic 21
  • 46. Bivariate kernel density ˆf(x; H) = 1 n n i=1 KH(x − Xi) Xi ∈ a bivariate random sample {X1, X2, . . . , Xn} KH(x) is the standard normal kernel function H estimated by minimizing the sum of AMISE Rank points based on ˆf values in 2d PCA space. Exploring the feature space of large time series collections Yahoo web traffic 21
  • 47. Bivariate density ranking Exploring the feature space of large time series collections Yahoo web traffic 22 −5 0 5 −8−6−4−20246 pc1 pc2 1 2 3 4 5
  • 48. Bivariate density ranking Exploring the feature space of large time series collections Yahoo web traffic 22 0 10000 20000 30000 40000 0 2000 4000 6000 0 10000 20000 30000 40000 50000 0 10000 20000 30000 40000 0 1000 2000 3000 4000 5000 S7793S8494S10464S7833S1715 2015−02−28 2015−03−01 2015−03−02 2015−03−03 2015−03−04 2015−03−05 2015−03−06 2015−03−07 2015−03−08 2015−03−09 2015−03−10 2015−03−11 2015−03−12 2015−03−13 2015−03−14 2015−03−15 2015−03−16 2015−03−17 2015−03−18 2015−03−19 2015−03−20 2015−03−21 2015−03−22 2015−03−23 2015−03−24 2015−03−25 2015−03−26 2015−03−27 2015−03−28 2015−03−29 2015−03−30 2015−03−31 2015−04−01 date value
  • 49. α-convex hulls The space generated by point pairs that can be touched by an empty disc of radius α. α → ∞ gives a convex hull. Points can become isolated when α is small. We rank points based on the value of α when they become isolated. Exploring the feature space of large time series collections Yahoo web traffic 23
  • 50. α-convex hulls The space generated by point pairs that can be touched by an empty disc of radius α. α → ∞ gives a convex hull. Points can become isolated when α is small. We rank points based on the value of α when they become isolated. Exploring the feature space of large time series collections Yahoo web traffic 23
  • 51. α-convex hulls The space generated by point pairs that can be touched by an empty disc of radius α. α → ∞ gives a convex hull. Points can become isolated when α is small. We rank points based on the value of α when they become isolated. Exploring the feature space of large time series collections Yahoo web traffic 23
  • 52. α-convex hulls The space generated by point pairs that can be touched by an empty disc of radius α. α → ∞ gives a convex hull. Points can become isolated when α is small. We rank points based on the value of α when they become isolated. Exploring the feature space of large time series collections Yahoo web traffic 23
  • 53. α-convex hull Exploring the feature space of large time series collections Yahoo web traffic 24 −5 0 5 −8−6−4−20246 q q qq q q q q q q q qq q q q q qq q qq q q q q q q qq q q q q qqqq qq q q q q q q q q q q q q q q q q q q q qqq q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q qqq q q qq q q q q qq q q q q q qq q q q q q q qq q q q q q q qq q q q q qq q q q q q q q q q q q q qq q qq q q q q q q qq q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qqq q qq q qq qqqqq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q qq qq qq q q q q q q q qq q q q q q q q q q qqq q qqqqq q q q q q q q qqq q q q q q qq qq q qq qqqq qq q q q q q q q q qq q qqq qqqq qqq q q q qq q q q q q q q qq qqq q q q q q q q q qq q qqq q q q q q q q q q q q q q q q q q q q qqq q q q qq q q q q q qq q q q q q q q qq q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q qq q q qq q q q q q q q q q q q q q q qq q q q q q q q q q qq qq q q q q qq q q qq q q q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q qqqq q q q q q q q q qq qq q qqq q q q qq q q q q q q q q q q q q q q q q qqq q q qq qq q q q q qqq q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q qq q q q q q q q qq q q q q qq q q q qq q q qq q q qq q q q q q q q q q qq q qqq q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q qqq q q q q q q qq q q q q q q q q q qqq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q q q q q q q q q q qq q q q q qq qq q q q q q q q q q q q q qqq q q q q q q q q qq q q q q q q q q q q q q q q qq q qq q q q q qq q q q q q q q qq q qq qq q q q q q q q q q q q q qq q q q qq q q q q qq q q q qq q q q qq q q q qqq q q q q q q q q q qqq qq q qq q q qq q qq qqqq qqq q q qqq q qq q q qq q q q q q q q q q q q q q q qq q q qq qqqq q q q q qq qq qq q q q q q q q q q q q q q q q q q q qqqq q q q q q qq q q qq qq qq q q qq qq q qq q q q q q q q q q q q q q q q q q q q qq qqq q qq qq q q q q qq qq q q qq q qq q q q q q q q q q qq q q q q q q q q q qq q q q q q q qq q qq qq qq q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q qqq q q q q q q qq q qq q q qq q q q q qq q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q qq q qq q q qqqq q q q q q q q q q q q q q q q q q q q qq q q q q qq qq q q q q q q q q q q qq q q q qq qqq q q q q q q q q q q q qq q qq q q q q q q qq q qq q qq q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q qqq q qq q q q q q q q qqq q q q q q q q q q qq q qq q q q q q q q q q q q q q q q q qq qq q q qq q q q q q q q q q q qqq q q q q q q q q q qq q q qq q q q q q q qq q q q q qqqq q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q q q q qqq qq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q qq q q qq q q q q q q qq q q q q q q q q q q q q q qq q qq q q q q q q qq q qq q q qq q q q q q q qq qq q q q q q q q q q q q qq q q q q q q qq q q q q q q q q qq qq q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q qq q q q q qqq q q qq q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q qq q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q qq q q q q q q q q q q q q q q q q q q qq q q q q qq q q q q q q q qq q q qq q q q q q q q q q q q q qqq q qq q qq qq qq q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q qq q q q q qq q q q qqq q q q q q q q q q q q q q q q q q q q q q q qq q q q q qq q q qq qq q q q q q q q q q q q q q q q q qq q q q q qq q q q q q q q q q q q q qq q q q q qq q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q qq qq q q q q q q q q qq qq q q q q q q q q qq q q q q q q qq q q qq qq q q q q q q q q q q qq q qq q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q q q qq q q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q qq q q q q qq q q q q q qq q q q q q qq q q qqq q q q q q q q q q q q q q q q q q q q q q q qq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q qq q q q q q q qq q q qq q q q q q q q q q q q qqq q q q q q q q qq q q q q qq q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q qq q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q qq q q q q q q q q q qq q qq q q q q q q q q qq q q q qq q q q q q q q q qq q q q qqqq qq q q q qq q q q q q qq qq q q q q qq q q q q q q qq q q q q qq q qq q q q q q q q q q q q q qq q qq qq qqq q q q qq q q q q q q q q q q q qq q q q q q qq q q q qqq q q q q q q q qq q q q q q q q q q q q q qq q qq q q qq q q q qq qq q q qq q q q q q q q q q qq qqq q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q q qq qq q q qq qq q q q q q q qq qq q q q q q qq q q q q q qq q q q q q q q q q qq q qq q q q q q q q q q q q q q q q q qq q q qq qq q q q q q q q q q q q q q qqqq qq q q qq q q qq q q q qq q q qq q q qq q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q qqq q q q q q q q q q q q q qq q q q qq qq q q q qqq q q q q q q q q qq q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q qq q q q q q q q qq qq q qqq q q qqqqq q q q q qq q qqq qq q q q q q q qqq q q q q q qq q q qq q q q q q qq q q q q q qq q q q q q q qq qqqq qq q q q q qq q q q q q q q q q q q q qq qq q qqqqq q q qqq q q q q qq q qq qqqq qqqq q q q q q q q q q q q q qq q q q q q q q qq q qq q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q qq q q qq q q q q q q qq qq q q q qq q q q q qq q q q q q q q q q q q qq qq q q q q q q q qqq q q q q q q q qq qq q q q q q q q qq q q q q qq q q qq q q q q q q qq q q q q q q q q q q qqq q q q qq qqq qq q q qq q q qqq q q qq q qq q q q qqq q qq q q qq q q q q qqq q q q q q q q q q q qqq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q qqq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q qqq q q q q q q q q q q q q q q qq q q q q q qqq q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q qq q q qq q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q qq q q q q q q q q qq q q q qq q qqq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q qqq q q q q q q q q q q q q q q qq q q qq q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q qq q q q q q q qq q q q q q qqq q q q q q q q q qq qq q q q q qq q q q q q q q q q q q qq q q q qq q q q q q q q q q q qq qq qq q q q q q q q q qq qq q q q q q q q q qq q q qq q q q q q q qq q q q q q q qq q q q q q q q q q q q q q qqq q q q q q q qq q qq q q q qq q q q qq q q q q q q q q q qq q q q q q qqq q q q q q q q q q q qq q q q q qq q q q q qq q q q q q q qq q q q q q q qq q qq q q q q q q q q qq q q q qqqqq qqq qqqqq qq q q qqq q q qqq qq q q qq qq qq q q q q q q q q qq q q q qqq q q qq q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q qq q qq q q q q q qq q q q q q q q q q q qq q q qq q q q qq q q qq qq q q qq q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q qq q qq q q q q q q q q q q q q q q q q q qqq q qq q q q q q q qq q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q q q q qq q q q qq q q qq q q q q q qq q q q q q qq q q q qq qq q q q q q q q q q q q q q qq q q q q q q qqq qq qq q q qq q qqq qq q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q qq q q q qq qq q q q q q q q q qq q q qq q q q q q q q q q q q q q q qq q q qq q q q q q qq q q qq q qq q q q q qq q q q q q q q qq q q q q q q qq q q q q q q q qq q q q q q qq q q qq qqq qq q q q q q q qq q q q qq q q q q q q q q q q qq qq q q qqq q qqq q q qq q q q q qq q qq q q q q q q q q q q q q q qq q q q q q q q q q qq q qq q qq q q qq qq q q q q q q q q q q q q qq q qq q q q qq q q q q q qqq qq q q qq q q q q q q q q q q qq q q q q q q q q qqq q q q q qq q q q qq q q q q qq q q qq q q q q q q q q q q qq q q qq q q q q q q q q qq q q q q qq q q qq qq q q q q qq q q q qq q q q q qqq q q q qq q q q q q q q q q q q q q q q qq q q q q qq qq q q qq q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q qq q qq qq q qq qq q q q q qq q q q q q q qq q q qq qq q qq q q q qq q q q q q q q q q qq q q q q q q q qq q q q q q q q q q q q qq q qq q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q qq qq q q q qqqq qq q q q qqqq q q q q q q qqq q q q q q qqq qqq q q q q q q q q q q q q qq q q qq qq q qq q q q q qq q q q q qq q qq qq q q q q q q q q q q q q q q q q q q q qq qq qq qq q q q q qq qq q q q q q q qq q q q q q q q qq q q qq q qqqq q q q qq q q qq qq q q q q q q q q q q qq q q qq qqq q q q qq qq qq qq q q q q q q q q q q q q q q qq q q qqq q q q q q q q q q q q q q qq q q q q qqqq q q q q q q q q q q q q q q q q q q q q q q q q q qqq q q q q q q q qq q q q qqqqq q q qq q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q qqq q qqq q qq q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q qq q q q q qq q q qqq q q q q q q q q q q q q q q q q q q q q q qq q qq q q q q qq q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q qqq q qqq q q q q q q q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q qqq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q qq q q qq q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q qq q q q q q q q qqq q q q q q q q q q q qq q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q q q q qq qq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q qq qq qq q q q q q q q q q q q q q q q q q qqq q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q qq q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q q q qq q q qq q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q qqq q q q q q q q q q q q q q q qq q q q q q q q q qq q q q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q qq qq q qq q q q q q q q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q qqqq q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q qq qq qq q qq qq q qqq q qq q qq q q q qq q q qq q qq q q q q q q qqq qqq q qqq q qq q q q q q qq qq qqqqq qq q qq q q q q q q qq qq q q q q qq q q q q q qq qqqq qqq qqqq q q q q qq q q qq q qq q qq qq qq q qqq q q q q q q q q q q q qqqqqq qq q q q qq q q q qq qqq q qq q q q qq q qq q qqqq qq q qq q q q q q q q q q qq q qq q q q q q q q q qqq q q q q q q q q qq qqq q q q q q q q q q q q q qq q q q q q q q q q q q qq qq q q q q q qq q q q q q q q qqq q qq q q qq q q q q q q q q q q qq qq q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q qq q q q q q q qq q q q qq q q q q qq qq qq q q q q q q qq q q q q q q qq q qq q q q q q qqq q q q qqq qq q q q qq q q q q q q qqq q qq q q q q q q q q qq q q q q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q qq q qqq qq q qq qqq q q qq qqqq q q qq q q qq q qq qq q qqq qqqq qqq q q qqq qq qq q q q q qq q q q q q q q q q q q qq q q qq q qq q qq qq q q qq q q q q q q q qq qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q qq q q q q q q q qq q q q qq q q q q qqq q q q q qq q q q qq q qq q q q q q q q q q q q q q qq q q q q q q q q q q q qq q q q qq qq qq q qqq q q q q qqqq qqqqq q q qq q q qq qq q qq q q q q q q q qq q q qq q q q qqq q q q q q qq q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q qq q q q qq q q qq q q q q qq q qqqq q qq q q q q qq q q qq q q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q qqq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q qq q q q q qq qq qq q q q q q q q qq q q q q q q q q q q q q qq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q q q q q qq qq q q q q q q q q q qq q q qq q qq q q qqqq q q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q qq q qq q qq q qq q qq qqqq qq qqqqq qq qq q q q q qq qq q q qq q qq qq qq q q q q q qqq q qq q q q q q qq q q qqqq q q qq q qq qq q q q q q q qq q q q q q qq qq q q qq q qqq q q q q q qq q q q q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q qq q qq qq qqq q q q q q q q q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qqq q q qqq q q q q q qq q q q qq qqq q q q qq qq q q q q q q q qq qq q q q q q qq q q q qqqqq qqq qq q qq q q q q q qq q qq q q q q q q qq q qq q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq qq q q q q qq q q q q q q q q q q q q q q qqqq q q qqqqqqq q q q qq q q q q q qq qq qqq q q q qq q q q q q qq qqq qqqq qqqqq qqqqqq qq q qq q qq q q q qq qq q qqqq q q qqq q qqq q q qqqq qq q q q q q q q q qq qq q q qq qq qqq qq qqqqq q q qq q q q qq qqq q q q q q qqq qq q qqq qqq q qqq q q q q qq q q q q q q q q q q qq qq q q qq qq q q q qqq qq q qqq qq qqqq q q q q q q qq q qq qq qqq q q qq q qqq qq qqqqqq q q qq q qqqqq qqq q qqq qqq qq q q q q q q q q q q q q q q q q q q q q qq q q q qq qqq q q q q q q q q q q q q q q q q q q q qq q qqqq qq q qq qq q q q q q q q q q q q qqqqq q q qq qq q q q q q q qq q q q qq q q q q q q q q q q q q q qq qq q q q qq q qq q q q q q q qqqq q q q q q qq q q q q q q q q q q q qq qq q q q q q q q q q q q q q q qq q q q q q q qq qq q q q q qqq q q q q qq q q q q q qq q q q qqqq q q q q q qqqqq q q qqq q q q q q q q q qq q q q qqq q q qqqq qqq q q q q qqq q q q q q q q q q q q q q q q qq q q q q q q q q q q q qq q q qq q q q q q q q q q q q q q q q q qqq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q qq q q q q q q q q q qq q q qq q q q q q q q q q qq q q q q q q q q q q q qq q q qq q qqq q qq q qq q q q q q q q q q q q q q q qqq q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qqq q q q q q q q q qq q q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q q q qqq qq q qq q q qq q q q q q q q q q qq q qq q q q qq qq qq q qq q q q q q qq q q q q q q q q q q qq q q q qq qq q q qqq q qq qq qq q q q q q q q q q q qq q q q q q q qq q qqqq qq q q qq q qqq q q q q qq qq q q q q q q q qq q q q q q qq qqq qq qq q q q qq qq q q q qq q q q qqq q q qqq q q q q q q qqqqq q q q q qq q q q qq q q q q q q qqqq q q qq q qq q q q q q qq q qq qq qq q q q q qq qq q q qq qq q q q q q q q q qq qqq q q q qq qqq qq q qq qqq q q q q qq q q q q q qq q qq q q qq q q q q q qq q q qq q q q q q q q q q q q q q qq q q q q q qqq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q qqq q q q q q qq q q q q q q q q q qq q q q q qq q q q q q qqq q q q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqqq q q qq qq q q qq q qq q q q q qq q q q qq q q qq q q q q q qq q q q q qq q q q qq q q qq qq q q q qqq q q q q qq qqq qqq q q qq q q q qq q qq qqq q q q q q qq q q qq qq q q qq q q q q qqqqqqq q qqqq q q q q qq q q q q q qq q q q qq q qqq q q q q q q q q qq q q qq q q q q q q q q q q q qq q q q q q q q q qq qq qq qq q q q q q q q qq q q qq q q q q q q q q q q q q q q q q q q q q q q q qq q q q qq q q q q qq q qq q q q qqq qqqq q q q q q q q q qqqq qqq q qq q q qqq q q q q q q qqq q q q q qqq q q q q q qq q q qq q q qq q q qq q q q q q qqq q q q q q qqq q q q q q q q q qq q qq q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q qq q qq q q qqq q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q q qqq q q q q qq qq q q q q q q qqqq q q q q qq qqq q q q qq q q q qqq q q q q qq q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q
  • 54. α-convex hull ranking Exploring the feature space of large time series collections Yahoo web traffic 25 −5 0 5 −8−6−4−20246 q q qq q q q q q q q qq q q q q qq q qq q q q q q q qq q q q q qqqq qq q q q q q q q q q q q q q q q q q q q qqq q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q qqq q q qq q q q q qq q q q q q qq q q q q q q qq q q q q q q qq q q q q qq q q q q q q q q q q q q qq q qq q q q q q q qq q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qqq q qq q qq qqqqq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q qq qq qq q q q q q q q qq q q q q q q q q q qqq q qqqqq q q q q q q q qqq q q q q q qq qq q qq qqqq qq q q q q q q q q qq q qqq qqqq qqq q q q qq q q q q q q q qq qqq q q q q q q q q qq q qqq q q q q q q q q q q q q q q q q q q q qqq q q q qq q q q q q qq q q q q q q q qq q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q qq q q qq q q q q q q q q q q q q q q qq q q q q q q q q q qq qq q q q q qq q q qq q q q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q qqqq q q q q q q q q qq qq q qqq q q q qq q q q q q q q q q q q q q q q q qqq q q qq qq q q q q qqq q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q qq q q q q q q q qq q q q q qq q q q qq q q qq q q qq q q q q q q q q q qq q qqq q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q qqq q q q q q q qq q q q q q q q q q qqq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q q q q q q q q q q qq q q q q qq qq q q q q q q q q q q q q qqq q q q q q q q q qq q q q q q q q q q q q q q q qq q qq q q q q qq q q q q q q q qq q qq qq q q q q q q q q q q q q qq q q q qq q q q q qq q q q qq q q q qq q q q qqq q q q q q q q q q qqq qq q qq q q qq q qq qqqq qqq q q qqq q qq q q qq q q q q q q q q q q q q q q qq q q qq qqqq q q q q qq qq qq q q q q q q q q q q q q q q q q q q qqqq q q q q q qq q q qq qq qq q q qq qq q qq q q q q q q q q q q q q q q q q q q q qq qqq q qq qq q q q q qq qq q q qq q qq q q q q q q q q q qq q q q q q q q q q qq q q q q q q qq q qq qq qq q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q qqq q q q q q q qq q qq q q qq q q q q qq q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q qq q qq q q qqqq q q q q q q q q q q q q q q q q q q q qq q q q q qq qq q q q q q q q q q q qq q q q qq qqq q q q q q q q q q q q qq q qq q q q q q q qq q qq q qq q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q qqq q qq q q q q q q q qqq q q q q q q q q q qq q qq q q q q q q q q q q q q q q q q qq qq q q qq q q q q q q q q q q qqq q q q q q q q q q qq q q qq q q q q q q qq q q q q qqqq q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q q q q qqq qq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q qq q q qq q q q q q q qq q q q q q q q q q q q q q qq q qq q q q q q q qq q qq q q qq q q q q q q qq qq q q q q q q q q q q q qq q q q q q q qq q q q q q q q q qq qq q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q qq q q q q qqq q q qq q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q qq q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q qq q q q q q q q q q q q q q q q q q q qq q q q q qq q q q q q q q qq q q qq q q q q q q q q q q q q qqq q qq q qq qq qq q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q qq q q q q qq q q q qqq q q q q q q q q q q q q q q q q q q q q q q qq q q q q qq q q qq qq q q q q q q q q q q q q q q q q qq q q q q qq q q q q q q q q q q q q qq q q q q qq q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q qq qq q q q q q q q q qq qq q q q q q q q q qq q q q q q q qq q q qq qq q q q q q q q q q q qq q qq q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q q q qq q q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q qq q q q q qq q q q q q qq q q q q q qq q q qqq q q q q q q q q q q q q q q q q q q q q q q qq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q qq q q q q q q qq q q qq q q q q q q q q q q q qqq q q q q q q q qq q q q q qq q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q qq q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q qq q q q q q q q q q qq q qq q q q q q q q q qq q q q qq q q q q q q q q qq q q q qqqq qq q q q qq q q q q q qq qq q q q q qq q q q q q q qq q q q q qq q qq q q q q q q q q q q q q qq q qq qq qqq q q q qq q q q q q q q q q q q qq q q q q q qq q q q qqq q q q q q q q qq q q q q q q q q q q q q qq q qq q q qq q q q qq qq q q qq q q q q q q q q q qq qqq q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q q qq qq q q qq qq q q q q q q qq qq q q q q q qq q q q q q qq q q q q q q q q q qq q qq q q q q q q q q q q q q q q q q qq q q qq qq q q q q q q q q q q q q q qqqq qq q q qq q q qq q q q qq q q qq q q qq q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q qqq q q q q q q q q q q q q qq q q q qq qq q q q qqq q q q q q q q q qq q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q qq q q q q q q q qq qq q qqq q q qqqqq q q q q qq q qqq qq q q q q q q qqq q q q q q qq q q qq q q q q q qq q q q q q qq q q q q q q qq qqqq qq q q q q qq q q q q q q q q q q q q qq qq q qqqqq q q qqq q q q q qq q qq qqqq qqqq q q q q q q q q q q q q qq q q q q q q q qq q qq q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q qq q q qq q q q q q q qq qq q q q qq q q q q qq q q q q q q q q q q q qq qq q q q q q q q qqq q q q q q q q qq qq q q q q q q q qq q q q q qq q q qq q q q q q q qq q q q q q q q q q q qqq q q q qq qqq qq q q qq q q qqq q q qq q qq q q q qqq q qq q q qq q q q q qqq q q q q q q q q q q qqq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q qqq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q qqq q q q q q q q q q q q q q q qq q q q q q qqq q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q qq q q qq q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q qq q q q q q q q q qq q q q qq q qqq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q qqq q q q q q q q q q q q q q q qq q q qq q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q qq q q q q q q qq q q q q q qqq q q q q q q q q qq qq q q q q qq q q q q q q q q q q q qq q q q qq q q q q q q q q q q qq qq qq q q q q q q q q qq qq q q q q q q q q qq q q qq q q q q q q qq q q q q q q qq q q q q q q q q q q q q q qqq q q q q q q qq q qq q q q qq q q q qq q q q q q q q q q qq q q q q q qqq q q q q q q q q q q qq q q q q qq q q q q qq q q q q q q qq q q q q q q qq q qq q q q q q q q q qq q q q qqqqq qqq qqqqq qq q q qqq q q qqq qq q q qq qq qq q q q q q q q q qq q q q qqq q q qq q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q qq q qq q q q q q qq q q q q q q q q q q qq q q qq q q q qq q q qq qq q q qq q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q qq q qq q q q q q q q q q q q q q q q q q qqq q qq q q q q q q qq q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q q q q qq q q q qq q q qq q q q q q qq q q q q q qq q q q qq qq q q q q q q q q q q q q q qq q q q q q q qqq qq qq q q qq q qqq qq q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q qq q q q qq qq q q q q q q q q qq q q qq q q q q q q q q q q q q q q qq q q qq q q q q q qq q q qq q qq q q q q qq q q q q q q q qq q q q q q q qq q q q q q q q qq q q q q q qq q q qq qqq qq q q q q q q qq q q q qq q q q q q q q q q q qq qq q q qqq q qqq q q qq q q q q qq q qq q q q q q q q q q q q q q qq q q q q q q q q q qq q qq q qq q q qq qq q q q q q q q q q q q q qq q qq q q q qq q q q q q qqq qq q q qq q q q q q q q q q q qq q q q q q q q q qqq q q q q qq q q q qq q q q q qq q q qq q q q q q q q q q q qq q q qq q q q q q q q q qq q q q q qq q q qq qq q q q q qq q q q qq q q q q qqq q q q qq q q q q q q q q q q q q q q q qq q q q q qq qq q q qq q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q qq q qq qq q qq qq q q q q qq q q q q q q qq q q qq qq q qq q q q qq q q q q q q q q q qq q q q q q q q qq q q q q q q q q q q q qq q qq q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q qq qq q q q qqqq qq q q q qqqq q q q q q q qqq q q q q q qqq qqq q q q q q q q q q q q q qq q q qq qq q qq q q q q qq q q q q qq q qq qq q q q q q q q q q q q q q q q q q q q qq qq qq qq q q q q qq qq q q q q q q qq q q q q q q q qq q q qq q qqqq q q q qq q q qq qq q q q q q q q q q q qq q q qq qqq q q q qq qq qq qq q q q q q q q q q q q q q q qq q q qqq q q q q q q q q q q q q q qq q q q q qqqq q q q q q q q q q q q q q q q q q q q q q q q q q qqq q q q q q q q qq q q q qqqqq q q qq q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q qqq q qqq q qq q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q qq q q q q qq q q qqq q q q q q q q q q q q q q q q q q q q q q qq q qq q q q q qq q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q qqq q qqq q q q q q q q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q qqq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q qq q q qq q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q qq q q q q q q q qqq q q q q q q q q q q qq q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q q q q qq qq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q qq qq qq q q q q q q q q q q q q q q q q q qqq q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q qq q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q q q qq q q qq q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q qqq q q q q q q q q q q q q q q qq q q q q q q q q qq q q q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q qq qq q qq q q q q q q q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q qqqq q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q qq qq qq q qq qq q qqq q qq q qq q q q qq q q qq q qq q q q q q q qqq qqq q qqq q qq q q q q q qq qq qqqqq qq q qq q q q q q q qq qq q q q q qq q q q q q qq qqqq qqq qqqq q q q q qq q q qq q qq q qq qq qq q qqq q q q q q q q q q q q qqqqqq qq q q q qq q q q qq qqq q qq q q q qq q qq q qqqq qq q qq q q q q q q q q q qq q qq q q q q q q q q qqq q q q q q q q q qq qqq q q q q q q q q q q q q qq q q q q q q q q q q q qq qq q q q q q qq q q q q q q q qqq q qq q q qq q q q q q q q q q q qq qq q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q qq q q q q q q qq q q q qq q q q q qq qq qq q q q q q q qq q q q q q q qq q qq q q q q q qqq q q q qqq qq q q q qq q q q q q q qqq q qq q q q q q q q q qq q q q q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q qq q qqq qq q qq qqq q q qq qqqq q q qq q q qq q qq qq q qqq qqqq qqq q q qqq qq qq q q q q qq q q q q q q q q q q q qq q q qq q qq q qq qq q q qq q q q q q q q qq qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q qq q q q q q q q qq q q q qq q q q q qqq q q q q qq q q q qq q qq q q q q q q q q q q q q q qq q q q q q q q q q q q qq q q q qq qq qq q qqq q q q q qqqq qqqqq q q qq q q qq qq q qq q q q q q q q qq q q qq q q q qqq q q q q q qq q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q qq q q q qq q q qq q q q q qq q qqqq q qq q q q q qq q q qq q q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q qqq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q qq q q q q qq qq qq q q q q q q q qq q q q q q q q q q q q q qq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q q q q q qq qq q q q q q q q q q qq q q qq q qq q q qqqq q q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q qq q qq q qq q qq q qq qqqq qq qqqqq qq qq q q q q qq qq q q qq q qq qq qq q q q q q qqq q qq q q q q q qq q q qqqq q q qq q qq qq q q q q q q qq q q q q q qq qq q q qq q qqq q q q q q qq q q q q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q qq q qq qq qqq q q q q q q q q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qqq q q qqq q q q q q qq q q q qq qqq q q q qq qq q q q q q q q qq qq q q q q q qq q q q qqqqq qqq qq q qq q q q q q qq q qq q q q q q q qq q qq q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq qq q q q q qq q q q q q q q q q q q q q q qqqq q q qqqqqqq q q q qq q q q q q qq qq qqq q q q qq q q q q q qq qqq qqqq qqqqq qqqqqq qq q qq q qq q q q qq qq q qqqq q q qqq q qqq q q qqqq qq q q q q q q q q qq qq q q qq qq qqq qq qqqqq q q qq q q q qq qqq q q q q q qqq qq q qqq qqq q qqq q q q q qq q q q q q q q q q q qq qq q q qq qq q q q qqq qq q qqq qq qqqq q q q q q q qq q qq qq qqq q q qq q qqq qq qqqqqq q q qq q qqqqq qqq q qqq qqq qq q q q q q q q q q q q q q q q q q q q q qq q q q qq qqq q q q q q q q q q q q q q q q q q q q qq q qqqq qq q qq qq q q q q q q q q q q q qqqqq q q qq qq q q q q q q qq q q q qq q q q q q q q q q q q q q qq qq q q q qq q qq q q q q q q qqqq q q q q q qq q q q q q q q q q q q qq qq q q q q q q q q q q q q q q qq q q q q q q qq qq q q q q qqq q q q q qq q q q q q qq q q q qqqq q q q q q qqqqq q q qqq q q q q q q q q qq q q q qqq q q qqqq qqq q q q q qqq q q q q q q q q q q q q q q q qq q q q q q q q q q q q qq q q qq q q q q q q q q q q q q q q q q qqq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q qq q q q q q q q q q qq q q qq q q q q q q q q q qq q q q q q q q q q q q qq q q qq q qqq q qq q qq q q q q q q q q q q q q q q qqq q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qqq q q q q q q q q qq q q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q q q qqq qq q qq q q qq q q q q q q q q q qq q qq q q q qq qq qq q qq q q q q q qq q q q q q q q q q q qq q q q qq qq q q qqq q qq qq qq q q q q q q q q q q qq q q q q q q qq q qqqq qq q q qq q qqq q q q q qq qq q q q q q q q qq q q q q q qq qqq qq qq q q q qq qq q q q qq q q q qqq q q qqq q q q q q q qqqqq q q q q qq q q q qq q q q q q q qqqq q q qq q qq q q q q q qq q qq qq qq q q q q qq qq q q qq qq q q q q q q q q qq qqq q q q qq qqq qq q qq qqq q q q q qq q q q q q qq q qq q q qq q q q q q qq q q qq q q q q q q q q q q q q q qq q q q q q qqq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q qqq q q q q q qq q q q q q q q q q qq q q q q qq q q q q q qqq q q q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqqq q q qq qq q q qq q qq q q q q qq q q q qq q q qq q q q q q qq q q q q qq q q q qq q q qq qq q q q qqq q q q q qq qqq qqq q q qq q q q qq q qq qqq q q q q q qq q q qq qq q q qq q q q q qqqqqqq q qqqq q q q q qq q q q q q qq q q q qq q qqq q q q q q q q q qq q q qq q q q q q q q q q q q qq q q q q q q q q qq qq qq qq q q q q q q q qq q q qq q q q q q q q q q q q q q q q q q q q q q q q qq q q q qq q q q q qq q qq q q q qqq qqqq q q q q q q q q qqqq qqq q qq q q qqq q q q q q q qqq q q q q qqq q q q q q qq q q qq q q qq q q qq q q q q q qqq q q q q q qqq q q q q q q q q qq q qq q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q qq q qq q q qqq q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q q qqq q q q q qq qq q q q q q q qqqq q q q q qq qqq q q q qq q q q qqq q q q q qq q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q 1 2 3 4 5
  • 55. α-convex hull ranking Exploring the feature space of large time series collections Yahoo web traffic 25 0 10000 20000 30000 40000 50000 0 10000 20000 30000 40000 0 2000 4000 6000 0 1000 2000 3000 4000 5000 0 10000 20000 30000 S10464S7793S8494S1715S7826 2015−02−28 2015−03−01 2015−03−02 2015−03−03 2015−03−04 2015−03−05 2015−03−06 2015−03−07 2015−03−08 2015−03−09 2015−03−10 2015−03−11 2015−03−12 2015−03−13 2015−03−14 2015−03−15 2015−03−16 2015−03−17 2015−03−18 2015−03−19 2015−03−20 2015−03−21 2015−03−22 2015−03−23 2015−03−24 2015−03−25 2015−03−26 2015−03−27 2015−03−28 2015−03−29 2015−03−30 2015−03−31 2015−04−01 date value
  • 56. HDR versus α-convex hull HDR boxplot −5 0 5 −8−6−4−20246 pc1 pc2 1 2 3 4 5 α-convex hull −5 0 5 −8−6−4−20246 q q qq q q q q q q q qq q q q q qq q qq q q q q q q qq q q q q qqqq qq q q q q q q q q q q q q q q q q q q q qqq q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q qqq q q qq q q q q qq q q q q q qq q q q q q q qq q q q q q q qq q q q q qq q q q q q q q q q q q q qq q qq q q q q q q qq q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qqq q qq q qq qqqqq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q qq qq qq q q q q q q q qq q q q q q q q q q qqq q qqqqq q q q q q q q qqq q q q q q qq qq q qq qqqq qq q q q q q q q q qq q qqq qqqq qqq q q q qq q q q q q q q qq qqq q q q q q q q q qq q qqq q q q q q q q q q q q q q q q q q q q qqq q q q qq q q q q q qq q q q q q q q qq q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q qq q q qq q q q q q q q q q q q q q q qq q q q q q q q q q qq qq q q q q qq q q qq q q q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q qqqq q q q q q q q q qq qq q qqq q q q qq q q q q q q q q q q q q q q q q qqq q q qq qq q q q q qqq q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q qq q q q q q q q qq q q q q qq q q q qq q q qq q q qq q q q q q q q q q qq q qqq q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q qqq q q q q q q qq q q q q q q q q q qqq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q q q q q q q q q q qq q q q q qq qq q q q q q q q q q q q q qqq q q q q q q q q qq q q q q q q q q q q q q q q qq q qq q q q q qq q q q q q q q qq q qqqq q q q q q q q q q q q q qq q q q qq q q q q qq q q q qq q q q qq q q q qqq q q q q q q q q q qqq qq q qq q q qq q qq qqqq qqq q q qqq q qq q q qq q q q q q q q q q q q q q q qq q q qq qqqq q q q q qq qq qq q q q q q q q q q q q q q q q q q q qqqq q q q q q qq q q qq qq qq q q qq qq q qq q q q q q q q q q q q q q q q q q q q qq qqq q qq qq q q q q qq qq q q qq q qq q q q q q q q q q qq q q q q q q q q q qq q q q q q q qq q qq qq qq q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q qqq q q q q q q qq q qq q q qq q q q q qq q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q qq q qq q q qqqq q q q q q q q q q q q q q q q q q q q qq q q q q qq qq q q q q q q q q q q qq q q q qq qqq q q q q q q q q q q q qq q qq q q q q q q qq q qq q qq q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q qqq q qq q q q q q q q qqq q q q q q q q q q qq q qq q q q q q q q q q q q q q q q q qq qq q q qq q q q q q q q q q q qqq q q q q q q q q q qq q q qq q q q q q q qq q q q q qqqq q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q q q q qqq qq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q qq q q qq q q q q q q qq q q q q q q q q q q q q q qq q qq q q q q q q qq q qq q q qq q q q q q q qq qq q q q q q q q q q q q qq q q q q q q qq q q q q q q q q qq qq q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q qq q q q q qqq q q qq q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q qq q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q qq q q q q q q q q q q q q q q q q q q qq q q q q qq q q q q q q q qq q q qq q q q q q q q q q q q q qqq q qq q qq qq qq q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q qq q q q q qq q q q qqq q q q q q q q q q q q q q q q q q q q q q q qq q q q q qq q q qq qq q q q q q q q q q q q q q q q q qq q q q q qq q q q q q q q q q q q q qq q q q q qq q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q qq qq q q q q q q q q qq qq q q q q q q q q qq q q q q q q qq q q qq qq q q q q q q q q q q qq q qq q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q q q qq q q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q qq q q q q qq q q q q q qq q q q q q qq q q qqq q q q q q q q q q q q q q q q q q q q q q q qq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q qq q q q q q q qq q q qq q q q q q q q q q q q qqq q q q q q q q qq q q q q qq q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q qq q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q qq q q q q q q q q q qq q qq q q q q q q q q qq q q q qq q q q q q q q q qq q q q qqqq qq q q q qq q q q q q qq qq q q q q qq q q q q q q qq q q q q qq q qq q q q q q q q q q q q q qq q qq qq qqq q q q qq q q q q q q q q q q q qq q q q q q qq q q q qqq q q q q q q q qq q q q q q q q q q q q q qq q qq q q qq q q q qq qq q q qq q q q q q q q q q qq qqq q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q q qq qq q q qq qq q q q q q q qq qq q q q q q qq q q q q q qq q q q q q q q q q qq q qq q q q q q q q q q q q q q q q q qq q q qq qq q q q q q q q q q q q q q qqqq qq q q qq q q qq q q q qq q q qq q q qq q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q qqq q q q q q q q q q q q q qq q q q qq qq q q q qqq q q q q q q q q qq q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q qq q q q q q q q qq qq q qqq q q qqqqq q q q q qq q qqq qq q q q q q q qqq q q q q q qq q q qq q q q q q qq q q q q q qq q q q q q q qq qqqq qq q q q q qq q q q q q q q q q q q q qq qq q qqqqq q q qqq q q q q qq q qq qqqq qqqq q q q q q q q q q q q q qq q q q q q q q qq q qq q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q qq q q qq q q q q q q qq qq q q q qq q q q q qq q q q q q q q q q q q qq qq q q q q q q q qqq q q q q q q q qq qq q q q q q q q qq q q q q qq q q qq q q q q q q qq q q q q q q q q q q qqq q q q qq qqq qq q q qq q q qqq q q qq q qq q q q qqq q qq q q qq q q q q qqq q q q q q q q q q q qqq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q qqq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q qqq q q q q q q q q q q q q q q qq q q q q q qqq q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q qq q q qq q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q qq q q q q q q q q qq q q q qq q qqq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q qqq q q q q q q q q q q q q q q qq q q qq q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q qq q q q q q q qq q q q q q qqq q q q q q q q q qq qq q q q q qq q q q q q q q q q q q qq q q q qq q q q q q q q q q q qq qq qq q q q q q q q q qq qq q q q q q q q q qq q q qq q q q q q q qq q q q q q q qq q q q q q q q q q q q q q qqq q q q q q q qq q qq q q q qq q q q qq q q q q q q q q q qq q q q q q qqq q q q q q q q q q q qq q q q q qq q q q q qq q q q q q q qq q q q q q q qq q qq q q q q q q q q qq q q q qqqqq qqq qqqqq qq q q qqq q q qqq qq q q qq qq qq q q q q q q q q qq q q q qqq q q qq q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q qq q qq q q q q q qq q q q q q q q q q q qq q q qq q q q qq q q qq qq q q qq q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q qq q qq q q q q q q q q q q q q q q q q q qqq q qq q q q q q q qq q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q q q q qq q q q qq q q qq q q q q q qq q q q q q qq q q q qq qq q q q q q q q q q q q q q qq q q q q q q qqq qq qq q q qq q qqq qq q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q qq q q q qq qq q q q q q q q q qq q q qq q q q q q q q q q q q q q q qq q q qq q q q q q qq q q qq q qq q q q q qq q q q q q q q qq q q q q q q qq q q q q q q q qq q q q q q qq q q qq qqq qq q q q q q q qq q q q qq q q q q q q q q q q qq qq q q qqq q qqq q q qq q q q q qq q qq q q q q q q q q q q q q q qq q q q q q q q q q qq q qq q qqq q qq qq q q q q q q q q q q q q qq q qq q q q qq q q q q q qqq qq q q qq q q q q q q q q q q qq q q q q q q q q qqq q q q q qq q q q qq q q q q qq q q qq q q q q q q q q q q qq q q qq q q q q q q q q qq q q q q qq q q qq qq q q q q qq q q q qq q q q q qqq q q q qq q q q q q q q q q q q q q q q qq q q q q qq qq q q qq q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q qq q qq qq q qq qq q q q q qq q q q q q q qq q q qq qq q qq q q q qq q q q q q q q q q qq q q q q q q q qq q q q q q q q q q q q qq q qq q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q qq qq q q q qqqq qq q q q qqqq q q q q q q qqq q q q q q qqq qqq q q q q q q q q q q q q qq q q qq qq q qq q q q q qq q q q q qq q qq qq q q q q q q q q q q q q q q q q q q q qq qq qq qq q q q q qq qq q q q q q q qq q q q q q q q qq q q qq q qqqq q q q qq q q qq qq q q q q q q q q q q qq q q qq qqq q q q qq qq qq qq q q q q q q q q q q q q q q qq q q qqq q q q q q q q q q q q q q qq q q q q qqqq q q q q q q q q q q q q q q q q q q q q q q q q q qqq q q q q q q q qq q q q qqqqq q q qq q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q qqqq qqq q qq q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q qq q q q q qq q q qqq q q q q q q q q q q q q q q q q q q q q q qq q qq q q q q qq q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q q q q qqq q qqq q q q q q q q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q qqq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q qq q q qq q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q qq q q q q q q q qqq q q q q q q q q q q qq q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q q q q qq qq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q qq qq qq q q q q q q q q q q q q q q q q q qqq q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q qqq qq q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q q q qq q q qq q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q qqq q q q q q q q q q q q q q q qq q q q q q q q q qq q q q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q qq q q qq qq q qq q q q q q q q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q qqqq q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q qq qq qq q qq qq q qqq q qq q qq q q q qq q q qq q qq q q q q q q qqq qqq q qqq q qq q q q q q qq qq qqqqq qq q qq q q q q q q qq qq q q q q qq q q q q q qq qqqq qqqqqqq q q q q qq q q qq q qq q qq qq qq q qqq q q q q q q q q q q q qqqqqq qq q q q qq q q q qq qqq q qq q q q qq q qq q qqqq qq q qq q q q q q q q q q qq q qq q q q q q q q q qqq q q q q q q q q qq qqq q q q q q q q q q q q q qq q q q q q q q q q q q qq qq q q q q q qq q q q q q q q qqq q qq q q qq q q q q q q q q q q qq qq q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq q q qq q q q q q q qq q q q qq q q q q qq qq qq q q q q q q qq q q q q q q qq q qq q q q q q qqq q q q qqq qq q q q qq q q q q q q qqq q qq q q q q q q q q qq q q q q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q qq q qqq qq q qq qqq q q qq qqqq q q qq q q qq q qq qq q qqq qqqq qqq q q qqq qq qq q q q q qq q q q q q q q q q q q qq q q qq q qq q qq qq q q qq q q q q q q q qq qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q qq q q q q q q q qq q q q qq q q q q qq q q q q q qq q q q qq q qq q q q q q q q q q q q q q qq q q q q q q q q q q q qq q q q qq qq qq q qqq q q q q qqqq qqqqq q q qq q q qq qq q qq q q q q q q q qq q q qq q q q qqq q q q q q qq q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q qq q q q qq q q qq q q q q qq q qqqq q qq q q q q qq q q qq q q q q q q qq qq q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q qqq q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q qq q q q q qq qq qq q q q q q q q qq q q q q q q q q q q q q qq q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqq q q q q q qq qq q q q q q q q q q qq q q qq q qq q q qqqq q q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q qq q qq q qq q qq q qq qqqq qq qqqqq qq qq q q q q qq qq q q qq q qq qq qq q q q q q qqq q qq q q q q q qq q q qqqq q q qq q qq qq q q q q q q qq q q q q q qq qq q q qq q qqq q q q q q qq q q q q q q q q qq q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q qq q qq qq qqq q q q q q q q q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q qq q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qqq q q qqq q q q q q qq q q q qq qqq q q q qq qq q q q q q q q qq qq q q q q q qq q q q qqqqq qqq qq q qq q q q q q qq q qq q q q q q q qq q qq q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q qq qq q q q q qq q q q q q q q q q q q q q q qqqq q q qqqqqqq q q q qq q q q q q qq qq qqq q q q qq q q q q q qq qqq qqqq qqqqq qqqqqq qq q qq q qq q q q qq qq q qqqq q q qqq q qqq q q qqqq qq q q q q q q q q qq qq q q qq qq qqq qq qqqqq q q qq q q q qq qqq q q q q q qqq qq q qqq qqq q qqq q q q q qq q q q q q q q q q q qq qq q q qq qq q q q qqq qq q qqqqq qqqq q q q q q q qq q qq qq qqq q q qq q qqq qq qqqqqq q q qq q qqqqq qqq q qqq qqq qq q q q q q q q q q q q q q q q q q q q q qq q q q qq qqq q q q q q q q q q q q q q q q q q q q qq q qqqq qq q qq qq q q q q q q q q q q q qqqqq q q qq qq q q q q q q qq q q q qq q q q q q q q q q q q q q qq qq q q q qq q qq q q q q q q qqqq q q q q q qq q q q q q q q q q q q qq qq q q q q q q q q q q q q q q qq q q q q q q qq qq q q q q qqq q q q q qq q q q q q qq q q q qqqq q q q q q qqqqq q q qqq q q q q q q q q qq q q q qqq q q qq qq qqq q q q q qqq q q q q q q q q q q q q q q q qq q q q q q q q q q q q qq q q qq q q q q q q q q q q q q q q q q qqq q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q qq q q q q q q q q q qq q q qq q q q q q q q q q qq q q q q q q q q q q q qq q q qq q qqq q qq q qq q q q q q q q q q q q q q q qqq q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q qqq q q q q q q q q qq q q q q q q q q q q q q q q q q q qq qq q q q q q q q q q q q q q q q qqq qq q qq q q qq q q q q q q q q q qq q qq q q q qq qq qq q qq q q q q q qq q q q q q q q q q q qq q q q qq qq q q qqq q qq qq qq q q q q q q q q q q qq q q q q q q qq q qqqq qq q q qq q qqq q q q q qq qq q q q q q q q qq q q q q q qq qqq qq qq q q q qq qq q q q qq q q q qqq q q qqq q q q q q q qqqqq q q q q qq q q q qq q q q q q q qqqq q q qq q qq q q q q q qq q qq qq qq q q q q qq qq q q qq qq q q q q q q q q qq qqq q q q qq qqq qq q qq qqq q q q q qq q q q q q qq q qq q q qq q q q q q qq q q qq q q q q q q q q q q q q q qq q q q q q qqq qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q qqq q q q q q qq q q q q q q q q q qq q q q q qq q q q q q qqq q q q q q q qq q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qqqq q q qq qq q q qq q qq q q q q qq q q q qq q q qq q q q q q qq q q q q qq q q q qq q q qq qq q q q qqq q q q q qq qqq qqq q q qq q q q qq q qq qqq q q q q q qq q q qq qq q q qq q q q q qqqqqqq q qqqq q q q q qq q q q q q qq q q q qq q qqq q q q q q q q q qq q q qq q q q q q q q q q q q qq q q q q q q q q qq qq qq qq q q q q q q q qq q q qq q q q q q q q q q q q q q q q q q q q q q q q qq q q q qq q q q q qq q qq q q q qqq qqqq q q q q q q q q qqqq qqq q qq q q qqq q q q q q q qqq q q q q qqq q q q q q qq q q qq q q qq q q qq q q q q q qqq q q q q q qqq q q q q q q q q qq q qq q q q q q q q q q q q qq q q q q q qq q q q q q q q q q q q q q q q q qq q qq q q qqq q q q q q q q qq q q q q q q q q q q q q q q q q qq q q q q qqq q q q q qq qq q q q q q q qqqq q q q q qq qqq q q q qq q q q qqq q q q q qq q q q q q q q qq q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q 1 2 3 4 5 Exploring the feature space of large time series collections Yahoo web traffic 26
  • 57. Top 5 anomalous time series HDR 0 10000 20000 30000 40000 0 2000 4000 6000 0 10000 20000 30000 40000 50000 0 10000 20000 30000 40000 0 1000 2000 3000 4000 5000 S7793S8494S10464S7833S1715 2015−02−28 2015−03−01 2015−03−02 2015−03−03 2015−03−04 2015−03−05 2015−03−06 2015−03−07 2015−03−08 2015−03−09 2015−03−10 2015−03−11 2015−03−12 2015−03−13 2015−03−14 2015−03−15 2015−03−16 2015−03−17 2015−03−18 2015−03−19 2015−03−20 2015−03−21 2015−03−22 2015−03−23 2015−03−24 2015−03−25 2015−03−26 2015−03−27 2015−03−28 2015−03−29 2015−03−30 2015−03−31 2015−04−01 date value α-convex hull 0 10000 20000 30000 40000 50000 0 10000 20000 30000 40000 0 2000 4000 6000 0 1000 2000 3000 4000 5000 0 10000 20000 30000 S10464S7793S8494S1715S7826 2015−02−28 2015−03−01 2015−03−02 2015−03−03 2015−03−04 2015−03−05 2015−03−06 2015−03−07 2015−03−08 2015−03−09 2015−03−10 2015−03−11 2015−03−12 2015−03−13 2015−03−14 2015−03−15 2015−03−16 2015−03−17 2015−03−18 2015−03−19 2015−03−20 2015−03−21 2015−03−22 2015−03−23 2015−03−24 2015−03−25 2015−03−26 2015−03−27 2015−03−28 2015−03−29 2015−03−30 2015−03−31 2015−04−01 date value Exploring the feature space of large time series collections Yahoo web traffic 27
  • 58. Outline 1 M3 competition data 2 Yahoo web traffic 3 What next? Exploring the feature space of large time series collections What next? 28
  • 59. What next? Develop a more comprehensive set of features that are reliable measures and fast to compute. e.g., for finance data. Consider application to functional data from other contexts (not time series). Is PCA the right approach? Perhaps we should use multidimensional scaling? Or something else? Should we use more than 2 PC dimensions? Develop dynamic and interactive visualization tools. Make methods available in an R package. Some of the methods are already available in the anomalous package for R on github. Exploring the feature space of large time series collections What next? 29
  • 60. What next? Develop a more comprehensive set of features that are reliable measures and fast to compute. e.g., for finance data. Consider application to functional data from other contexts (not time series). Is PCA the right approach? Perhaps we should use multidimensional scaling? Or something else? Should we use more than 2 PC dimensions? Develop dynamic and interactive visualization tools. Make methods available in an R package. Some of the methods are already available in the anomalous package for R on github. Exploring the feature space of large time series collections What next? 29
  • 61. What next? Develop a more comprehensive set of features that are reliable measures and fast to compute. e.g., for finance data. Consider application to functional data from other contexts (not time series). Is PCA the right approach? Perhaps we should use multidimensional scaling? Or something else? Should we use more than 2 PC dimensions? Develop dynamic and interactive visualization tools. Make methods available in an R package. Some of the methods are already available in the anomalous package for R on github. Exploring the feature space of large time series collections What next? 29
  • 62. What next? Develop a more comprehensive set of features that are reliable measures and fast to compute. e.g., for finance data. Consider application to functional data from other contexts (not time series). Is PCA the right approach? Perhaps we should use multidimensional scaling? Or something else? Should we use more than 2 PC dimensions? Develop dynamic and interactive visualization tools. Make methods available in an R package. Some of the methods are already available in the anomalous package for R on github. Exploring the feature space of large time series collections What next? 29
  • 63. What next? Develop a more comprehensive set of features that are reliable measures and fast to compute. e.g., for finance data. Consider application to functional data from other contexts (not time series). Is PCA the right approach? Perhaps we should use multidimensional scaling? Or something else? Should we use more than 2 PC dimensions? Develop dynamic and interactive visualization tools. Make methods available in an R package. Some of the methods are already available in the anomalous package for R on github. Exploring the feature space of large time series collections What next? 29
  • 64. What next? Develop a more comprehensive set of features that are reliable measures and fast to compute. e.g., for finance data. Consider application to functional data from other contexts (not time series). Is PCA the right approach? Perhaps we should use multidimensional scaling? Or something else? Should we use more than 2 PC dimensions? Develop dynamic and interactive visualization tools. Make methods available in an R package. Some of the methods are already available in the anomalous package for R on github. Exploring the feature space of large time series collections What next? 29
  • 65. What next? Develop a more comprehensive set of features that are reliable measures and fast to compute. e.g., for finance data. Consider application to functional data from other contexts (not time series). Is PCA the right approach? Perhaps we should use multidimensional scaling? Or something else? Should we use more than 2 PC dimensions? Develop dynamic and interactive visualization tools. Make methods available in an R package. Some of the methods are already available in the anomalous package for R on github. Exploring the feature space of large time series collections What next? 29
  • 66. Further information Exploring the feature space of large time series collections What next? 30 ¯ Papers: robjhyndman.com ¯ Code: github.com/robjhyndman ¯ Email: Rob.Hyndman@monash.edu