Real time clustering of time series

Dhinaharan Nagamalai et al. (Eds) : CCSEA, DKMP, AIFU, SEA - 2015
pp. 197–212, 2015. © CS & IT-CSCP 2015 DOI : 10.5121/csit.2015.50217
REAL TIME CLUSTERING OF TIME SERIES
USING TRIANGULAR POTENTIALS
Aldo Pacchiano1
and Oliver J. Williams2
1
Massachusetts Institute of Technology
aldopacchiano@gmail.com
2
Markham Rae LLP, London, UK. (The views and opinions expressed herein are
those of the author and do not necessarily reflect the views of Markham Rae
LLP.)
oliver.williams@markhamrae.com
ABSTRACT
Motivated by the problem of computing investment portfolio weightings we investigate various
methods of clustering as alternatives to traditional mean-variance approaches. Such methods can
have significant benefits from a practical point of view since they remove the need to invert a
sample covariance matrix, which can suffer from estimation error and will almost certainly be
non-stationary. The general idea is to find groups of assets which share similar return
characteristics over time and treat each group as a single composite asset. We then apply inverse
volatility weightings to these new composite assets. In the course of our investigation we devise a
method of clustering based on triangular potentials and we present associated theoretical results
as well as various examples based on synthetic data.
KEYWORDS
Clustering, Expected Utility, Graphical Models, k-Clique Problem
1. INTRODUCTION
A common problem in finance is the question of how best to construct a diversified portfolio of
investments. This problem is ubiquitous in fund management, banking and insurance and has led to
an extensive evolving literature, both theoretical and empirical. From an informal mathematical
perspective the central challenge is to devise a method for determining weightings for a set of
random variables such that ex post realisations of the weighted sum optimise some objective
function on average. The objective function most typically used in financial economics is a
concave utility function, hence from an ex ante pespective the portfolio construction problem is a
matter of optimising so-called expected utility. Koller and Friedman provide a detailed discussion
of utility functions and decision theory in the general machine learning context [1].
The theoretical literature analyses many alternative weighting strategies which can be distinguished
based on such criteria as: (a) the investor’s time horizon (e.g. does utility depend on realisations on
a single time horizon in a ‘one-shot’ scenario or does uncertainty resolve over multiple time
periods, affording the investor opportunities to alter portfolio composition dynamically?), (b) the
nature of the information available to investors regarding the distribution of future returns (this may
be extremely limited or highly-structured for mathematical expediency), and (c) the investor’s

198 Computer Science & Information Technology (CS & IT)
particular utility function (where, for instance, it can be shown that curvature can be interpreted as
representing the investor’s risk-preferences [2]).
One of the most prominent theoretical results is the concept of mean-variance efficiency which has
its roots in the work of Markowitz [3]: the idea is that in a one period model (under certain
restrictive assumptions) if investors seek to maximize return and minimise portfolio variance, the
optimal ex ante weighting vector w is given by
)(
1
= 1
ιµ
λ
rw −Ω−
(1)
where Ω is the covariance matrix of future returns, µ is the mean vector of expected returns, λ
is a risk-aversion parameter and r is the risk-free rate of return [4]. A key aspect of this formula is
the dependency on the inverse of the covariance matrix which is never known with certainty and
will in practice be a forecast in its own right (and the same will be true for µ and quite possibly r ).
When deploying this formula in real-world investment, practitioners are divided over how to
account for parameter uncertainty, with a number of alternative approaches in common usage
(including ignoring uncertainty entirely).
Unfortunately it is widely recognised that the exact weightings in (1) have a sensitivity to
covariance assumptions which is unacceptably high; in other words small changes in covariance
assumptions can lead to large changes in prescribed weightings. Further significant concerns are
raised by the fact that long time series are required to generate acceptable estimates for a large
covariance matrix but financial returns series are notoriously non-stationary – it is therefore easy
for an analyst to fall into the trap of thinking that they are applying prudent statistical methods when
in reality their input data may be stale or entirely inappropriate. The forecasting of expected returns
is also regarded as an exceptionally difficult task.
In these circumstances one strand of literature considers simpler weighting schemes which are
predicated on relatively few assumptions; one prominent example, popular with practitioners, is the
self-explanatory equally-weighted (or
n
1
) approach [5]. This method requires no explicit forecasts
of correlation or returns and it can be shown that this is equivalent to mean-variance methods if the
correlation between all possible pairs of investments is equal, along with all means and variances.
Although this may be far from the truth it may be more innocuous to assume this than to suffer
potentially negative effects of erroneous statistical forecasts and there is a body of empirical
literature which demonstrates the efficiency of the approach [6]. Refinements to the basic method
can include weighting each asset by the inverse of the forecast standard deviation of its returns
(known as volatility) which allows some heterogeneity to be incorporated.
Nevertheless it is intuitively obvious that such a simple method presents potential dangers of its
own, and is particularly inappropriate when the universe of alternative investments contains
subgroups of two or more investments which are highly correlated with each other. Suppose, for
instance, a portfolio of investments in world stock market indices which includes several
alternative indices for the United States (e.g. Dow Jones, S&P 500, Russell 2000) but only single
indices for other markets (e.g. the CAC-40 for France, FTSE-100 for UK, etc.). In this setting the
n
1
approach may (arguably) significantly overweight US equities in comparison to each foreign
market and in general regional weightings will be more dependent on the cardinality of available
indices than any economic properties of the markets. In a systematic investment process it is clearly

Computer Science & Information Technology (CS & IT) 199
impractical to have analysts manually sift through investments to ensure an appropriate ‘balance’
(which defeats the object of a weighting algorithm) and indeed potential diversification benefits
argue in favour of including a broad range of investments anyway.
The contribution of this paper is to explore potential weighting methods based on clustering, such
that highly ‘similar’ investments can be identified, grouped together and treated (for weighting
purposes) as if they are a single ‘composite’ investment. By contrast, investments which exhibit
relatively little similarity to each other are treated individually in their own right. Our focus here is
on a process for identifying clusters rather than evaluation of ex post investment performance,
which we leave for a separate analysis, and in fact we draw attention to the applicability of our
methods to fields beyond finance where clustering may be required, e.g. well-known problems in
biology, medicine and computer science. We also present an intriguing theoretical result arising
from our work, which emphasises limitations of certain clustering techniques and may help to
guide other researchers in their search for suitable methods.
The paper is organised as follows: in Section 2 we formally specify the problem at hand, in Section
3 we demonstrate spectral clustering as a preliminary benchmark approach and in Section 4 we
explore an alternative method based on a graphical model where we propose a specific estimation
technique involving triangular potentials and provide illustrative examples. Section 5 briefly
considers extension to a more dynamic setting (via a Hidden Markov Model) and Section 6
concludes.
2. PROBLEM SPECIFICATION
Definition 1 Let n ∈N, define },{1,=][ nn Λ the set of natural numbers from 1 to n.
Let }{,},{ 1 ntt Λ be n time series, where },,{=
1 miii ttt Λ for N∈nm, .
Definition 2 Clustering.
A clustering of }{,},{ 1 ntt Λ is an equivalence relation ~ over },{1,=][ nn Λ such that:
1. Reflexivity: If i ~ j then j ~ i.
2. Transitivity: If i ~ j and j ~ k then k ~ i.
Definition 3 Time dependent clustering.
We say i~
k
j if i and j are clustered at time k .
Our aim is to find a sequence {~
k
}k =1
m
, i.e. we allow the nature of the clustering relation to evolve
over time.
We denote the distance between series at time k as ),( ji
k
ttd for all ji, and the similarity at
time k defined as ),( ji
k
tts . The functions kk
sd , are specified by the user of the algorithm and
may be chosen based on prior domain-specific knowledge, or perhaps by a more systematic process
of searching across alternative specifications guided by out-of-sample performance.
Definition 4 Distance Matrix.
Given a family of time-dependent distance functions m
k
k
d 1=)},({ ⋅⋅ , we define a family of distance
matrices as )},({=, ji
kk
ji ttdD .

Definition 5 Similarity Matrix.
Given a family of time-dependent similarity functions m
k
k
s 1=)},({ ⋅⋅ , we define a family of similarity
matrices as )},({=, ji
kk
ji ttsS .
Definition 6 Similarization function.
We say [0,1]: →+Rz is a similarization function if for any distance function +→× RRRn
d :
, dz ο is a valid similarity function.
In what follows we restrict our attention to reflexive and non-negative distance and similarity
functions and thus to symmetric similarity and distance matrices. We will also use the variable n
to represent the number of data points observed at each time step when the clustering algorithm will
be applied.
3. SPECTRAL CLUSTERING
Here we introduce the Spectral Clustering algorithm, which is suitable for data where the cluster
structure does not change over time. Later in the paper we will compare the performance of our
proposed approach with this benchmark method.
Definition 7 The Laplacian matrix L of a similarity matrix S is defined as follows:
2
1
2
1
=
−−
− DD SIL
where ijjii S∑=D .
The most basic spectral clustering algorithm for bipartition of data is the Shi Malik bipartition
algorithm which we describe below.
3.1. Shi Malik algorithm
Given n items and a similarity matrix nn
S ×
∈R , the Shi Malik algorithm bipartitions the data
into two sets ),( 21 BB with ][=21 nBB ∪ and ∅∩ =21 BB based on the eigenvector v
corresponding to the second smallest eigenvalue of the normalized Laplacian matrix L of S .
Algorithm 1 The Shi Malik bipartition algorithm:
1. Compute the Laplacian from a similarity matrix.
2. Compute the second smallest eigenvalue and its corresponding eigenvector v .
3. Compute the median m of its corresponding eigenvector.
4. All points whose component in v is greater than m are allocated to 1B , the remaining
points are allocated to 2B .
Unfortunately the Shi Malik algorithm is not a dynamic procedure, i.e. it is not intended to identify
an underlying cluster structure which is time-varying. However various clustering approaches are
available which specifically seek to address this and we outline one such approach next.

3.2. A generalized spectral clustering approach
The following algorithm is an extension of the Shi Malik algorithm that can handle two or more
clusters. It can be found at [7]. Given n items and a similarity matrix nn
S ×
∈R the goal of
Dynamic Spectral Clustering is to find a clustering ~ of ][n .
Algorithm 2 Dynamic Spectral Clustering
1. Compute the Laplacian of the similarity matrix.
2. Compute the Laplacian’s eigenvalues and eigenvectors
3. Let c be a desired number of clusters.
4. Find the eigenvectors of the corresponding eigenvalues found on the previous step. Let the
corresponding cn× matrix be called V .
5. Rotate V , by multiplying it with an appropriate rotation matrix R so each of the
corresponding rows of VRZ = have (ideally) only one nonzero entry. In reality the
resulting matrix we will use the largest (in absolute value) entry of the matrix. R is a
rotation matrix in
cc
R ×
.
6. The cluster to which point i is assigned is ||maxarg ,},{1, jicj ZΛ∈ .
In order to find an appropriate rotation matrix R , there is a theorem that guarantees that any
rotation matrix
cc
RR ×
∈ can be written as a product kGG ⋅⋅Λ1 where k =
c(c −1)
2
and each
iG equals a Givens rotation matrix.
Givens rotation matrices ),,( θjiG are parameterized as follows:






















−
1000
0cossin0
0sincos0
0001
=),,(
ΛΛΛ
ΜΟΜΜΜ
ΛΛΛ
ΜΜΟΜΜ
ΛΛΛ
ΜΜΜΟΜ
ΛΛΛ
θθ
θθ
θjiG
Hence for each iG there is an associated angle iθ and we represent these k angles by the vector
1)/2( −
∈Θ cc
R . In order to find the optimal Θ for a given number of clusters c , we use gradient
descent on the following objective function:
2
1=1=
=min 





∑∑Θ
i
ij
c
j
n
i M
Z
J
subject to the constraint
.)(= cccncn RVZ ××× Θ
Following [7] we set ||max= ijji ZM .

As suggested by [7], the optimal number of clusters can be obtained by choosing the value of c
that maximizes a scoring function given by
.11=),( 





−−
n
ncq
J
3.2.1. A dynamic clustering algorithm
Given a family of time dependent similarity functions m
k
k
s 1=)},({ ⋅⋅ defining a family of similarity
matrices )},({=, ji
kk
ji ttsS , an optimal time-varying clustering structure ~
k
can be estimated by
applying Algorithm 2 at time k using input similarity matrix k
jiS , . Hence for time series data we
propose the following algorithm:
Algorithm 3 Let }{,},{ 1 ntt Λ be n time series. Where },,{=
1 miii ttt Λ for N∈nm, . Let
N∈w be a window parameter, +→×⋅⋅ RRR ww
d :),( be a distance function and
[0,1]: →+Rz be a similarization function.
1. Let m
ijD be the distance matrix having
]),,[],,,([=
11, mjwmjmiwmi
m
ji ttttdD ΛΛ
+−+−
for every pair ][, nji ∈ .
2. Let m
jiS , be the similarity matrix having )(= ,,
m
ji
m
ji DzS for every pair ][, nji ∈ .
3. Let ~
m
be the clustering resulting from running Algorithm 2 with input similarity matrix
m
jiS , .
4. Output clustering ~
m
.
Extensions of this approach include considering a geometric decay factor in the distance
computation, alternative distance functions and different similarization functions. We tried various
combinations as shown in Table 1 but found no significant improvement on the stability of the
resulting clusters.
We did not consider a scenario where the distance or similarization functions change through time
although there may be certain applications where this might be appropriate.
Table 1: Alternative distance and similarity functions. The second similarity function is a generalization of
the first.
Distances 1
L norm 2
L norm
Similarities exp −
x1
x1
−
x2
x2





 exp −c⋅
x1
x1
−
x2
x2






set to zero when it achieves
values less than λ for
different combinations of λ,c

Computer Science & Information Technology (CS & IT)
3.3. Overview
We present the performance of this algorithm in Figure 1. Some of the observed characteristics of
this method are the following:
• The resulting clustering values are notably sensitive to the similarity function used in the model.
• The clustering structure estimated by this method tends to be relatively unstable over time.
Although in some applications this may be plausible, in the context of financial time series we have
a strong prior belief that clusters typically arise due to com
fundamentals (e.g. similar commodities, currency pairs belonging to close trading partners, etc.)
which would tend to change very slowly relative to the frequency of market data.
Figure 1: Performance of the spectral clustering algorithm on 5,000 periods of synthetic data with
each time step we generate random standard normal variates which are common to each cluster, then for each
of the 3 returns we add independent Gaussian noise with
cluster therefore have a large portion of randomness in common, but each observation also includes its own
independent noise. The cluster structure is randomly changed over time and represented by coloured b
each row, i.e. all columns with the same colour belong in the same cluster.
4. GRAPHICAL MODEL A
Instead of representing clusterings as a binary matrix
j as the authors of [8] do, we approach the problem in a different way. Consider a symmetric (
ijji CC ,, = ) family of Bernoulli random variables,
€
Ci, j =
We wish to learn a distribution over the ensemble
is the following:
where S is a similarity matrix; in other words, we consider that the observed si
pair of points will come from one of two distributions, depending on whether or not the two points
belong to the same cluster.
In what follows it will be useful to think of the matrix
graph ),(= EVG where [=V
nodes that are in the same cluster. Learning a distribution over
a distribution over the set of undirected graphs
The resulting clustering values are notably sensitive to the similarity function used in the model.
The clustering structure estimated by this method tends to be relatively unstable over time.
a strong prior belief that clusters typically arise due to common factors relating to economic
which would tend to change very slowly relative to the frequency of market data.
ctral clustering algorithm on 5,000 periods of synthetic data with
of the 3 returns we add independent Gaussian noise with a relatively small variance. The members of each
independent noise. The cluster structure is randomly changed over time and represented by coloured b
each row, i.e. all columns with the same colour belong in the same cluster.
APPROACH
Instead of representing clusterings as a binary matrix ),( jiZ such that 1=),( jiZ if
as the authors of [8] do, we approach the problem in a different way. Consider a symmetric (
) family of Bernoulli random variables, ][,, }{= njijiC ∈C such that:
1 if i, j are in the same cluster, or 0 otherwise.
We wish to learn a distribution over the ensemble }{= , jiCC . The model we will use in this paper
S→C
is a similarity matrix; in other words, we consider that the observed similarity between a
In what follows it will be useful to think of the matrix }{ , jiC as an adjacency matrix. The resulting
][n and 1}=|),{(= , jiCjiE , has an edge between every two
nodes that are in the same cluster. Learning a distribution over }{ , jiC can be thought of as learning
a distribution over the set of undirected graphs ),( EV with ][= nV .
203
The resulting clustering values are notably sensitive to the similarity function used in the model.
The clustering structure estimated by this method tends to be relatively unstable over time.
mon factors relating to economic
ctral clustering algorithm on 5,000 periods of synthetic data with 3=n : at
a relatively small variance. The members of each
independent noise. The cluster structure is randomly changed over time and represented by coloured bars in
if ∈i cluster
as the authors of [8] do, we approach the problem in a different way. Consider a symmetric (
. The model we will use in this paper
milarity between a
as an adjacency matrix. The resulting
, has an edge between every two
t of as learning

The goal of this section is to compute the following posterior:
).|( SP C
The algorithms we present here output )|(maxarg SPC CC∈ , the MAP estimator for the posterior.
A short algebraic manipulation (Bayes Theorem) yields:
.
)(
)()|(
=)|(
SP
PSP
SP
CC
C
⋅
Since S is fixed:
).()|(maxarg=)|(maxarg CCC
CC
PSPSP
CC ∈∈
In the following two sections we present different models for inference on the ensemble C, their
performance and their relationship to clusterings.
The training data will be:
1. A set of similarity matrices m
i
k
jiS 1=, }{ .
2. The set of corresponding clusterings {~
k
} produced via a clustering algorithm such as the
ones described earlier in Section 3.
4.1. Exponential model
As a starting point we propose the following model for the ensemble C, in which we impose
conditional independence assumptions between observed similarities. We therefore assume the
following factorization:
)(),(
)(
1
=),(
)(
1
=)|( ,
2
,,
1
, jijijiji CSC
SZ
S
SZ
SP ΨΨΦ ∏CC
)(),(=)( ,
2
,,
1
, jijijiji
C
CSCSZ ΨΨ∏∑∈C
In this model we assume )|(=),( ,,,,
1
, jijijijiji CSPSCΨ , and )(=)( ,,
2
jiji CPCΨ . This is
equivalent to assuming full pairwise independence of the variables jiC , and the conditionals
)|( ,, jiji CSP .
For implementational purposes we assume jiji SC ,, → are exponentially distributed and the jiC ,
are Bernoulli random variables.
4.1.1. Training
{~
k
} can be translated into a training sequence of ensemble values }{ k
C via the transformation

1=,
k
jiC if
€
i~
k
j . Because of the independence assumptions underlying this model, the ML
estimate for the posterior distribution of the ensemble can be computed by obtaining the ML
estimate for each of the distributions
parameter of )|( ,, jiji CSP equals the inverse of the sample mean, and the ML estimate for the
mean of )( , jiCP equals the sample frequency of
Observation 1 Define ji
m
,
1
=λ
The ML estimator of the parameters for the posterior distribution
)|(
)(
1
=)|( ,, CC PSP
SP
SP jiji
€
4.1.2. Prediction
Prediction under this model is performed by finding the MAP assignment for the ensemble
and turning it into a clustering
)()|( ,,, jijiji CPCSP independently:
For the ensemble assignment *
C
component of the graph corresponding to
Figure 2: Performance of the exponential model on 5,000 periods of synthetic data with
The prediction algorithm is linear.
4.1.3. Limitations
Consider the following joint posterior distribution over clusterings of
. Because of the independence assumptions underlying this model, the ML
estimate for each of the distributions )|( ,, jiji CSP and )( , jiCP . The ML estimate for the rate
equals the inverse of the sample mean, and the ML estimate for the
equals the sample frequency of 1=, jiC . More formally:
k
jik
S ,∑ . And let k
jikji C
m
p ,,
1
= ∑ .
)(CP has
€
P(Si, j |Ci, j ) ~ exp(λi, j ) and jiCP , 1)=(
Prediction under this model is performed by finding the MAP assignment for the ensemble
nd turning it into a clustering. *
C is obtained by maximizing each likelihood
independently:
).()|(maxarg= ,,,
{0,1},
*
, jijiji
jiC
ji CPCSPC
∈
*
we output a clustering composed of a cluster for each connected
component of the graph corresponding to *
C . Results are presented in Figure 2.
2: Performance of the exponential model on 5,000 periods of synthetic data with
The prediction algorithm is linear.
Consider the following joint posterior distribution over clusterings of {1,2,3}.
205
. Because of the independence assumptions underlying this model, the ML
. The ML estimate for the rate
equals the inverse of the sample mean, and the ML estimate for the
jip ,=1) .
Prediction under this model is performed by finding the MAP assignment for the ensemble *
C
is obtained by maximizing each likelihood
we output a clustering composed of a cluster for each connected
2: Performance of the exponential model on 5,000 periods of synthetic data with 3=n .

The marginals
€
p((1,2)),p((2,3))
4.2. Triangular Potentials
The main limitation of the approach described in the previous section is that there is potential for
spurious large clusters to emerge solely from the independent optimization of the potentials. If the
marginal probability jip , is large,
regardless of the values of any of the other similarities
also possible for the algorithm to suggest cluster shapes which are intuitively implausible (and do
not conform to prior notions of cluster structure which may be appropriate to a particular domain);
we illustrate this in Figure 3.
Figure 3: Alternative cluster structures: (top to bottom) the first implausible configuration is ruled
use of triangular potentials, however the second and third configurations are possible (which is our
We therefore proceed to address these issues by a modification to the basic model as described by
the following observations:
Observation 2 C is a valid clustering
€
1=1== ,,, ikkjji CCC ⇒ .
Observation 3 C is a valid clustering if the graph whose adjacency matrix equals
composed of a disjoint union of cliques.
In this section we assume the following factorization:
(
1
=)|(
SZ
SP C
p(⋅) =
0.1 if ⋅ = (1,2,3)
0.41 if ⋅ = (1,2),(3)
0.41 if ⋅ = (1,3),(2)
0 if ⋅ = (2,3),(1)
0.17 if ⋅ = (1),(2),(3)








3)) > 0.5. The current algorithm will output (1,2,3)
is large, it is likely that the MAP of the ensemble C will have
regardless of the values of any of the other similarities mkS , or clustering assignments
algorithm to suggest cluster shapes which are intuitively implausible (and do
3: Alternative cluster structures: (top to bottom) the first implausible configuration is ruled
, however the second and third configurations are possible (which is our
deliberate intention).
ed to address these issues by a modification to the basic model as described by
is a valid clustering ~ if, for all triplets of distinct numbers , ji
is a valid clustering if the graph whose adjacency matrix equals
composed of a disjoint union of cliques.
In this section we assume the following factorization:
),(
)
1
S
S
CΦ′
.
will have 1=, jiC
or clustering assignments mkC , . It is
algorithm to suggest cluster shapes which are intuitively implausible (and do
3: Alternative cluster structures: (top to bottom) the first implausible configuration is ruled-out by the
, however the second and third configurations are possible (which is our
ed to address these issues by a modification to the basic model as described by
][, nkj ∈ ,
is a valid clustering if the graph whose adjacency matrix equals C is

),,()(),(
)(
1
= ,,,
3
,,
,,
,
2
,,,
1
,
,
kjkijikji
kji
jijijijiji
ji
CCCCSC
SZ
Ψ







ΨΨ ∏∏ (2)
),,()(),(=)( ,,,
3
,,
,,
,
2
,,,
1
,
,
kjkijikji
kji
jijijijiji
jiC
CCCCSCSZ Ψ







ΨΨ ∏∏∑∈C
where
Ψi, j,k
3
(Ci, j ,Ci,k,Cj,k ) =
0 if Ci, j = Ci,k =1,Cj,k = 0
0 if Ci, j = Cj,k =1,Ci,k = 0
0 if Ci,k = Cj,k =1,Ci, j = 0
1 otherwise







This has the effect of turning ),( SCΦ into a potential function ),( SCΦ′ such that all the
assignments of the joint distribution of the ensemble C with a nonzero probability are valid
clusterings.
4.2.1. Training algorithm
We use the same construction for the univariate and bivariate potentials as the one used in the
previous section. The distribution over clusterings will vary because the triangular potentials
restrict the mass of the distribution to the space of valid clusterings. It is of course also possible to
add other potentials relating different sets of clustering variables although we leave that direction
for future research.
4.2.2. Prediction algorithms
This model can be thought of as an undirected graphical model with variables }{ , jiC for ji <
and ][, nji ∈ and edges kiji CC ,, , , kjji CC ,, , and jikj CC ,, , for all kji << . If the variable
jiC , is identified with the point ),( ji , then there is an edge between every two variables on the
same vertical line and between every two variables on the same horizontal line.
We tackle the problem of obtaining the MAP assignment over clusterings under this model using
either the Elimination Algorithm or MCMC. To obtain an estimate for the MAP assignment using
MCMC we sample from the posterior and output the clustering arrangement which appears most
often. The MCMC chain construction is described in the next section.
By construction there is a clique of size 1−n along the horizontal line )(1,i for ni ,2,= Λ As a
consequence, the elimination algorithm has an exponential running time over this graphical model.
Similarly, there are no easy theoretical guarantees for the performance of the MCMC method. In
particular, it is possible for the probability mass over the optimal assignment to be so small that
there are no concentration inequalities to guarantee that the proposed algorithm will output the
MAP with high probability in polynomial time.
In the following section we show this behavior is not only a result of the graphical model
formulation or our proposed algorithm but an intrinsic limitation of the model itself.

4.3. Results and Limitations
We next apply the classic sumproduct algorithm or the MAP elimination algorithm to find the best
clustering, with results shown in Figure 4, however the drawbacks are that this solution becomes
intractable as the number of products becomes large. The elimination algorithm could be worst case
2
2n
which becomes intractable quite fast.
Figure 4: Performance of the graphical model with triangular potentials;
4.3.1. Theoretical limitations
Let [0,1]ˆ , ∈jip be an ensemble of probabilities with
Define a distribution over simple graphs via
=)(GP
Let ˆP(G) = P(G |G is a disjoint union of cliques
It is easy to see that finding the MAP assignment for the dist
equivalent to finding the MAP assignment for
(
=ˆ ,
i
ji
SP
p
Since it is conceivable that any arrangement of the values
the two problems are equivalent.
ults shown in Figure 4, however the drawbacks are that this solution becomes
which becomes intractable quite fast.
4: Performance of the graphical model with triangular potentials; 3=n
be an ensemble of probabilities with ji ≠ such that ijji pp ,,
ˆ=ˆ and
Define a distribution over simple graphs via
].[=),,(=)ˆ(1ˆ ,
),(
,
),(
nVEVGpp ji
Eji
ji
Eji
∀







−







∏∏ ∈∈
is a disjoint union of cliques ).
It is easy to see that finding the MAP assignment for the distribution defined via Equation (2)
equivalent to finding the MAP assignment for )(ˆ GP with:
=(0)=|(1)=(1)=|
1)=(1)=|(
,,,,,,
,,,
jijijijijiji
jijiji
CPCSPCPC
CPCSP
+
Since it is conceivable that any arrangement of the values jip ,ˆ can result from the training data,
the two problems are equivalent.
ults shown in Figure 4, however the drawbacks are that this solution becomes
3.
and ][, nji ∈ .
ribution defined via Equation (2) is
0)=
can result from the training data,

Theorem 1 If there is a polynomial time algorithm for finding the MAP assignment over )(ˆ GP
then P = NP.
Proof. Let })ˆ({ , jipA be an algorithm for finding the MAP over the distribution )(ˆ GP as
defined by }ˆ{ , jip . We show A can be used to construct an algorithm for solving the −k clique
problem. The −k clique problem is the problem of deciding whether a graph ),(= EVG has a
clique of size k where both G and k are inputs to be specified. If A was polynomial, the
algorithm we propose for −k clique would run in polynomial time. Because −k clique is NP
complete we conclude the existence of A would imply P = NP. The following algorithm solves
−k clique:
Algorithm 4 Inputs: >,< kG .
Let |}|2,2||{2max> VEN +⋅ and
2
1
>q .
1. Construct ),(= EVG ′′′ with NVVV ∪′ = and
},,|},{{},|},{{= 2121212121 vvVvvvvVvVvvvEE NN ≠∈∪∈∈∪′ . The edges of G′ equal all
edges in G , plus all possible edges between V and NV and all possible edges among elements
of NV .
2. For all pairs |][|, Vji ′∈ define:


 ∈/
otherwise
,if0
=ˆ ,
q
Eji
p ji
3. Let *ˆE be the output edges in the MAP assignment from })ˆ({ , jipA .
4. If kEE ≥∩ |)ˆ(MaxClique| *
output 1, else output 0 . This step runs in polynomial time
because every connected component of EE ∩*ˆ is a clique graph.
The probability of the MAP assignment equals








−








∏∏
∈/∈
)ˆ(1ˆ ,
*ˆ),(
,
*ˆ),(
ji
Eji
ji
Eji
pp
which can be written as a product ***
VVNN PPP ×⋅ of the product of the chosen probabilities of pairs
belonging to NN VV × , a cross component of probabilities from VVN × and a component of
probabilities from VV × . By construction, the edges in VV × but not in E are not chosen. The
MAP restricted to NV and V is a disjoint union of cliques. Because 1/2||*
< −
≤ NE
V qqP we can
conclude:
1. The MAP assignment restricted to NV must be a complete graph: Suppose the MAP
restricted to NV had more than one component, say rKK ,,1 Λ , |||| 1 rKK ≥≥Λ with
11
1 ,, rKK Λ their (possibly empty) corresponding clique intersections in V . It can be shown via
the rearrangement inequality that the MAP must have |||| 11
1 rKK ≥≥Λ . Let MAP1 be the
assignment obtained via joining rKK ,,1 Λ into NK (the complete graph on NV ) and

reconnecting all to 1
1K . If
)(edges)(edges 1=
KK i
r
iN ≥− ∑
)MAP(>)MAP1( PP , a contradiction.
2. The VVN × edges must connect
The correctness of the algorithm follows. The algorithm above runs in polynomial time, provided
A is in P.
5. EXTENSIONS
5.1. HMM
Because the training procedure we propose is done over fully annotated data, more sophisticated
and time-dependent models can be explored. We propose a generalization of the previous models
via an HMM.
In this model, each hidden state is a clustering and the transition probabilities are obtained from the
sampled frequencies of the transitions in the
data are known, the ML estimate of the transition probabilities of an HMM equals the transitions
sample frequencies.
The results of applying this method are show
performance is achieved.
Figure 5: Performance of the HMM clustering algorithm on 2,000 periods of synthetic data with
The version implemented here is hard
clustering states. The length of the chain can be adjusted as desired.
5.2. Coagulation Fragmentation
The underlying chain for the MCMC sampler uses a fragmentation coagulation process to walk
over clusterings. At each step, the chain either selects a random cluster, and divides it into two, or
selects two random clusters and joins them together. The acceptance/rejection probabilities can be
computed with respect to any coagulation fragmentation p
either a uniform random cluster and a random bipartition of it (fragmentation), or a uniform random
pair of clusters (coagulation). We believe the mixing time of this process should be fast as it is
related to a coagulation fragmentation process known as the random transposition walk. Diaconis
and Shahshahani provided a polynomial upper bound for this walk’s mixing time [9].
. If 2≥r , a simple counting argument shows that
2
N
. The latter, and |||| 11
1 rKK ≥≥Λ
, a contradiction.
edges must connect NV with one of the largest cliques of G .
dependent models can be explored. We propose a generalization of the previous models
sampled frequencies of the transitions in the training phase. When the hidden states of the training
The results of applying this method are shown in Figure 5, where it is apparent that relatively good
5: Performance of the HMM clustering algorithm on 2,000 periods of synthetic data with
The version implemented here is hard-coded for only 3 series and therefore only
clustering states. The length of the chain can be adjusted as desired.
Coagulation Fragmentation
er clusterings. At each step, the chain either selects a random cluster, and divides it into two, or
computed with respect to any coagulation fragmentation process. In our implementation, we pick
lation fragmentation process known as the random transposition walk. Diaconis
, a simple counting argument shows that
imply that
.
dependent models can be explored. We propose a generalization of the previous models
training phase. When the hidden states of the training
t relatively good
5: Performance of the HMM clustering algorithm on 2,000 periods of synthetic data with 3=n .
series and therefore only 5 possible
er clusterings. At each step, the chain either selects a random cluster, and divides it into two, or
rocess. In our implementation, we pick
lation fragmentation process known as the random transposition walk. Diaconis

5.2.1. Alternative model
We believe a worthwhile alternative to the ideas described above is to represent the clustering
evolution as an HMM on fragmentation-coagulation parameters: the simplest model having only
two parameters ),( qp , one controlling the probability of fragmentation and the other controlling
the probability of coagulation. If the number of fragmentation-coagulation parameters is small,
inference could be tractable.
6. CONCLUSIONS
Our intention in this paper has been to show how various clustering methods can be applied to
datasets which arise in financial markets. We have documented the process by which we analysed
the problem and considered a method for determining clusters using triangular potentials. This
latter method can be computationally intensive and we have provided some preliminary theoretical
results concerning its limitations. However, notwithstanding these considerations, we have found
promising empirical results from applying the method to simulated datasets and we look forward to
extending this to real-world data in due course.
In future work we aim to extend the idea to a setting where we place a non-uniform prior on
clusterings, e.g. if expert knowledge suggests that a group of investments are likely to share similar
return characteristics then we can configure potentials such that appropriate weighted links are
established among these products.
There is also considerable scope to investigate efficiency improvements to the MCMC estimation
process, based on the particular structure of potentials in this context.
REFERENCES
[1] Koller, D. & Friedman, N. (2009) Probabilistic Graphical Models, MIT Press.
[2] Mas-Colell, A., Whinston, M.D. & Green, J.R. (1995) Microeconomic Theory, Oxford University
Press.
[3] Markowitz, H. (1952) ‘Portfolio Selection’, Journal of Finance, Vol. 7, No. 1.
[4] Ingersoll Jr., J.E. (1987) Theory of Financial Decision Making, Rowman and Littlefield.
[5] Benartzi, S. & Thaler, R.H. (2001) ‘Naive diversification strategies in defined contribution saving
plans’, American Economic Review, 91(1), pp. 79-98.
[6] De Miguel, V., Garlappi, L. & Uppal, R. (2009) ‘Optimal versus naive diversification: How inefficient
is the portfolio strategy?’, Review of Financial Studies, 22.
[7] LaViers, A., Rahmani, A. & Egerstedt, M. (2010) ‘Dynamic Spectral Clustering’, Proceedings of the
19th International Symposium on Mathematical Theory of Networks and Systems – MTNS 2010, July.
[8] Chi, Y., Song, X., Zhou, D., Hino, K. & Tseng, B.L. (2007) ‘Evolutionary Spectral Clustering by
Incorporating Temporal Smoothness’, Proceedings of the 13th ACM SIGKDD International
Conference on Knowledge Discovery and Data Mining, ACM, New York.
[9] Diaconis, P. & Shahshahani, M. (1981) ‘Generating a Random Permutation with Random
Transpositions’, Zeitschrift für Wahrscheinlichkeitstheorie und Verwandte Gebiete, 57(2), pp 159-179.

AUTHORS
Aldo Pacchiano is a Bachelor of Science in Computer Science and Mathematics from MIT,
with a masters degree in Mathematics from Cambridge University and a recently obtained
Masters of Engineering degree from MIT; his interests span the areas of Machine Learning,
Computational Biology and he has a special interest in quantitative finance.
Oliver Williams is a quantitative investment specialist with interests including financial
economics, asset pricing and systematic trading; he has co-authored a number of papers in
this area and his career has been spent in investment banking and asset management. He
holds an MA in Computer Science and Management Studies, MPhil and PhD in Financial
Economics from Cambridge University.

Real time clustering of time series

More Related Content

What's hot (19)

Viewers also liked (20)

Similar to Real time clustering of time series (20)

Recently uploaded (20)

Real time clustering of time series