Learning in non stationary environments

Chapter 3
A Granular Description of Data: A Study
in Evolvable Systems

Witold Pedrycz, John Berezowski, and Iqbal Jamal

Abstract A human-centric way of data analysis, especially when dealing with
data distributed in space and time, is concerned with data representation in an
interpretable way where a perspective from which the data are analyzed is actively
established by the user. Being motivated by this essential feature of data analysis,
in the study we present a granular way of data analysis where the data and
relationships therein are described through a collection of information granules
deﬁned in the spatial and temporal domain. We show that the data, expressed in
a relational fashion, can be effectively described through a collection of Cartesian
products of information granules forming a collection of semantically meaningful
data descriptors. The design of the codebooks (vocabularies) of such information
granules used to describe the data is guided through a process of information
granulation and degranulation. This scheme comes with a certain performance index
whose minimization becomes instrumental in the optimization of the codebooks. A
description of logical relationships between elements of the codebooks used in the
granular description of spatiotemporal data present in consecutive time frames is
elaborated on as well.

W. Pedrycz ( )
Department of Electrical & Computer Engineering University of Alberta, Edmonton AB T6R
2G7 Canada, and System Research Institute, Polish Academy of Sciences, Warsaw, Poland
e-mail: wpedrycz@ualberta.ca
J. Berezowski
Ross University School of Veterinary Medicine, St Kitts, West Indies
e-mail: john.berezowski@gov.ab.ca
I. Jamal
AQL Management Consulting Inc., Edmonton, AB, Canada
e-mail: iqbaljamal@aqlmc.com

M. Sayed-Mouchaweh and E. Lughofer (eds.), Learning in Non-Stationary Environments: 57
Methods and Applications, DOI 10.1007/978-1-4419-8020-5 3,
© Springer Science+Business Media New York 2012

58 W. Pedrycz et al.

3.1 Introduction

Spatiotemporal data become visible in numerous applications. We envision data
distributed in time and space: readings associated with oil wells distributed over
space and reported in time, recordings of sensors distributed over some battleﬁeld
area or disaster region, counts of animal disease recorded in several counties over
time, etc.—all of those are compelling examples of spatiotemporal data.
Spatial and temporal phenomena (systems) can be effectively analyzed and
described at various levels of abstraction. The selected level of abstraction is of
paramount importance as it helps realize a user-friendly approach to data analysis. In
essence, the user can establish the most suitable perspective to view the data (iden-
tifying a suitable level of detail to be included in the analysis); making an overall
analysis more humancentric. This view inherently brings a concept of information
granules into the picture. Information granules, no matter how they are formalized
(either as sets, fuzzy sets, rough sets, or others) are regarded as a collection of
conceptual landmarks using which one looks at the data and describes them. We
show that the relational way of capturing data naturally invokes the mechanisms of
relation calculus where the concepts of granulation and degranulation are crucial to
the optimization of the vocabulary (codebook) of information granules by means
of which the main dependencies in data become revealed and quantiﬁed. The key
objective of this study is to introduce a relational way of data analysis through their
granulation and degranulation and show a constructive way of forming information
granules leading to the best granular data representation. Furthermore, we introduce
some way of characterizing relationships between granular codebooks pertaining to
the description of data collected in successive time frames.
The presentation of the material is organized in a top–down fashion. We start
with an illustrative practical example (Sect. 3.2) and then move on to the problem
statement (Sect. 3.3) by stressing the relational character of data to be processed.
A mechanism of granulation of information and a way of representing data in the
language of information granules is covered in Sect. 3.4, which leads to a concept
of a so-called granular signature of data (Sect. 3.5). The fundamental concept of
information granulation–degranulation is introduced in Sect. 3.6 where we also
show how a vocabulary (codebook) of information granules can be optimized
so that the reconstruction of data implies a minimal value of the reconstruction
error. With regard to this optimization, it is discussed how fuzzy clustering leads
to the realization of information granules (Sect. 3.7). The issue of evolvability of
the relational description of data emerging in presence of a series of data frames
reported in successive time slices is covered in Sect. 3.8.

3 A Granular Description of Data: A Study in Evolvable Systems 59

3.2 From Conceptual Developments of Information Granules
to Applications

As an example of a real-world problem whose formulation and solutions can be
naturally cast within the conceptual framework to be discussed here, relates to
a comprehensive analysis of spatiotemporal data and associated event detection
schemes formed directly on a basis of information granules. An interesting system
where we encounter a wealth of spatiotemporal data is found at http://www1.agric.
gov.ab.ca/department/$deptdocs.nsf/all/cl12944. It concerns a set of data collection
stations distributed across the province of Alberta, see Fig. 3.1. This system helps
graph, compare, and download data in almost real-time mode for more than
270 stations where data themselves go back to April 2005. There are some
other data sets like precipitation and temperature (normal), temperature extremes,
solar radiation, soil temperature, and other important weather characteristics. In
a nutshell, we encounter a collection of spatial data—stations identiﬁed by some
x–y coordinates. For each station, there is an associated suite of time series (say,
dealing with the temperature, precipitation, radiation), see Fig. 3.2. An analysis

Fig. 3.1 A screen snapshot of the AgroClimatic Information Service (ACIS)


Fig. 3.2 Collection stations along with associated time series

Fig. 3.3 From collection
stations to spatiotemporal
information granules
(clusters)

of such spatiotemporal data offers interesting and practically relevant insights into
their nature and helps identify main trends and relationships. Several challenging
problems are worth identifying in this context:
(a) Detection of information granules in the spatiotemporal data space, Fig. 3.3.
As shown there, the resulting information granules form regions (information
granules) in the space x–y by bringing together points (stations) characterized
by similar temporal data (time series) and being in close vicinity to each
other. From the algorithmic perspective, one can envision using a clustering
algorithm (which has to be carefully designed to capture the spatial as well as


Fig. 3.4 Evolution of information granules through revealing dependencies among information
granules observed in successive discrete time moments (determined by the predetermined length
of the time horizon T)

temporal component). This implies that we have to carefully specify a distance
function used to quantify the closeness of elements in the spatiotemporal data
space. From the perspective of practical application, by forming the information
granules the user can get a better insight into the regions of the province, their
location, and size which exhibit some level of inner homogeneity while being
distinct from the others. Such findings could be of immediate value to decision-
makers in agriculture sector.
(b) Alluding to the constructs noted above, it is apparent that the temporal compo-
nent of the data depends directly on the length of the corresponding time series.
In particular, this impacts a way in which time series are represented (as the
time series could be either stationary or nonstationary, which is usually the case
when dealing with longer time horizons for which the information granules are
to be formed). This brings a concept of the temporal scale (which again triggers
some temporal granularity). The information granules are constructed over
some time horizon. They evolve so in successive time moments, say T, 2T, . . .
we obtain a collection of information granules, see Fig. 3.4.
These information granules are related. By describing relationships (dependen-
cies) between the information granules arising in time horizons T, 2T, etc., an
interesting dynamic pattern of changes (dynamics) at a more abstract level can be
revealed. It could be regarded as a generalization of the dynamic process where now
the dynamics is captured at a certain nonnumeric level and with the varying number
of granules, we witness changes in the level of specificity of the granular descriptors
(which becomes reflective of the varying level of complexity of the associated time
series).
The data collected in the system described above can be used in conjunction with
other sources of data (e.g., related with livestock and humans in the realization of
event detection systems). Those are essential to prevention of disasters and effective
risk management for livestock production, public health, and rural communities.
The need for animal health surveillance systems and ensuing decision-making
architectures capable of rapid disease detection is of paramount relevance. This is


exacerbated by a number of factors. The speed of transportation has never been
faster, facilitating the rapid spread of pathogens around the world, and makes timely
detection more difficult. The average time to transport people and goods around the
world is less than the incubation period for most infectious diseases. The volume
of people, animals, and animal products that is moved everyday is large. Event
detectors, which could be sought as an integral functional component are formalized
and described in the language of information granules and are built on a basis of
reported weather data, disease cases reported and a way of their spread as well as
an intensity of spread itself [3–5, 7, 8]. All these phenomena can be conveniently
described and operated on through the calculus of information granules.

3.3 Problem Formulation

The data we are concerned with are of spatiotemporal character. The temporal data
(time series) are associated with some geographical x–y coordinates. Their descrip-
tion is provided in the form of a certain vector z positioned in the p-dimensional
unit hypercube. We can encounter here a genuine diversity of available ways used
to describe time series such as parameters of the autoregressive time series model
(AR model), coefficients of the Fourier expansion, cepstral coefficients, components
of discrete wavelet transforms (DWT), parameters of fuzzy rule-based system (in
case of fuzzy modeling of the series), and others. For instance, in case of the AR
model we have z = [a0 a1 a2 . . . as ]T where the model comes in the form z(K + 1) =
∑sj=0 a j z(K − j). One could also encounter models that are more specialized to fit a
certain category of signals such as this is the case in ECG signals modeled in terms
of Hermite expansion. In all these cases, we assume that the values of the descriptors
have been normalized so that the results are located in the [0, 1] p hypercube.
The available data come in the form of the triples (xk , yk , zk ) with xk = [xk yk ]T
being the spatial coordinates of the temporal data zk , k = 1, 2, . . . , N. The equivalent
representation that is convenient in our considerations, comes in a form of a fuzzy
relation R(x, y, z) assuming vector values in [0, 1] p for specific values of xk and yk .
In other words, the above notation underlines that the pair (xk , yk ) is associated with
a vector of temporal data zk . An illustration of an example fuzzy relation is included
in Fig. 3.5.
Given the relational data R, we are interested in the following two important
characterization and design issues:
(a) A description of R realized in a terms of a certain vocabulary of granular
descriptors, see Fig. 3.6. We form a certain focused view at the data provided
by the vocabulary. It has to be noted, as shown in Fig. 3.6, that different
vocabularies (codebooks) facilitate different, sometimes complementary views
at the available experimental evidence.
(b) Analysis of granulation and degranulation capabilities of the vocabularies. The
intriguing question is about capabilities of the given vocabulary to capture the
essence of the data (granular descriptors). Subsequently based upon the granular


Fig. 3.5 Visualization of the fuzzy relation R; associated with some combination of the (x, y)-
coordinates are sine—like time series characterized, e.g., by a collection of their Fourier expansion
coefficients

Fig. 3.6 Formation of various views at relational data supported by the use of different
vocabularies

representation of the data, we discuss how to reconstruct (degranulate) the
original data, see Fig. 3.7. The quality of this reconstruction (as we anticipate
that the process of reconstruction could impart some losses) can be quantified
and the figure of merit associated with this process can be used in the
optimization of the codebook.

3.4 Granulation of Information and Granular Data
Representation

We characterize the available data R in terms of a collection of information granules-
semantically meaningful entities [1, 2, 6, 17] that are formed in the spatial and
temporal coordinates. Formally, information granules can be captured in different
ways, say in the form of sets, rough sets, fuzzy sets, to name a few alternatives.
Here we are concerned with their realization by means of fuzzy sets. We define a


Fig. 3.7 The concept of
granulation and degranulation
realized with the aid of a
vocabulary of information
granules

collection of spatial information granules (located in the x- and y-coordinates) and
temporal information granules used to describe the spatiotemporal signal

Ai : R → [0, 1], i = 1, 2, . . . , n
B j : R → [0, 1], j = 1, 2, . . . , m
Cl : [0, 1] p → [0, 1] p, l = 1, 2, . . . , r

Those are fuzzy sets described by the corresponding membership functions, that
is Ai (x), B j (y), and Cl (z). The collections of these granules are organized in the
form of families of fuzzy sets A, B,C, respectively. For instance, we have A =
{A1 , A2 , . . . , An }, B = {B1 , B2 , . . . , Bm } and C = {C1 ,C2 , . . . ,Cr }. These fuzzy sets
come with a well-defined meaning. For instance, we may have linguistic descriptors
such as
• Ai —east region of x-coordinate, B j —north region of the y-coordinate.
• Cl —high frequency and low amplitude of the first component of the Fourier
expansion of the signal.
See Fig. 3.8. We form Cartesian products of the granular descriptors. For instance,
the Cartesian product of Ai and B j defined above, the Cartesian product Ai × B j ,
can be regarded as a descriptor of the region of x − y space located in the north-east
region of the plane.
The Cartesian product of the above information granules can be treated as
a granular descriptor—a linguistic “probe” used to describe the available data.
A collection of all probes is used to describe/perceive the essence of data. Taken
altogether they can be sought as a granular codebook (vocabulary) supporting a
concise description of the collected experimental evidence.
The number of information granules for the spatial aspect of data could be
reduced if we consider treating both coordinates en block meaning that a family of
fuzzy sets is with respect to x- and y-coordinate. This gives rise to the membership
functions of both arguments, say A1 (x, y), A2 (x, y), . . . .


Fig. 3.8 Examples of linguistic descriptors used in the granular representation of relational data:
spatial granular descriptors Ai and B j (a) and spatial granular descriptor Cl (b)

3.5 A Granular Signature (Descriptor) of Relational Data

Given the collection of information granules—vocabularies {Ai , B j ,Cl }i =
1, 2, . . . , n; j = 1, 2, . . . , m; l = 1, 2, .., r they are used to describe (represent) the
relational data R. This description is realized by performing a convolution of the
elements of the vocabularies A, B, and C with the fuzzy relation of data, cf. [14]. This
convolution is realized in a way it is quite commonly present in signal processing
as well as the technology of fuzzy sets [16] that is

gi jl = (Ai × B j × Cl ) ◦ R (3.1)

i = 1, 2 . . . , n; j = 1, 2, . . . , m; l = 1, 2, . . . , r. Expressing the fuzzy sets of the code-
books in terms of the corresponding membership functions we obtain the values of
gi jl as a result of the max-t composition operator [14] of the fuzzy sets and the fuzzy
relation
gi jl = max [Ai (xk )tB j (yk )tCl (z)tR(xk , yk , zk )] (3.2)
xk ,yk ,zk

where t stands for a certain t-norm [12] (the minimum operator can be viewed as a
particular case).
This max-t composition comes with a useful interpretation: gi jl is then a
possibility measure of the relational data with respect to the selected information
granules indexed (i, j, l), Poss(Ai , B j ,Cl ; R). The possibility measures arise here in
an explicit manner: they describe a degree of activation of a certain element of the
Cartesian product of the codebooks by the fuzzy relation R. As an example, let us
consider the fuzzy relation R with the following entries
⎡ ⎤
1.0 0.7 0.2 0.3
⎣ 0.5 0.9 1.0 0.0 ⎦
0.0 0.4 0.8 1.0


and
⎡ ⎤
0.0 0.2 0.7 1.0
⎣ 1.0 0.3 0.6 0.3 ⎦
0.3 0.5 0.5 0.2

The columns of R are indexed by x-coordinates while the rows are linked with the
y-coordinate. The first relation includes the first coordinates of zk , zk1 . The second
coordinates of zk form the second matrix. With each entry of the matrix associated
are the successive descriptors of the time series; the length of the descriptor vectors
of the time series is equal to 2. For instance, considering the location at (3, 3)—entry
of the relation R, the descriptor of the time series positioned there is zk = [0.8 0.5]T
while zk = [0.0 0.3]T for the entry (4, 2).
The vocabularies formed by information granules are as follows:
• For A with n = 2, A1 = [1.0 0.5 0.2 0.0], A2 = [0.3 1.0 0.7 0.6]
• For B with m = 2, B1 = [1.0 0.6 0.3], B2 = [0.0 0.5 1.0]
• For C with r = 2, C1 = [1.0 0.2], C2 = [0.1 1.0]
Let us now compute g111 . Following the formula (3.2), we proceed with the
successive calculations, g111 = maxx,y [min(min(A1 (x), B1 (y)), maxz (min(C1 (z),
R(x, y, z)))]. The inner expressions result in the following:
⎡ ⎤
1.0 0.5 0.2 0.0
(A1 × B1)(x, y) = 1.0 0.5 0.2 0.0 1.0 0.6 0.3 = ⎣ 0.6 0.5 0.2 0.0 ⎦
T

0.0 0.0 0.0 0.0
⎛⎡ ⎤⎡ ⎤⎞
1.0 0.7 0.2 0.3 0.0 0.2 0.2 0.2
max(min(C1 (z), R(x, y, z))) = max ⎝⎣ 0.5 0.9 1.0 0.0 ⎦, ⎣ 0.2 0.2 0.2 0.2 ⎦⎠
z
0.0 0.4 0.8 1.0 0.2 0.2 0.2 0.2
⎡ ⎤
1.0 0.7 0.2 0.3
= ⎣ 0.5 0.9 1.0 0.2 ⎦
0.2 0.4 0.8 0.0
⎡ ⎤
1.0 0.5 0.2 0.0
Finally, we find the minimum of the corresponding entries of ⎣ 0.6 0.5 0.2 0.0 ⎦
0.0 0.0 0.0 0.0
⎡ ⎤ ⎡ ⎤
1.0 0.7 0.2 0.3 1.0 0.5 0.2 0.0
and ⎣ 0.5 0.9 1.0 0.2 ⎦, which yields the relation ⎣ 0.5 0.5 0.2 0.0 ⎦. The maxi-
0.2 0.4 0.8 0.0 0.0 0.0 0.0 0.0
mum taken over all entries of this relation returns g111 = 1.0.
The result conveyed by (3.2) can be interpreted as a granular characterization
of R. Obviously, when we consider some other granular codebook, we arrive at
a different description or view of the same data. The flexibility available here is
essential; we customize the analysis of the data and the mechanism of customization


Fig. 3.9 Realization of the
granular signature G of
relational data R

is offered in the form of the vocabulary. The elements of gi jl being arranged into a
single fuzzy relation G = [gi jl ] can be treated as a granular signature of R realized
for a certain collection of generic information granules (codebook) A, B, C, refer
to Fig. 3.9. It is worth noting that G supplies an important compression effect: the
original data are reduced to granular data of dimensionality n ∗ m ∗ r. Obviously, the
reduction depends on the size of the codebook. The Cartesian products of elements
of A × B × C of information granules can be ordered linearly depending upon the
values of G = [gi jl ]; higher values of its entries indicate that the corresponding
entries are more consistent, supported by experimental evidence conveyed by R.
In this way, we can identify the most essential relational descriptors of R. More
speciﬁcally, the relational data R translate into a family of triples of descriptors
coming from the Cartesian product of the vocabularies A × B × C indexed by the
associated values of possibilities:

R = {(Ai1 , B j1 ,Cl1 ), gi1 j1l1 } (3.3)

We organize the entries of the granular representations gi1 j1l1 in a nonincreasing
order; the ﬁrst entries of the sequence provides the essential granular descriptors of
R, that is gI ≤ gJ ≤ gK where I = (i1 , j1 , l1 ), J = (i2 , j2 , l2 ), etc.

3.6 The Concept of Granulation–Degranulation: A Way
of Describing Quality of Vocabulary

Given the granular signature of the relational data captured by G as well as the
elements of the vocabulary itself, we can reconstruct R. The result is an immediate
consequence that (3.2) is a fuzzy relational equation and for each combination of the
elements of A and B, we arrive at an individual fuzzy relational equation indexed
by the elements of the codebook of A and B [11, 14]. The degranulation problem
associates with a solution to the system of relational equations (3.1) in which the
fuzzy sets Ai , B j , Cl as well as the entries of the fuzzy relation gi jl are given while
the fuzzy relation R is to be determined. The solution to the problem comes as the
maximal fuzzy relation expressed as follows [10]:

R = |i, j,l ((Ai × B j × Cl )Φ gi jl )
ˆ (3.4)


One can show that there is a solution to the system of fuzzy relational equations
expressed by (3.1). In terms of the membership values of the fuzzy relation, we
obtain
R(x, y, z) = mini, j,1 [(Ai (x)tB j (y)tC1 (z))Φ gi jl ]
ˆ (3.5)

The pseudo-residual operator [10] associated with a given t-norm is defined in the
following form:
aΦ b = sup{c ∈ [0, 1]|atc ≤ b} (3.6)

The expression (3.8) requires some attention as we encounter here the vector-
valued membership function Cl(z). The calculations realized there are considered
ˆ
coordinate-wise that is we compute consecutive entries of R for the arguments
z1 , z2 , . . . , z p by computing the corresponding expressions. For z1

R(x, y, z) = mini, j [(Ai (x)tB j (y)tC1 (z1 ))Φ gi j1 ]
ˆ (3.7)

for z2
R(x, y, z) = mini, j [(Ai (x)tB j (y)tC1 (z2 ))Φ gi j2 ]
ˆ (3.8)

In case we use the minimum operator, the corresponding pseudo-inverse (3.6) reads
as follows:
b a>b
aΦ b = sup{c ∈ [0, 1]|min(a, c) ≤ b} = (3.9)
1 a≤b

Observe that in all cases the inclusion relationship holds, namely R ⊆ R. The
ˆ
quality of the granulation–degranulation is expressed by comparing how much
ˆ
the reconstruction of R given by (3.8), namely R, coincides with the relational
data themselves (R). The associated performance index expressing the quality of
reconstruction is defined as follows:
N
Q= ∑ R(xk , yk , zk ) − R(xk , yk , zk )
ˆ 2
(3.10)
K=1

Again in light of the vector values assumed by the fuzzy relation, the above
distance is computed by taking distances over the successive coordinates of z.
For the Euclidean distance one explicitly specifies the components of the fuzzy
relation, which yields Q = ∑ p ∑N j=1 K=1 R(xk , yk , zk j ) − R(xk , yk , zk j ) . The associ-
ˆ 2

ated optimization process may then link with a formation of information granules
minimizing the distance expressed by (3.10). This can be taken into consideration
when designing fuzzy sets (membership functions) as well as selecting the number
of fuzzy sets. Referring to the original expression for the max-t composition, the cal-
culations are carried out coordinate-wise, that is we compute min(Cl (z1 ), R(x, y, z1 )),
min(Cl (z2 ), R(x, y, z2 )), etc.


In summary, the presented approach offers a general methodology of describing
data in the language of information granules, which can be schematically captured
as D → G(D), where G denotes a granulation mechanism exploiting a collection
of information granules in the corresponding vocabularies. In view of flexibility
offered by different views at the data, we can highlight this effect in Fig. 3.7:
different granulation schemes cast an analysis of D in different perspectives and
give rise to different sequences of dominant granular descriptors.

3.7 Construction of Information Granules

Fuzzy clustering is a vehicle of information granulation. As such it can be used
here to form the elements of the codebook. As an example, we use here the well-
known method of the fuzzy c-means (FCM) [6]. The results of clustering come in
the form of the prototypes of the clusters and the partition matrix, which describes
an allocation of elements (membership degrees) to the clusters.
Let us discuss in detail on how the corresponding codebooks A, B, and C are
formed. For the spatial component, the data for the construction of elements of A
and B are one dimensional. The membership functions A1 , A2 , . . . , An are expressed
in the form
1
Ai (x) = 2/(ζ −1)
(3.11)
x−vi
∑n
i=1 x−v j

i = 1, 2, . . . , n, where v1 , v2 , . . . , vn ∈ R are the prototypes constructed by running the
FCM using the data {x1 , x2 , . . . , xN }. The fuzzification coefficient is denoted by ζ ,
ζ > 1. In the same way formed are the elements of the second codebook, B1 , B2 ,
. . . , Bm by clustering one-dimensional data {y1 , y2 , . . . , yN }. The codebook forming
the elements of C is constructed by clustering multidimensional data z1 , z2 , . . . , zN .
Here, the membership functions of the elements of C are expressed in the form

1
C1 (z) = 2/(ζ −1)
, (3.12)
z−vi
∑r
i=1 z−vj

where v1 , v2 , . . . , vr are the prototypes located in [0, 1] p . The successive coordinates
of the membership function Cl , Cl (z) can be expressed explicitly by projecting the
prototypes onto the corresponding axes of the unit hypercube and computing the
membership functions based upon these projected prototypes, see Fig. 3.10.
Take the i0 -th coordinate of z, that is zi0 . Project the original prototypes
v1 , v2 , . . . , vc on this coordinate obtaining the real numbers v1i0 , v2i0 , . . . , vci0 .
By making use of these scalar prototypes, we obtain the membership functions
of the i0 -th coordinate of z.


Fig. 3.10 Computing
membership values based on
i0 -th coordinate of the
prototypes

There are several strategies of organizing a clustering process; the three main
categories are worth highlighting here:
(a) Clustering realized separately for the spatial data and temporal data. In this
approach, we exercise two separate clustering algorithms, which do not interact.
The results are used together. The advantage of this design strategy is that one
can consider a number of well-established clustering techniques. A disadvan-
tage comes from a lack of interaction when forming the clusters.
(b) Clustering temporal and spatial data realized by running a single clustering
method. The interplay between the temporal and spatial data is realized by
forming an aggregate distance function composed of two parts quantifying
closeness of elements in the spatial domain and the space in which temporal
data are described. The approach by [15] is an example of the clustering strategy
falling within this realm. Further more advanced approaches in which the
distance functions are quite different are worth pursuing here (say, an Euclidean
distance used for spatial data and Itakura distance with regard to the temporal
data).
(c) Clustering temporal and spatial data realized by two clustering methods with a
granular interaction realized at the structural level of data. While the clustering
methods run individually, they receive feedback about the structure discovered
for the other data set (spatial or temporal). The interaction is typically realized at
the level of proximity matrices induced at the locally formed partition matrices
[13].
There are several parameters of the codebook formed by the information granules
that could be effectively used in the optimization of the description of the relational
data. This list includes:
(a) The number of information granules defined for the corresponding spatial and
temporal coordinates.
(b) The values of the fuzzification coefficients (ζ ) used in the construction of
information granules. As discussed in the literature, by changing the value of
the fuzzification coefficient, we directly affect the shape of the membership
function. The values close to 1 yield membership functions whose shape is


close to the characteristic functions of sets. The commonly used value of the
fuzzification coefficient is equal to 2. Higher values of this parameter result in
spike-like membership functions.
The first group of the parameters is of combinatorial nature. In general, there is a
general monotonicity relationship: higher number of clusters can help reduce the
value of the optimization criterion (3.10). The optimization is of a combinatorial
nature, though so as to assure an effective design, one has to engage some techniques
of global optimization such as, e.g., evolutionary optimization. In light of the
monotonicity relationship, it could be more legitimate to look at the allocation
of the number of information granules to the vocabularies of the information
granules formed for the spatial and temporal dimensions of the variables. Let c
be a number of available information granules used in the granulation process
and specified in advance. We choose the number of granules for the x-, y-, and
temporal coordinates of the variables, say c1 , c2 , and c3 so that they sum is equal
to c. How to do this allocation, comes as a result of combinatorial optimization.
The parametric aspects of the optimization are implemented by selecting suitable
values of the fuzzification coefficients; those could be assume different values for
the corresponding codebooks, say ζ1 for A, ζ2 for B and ζ3 for C.

3.8 Describing Relationships Between Granular Descriptors
of Data

The spatiotemporal data discussed so far along with their granular description
(signature) may be viewed as a certain snapshot of a certain phenomenon reported
over some period of time, say a week or a month. The data could be acquired over
consecutive time periods (time slices) thus forming a sequence of spatiotemporal
data D(T ), D(2T ), . . . D(MT ). As each data is subject to its granular description
(and behind each of these description comes a certain codebook), it is of interest
to look at the relationships between D(T ), D(2T ), etc. or better to say between
G(D(T )) and G(D(2T )). The latter task translates into a formation of relationships
between the components of the codebooks being used for the successive data
“slices” say, D(T ), D(2T ), etc. Another way of expressing such dependencies is
to look at the characterization of information granules forming a certain vocabulary
at time slice T by the elements of the vocabulary pertaining to the successive 2T th
time slice. We discuss these two approaches in more detail.

3.8.1 Determination of Relationships Elements of Vocabularies

We construct the relationships between {Ai (2T ), B j (2T ),Ck (2T )} and Ai (T ), B j (T ),
Ck (T ), so the evolution of the underlying system is expressed in terms of evolution


Fig. 3.11 Computing the
degrees of activation of the
elements of the codebook
A(2T ) by the elements of
A(T ); vi (T ) and v j (2T )
(black and grey dots) are the
prototypes of the
corresponding fuzzy sets in
the T and 2T time periods

of information granules describing the data structure. The number of elements in
the vocabularies Ai (T ) and Ai (2T ) need not to be same as the granularity of the
vocabularies can evolve to reflect the complexity of the data and structures being
encountered. The relationships between Ai (T ) and Al (2T ) can be expressed in terms
of logic dependencies. In general, we formulate a logic relationship in the form of
the fuzzy relational equation:

a(2T ) = a(T ) ◦ L(T, 2T ) (3.13)

where ◦ stands for a certain composition operator, say a max-t or a max–min
one (being already used in the previous constructs) and L(T, 2T ) captures the
relationships between coordinates of the vectors a(T ) and a(2T ). The vectors
a(T ) and a(2T ) being the elements of the unit hypercubes, contain the degrees of
activation of the elements of the vocabulary A. The dimensionality of a(T ) and
a(2T ) is equal to n(T ) and n(2T ), respectively. As noted, the dimensionality of
these two vectors might not be the same. The estimation of the fuzzy relation can
be realized in a number of different ways depending upon a formation of the input–
output data to be used in the estimation procedure of L(T, 2T ). We note that the
above relational description is a nonstationary model as the fuzzy relation L depends
on the time indexes (T and 2T ). One could form a stationary model where L is time
independent that is we have a(nT ) = a(nT − T ) ◦ L, n = 2, 3, . . . .
A certain way, using which this could be realized in an efficient way is to consider
the data composed of the pairs (here we are considering the elements of the first
codebook, A)

a1 (T ) = [1 0 . . . 0] a1 (2T ) = [λ11 λ12 . . . λ1n(2T ) ] = λ1 (3.14)

Here λ1 (T ) is a degree of activation of the vocabulary at 2T by the 1st element of
the vocabulary used at T . The degrees of activation λ1i are computed as follows:

1
λ1i = 2/(ζ −1)
(3.15)
n(2T ) v1 (T )−vi (2T )
∑ j=1 v1 (T )−v j (2T )

Refer to Fig. 3.11 for a more detailed explanation.


Likewise, we have the remaining n(T ) input–output pairs

a2 (T ) = [0 1 . . . 0] a2 (2T ) = [λ21 λ22 . . . λ2n(2T ) ] = λ2 (3.16)
an(T ) (T ) = [0 0 . . . 1] an(T ) (2T ) = [λn(T )1 λn(T )2 . . . λn(T )n(2T ) ] = λ2 (3.17)

We consider all pairs of data {(a1 (T ), a1 (2T )), . . . , (an(T ) (T ), an(T ) (2T )} and solve
a system of relational equations (3.13) with regard to L(T, 2T ). In case of the max-t
composition, the result is straightforward; the solution does exist and the largest one
is expressed in the form

L(T, 2T ) = [1 0 . . . 0]T Φ [λ11 λ12 . . . λ1n(2T) ]
ˆ [0 1 . . . 0]Φ [λ21 λ22 . . . λ2n(2T ) ]

× ... [0 0 . . . 1]Φ [λn(T )1 λn(T )2 . . . λn(T )n(2T ) ] (3.18)

In other words, the fuzzy relation computed in this way comes in the form

L(T, 2T ) = |λ1 λ2 . . . λn(T ) |T (3.19)

We can quantify the changes when moving from one vocabulary to the next one by
expressing uncertainty of expressing the elements of the codebook at 2T , A(2T ) in
terms of the elements of the codebook formed for the data at T .

3.8.2 Uncertainty Quantification of Variable Codebooks

We use the entropy measure [9] to quantify the level of uncertainty. More specifi-
cally, we look at the successive rows of L(T, 2T ) and for each of them compute the
entropy. For instance, for the i-th row we have

n(2T )
H λi = − ∑ λi j log2 (λi j ) (3.20)
j=1

The highest uncertainty (viz. difficulty of expressing the elements of one codebook
by another one) occurs when all elements of i are the same and equal to 1/n(2T ).
n(T )
At a more aggregate level, we consider an average entropy that is n(T ) ∑i=1 H(λi ).
1

Its value is reflective of the level of variability between the codebooks used in
successive time epochs and uncertainty when expressing the elements of A(T ) in
terms of the components of A(2T ). In the same way, we compute the entropy
associated with the evolution of the codebooks B and C. This type of evaluation
is useful in the assessment of the evolution of the granularity of the descriptors of
data. The granular description of the system presented in this way highlights and
quantifies two important facets of evolvability, that is evolvability being reflective
of the changes of the system itself and evolvability being an immediate result


of the adjustable perception of the data caused directly by changes made to the
vocabularies used in the description of the data. These two are interlinked.
In case we fix our perception of the system that is the codebooks are left
unchanged A(T ) = A(2T ) = . . . B(T ) = B(2T ) = . . . , etc., the matrices of granular
descriptors of the data, G(T ), G(2T ), . . . , etc. are essentially reflective of the
dynamics of the system. The dynamics being captured in this way depends upon
a level of specificity of the codebooks. What is intuitive, the less detailed the
codebooks, the closer the granular descriptors become and some minor evolvability
trends present in the system are not taken into consideration. When the codebook
(vocabulary) itself changes (because of the structural complexity of the data
themselves or perception at the system, which has been changed), we envision an
aggregate effect of evolvability stemming from the dynamics of the system as well
as its varying level of perception.

3.9 Conclusions

There is an ongoing quest in data analysis to make the results more user friendly
(interpretable) as well as make the role of the user more proactive in the pro-
cesses of data analysis. The study offered in this paper arises as one among
interesting possibilities with this regard. It should be noted that the closeness
among data is expressed by taking into account a certain aggregate of closeness
in the spatial domain and temporal domain. While the x–y coordinates could
be left unchanged in successive time slices (which reflects a situation of fixed
measuring stations/sensors), one can easily handle a situation of mobile sensors
whose distribution varies over time slices.
It has been demonstrated that the concept of information granularity not only
helps conceptualize a view at the data but also construct effective algorithms
supporting the development of relational models.
The nonstationarity phenomenon might be inherent to the data themselves (viz.
the system or phenomenon producing the data). The evolution effect could be
also related to the varying position one assumes when studying the system. These
two facets of dynamical manifestation of the system/perception are captured and
quantified in terms of the relational descriptors and dependencies between them.

Acknowledgments Support from Natural Sciences and Engineering Research Council (NSERC)
is greatly appreciated.

References

1. Bargiela, A., Pedrycz, W.: Granular Computing: An Introduction. Kluwer Academic Publish-
ers, Dordrecht (2003)
2. Bargiela, A., Pedrycz, W.: Toward a theory of granular computing for human-centered
information processing. IEEE Transactions on Fuzzy Systems 16(2), 320–330 (2008)


3. Berezowski, J.: Animal health surveillance. In: The Encyclopedia of Life Support Systems
(EOLSS). EOLSS Publishers Co. Ltd (2009)
4. Berezowski, J.: Alberta veterinary surveillance network, advances in pork production. In:
Proceedings of 2010 Banff Pork Seminar vol. 21, pp. 87–93. Quebec, Canada (2010)
5. Berezowski, J., Renter, D., Clarke, R., Perry, A.: Electronic veterinary practice surveillance
network for alberta’s cattle population. In: Proceedings of the Twenty Third World Buiatrics
Congress. Quebec, Canada (2004)
6. Bezdek, J.: Pattern Recognition with Fuzzy Objective Function Algorithms. Kluwer Aca-
demic/Plenum Publishers, U.S.A. (1981)
7. Burkom, H., Murphy, S., Shmueli, G.: Automated time series forecasting for biosurveillance.
Statistics in Medicine 26, 4202–4218 (2007)
8. Checkley, S., Berezowski, J., Patel, J., Clarke, R.: The alberta veterinary surveillance network.
In: Proceedings of the 11th International Symposium for Veterinary Epidemiology and
Economics. Cairns, Australia (2009)
9. Cover, T.M., Thomas, J.A.: Elements of Information Theory. John Wiley & Sons, New York
(1991)
10. Di-Nola, A., Sessa, S., Pedrycz, W., Sanchez, E.: Fuzzy Relational Equations and Their
Applications in Knowledge Engineering. Kluwer Academic Publishers, Dordrecht (1989)
11. Hirota, K., Pedrycz, W.: Fuzzy relational compression. IEEE Transactions on Systems, Man,
and Cybernetics, Part B: Cybernetics 29(3), 407–415 (1999)
12. Klement, E., Mesiar, R., Pap, E.: Triangular Norms. Kluwer Academic Publishers, Dordrecht
Norwell New York London (2000)
13. Loia, V., Pedrycz, W., Senatore, S.: Semantic web content analysis: A study in proximity based
collaborative clustering. IEEE Transactions on Fuzzy Systems 15(6), 1294–1312 (2007)
14. Nobuhara, H., Pedrycz, W., Sessa, S., Hirota, K.: A motion compression/reconstruction method
based on max t-norm composite fuzzy relational equations. Information Sciences 176(17),
2526–2552 (2006)
15. Pedrycz, W., Bargiela, A.: Fuzzy clustering with semantically distinct families of variables:
descriptive and predictive aspects. Pattern Recognition Letter 31(13), 1952–1958 (2010)
16. Pedrycz, W., Gomide, F.: Fuzzy Systems Engineering: Toward Human-Centric Computing.
John Wiley & Sons, Hoboken, NJ (2007)
17. Zadeh, L.: Towards a theory of fuzzy information granulation and its centrality in human
reasoning and fuzzy logic. Fuzzy Sets and Systems 90, 111–117 (1997)

Learning in non stationary environments

More Related Content

What's hot (20)

Viewers also liked (16)

Similar to Learning in non stationary environments (20)

More from Springer (20)

Recently uploaded (20)

Learning in non stationary environments