A Novel Penalized and Compensated Constraints Based Modified Fuzzy Possibilistic C-Means for Data Clustering

IJSRD - International Journal for Scientific Research & Development| Vol. 2, Issue 07, 2014 | ISSN (online): 2321-0613
All rights reserved by www.ijsrd.com 463
A Novel Penalized and Compensated Constraints based Modified Fuzzy
Possibilistic C-Means for Data Clustering
Duraisamy K.1
Haridass K.2
1
Research Scholar 2
Assistant Professor& Head of Department
1
Department of Computer Science 2
Department of Computer Application
1,2
NGM College Pollachi India
Abstract— A cluster is a group of objects which are similar
to each other within a cluster and are dissimilar to the
objects of other clusters. The similarity is typically
calculated on the basis of distance between two objects or
clusters. Two or more objects present inside a cluster and
only if those objects are close to each other based on the
distance between them.The major objective of clustering is
to discover collection of comparable objects based on
similarity metric. Fuzzy Possibilistic C-Means (FPCM) is
the effective clustering algorithm available to cluster
unlabeled data that produces both membership and typicality
values during clustering process. In this approach, the
efficiency of the Fuzzy Possibilistic C-means clustering
approach is enhanced by using the penalized and
compensated constraints based FPCM (PCFPCM). The
proposed PCFPCM approach differ from the conventional
clustering techniques by imposing the possibilistic reasoning
strategy on fuzzy clustering with penalized and compensated
constraints for updating the grades of membership and
typicality. The performance of the proposed approaches is
evaluated on the University of California, Irvine (UCI)
machine repository datasets such as Iris, Wine, Lung Cancer
and Lymphograma. The parameters used for the evaluation
is Clustering accuracy, Mean Squared Error (MSE),
Execution Time and Convergence behavior.
Key words: Unsupervised Learning, Fuzzy C-Mean, Fuzzy
Possibility C-Means, Penalized and Compensated
constraints based FPCM
I. INTRODUCTION
Clustering (also known as unsupervised learning) is the task
of recognizing a finite group of categories (or clusters) to
illustrate the data. Therefore, similar objects are clustered to
the similar category and dissimilar objects to different
clusters. Clustering is also known as unsupervised learning
since the data objects are pointed to a collection of clusters
which can be interpreted as classes additionally.Clustering is
the process of assembling the data records into significant
subclasses (clusters) in a way that increases the relationship
within clusters and reduces the similarity among two
different clusters. Other names for clustering are
unsupervised learning (machine learning) and segmentation.
Clustering is used to get an overview over a given data set.
A set of clusters is often enough to get insight into the data
distribution within a data set. Another important use of
clustering algorithms is the preprocessing for some other
data mining algorithm.
Fuzzy clustering methods allow the objects to
belong to several clusters simultaneously, with different
degrees of membership. Fuzzy clustering is a powerful
unsupervised method for the analysis of data and
construction of models. In many situations, fuzzy clustering
is more natural than hard clustering. Objects on the
boundaries between several classes are not forced to fully
belong to one of the classes, but rather are assigned
membership degrees between 0 and 1 indicating their partial
membership. The discrete nature of the hard partitioning
also causes difficulties with algorithms based on analytic
functional, since these functional are not differentiable.The
concept of fuzzy partition is essential for cluster analysis,
and consequently also for the identification techniques that
are based on fuzzy clustering. Fuzzy and possibilistic
partitions can be seen as a generalization of hard partition
which is formulated in terms of classical subsets.
The remainder of this is organized as follows.
Section 2 summarizes the concepts and literature survey.
Section 3 discusses the proposed method, and section 4
provides the experiments with high accuracy. Finally,
Section 5 presents the conclusions of the work.
II. LITERATURE SURVEY
A.M. Fahim et al., (2006) proposed an enhanced method for
assigning data points to the suitable clusters. In the original
K-Means algorithm in each iteration the distance is
calculated between each data element to all centroids and
the required computational time of this algorithm is depends
on the number of data elements, number of clusters and
number of iterations, so it is computationally expensive.
Likas et al., (2003) put forth a global K-Means
clustering algorithm. The technique was an incremental
move towards to clustering that dynamically includes one
cluster center at a particular time in the course of a
deterministic global exploration procedure comprises of N
(with N being the size of the data set) executions of the K-
Means algorithm from appropriate initial positions. Baolin
Yi et al., (2010) proposed a new method to find the initial
center and improve the sensitivity to the initial centers of K-
Means algorithm. Barakbah et al., (2009) proposes a new
approach to optimizing the designation of initial centroids
for K-Means clustering. Celikyilmaz et al., (2008) proposed
a new fuzzy system modeling approach based on improved
fuzzy functions to model systems with continuous output
variable.
Chen Zhang et al., (2009) presented a new
clustering method based on K-Means that have avoided
alternative randomness of initial center. This approach
focused on K-Means algorithm to the initial value of the
dependence of K selected from the aspects of the algorithm
is improved. Chunhui et al., (2008) presented a Similarity
based Fuzzy and Possibilistic C-Means algorithm called
SFPCM. It is derived from original fuzzy and FPCM which
was proposed by Bezdek.
Fang Yuan et al., (2004) investigated the standard
K-Means clustering algorithm in this work and give our

A Novel Penalized and Compensated Constraints based Modified Fuzzy Possibilistic C-Means for Data Clustering
(IJSRD/Vol. 2/Issue 07/2014/103)
improved version by selecting better initial centroids that the
algorithm begins with.
Filippone et al., (2010) investigated a kernel extension of the
classic possibilistic C-Means. In this extension, the author
implicitly mapped input patterns into a possibly high-
dimensional space by means of positive semidefinite
kernels. However, it is not good for the image with noise
and it also takes more time for execution. A new modified
FPCM clustering algorithm is proposed by Ganesan et al.,
(2010) for color image segmentation of any type of color
images. This new proposed clustering algorithm exhibits the
robustness to noise, and also faster as compared to the
traditional one.
Jiang-She Zhang et al., (2004) modified and
improve these algorithms to overcome their shortcoming. A
fast PCM clustering algorithm is proposed by Kai Li et al.,
(2003). Ojeda-Magafia et al., (2006) proposed a new
technique to use the Gustafson-Kessel (GK) algorithm
within the PFCM, such that the cluster distributions have a
better adaptation with the natural distribution of the data.
Xiao-Hong et al., (2005) presented a novel approach on
Possibilistic Fuzzy C-Means Clustering Model Using Kernel
Methods. The author insisted that fuzzy clustering method is
based on kernel methods. This technique is said to be Kernel
Possibilistic Fuzzy C-Means model (KPFCM).
III. PROPOSED METHODOLOGY
This work presents a clustering algorithm called Fuzzy
Possibilistic C-Means that merges the characteristics of both
Fuzzy and Possibilistic C-Means. To enhance the FPCM
approach MFPCM is presented. This novel technique aims
to give good results relating to the previous algorithms by
modifying the Objective function used in FPCM. The
objective function is based by adding new weight of data
points in relation to every cluster and modifying the
exponent of the distance between a point and a class.
A. Fuzzy Possibilistic Clustering Algorithm
Data analysis is considered as a very important science inthe
real world. Cluster analysis is a technique for classifying
data; it is a method for finding clusters of a data set with
most similarity in the same cluster and most dissimilarity
between different clusters. The conventional clustering
methods puteach point of the data set to exactly one cluster.
A fuzzy version of clustering appeared; it is Fuzzy C-Means
with a weighting exponent m>1, that uses the probabilistic
constraint that the memberships of a data point across
classes sum to one. The FCM is sensitive to noise.
Tomitigate such an effect, Krishnapuram and Keller throw
away the constraint of memberships in FCM and propose
the Possibilistic C-Means (PCM) algorithm. This work
deducted that to classify a data point, cluster centroid has to
be closest to the data point, and it is the role of membership.
Also for estimating the centroids, the typicality is used for
alleviating the undesirable effect of outliers. So Pal defines a
clustering algorithm called Fuzzy Possibilistic C-Means that
combines the characteristics of both fuzzy and possibilistic
c-means. The proposed algorithm called Modified Fuzzy
Possibilistic C-Means (MFPCM) aims to give good results
relating to the previous algorithms by modifying the
Objective function used in FPCM.
B. 3.2 Modified Fuzzy Possibilistic C-Means Technique
(MFPCM)
The selection of suitable objective function is the major
factor for the success of the cluster technique and to achieve
enhanced clustering. Hence the clustering optimization is
based on objective function to be used for clustering. To
obtain an appropriate objective function, the following set of
necessities is considered:
 The distance between clusters and the data points
allocated to them must be reduced
 The distance between clusters must to be reduced
The desirability between data and clusters is
modeled by the objective function. Also it provides a new
technique called Modified Suppressed Fuzzy C-Means,
which considerably improves the function of FCM because
of a prototype-driven learning of parameter α. The learning
procedure of α is dependent on an exponential separation
strength between clusters and is updated at every iteration.
1) Penalized and Compensated Constraints based Fuzzy
Possibilistic C-Means (PCFPCM)
This work presents Penalized and compensated constraints
that are embedded with the previously discussed Modified
Fuzzy Possibilistic C-Means algorithm. The objective
function of the MFPCM is given in equation (1.1).
∑ ∑ ( )
(1.1)
In the proposed approach the penalized and
compensated terms are added to the objective function of
MFPCM to construct the objective function of PCFPCM.
The penalized constraint can be represented as follows
∑ ∑
(1.2)
Where
∑
∑ ∑
∑
∑ ∑
(1.3)
Where i is a proportional constant of class i; j is a
proportional constant of training vector zi, and v (v0); 
(0) are also constants. In these functions, i and j are
defined in equations above. Membership and typicality
for the penalized component is presented below.
(∑
‖ ‖ ⁄
‖ ‖ ⁄
)
( ) (∑
‖ ‖ ⁄
‖ ‖ ⁄
)
(1.4)

In the previous expression
∑
∑
which is the centroid. The
compensated constraints can represented as follows
∑ ∑
(1.5)
Where membership and typicality for the
compensation is presented below
(∑
‖ ‖ ⁄
‖ ‖ ⁄
)
( )
(∑
‖ ‖ ⁄
‖ ‖ ⁄
)
(1.6)
To obtain an efficient clustering the penalization
term must be removed (i.e. the noise is removed) and the
compensation term must be added to the basic objective
function of the existing MFPCM. This brings out the
objective function of PCFPCM and it is given in (5.21).
∑ ∑
∑ ∑(
)
∑ ∑(
)
(1.7)
The final objective function is presented in (1.7).
2) 3.2.1 Algorithm of PCFPCM
Step 1: FCM algorithm is an iterative clustering method that
brings out an optimal c partition by minimizing the weighted
within group sum of squared error objective function.
Step 2: is the dataset in the p-dimensional vector space,
the number of data items is represented as is the number
of clusters with .
Step 3: is the centers or prototypes of the clusters,
represents the p-dimension center of the cluster , and
represents a distance measure between object
and cluster centre .
Step 4: represents a fuzzy partition matrix with
is the degree of membership of in the ith
cluster; is the jth
of p-dimensional measured data.
Step 5: To recover this weakness of FCM, relaxed the
constrained condition of the fuzzy c-partition to obtain a
possibilistic type of membership function.
Step 6: The characteristics of both Fuzzy and Possibilistic
C-Means are combined in the objective function of the
FPCM. But the estimation of centroids is influenced by the
noise data.
Step 7: Exponential separation strength between clusters is
the base for the learning process of α and is updated at each
of the iteration.
Step 8: A new parameter is added with this which
suppresses this common value of and replaces it by a new
parameter like a weight to each vector. Add Weighting
exponent as exhibitor of distance in the objective functions.
Step 9: Penalization term must be removed (i.e. the noise is
removed) and the compensation term must be added to the
basic objective function of the existing MFPCM.
IV. EXPERIMENTAL RESULTS
To evaluate the proposed penalized and compensated
constraints based Fuzzy Possibilistic C-Means (PCFPCM)
against Fuzzy Possibilistic C-Means (FPCM) and Modified
Fuzzy Possibilistic C-Means (MFPCM), experiments were
carried out using the similar experimental setup and
parameters as discussed in this chapter. The experiment is
done in MATLAB with such datasets such as Iris, Wine,
Lung Cancer and Lymphograma.
Datas
ets
FPCM MFPCM PCFPCM
Acc
ura
cy
(%)
Ti
m
e
(S
ec
)
M
S
E
Acc
ura
cy
(%)
Ti
m
e
(S
ec
)
M
S
E
Acc
ura
cy
(%)
Ti
m
e
(S
ec
)
M
S
E
Iris 68
5
8
0.
52
00
72
5
2
0.
50
22
84
4
9
0.
49
21
Wine 73
5
5
0.
48
31
79
4
7
0.
45
00
86
4
3
0.
40
10
Lung
cance
r
75
4
8
0.
45
01
83
4
4
0.
44
69
89
3
9
0.
43
02
Lymp
hogra
ma
80
3
7
0.
43
12
86
3
0
0.
42
80
91
1
9
0.
41
00
Table 1: Accuracy, Execution Time and Mean Squared
Error
Table1 shows the accuracy, execution time and
mean square error rate for clustering methods
Fig. 1: Accuracy for clustering methods
0
20
40
60
80
100
Accuracy(%)
Datasets
FPCM MFPCM

Fig. 2: Execution Time for clustering methods
Fig. 3: Mean Square Rate for clustering methods
Figure 1 and figure 2 show the accuracy and
execution time for clustering methods. Thus the proposed
method of Penalized and Compensated Constraints based
Fuzzy Possibilistic C-Means (PCFPCM) has high accuracy
with les execution time. Figure 3 shows the mean square
error rate. The proposed method has less error rate when
compare with others.
V. CONCLUSION
This work provides a penalized constraint of FCM is
improved by using NEM algorithm and it is combined with
compensated constraints which is said to be Improved
Penalized and Compensated constraints for Fuzzy
Possibilistic C-Means (IPCFPCM) clustering algorithm. The
usage of improved penalized constraints in MFPCM will
help in better calculation of distance between the clusters
and increasing the accuracy of clustering.Thus the proposed
method may perform better performance with high
accuracy.
REFERENCES
[1] A.M. Fahim, A.M. Salem, F.A. Torkey and M.A.
Ramadan, “An Efficient enhanced k-means clustering
algorithm,” Journal of Zhejiang University, Vol. 10,
No. 7, Pp. 1626-1633, 2006.
[2] AristidisLikas, Nikos Vlassis and Jakob J. Verbeek,
“The global k-means clustering algorithm”, Journal
of Pattern Recognition society, Elsevier, Vol. 36, No.
2, Pp. 451-461, 2003.
[3] Baolin Yi, HaiquanQiao, Fan Yang and ChenweiXu,
“An Improved Initialization Center Algorithm for K-
Means Clustering”, International Conference on
Computational Intelligence and Software Engineering
(CiSE), Pp. 1–4, 2010.
[4] Barakbah A.R and Kiyoki Y “A pillar algorithm for
K-means optimization by distance maximization for
initial centroid designation”, IEEE Symposium on
Computational Intelligence and Data Mining (CIDM
'09), Pp. 61–68, 2009.
[5] Celikyilmaz A and BurhanTurksen I, “Enhanced
Fuzzy System Models With Improved Fuzzy
Clustering Algorithm”, IEEE Transactions on Fuzzy
Systems, Vol. 16, No. 3, Pp. 779–794, 2008.
[6] Chen Zhang and Shixiong Xia, “K-means Clustering
Algorithm with Improved Initial Center”, Second
International Workshop on Knowledge Discovery
and Data Mining (WKDD), Pp. 790–792, 2009.
[7] Chunhui Zhang, Yiming Zhou and Trevor Martin,
“Similarity Based Fuzzy and Possibilistic c-means
Algorithm”, Proceedings of the 11th Joint Conference
on Information Sciences, Pp. 1-6, 2008.
[8] D. Vanisri, Dr.C. Loganathan, “An Efficient Fuzzy
Clustering Algorithm Based on Modified K-Means”,
International Journal of Engineering Science and
Technology, Vol. 2, No. 10, Pp. 5949-5958, 2010.
[9] Fang Yuan, Zeng-Hui Meng, Hong-Xia Zhang and
Chun-Ru Dong, “A new algorithm to get the initial
centroids”, Proceedings of International Conference
on Machine Learning and Cybernetics, Vol. 2, Pp.
1191–1193, 2004.
[10]Filippone M, Masulli F and Rovetta S, “Applying the
Possibilistic c-Means Algorithm in Kernel-Induced
Spaces”, IEEE Transactions on Fuzzy Systems, Vol.
18, No. 3, Pp. 572–584, 2010.
[11]Ganesan P and Rajini V, “A method to segment color
images based on modified Fuzzy-Possibilistic-C-
Means clustering algorithm”, Recent Advances in
Space Technology Services and Climate Change
(RSTSCC), Pp. 157–163, 2010.
[12]Jiang-She Zhang and Yiu-Wing Leung, “Improved
possibilistic C-means clustering algorithms”, IEEE
Transactions on Fuzzy Systems, Vol. 4, No. 2, Pp.
209–217, 2004.
[13]Kai Li, Hou-Kuan Huang and Kun-Lun Li, “A
modified PCM clustering algorithm”, International
Conference on Machine Learning and Cybernetics,
Vol. 2, Pp. 1174–1179, 2003.
[14]Ojeda-Magafia B, Ruelas R, Corona-Nakamura M.A
and Andina D, “An Improvement to the Possibilistic
Fuzzy c-Means Clustering Algorithm”, Automation
Congress, Pp. 1–8, 2006.
[15]Xiao-Hong Wu and Jian-Jiang Zhou, “Possibilistic
Fuzzy c-Means Clustering Model Using Kernel
Methods”, International Conference on Intelligent
Agents, Web Technologies and Internet Commerce,
Vol. 2, Pp. 465-470, 2005.
0
10
20
30
40
50
60
70
ExecutionTime(Sec)
Datasets
FPCM MFPCM
0
0.2
0.4
0.6
0.8
1
MeanSquareRate
Datasets
FPCM MFPCM

A Novel Penalized and Compensated Constraints Based Modified Fuzzy Possibilistic C-Means for Data Clustering

More Related Content

What's hot (19)

Viewers also liked (19)

Similar to A Novel Penalized and Compensated Constraints Based Modified Fuzzy Possibilistic C-Means for Data Clustering (20)

More from ijsrd.com (20)

Recently uploaded (20)

A Novel Penalized and Compensated Constraints Based Modified Fuzzy Possibilistic C-Means for Data Clustering