Strengthening support vector classifiers based on fuzzy logic and evolutionary methods_Presentation

1. Introduction
2. Problem definition
3. Background
4. The proposed methods
5. Conclusion and future work

3/20
[1]. B. Razavi, Design of integrated circuits for optical communications, McGraw-Hill, 2003.
Introduction

4/20
[1]. C. Aggarwal, Outlier analysis, Springer, 2013.
[2]. V. Chandola, A. Banerjee, V. Kumar, Anomaly Detection: A Survey, ACM, 2009
[3]. J. Guo, W. Huang, B. M. Williams, Real time traffic flow outlier detection using short-term traffic conditional variance
prediction, Transportation Research, Vol 50, 2015
[4]. J. A. Jablonski, T. J. Bihl, Principal Component Reconstruction Error for Hyperspectral Anomaly Detection, IEEE
Geoscience and Remote Sensing Letters, 2015
[5]. A. E. Lazzaretti, H. V. Neto, V. H. Ferreira, An Accurate Approach for Automatic Segmentation of Power Distribution
Voltage Waveforms, IEEE Transactions on Power Delivery, 2015
[6]. Z. Gao, C. Cecati, S. X. Ding, A Survey of Fault Diagnosis and Fault-Tolerant Techniques Part II: Fault Diagnosis with
Knowledge-Based and Hybrid/Active Approaches, IEEE Transactions on Industrial Electronics, 2015
Data cleaning[1]
Intrusion Detection Systems[2]
Traffic management[3]
Image processing[4]
Distributed system[5]
Fault diagnosis[6]
Problem definition

5/20
SVM inability[1]
[1]. R. Akbani1, S. Kwek1, N. Japkowicz, Applying Support Vector Machines
to Imbalanced Datasets, Springer, 2004
Wrong dicisoin of K-NNOne-Class Classifiers
[1]. M. A. F. Pimentel et al., A review of novelty detection, Signal
Processing, Vol 99, 2014
[2]. D. M. J. TAX, One-class classification, Doctoral thesis, TD
University
Support Vector Data Description (SVDD)
Samples
Center of SVDD
Support vector samples
Hyper sphere boundary
[1]. D. M. J. TAX, R. P. W. DUIN, Support Vector Data Description,
Machine Learning, 2004
Problem definition

6/20
SVDD improvements
Data structureKernel issue
Density
Kernel reduction
SVDD-neg
QP dividinghybrid Marginal Maximization
Special kernels
Non-stationary data
Distance Fuzziness
Multi hyper sphereHyper ellipse
SV prioritization
Background

7/20
[1]. Y. Forghani, H. Sadoghi Yazdi, S. Effati, An extension to fuzzy support vector data description-FSVDD, Pattern Analysis
and Applications, vol. 15, no. 3, pp. 237-247, 2012.
SVDD improvements
Density
Kernel reduction
SVDD-neg
Special kernels
Non-stationary data
Distance Fuzziness
SV prioritization
Background

8/20
[1]. M. GhasemiGol & et al., A New Support Vector Data Description with Fuzzy Constraints, in In Intelligent Systems,
Modelling and Simulation (ISMS), International Conference on. IEEE, pp. 10- 14, 2010.
[2]. Y. Allahyari, H. Sadoghi-Yazdi, Quasi Support Vector Data Description (QSVDD), International Journal of Signal
Processing, Image Processing and Pattern Recognition, vol. 5, no. 3, September, 2012.
[3]. B. Liu & et al., SVDD-based outlier detection on uncertain data, Knowledge and information systems, vol. 34, no. 3, pp. 597-
618, 2012.
SVDD improvements
Density
Kernel reduction
SVDD-neg
Special kernels
Non-stationary data
Distance Fuzziness
SV prioritization
Background

9/20
[1]. L. K. Young & et al., Improving support vector data description using local density degree, Pattern Recognition, vol. 38, no.
10, p. 1768 – 1771, 2005
[2]. L. K. Young & et al., Density-Induced Support Vector Data Description, IEEE Transactions on Neural Networks, vol. 18, no.
1, JANUARY 2007.
[3]. B. M. EL & et al., Support Vector Domain Description with a new confidence coefficient, in Intelligent Systems: Theories
and Applications (SITA-14), 2014 9th International Conference on. IEEE, 2014, pp. 1- 8.
[4]. C. Myungraee, K. J. Seok, B. J. Geol, Density weighted support vector data description, Expert Systems with Applications,
vol. 41, no. 7, pp. 3343-3350, 2014.
SVDD improvements
Density
Kernel reduction
SVDD-neg
Special kernels
Non-stationary data
Distance Fuzziness
SV prioritization
Background

10/20
, ,
(s ) ( )
( )
2
R Rc F c F
F i iF
i
s
VD s
 
 
 
Rough set[1]
[1]. Z. Pawlak, Rough sets, vol. 11, no. 5, pp. 341-356, 1982
Boundary area
Lower approximation set
Outer of decision boundary
Decision boundary
Fuzzy Rough set
[1]. D. Dubois, H. Prade, Rough fuzzy sets and fuzzy rough sets*, vol. 17,
no. 2-3, pp. 191-209, 1990
[2]. A. M. Radzikowska, E. E. Kerre, A comparative study of fuzzy rough
sets, Fuzzy Sets and Systems, vol. 126, no. 2, pp. 137-155, 2002
, , : ( , ) (max(0,1 ( , )))A a
x y X a A R x y T x y     
2
, : ( , ) ( ( ) ( ))a
x y X x y a x a y   
0 ( ) ( )
, : ( , )
1
a
if a x a y
x y X x y
else

 
   

, ,
(s ) inf ( ( , ), ( ))Rc F j i j
F i c i j F j
s DS s s
R s s s
  
 
 
,
,
(s ) ( ( , ), ( ))
Rc F
j i j
i c i j F jF
s DS s s
sup R s s s

  
 

Fuzzy rough lower and upper approximation membership
[1]. N. Verbiest, C. Cornelis, F. Herrera, FRPS: a fuzzy rough
prototype selection method, vol. 46, no. 10, pp. 2770-2782, 2013
, min
,
(s ) ( ( , ), ( ))Rc F
j i j
F i c i j F j
s DS s s
OWA R s s s
  
 

,
max
,
(s ) ( ( , ), ( ))
Rc F
j i j
i c i j F jF
s DS s s
OWA R s s s

  
 

An example
4 5 6 7 8
2
2.5
3
3.5
4
4.5
Feature 1 from iris
Feature2fromiris
Validation degree
0.8
0.85
0.9
0.95
The proposed methods

11/20
1
2
, ,
1
2 2
min ( , , ) ( )
1,...,
.
0 1,...,
i
L
i i i
R c
i
i i
i
OF R C R P VD s
s C R i L
s t
i L

 



 
     

  

Primary
problem
2 22 2
1 1 1
( , , , , ) ( ) ( )
L L L
i i i i i i i i i i
i i i
PP R C R P VD s R s C       
  
         
Samples
Center of SVDD
Support vector samples

12/20
3
L L L
R 0
i i i
i 1 i 1 i 1
PP
0 2R 2R 0 2R(1 ) 0 1
R

  

            

  
4 L
i
i 1
L L L
i i i i i
i 1 i 1 i 1
L
i iL L L1
i 1
i i i i iL
i 1 i 1 i 1
i
i 1
PP
0 ( 2s C 2 s 2 C
C
s
C 2 s C C s
  
 

  


                

 
         

  

  

5 i i
L L L L
i i i i i i
i 1 i 1 i 1 i 1i
0, 0
i i i i
PP
0 P.VD(s ) (P.VD(s )
P P.VD(s )
   
   

              

       
   

13/20
6
1 1, 1
1
max ( . ) ( . )
0 ( ) 1,2,...,
.
1 1,2,...,
L L
i i i i j i j
i i j
i i
L
i
i
DP s s s s
VD s P i L
s t
i L
  


  

  
   


  

 

Quadratic
programming
7 ,s ,
[ (s . ) 2 ( . ) |SV | ( . )]/ |SV |
k i k i j
k k i i k i j i j
s SV s DS SV s s DS
R s s s s s  
   
     
8
2
,
(s . ) 2 ( . ) ( . )]
i i j
new new new i new i i j i j
s DS s s DS
s C s s s s s  
 
    

14/20
-1 0 1 2 3 4 5 6 7 8 9 10
-2
-1
0
1
2
3
4
5
6
Iris(A1,A2),(A3,A4)
Feature 1
Feature2
Target
Outlier
2
,C,
1
2 2
min
.
0 1,...,
i
L
i
R
i
i i
i
i
F R P
s C R
s t
i L

 



 
    

  


15/20
Initialization Loudness L, Maximum Number of Iteration MNI, and Pulse rates P1
Creation N bates with location xi in intervals of variables, x=( )
Solution=SVDD Accuracy (xi), i=1,…,N
Best=Max(Solution)
I=1
While (MNI<I)
For(i=1:N)
Adapt xi based on pulse frequency fi and velocity Vi
If (Random Number > P I
i)
xi = x Best + η*x Random, η=0.001
End if
xi =Check Boundary of Variable(xi)
New solution=SVDD Accuracy (xi)
If(New solution>=Best solution & Random Number > LI
i)
Solution i =New solution;
LI+1
i=ω*LI
i
PI+1
i =Sinusoidal map (PI
i)
End if
End for
I=I+1
Best=Max(Solution)
x Best solution= x Best
End While
Return SVDD(x Best solution)
i
[1]. A. H. Gandomi , X. S. Yang, Chaotic bat algorithm, Journal of Computational Science, vol. 5, no. 2, pp. 224-232, 2014

16/20
Efficient
data structure
Not the end
Conclusions and Future work

17/20Conclusions and Future work

1. Sadeghi, R. & Hamidzadeh, J. (2015). Fuzzy rough support vector data
description. submitted in Pattern Recognition.
2. Sadeghi, R. & Hamidzadeh, J. (2015). Support Vector Data Description
based on Chaotic Bat Algorithm. submitted in Neurocomputing.
18/20Conclusions and Future work

[1]. B. Razavi, Design of integrated circuits for optical communications, McGraw-Hill, 2003.

[1]. C. Aggarwal, Outlier analysis, Springer, 2013.

Extreme value
{3, 2 , 3, 2, 3, 87, 2, 2, 3, 3, 3, 84, 91, 86, 91, 81}
Probabilistic and statistical
Linear
Information theoretic
ABABABABABABABAB
ABABCBABABABABAB
[1]. V. Chandola, A. Banerjee, V. Kumar, Anomaly Detection: A Survey, ACM, 2009
[1]. Type of Outliers

Proximity-based [1]
High-Dimensional [2]
[1]. M. Ankerst, M. M. Breunig, H. P. Kriegel, J. Sander, OPTICS: Ordering Points To Identify the Clustering Structure, ACM,
1999
[2]. Y. Yuan, Q. Wang, Fast hyperspectral anomaly detection via high-order 2-D crossing filter, IEEE TRA., 2015

Supervised
BN, DT, HMM, SVM, SVD,SVDD, SVDD+, ANN
[1]. M. Moghtadaei, M. R. HashemiGolpayegani, R. Malekzadeh, A variable structure fuzzy neural network model of squamous
dysplasia and esophageal squamous cell carcinoma based on a global chaotic optimization algorithm, elsevier, 2013
[2]. X. Z. Wang, L. C. Dong, J. H. Yan, Maximum Ambiguity-Based Sample Selection in Fuzzy Decision Tree Induction, IEEE
TRA., 2012
FNN [1] MABS [2]

Strengthening support vector classifiers based on fuzzy logic and evolutionary methods_Presentation

More Related Content

What's hot (16)

Similar to Strengthening support vector classifiers based on fuzzy logic and evolutionary methods_Presentation (20)

More from Reza Sadeghi (17)

Recently uploaded (20)

Strengthening support vector classifiers based on fuzzy logic and evolutionary methods_Presentation