Development of Robust Adaptive Inverse models using Bacterial Foraging Optimization

ACEEE Int. J. on Control System and Instrumentation, Vol. 02, No. 02, June 2011

Development of Robust Adaptive Inverse models using
Bacterial Foraging Optimization
H Pal Thethi
Dept. of Electronics & Communication Engg., KIIT University, Bhubaneswar, India
Email: harpalnitr@gmail.com
Babita Majhi and Ganapati Panda
Dept. of IT, Institute of Technical Education and Research, S ‘O’ A University, Bhubaneswar, India
School of Electrical sciences, Indian Institute of Technology Bhubaneswar, India
Email:babita.majhi@gmail.com/ganapati.panda@gmail.com

Abstract— Adaptive inverse models find applications in The possibility of BFO being trapped to local minimum is less.
communication and magnetic channel equalization, recovery In recent years the BFO has been proposed and has been
of digital data and adaptive linearization of sensor applied to many areas such as harmonic estimation of power
characteristics. In presence of outliers in the training signal,
system signals [12] and adaptive inverse modeling [13]. In
the model accuracy is severely reduced. In this paper three
case of derivative free algorithms conventionally the mean
robust inverse models are developed by recursively minimizing
robust norms using BFO based learning rule. The performance square error (MSE) is used as the fitness or cost function.
of these models is assesses through simulation study and is Use of MSE as cost function leads to improper training of
compared with those obtained by standard squared norm based the parameters of adaptive model when outliers are present
models. It is in general, observed that the Wilcoxon norm in the desired signal. The traditional regressors employ least
based model provides best performance. Moreover the squared square fit which minimizes the Euclidean norm of the error,
error based model is observed to perform the worst. while the robust estimator is based on a fit which minimizes
another rank based norm called Wilcoxon norm [14]. It is
Index Terms— Adaptive inverse models, Robust norms, Robust
reported that linear regressors developed using Wilcoxon
adaptive inverse model, Bacterial foraging optimization
norm are robust against outliers. Using this new norm robust
machines have recently been proposed for approximation of
I. INTRODUCTION
nonlinear functions [15]. Inverse modeling finds many
In digital data communication systems, high speed data applications such as channel equalization in
transmission over band limited channel often causes inter- telecommunication [8], adaptive linearization in intelligent
symbol interference (ISI) due to adverse effect of dispersive sensors [16] and digital control [17]. In all these applications
channel [1]. The performance of linear channel equalizers is the learning tools used are derivative based such as least
poor especially when the nonlinear distortion is severe [2]. mean square (LMS) or back propagation (BP) algorithms.
In these cases, nonlinear equalizers are preferable [1]-[6]. When outliers are present in the training signals the inverse
Since artificial neural network (ANN) can perform complex modeling performance of these algorithms degrades
mapping between its input and output space, different ANNs substantially. To the best of our belief no literature has dealt
have been successfully used in nonlinear inverse modeling on the problem of development of robust inverse model in
problem [1]-[6]. Functional link artificial neural network presence of outliers. Therefore our motivation in this paper
(FLANN) possess a low complexity structure and has been is to address this problem and to provide effective solution
employed as an adaptive inverse model [7]-[8]. Recently, using BFO based training of the models using a robust norms
genetic algorithm (GA) has been used for nonlinear and blind of error [15, 18, 19] as cost function. The choice of BFO as a
channel equalization [9]-[10]. The operations in GA such as training tool is due to its advantages over other evolutionary
the crossover and the mutation, help to avoid local minima computing tool stated earlier. The paper is organized into
problem and thus provide improved solution. However there following sections : Section II discusses inverse modeling
are some situations when the weights in GA are trapped to problem. Robust adaptive inverse model is given in Section
local minima. The bacterial foraging optimization (BFO) [11] III. In Section IV the BFO based update algorithm is developed
like GA is a derivative free optimization technique and acts as for the inverse model. Development of robust inverse model
an useful alternative to GA. The number of parameters that using BFO based training and robust norm minimization is
are used for searching the total solution space is higher in dealt in Section V. For performance evaluation, the simulation
BFO compared to those in GA but on the other hand requires study is carried out and is dealt in Section VI. Finally
less number of computations. conclusion of the paper is outlined in Section VII.

18
© 2011 ACEEE
DOI: 01.IJCSI.02.02.44


II. ADAPTIVE INVERSE MODEL performed iteratively by derivative based LMS algorithm.

III. ROBUST ADAPTIVE INVERSE MODEL
Three robust cost functions (RCF) defined in literature [15,
18, 19] are chosen in the development of robust adaptive
inverse models. The RCFs are defined as
(a) Robust Cost Function-1 (Wilcoxon Norm) [14,15]
The Wilcoxon cost function is a pseudo-norm and is
l l '

defined as CF1   a( R(vi ))vi   a (i )vi (4)
i 1 i 1

where R (v i ) denotes the rank of vi among
v1 , v 2 ,......... ......, vl , v (1)  v ( 2 )  ...... v (l ) are the ordered

values of v1 , v 2 ,..............., vl , a(i)  [i /(l  1)] . In
Fig.1. Development of a robust inverse model using BFO based
training cost function statistics, different types of score functions have been dealt
but the commonly used one is given by
The input symbols are represented as x(k ) at time
 (u )  12 (u  0.5) .
instance, k . They are then passed to a channel (plant) which
may be linear or nonlinear. An FIR filter is used to model a (b) Robust Cost Function-2 [18]
linear channel whose output at time instant k may be written It is defined as
as
CF3   (1  exp(e 2 / 2 )) (5)
N 1
where  is a parameter to be adjusted during training and
y (k )   w (i ) x ( k  i )
i0
(1)
e 2 is the mean square error
where w(i) are the weight values and N is the length of (c) Robust Cost Function-3 (Mean Log Squared error)[19]
the FIR plant. The “NL” block represents the nonlinear The next cost function is defined as
distortion introduced in the channel and its output may be
expressed as a nonlinear function of input and channel e2
CF4  log(1  ) (6)
coefficients. 2
z ( k )   ( x ( k ), x ( k  1),........ , x ( k  N  1) where is mean square error.
(2) The weight-update mechanism of inverse model of Fig. 1 is
w ( 0 ), w (1),........ .......... .......... ....., w ( N  1)),
carried out by minimizing the cost functions of the errors
where  (.) is some nonlinear function generated by the “NL” defined in (4), (5) and (6) using BFO algorithm.
block. The channel output z (k ) is corrupted with additive
IV. DEVELOPMENT OF ROBUST NONLINEAR INVERSE MODELS USING
white Gaussian noise q(k ) of variance  2 . This corrupted BFO

received output is given by r (k ) . The received signal The updating of the weights of the BFO based inverse
model is carried out using the training rule as outlined in the
r (k ) is then passed into the adaptive inverse model to following steps:
ˆ
produce x( k ) which represents the input symbols x(k ) . Step -1 Initialization of parameters
From initial parameters, the weights are updated until the (i) Sb = No. of bacteria to be used for searching the total
conventional cost function region
N (ii) N = Number of input samples
1 2
CF2 
N
e
k 1
(k ) (3) (iii) p = Number of parameters to be optimized

is minimized. The term N stands for input samples used for (iv) N s = Swimming length after which tumbling of bacteria
is undertaken in a chemotactic loop.
ˆ
training and the error term e( k )  x d ( k )  x( k ) . The
(v) Nc =Number of iterations carried out in a chemotactic
received data is given by r ( k )  z ( k )  q( k ) .
Conventionally the minimization of this cost function is loop, Nc > .
19
© 2011 ACEEE
DOI: 01.IJCSI.02.02.44


(vi) = Maximum number of reproduction loop (d)If min( J ) {minimum value of J among all the bacteria} is
(vii) = Maximum number of elimination and dispersal loop. less than the tolerance limit then all loops are broken.
(viii) Ped = Probability with which the elimination and Step-4. If j  N c ,go to (iii) to continue chemotaxis loop since
dispersal takes place. the lives of bacteria are not over.
(ix) The location of each bacterium P (1- p , 1- S b , 1) is
Step-5 Reproduction
specified by random numbers on [0,1].
(a) For the given k and l , and for each
(x) The value of run length unit, C (i ) is assumed to be
constant for all bacteria. i  1,2,................., S b the bacteria are sorted in
Step-2 Generation of model input ascending order of cost functions, J (higher cost function
means lower health).
(i) Random binary input [1,-1] is applied to the channel/plant.
(b) The first half of bacteria population is allowed to die and
(ii) The output of the channel is contaminated with white
the remaining half bacteria with least cost functions split into
Guassian noise of known strength to generate the input signal.
two and the copies are placed at the same location as their
(iii) The binary input is delayed by half of the order of the
parent.
inverse model to obtain the signal, x d (k ) .
Step-6. If k  N re go to Step-2. In this case, the number of
Step -3 Weight update algorithms specified reproduction steps has not reached and the next
In this step the bacterial population, chemotaxis, reproduction, generation in the chemotactic loop is started.
elimination and dispersal are modeled. Initially j  k  l  0
Step-7. Elimination –Dispersal
(i) Elimination dispersal loop l  l  1 The bacterium with an elimination-dispersal probability above
(ii) Reproduction loop k  k 1 a preset value Ped is eliminated by dispersing to a random
,
(iii) Chemotaxis loop j  j  1 location and new replacements are randomly made in the
search space. By this approach the total population is
(a) For i  1,2,......... ...S b , the cost function J (i , j , k , l ) for
maintained constant.
each i th bacterium is evaluated by the following way :
V DEVELOPMENT OF ROBUST INVERSE MODEL USING BFO BASED
.
(1) N number of random binary samples are generated ,
TRAINING
passed through the model and the output is computed.
(2)The output is then compared with the corresponding Three robust cost functions defined in literature [15, 18,
19] are used to develop the robust inverse models. The BFO
training signal, x d (k ) to calculate the error, e(k ) . is then used to iteratively minimize these cost functions of
(3)The robust cost functions of the error terms are computed the error obtained from the model. The resulting inverse
as . model is expected to be robust against outliers. The weight-
(4)End of For Loop. update of inverse model of Fig. 1 is carried out by minimizing
(b)For the tumbling/swimming decision is taken. these cost functions of the errors defined in (4), (5) and (6)
Tumble : A random vector with each element, using BFO algorithm.
in the range of [-1, 1] is generated. The new position of th In this approach, the procedure outlined in Steps-1 to 7 of
bacterium in the direction of tumble is computed as Section IV remains the same. The only exception is detailed
as follows :
Let the error vector of p th bacterium at k th generation due
The updated position is used to compute the new cost to application of N input samples to the model be
function (mean squared error) T
represented as [e1, p (k ), e2, p (k ),............., e N , p (k )] .
Swim – (i) Let c (counter for swim length) = 0
(ii) While that is bacteria have not climbed down too The errors are then arranged in an increasing manner from
long then update
which the rank R{en , p ( k )} of each n th error term is
If then the new position of th bacterium is
computed by using (7). The updated position, obtained. The score associated with each rank of the error
term is evaluated as
P ( j  1, k , l ) is used to compute the new cost function,
i
J (i, j  1, k , l ) a (i )  12 (  0.5) (8)
N 1
ELSE c  N s . (End of the WHILE statement).
where (1  i  N ) denotes the rank associated with each
(c)Go to next bacterium (i  1) , if i  Sb the process is error term. At k th generation of each p th particle the
repeated. Wilcoxon norm is then calculated as
20
© 2011 ACEEE
DOI: 01.IJCSI.02.02. 44


N (a)Keeping the CF, SNR and percentage of outliers in the
C p ( k )   a (i ) e i , p ( k ) (9) training signal same, the BER increases with increase in the
i 1 EVR of the channel. Similarly under identical simulation
conditions the squared error cost function based inverse
Similarly other two robust CFs are also computed using (5)
model performs the worst where as the Wilcoxon norm based
and (6). The learning process continues until the CF
model provides the best performance (least BER).
decreases to the possible minimum values. At this stage the
(b)As the outliers in the training signal increases the Wilcoxon
training process is terminated and the resulting weight vector
norm based inverse model continues to yield lowest BER
represents the final weights of the inverse model
compared to those provided by other norms.
(c)With no outlier present in the training signal, the BER plot
VI. SIMULATION STUDY
of all four CFs are almost same
In this section, the simulation study of the proposed (d)In presence of high outliers the conventional CF2 based
inverse model in presence of 10% to 50% of outliers in the model performs the worst followed by CF4 based model. In
training signal is carried out. Fig. 1 is simulated for various all cases the Wilcoxon norm (CF1) based inverse model
nonlinear channels using the algorithm given in Sections V performs the best and hence is more robust against low to
and VI. The transfer functions of three standard linear systems high outliers in the training signal.
used in the simulation study are : (e)The accuracy of inverse model based on CF3 and CF4
norms developed using outliers is almost identical.
H 1 ( z ) : 0.209  0.995z 1  0.209 z 2 (f)In addition, the plots of Fig. 3 indicate that at 50% outliers
H 2 ( z ) : 0.260  0.930 z 1  0.260 z  2 in the training signal the BER increases with increase in the
(10) EVR of the nonlinear plants.
H 3 ( z ) : 0.304  0.903z 1  0.304 z  2 (g) Further, the BER of the inverse models of all plants and
The eigen-value-ratio (EVR) of the plants given in (10) are SNR conditions is highest in square error norm (CF2) based
6.08, 11.12 and 21.71 respectively [8] which indicate the training compared to three other robust norms used. However
complexity of the plant or channel. A higher EVR indicates a the Wilcoxon norm (CF1) based inverse model yields minimum
bad channel in the sense that the convergence of the model BER among all cases studied.
becomes poor. To study the effect of nonlinearity on the
inverse model performance, two different nonlinearities are
introduced
NL1 : z(k)  tanh(y(k))
(11)
NL2 : z(k )  y(k )  0.2 y2 (k)  0.1y3 (k)
where y(k ) is the output of each of linear systems defined in
(10). The additive noise in the channel is white Gaussian with
-30dB strength. In this study an 8-tap adaptive FIR filter is
used as an inverse model. The desired signal is generated by
delaying the input binary sequence by half of the order of
the inverse model. Outliers are added by simply replacing the
bit value from 1 to -1 or -1 to 1 at randomly selected locations Fig. 2(a) Comparison of BER of four different CFs based nonlinear
equalizers with [.209, .995, .209] as channel coefficients and NL1
(10% to 50%) of the desired signal. In this simulation study, with 50% outliers
we have used the following parameters of BFO : Sb = 8, N is
= 100, p =8, N s =3, Nc =5, Nre =40-60, N ed =10, Ped = 0.25,
C (i ) = 0.0075. This selection of parameters is based on
achieving best inverse modeling performance through
simulation.
The bit-error-ratio (BER) plot of BFO trained inverse model
pertaining to different nonlinear plants with different cost
functions in presence of 0%-50% of outliers are obtained
through simulation. The BER plots with 50% outliers only
are shown in the Figs.2(a)-(f). In these figures the BER plots
of four cost functions have been compared. To study the
Fig.2(b) Comparison of BER of four different CFs based nonlinear
effects of EVR of the plant on BER performance the SNR is equalizers with [.209, .995, .209] as channel coefficients and NL2
set at 15dB in presence of 0% and 50% outliers in the training with 50% outliers
signal and the results are shown in Fig. 3 for 50% outliers.
Few notable observations obtained from these plots are :
21
© 2011 ACEEE
DOI: 01.IJCSI.02.02. 44


Fig.2(c) Comparison of BER of four different CFs based nonlinear Fig.2(f) Comparison of BER of four different CFs based nonlinear
equalizers with [.260, .930, .260] as channel coefficients and NL1 equalizers with [.304, .903, .304] as channel coefficients and NL2
with 50% outliers with 50% outliers

(a) NL1
Fig. 2(d) Comparison of BER of four different CFs based nonlinear
with 50% outliers

(b) NL2
Fig. 3 Effect of EVR on the BER performance of the four CF-based
equalizers in presence of 50% outliers
Fig. 2(e) Comparison of BER of four different CFs based nonlinear
with 50% outliers

22
© 2011 ACEEE
DOI: 01.IJCSI.02.02. 44


VII. CONCLUSION [7] J. C. Patra, Wei Beng Poh, N. S. Chaudhari and Amitabha Das,
“Nonlinear channel equalization with QAM signal using Chebyshev
This paper examines and evaluates the learning capability artificial neural network”, Proc. of International joint conference
of different robust norms of error when the training signal (of on neural networks, Montreal, Canada, pp. 3214-3219, August
the inverse model) is contaminated with strong outliers. To 2005.
[8] J. C. Patra, R. N. Pal, R. Baliarsingh and G. Panda, “Nonlinear
facilitate such evaluation different nonlinear plants with
channel equalization for QAM signal constellation using Artificial
varying EVRs are used. The population based BFO learning Neural Network”, IEEE Trans. on systems, man and cybernetics-
tool is developed to minimize four different norms. The Part B:cybetnetics, vol. 29, no. 2, April 1999.
robustness of these norms is assessed through extensive [9] Saman S. Abeysekera, “Adaptive blind channel equalization
simulation study. It is in general observed that the using orthogonalization and plane rotations via the Genetic
conventional squared error norm (CF2) is least robust to Algorithm”, International Conference on Communications, circuits
develop inverse models of nonlinear systems under varying and systems, vol. 2, pp.895-899, 27-30 May 2005.
noise conditions whereas the Wilcoxon norm (CF1) is the [11] K. M. Passino, “Biomimicry of Bacterial Foraging for
most robust one. In terms of quality of performance, the norms distributed optimization and control”, IEEE control system
magazine, vol 22, issue 3, pp. 52-67, June 2002.
are grouped in the order CF1, CF3, CF4 and CF2.
[10] G. Panda, Babita. Majhi, D. Mohanty, A. Choubey and S.
Mishra, “Development of Novel Digital Channel Equalizers using
ACKNOWLEDGMENT Genetic Algorithms”, Proc. of National Conference on
The work was supported by the Department of Science Communication (NCC-2006), IIT Delhi, pp.117-121, 27-
29,January, 2006.
and Technology, Govt. of India under grant no. SR/S3/EECE/
[12] S. Mishra, “A Hybrid least square Fuzzy bacterial foraging
065/2008.
strategy for harmonic estimation”, IEEE Trans. on Evolutionary
Computation, vol 9, no. 1, pp. 61-73, Feb. 2005.
REFERENCES [13] Babita Majhi, G. Panda and A. Choubey, “On The
Development of a new Adaptive Channel Equalizer using Bacterial
[1] S. Siu, G. J. Gibson and C. F. N. Cowan, “Decision feedback Foraging Optimization Technique”, Proc. of IEEE Annual India
equalization using neural network structures and performance Conference (INDICON-2006), New Delhi, India, 15 th-17 th
comparison with standard architecture, Proc. of Inst. Elect. Eng., September, 2006, pp. 1-6.
vol. 137, pp. 221-225, 1990. [14] Joseph W. McKean, “Robust analysis of Linear models”,
[2] S. Chen, G. J. Gibson, C. F. N. Cowan and P. M. Grant, Statistical Science, vol. 19, no. 4, pp. 562-570, 2004.
“Adaptive equalization of finite nonlinear channels using multilayer [15] Jer-Guang Hsieh, Yih-Lon Lin and Jyh-Horng Jeng,
perceptrons, EURASIP Journal of Signal Processing, vol. 20, pp. “Preliminary study on Wilcoxon learning machines”, IEEE Trans.
107-119, 1990. on neural networks, vol.19, no. 2, pp. 201-211, Feb. 2008.
[3] M. Meyer and G. Pfeiffer, “Multilayer perceptron based [16] J. C. Patra, A. C. Kot and G. Panda, “An intelligent pressure
decision feedback equalizers for channels with intersymbol sensor using neural networks”, IEEE Trans. on Instrumentation
interference”, IEE Proceeding, part-I, vol. 140, issue 6, pp. 420- and Measurement, vol. 49, issue 4, pp. 829-834, Aug. 2000.
424, 1993.. [17] K. S. Narendra and K. Parthasarathy, “Identification and
[4] S. Chen, B. Mulgrew and P. M. Grant, “A clustering technique control of dynamical systems using neural networks”, IEEE Trans.
for digital communication channel equalization using radial basis on Neural Networks, vol. 1, pp. 4-26, January 1990.
function networks”, IEEE Trans. on Neural Network, vol. 4,issue [18] Wei-Yen Wang, Tsu-Tian Lee, Ching-Lang Liu and Chi-Hsu
4, pp. 570-579, 1993. Wang, “Function approximation using fuzzy neural networks
[5] P. Kumar, P. Saratchandran and N. Sundararajan, “Minimal with robust learning algorithm”, IEEE Trans. on Systems, Man and
radial basis function neural networks for nonlinear channel Cybernetics-Part B : Cybernetics, vol. 27, no. 4, pp. 740-747, Aug.
equalization”, IEE Proc. Vis. Image Signal Processing, vol. 147, pp. 1997.
428-435, 2000. [19] Hung-Hsu Tsai and Pao-Ta Yu, “On the optimal design of
[6] J. Lee and R. Sankar, “Theoretical derivation of minimum fuzzy neural networks with robust learning for function
mean square error of RBF based equalizer”, Signal Processing, vol. approximation”, IEEETrans.on Systems, Man and Cybernetics-
87, pp. 1613-1625, 2007. Part B : Cybernetics, vol. 30, no. 1, pp. 217-223, Feb. 2000.

23
© 2011 ACEEE
DOI: 01.IJCSI.02.02. 44

Development of Robust Adaptive Inverse models using Bacterial Foraging Optimization

More Related Content

Similar to Development of Robust Adaptive Inverse models using Bacterial Foraging Optimization (20)

More from IDES Editor (20)

Recently uploaded (20)

Development of Robust Adaptive Inverse models using Bacterial Foraging Optimization