Ideas and Problems in adaptive Fuzzy ControlCMLC2011.ppt

A talk in ICMLC 2011
Feb., 2011
®Copyright by Shun-Feng Su
1
Learning Control
Ideas and Problems
in Adaptive Fuzzy Control
Offered by Shun-Feng Su,
E-mail: su@orion.ee.ntust.edu.tw
Department of Electrical Engineering,
National Taiwan University of Science and Technology
Feb. 27, 2011

Feb., 2011
2
Preface
 Intelligent control is a promising way of control
design in recent decades.
 Intelligent control design usually needs some
knowledge of the system considered.
 However, such knowledge usually may not be
available.
 Learning become a important mechanism for
acquiring such knowledge.

Feb., 2011
3
Preface
 Learning control seems a good idea for control
design for unknown or uncertain systems.
 To learn controllers is always a good idea, but
somehow like a dream. It is because learning is
to learn from something. But when there is no
good controller, where to learn from?
 This talk is to discuss fundamental ideas and
problems in one learning controller -- adaptive
fuzzy control.

Feb., 2011
4
Outline
 Preface
 Introduction to learning control and fuzzy
systems
 Basic ideas in adaptive fuzzy control
 Problems and possible approaches for
resolving those problems
 Conclusive remarks

Feb., 2011
5
Learning Control
Learning control is to learn to know some
unknown quantities. In other words, it is to
find a way of estimating or successively
approximating those unknown quantities.
Categories of targets for learning in control:
 Learning about the plant;
 Learning about the environment;
 Learning about the controller; and
 Learning new design goal and constraints.

Feb., 2011
6
Learning Control
 The first two categories are more like modeling.
It is easy to achieve by using supervised
learning schemes, but have difficulties in
designing controller (model based control) due
to nonlinearity and insufficient learning.
Most learning control research efforts are in
these two categories.
 The last one is AI related issue and is usually
considered for general systems instead of
control systems.

Feb., 2011
7
Learning Control
 To learn controllers is always a good idea, but
somehow like a dream. It is because learning is
to learn from something. But when there is no
good controller, where to learn from?
 Nevertheless, there still exist approaches, such
as adaptive fuzzy control, that can facilitate
such an idea.  performance based learning
(reinforcement learning and Lyapunov stability)

Feb., 2011
8
Learning Control
Performance based approach — find a way of
optimizing an index which is directly related to
the control performance.
1. Reinforcement learning is to tune parameters
directly under a temporal difference fashion to
optimize the performance index (external
reinforcement).
It is usually in a trial-and-error manner due to
no knowledge about the system.

Feb., 2011
9
Learning Control
2. Lyapunov stability is to derive update rules
of parameters from a Lyapunov function, which
is always positive and is a function of the
control error.
The idea is to let the derivative of the Lyapunov
function is negative under a certain condition
and this condition will be the update law.
In that case, the system can be said to be
stable and the error will eventually become
zero.

Feb., 2011
10
Fuzzy Systems
In recent development, fuzzy systems have been
considered as an alternative representation of a
nonlinear system but with a linear system in
each rule so that approaches for linear systems
can also be applied.
For unknown systems, parameters of the fuzzy
systems must be estimated.
When the considered fuzzy system is a controller
and those parameters are adaptively tuned or
updated, it can be called adaptive fuzzy control.

Feb., 2011
11
Fuzzy Systems
A fuzzy approximator is constructed by a set of
fuzzy rules as
Generally, is a fuzzy singleton (TS fuzzy model).
Another type is to use a linear combination of input
variables. In that case, usually, the recursive
least square (RLS) approach (or recursive
Kalman filter) can be used to identify those
coefficients.
M
l
y
A
x
A
x
R l
F
l
n
n
l
l
,
,
2
,
1
for
,
is
THEN
is
and
,
and
,
is
IF
: 1
1




l


Feb., 2011
12
Fuzzy Systems
The fuzzy systems with the center-of area like
defuzzification and product inference can be
obtained as
It is a universal function approximator and is
written as .
 
 
 
 

M
l
n
i
i
A
M
l
n
i
i
A
l
f
x
x
y
l
i
l
i
1 1
1 1
)
)
(
(
)
)
(
(
)
(



x
ω
θ
θ
x T
f
y 
)
(
t-norm operation for
all premise parts

Feb., 2011
13
Fuzzy System
It should be noted that the above system is a
nonlinear system. But, it can be seen that the
form is virtually linear.( )
Thus, various approaches have been proposed to
handle nonlinear systems by using the linear
system techniques for the linear property
bearing in each rule, such as common P
stability or LMI design process.
ω
θ
θ
x T
f
y 
)
(

Feb., 2011
14
 Preface
systems

Feb., 2011
15
Adaptive Fuzzy Control
Consider the following nth order nonlinear
system :
is the system output.
1 2
2 3
( )
:
( ) ( )
n
x x
x x
N
x f g u f gu


 



    
 x x
1
x
y 

Feb., 2011
16
The objective is to design an controller such that
y tracks a desired signal ym(t).
Let be the tracking error and can be
written as
Based on the feedback linearization method, if
and are known, the reference controller is [1]
.
y
y
e m 

T
n
T
n
e
e
e
e
e
e ]
,
,
,
[
]
,
,
,
[ 2
1
)
1
(


 
 
e
* ( )
1
( )
n T
m
u f y
g
    k e
This is also referred
to as the perfect
control law

Feb., 2011
17
Feedback linearization
For tracking control, we need to obtain an perfect
control law referred to “Applied Nonlinear
Control” a method which is called “Feedback
Linearization Method.”
( )
1
[ f( ) ]
g( )
n T
d
u x
   
x K E
x
f( ) g( )
n
x u
 
x x ( ) ( 1)
1 0
n n
n
e k e k e

   
lim ( ) 0
t
e t


substitute
Hurwitz
Assum
e
g(x)≠0

Feb., 2011
18
where is selected such that all
roots of
are in the open left-half plane.
The tracking error dynamics
can have .
If f and g are known, the control law can be
fulfilled and then the control performance can
be guaranteed.
T
n
n k
k
k ]
,
,
,
[ 1
1 


k
0
0
1
)
1
(
1
)
(




 
 k
s
k
s
k
s n
n
n

0
)
(
)
(


 e
kT
n
m
n
y
x
0
lim 


e
t

Feb., 2011
19
But, when f and g are unknown or subject to
uncertainties, the linear feedback control may
not work.
Adaptive fuzzy control is then use fuzzy
approximator (systems) to approximate them.
Direct adaptive fuzzy control – to estimate
directly the controller .
In other words, it is to use
to model the perfect control law directly.
ω
θ
θ
x T
f
y 
)
(
* ( )
1
( )
n T
m
u f y
g
    k e

Feb., 2011
20
But, when f and g are unknown or subject to
uncertainties, the linear feedback control may
not work.
Adaptive fuzzy control is then use fuzzy
approximator (systems) to approximate them.
Direct adaptive fuzzy control – to estimate
directly the controller .
Indirect adaptive fuzzy control – to estimate f
and g.
the perfect control law
*
u
* ( )
1
( )
n T
m
u f y
g
    k e

Feb., 2011
21
Direct Adaptive Fuzzy Control
To approximate the controller by using a fuzzy
system as [3].
Consider the following Lyapunov function
where is the error of the
estimated parameter and is the optimal
parameter vector and is defined as
D
T
D
T
V θ
θ
Pe
e
~
~
2
1
2
1



*
( )
D D D
 
θ θ θ
*
D
θ
}
)
(
ˆ
sup
{
min
arg *
*
*
x
D
D
D u
u
d
D
θ
x
θ
x
θ





 
ˆ( ) T
D
u 
x θ θ ω

Feb., 2011
22
function is negative. In that case, the system can
be said to be stable and the error will eventually
become zero if possible.
The second term of the Lyapunov function can be
view as to minimize the approximation errors.
In fact, it is to generate the derivative of ,
which will be used to form the update rule for .
D
θ
D
θ
... ( ) ... ... ... ( terms) ...
T T T
D D D D D D
d
dt
       
θ θ θ θ θ θ
terms update law
D  
θ

Feb., 2011
23
function is negative. In that case, the system can
be said to be stable and the error will eventually
become zero if possible.
The second term of the Lyapunov function can be
view as to minimize the approximation errors.
In fact, it is to generate the derivative of ,
which will be used to form the update rule for .
This kind of approach can be seen in lots of
learning or adaptive control schemes.
D
θ
D
θ

Feb., 2011
24
is approximate error (assumed to
be small enough.)
The idea is to let and to prove
the remaining terms are negative in general.
Then, it can be claimed that is negative.
D
T
D
T
T
V θ
θ
e
P
e
Pe
e 


 ~
1
2
1
2
1




)
(
~
1
2
1
D
D
T
T
D
D
T
T
θ
ω
PB
e
θ
PB
e
Qe
e 




 


1
T
D D


θ e PB ω
V
*
*
ˆD
D u
u 


Traditiona
l
Lyapunov
derivation
Q
PΛ
P
Λ 


T
, it is the update law for 
1
T
D D t

  
θ e PB ω

Feb., 2011
25
If you actually use this approach, the results may
not be satisfactory.
The example shown in the paper is only for
regulation control.
The main problem is whether there exist the optimal
control and whether it can be approximated by
the fuzzy approximator; that is, whether
is small enough?
*
*
ˆD
D u
u 



Feb., 2011
26
(regulation control)
The control results in the
original paper
(regulation control)
Another approach

Feb., 2011
27
(Tracking Control)
0 5 10 15 20
-1.5
-1
-0.5
0
0.5
1
1.5
2
2.5 Original
approach
Reference
trajectory
Another approach

Feb., 2011
28
Direct Adaptive Fuzzy Control [6]
0 5 10 15 20
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
Original
approach
Reference
trajectory Another approach

Feb., 2011
29
Indirect Adaptive Fuzzy Control [4]
To approximate f and g by using two fuzzy systems
as and .
Consider the following Lyapunov function
where those variables are similar to those defined
in direct adaptive control.
f
f f
T
f
f
ˆ
)
(
ˆ 
 ω
θ
θ
x g
g g
T
g
g ˆ
)
(
ˆ 
 ω
θ
θ
x
g
T
g
f
T
f
T
V θ
θ
θ
θ
Pe
e
~
~
2
1
~
~
2
1
2
1
2
1 





Feb., 2011
30
Indirect Adaptive Fuzzy Control
The approximate error is
Similarly we can have the update rules as
)
ˆ
(
~
1
)
(
~
1
2
1
1
2
2
1
1
1
1
g
g
T
I
T
g
f
f
T
T
f
I
T
T
u
V
θ
ω
PB
e
θ
θ
ω
PB
e
θ
PB
e
Qe
e















f
T
f ω
PB
e
θ 1
1




g
T
I
g u ω
PB
e
θ 1
2 ˆ




I
I u
g
g
f
f ˆ
)
ˆ
(
)
ˆ
( *
*





Assume to be small
enough

Feb., 2011
31
Original adaptive control scheme
Update Laws:
1
T
f f
e
  
  PB
2
T
g g
e u
  
  PB
Fuzzy Approximator

Feb., 2011
32
Adaptive fuzzy control is to use fuzzy approximator –
.
Fuzzy systems are universal approximators [2].
Other universal approximators can also be used, such
as:
 Radial Basis Functions;
 Cerebellar Model Articulation Controllers;
 Wavelets; etc.
As long as they can also be written as a linear form
like .
ω
θ
θ
x T
f
y 
)
(
ω
θ
θ
x T
f
y 
)
(

Feb., 2011
33
Some approaches claimed that they also used
neural networks to act as the approximator in
their approach. In fact, it is one kind of radial
basis function neural networks, which can be
equivalent to a fuzzy system.
If you want to use other approximators which are
not of linear form, some linear approximation
approaches (like first order Taylor expansion)
may be employed to make it workable in the
framework.

Feb., 2011
34
Also, such an idea can be employed to find
adaptive laws for other parameters.
Again, a linear form is needed (or some linear
approximation mechanism is employed) to
ensure a simple form of the update law.
Besides, the squared term of the parameter must
be added into the Lyapunov function to have a
basic formation of the update law.

Feb., 2011
35
Some approaches also adapt the idea of sliding
control, by defining the sliding surface as the
integral of the characteristic polynomial as
, where [7].
Then, the idea is to replace all error terms by the
sliding term. For example, the Lyapunov function
is defined as:
Similar results can be obtained.
( ) ( )
S t S t dt
 
( ) ˆ ˆ
( ) n T
S t e k e
 
2
2
1 1 1
( )
2
T T
g g f f
g f
V S
r r
   
  

Feb., 2011
36
 Preface
systems

Feb., 2011
37
There are problems in the above approaches:
 Control aspect — (the approximate
errors) may not be small. It may cause a
system stability problem.
 Learning aspect — Large error (chattering
phenomenon) in the Initial stage and
convergence problem (parameter drifting) in
the final stage.
or
D I
 

Feb., 2011
38
 Approximate errors and robust control
 Initialization and supervisory control.
 Parameter drifting

Feb., 2011
39
Approximate Errors
Errors may exist due to rule resolution (rule
numbers) and rule dependency (input-output
deterministic).
Rule resolution may not be sufficient if the rule
number used is small.  universal
approximator theorem
1 1
V= ( )
2
T T T T
D D D D
 

   
e Qe e PB θ e PB ω θ
When is large, then this part
may not be negative.
With the update law, it
is zero.
D

D


Feb., 2011
40
Another possible error-- Rule dependency may
not be sufficient if the input variables used to
define the input-output relationship is not
sufficient.  It is called nondeterministic in
traditional learning.
For current published work, only the error and the
error derivative are used as the input variables.
If the system considered is more complicated,
maybe more terms must be included.
Approximate Errors

Feb., 2011
41
)
( *
u
u 

 B
Λe
e

)
( )
(
1
*
e
kT
n
m
y
f
g
u 


 







e
B
Λe
e
e
design
dI
y
u )
(
:


 Differential input
An assignable stable
inner linear system
System output
, Q
PΛ
P
Λ 


T
Robust Control
else
dI u
u 
 *

Unknown input
, u = udesign + uelse
Designed input
error
It can be
viewed
as an
unknown
input
or
D I
 
It is in the above.

Feb., 2011
42
How to design udesign to yield that
the energy dynamics ( ) fits in with a special form.











else
dI
e
design
dI
u
u
y
u
*
)
(
:


 e
B
Λe
e

Pe
eT
V 
Use a Lyapunov function to find the energy change of systemΨ.
dI
T
design
T
T
u
V 
2
2 PB
e
PB
e
Qe
e 




We have
Dissipative
H∞ tracking performance
L2-gain inequality
V

(Energy dynamics equation)
Robust Control

Feb., 2011
43
u
d
T
u
d
d
u
T
d
T
g
k
g
g
k
g
g
k
g
g
k
V 


)
2
(
)
2
(
)}
2
(
)
2
{( 2
1 



 PB
e
Qe
e

u
T
T
d
T
g
g
k
V 
2
)
(
2 1
2
1 PB
e
PB
e
Qe
e 




u
low
d
up
T
u
low
d
up
T
g
k
g
g
k
g
V 
 )
2
(
)
2
(


 Qe
e

u
low
d
up
T
u
low
d
up
T
u
g
k
g
g
k
g



 )
2
(
)
2
(
)
,
( 

 Qe
e
e
Supply rate
Robust Control

Feb., 2011
44
u
low
d
up
T
u
low
d
up
T
g
k
g
g
k
g
V 
 )
2
(
)
2
(


 Qe
e







0
2
0
)
2
(
)
0
(
)
0
( dt
g
k
g
dt u
T
u
low
d
up
T
T


e
P
e
Qe
e



0
2
2
dt
d
T
L
Q e
Q
e
e



0
2
2
dt
u
T
u
L
u 


Controllable
attenuation level δ
 H∞ tracking performance
 L2-gain-like inequality
0
))
0
(
(
2
2
2
2
)
2
(
2
2


 e
e
V
low
d
up
L
u
L
Q
g
k
g


,
1
2
)
2
(
)
( 
 low
up
d g
g
k 
Integral
Robust Control

Feb., 2011
45
Consider an often-used inverted pendulum
system as
Considered Example
1 2
2
x x
x f gu d

  
2
1 2 1 1
2
1
sin ( sin cos )/( )
(4/3 cos / )
c
c
g x mlx x x m m
f
l m x m m
 

 
1
2
1
cos /( )
(4/3 cos / )
c
c
x m m
g
l x m m


 

Feb., 2011
46
Simulation 1: The assignable control performance test.
We let δ=0.2, 0.3, 0.4, and 0.5.
0 2 4 6 8 10
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
time (sec.)
y
and
ym
(rad)
7 7.5 8 8.5 9
0.8
0.85
0.9
0.95
1
1.05
1.1
time (sec.)
y
and
ym
(rad) ym
Delta=0.2
Delta=0.3
Delta=0.4
Delta=0.5
Tracking control performance
Robust Control

Feb., 2011
47
0 0.2 0.4 0.6 0.8 1
-600
-400
-200
0
200
400
600
800
time (sec.)
Control
action
(U=Ud)
Nt-m
Delta=0.2
Delta=0.3
Delta=0.4
delta=0.5
High Initial Gain Problem.
A solution is provided in later.
δ=0.2
Control actions
Robust Control

Feb., 2011
48
1
)
sgn( PB
eT
c g
K
u 
)
( *
u
u
V T
T



 PB
e
Qe
e

*
2
1)
( u
g
K
V T
T
c
T
PB
e
PB
e
Qe
e 




Bounded
Negative define term
L2-gain state
feedback
controller
A suitable value of Kc leads the equation to be minimum,
which results in a more negative value of the derivation
of V, and the initial control action does not have the
oscillation (high-gain) problem.
How to find a suitable Kc ?
Robust Control

Feb., 2011
49
A large Kc will have nice control performance (small ) but
will have large initial control gain, but a small Kc may have
a large error in the final stage.
The research goal is that how to reduce the
oscillation phenomenon of the initial control
action and remain the satisfactory initial state
response.
A idea is to use a small Kc in the initial stage and a large Kc
in the final stage. But how to change?
Use genetic algorithm to in adjusting Kc.
Robust Control

Feb., 2011
50
Search region
The assignable control performance is a inherent
property of the L2-gain control, which can be applied to
define the search region.
The attenuation level
determines the tracking
control accuracy, and we
can use the selection of Kc
to adjust ,
such as

low
c
up g
K
g 2
/


up
low g
g
g 


7 7.5 8 8.5 9
0.8
0.85
0.9
0.95
1
1.05
1.1
time (sec.)
y
and
ym
(rad)
ym
Delta=0.2
Delta=0.3
Delta=0.4
Delta=0.5
Example:
2
.
0
,
5
.
0 min
max 
 


Feb., 2011
51
Random population
Cost function and
Auxiliary search condition
Roulette wheel selection
and
the elite reproduction
Living population
Crossover
Mutation
Population
of next
generation
The l-th chromosome is represented as
)
( l
c
l
K
Ch  m
l ,
,
2
,
1 

m is the number of the used chromosomes.
is a gene (solution) of the l-th chromosome.
,
l
c
K
If a chromosome cannot satisfy the
auxiliary search condition, or its cost is
larger than a threshold cost (CostT) ,
then the chromosome will be replaced
by another good chromosome.
is the minimum cost of the k-th generation.
is the average cost of the k-th generation.
ho ≦ 1 is a retaining constant;
it provides the multiplicity for the population.
)
(k
CostMin
)
(k
CostAvg
1
2
3
)
(
}
)
(
)
(
{
)
( k
CostMin
h
k
CostAvg
k
CostMin
k
CostT o 



Feb., 2011
52
Cost function
*
2
1)
( u
g
K
V T
T
c
T
PB
e
PB
e
Qe
e 




Bounded
Negative definite term
Cost function 2
1)
( PB
eT
c
K
CostE 

A suitable value of Kc leads the cost function to
be minimum.

Feb., 2011
53




 u
t
t
u
t
u )
(
)
(
In order to resolve the oscillation problem in
the initial stage, we must avoid the minimum
solution being found too early.
It is the sample time (0.01 seconds).
A constant which is used to
restrict the evolution speed.
 An auxiliary search condition is defined under the change
of the control action as
The evolution speed is needed to be restricted
that is the design basis of the auxiliary search condition.

Feb., 2011
54
0 2 4 6 8 10
-1.5
-1
-0.5
0
0.5
1
1.5
time (sec.)
y
and
ym
(rad)
y
ym
0 1 2 3 4 5
-300
-250
-200
-150
-100
-50
0
50
100
150
time (sec.)
Control
action
(Nt-m)
High initial gain problem
0 2 4 6 8 10
-1.5
-1
-0.5
0
0.5
1
1.5
time (sec.)
y
and
ym
(rad)
y
ym
0 1 2 3 4 5
-300
-250
-200
-150
-100
-50
0
50
100
150
time (sec.)
Control
action
(Nt-m)
High initial gain problem
1
)
sgn(
80 PB
eT
g
u 
)
(
)
10
sin(
10 m
Nt
t
d 

1
)
sgn(
80 PB
eT
g
u 
0

d
Without the genetic adaptive scheme
Without the genetic adaptive scheme
The tracking control
performance
Control action

Feb., 2011
55
0 2 4 6 8 10
-1.5
-1
-0.5
0
0.5
1
1.5
time (sec.)
y
and
ym
(rad)
y
ym
0 1 2 3 4 5
-120
-100
-80
-60
-40
-20
0
20
40
time (sec.)
Control
action
(Nt-m)
0 2 4 6 8 10
-1.5
-1
-0.5
0
0.5
1
1.5
time (sec.)
y
and
ym
(rad)
y
ym
0 1 2 3 4 5
-300
-250
-200
-150
-100
-50
0
50
time (sec.)
Control
action
(Nt-m)
1
)
sgn(
)
( PB
eT
g
GA
u 
)
(
)
10
sin(
10 m
Nt
t
d 

1
)
sgn(
)
( PB
eT
g
GA
u 
0

d
With the genetic adaptive scheme
With the genetic adaptive scheme
The initial tracking performance
is not sacrificed.
The initial tracking performance
is not sacrificed.

Feb., 2011
56
c
T
t
T
T
u
V PB
e
PB
e
Qe
e 2
2 


 

We can add an integral term to have more stable
dt
g
k
g
k
u T
t
c
T
c
c 1
0
2
1
1 )
sgn(
)
sgn( PB
e
PB
e 


The compensative controller is defined as







t
T
c
t
c
c
t
T
c
T
dt
g
k
g
k
g
g
k
g
g
k
V
0
2
1
2
2
2
1
2
1
1
1
)
(
2
)
2
(
)
2
2
(
PB
e
PB
e
Qe
e 


0

1
c
k 0

2
c
k
By substituting uc into .
V

Robust Control
Additional negative energy
Added integral term

Feb., 2011
57
Robust Control
0 2 4 6 8 10
-1.5
-1
-0.5
0
0.5
1
1.5
2
time (sec.)
y
and
ym
(rad)
y (Kc1=20, Kc2=0)
ym
y (Kc1=20, Kc2=300)
0 2 4 6 8 10
-40
-30
-20
-10
0
10
20
30
40
50
60
time (sec.)
Total
control
action
(Nt-m)
u (Kc1=20, Kc2=0)
u (Kc1=20, Kc2=300))
An integral term provides a more stable edge to
have better control performance.
Without the
integral term
With the
integral term
Without the
integral term
With the
integral term
Tracking
performances
Control actions

Feb., 2011
58
Another way of handling errors is to consider
those errors in the controller (error feedback
controller).
For indirect adaptive fuzzy control, it is easy to
find ways of estimating those errors and/or
compensating them.
For direct adaptive fuzzy control, it may be difficult
to compensate the approximate error because
it is difficult to define control errors.
Approximate Errors

Feb., 2011
59
Error Feedback Controller (Indirect)
An approach is proposed to improve the
accuracy of estimated value. Define a
modeling plant as
Define the estimated state error as ;
that is, ( )
( ) ( )
n
x f x g x u
 
ˆ
f f f
  ˆ
g g g
 
( ) ˆ
ˆ ˆ
( ) ( ) ( )
n
x f x g x u t
 
( ) ( ) ( )
ˆ
n n n
x x x
 
Estimated f
by the fuzzy
system
Estimated g
by the fuzzy
system

Feb., 2011
60
Approximate Errors (Indirect)
Now, define a new Lyapunov function as
The idea is to minimize the modeling error while
adaptive.
Similarly we can we the update rules as
2
g g
1 2
1 1 1 1
V
2 2 2 2
T T
n f f
e e x    
 
   
T
P
1( )
T
f n f
x e
  
 B P
2 ( )
T
g n g
x e u
  
 B P
Approach I

Feb., 2011
61
Approximate Errors (Indirect)
Fuzzy Approximator
Note:
Lyapunov Laws:
1( )
T
f n f
x e
  
 B P
2 ( )
T
g n g
x e u
  
 B P
To calculate
the estimated
model
Model error

Feb., 2011
62
Simulation – chaotic eq.
Simulate the chaotic equation
select vector k and matrix Q are
The desire output .
Some parameter β1=70,β2=0.01, gL=0.01.
There three conditions are simulated
1. Simulation with noise-free
2. Simulation with disturbance: with disturbance at 10
sec which function is .
3. Simulation with noise: with noise whose mean is 0,
and standard deviation is 0.01.
3
0.1 12cos( )
x x x t u
   
 
30 5
12 7 ,
5 30
T  
   
 
k Q
cos( )
d
x t

2 2
0.05exp( / 0.1 )
x


Feb., 2011
63
Original approach
Tracking error converges at 0.003.
Proposed approach
Condition 1.
20% improvement in error reduction

Feb., 2011
64
Original approach
Proposed approach
Condition 2.
26.8% improvement in error reduction

Feb., 2011
65
Original approach
Proposed approach
Condition 3.
21.9% improvement in error reduction

Feb., 2011
66
Another Approach [8]
CMAC
based
Learning
New added
compensated
control

Feb., 2011
67
Approach II
CMAC
based
parameter
Learning
compensated
control also has
some bounded
adaptive effects
(discussed later)

Feb., 2011
68
Simulation
Original approach Approach II

Feb., 2011
69
Simulation
Approach I Approach II
0 1 2 3 4 5 6 7 8 9 10
-0.4
-0.2
0
0.2
0.4
supervisory adaptive FCMAC with CMAC compensator Dmax=4
(a) sec
RMSE=0.016899
rad
Nt
0 1 2 3 4 5 6 7 8 9 10
-20
-15
-10
-5
0
5
(b) sec
y
r
U
d
0 1 2 3 4 5 6 7 8 9 10
-0.4
-0.2
0
0.2
0.4
supervisory adaptive FCMAC with fictitious model Dmax=4
(a) sec
RMSE=0.016452
rad
Nt
0 1 2 3 4 5 6 7 8 9 10
-20
-15
-10
-5
0
5
(b) sec
y
r
U
d
The control performance is comparable.
Disturbance

Feb., 2011
70
0 1 2 3 4 5 6 7 8 9 10
-20
-10
0
10
20
0 1 2 3 4 5 6 7 8 9 10
1.1
1.2
1.3
1.4
1.5
1.6
uf
f
ug
g
0 1 2 3 4 5 6 7 8 9 10
-20
-10
0
10
20
(c) sec
uf
f
0 1 2 3 4 5 6 7 8 9 10
1.1
1.2
1.3
1.4
1.5
1.6
(d) sec
ug
g
Simulation – Approach I
Without disturbance With disturbance
Modeling performance
is acceptable.
The disturbance
is modeled into
the system
function.
Modeling f
Modeling g

Feb., 2011
71
Simulation – Approach II
Without disturbance With disturbance
0 1 2 3 4 5 6 7 8 9 10
-5
0
5
uf
f
0 1 2 3 4 5 6 7 8 9 10
1.1
1.2
1.3
1.4
1.5
ug
g
0 1 2 3 4 5 6 7 8 9 10
-5
0
5
(c) sec
0 1 2 3 4 5 6 7 8 9 10
1.3
1.4
1.5
1.6
(d) sec
uf
f
ug
g
Modeling
performance is
unacceptable.
The modeling
effects will become
worse.
Modeling f
Modeling g

Feb., 2011
72
where represents the system
output energy and represents
the system input energy.
Definition:
A control system is said to have the finite
-gain property if there exists an assignable finite
gain and a bias constant
representing the initial condition such that the
following inequality holds
2
L
0

 
R
bias

bias
L
T
t
L
T f
f


 

2
2
e


f
f
T
T
L
T dt
0
2
e
e
e


f
f
T
t
L
T
t dt
0
2
2


Error Feedback Controller (Direct)
A way of defining errors must be developed for direct
adaptive fuzzy control.

Feb., 2011
73
d
T
d
d
c u
u
u ω
θ
~
*



is estimated as .
T
d
c
d u )
(
~ 1

 ω
θ
*
lim u
u
u c
d
t




Consider that
d
θ
~
[6] E. Kim, “A fuzzy disturbance
observer and its application to control,”
IEEE Trans. Fuzzy Systems, vol. 10, no.
1, Feb. 2002.
The inequity indicates that --
bias
L
tT
L
T f
f


 

2
2
e
the tracking control error is bounded in a region
around origin, the size of the region can be
arbitrarily small with the choice of δ. Thus, the
following equation is guaranteed as [6].

Feb., 2011
74
By substituting the estimative value into the earlier
adaptive law, the proposed adaptive law is found as
follows:
T
d
c
m
d
T
m
d
u
g )
(
)
sgn( 1
2
1


 ω
e
ω
PB
e
θ 


is a simple adaptation
scheme to enhance the learning stability
more.
Q.E.D.
The estimative value can be multiplied by another
adaptive rate as
T
d
c
m
d u )
(
~ 1
2

 ω
e
θ 
2
e
m

2
2
2
2
1
2 n
e
e
e 


 
e

Feb., 2011
75
1. If , then the approximate error feedback term
is not used.
is still used, the other parameters and will be
adjusted to show the change of the learning speed.
Simulation 2: The learning speed tests are illustrated in this
simulation.
15
1 
c
k m

m

T
d
c
m u )
( 1
2

ω
e

0

m

2. The value of is also increased to illustrate the effects of
the learning speed.
m

Adaptive rate 1
Adaptive rate 2
Simulations

Feb., 2011
76
Reached Time (sec.) and Cycle
0.1 20 25.12 (4th cycle)
0.1 0 Unstable
1 20 25.12 (4th cycle)
1 0 Unstable
10 20 18.84 (3th cycle)
10 0 Unstable
20 20 18.84 (3th cycle)
20 0 43.96 (7th cycle)
30 20 18.84 (3th cycle)
30 0 31.40 (5th cycle)
40 20 18.84 (3th cycle)
40 0 25.12 (4th cycle)
50 20 18.84 (3th cycle)
50 0 18.84 (3th cycle)
m

m

1. A suitable βm
provides more stable
learning speed even
if the adaptive rates
are different.
2. It can be found that
the selection of the
adaptive rate can be
relaxed because the
proposed approach.
m

T
d
c
m
d
T
m
d u
g )
(
)
sgn( 1
2
1


 ω
e
ω
PB
e
θ 


d
T
m
d g ω
PB
e
θ 1
)
sgn(



Simulate results
 The stable learning speed
is guaranteed.
The initial learning stability
is guaranteed.

Feb., 2011
77
 Approximate errors and robust control

Feb., 2011
78
Initial status (initial states and initial parameter
values) may cause various problems for a
learning control system.
A so-called supervisory controller [3,5] is often
used and the effects are satisfactory. In above
examples, all use supervisory controllers. It is
similar to hitting control for sliding model control.
The supervisory controller is proposed in the early
version of adaptive fuzzy control and can also
act as one kind of robust control.
Initialization

Feb., 2011
79
Supervisory Control
Adaptive
fuzzy
control
supervisory
control
Robust control if
used, such as
compensated
control

Feb., 2011
80
The supervisory control is to let ,
where is the approximated perfect control
law and is the supervisory controller.
Consider the derivative of Lyapunov function
, where is
the optimal control.
Thus, if is large enough, the derivate of V will
always be negative.
Supervisory Control
p s
u u u
 
p
u
s
u
*
(1 2) ( )
T T
s s s p s
V u u u
    
e Q e e P B *
u
s
u
It is I


Feb., 2011
81
Controller p s
u u u
  Supervisory controller
sgn( )( )
s su
u g C
 
)
(
)
2
1
( *
1
1 m
s
s
T
su
s
T
s
T
s gu
gu
C
g
V 



 B
P
e
B
P
e
e
Q
e

T
]
1
0
0
0
[
1 

B
e
kT
s
n
m
s y
f
gu 


 )
(
*
( )
1 1
1
( )
2
T T T n T
s s s su s m s p
V g C f y gu
       
e Q e e P B e P B k e
Supervisory Control

Feb., 2011
82
( )
1 1
1
( )
2
T T T n T
s s s su s m s p
V g C f y gu
       
e Q e e P B e P B k e
)
)
(
(
)
2
1
( )
(
1
1 up
m
T
s
n
m
up
s
T
su
su
s
T
s
T
s gu
y
f
C
K
V 





 e
k
B
P
e
B
P
e
e
Q
e

1
su
K  : A positive constant.
0
up
f  : The upper-bound of f.
( )
p up
gu : The upper-bound of the gum.
( )
1
sgn( )( ( ) )
T n T
su s up m s p up
C f y gu
    
e P B k e
0

s
V

Stable condition :
Design
( )
1
sgn( )sgn( ) ( ( ) )
T n T
s s su up m s p up
u g K f y gu
   
e P B k e
sgn( )( )
s su
u g C
 
Design result :
Supervisory Control

Feb., 2011
83
Thus, the supervisory controller can be selected
as
where the subscript up is the upper bound of
that function and Ks is a constant.
It can be found that the supervisory controller is a
function of the upper bound of the system
function.
If the bound is not properly selected, the control
performance may not be satisfactory.
Supervisory Control
( )
1
sgn( )sgn( ) ( ( ) )
T n T
s s s up m s p up
u g K f y gu
   
e P B k e

Feb., 2011
84
0 1 2 3 4 5 6 7 8 9 10
-0.4
-0.2
0
0.2
0.4
adaptive fuzzy without supervisory control Dmax=4 maxfzi=1 maxfzo=5
(a) sec
RMSE=0.013749
rad
Nt
y
r
0 1 2 3 4 5 6 7 8 9 10
-20
-10
0
10
20
(b) sec
U
d
Without supervisory control
Simulation

Feb., 2011
85
With supervisory control
Simulation
0 1 2 3 4 5 6 7 8 9 10
-0.4
-0.2
0
0.2
0.4
Dmax=4 maxfzi=1 maxfzo=5
(a) sec
RMSE=0.0062999
rad
Nt
y
r
0 1 2 3 4 5 6 7 8 9 10
-20
-10
0
10
20
(b) sec
U
d
Improved from
0.013749 (50%
reduction)

Feb., 2011
86
Supervisory Control
Consider another system as
This system do not have a bound for the system
function.  The system diverges.
1
x
x e u
  

Feb., 2011
87
As mentioned earlier, the approach (approach II) with
compensated control also considers the bound in
the supervisory controller,
Supervisory Control
Improved from
0.0062999 (simple
supervisory control)

Feb., 2011
88
With compensated control
Simulation– exponential case
0 1 2 3 4 5 6 7 8 9 10
-0.4
-0.2
0
0.2
0.4
0.6
Dmax=4 Mo=1 maxfzi=1 maxfzo=5
(a) sec
RMSE=0.046899
rad
Nt
0 1 2 3 4 5 6 7 8 9 10
-5
0
5
(b) sec
y
r
U
d
Not good
enough

Feb., 2011
89
The problem is that the bound is a function of f.
The idea is to use previous control action so that
the bounded for the perfect control law can be
reduced so that the supervisory control can
easily be implemented.
The term of the system function becomes the
difference of the system function, of which the
bound is much smaller than that of the system
function.
Supervisory Control
( )
1
sgn( )sgn( ) ( ( ) )
T n T
s s s up m s p up
u g K f y gu
   
e P B k e

Feb., 2011
90
Supervisory Control
( ) ( )
0
* ( ) ( 1) [ ( ) ( 1) ( ) ( 1)
ˆ ˆ ˆ
( ( ) ( 1)) ]
n n
m m
T
g
u u k u k g y k y k f k f k
k e k e k err
        
   
( )
0 0
* ( 1) ( )
n
u u k g e f g E
     
g tracking transition
E err err err
  
( )
0
( 1) ( )
n T
f f CMAC s
u u k M e u u
 
     
Compensated
learning for E.

Feb., 2011
91
Supervisory Control
s
f
)
n
(
rf u
e
M
k
u
u 



  )
(
)
1
( T
0 

max
max )
/
( D
D
S
sat
us 

k
v
s
old
new C
u
LM
W
W
0
1


W
C
u T
v
CMAC k

s
u
y
r +
-
CMAC
u
d
bu
f
x n



)
(
Robust fuzzy controller
CMAC learning process
CMAC controlling process
+
+

Feb., 2011
92
Inverted Pendulum
0 1 2 3 4 5 6 7 8 9 10
-0.4
-0.2
0
0.2
0.4
CMAC compensator Dmax=4 Mo=0.7 maxfzi=1 maxfzo=5
(a) sec
RMSE=0.000777
rad
Nt
y
r
0 1 2 3 4 5 6 7 8 9 10
-5
0
5
10
15
(b) sec
U
d
0.0062999 for
supervisory control
0.0028552 for with
compensated control

Feb., 2011
93
Exponential System
0 1 2 3 4 5 6 7 8 9 10
-0.4
-0.2
0
0.2
0.4
0.6
CMAC compensator Dmax=4 Mo=1
(a) sec
RMSE=0.030789
rad
Nt
0 1 2 3 4 5 6 7 8 9 10
-5
0
5
(b) sec
y
r
U
d
Much
better than
other
approache
s

Feb., 2011
94
 Modeling errors

Feb., 2011
95
For adaptive fuzzy control, it can be found that the
parameter is a function of errors:
When there are errors, the parameters will be
changed. It can be expected that for tracking
problems, there are always errors and the
parameters are always changing.
Parameter Drifting
f
T
f ω
PB
e
θ 1
1




g
T
I
g u ω
PB
e
θ 1
2 ˆ




This referred to as
the parameter drifting problem.

Feb., 2011
96
Parameter Drifting
Two situations occur:
 The parameters may drift to some unwanted
regions (in fact, some values may go
unbounded.)
 The parameters in the optimal controller are not
constants. This violates the basic assumption in
the derivation of the update rules.
 no longer true!
*
( )
   
θ θ θ θ

Feb., 2011
97
The actual output y The control u
The consequence
Parameter Drifting (without disturbance)
0 5 10 15 20 25 30 35 40 45 50
-15
-10
-5
0
5
10
15
0 5 10 15 20 25 30 35 40 45 50
-5
-4
-3
-2
-1
0
1
2
3
4
5
0 5 10 15 20 25 30 35 40 45 50
-2.5
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
2.5
y
ym

The tracking error
is small enough in
5 sec.
All parameters are
still changing

Feb., 2011
98
0 10 20 30 40 50 60 70 80 90 100
-0.5
0
0.5
1
The consequence
0 10 20 30 40 50 60 70 80 90 100
-60
-40
-20
0
20
40
60
Parameter Drifting (with disturbance)
0 10 20 30 40 50 60 70 80 90 100
-20
-15
-10
-5
0
5
10
15
20
25

With external noise,
the system may
become unacceptable

Feb., 2011
99
Parameter Drifting
For the unbounded phenomenon, the original
adaptive fuzzy control [3] has proposed a
simple way of restraining it.
 By simply clipping the bounded
 By using the projection onto the boundary
surface. (Projection methods)

Feb., 2011
10
0
Clipping – Regulation control
The consequence
M
0 10 20 30 40 50 60 70 80 90 100
-0.5
0
0.5
1
0 10 20 30 40 50 60 70 80 90 100
-8
-6
-4
-2
0
2
4
6
0 10 20 30 40 50 60 70 80 90 100
-10
-8
-6
-4
-2
0
2
4
6
8
10

Most of the
parameters go to
the boundaries.
non-reasonable

Feb., 2011
10
1
Projection
The update parameters = *
The consequence
 
0 10 20 30 40 50 60 70 80 90 100
-0.5
0
0.5
1
0 10 20 30 40 50 60 70 80 90 100
-8
-6
-4
-2
0
2
4
0 10 20 30 40 50 60 70 80 90 100
-8
-6
-4
-2
0
2
4
6
8

| |
n
M 

The tracking
error is small
enough in 5 sec.
All parameters
are saturated
after 30 sec.

Feb., 2011
10
2
Projection– Tracking control

For tracking
cases, the
parameters
are not
constants.

Feb., 2011
10
3
Parameter Drifting
In above situations, it can be found that the
parameters in the learned control are never
constants. This violates the basic assumption
in the derivation of the update rules.
Besides, it becomes an adaptive controller
because the learned controller may not work
well when the system stops learning.
Note that such a controller still works well, but the
adaptive mechanism cannot be stopped.

Feb., 2011
10
4
Parameter Drifting
Another approach is to consider the dead-zone
modification. The idea is simple. It is to stop
learning under certain conditions. It is similar
to the early stopping approach in neural
network learning to avoid overfitting.
The problem is when to stop learning? Can the
learned controller can work fairly without
adaptation?

Feb., 2011
10
5
Parameter Drifting
The dead-zone approach is to modify the
adaptive rule as
How to select ?
It is desired that the error will not become larger
than when the learning is stopped.
1 2
2
sgn( ) , if
0 , if
T
m D dz
D
dz
g e
e

 

 



e PB ω e
θ
e
dz
e
dz
e

Feb., 2011
10
6
Parameter Drifting
0 5 10 15 20 25 30
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0.09
0.1
time (sec.)
2-Norm
of
error
vector
edz
In this example, the learning
is turned on when the error
exceeds the threshold.
How can the
remaining error
be assured to be
smaller than the
threshold?
An ideal case

Feb., 2011
10
7
Parameter Drifting
If such a learning control is desired, a robust
mechanism must be employed to ensure that
the error bound be restrained in the control
process.
We have employed the dissipative control (HTAC)
in designing the supervisory controller as:
with the H-infinity tracking
performance having an attenuation level as
.
1
2
sgn( )
8
T
a
g
u

 e PB
2
low
g

 

Feb., 2011
10
8
Parameter Drifting
It can be any
adaptive controller,
such as adaptive
fuzzy or CMAC, etc.

Feb., 2011
10
9
Parameter Drifting
0 5 10 15 20 25 30
-0.3
-0.25
-0.2
-0.15
-0.1
-0.05
0
0.05
0.1
0.15
time (sec.)
Learning
results
of
fuzzy
system
(f-head)
0 5 10 15 20 25 30
0
0.02
0.04
0.06
0.08
0.1
0.12
time (sec.)
Learning
results
of
fuzzy
system
(g-head)
In this example, the learning
is turned on when the error
exceeds the threshold.
The learned f the learned g

Feb., 2011
11
0
Parameter Drifting
The learning is stopped after 15 seconds and
an external disturbance is added into the
system at the same time.
0 5 10 15 20 25 30
-1.5
-1
-0.5
0
0.5
1
1.5
time (sec.)
y
and
ym
y
ym
15 20 25 30
-1.5
-1
-0.5
0
0.5
1
1.5
time(sec.)
y
and
ym
y
ym
With HTAC, the
learned controller
can work well.
Without HTAC, the
learned controller
does not work well.

Feb., 2011
11
1
Conclusions
Adaptive fuzzy control can be viewed as one
learning control mechanism.
The idea is simple and can be extended to
various learning mechanisms.
In fact, such an idea can also be employed in
various learning control schemes.
Some deficits of such an approach are discussed.
If you want to use such kind of approaches,
those issues must be considered in your study.

Feb., 2011
11
2
Epilogue
Those ideas are from different papers. Thus, I do
not try to combine all approaches together.
Sometimes, some approaches may have similar
or conflict roles. If you are interested, you may
try them by yourself.
In fact, some approaches may not be complete. In
other words, you may find more problems and
more suitable approaches in you study .

Feb., 2011
11
3
Thankyoufor yourattention!
Any Questions ?!
Shun-Feng Su,
Chair Professor of Department of Electrical Engineering,
National Taiwan University of Science and Technology
E-mail: sfsu@mail.ntust.edu.tw,

Ideas and Problems in adaptive Fuzzy ControlCMLC2011.ppt

More Related Content

Similar to Ideas and Problems in adaptive Fuzzy ControlCMLC2011.ppt (20)

Recently uploaded (20)

Ideas and Problems in adaptive Fuzzy ControlCMLC2011.ppt