Geometrical control theory

E

extracta mathematicae Vol. 18, N´m. 2, 129 – 151 (2003)
u

A Panorama of Geometrical Optimal Control Theory
´
M. Delgado-Tellez, A. Ibort
Dpto. de Matem´ticas, Unidad Asociada de Matem´ticas UCIIIM-CSIC, Universidad
a
a
Carlos III de Madrid, 28911-Legan´s, Madrid, Spain
e

AMS Subject Class. (2000): 49J15, 34A09, 34K35

Received February 14, 2002

1. Introduction
Control theory is a young branch of mathematics that has developed
mostly in the realm of engineering problems. It is splitted in two major
branches; control theory of problems described by partial differential equations where control are exercized either by boundary terms and/or inhomogeneous terms and where the objective functionals are mostly quadratic forms;
and control theory of problems described by parameter dependent ordinary
differential equations. In this case it is more frequent to deal with non-linear
systems and non-quadratic objective functionals [49]. In spite that control
theory can be consider part of the general theory of differential equations,
the problems that inspires it and some of the results obtained so far, have
configured a theory with a strong and definite personality that is already offering interesting returns to its ancestors. For instance, the geometrization of
nonlinear affine-input control theory problems by introducing Lie-geometrical
methods into its analysis started already by R. Brockett [9] is inspiring classical Riemannian geometry and creating what is called today subriemannian
geometry.
In any case, the breadth of mathematics involved in modern control theory
is so promising that mathematicians, and in particular “applied” mathematicians, should be aware of their developments, having the total certainty that
they will find in this theory an important source of inspiration for their research.
In this review article we present a panorama of modern geometrical optimal
control theory for dynamical systems. Optimal control theory for dynamical
systems is perhaps, inside the vast subject of control theory, one of the oldest
129

130

´
m. delgado-tellez, a. ibort

and most developed parts. Optimal control theory sinks directly to the sources
of mechanics and the calculus of variations, and reaches its most profound
results by the hand of Pontriaguine’s maximum principle [49]. Rather recently,
a battery of geometrical methods has been developed to address from a new
perspective some old problems in the theory of control. Starting with the
already mentioned pioneering work by R.W. Brockett [9], [10], to the most
recent developments in the field, as reported for instance by H.J. Sussmann
[52], and the work of young mathematicians that are conducting their research
work in this direction [17], [42], [47], [18], etc., we are witnessing a flourishing
of geometrical ideas at the foundations of control theory.
In these notes we will report on some problems in optimal control theory
where, we believe, geometrical ideas will play a relevant role towards its understanding and eventual solution. Moreover, geometrically inspired ideas will
provide alternative treatments to previously considered approaches. Concretely, we will discuss the problem of cheap and singular optimal control and its
links to singular perturbations from a new perspective inspired by traditional constraints theory. We will also present the problem of implicit optimal
control and we will analize the various singular behaviours arising in such
situations. We will discuss also the problem of the integrability of optimal
control problems and the notion of symmetry and, finally, we will present a
few remarks on the problem of feedback linearizability and normal forms. We
will not, however, touch such interesting problems as controllability or observability, that have already deserved a lot of attention, and where Lie methods
have proved to be essential. Finally, we will leave the full discussion and details of the material presented here to the more technical papers quoted or
announced in the list of references at the end.
2. Singular optimal control
2.1. Regular and singular optimal control We will consider the
problem of finding C 1 -piecewise smooth curves γ(t) = (x(t), u(t)) with fixed
endpoints in state space, x(t0 ) = x0 and x(T ) = xT , satisfying the control
equation
(1)

xi (t) = f i (x(t), u(t)),
˙

and minimizing the objective functional
T

(2)

S(γ) =

L(x(t), u(t))dt.
t0

a panorama of geometrical optimal control theory

131

It is well-known that the solution of such problem is provided by Pontriaguine’s maximum principle; this is, the curve γ(t) = (x(t), u(t)) is an optimal
trajectory if there exists a lifting of x(t) to the coestate space, (x(t), p(t))
satisfying Hamilton’s equations
∂H
,
∂pi

xi =
˙

(3)

pi = −
˙

∂H
,
∂xi

and such that
(4)

H(x(t), p(t), u(t)) = max H(x(t), p(t), v),
v

a.e.

t ∈ [t0 , T ],

where H denotes the Hamiltonian function (Pontriaguine’s Hamiltonian),
(5)

H(x, p, u) = pi f i (x, u) − L(x, u).

It is clear that a necessary condition for the maximum condition (4) is that
∂H
the function defined by ϕa = ∂ua will vanish. Hence, the trajectories solution
of the optimal control problem will lie in the submanifold
M1 = {(x, p, u) | ϕa (x, p, u) = 0},
of the total space M with coordinates (x, p, u).
The constraint functions ϕa = 0, called in what follows zeroth order
constraints, will define implicitely a function
(6)

ua = ψ a (x, p)

whenever the matrix Wab defined by
Wab =

∂ϕa
∂ub

will be invertible. Thus, under these circumstances, on the submanifold M1
there will be defined an optimal feedback function ψ a given by eq. (6), i.e.,
whenever
(7)

det Wab = det pi

∂2f i
∂2L
− a b
∂ua ∂ub ∂u ∂u

= 0.

If such condition is satisfied at a given point m we will say that the optimal
control problem is regular at m. If det Wab = 0 at m we will say that the

132

´

optimal control problem is singular at m. Generically, the singular points for
a given optimal control problem will lie in a union of submanifolds of M1 , the
singular locus of the problem. At regular points we will substitute (6) at 3
obtaining a first order system of differential equations that possess locally a
unique solution for each initial data.
If we consider the particular example of control systems affine in the control
variables, i.e.,
i
xi = Ai (x) + Ba (x)ua
˙
the regularity condition eq. (7) becomes simply
det Wab = det

∂2L
= 0,
∂ua ∂ub

that coincides with the well-known regularity condition for Lagrangians in the
ordinary calculus of variations, where the control equation eq. (1) is xi = ui .
˙
Even more, in the case of linear control problems with quadratic regulators
(LQ systems), i.e., of the form,
i
xi = Ai xj + Ba ua ,
˙
j

(8)
with Lagrangian function,
(9)

1
1
L = Pij xi xj + Qia xi ua + Rab ua ub .
2
2

the regularity condition becomes simply the regularity of the matrix Rab .
The system will be singular if and only if the matrix R is singular. Notice
that regularity does not requires that R will be definite, condition that will be
needed for the extremal trajectories to be an actual minimun of the objective
functional. In obvious matrix notation
(10)

x = A · x + B · u,
˙

and
(11)

1
1
L = xt · P · x + xt · Q · u + ut · R · u.
2
2

Thus for regular LQ systems, the optimal feedback function (6) will be
u = R−1 pt · B − R−1 Q · x,


133

a
where R−1 denotes the inverse of the matrix Rab , (R−1 )ab Rbc = δc . Then
substituting in eq. (8) and the corresponding coestate equation, we obtain
the linear system

x = (A − BR−1 Q) · x + BR−1 pt · B
˙
and
p = pt · (A + QR−1 B) + (P − QR−1 Qt ) · x.
˙
Moreover, introducing the partial feedback p(t)t = K(t)x(t), we obtain the
linear system
(12)

x = (A − BR−1 Q + BR−1 BK)x
˙

and the matrix function K satisfies the matrix Ricatti equation,
˙
K = K[A − BR−1 Q] − K t [A − QR−1 B] + KBR−1 BK − QR−1 QT + P.
Thus, solving this last equation in the interval [t0 , T ], and replacing in the
equation for x, we will obtain a time-dependent system that we can eventually
solve explicitely.
An important observation is in place here. Let us suppose that R is regular
but has a small eigenvalue that we denote by . We can think alternatively
that R = R( ) is a one-parameter family of matrices such that R(0) is not
invertible. For instance R( ) = R0 where R0 is regular. In such case, R−1 =
−1 R−1 , and eq. (12) becomes the singularly perturbed problem
0
−1
−1
x = ( A − BR0 Q + BR0 BK)x.
˙

Thus, the analysis of a singular LQ systems leads naturally to a singularly
perturbed linear system. This situation has been studied extensively in [16].
This singular perturbation is different to the often-studied singular perturbation problem that results when the state equation is singularly perturbed [44],
[38], [39]. Generalized necessary and sufficient conditions for minimality of
singular arcs are studied in [5].
The analysis of these systems has great interest. We must notice first
that for a general singular problem, even an LQ system, the behavior of the
matrix R near the singular set can be much more complicated that the naive
example before. Then, the reduced system, i.e., → 0, will be in general
quite involved. We have addressed such problem in the general setting by
adapting ideas taken from the general theory of singular lagragians. The
analysis proceeds as sketched below and is described in detail in [20], [21].

134

´

Thus, either if we have a singular system,
det Wab = 0,
or we have a regular family Wab ( ), where = ( 1 , . . . , r ) is a multiparameter, with singular limit at → 0, we shall obtain a reduced system that will
be defined recursively by imposing the existence of solutions compatible with
the constraints. The natural way to express these conditions is by means of
the geometric formulation that receives the name, among others, of the presymplectic constraint algorithm (PCA), originally developed in its primitive
form by P.A.M. Dirac to quantize mechanical systems defined by singular lagrangians [24] and brought to maturity by the work of many people, ending
with the formulation presented in [28]. The PCA will eventually produce
a reduced submanifold M∞ where the state and coestate equations will be
well posed, i.e., its integral curves will lie in M∞ (even if they will not be
unique). Next step will be to match the resulting reduced solutions with the
endpoints. To do that we will proceed by considering a singularly perturbed
system, constructed step by step of the PCA, ending after this process with
a “cheap” or nearly optimal control singularly perturbed system called the
singular turnpike [20].
2.2. The geometrical setting of optimal control theory To
proceed further with the programme sketched at the end of the last paragraph,
we must clarify first the geometrical nature of the different objects involved
in the previous discussion. All the discussion before was done locally, hence
spurious identifications took place sometimes among various coordinates. We
will assume that the state space of the system, or configuration space, is a
smooth manifold P without boundary and with local coordinates xi . Control
variables will be local coordinates on an affine bundle over P . Such bundle
structure for the control variables π : C → P means that the control variables
are attached to points of the state space and transform under changes of
coordinates in state space
xi = ϕi (x),
´
affinely, this is,
ua = ϕa (x)ub + ρa (x).
´
b
Later on (see Section 5.1) we will return to discuss some consequences of this
assumption.
Thus, an ordinary differential equation on P depending on the parameters
ua , is nothing but a vector field Γ along the projection map π, i.e., Γ is a


135

smooth map Γ : C → T P such that τP ◦ Γ = π, where τP : T P → P denotes
the canonical projection. Locally the vector field Γ will be written as
Γ = f i (x, u)

∂
,
∂xi

and could also be considered as a map from smooth functions on P to smooth
functions on C with properties similar to a derivation (see [48], [46], [14], [15]
for a thorough discussion on calculus along maps). Finally, the coestate space
of the system is simply the contangent bundle πP : T ∗ P → P .
The first observation is that the Pontriguine’s necessary conditions for
extremal trajectories have a sound geometrical description as a presymplectic
system. In fact, the total space of the system will be M = T ∗ P ×P C.
The previous notation means that M is the bundle over P with fibre Mx
∗
at x ∈ P , given by Tx P × Cx . In M there is a canonical closed 2-form Ω
∗ ω , where ω is the canonical symplectic form on T ∗ P and
defined as Ω = pr1 0
0
pr1 : M → T ∗ P is the canonical projection. The coordinate expression for Ω
is simply,
Ω = dxi ∧ dpi .
The 2-form Ω is degenerate and its characteristic distribution is spanned by the
“vertical” vectors ∂/∂ua . The Pontriaguine’s hamiltonian function H : M →
R, eq. (3), defines a presymplectic system (M, Ω, H), whose dynamical vector
˜
fields Γ are the solutions of the dynamical equation
(13)

iΓ Ω = dH.
˜

The relevance of the presymplectic system eq. (13) comes from the following
result.
Theorem 1. The curve γ(t) is an extremal trajectory for the optimal
control system (1)-(2) if there exists a lifting γ (t) of γ(t) to M which is an
˜
integral curve of a vector field defined by the dynamical equation (13).
See [21] for a proof. Similar ideas have been discussed in [42].
2.3. The PCA and singular perturbation It is important to notice that there are vector fields satisfying the dynamical equation (13) only at
points m of M such that Z(H) = 0 for all Z in the kernel of Ωm . Thus, the
dynamical equation (13) will have in principle solutions only on the submanifold M1 defined by the equations (as we notice already by means of a different

136

´

analysis),
(14)

ϕ(1) =
a

∂H
= 0.
∂ua

˜
Now on M1 there is at least a vector field Γ satisfying (13). However, such
vector fields are not, in general, tangent to M1 . A simple computation shows
that if the system is regular, i.e., det Wab = 0 on M1 , then there is a unique
˜
dynamical vector field Γ parallel to M1 . Such vector field will be obtained
explicitely by using the optimal feedback condition (6). At regular points
the manifold M1 projects (locally) symplectomorphically onto T ∗ P , and the
˜
vector field Γ projects to a Hamiltonian vector field on T ∗ P whose integral
curves satisfy hamilton’s equations for the hamiltonian function
h(x, p) = H(x, p, ψ(x, p)).
If the system is singular however, there are points of M1 where the vector
fields solution to (13) will not be tangent to M1 . Thus, their integral curves
will exit the submanifold where the extremal trajectories must lie. Those
points must be removed from the analysis leaving in this way a subset M2 ⊂
M1 where some vector field solution to (13) is tangent to M1 . In fact, such
subset is defined by the conditions
˜ a
Γ(ϕ(1) ) = 0,

on M1 ,

˜
for all Γ solution of (13). We will assume that the subset M2 is a submanifold
(2)
of M1 . We shall denote the functions defining locally M2 on M1 by ϕb and
we will call them secondary constraints. Clearly the argument goes on, and
we will obtain a family of submanifolds defined recursively as follows.
˜ (k)
Mk+1 = { x ∈ Mk | Γ(ϕb )(x) = 0 },

k ≥ 1.

Eventually the recursion will stop and Mr = Mr+1 = Mr+2 = · · · for certain
finite r (we will say then that the system is noetherian). In this way we will
obtain a stable submanifold
M∞ =

Mk
k≥0

where we can try to solve the dynamical equation (13) and the integral curves
of the corresponding vector fields will be the extremal singular arcs of the


137

singular optimal control problem. Next subsection will show an example of
such procedure.
Finally, we can complete our picture by singularly perturbing the vector
˜
fields Γ and construct cheap optimal controls of the singular problem. Such
singular perturbation is done iteratively step by step in the PCA (see [20] for
details of the construction).
2.4. A simple example: The cusp singular optimal control problem We shall consider the optimal control problem on P = R with control
space the 1 dimensional trivial bundle C = P × R, given by the equation
x=u
˙
and Lagrangian density
L(x, u) = −xu2 − u4 .
Pontriaguine’s Hamiltonian is given by
H(x, p, u) = pu + xu2 + u4
and the primary constraint ϕ(1) = ∂H/∂u is given by
ϕ(1) (x, p, u) = p + 2xu + 4u3 .
The manifold M1 = { ϕ(1) = 0 } is the well-known cusp singularity [4]. On
˜
M1 the vector fields Γ solution of eq. (13) will have the form
∂
∂
∂
˜
Γc = u
− u2
+c ,
∂x
∂p
∂u
˜
where c is an undetermined constant. The vector field Γc will be tangent to
(1) = 0 on M . Thus, this condition implies that
˜
M1 iff Γc (ϕ
1
u2 + cW = 0
on M1 , with W = ∂ 2 H/∂u2 = 2x + 12u2 . Thus, if we are at a regular point
W = 0 and then c = −u2 /2(x + 6u2 ). However there are singular points given
precisely by the equation W = 0. The singular locus Σ1 of the equation will
be defined then as
Σ1 = { (x, p, u) | ϕ(1) (x, p, u) = 0, W (x, p, u) = 0 }.

138

´

The set Σ1 corresponds clearly to the points where the surface M1 does not
project transversaly into the plane (x, p).
The points such that W = 0 do not have vectors solution of (13) tangent
to M1 unless u = 0. If this is the case, then there are vectors tangent to M1
given by (0, 0, c), c arbitrary. Notice that the singular subset Σ2 = { (x, p, u) |
ϕ(1) = 0, W = 0, u = 0 } consists simply of the point (0, 0, 0) and because
the tangent space to M1 at (0, 0, 0) is the plane (x, u), the vector (0, 0, c) is
tangent to M1 . In this case the final constraint submanifold is given by
M∞ = { (x, p, u) | ϕ(1) = 0, W = 0 } ∪ { (0, 0, 0) }.
3. Implicit Optimal Control
3.1. An elementary example of implicit optimal control An
important class of problems in control theory have the form of the following
simple example, (x, y) ∈ Rn+m :

(15)

A·

x
˙
y
˙

=B·

x
y

+C ·u

where the matrix A is of rank n. If the matrix A is diagonalizable, in a new set
of coordinates, denoted again by x, y, the previous equation takes the form,
(16)

x = ax + by + cu
˙

(17)

0 = cx + dy + du

Such differential-algebraic equations have deserved to be studied by themselves. See for instance [12], [7] for a discussion of nonlinear differentialalgebraic equations. One of the natural ways to address such equations is by
introducing a small parameter and consider the singularly perturbed system,
(18)

x = ax + by + cu
˙

(19)

y = cx + dy + du.
˙

If the matrix A were not diagonalizable, the resulting system will be even more
involved and it will not be so obvious how to introduce small parameters
to render the problem a purely differential one. It could happen moreover,
that the matrix A is of non constant coefficients and we will have to consider
the structure of the bifurcation set where the rank of the matrix jumps. This


139

situation will correspond to consider the first order approximation to a general
implicit problem in control theory given by an implicit differential equation
of the form
Φ(x, y, x, y, u) = 0.
˙ ˙
We shall address here not the most general implicit problem but the first
order approximation to the general situation, i.e., systems of the form,
(20)

A(x) · x = f (x, u),
˙

where x denotes the state variables on the state space P , the vector function f
denotes generalized “forces” and live in an auxiliary “force” space F (that can
be thought as a vector bundle over F → P ) and A is a matrix depending on
the state variables x (that can be thought as a vector bundle map A : T P → F
or as an F –valued 1–form on P ). Thus the implicit optimal control problem
will consists in finding the curves γ(t) = (x(t), u(t)) such that x(t0 ) = x0 ,
x(T ) = xT ,
(21)

A(x(t)) · x(t) = f (x(t), u(t))
˙

minimizing the objective functional eq. (2).
The strategy to solve this problem will be to extend Pontriaguine’s maximum principle to this setting, and afterwards to proceed geometrically as in
the non-implicit (or normal) optimal control problem discussed in Section 2.
Theorem 2. [22] A C 1 -piecewise smooth curve γ(t) = (x(t), u(t)) is a
solution of the cuasilinear system
Aa (x, u)xi = f a (x, u)
˙
i
and an extremal of the objective functional
t1

S(x, u) =

L(x, u)dt
t0

with fixed endpoints x(t0 ) = x0 ; x(T ) = xT if there exists a lifting γ =
˜
(x(t), u(t), ζ(t)) of the curve γ to the dual bundle F ∗ → P of the force bundle
F , satisfying the set of implicit differential equations:
1. Aa (x, u)xi =
˙
i

∂H
∂ξa

´

140

∂Aa
∂H
i
˙
2. Aa (x, u)ξa + (FA (x, u))a xj ξa + ξa u˙α α = − i
˙
i
ij
∂u
∂x
3. ξa

∂Aa i
∂H
i
x =
˙
α
∂u
∂uα

where H denotes the Hamiltonian function on M = F ∗ ×P C given by,
(22)

H(x, u, ζ) = ζa f a (x, u) − L(x, u).

3.2. Presymplectic description of extremal trajectories It is
a simple observation that the bundle F ∗ is equipped with a canonical 1-form
θA defined by
θA = ζa Aa (x)dxi
i
that can be thought also as the pull-back of the canonical Liouville 1-form
on T ∗ P by the dual A∗ of the map A. Then, the 2-form dθA induces a presymplectic 2-form Ω on M . Hence as in the case of normal optimal control,
we have a presymplectic system (M, Ω, H) defined on M that provides a dynamical equation formally identical to (13). That such presymplectic system
is useful to solve the implicit optimal control problem is the content of the
following theorem.
Theorem 3. [22] Obtaining smooth extremal trajectories for an implicit
optimal control system (21)-(2) is equivalent to find the integral curves of the
∗
presymplectic system defined by the presymplectic 2–form Ω = pr1 dθA on
M and the Hamiltonian function H (22), in the sense that solutions of the
˜
equations (1)-(3) of Thm. 2, are integral curves of vector fields Γ satisfying
(13).
The presymplectic system defines implicit equations whose consistency and
uniqueness must be carefully analyzed. We will reproduce the P CA algorithm
described in the section above restricting eventually the space M to a subspace
M∞ where the solutions of (13) live. Such analysis follows closely the work in
[29], [30], [31].
4. Optimal control with symmetries and integrability
An important aspect of the theory of mechanical systems is the existence
of symmetries and in consequence, because of Noether’s theorem, of constants
of the motion that are very useful in its integration. Such ideas have played a


141

crucial role in the development of celestial mechanics in the past and, mainly
through its quantum mechanical counterpart, in modern physics in general.
In the theory of control such ideas do not have played such a significant role
so far partly because the strong interest in various issues that in a sense a
contrary to integrability (controllability for instance). Nevertherless it was
already noticed by R. Brockett in [8] the importance of control on Lie groups
and homogeneous spaces. In fact, the systems on Lie groups have a close
resemblance with the situation discussed by S. Lie [43] and that give rise to
the so called nonlinear superposition principles which represents a generalized
form of integrability as discussed for instance in [35].
Apart from these observations, little has been worked out around the problem of integrability on control theory. L.E. Faybusovich has discussed more
recently the issue of explicit solvability for control problems [25], [26] and,
even more recently, A. Bloch and P. Crouch offer in [6] a presentation of optimal control systems on coadjoint orbits related with reduction problems and
integrability. The issue is gaining interest and some recent developments can
be found in [18].
In these notes we address the problem of integrability of optimal control
systems, from the point of view of its symmetries, searching for a generalized
Noether’s theorem that will associate any one-parameter group of symmetries
with a conserved quantity.
4.1. Noether’s theorem Let us consider a regular optimal control
problem (C, Γ, L) as stated in Section 2. A symmetry transformation will
be a map Φ of state and control variables that will transform optimal trajectories and controls into themselves [23]. More precisely, Φ will be of the
form
Φ(x, u) = (ϕ(x), ϕ(x)u),
˜
i.e., Φ is a bundle map of the control vector bundle C → P . It is not difficult
to check that Φ maps optimal trajectories into optimal trajectories if it verifies
ϕ∗ ◦ Γ = Γ ◦ Φ,
and
˙
Φ∗ L = L + h,
˙
˜
˜
where h = Γ(h) and Γ denotes the unique vector field extending Γ to the
submanifold M0 and satisfying the dynamical equation (13). Hence if we have

´

142

a one-parameter group of symmetries Φs , then its infinitesimal generator X,
X ◦ Φt =
has the form
X = ζ i (x)
Moreover,

d
Φt ,
dt

∂
∂
b
+ ua ζa (x) b .
i
∂x
∂u

˙
Φ∗ L = L + ht ,
t

and

d ˙
ht |t=0 .
dt
We shall denote the gauge function in the r.h.s. of the previous equation by
µ.
d ˙
µ = ht |t=0 .
dt
Then, we have the following result:
X(L) =

Theorem 4. [23] Let X be the infinitesimal generator of a one-parameter
group of symmetries of the optimal control problem (C, Γ, L). Then the function
J(x, p, u) = ζ i (x)pi − µ(x),

(23)

is a constant of the motion, i.e., if (x(t), p(t), u(t)) is an optimal trajectory,
then
(24)

J(x(t), p(t), u(t)) = J(x(t0 ), p(t0 ), u(t0 )),

∀t ∈ [t0 , T ].

4.2. Particular cases The first application of the previous result,
Thm. 4, comes when particularizing it to control systems of the form
xi = ui .
˙
Then, on M0 , we have

∂L
,
∂ui
and the conserved quantity associated to any one-parameter group of symmetries, eq. (24), becomes
pi =

J(x, u) =

∂L i
ζ (x) − µ(x),
∂ui


143

which is the statement of the well-known Noether’s theorem for Lagrangian
mechanics.
If we have vertical symmetries, i.e., symmetries X such that ζ i = 0, then
the conserved quantity is 0. This is not strange as vertical symmetries affect
only to control variables, thus the system is insensitive to these changes.
5. Linearization and normal forms
5.1. Linearization and feedback linearization In the previous
sections we have used as simple examples linear systems of the form eq. (10)
Thus, if we are considering a nonlinear problem
(25)

x = f (x, u)
˙

we can face the question of whether there is a new local system of coordinates
y i in state space, and new controls v a such that (25) transforms in the simpler
linear form (10). Before determining the type of transformations that we will
allow in state and control space, let us consider first the simpler problem of
the linearization of a vector field X in the neighborhood of a critical point.
Let us then consider then a vector field X = f i (x)∂/∂xi such that X(m) =
0. Then, we would like to know if the vector field is (locally) linear, that is,
if there exists a set of coordinates y i such that the vector field takes the form
X = Aj y j
i

∂
.
∂y i

We shall consider in what follows local coordinates such that x(m) = 0.
The problem of the local linearizability of a vector field was already considered
by H. Poincar´. If we expand f around 0, we obtain
e
x = A · x + h.o.t.,
˙
with

∂f j
(0).
∂xi
We shall denote by λ1 , . . . , λn the eigenvalues of A. We will say that they are
resonant if there is collection of nonnegative integers m1 , . . . , mn , k mk ≥ 2,
such that
λs = m, λ .
Aj =
i

The number |m| =

k

mk will be called the order of the resonance.

144

´

Theorem 5. [3] If the eigenvalues of the matrix A are non resonant, then
there is a formal change of coordinates y = y(x) such that in the coordinates
y, the vector field X becomes linear.
Similar local results can be obtained for families of vector fields closing a
semisimple Lie algebra with a common critical point [33], [32].
A geometrical characterization of (local and global) linearizability can be
used to address the same question. Such aspects are discussed in depth in
[13] (see [27] for some elementary applications). Global linearizability can be
characterized by means of a global dilation vector field ∆. Then, X will be
linearizable iff [∆, X] = 0. Locally, linearizability can be characterized by a
sode. The precise way to do this will be discussed elsewhere.
Now we arrive to the point about the linearizability of a general control
system. If we think that the system (25) defines a vector field along the projection map of an affine bundle π : C → P , then the coordinate transformations
we consider will be of the form
(26)

y i = ϕi (x)

(27)

v a = ϕ(x)a ub + ρa (x).
˜ b

This is exactly what has been called feedback linearization as was presented
by R. Brockett [11]. If the system (25) is affine in the controls
x = f (x) + g(x)u,
˙
such problem has been solved in different ways [40], [36], [51]. See also [41]
for a review of the problem and references therein.
5.2. Normal forms Similar questions to those addressed in the previous paragraphs, section 5.1, can be posed for optimal control systems. Now,
the linear model for such systems will be an LQ systems as described in Section 2, eqs. (10-11). Hence we would like to know if a regular non-linear
optimal control system can be brought locally to an LQ form.
More precisely, let us suppose that we perform a change of variables in the
state, coestate and control variables, Φ(y, ξ, v) = (x, ζ, u), of the particular
form
(28)
(29)
(30)

xi = ϕi (y)
∂y j
ζi =
ξj
∂xi
ua = ϕa (y)v b + ρa (y)
˜b


145

and that the resulting system is of LQ type. That is,
1
1
K(y, ξ, v) = H ◦ Φ(y, ξ, v) = ξ t · A · y + ξ t · B · v − y t · P · y − y t · Q · v − ξ t · R · ξ.
2
2
Then, the linear equations of motion will have the form
(31)
(32)

y = A·y+B·v
˙
˙
ξ = −At · ξ + P · y + Qt · v

(33)

0 = B t · ξ − Qt · y − R · v

Thus, the optimal feedback condition becomes,
v = C · ξ + D · y,
with C = R−1 B, D = −R−1 Q, and we will obtain, after substituting it, the
linear hamiltonian equations,
(34)

y = (A + BD) · y + BC · ξ
˙
˙
ξ = −(At − Qt C) · ξ + (P + Qt D) · y.

(35)

Thus, the system has to be feedback linearizable and simultaneously, the
Lagrangian L has to become quadratic. Notice that the point (y, v) = (0, 0)
must be a critical point for the vector ﬁeld f (x, u). We will assume that
Φ(0) = 0. Now, in order for L to be reducible to its quadratic part, we need,
L(0, 0) = 0,

∂L
(0, 0) = 0,
∂x

∂L
(0, 0) = 0.
∂u

Thus,
f (x, u) = A·x+B ·u+h.o.t.,

1
1
L(x, u) = xt ·P ·x+xt ·Q·u+ ut ·R·u+h.o.t.,
2
2

with
A=

∂f
∂f
∂2L
(0, 0), B =
(0, 0),
P =
(0, 0),
∂x
∂u
∂x∂x
∂2L
∂2L
Q=
(0, 0),
R=
(0, 0).
∂x∂u
∂u∂u

The ﬁrst important observation is that in general this problem has no
solution. The reason for this is that because of the regularity condition, the

146

´

optimal feedback control ua = ψ(x, ζ), where ζi denotes the coestate variables,
allows to reduce the problem to a Hamiltonian system with equations
(36)
(37)

xi = f i (x, ψ(x, ζ))
˙
∂f j
∂L
˙
ζi = −ζj i (x, ψ(x, ζ)) + i (x, ψ(x, ζ)).
∂x
∂x

The Hamiltonian function of such system becomes,
h(x, ζ) = ζi f i (x, ψ(x, ζ)) − L(x, ψ(x, ζ)).
The conclusion of the argument is based on the following (non-completely
trivial) observation. The transformed hamiltonian function k = k(y, ξ) of
hamiltonian h = h(x, ζ) corresponding to a canonical map (ϕ∗ )−1 obtained
by lifting a map ϕ on state variables, coincides with the Hamiltonian function
obtained by first transforming the function H by means of the transformation
Φ on the total space M with coordinates (x, ζ, u), and then restricting to a
hamiltonian function by using the transformed feedback optimal condition.
Thus, if a regular optimal control problem could be reduced to an LQ system
by a change of coordinates of the form (36), then, there will exits a canonical
change of coordinates such that the hamiltonian h will be linearized, but it
is well-known that a Hamiltonian vector field cannot be linearized in general
(not even formally). H. Poincar´ noticed the impossibility of removing fourth
e
order terms in the Taylor expansion by formal changes of coordinates.
Thus, we have arrived to the first negative result concerning the linearizability of optimal control problems. However, regarding the stability of
equilibrium points, it is important to understand the local structure of such
points. We know that the stability of equilibrium points for Hamiltonian systems is neutral. It is possible to reduce formally h to normal forms, called
Poincar´-Birkhoff normal forms, in the neighborhood of an stable point. In
e
fact, at an stable point, the quadratic part of the hamiltonian can be written
in the form
2
h(q, p) =
ωk (p2 + qk ),
k
where (q, p) are canonical coordinates and the frequences ωk are, generically,
all different. Then, if the frequences ωk have no resonances of order less
than N , there exists a formal change of canonical coordinates that brings the
Hamiltonian h to the form
ak τ k ,

k(Q, P ) =
|k|≤N


147

up to terms of order N +1, where k denotes the multiindex k = (k1 , k2 , . . . , kr ),
k
1
k
|k| = k1 + k2 + · · · + kr , τi = 2 (Pi2 + Q2 ) and τ k = τ1 1 · · · τr r [2].
i
We can summarize the previous discussion as follows:
Theorem 6. Let xi = f i (x, u) denote a regular optimal control system
˙
with objective functional defined by a Lagrangian function L = L(x, u). We
shall assume that f i (0, 0) = 0, L(0, 0) = 0 and dL(0, 0) = 0. Let h(x, ζ) =
H(x, ζ, u(x, ζ)) be the hamiltonian function defined by the optimal feedback
functions ua = ψ a (x, ζ). The linear approximation of the hamiltonian vector
field defined by the hamiltonian h, is defined by a quadratic hamiltonian
function h0 ,
1
1
h0 (x, ζ) = ζLζ + ζM x + xN x.
2
2
with
∂f ∂f
∂ 2 L ∂ψ 2 ∂L ∂ 2 ψ
+
+
∂u ∂u ∂u∂u ∂ζ
∂u ∂ζ 2
∂f
∂2f
∂2L
∂2L
∂ 2 L ∂ψ ∂L ∂ 2 ψ
∂f
− W −1
ζ
−
+
+
+
(39) M =
∂x
∂u
∂x∂u ∂x∂u
∂x∂x ∂x∂u ∂x
∂u ∂x2
∂2L
∂ 2 L ∂ψ ∂L ∂ 2 ψ
N=
+
+
(40)
∂x∂x ∂x∂u ∂x
∂u ∂x2
(38)

L = −W −1

Then, if the linear approximation is estable, i.e., all eigenvalues of the hamiltonian matrix
L M
T =J
,
Mt N
are imaginary and different, and do not have resonances of order smaller than
N , the hamiltonian h can be brought to a Birkhoff normal form of degree N
by a formal canonical change of canonical coordinates.
Several remarks are in order here. First, we must notice that the canonical
change of coordinates that we will eventually find as a result of the previous
theorem, will mix the state and coestate variables, but will not destroy the
structure of state/control variables.
The linear approximation of the optimal control problem given by the
hamiltonian h0 , can be brought to a normal form by linear symplectic changes
of coordinates. The family of normal forms were obtained by Williamson [54]
and can also be consulted in Appendix 6 of [2]. It will be important to
understand the existence of approximate feedback linearizations for nonstable
configurations.

148

´
6. Outlook

From the previous discussion, it is clear that geometrical ideas are providing a powerful insight in old and new problems in the theory of control and
optimal control. In this paper we have reviewed some aspects of classical problems in optimal control theory such as the construction of singular arcs. We
have also addressed recent problems like the integrability of optimal control
problems and we have posed new problems like the construction of normal
forms for optimal control. There are many other problems which are waiting
for a geometrical perspective, like the inverse problem that could be analyzed
along the lines suggested in [34].
As a consequence of all this we hope to have convinced the reader that
geometrical ideas, combined with new analytical and numerical tools will lead
to new applications and solutions to important problems in this domain of
applied mathematics. In particular the use of numerical methods with a geometrical background will prove to be particularly stimulating. In this sense
the adaptation of symplectic integrators to the optimal control setting [50],
or Veselov discretization methods [53], [45] will certainly provide an outburst
of activity in the field. Such ideas will be discussed in forthcoming papers.
Acknowledgements
Partial financial support for this work was provided by MCYT project
BFM-2001/2272.

References
[1] Agrachev, A.A. GamKrelidze, R.V., Simplectic geometry for optimal
[2]
[3]
[4]
[5]
[6]
[7]

control, in “Nonlinear Controllability and Optimal Control” Ed. H.J. Sussmann, Marcel & Dekker Inc., N.Y., 1990, 263 – 267.
Arnol’d, V.I., “Les M´thodes Math´matiques de la Mćanique Classique”,
e
e
e
´
Editions Mir., Moscou, 1976.
Arnol’d, V.I., “Geometrical Methods in the Theory of Ordinary Differential
Equations”, Springer-Verlag, Berlin, 1983.
´
Arnol’d, V.I., Varchenko, A., Goussein-Zade, S., “Singularit´s des
e
´
Applications Diff´rentiables”, Vol. 1. Editions Mir, Moscou, 1986.
e
Bell, D.J., Jacobson, D.H., “Singular Optimal Control Problems”, Academic Press, London, 1975.
Bloch, A.M., Crouch, P.E., Optimal control, optimization and analytical
mechanics, in “Mathematical Control Theory”, J. Baillieul and J.C. Willems,
eds., Springer-Verlag, 1999, 265 – 321.
Brenan, K.E., Campbell, S.L., Petzold, L.R., “Numerical Solutions of Initial Value Problems in Differential-Algebraic Equations”, NorthHolland, Amsterdam, 1989.


149

[8] Brockett, R.W., Systems theory on group manifolds and coset spaces, SIAM
J. Control Optim., 10 (1970), 265 – 284.

[9] Brockett, R.W., Lie theory and control systems defined on spheres, SIAM
J. Appl. Math., 25 (1973), 213 – 225.

[10] Brockett, R.W., Lie algebras and Lie groups in control theory, in “Geome-

trical Methods in System Theory”, D.Q. Mayne, R. W. Brockett eds., Reidel,
Drodrechet, 1973, 43 – 82.
[11] Brockett, R.W., Feedback invariants for nonlinear systems, in “Proceedings
of the International Congress of Mathematicians”, Helsinki, 1978, 1357 – 1368.

S.L. . . (1980); .
[12] Campbell, S.L., “Singular Systems of Differential Equations”, Pitman, San
[13]
[14]
[15]
[16]
[17]
[18]
[19]
[20]

[21]
[22]
[23]
[24]
[25]
[26]

Francisco, 1980; ibid. “Singular Systems of Differential Equations II”, Pitman,
San Francisco, 1982.
˜
Carinena, J.F., Ibort, A., Marmo, G., Morandi, G., Dynamical
systems, (2000) in preparation.
˜
´
Carinena, J.F., Lopez, C., Mart´
ınez E., A new approach to the
converse of Noether’s theorem, J. Phys. A: Math. & Gen., 22 (1989), 4777 –
4486.
˜
´
Carinena, J.F., Lopez, C., Mart´
ınez E., Sections along a map applied
to higher-order Lagrangian Mechanics, Acta Apli. Mathematicae, 25 (1991),
127 – 151.
Clements, D.J., Anderson, B.D.O., “Singular Optimal Control: The
Linear-Quadratic Problem”, Springer-Verlag, Berlin, 1978.
´
Cortes, J., Mart´
ınez, S., Optimal control for nonholonomic systems with
symmetry, in “Proc. IEEE Conf. Decision & Control”, Sidney, Australia
(2000), 5216 – 18.
´
Cortes Monforte, J., “Geometric, Control and Numerical Aspects of Nonholonomic Systems”, Ph. Thesis, to be published in Lect. Notes in Maths.,
2003.
Crouch, P.E.. van der Schaft, A.J., “Variational and Hamiltonian
Control Systems”, Lect. Notes in Control and Inf. Theory, Vol. 101, SpringerVerlag, 1987.
´
Delgado-Tellez, M., Ibort, A., “On the Geometry and Topology of
Singular Optimal Control Problems and their Solutions”, Proceedings of the
fourth international conference on dynamical systems and differential equations, Wilmington 2002.
´
Delgado-Tellez, M., Ibort, A., Singular Optimal Control: A geometric
approach, in preparation (2002).
´
Delgado-Tellez, M., Ibort, A., Implicit optimal control: a maximum
principle, in preparation (2002).
´
Delgado-Tellez, M., Ibort, A., Optimal control with symmetries and
integability, in preparation (2002).
Dirac, P.A.M., Generalized hamiltonian dynamics, 392 – ???; Canad. J.
Math., 2 (1950), 129 – 148.
Faybusovich, L.E., Explicitly solvable optimal control problems, Int. J.
Control, 48 (1988), 235 – 250.
Faybusovich, L.E., Hamiltonian structure of dynamical systems which solve
linear programming problems, Physica D, 53 (1991), 217 – 232.

´

150

[27] Gaeta, G., Marmo, G., Non-perturbative linearization of dynamical systems, J. Phys. A: Math. & Gen., 29 (1996), 5035 – 5048.
M.J., Nester, J.M., Presymplectic manifolds, geometric
constraint theory and the Dirac-Bergmann theory of constraints, Ann. Inst.
Henri Poincar´, Ser. A, 30 (1979).
e
Gracia, X., Pons, J.M., Constrained systems: a unified approach, Int. J.
Theor. Phys., 30 (1991), 511 – 516.
Gracia, X., Pons, J.M., A generalized framework for constrained systems,
Diff. Geom. Appl., 2 (1992), 223 – 247.
˜
´
Gracia, X., Munoz-Lecanda, M., Roman-Roy, N., Singular systems:
their origins, general features and non-numerical solutions, in “IFAC conference system structure and control. Ed. M. Gugliemi-IFAC”, (1995), 37 – 42.
Guillemin, V., Sternberg, S., Remarks on a paper by Hermann, Trans.
Am. Math. Soc., 130 (1968), 110 – 114.
Hermann, R., The formal linearization of a semisimple Lie algebra of vector
fields, Trans. Am. Math. Soc., 130 (1968), 105 – 110.
Ibort, L.A., Mar´
ın-Solano, J., On the inverse problem of the calculus
of variations for a class of coupled dynamical systems, Inverse Problems, 7
(1991), 1 – 13.
Ibort, A., Marmo, G., A new look at completely integrable systems and
double Lie groups, Contemporary Maths., 219 (1998), 154 – 172.
Jakubczyk, B., Respondek, W., On linearization of control systems,
Bull. Acad. Pol., Sci. Ser. Sci. Math., 28 (1980), 517 – 522.
Jurdjevic, V., “Geometric Control Theory”, Cambridge Univ. Press, Cambridge, 1996.
Kokotovic, P.V., Applications of singular perturbation techniques to
control problems, SIAM Review, 26 (1984), 501 – 550.
Kokotovic, P.V., Khalil, H.K., O’Reilly, J., “Singular Perturbation
Methods in Control”, Academic Press, London, 1986.
Korobov, V.I., A general approach to the solution of the problem of synthesizing bounded controls in a control problem, Math. USSR Sb., 37 (1979),
535 – 539.
Krener, A.J., Feedback linearization, in “Mathematical Control Theory”, J.
Baillieul and J.C. Willems eds., Springer Verlag, (1999), 66 – 98.
´
Lopez, C., Mart´
ınez, E., Subfinslerian metrics associated to an optimal
control problem, J. Control Optim., 39 (2000), 798 – 811.
Lie, S., “Vorlesungen uber Differenialgleichungen mit bekannten infinitesima¨
len Transformationen. Bearbit und herausgegeben von Dr. G. Scheffers”, Liepzig, B.G. Teubner, 1981.
R.E. O’Malley, R.E., Jr, Singular perturbations and optimal control, in
Lect. Notes in Maths., Vol. 680, Springer-Verlag, Berlin (1978), 170 – 218.
Marsden, J.E., Patrick, G.W., Shkoller, S., “Multisymplectic Geometry, Variational Integrators, and Nonlinear PDEs”, math.DG/9807080,
1998.
Mart´
ınez, E., Geometr´ de ecuaciones diferenciales aplicada a la Mecńica,
ıa
a
Publ. Sem. Garc´ Galdeano, Sec 2, 36, 1989.
ıa

[28] Gotay,
[29]
[30]
[31]
[32]
[33]
[34]
[35]
[36]
[37]
[38]
[39]
[40]
[41]
[42]
[43]
[44]
[45]
[46]


151

[47] Mart´
ınez D´
ıaz, S., “Geometric Methods in Nonlinear Control Theory with

Applications to Dynamic Robotic Systems”, Ph. D. dissertation, Univ. Carlos
III de Madrid, 2002.
[48] Pidello, G., Tulczyjew, W., Derivations of diﬀerential forms and jet
bundles, Ann. Math. Pure et Appl., 147 (1987), 249 – 265.

[49] Pontryaguin, L.S., Boltyanskii, V.G., GamKrelidze, R.V., Mishchenko, E.R., “The Mathematical Theory of Optimal Processes”, Interscience, N.Y., 1962.

[50] Sanz-Serna, J., Calvo, M., “Numerical Hamiltonian Problems”, Chapman and Hall, London 1994.

[51] Su, R., On the linear equivalents of control systems, Syst. Control Lett., 2
(1982), 48 – 52.

[52] Sussmann, H.J., Geometry and Optimal Control, in “Mathematical Control
Theory”, J. Baillieul and J.C. Willems eds., Springer Verlag, (1999), 140 –
198.
[53] Veselov, A.P., Integrable Lagrangian correspondences and the factorization
of matrix polynomials, Funct. Anal. Pril., 25 (1991), 38 – 49.
[54] Williamson, J., On the algebraic problem concerning the normal forms of
linear dynamical systems, Amer. J. Maths., 58 (1936), 141 – 163.

Geometrical control theory

More Related Content

What's hot (17)

Viewers also liked (16)

Similar to Geometrical control theory (20)

More from M Reza Rahmati (20)

Recently uploaded (20)

Geometrical control theory