Probabilistic Control of Switched Linear Systems with Chance Constraints

Control & System
Theory
Probabilistic Control of Switched Linear Systems
with Chance Constraints
Leonhard Asselborn Olaf Stursberg
Control and System Theory
University of Kassel (Germany)
l.asselborn@uni-kassel.de
stursberg@uni-kassel.de
European Control Conference 2016
www.control.eecs.uni-kassel.de 01.07.2016

Introduction: Motivation in Uncertain Environment Control & System
Theory
• dynamic system with different modes (1. fast straight motion, 2. slow
curved motion, etc.)
• stochastic disturbances (wind)
• stochastic initialization (GPS coordinates)
• input constraints: uk ∈ U
• state constraints (safety critical): xk ∈ Xk cannot be guaranteed for all
disturbances → state chance constraint: Pr(xk ∈ Xk) ≥ δx
• steer the system into a terminal region T with confidence δ
x0
p
p
xN
estimated
position of
an obstacle
T
Introduction Problem Definition Method Example Conclusion Appendix 2

Relevant Literature (Excerpt): Control & System
Theory
Switched Linear Systems
• Liberzon [2003], Sun [2006], Sun and Ge [2011]: stability conditions for arbitrary
switching, no input constraints
Chance Constraint Optimization
• Galaﬁore and Campi [2005], Blackmore et al. [2010], Asselborn et al. [2012],
Prandini et al. [2015]: scenario-based approach
• van Hessem et al. [2001,2002], Ma and Borelli [2012]: set-based approach
• Prekopa [1999], Blackmore and Ono [2009], Blackmore et al [2011], Vitus and
Tomlin [2011]: Boole’s inequality for separate handling of chance constraints
Reachability sets for stochastic hybrid systems
• Hu et al. [2000], Blom and Lygeros [2006], Cassandras and Lygeros [2006],
Kamgarpour et al. [2013], Abate et al. [2008]: control design
Previous own work
• Controller synthesis for nonlinear systems: [NOLCOS, 2013]
• Synthesis for stochastic discrete-time linear systems: [ROCOND, 2015]
• Synthesis for stochastic discrete-time switched linear systems: [ADHS, 2015]

Contribution Control & System
Theory
Controller synthesis based on probabilistc reachability computation for
discrete-time stochastic hybrid systems with state chance constraints
Solution Approach:
• forward propagation of ellipsoidal reachable sets Xδ
k with confidence δ
Xδ
0
Xk
Xδ
N
T
• offline controller synthesis by semi-definite programming (SDP) and tree
search
• approximation of chance constraints via Boole’s inequality

Sets, Distributions and Dynamic System (1) Control & System
Theory
Set representation:
• Ellipsoid: E := ε(q, Q) = x ∈ Rn
| (x − q)T
Q−1
(x − q) ≤ 1
with q ∈ Rn
, Q ∈ Rn×n
• Polytope: P := {x ∈ Rn
| Rx ≤ b} with R ∈ Rnp×n
, b ∈ Rnp
Multivariate Normal Distribution:
ξ ∼ N(µ, Ω)
The sum of two Gaussian variables ξ1 ∼ N(µ1, Ω1) and ξ2 ∼ N(µ2, Ω2) is
again a Gaussian variable:
ξ1 + ξ2 ∼ N(µ1 + µ2, Ω1 + Ω2)

Sets, Distributions and Dynamic System (2) Control & System
Theory
Probabilistic Switched Aﬃne Systems (PSAS):
xk+1 = Azk xk + Bzk uk + Gzk vk, k ∈ {0, 1, 2, . . .}
x0 ∼ N(q0, Q0), xk ∈ Rn
Xk = {xk | Rx,kxk ≤ bx,k}
vk ∼ N(0, Σ), vk ∈ Rn
, iid
uk ∈ U = {uk | Ruuk ≤ bu} ⊆ Rm
zk ∈ Z = {1, 2, . . . , nz}
Feasible system execution for k ∈ N0:
1. select zk ∈ Z to determine the tuple (Azk , Bzk , Gzk )
2. sample the disturbance vk ∼ N(0, Σ)
3. choose a suitable input uk ∈ U
4. evaluate the continuous dynamics to get the new state xk+1

Probabilistic Reachable Sets with Confidence δ Control & System
Theory
• Surfaces of equal density for ξ ∼ N(µ, Ω)
(Krzanowski and Marriott [1994]):
(ξ − µ)T
Ω−1
(ξ − µ) = c
with χ2
-distributed random variable and:
δ := Pr (ξ ∈ ε(µ, Ωc)) = Fχ2 (c, n)
cumulative distribution function
• Initial state confidence ellipsoid:
Xδ
0 := ε(q0, Q0c) with Pr(x0 ∈ Xδ
0 ) = δ contour of pdf
samples of ξ ∼ N (µ, Ω)
ξ2
ξ1
• Evolution of the state distribution:
qk+1 = Azk qk + Bzk uk, Qk+1 = Azk QkAT
zk
+ Gzk ΣGT
zk
Xδ
k+1 := ε(qk+1, Qk+1c
=:Qδ
k+1
)
Xδ
k+1 is the confidence ellipsoid for xk+1 with confidence δ.

Problem Definition Control & System
Theory
Problem
Given PSAS, determine a hybrid control law κk = (λk(xk), νk) for which it
holds that:
• uk = λk(xk) ∈ U, zk = νk ∈ Z and xk ∈ Xδ
k
∀ k ∈ {0, 1, . . . , N − 1}, N ∈ N
• the chance constraint Pr(xk ∈ Xk) ≥ δx is satisfied ∀k ∈ {0, . . . , N − 1}
• Xδ
N ⊆ T for a finite N ≤ Nmax.
Thus, any initial state x0 ∈ Xδ
0 has to be transferred into the terminal set T
with probability δ after N steps.

Main Idea Control & System
Theory
Solution procedure:
1. tree search algorithm (Asselborn and Stursberg [2015]) to determine the
sequence of discrete control inputs νk ∈ Z
2. solution of an SDP provides continuous control law:
uk = λ(xk) = −Kkxk + dk
with closed-loop dynamics:
xk+1 = Azk xk + Bzk uk + Gzk vk
= (Aνk − Bνk Kk)
:=Acl,k,νk
xk + Bνk dk + Gνk vk
Main Challenge:
• state chance constraint with multi-dimensional integral:
Pr(xk ∈ Xk) =
w∈Xk
N (qx,k, Qx,k) dw ≥ δx
• no closed form solution available
Idea: separate consideration of each half-space of Xk

Approximation of State Chance Constraints Control & System
Theory
• projection of the random variables onto each hyperplane
rx,k,1
rx,k,2
Rx,k
xk ≤
bx,k,1
bx,k,2
bx,k
yk,i := rx,k,i · xk ∼ N(qy,k,i, Qy,k,i)
qx,k
rx,k,1
rx,k,2
• yk,1 is a univariate random variable (Blackmore and Ono [2009])
qy,k,i = rx,k,iqx,k,i, Qy,k,i = rx,k,iQx,k,irT
x,k,i
• consideration of the violation of the chance constraint: Pr(yk,i > bx,k,i)
• with Boole’s inequality Pr nx
i=1 yk,i > bx,k,i ≤ nx
i=1 Pr(yk,i > bx,k,i)
the following Lemma holds:
Lemma
The chance constraint Pr(xk ∈ Xk) ≥ δx is satisﬁed, if it holds for
i ∈ {1, 2, . . . , nx} that:
ǫi := Pr(yk,i > bx,k,i),
nx
i=1
ǫi < 1 − δx

Approximation of State Chance Constraint (2) Control & System
Theory
Transformation into a standard normal distribution
yk,i ∼ N(qy,k,i, Qy,k,i), bx,k,i → yk,i ∼ N(0, 1),
bx,k,i − qy,k,i
Qy,k,i
Evaluation of the probability with the cumulative distribution function:
Pr(yk,i > bx,k,i) = 1 − cdf
bx,k,i − qy,k,i
Qy,k,i
with:
cdf(x) =
1
√
2π
x
−∞
e− z2
2 dz, no closed form solution available
In Soranzo [2014]:
cdf(x) ≈ fcdf (x) = 2−221−41x/10
with max error: 1.27e − 4 for x ≥ 0

Solution based on SDP (1) Control & System
Theory
Linearization of fcdf (x):
¯fcdf (qy,k,i, Qy,k,i, bx,k,i) :=fcdf (¯qy, ¯Qy, bx,k,i) + . . .
∇qy fcdf (qy,k,i − ¯qy) + ∇Qy fcdf (Qy,k,i − ¯Qy)
Approximation of Pr(xk ∈ Xk) ≥ δx by:
ǫi = 1 − ¯fcdf (qy,k,i, Qy,k,i, bx,k,i) ,
nx
i=1
ǫi < 1 − δx, ∀i = {1, 2, . . . nx}

Theory
• Convergence of the covariance matrix of N(qx,k+1, Qx,k+1):
Sk+1 ≥ Qx,k+1 = Acl,k,zk
Qx,kAT
cl,k,zk
+ GΣGT
or with Schur complement:


Sk+1 Acl,k,zk
Qx,k GΣ
Qx,kAT
cl,k,zk
Qx,k 0
ΣGT
0 Σ

 ≥ 0
• Convergence of the expected value qx,k
use of ﬂexible Lyapunov functions. (Lazar et al. [2009] ) suitable for
switched dynamics
V
k

Theory
Proposition
The input constraint uk = −Kkxk + dk ∈ U holds for Kk, dk and all xk ∈ Xδ
k
if:
(bu,i − ru,idk)In −ru,iKk(Qδ
x,k)− 1
2
−(Qδ
x,k)− 1
2 KT
k rT
u,i bu,i − ru,idk
≥ 0 ∀i = {1, . . . , nu}.
• ru,i and bu,i denote the i−th row of Ru and bu, respectively.
• Xδ
k is mapped into a unit ball by a suitable coordinate transformation
h(xk)
• the Euclidean norm ||h(xk)||2 ≤ 1 can be expressed as LMI, which results
in the above formulation
• complete proof can be found in Asselborn et al. [2013]

Determination of the Continuous Controller Control & System
Theory
Semideﬁnite program to be solved for chosen zk ∈ Z:
min
Sk+1,Kk,dk
Jk,zk
center point convergence:



qT
x,k+1,zk
Lqx,k+1,zk
− ρqT
x,kLqx,k ≤ αk
qx,k+1,zk
= (Azk − Bzk Kk)qx,k + Bzk dk
αk ≤ maxl∈{1,...,k} ωl
αk−l
ellipsoidal shape convergence:






Sk+1 Acl,k,zk
Qx,k Gzk Σ
Qx,kAT
cl,k,zk
Qx,k 0
ΣGT
zk
0 Σ


 ≥ 0
trace(Sk+1) ≤ trace(Qk)
input constraint:



(bu,i − ru,idk)In −ru,iKk(Qδ
x,k)− 1
2
−(Qδ
x,k)− 1
2 KT
k rT
u,i bu,i − ru,idk
≥ 0,
∀i = {1, . . . , nu}
state chance constraint:
ǫi = 1 − ¯fcdf (qy,k+1,i, Qy,k+1,i, bx,k+1,i) ,
nx
i=1 ǫi < 1 − δx, ∀i = {1, 2, . . . nx}

Determination of the Hybrid Controller Control & System
Theory
Probabilistic Ellipsoidal Control Algorithm (PECA)
Given: x0 ∼ N(qx,0, Qx,0), vk ∼ N(0, Σ), U = {uk | Ruuk ≤ bu},
Xk = {xk | Rx,kxk ≤ bx,k}, T, δ, δx, γmin, ω, ρ, α0
Deﬁne: k := 0, γ0 = γmin, O0 = ∅
while Xδ
k T and γk ≥ γmin do
(1) for i = 1, . . . , nz do
◮ compute Xδ
k, and solve the SDP problem for z = i and Xδ
k
◮ if solution exists do Ok := Ok ∪ {i} else Ok := Ok end
end
(2) if Ok = ∅ do choose the tuple (Kk, dk, νk) with best performance
else
if k = 0 do Termination without success
else k = k − 1, Ok := Ok {νk}, go to step (2) end
end
(3) compute (qx,k+1, Qx,k+1) with the selected controller (Kk, dk, νk)
(4) check state chance constraint:
if nx
j=1 1 − fcdf (qy,k+1,j, Qy,k+1,j, bx,k,j ) > 1 − δx do mark νk = i as
infeasible solution, go to step 2
(5) compute γk+1 = qk+1 − qk , k := k + 1, end while

Termination with success Control & System
Theory
Lemma
The control problem with a conﬁdence δ, an initialization x0 ∼ N(qx,0, Qx,0),
vk ∼ N(0, Σ), (λk(xk), νk) ∈ U × Z ∀ k, and Pr(xk ∈ Xk) ≥ δx is
successfully solved with selected parameters γmin, ω, ρ and α0, if PECA
terminates in N steps with Xδ
N ⊆ T.
Proof: by construction
If no success: adjust δ, γmin, ω, ρ, tree search strategy

Numerical Example (1) Control & System
Theory
Initial distribution and disturbance:
x0 ∼ N(qx,0, Qx,0) with qx,0 =
−10
50
, Qx,0 =
1 0
0 1
vk ∼ N(0, Σ) with Σ =
0.02 0.01
0.01 0.02
.
Discrete input set: Z = {1, 2, 3}
The continuous dynamic is speciﬁed by the following system matrices:
A1 =
1.35 −0.06
0.11 0.95
, A2 =
0.82 0.05
−0.14 1.10
, A3 =
0.86 0.05
−0.09 0.99
B1 =
0.58 −0.03
0.03 0.97
, B2 =
0.48 0.01
1.01 0.53
, B3 =
0.49 0.01
0.98 0.50
G1 = G2 = G3 =
0.1 0.05
0.08 0.2
Note that all three subsystems are chosen to have unstable state matrices.

Theory
Input constraints:
uk ∈ U =



u ∈ R2
|




1 0
0 1
−1 0
0 −1



 u ≤




3
3
3
3







,
Target set:
T = ε 0,
0.96 0.64
0.64 0.8
Cost function:
Jk = trace
Sk+1 0
0 0.8 qx,k+1
State chance constraint:
Pr(xk ∈ Xk) ≥ δx = 0.95, Xk = {x ∈ Rn
| Rx,kxk ≤ bx,k}
with:
Rx,k =
1 0 −1 0 − 2√
2
3
0 1 0 −1 − 1√
2
1
T
bx,k = 5 60 20.5 5 2.5 + 0.2k 40
T
Parameters: δ = 0.95, γmin = 0.01, α0 = 10−4
, ω = 0.8 and ρ = 0.98

Theory
x2
x1
Xδ
0
T
−20 −15 −10 −5 0 5
0
10
20
30
40
50
60
• Termination with N = 38 steps
in 80s using 2.8 Ghz Quad-Core
CPU
• Implementation with Matlab
7.12.0, YALMIP 3.0, SeDuMi
1.3, and ellipsoidal toolbox ET
(Kurzhanskiy and Varaiya
[2006]).
Xδ
27
Xδ
26
Xδ
25
x2
x1
−8 −7.5 −7 −6.5 −6
2
3
4
5
6
7

Conclusion and Outlook Control & System
Theory
Summary:
• Algorithm for control of PSAS with state chance constraints
• Oﬄine control law synthesis based on a combination of probabilistic
reachability analysis and tree search
• Explicit consideration of input constraints
• Tight approximation of state chance constraints
Future work:
• Consideration of piecewise-aﬃne systems with autonomous switching
• Explore measures to reduce the computational complexity

End Control & System
Theory
Thank you for your attention!

Attractivity and Stochastic Stability Control & System
Theory
Stability with confidence δ
The switched uncertain linear system is called attractive with confidence δ on a
bounded time domain [0, N], if for any initial condition x0 ∈ Xδ
0 and any
vk ∈ ε(0, Σc), finite parameters ¯q ∈ Rn
and ¯Q ∈ Rn×n
exist such that:
||qN || ≤ ||¯q||, ||QN || ≤ || ¯Q||.
The system is said stable with confidence δ on a bounded time domain [0, N] if
in addition
||qk+1|| ≤ ||qk||, ||Qk+1|| ≤ ||Qk||.
holds for any 0 ≤ k ≤ N − 1.
Interpretation:
• qk converges to a finite neighborhood of the origin
• covariance matrix Qk converges, such that the confidence ellipsoid is of
decreasing size over k (while rotation is still possible).

References Control & System
Theory
Liberzon, D.
Switching in Systems and Control
Birkhaeuser, 2003
Sun, Z.
Switched Linear Systems: Control and Design
Springer, 2006
Sun, Z. and Ge, S.S.
Stability theory of switched dynamical systems;
Springer. 2011
Blackmore, L. and Ono, M.
Convex chance constrained predictive control without sampling
Proceedings of the AIAA Guidance, Navigation and Control Conference, 2009
Blackmore, L., Ono, M., William, B.C.
Chance-constrained optimal path planning with obstacles
IEEE Transactions on Robotics, 2011
Vitus, M.P. and Tomlin, C.J.
Closed-loop belief space planning for linear gaussian systems
IEEE Conference on Robotics and Automation, 2011
Calaﬁore, G. and Campi, M.C.
Uncertain convex programs: randomized solutions and conﬁdence levels
Mathematical Programming, 2005
Blackmore, L., Ono, M., Bektassov, A., Williams, B.C.
A probabilistic particle-control approximation of chance-constrained stochastic predictive control
IEEE Transactions on Robotics, 2010

Theory
Asselborn, L., Jilg, M., and Stursberg, O.
Control of uncertain hybrid nonlinear systems using particle ﬁlters
In 4th IFAC Conf. on Analysis and Design of Hybrid Systems, 2012
Prandini, M., Garatti, S., Vignali, R.
Performance assessment and design of abstracted models for stochastic hybrid systems through a randomized approach
Autmatica, vol. 50, 2014
Hu, J., Lygeros, J., and Sastry, S.
Towards a theory of stochastic hybrid systems;
In Hybrid systems: Computation and Control, volume 1790, 160-173. Springer. 2000
Blom, H.A. and Lygeros, J.
Stochastic hybrid systems: theory and safety critical applications;
volume 337. Springer. 2006
Cassandras, C.G. and Lygeros, J.
Stochastic hybrid systems;
CRC Press.2006
Kamgarpour, M., Summers, S., and Lygeros, J.
Control design for speciﬁcations on stochastic hybrid systems;
In Hybrid systems: computation and control, 303-312. ACM. 2013
Abate, A., Prandini, M., Lygeros, J., and Sastry, S.
Probabilistic reachability and safety for controlled discrete time stochastic hybrid systems;
Automatica,44(11), 2724-2734. 2008
Asselborn, L., Gross, D., and Stursberg, O.
Control of uncertain nonlinear systems using ellipsoidal reachability calculus;
In 9th IFAC Symp. on Nonlinear Control Systems, 50-55. 2013

Theory
Asselborn, L. and Stursberg, O.
Probabilistic control of uncertain linear systems using stochastic reachability;
In 8th IFAC Symp. on Robust Control Design. 2015
Asselborn, L. and Stursberg, O.
Robust Control of Uncertain Switched Linear Systems based on Stochastic Reachability
In 5th IFAC Conf. on Analysis and Design of Hybrid Systems, 2015
Soranzo, E.E.A.
Very simple explicit invertible approximation of normal cumulative and normal quantile function
Applied Mathematical Science, 2014
Boyd, S.P., El Ghaoui, L., Feron, E., and Balakrishnan, V.
Linear matrix inequalities in system and control theory;
volume 15. SIAM. 1994
Lazar, M.
Flexible control lyapunov functions;
In American Control Conf., 102-107.2009
Kurzhanski, A. and Varaiya, P.:
Ellipsoidal Calculus for Estimation and Control;
Birkh¨auser, 1996.

Probabilistic Control of Switched Linear Systems with Chance Constraints

More Related Content

What's hot (20)

Similar to Probabilistic Control of Switched Linear Systems with Chance Constraints (20)

Recently uploaded (20)

Probabilistic Control of Switched Linear Systems with Chance Constraints