Krylov Subspace Methods in Model Order Reduction
2 likes1,585 views
The document discusses Krylov subspace methods for model order reduction. It begins by introducing the model reduction problem and defining moments and Markov parameters. It then defines Krylov subspaces, distinguishing between input and output Krylov subspaces. It shows that if the projection matrices are chosen as bases for the input/output Krylov subspaces, then the first few moments of the original and reduced systems will match. The document provides a proof that the zero-th and first moments will match and hints that higher moments can also be shown to match through properties of the Krylov subspace.
![Krylov
Methods in
MOR
Introduction
Moments &
Markov
Parameters
Krylov
Subspace
Moment
Matching
Issues with
Krylov
Methods
Orthogonalization
Stopping
Criteria
Remarks 2
For the second moment, the results of first moment can
be used and the fact that A−1E
2
A−1b can be written
as a linear combination of columns of matrix V
The proof can be continued by repeating these steps
(Induction) until mr(q−1) = m(q−1) i.e. q moments match.
The method discussed above was one-sided as we did not
go for computing W. Usually, W = V is chosen
In two-sided method W is chosen to be the basis of output
Krylov subspace, then 2q moments can be matched.
Proof is similar for matching Markov parameters and the
MIMO case [3,4].
13 / 20](https://image.slidesharecdn.com/krylov-methods-mor-160308115953/85/Krylov-Subspace-Methods-in-Model-Order-Reduction-27-320.jpg)
Krylov Subspace Methods in Model Order Reduction
- 1. Krylov Methods in MOR Introduction Moments & Markov Parameters Krylov Subspace Moment Matching Issues with Krylov Methods Orthogonalization Stopping Criteria Krylov Subspace Methods in Model Order Reduction Mohammad Umar Rehman PhD Candidate, EE Department, IIT Delhi umar.ee.iitd@gmail.com March 8, 2016 1 / 20
- 2. Krylov Methods in MOR Introduction Moments & Markov Parameters Krylov Subspace Moment Matching Issues with Krylov Methods Orthogonalization Stopping Criteria Outline 1 Introduction 2 Moments & Markov Parameters 3 Krylov Subspace 4 Moment Matching 5 Issues with Krylov Methods Orthogonalization Stopping Criteria 2 / 20
- 3. Krylov Methods in MOR Introduction Moments & Markov Parameters Krylov Subspace Moment Matching Issues with Krylov Methods Orthogonalization Stopping Criteria Introduction Model Reduction Problem Revisited Given a MIMO state space model E˙x = Ax + Bu y = Cx (1) where, E, A ∈ Rn×n, B ∈ Rn×m, C ∈ Rp×n, u ∈ Rm, y ∈ Rp, x ∈ Rn and n is sufficiently large. 3 / 20
- 4. Krylov Methods in MOR Introduction Moments & Markov Parameters Krylov Subspace Moment Matching Issues with Krylov Methods Orthogonalization Stopping Criteria Introduction Model Reduction Problem Revisited Given a MIMO state space model E˙x = Ax + Bu y = Cx (1) where, E, A ∈ Rn×n, B ∈ Rn×m, C ∈ Rp×n, u ∈ Rm, y ∈ Rp, x ∈ Rn and n is sufficiently large. It is required to obtain the following reduced order model Er ˙z = Arz + Bru y = Crz (2) where, Er, Ar ∈ Rq×q, Br ∈ Rq×m, Cr ∈ Rp×q, u ∈ Rm, y ∈ Rp, z ∈ Rq q << n Er = WTEV, Ar = WTAV, Br = WTB, Cr T = CTV 3 / 20
- 5. Krylov Methods in MOR Introduction Moments & Markov Parameters Krylov Subspace Moment Matching Issues with Krylov Methods Orthogonalization Stopping Criteria Introduction Model Reduction Problem Revisited Given a MIMO state space model E˙x = Ax + Bu y = Cx (1) where, E, A ∈ Rn×n, B ∈ Rn×m, C ∈ Rp×n, u ∈ Rm, y ∈ Rp, x ∈ Rn and n is sufficiently large. It is required to obtain the following reduced order model Er ˙z = Arz + Bru y = Crz (2) where, Er, Ar ∈ Rq×q, Br ∈ Rq×m, Cr ∈ Rp×q, u ∈ Rm, y ∈ Rp, z ∈ Rq q << n Er = WTEV, Ar = WTAV, Br = WTB, Cr T = CTV W, V are suitable Krylov subspace based projection matrices. 3 / 20
- 6. Krylov Methods in MOR Introduction Moments & Markov Parameters Krylov Subspace Moment Matching Issues with Krylov Methods Orthogonalization Stopping Criteria Moments and Markov Parameters The transfer function of the system in (1) is G(s) = C(sE − A)−1 B By assuming that A is nonsingular, the Taylor series of this transfer function around zero is: G(s) = −CA−1 B − C(A−1 E)A−1 Bs − · · · − C(A−1 E) i A−1 Bsi − · · · Coefficients of powers of s are known as moments i-th moment: M0 i = C(A−1 E)i A−1 B, i = 0, 1, . . . Also, M0 i = − 1 i diG(s) dsi s=0 is the value of subsequent derivatives of the transfer function G(s) at the point s = 0 4 / 20
- 7. Krylov Methods in MOR Introduction Moments & Markov Parameters Krylov Subspace Moment Matching Issues with Krylov Methods Orthogonalization Stopping Criteria Contd... A different series in terms of negative powers of s is obtained when expanded about s → ∞ G(s) = CE−1 Bs−1 +C(E−1 A)E−1 Bs−2 +· · ·+C(E−1 A)i E−1 Bs−i +· · · and the coefficients are known as Markov parameters. Model reduction is achieved by the means of matching of Moments (Markov parameters) Explicit moment matching becomes numerically cumbersome for large system order 5 / 20
- 8. Krylov Methods in MOR Introduction Moments & Markov Parameters Krylov Subspace Moment Matching Issues with Krylov Methods Orthogonalization Stopping Criteria Contd... A different series in terms of negative powers of s is obtained when expanded about s → ∞ G(s) = CE−1 Bs−1 +C(E−1 A)E−1 Bs−2 +· · ·+C(E−1 A)i E−1 Bs−i +· · · and the coefficients are known as Markov parameters. Model reduction is achieved by the means of matching of Moments (Markov parameters) Explicit moment matching becomes numerically cumbersome for large system order Go for implicit moment matching: Krylov subspace based Projection 5 / 20
- 9. Krylov Methods in MOR Introduction Moments & Markov Parameters Krylov Subspace Moment Matching Issues with Krylov Methods Orthogonalization Stopping Criteria Remarks 1 Asymptotic Waveform Evaluation (AWE) method is based on explicit moment matching Matching at s = 0 is known as Pad´e Approximation, and steady state response (low frequency) is reflected in the reduced order model. Matching at s → ∞ is known as Partial Realization, and the reduced order model is a good approximation of the HF response. Matching at s = s0, i. e. at some arbitrary value of s is known as Rational Interpolation and is aimed at approximating system response at specific frequency band of interest. 6 / 20
- 10. Krylov Methods in MOR Introduction Moments & Markov Parameters Krylov Subspace Moment Matching Issues with Krylov Methods Orthogonalization Stopping Criteria Defining Krylov Subspace Kq(A, b) = span{b, Ab, . . . , Aq−1 b}, A ∈ Rn×n and b ∈ Rn is called the starting vector. q is some given positive integer called index of the Krylov sequence. 7 / 20
- 11. Krylov Methods in MOR Introduction Moments & Markov Parameters Krylov Subspace Moment Matching Issues with Krylov Methods Orthogonalization Stopping Criteria Defining Krylov Subspace Kq(A, b) = span{b, Ab, . . . , Aq−1 b}, A ∈ Rn×n and b ∈ Rn is called the starting vector. q is some given positive integer called index of the Krylov sequence. The vectors b, Ab, . . . , constructing the subspace are called basic vectors. 7 / 20
- 12. Krylov Methods in MOR Introduction Moments & Markov Parameters Krylov Subspace Moment Matching Issues with Krylov Methods Orthogonalization Stopping Criteria Defining Krylov Subspace Kq(A, b) = span{b, Ab, . . . , Aq−1 b}, A ∈ Rn×n and b ∈ Rn is called the starting vector. q is some given positive integer called index of the Krylov sequence. The vectors b, Ab, . . . , constructing the subspace are called basic vectors. The Krylov subspace is also known as controllability subspace in control community. 7 / 20
- 13. Krylov Methods in MOR Introduction Moments & Markov Parameters Krylov Subspace Moment Matching Issues with Krylov Methods Orthogonalization Stopping Criteria Defining Krylov Subspace Kq(A, b) = span{b, Ab, . . . , Aq−1 b}, A ∈ Rn×n and b ∈ Rn is called the starting vector. q is some given positive integer called index of the Krylov sequence. The vectors b, Ab, . . . , constructing the subspace are called basic vectors. The Krylov subspace is also known as controllability subspace in control community. For each state space, there are two Krylov subspaces that are dual to each other, input Krylov subspace and output Krylov subspace. Either or both of subspaces can be used as projection matrices for model reduction. 7 / 20
- 14. Krylov Methods in MOR Introduction Moments & Markov Parameters Krylov Subspace Moment Matching Issues with Krylov Methods Orthogonalization Stopping Criteria Defining Krylov Subspace Kq(A, b) = span{b, Ab, . . . , Aq−1 b}, A ∈ Rn×n and b ∈ Rn is called the starting vector. q is some given positive integer called index of the Krylov sequence. The vectors b, Ab, . . . , constructing the subspace are called basic vectors. The Krylov subspace is also known as controllability subspace in control community. For each state space, there are two Krylov subspaces that are dual to each other, input Krylov subspace and output Krylov subspace. Either or both of subspaces can be used as projection matrices for model reduction. The respective method is then called One-Sided/Two-sided 7 / 20
- 15. Krylov Methods in MOR Introduction Moments & Markov Parameters Krylov Subspace Moment Matching Issues with Krylov Methods Orthogonalization Stopping Criteria Input and Output Krylov Subspaces Input Krylov subspace Kq1 A−1 E, A−1 b = span A−1 b, . . . , A−1 E q1−1 A−1 b Output Krylov Subspace Kq2 A−T ET , A−T c = span A−T c, . . . , A−T ET q2−1 A−T c 8 / 20
- 16. Krylov Methods in MOR Introduction Moments & Markov Parameters Krylov Subspace Moment Matching Issues with Krylov Methods Orthogonalization Stopping Criteria Input and Output Krylov Subspaces Input Krylov subspace Kq1 A−1 E, A−1 b = span A−1 b, . . . , A−1 E q1−1 A−1 b Output Krylov Subspace Kq2 A−T ET , A−T c = span A−T c, . . . , A−T ET q2−1 A−T c V is any basis of Input Krylov Subspace W is any basis of Output Krylov Subspace 8 / 20
- 17. Krylov Methods in MOR Introduction Moments & Markov Parameters Krylov Subspace Moment Matching Issues with Krylov Methods Orthogonalization Stopping Criteria Moment Matching: SISO Theorem If the matrix V used in (2), is a basis of Krylov subspace Kq1 A−1E, A−1b with rank q and matrix W is chosen such that the matrix Ar is nonsingular, then the first q moments (around zero) of the original and reduced order systems match. 9 / 20
- 18. Krylov Methods in MOR Introduction Moments & Markov Parameters Krylov Subspace Moment Matching Issues with Krylov Methods Orthogonalization Stopping Criteria Moment Matching: SISO Proof: The zero-th moment of the reduced system is mr0 = cT r A−1 r br = cT V WT AV −1 WT b The vector A−1b is in the Krylov subspace and it can be written as a linear combination of the columns of the matrix V, ∃r0 ∈ Rq : A−1 b = Vr0 Therefore, WT AV −1 WT b = WT AV −1 WT AA−1 b 10 / 20
- 19. Krylov Methods in MOR Introduction Moments & Markov Parameters Krylov Subspace Moment Matching Issues with Krylov Methods Orthogonalization Stopping Criteria Moment Matching: SISO Proof: The zero-th moment of the reduced system is mr0 = cT r A−1 r br = cT V WT AV −1 WT b The vector A−1b is in the Krylov subspace and it can be written as a linear combination of the columns of the matrix V, ∃r0 ∈ Rq : A−1 b = Vr0 Therefore, WT AV −1 WT b = WT AV −1 WT AA−1 b = WT AV −1 WT AVr0 10 / 20
- 20. Krylov Methods in MOR Introduction Moments & Markov Parameters Krylov Subspace Moment Matching Issues with Krylov Methods Orthogonalization Stopping Criteria Moment Matching: SISO Proof: The zero-th moment of the reduced system is mr0 = cT r A−1 r br = cT V WT AV −1 WT b The vector A−1b is in the Krylov subspace and it can be written as a linear combination of the columns of the matrix V, ∃r0 ∈ Rq : A−1 b = Vr0 Therefore, WT AV −1 WT b = WT AV −1 WT AA−1 b = WT AV −1 WT AVr0 = r0 10 / 20
- 21. Krylov Methods in MOR Introduction Moments & Markov Parameters Krylov Subspace Moment Matching Issues with Krylov Methods Orthogonalization Stopping Criteria Contd... With this, mr0 becomes mr0 = cT V WT AV −1 WT b = cT Vr0 = cT A−1 b = m0 For the next moment (first moment) consider the following result: WT AV −1 WT EV WT AV −1 WT b = WT AV −1 WT EVr0 = WT AV −1 WT EA−1 b and the fact that A−1EA−1b is also in the Krylov subspace can be written as A−1EA−1b = Vr1 11 / 20
- 22. Krylov Methods in MOR Introduction Moments & Markov Parameters Krylov Subspace Moment Matching Issues with Krylov Methods Orthogonalization Stopping Criteria Contd... Thus, WT AV −1 WT AA−1 EA−1 b = WT AV −1 WT AVr1 12 / 20
- 23. Krylov Methods in MOR Introduction Moments & Markov Parameters Krylov Subspace Moment Matching Issues with Krylov Methods Orthogonalization Stopping Criteria Contd... Thus, WT AV −1 WT AA−1 EA−1 b = WT AV −1 WT AVr1 = r1 12 / 20
- 24. Krylov Methods in MOR Introduction Moments & Markov Parameters Krylov Subspace Moment Matching Issues with Krylov Methods Orthogonalization Stopping Criteria Contd... Thus, WT AV −1 WT AA−1 EA−1 b = WT AV −1 WT AVr1 = r1 mr1 = cT V WT AV −1 WT EV WT AV −1 WT b 12 / 20
- 25. Krylov Methods in MOR Introduction Moments & Markov Parameters Krylov Subspace Moment Matching Issues with Krylov Methods Orthogonalization Stopping Criteria Contd... Thus, WT AV −1 WT AA−1 EA−1 b = WT AV −1 WT AVr1 = r1 mr1 = cT V WT AV −1 WT EV WT AV −1 WT b = cT Vr1 = cT A−1 EA−1 b 12 / 20
- 26. Krylov Methods in MOR Introduction Moments & Markov Parameters Krylov Subspace Moment Matching Issues with Krylov Methods Orthogonalization Stopping Criteria Contd... Thus, WT AV −1 WT AA−1 EA−1 b = WT AV −1 WT AVr1 = r1 mr1 = cT V WT AV −1 WT EV WT AV −1 WT b = cT Vr1 = cT A−1 EA−1 b = m1 12 / 20
- 27. Krylov Methods in MOR Introduction Moments & Markov Parameters Krylov Subspace Moment Matching Issues with Krylov Methods Orthogonalization Stopping Criteria Remarks 2 For the second moment, the results of first moment can be used and the fact that A−1E 2 A−1b can be written as a linear combination of columns of matrix V The proof can be continued by repeating these steps (Induction) until mr(q−1) = m(q−1) i.e. q moments match. The method discussed above was one-sided as we did not go for computing W. Usually, W = V is chosen In two-sided method W is chosen to be the basis of output Krylov subspace, then 2q moments can be matched. Proof is similar for matching Markov parameters and the MIMO case [3,4]. 13 / 20
- 28. Krylov Methods in MOR Introduction Moments & Markov Parameters Krylov Subspace Moment Matching Issues with Krylov Methods Orthogonalization Stopping Criteria Issues with Krylov Methods Major issues with Krylov Subspace based MOR Methods: 1 Orthogonalization 2 Stopping Point of Iterative Scheme 14 / 20
- 29. Krylov Methods in MOR Introduction Moments & Markov Parameters Krylov Subspace Moment Matching Issues with Krylov Methods Orthogonalization Stopping Criteria Orthogonalization The Krylov vectors are known to lose independence readily and tend to align towards the dominant vector, even for moderate values of n and q. 15 / 20
- 30. Krylov Methods in MOR Introduction Moments & Markov Parameters Krylov Subspace Moment Matching Issues with Krylov Methods Orthogonalization Stopping Criteria Orthogonalization The Krylov vectors are known to lose independence readily and tend to align towards the dominant vector, even for moderate values of n and q. The remedy lies in constructing an orthogonal basis using Gram-Schmidt process. 15 / 20
- 31. Krylov Methods in MOR Introduction Moments & Markov Parameters Krylov Subspace Moment Matching Issues with Krylov Methods Orthogonalization Stopping Criteria Orthogonalization The Krylov vectors are known to lose independence readily and tend to align towards the dominant vector, even for moderate values of n and q. The remedy lies in constructing an orthogonal basis using Gram-Schmidt process. However, classical GS is also known to be unstable 15 / 20
- 32. Krylov Methods in MOR Introduction Moments & Markov Parameters Krylov Subspace Moment Matching Issues with Krylov Methods Orthogonalization Stopping Criteria Orthogonalization The Krylov vectors are known to lose independence readily and tend to align towards the dominant vector, even for moderate values of n and q. The remedy lies in constructing an orthogonal basis using Gram-Schmidt process. However, classical GS is also known to be unstable Go for Modified GS methods — 15 / 20
- 33. Krylov Methods in MOR Introduction Moments & Markov Parameters Krylov Subspace Moment Matching Issues with Krylov Methods Orthogonalization Stopping Criteria Orthogonalization The Krylov vectors are known to lose independence readily and tend to align towards the dominant vector, even for moderate values of n and q. The remedy lies in constructing an orthogonal basis using Gram-Schmidt process. However, classical GS is also known to be unstable Go for Modified GS methods — Arnoldi (Unsymmetric A) 15 / 20
- 34. Krylov Methods in MOR Introduction Moments & Markov Parameters Krylov Subspace Moment Matching Issues with Krylov Methods Orthogonalization Stopping Criteria Orthogonalization The Krylov vectors are known to lose independence readily and tend to align towards the dominant vector, even for moderate values of n and q. The remedy lies in constructing an orthogonal basis using Gram-Schmidt process. However, classical GS is also known to be unstable Go for Modified GS methods — Arnoldi (Unsymmetric A) / Lanczos (Symmetric A) 15 / 20
- 35. Krylov Methods in MOR Introduction Moments & Markov Parameters Krylov Subspace Moment Matching Issues with Krylov Methods Orthogonalization Stopping Criteria Arnoldi Algorithm Using Modified Gram-Schmidt Orthogonalization Algorithm 1 Arnoldi 1: Start: Choose initial starting vector b, v = b b 2: Calculate the next vector: ˆvi = Avi−1 Orthogonalization: 3: for j = 1 to i − 1 do 4: h = ˆvT i vj, ˆvi = ˆvi − hvj Normalization: 5: i-th column of V is vi = ˆvi ˆvi stop if ˆvi = 0 6: end for Output of Arnoldi Iteration: 1 Orthonormal Projection matrix V, 2 Hessenberg Matrix H = VTAV 16 / 20
- 36. Krylov Methods in MOR Introduction Moments & Markov Parameters Krylov Subspace Moment Matching Issues with Krylov Methods Orthogonalization Stopping Criteria Stopping Criterion When to stop the iterative scheme? is another question to be answered This also decides the size of the ROM TU-M: Singular values based stopping criterion.1 IIT-D: A more efficient criterion based on a index known as CNRI2 is proposed.3 1 B. Salimbahrami and Lohmann, B., “Stopping Criterion in Order Reduction of Large Scale Systems Using Krylov Subspace Methods”, Proc. Appl. Math. Mech., 4: 682–683, 2004. 2 Coefficent of Numerical Rank Improvement 3 M. A. Bazaz, M. Nabi and S. Janardhanan. “A stopping criterion for Krylov-subspace based model order reduction techniques”. Proc. Int. Conf. Modelling, Identification & Control (ICMIC), pp. 921 - 925, 2012 17 / 20
- 37. Krylov Methods in MOR Introduction Moments & Markov Parameters Krylov Subspace Moment Matching Issues with Krylov Methods Orthogonalization Stopping Criteria Comparison with Balanced Truncation Parameter BT Krylov No. of Flops O n3 O q2n Numerical Reliability for large n No Yes Accuracy of the reduced system More Accurate Less Accurate Range of Applicability ∼ 103 ∼ 104 or higher Stability Preservation Yes No Iterative Method No Yes Reliable Stopping Criterion Yes No* 18 / 20
- 38. Krylov Methods in MOR Introduction Moments & Markov Parameters Krylov Subspace Moment Matching Issues with Krylov Methods Orthogonalization Stopping Criteria Selected References 1 A. C. Antoulas, Approximation of Large Scale Dynamical Systems, SIAM, 2005. 2 Behnam Salimbehrami, Structure Preserving Order Reduction of Large Scale Second Order Models, PhD Thesis, TU Munich, 2005. 3 Rudy Eid, Time Domain Model Reduction By Moment Matching, PhD Thesis, TU Munich, 2008. 4 B. Salimbehrami, Boris Lohmann, Krylov Subspace Methods in Linear Model Order Reduction: Introduction and Invariance Properties. Scientific Report, Univ. of Bremen, 2002. 5 Zhaojun Bai, Krylov subspace techniques for reduced-order modeling of large-scale dynamical systems, Applied Numerical Mathematics, 43 (2002), pp 9-44. 19 / 20
- 39. Krylov Methods in MOR Introduction Moments & Markov Parameters Krylov Subspace Moment Matching Issues with Krylov Methods Orthogonalization Stopping Criteria Thanks! 20 / 20