Hierarchical Matrices: Concept, Application and Eigenvalues

November 5, 2010

Hierarchical Matrices:
Concept, Applications and Eigenvalues

Thomas Mach

Max Planck Institute for Dynamics of Complex Technical Systems
Computational Methods in Systems and Control Theory

MAX PLANCK INSTITUTE
FOR DYNAMICS OF COMPLEX
TECHNICAL SYSTEMS
MAGDEBURG
HIERARCHICAL STRUCTURES

Max Planck Institute Magdeburg Thomas Mach, Hierarchical Matrices 1/28

Concept Application Eigenvalues

Householder Notation

Scalar

λ∈R




Vector
 
x1
 x2 
 
x =  x 3  ∈ Rn
 
.
..
xn




Matrix
 
a11 a12 a13 . . . a1n
a21 a22 a23 . . . a2n 
 
A = a31 a32 a33 . . . a3n  ∈ Rn×n

 . . . .. . 

 .. .
. .
. . . 
.
an1 an2 an3 . . . ann



Dense Matrix Format
Store n vectors (Fortran):
       
a11 a12 a13 a1n
a21  a22  a23  a2n 
       
a31  a32  a33 
      · · · a3n 
 
 .   .   .   . 
 .   .   . 
. . .  . 
.
an1 an2 an3 ann

n2 entries in the storage
Ax costs O(n2 ) flops
AB costs O(nδ ) flops, δ ≥ 2.376 usually δ = 3
A−1 costs O(n3 ) flops



Dense Matrix Format
Landau Symbol
   that    more than cn2 ﬂops
There is a constant c, so  Ax costs not
a11 a12 a13 a1n
a21  a22  a23  a2n 
       
      · · · a3n 
a31  a32  a33    Flops
 .   .   . 
.  .  .   . 
. 
 . . .  . 1 ﬂop = (αβ + γ → γ)
an1 an2 an3 ann

n2 entries in the storage



Dense Matrix Format
       
a11 a12 a13 a1n
a21  a22  a23  a2n 
       
a31  a32  a33 
      · · · a3n 
 
 .   .   .   . 
 .   .   . 
. . .  . 
.
an1 an2 an3 ann

n2 entries in the storage expensive!



Sparse Matrix Format

4 −1 0 −1 0 0 0 0 0
 
−1 4 −1 0 −1 0 0 0 0
 
 0 −1 4 0 0 −1 0 0 0
 
−1 0 0 4 −1 0 −1 0 0
 
 0 −1 0 −1 4 −1 0 −1 0 
 
0
 0 −1 0 −1 4 0 0 −1
0
 0 0 −1 0 0 4 −1 0 

0 0 0 0 −1 0 −1 4 −1
0 0 0 0 0 −1 0 −1 4




4 −1 0 −1 0 0 0 0 0
 
−1 4 −1 0 −1 0 0 0 0
 
 0 −1 4 0 0 −1 0 0 0
 
−1 0 0 4 −1 0 −1 0 0
 
 0 −1 0 −1 4 −1 0 −1 0 
 
0
 0 −1 0 −1 4 0 0 −1
0
 0 0 −1 0 0 4 −1 0 

0 0 0 0 −1 0 −1 4 −1
0 0 0 0 0 −1 0 −1 4

non-zero entries per row/column = c ≤ 5




4 −1 −1 −1
 
−1 −1 −1 4
 
−1 4 −1 −1
 
−1 −1 4 −1
 
4
 −1 −1 −1

−1 −1 −1 4
 
−1 −1 4 
 
−1 4 −1 
4 −1 −1

non-zero entries per row/column = c ≤ 5



Sparse Matrix Format (Matlab)

Store vector of tripel:
 
(1, 1, 4)
(2, 1, −1)
 
(4, 1, −1)
 
(1, 2, −1)
.
 
.
.

O(n) entries in the storage (if #non-zeros/column < c n)
Ax costs O(n) ﬂops
AB may be much denser ⇒ better use A(Bx)
A−1 not possible, only solve Ax = b



Is there something in between?

Yes, e.g.
Low rank matrices and Tucker tensor format
Toeplitz/Hankel matrices
Semiseparable matrices [Gantmacher, Krein 1937]
Cauchy matrices
Fast Multipole Method (FMM) [Greengard, Rokhlin ’87]
Mosaic-skeleton matrices [Tyrtyshnikov ’96]
...



Is there something in between?

Yes, e.g.
Low rank matrices and Tucker tensor format
Toeplitz/Hankel matrices
Semiseparable matrices [Gantmacher, Krein 1937]
Cauchy matrices
Fast Multipole Method (FMM) [Greengard, Rokhlin ’87]
Mosaic-skeleton matrices [Tyrtyshnikov ’96]
Hierarchical (H-) Matrices [Hackbusch ’98]



Integral Equation

Fredholm equation of ﬁrst kind:

Ω g (x, y )u(y )dy = f (x)



Integral Equation

unknown

Ω g (x, y )u(y )dy = f (x)



Integral Equation


Ω g (x, y )u(y )dy = f (x)

electrostatic problem:
g (x, y ) = 4π 1x−y
u(y ) charge density
f (x) electrostatic potential
inverse of elliptic diﬀerential operators
population dynamics [Koch, Hackbusch, Sundmacher]



H-Matrices [Hackbusch ’98]

Ω g (x, y )u(y )dy = f (x)

Ritz-Galerkin method:
u(y ) = i ui ψi (y )

⇒ g (x, y )ψi (y )dy ψj (x)dx ui = f (x)ψj (x)dx
i Ω Ω Ω
:=Mji =fj




discretization error ∼ Ω1g,(x, <)u(y )dy = f (x)
nκ 0
y κ<1


i Ω Ω Ω
:=Mji =fj




Ω g (x, y )u(y )dy = f (x)


i Ω Ω Ω
:=Mji =fj

Mu = f




Ω g (x, y )u(y )dy = f (x)


i Ω Ω Ω
:=Mji =fj
k x,s y ,t
x ∈ Ωs , y ∈ Ωt : g (x, y ) ≈ p=1 gp (x)gp (y )




Ω g (x, y )u(y )dy = f (x)


i Ω Ω Ω
:=Mji =fj
k x,s y ,t
x ∈ Ωs , y ∈ Ωt : g (x, y ) ≈ p=1 gp (x)gp (y )

Mji = g (x, y )ψj (x)ψi (y )dydx
Ω Ω
k
x y T
≈ gp (x)ψj (x)dx gp (y )ψi (y )dy = Aj· Bi·
p=1 Ω Ω



3 3

22 7 10 14 8 11 9 8 5
3 7 19 10 11 8

3 10 10 31 11 11 9 16 12 8 15 8 12 11 8
19 10 11 8

14

8
11
11
10

11
31 11
31 11
9
16 11
7
6

15
5
6
13 8

8
15 9

15 12 13
11 8 3 3
8 7 5

19
13
11 61 11 10 7
8

9 16 11 11 8 11 8

13 13
25
9

7
3

11 10
10

19 11 6 5
13
8
8

11 9 k
11 16
3

11 11 x,s y ,t
11 8
8 11 11 31 15 6 9 15
x ∈ Ωs , y ∈ Ωt : g (x, y ) ≈ gp (x)gp (y )
8 8 15 10 10 15 9

5 8 11 61 3

6 14 9 11 10 10 7 p=1
13
6 15 10 3 6 25 10 6 13 8

12 5 6 10 14 6
10 19

10
10

31 10 16 9
11

8
8

15 11 12

20
13 8
7 8 11 15 9 10 10
5 8 11 8 11 10 51 7 9
3
7
3
9 7
6 15 10 25 11 8

13 13
7 13

11 5 6 10 16 10
9

7
3

3
11
25
10
10

19 9
13
8
8

11
11
11 8 3 7

8 15 8 39 10 10 10 6 5
10
3

15 10 11
9 15 3
10 3
25 7 10
3
6
3
13 6
Ω Ω
13
7 10 7 22 7 10 11 9 8 5

7 12 10
10

6
3

3
7

10 10
19 10

31 11 9 16 12
11

8
8

15 8
12 k
x y T

20
13

gp (x)ψj (x)dx gp (y )ψi (y )dy = Aj· Bi·
8

10
8

9
11 8
15 15 11
34
10
10

25
13

7
10
11
6

13
5
6 11 ≈
Ω Ω
13 13 p=1
11 8 13 7

7 11 10 9 16 10 11 61 11 23
13 9 6 13 20 9 9 7
11 8 3 7

9 8 8 15 5 6 12 9 39 10 3 10 15 10

12 13
11 8

8 8 15 9 10 15 9

7 11 11 5 8 7
3

7
3

10 61 10 9 11

19
13 9 6 13 20 9 9 7

9 8
13
8
8
11 5 6 10 15 10 9 34 10 13

12 7 11 12 11 23 10
15
8
9
11
9
7
10
13 51




3 3

22 7 10 14 8 11 9 8 5
3 7 19 10 11 8

3 10 10 31 11 11 9 16 12 8 15 8 12 11 8
19 10 11 8

14

8
11
11
10

11
31 11
31 11
9
16 11
7
6

15
5
6
13 8

8
15 9

15 12 13
11 8 3 3
8 7 5

19
13
11 61 11 10 7
8

9 16 11 11 8 11 8

13 13
25
9

7
3

11 10
10

19 11 6 5
13
8
8

11 9 k
11 16
3

11 11 x,s y ,t
11 8
8 11 11 31 15 6 9 15
x ∈ Ωs , y ∈ Ωt : g (x, y ) ≈ gp (x)gp (y )
8 8 15 10 10 15 9

5 8 11 61 3

6 14 9 11 10 10 7 p=1
13
6 15 10 3 6 25 10 6 13 8

12 5 6 10 14 6
10 19

10
10

31 10 16 9
11

8
8

15 11 12

20
13 8
7 8 11 15 9 10 10
5 8 11 8 11 10 51 7 9
3
7
3
9 7
6 15 10 25 11 8

13 13
7 13

11 5 6 10 16 10
9

7
3

3
11
25
10
10

19 9
13
8
8

11
11
11 8 3 7

8 15 8 39 10 10 10 6 5
10
3

15 10 11
9 15 3
10 3
25 7 10
3
6
3
13 6
Ω Ω
13
7 10 7 22 7 10 11 9 8 5

7 12 10
10

6
3

3
7

10 10
19 10

31 11 9 16 12
11

8
8

15 8
12 k
x y T

20
13

gp (x)ψj (x)dx gp (y )ψi (y )dy = Aj· Bi·
8

10
8

9
11 8
15 15 11
34
10
10

25
13

7
10
11
6

13
5
6 11 ≈
Ω Ω
13 13 p=1
11 8 13 7

7 11 10 9 16 10 11 61 11 23
13 9 6 13 20 9 9 7
11 8 3 7

9 8 8 15 5 6 12 9 39 10 3 10 15 10

12 13
11 8

s 7 11 11
8

5
8

8
15 9

7
3

7
10 3

10 61 10
15
9
9

11

19
13 9 6 13 20 9 9 7

9 8
13
8
8
11 5 6 10 15 10 9 34 10 13

12 7 11 12 11 23 10
15
8
9
11
9
7
10
13 51

t




3 3

22 7 10 14 8 11 9 8 5
3 7 19 10 11 8

3 10 10 31 11 11 9 16 12 8 15 8 12 11 8
19 10 11 8

14

8
11
11
10

11
31 11
31 11
9
16 11
7
6

15
5
6
13 8

8
15 9

15 12 13
11 8 3 3
8 7 5

rank (Ms×t ) = 19 19
13
11 61 11 10 7
8

9 16 11 11 8 11 8

13 13
9 3 25 10 13 8

7 11 10 19 11 6 5 8 11 9

11 16
3

11 11
store: 10 7
8 11 11 31 15 6 9 15
11 8

8 8 15 10 10 15 9

5 8 11 61 3

6 14 9 11 10

13
6 15 10 6 25 10 6 13 8

|s|×19
3

A ∈11R12 ,
10 19 10 11 8

12 16
5 6 10 14 6 10 31 10 9 8 15

=
20 B 13 R|t|×19
13 8
7 8 11 15 9 10 10
5 8 11 8 11 10 51 7 9
3
7
3
9 7

∈
6 15 10 25 11 8

13
7 13

11 5 6 10 16 10
9

7
3

3
11
25
10
10

19 9
13
8
8

11
11
11 8 3 7

8 15 8 39 10 10 10 6 5
10
3

15 10 11
with Ms×t 12 AB T
=
9 15 3
10 3
25 7 10
3
6
3
13 6

13
7 10 7 22 7 10 11 9 8 5
10 3 7 19 10 11 8

7 12 10 6 3
11
10 10 12 31 9 16 8 15 8

⇒ only 19(|s| + |t|) instead of |s| |t| storage
20
13 8
8 11 8 34 10 13 10 6 5
10 9 15 15 11 10 25 7 11 13 6 11

13 61 13
11 8 13 7

7 11
9
13

8
11

8
9
8

15
6

5
10
13

6
⇒ reduces the required storage to 15%
9

12
16 10 11
20

9
9

39 10
9
3

3
7
7

10
11 23

15 10

12 13
11 8

s 7 11 11
8

5
8

8
15 9

7
3

7
10 3

10 61 10
15
9
9

11

19
13 9 6 13 20 9 9 7

9 8
13
8
8
11 5 6 10 15 10 9 34 10 13

12 7 11 12 11 23 10
15
8
9
11
9
7
10
13 51

t




3 3

22 7 10 14 8 11 9 8 5
3 7 19 10 11 8

3 10 10 31 11 11 9 16 12 8 15 8 12 11 8
19 10 11 8

14

8
11
11
10

11
31 11
31 11
9
16 11
7
6

15
5
6
13 8

8
15 9

15 12 13
11 8 3 3
8 7 5

19
13
11 61 11 10 7
8

9 16 11 11 8 11 8

13 13
9 3 25 10 13 8

7 11 10 19 11 6 5 8 11 9

11 16
3

8 11 11 31 11 15 6 9 15 11
11 8

10 15
1
if x ≈ y then x−y is large
8 8 15 10 9

5 8 11 61 3

6 14 9 11 10 10 7

13
6 15 10 3 6 25 10 6 13 8

12
10 19 10

10 16
11

128

11
small matrices on the diag-
5 6 10 14 6 10 31 9 8 15

20
13 8
7 8 11 15 9 10 10
5 8 11 8 11 10 51 7 9
3
7
3
9 7
6 15 10 25 11 8

13 13 onal have no low rank ap-
7 13

11 5 6 10 16 10
9

7
3

3
11
25
10
10

19 9
13
8
8

11
11
11 8 3 7

8 15 8 39 10 10 10 6 5
10
proximation
3

9 15 3
10 3
25 7 10
3
6
3
15 10 13 116

13
7 10 7 22 7 10 11 9 8 5

7 12 10
10 3 7 19 10

11 16 12 12 11 8

⇒ use dense matrix format
6 3 10 10 31 9 8 15 8

20
13 8
8 11 8 34 10 13 10 6 5
10 9 15 15 11 10 25 7 11 13 6 11

13 13
11 8 13 7

7 11 10 9 16 10 11 61 11 23
13 9 6 13 20 9 9 7
11 8 3 7

9 8 8 15 5 6 12 9 39 10 3 10 15 10

12 13
11 8

8 8 15 9 10 15 9

7 11 11 5 8 7
3

7
3

10 61 10 9 11

19
13 9 6 13 20 9 9 7

9 8
13
8
8
11 5 6 10 15 10 9 34 10 13

12 7 11 12 11 23 10
15
8
9
11
9
7
10
13 51




3 3

22 7 10 14 8 11 9 8 5
3 7 19 10 11 8

3 10 10 31 11 11 9 16 12 8 15 8 12 11 8
19 10 11 8

14 11 10 31 11 6 5
13 8 15 9
13
8
11
11
8
11 31 11
9
3
16 11
7
3
8
15 6
7 5
8 15 12
non-admissible block
19
13
11 61 11 10 7
8

9 16 11 11 8 11 8

13 13
9 3 25 10 13 8

7 11 10 19 11 6 5 8 11 9

11 16
3

8 11 11 31 11 15 6 9 15 11
11 8

8 8 15 10 10 15 9

5 8 11 61 3

6 14 9 11 10 10 7

13
6 15 10 3 6 25 10 6 13 8

12 5 6 10 14 6
10 19

10
10

31 10 16 9
11

8
8

15 11 12

20
13 8
7 8 11 15 9 10 10
5 8 11 8 11 10 51 7 9
3
7
3
9 7
6 15 10 25 11 8

13 13
7 13

11 5 6 10 16 10
9

7
3

3
11
25
10
10

19 9
13
8
8

11
11
11 8 3 7

8 15 8 39 10 10 10 6 5
10
3

9 15 3
10 3
25 7 10
3
6
3
15 10 13 6 11

13
7 10 7 22 7 10 11 9 8 5

7 12 10
10

6
3

3
7

10 10
19 10

31 11 9 16 12
11

8
8

15 8
12

20
13 8
8 11 8 34 10 13 10 6 5
10 9 15 15 11 10 25 7 11 13 6 11

13 13
11 8 13 7

7 11 10 9 16 10 11 61 11 23
13 9 6 13 20 9 9 7
11 8 3 7

9 8 8 15 5 6 12 9 39 10 10 15 10
admissible block
3

12 13
11 8

8 8 15 9 10 15 9

7 11 11 5 8 7
3

7
3

10 61 10 9 11

19
13 9 6 13 20 9 9 7

9 8
13
8
8
11 5 6 10 15 10 9 34 10 13

12 7 11 12 11 23 10
15
8
9
11
9
7
10
13 51




rank-k-matrix: Ms×t = AB T , A ∈ Rn×k , B ∈ Rm×k (k n, m)

hierarchical tree TI block H-tree TI × I
I = {1, 2, 3, 4, 5, 6, 7, 8}
12345678 12345678 12345678 12345678
1 1 1 1
2 2 2 2
{1, 2, 3, 4} {5, 6, 7, 8} 3 3 3 3
4 4 4 4
5 5 5 5
6 6 6 6
{1, 2} {3, 4} {5, 6} {7, 8} 7 7 7 7
8 8 8 8

{1}{2}{3}{4}{5}{6}{7}{8} dense matrices, rank-k-matrices




Hierarchical matrices
H(TI × I , k) = M ∈ RI × I rank (Ms×t ) ≤ k ∀s × t admissible
3 3

22 7 10 14 8 11 9 8 5
3 7 19 10 11 8

3 10 10 31 11 11 9 16 12 8 15 8 12 11 8
19 10 11 8

11 5
13 13
adaptive rank k(ε)
14 10 31 11 6 8 15 9

8 11 11 31 11 16 11 15 6 8 15 12
11 8 9 7 8 7 5
3 3

19
13
11 61 11 10 7
8

9 16 11 11 8 11 8

13 13
9 3 25 10 13 8

11 11 6 5 9

storage NSt,H (T , k) = O(n log n k(ε))
7 10 19 8 11

11 16
3

8 11 11 31 11 15 6 9 15 11
11 8

8 8 15 10 10 15 9

5 8 11 61 3

6 14 9 11 10 10 7

13
6 15 10 3 6 25 10 6 13 8

12
10 19 10

16
11 8

12
complexity of approximate arithmetic
5 6 10 14 6 10 31 10 9 8 15 11

20
13 8
7 8 11 15 9 10 10
5 8 11 8 11 10 51 7 9
3
7
3
9 7
6 15 10 25 11 8

13 13
7 13

11 5 6 10 16 10
9

7
3

3
11
25
10
10

19 9
13
8
8

11
11
11 8 3 7

O(n log n k(ε))
39 10 5

MH v
8 15 8 10 10 6

10
3

9 15 3
10 3
25 7 10
3
6
3
15 10 13 6 11

13
7 10 7 22 7 10 11 9 8 5

7 12 10
10

6
3 7

10 10
19 10

31 11 9 16 12
11

8
8

15 8
12
O(n log n k(ε)2 )
3

20 +H , −H
13 8
8 11 8 34 10 13 10 6 5
10 9 15 15 11 10 25 7 11 13 6 11

13 13
11 8 13 7

7 11 10 9 16 10 11 61 11 23

∗H , HLU(·), (·)−1 O(n (log n)2 k(ε)2 )
13 9 6 13 20 9 9 7
11 8 3 7

9 8 8 15 5 6 12 9 39 10 3 10 15 10

12 13 H
11 8

8 8 15 9 10 15 9

7 11 11 5 8 7
3

7
3

10 61 10 9 11

19
13 9 6 13 20 9 9 7

9 8
13
8
8
11 5 6 10 15 10 9 34 10 13

12 7 11 12 11 23 10
15
8
9
11
9
7
10
13 51




Hierarchical matrices
H(TI × I , k) = M ∈ RI × I rank (Ms×t ) ≤ k ∀s × t admissible
3 3

22 7 10 14 8 11 9 8 5
3 7 19 10 11 8

3 10 10 31 11 11 9 16 12 8 15 8 12 11 8
19 10 11 8

11 5
13 13
adaptive rank k(ε)
14 10 31 11 6 8 15 9

8 11 11 31 11 16 11 15 6 8 15 12
11 8 9 7 8 7 5
3 3

19
13
11 61 11 10 7
8

9 16 11 11 8 11 8

13 13
9 3 25 10 13 8

11 11 6 5 9

storage NSt,H (T , k) = O(n log n k(ε))
7 10 19 8 11

11 16
3

8 11 11 31 11 15 6 9 15 11
11 8

8 8 15 10 10 15 9

5 8 11 61 3

6 14 9 11 10 10 7

13
6 15 10 3 6 25 10 6 13 8

12
10 19 10

16
11 8

12
complexity of approximate arithmetic
5 6 10 14 6 10 31 10 9 8 15 11

20
13 8
7 8 11 15 9 10 10
5 8 11 8 11 10 51 7 9
3
7
3
9 7
6 15 10 25 11 8

13 13
7 13

11 5 6 10 16 10
9

7
3

3
11
25
10
10

19 9
13
8
8

11
11
11 8 3 7

O(n log n k(ε))
39 10 5

MH v
8 15 8 10 10 6

10
3

9 15 3
10 3
25 7 10
3
6
3
15 10 13 6 11

13
7 10 7 22 7 10 11 9 8 5

7 12 10
10

6
3 7

10 10
19 10

31 11 9 16 12
11

8
8

15 8
12
O(n log n k(ε)2 )
3

20 +H , −H
13 8
8 11 8 34 10 13 10 6 5
10 9 15 15 11 10 25 7 11 13 6 11

13 13
11 8 13 7

7 11 10 9 16 10 11 61 11 23

∗H , HLU(·), (·)−1 O(n (log n)2 k(ε)2 )
13 9 6 13 20 9 9 7
11 8 3 7

9 8 8 15 5 6 12 9 39 10 3 10 15 10

12 13 H
11 8

8 8 15 9 10 15 9

7 11 11 5 8 7
3

7
3

10 61 10 9 11

19
13 9 6 13 20 9 9 7

9 8
13
8
8
11 5 6 10 15 10 9 34 10 13

12 7 11 12 11 23 10
15
8
9
11
9
7
10
13 51

T
B1 T
B2 T
B1
A1 + A2 = A1 A2 T
B2



Special Case: H -Matrices

F1 B2 AT
2
B4 AT
4
T F3
A2 B2
B8 AT
8
F5 B6 A T
6
T
A4 B4
T F7
A6 B 6

F9 B10 AT
10
B12 AT
12
T
A10 B10 F11
T
A8 B8
F13 B14 AT
14
T
A12 B12
T
A14 B14 F15

Figure: Structure of an H3 (k)-matrix



Applications



Applications

Finite-Element-Method (FEM)
Inverse of FEM matrices
Integral equations
Solving matrix equations for model order reduction
AX + XAT + BB T = 0, A ∈ H, B ∈ Rn×k
Sign function iteration
[Baur ’07, Grasedyck ’03, et al.]
Boundary-Element-Method (BEM)



BEM for Electrostatic Computations
·103

4

2

0

−2

−4
2
−4
−2 0 ·103
0
2 −2
4
3
·10


·103
Electrostatic problem:

4 Ω g (x, y )u(y )dy = f (x)

1
2
g (x, y ) = 4π x−y
unknown: u(y ) charge density
0 given: f (x) electrostatic potential

−2

−4
2
−4
−2 0 ·103
0
2 −2
4
3
·10


·103

4

2

0

−2

−4
2
−4
−2 0 ·103
0
2 −2
4
3
·10


26 14

14 20 19 23 16

19
25 14
14 20 17 23 13 20 16 17 12
22 23 13 10 11
22 20
23 17 27 19
25 25
9
22 15

16 23
20 22

22 36
17
18
19 25 8

19 26 20 14 11 22 9 12
19 15 22 7
13 15 11 18 13 10 11 23 9 16 10 13 18 10 8 19 8 17 10 9 7 8 6 7 7 6 7 7

19 10 15 13 9
17 18 15 21 8 23 6

9
25 14
14 20 16 22 15
20 19 11 16 26 13 15 16 10 13 22
30 43 27 19 11
17
22 13 36 19 26 16 21
16 25 18 15 15 19 28
8

16 18 17 16 20 13
18

17
12

22
24 20 8
8 8 8 8 8 8 8 6

22 20
27 19 13 20 22 22 28 20 18 15
17 19 26 23 22 36 20 28 36 22 19 12 34 17 11 10 18 11
21 11

23 48 12 25 11 14
11 12 20
21 6 7 30
12 20
10

11
28 20

20 28 29
20 22 29
28
18
18

18

19
32
13
28
17
22
18
50 15 20
10
13 16

25
20 19
16 17
12 11 8 9 6 22
21

29 19 29 25 10

35 21
13 15 23 20 18

25
23

9
30
18 36

22
32
21
28

22
17

18
23

25
23
13
13

15

20
12

23
25 24 20
12

15 23
20
12
12

15
17 25
20 14

13 20
19
6

6
16

11
10
9
17 24

20
19
20 17

18 20
27
18
13
12
18
27
12
13 10 12 12 10 29
10 8

25 10
32 22 18
10 9
12
8
10
7
9 8 9
6
9
7
7
7
7
10
22 19 13
26 36 27 19 36 26

14
16 19
16 16 16 18 24 17 19 28 40 20 14 10 16 19 23 14 20 24 33 15 11

25 50
26 18

18 18 23 44 29

14
22
10

13
10 17
16
20
8

8
19
12

19
15
20
20 25 40
20
14
13
20
20
14

15
23
17
20
21
13
13

14
17 20

20 12 15
12 20

22
6

17
18
19 15
8
19 24
20
14

15
13
20
46
20
17

15
23
16
20
30
24
11
8 6
32 13 33 42 16
18
18

14
21 46 21 11
19 13 19 12 17 11 18 12
6 13 13 6 6 10 6 13 18 9 17 13 17 14 10 6 6 11 17 15 12 17 15 9
31 19

18
17 18
12
17 16 27 16 9 19
12
27 12

12
27 9 11 35 25 8 11 8 12 7
18 18 12 13 16 13

22 23 24 18 18 13 10 10

19
20 15

33 14 9 14 9 13 9 13 9
21 17 13

18
12

10 22 21 22
8

10 11 19 19
19 17
17 27 18 18 17
24
19
19

19
16 16 15 24 17 12 27 17 19 15 40 15 15 25 12
19 13 28 14 18 12 24 13

18
12 17 9 16 14 17 14 8 9 6 6 12 16 15 13 17 15 9 10
11 8 24 6 10 9 15 8 14 11 12
18
13
16
15 14
18 12
12 15
8 11 19 9 14 13
18
12
14
15 13
18
12
14
15 10
20 17 20 17

19 15 34 13 25 18 24 20 23 19 20 14 17 24 19 29 12 17 10
22 8 20 17 16
20

19
16

17 19
20

17
18

20 14 24 13 20 15 18 8 19
20 17

17 20 16 10 26 11 11 9 5 11 7 24 22 21 21 19 8 8 9 8
20 15

15 15 27 15 17

23 15
27
17 27 18 36 20 35 24 8 11 29 16 15

17
15

15

18
15 15

15 14

14
18

18
49 39 27 13 8 15 12 15 11 10 8 7 7 7
13 17 31 31 6
12 12 15 15

20 13
27 20
10 13 27 37

23
15 20 20

26
18 17

49 9 7
15 15 14 14

10 13 12 26 24
46 20 17 10
25 16
10 17 9 19 12 10 49 31 37
15
14
24
14
14

14
21
13
25 26 17 9
24 9 8 45 19 46 16 9 7 19 11 9 7 8 11 8

49
17
14 21 13 21
20 15

15 15 28

27
20 15
28 30
18 27 44 23 6 6 17 26 39 25 30
15 15

30 45 6 9 6 12 12 12 11 18 10 39 26 11
17 16

15 13
16 18 35 17 21

23 48 26 20 15 13
15 15 13 13

9
43 12 13 33 29 30
16 20
15
6
19 6 9
10 11 27
26
49 45
18
35
15

13

13
28
16
12

29
16

16 12
15

15
29
15

27
18
18

18
18 26

26 13
6 10 6 14 8 22 36
26 15
25
26
38
16
17 14 12 9 41 15
19

19 26 21
13

13
17 14
13

29 17
19
19 11 10 19 12 11 10
9 7 12 10 24 11 19 14 11 17 26 13 26 17 18 8 14 26 16 26 9 8 14 8 19 14 8 16 19 9
26 16 15 12
20 14

16 14 25 15 23 16 11 13 20 11
15

12
15

23
28

19
19

36
20 14
40 18

16
16

26 17 24 34 13
20

18
16

27 17 23 36 21
18 22

18 10 14 10
14 25 11 11

18
20

14
14

26 16 24 18 11 13

40
18 16
22
11

11
16

24 22
36 22
22 20

20 22

28
25
19
26 19

19 27 20 24 26 13 23
23 19
26
16 40 34 20 31 16 11

35 49 21 23
16 26 19 36 24 36
21 12

11 17 25 40
12

23
14 23
28
18
18

18
50 41 18 26 38 41
27

24
22

42 45 42 15 26
26 19 36 22 28 20

18 36 23 19 11
18 57 15
22 20
19 27 22 20 22 20 29 23
26 18

17 11 18 12
24 11 20 50
28

20
20

29 29
18 18
12

12

21
29
12

21 21

22
19 12
23 19 13
14 19 47 30 40 27
29

22 30
20

18
11 17 8 15 12 16 13 9 6 6 12 17 16 13 17 17 9 9 5 9 5 6
18 12 21 16 21 16

12 8 15 11 40 15 14 15 13 24 9 10 17 34 16 16
31
13
13

13
16 16
27 15
13 13
18 9 30 10 30 18 10 10
18 12 24 13 19 13 28 15
6 12 17 8 15 13 17 13 8 9 6 6 13 13 13 17 16 15 18 16 9 6 8 5 10 5 6
20 15 18 12 21 16 21 16

13
27 13 27 16
12 24 11 29 32 19
13 7
27
22 8
15

9 7
13 15

12
25 15

9
14 15

10
9 26 14 18 20
20

16
18

27
24 41 23
17

9
16 15

18
13
16

9
16

24
16 26

32
17

25
17

21 25

25
10
27
10
17 12 21 41 18 26 13 9 12 9 15 10 10 10
29 17
7 8 6 10 5 8 8 8 11 17 13 26 18 23 14 9 6 11 27 10
17 23 10 12 23 11 34 11 7 6 9 7 8 6
32 13 27 15
13 13 29 13 13 29
16 16 16

7 15 45 12 22 26 14 34 26 9 30 8 29 17 29 16
14
12

12
12

21 14

14
31 27
12

14
14
12

20 16
14
32 29 19 12 9 11 8 13 9
10 6
12 12 12 19 12 36 15 26
23 13 20 38 19 5 10 5 12 10
27
31
13
30
13

29
13

15 29
30
14
14

15

27
32
15
18
30
14

28
13

15

30
14
27
13

15
31 18 11 9 11 20 11 16 25 32 11 30 18 12 13 7

15 13 27 15 17 29
12 14 17 17

15 46 23 24 30 32 21 12 29
16

16
12 20 14

14
32 29
17

16
23 16
19
37 31 22 15
20 13 17 12
25 23

25 20 13
13 15 12 13

17 12 21 13
20 30

14 11
12 13 13 15

7 11 11 11 16 26 16
19 36
41 9
18 10 19 41 23 32 18 37 25

23
20

30
25 20

20 30
19 45 48 25 9 7 12 9 7

27
30 14 30 14

13 15 28 14 15 30

11 7
19
10
6

7
8

9

6
12 32 11
23
12

11
38

17 16 26
36 23 31
27

22
24

42
47
5

6
10

10
5

6 18 11
29

19 31 22 45 31
27
30
13
14

15
31 19 30
30
14
14

15

20
13

28
25
21
31
13

15 28
20
13

22
25
13

15
25 21

22 25

25 20
20 25
11

27 45
12
14 9
6

7
20 9 19 38 24
30

18 13 18 46
32 18

20
10

25 12
8
13 6

8
20 20

33 13 13
20 17

∈ R7416×7416
14
17 24 29

36 30 34 15 48 11 27 20 29
23 19

19 29
37 7 8 7 8 12 11 12 10 6

10 14 26 10 10
20 20

9 11 13 7 21 26 45 23 40 11 18 25 12 45 37 20
20 17

17

29
25 29
20 17

17 25
22 21
11 8 11 7 8 9 22 28 13 10 25 14 38 12 8
30 22 21
21
12
12

12
29 29 20 21 10 16 13
18 21 29 29
18

18
18

26 20 28
18

21
17

26
30 20 15 20 11 11
17

18 20
16

24

11 19 29 24 26 9 11 14 29
20
20
28
22 20
20 22

22
22

36
20
23
12
18
17
19
18
18
50 13 19
27 19
19 26 18
13
23 18 34
15
47 12 22 10
18 21 12 15 23 20 18

10 25
33
10 9 18 12 9 9
10 21 23 25 27 16 27 13 18 12 11
20 12
26 23 20 13 9 20 6

29
17 12 15 17 20

29 9 26
23 13
23 24 17

34
22 8
18 17

9
13 15 19

8 10 7 10 9 8 7 6 7 30 19 18 18 25 23
19
13
13

15
21 24 14 20 23 19 18 20 18 12 27 13 10 16 10 6

19 13 21
18 18

14
18 28 23 46 30
15 42 19 41 27 21 23
14

13
13

21
40 20 17 11 17 11 24 20 24 30 48 19 6

32 11 16 18
21
11 15 14
20 11 20
50 13
16
20
24
14
40
20
36
19
14
19

28 14 20

20 12
25 16
26
16
16

18 16 8

15 22
20 14
34
20
36
20
17
26
24
24
16
27
16
19
19 10 15
18 14 19 8 15 19 13 20 17 20 12 20 15 13 15 8
13 20 16 20 11 17 8
22 20 11
27 19 20 22 22 24 11 15
20

21 35
18
19 26 22 36 16 22 16

24 23 10 12 13
26 14

26 25 24 16 18
40
9
14 20

11 18 15
11 11 22 18
25 14
14 20
13 8 22 10 11 11
19 15 22 7

10 19 15 9 13
10 11 13 23 9 23 11 16 13 15 18 16 13 15 21 17 11 8 8 19 9 17 10 7 18 10 6 8 6 7 7 7 8 7 6

9 16
29
17

16
18
20 18
20
18
17
20 19
17

11
24
8
26
16

16
15
16

18

15
8

20 13
40
8
17

11
8

8
36
19

23
12
8
19

28

15
15
23
15
25 14
14

16
20
12

15

16

26
8

8

20 17
17 20
20
19 26
43
11 16 10
20
30
17

21
8
21
22 13

13 23
6
19 11 26 8

25 22 21 21 19 9 8 13 24 15 34 20
24
17
18
20 20 22 14 20 22 19 20 19
24 17
17 20 17 10 29 12 11 15 17 8 11 7 9 5
20 15

15 15 28
20 15
28 30
10 18 39 26 11 7 27 18 46 24 26 17 30
15 15

30 45 39 25 6 6 12 12 12 11 9 6 6
18 16
17 21 16 17 35

20
13 13 15 15 13 12

9 12 26 21
17

14 17
13 29
12 20 8 11 13 12 30 30
34 20 16 9 11 10 45 35
15

13

12
27
18
13

29
18

18 13
15

15
29
15

28
16
16

16

23
27
15
49 26 18 19 6
9
6 36 22 38 17 25 10 6
14 6 8
15 15 27 18 15

20 15
27
8 10 7 18 27 16 29 39 27 15 15 17 15

49 35 23 8 12 15 15 11 13 8

27
36 20 17 18
15
17
15
18 14
14 15
11

31 31 13 17

12
18 17

15 14 13 20
24 15 21 14

25 47 24
15 15 14 14

37 17
23 14
21 14 21 13

26
17 10
19
46 7 49
14 14 13 14

7 9 19
9 11 8 12 13 48 26 24
17 20
10
43 10 12 25 49 31
37
15

15
15

27 20
17 26
15
25 16
12

13
12

27 10
10
19 9 19 45 46 16 24 9
9 8
20 20

26 17 26 13 26 14 26 16
10 8 19 14 16 19 9 12 10 24 11 10 7 11 18 26 13 26 19 14 15 26 16 26 17 18 8

23 49
19 16 22 20 15 15 22 18 8 6

31 19

16 27 16 18 30 17 35 25 19
12
27 12

12
27 9 11 9 11 8 12 7 8
13 16 18 18 12 13

19
12 13 21 17

24
17 19
25
22 11 10 19
23 21 23 12 27
27
17 19 19 18 15 16

40
15
12

9 14 9 15 9 13 33
19 19 10 21 22 17 18 17 16 15 15
6 6 6 10
17
6 13 13
8 17 6 6 10
10 10
19
9 17
18 24
18 13

13 19
12

14
18

17

12
14

19 15
17 11

11 17
15

15
20

9
17

12
12

18
15 12 15 12 15 12 15 12

9 11 19 11 12 16 18 14 12 18 14 14 18 13 14 18 10 13
18 10
10 9 15 11 23 8
17 12 17 14 16 12 17 13

8 8 6
6 6 8 9 14 9 16 14 19 14 14 28 15 12 18 15 9 13 24
14 13

13 21 23 27 23

42 31 22 17 10 38 26 42
18 18

16 23 18 26 40 24 40

8 19 11 15 27 25 38 22 40
28

19
19

36
18
21
18
25
18

16
16

26 16 25 19 34 31
23

24
19

36
44 41 18
7 6 9 29 22 28 29 28 20

15
18

13 11
18 40 31
22 21

7 7 19 20 30 31 21 29 21
12

11
11

21 20 28 20 12 14

8 8 10 8 26 11 18 24 28 22 25
18 16
28

20
20

28
36 22

22
22 20
20 22

28
25
19 15 15
26 19

19 27 17 19 47 23 37 23 21 16
24

21 35 49
17 16 26 19 36 13 22 18 27 13 18

25 40 23 36 21
15 13 26 16

10 16 15 22 16
20 14

14 25
34 20 13 11
26 19 36 22 23 16

9 13 10 18 10 14 20 22 20
19 27 22 20 22 17 23 25
20 14

25 12 17 40 23

16
17

23 19
14 25 19
20

14
14

26
23 26 24 14
30 14 30 13

14 15 29 14 15 28
11 11 19 11 32 12 30 18 6 12 36 12 19 26 15 38 19 29
30
13
13

15
31 27
30
14
13

15
31 32 18 10 5 12 5
16 16 16

11 13 10 19 18
21 12 20 14
16

9
14

23
16
14
12 12

14
29 12

14
12

14
28
9 12 16 7 8 11 9 13 7 12 22 15 45 14 26 26 34 31 29
32
13
13

13
29 19 29
27
13
15
13
32 30 9 29 17 8 10
30 14 30 14

13 15 27 14 15 30
6 6 30 32 18 8 10 12 38 27 24 28
30
13
14

15
29 30
30
14
14

15
31 31 19 12 11 10 5 5
20 13
28 25 22

20 23
13 15

18
20 13
28 21 25

13 18
13 15

7 9 9 18 46 12 25 7 11 17 26 16
23 40 31 31 19 31 25
22
21
25
25 20
20 25
22 45 45 27 10 6 6 11
20 12 23 17

27
16 14 12 14 17 17 16

16 14
29 16 19 29
15 46 22 37 15
7

13 32 31 30 32 21 12
27 13 27 15
23 24 28 15 13 29 15 17

14 21 9 19
9 13 20 11 14 12 19 11
25 47 36 23
21
13

17
12
13

15

12

13
17
12

20
13
12

13

13

15
30 20

23 25

25 48
7 7 12 11 16 42 18 32 19 45 37 20 41

34
7 7 16 26
19 36
30 23 30 20
18 9 10 23
20 25 20 25
20 20

13 13 14 57 33
20 17

17 25 29
7 8 11 8 14 10 25 12 38 40 23 12 45 25 20 29
20 17

17 25
37 11
20 20

10 11 8 7 10 9 10 6 13 11 8 12 10 12 9 11 7 14 44 31 37 21 26 18 11 27 15 48 37 20
23 19

19

29
29 29
20 17

17 24

31 13
26 27 15
34
15 14 17 16
9 10 13 24 11 40 15 15 10 30 10 10 30 18 13 13

16
13 13

18 16
11 34
12 18 17 21 16 21

8 6 6 9 17 8 15
17

11 17
11

13
16

12
12

18 5 9 5 6 9 16
17 12

12 19 17 9
17

13
13

19
9
27 13 27 16
25 15
29 32 20
16 16 26 16
24
14 15
12
15

11
13

13 15 15 21 18 17 16

13
15 20 12 18 16 21 16 21

10 14 26 18 26
32 17

9 13 9 13 10 15 10 21 23 20
24 12 17 21 25
12 21 41
13 26 41
17
7 7 9 10 9 16 27 18 9 24 9 27
9 18 8 17 12 17 13 17 13
25 25
18 15

6 6 6 8 9 8 15 12 18 13 13 24 13 13 5 8 5 6 10 16 13 19 16 9 15 28 6
10 10

29 18
7 7 6 8 6 8 7 8 5 8 8 8 6 10 18 14 24 18 11 25 14 10 11 12 23 11 34 9 11 27 6 10
18 23



26 14

14 20 19 23 16

19
25 14
14 20 17 23 13 20 16 17 12
22 23 13 10 11
22 20
23 17 27 19
25 25
9
22 15

16 23
20 22

22 36
17
18
19 25 8

19 26 20 14 11 22 9 12
19 15 22 7
13 15 11 18 13 10 11 23 9 16 10 13 18 10 8 19 8 17 10 9 7 8 6 7 7 6 7 7

19 10 15 13 9
17 18 15 21 8 23 6

9
25 14
14 20 16 22 15
20 19 11 16 26 13 15 16 10 13 22
30 43 27 19 11
17
22 13 36 19 26 16 21
16 25 18 15 15 19 28
8

16 18 17 16 20 13
18

17
12

22
24 20 8
8 8 8 8 8 8 8 6

22 20
27 19 13 20 22 22 28 20 18 15
17 19 26 23 22 36 20 28 36 22 19 12 34 17 11 10 18 11
21 11

23 48 12 25 11 14
11 12 20
21 6 7 30
12 20
10

11
28 20

20 28 29
20 22 29
28
18
18

18

19
32
13
28
17
22
18
50 15 20
10
13 16

25
20 19
16 17
12 11 8 9 6 22
21

29 19 29 25 10

35 21
13 15 23 20 18

25
23

9
30
18 36

22
32
21
28

22
17

18
23

25
23
13
13

15

20
12

23
25 24 20
12

15 23
20
12
12

15
17 25
20 14

13 20
19
6

6
16

11
10
9
17 24

20
19
20 17

18 20
27
18
13
12
18
27
12
13 10 12 12 10 29
10 8

25 10
32 22 18
10 9
12
8
10
7
9 8 9
6
9
7
7
7
7
10
22 19 13
26 36 27 19 36 26

14
16 19
16 16 16 18 24 17 19 28 40 20 14 10 16 19 23 14 20 24 33 15 11

25 50
26 18

18 18 23 44 29

14
22
10

13
10 17
16
20
8

8
19
12

19
15
20
20 25 40
20
14
13
20
20
14

15
23
17
20
21
13
13

14
17 20

20 12 15
12 20

22
6

17
18
19 15
8
19 24
20
14

15
13
20
46
20
17

15
23
16
20
30
24
11
8 6
32 13 33 42 16
18
18

14
21 46 21 11
19 13 19 12 17 11 18 12
6 13 13 6 6 10 6 13 18 9 17 13 17 14 10 6 6 11 17 15 12 17 15 9
31 19

18
17 18
12
17 16 27 16 9 19
12
27 12

12
27 9 11 35 25 8 11 8 12 7
18 18 12 13 16 13

22 23 24 18 18 13 10 10

19
20 15

33 14 9 14 9 13 9 13 9
21 17 13

18
12

10 22 21 22
8

10 11 19 19
19 17
17 27 18 18 17
24
19
19

19
16 16 15 24 17 12 27 17 19 15 40 15 15 25 12
19 13 28 14 18 12 24 13

18
12 17 9 16 14 17 14 8 9 6 6 12 16 15 13 17 15 9 10
11 8 24 6 10 9 15 8 14 11 12
18
13
16
15 14
18 12
12 15
8 11 19 9 14 13
18
12
14
15 13
18
12
14
15 10
20 17 20 17

19 15 34 13 25 18 24 20 23 19 20 14 17 24 19 29 12 17 10
22 8 20 17 16
20

19
16

17 19
20

17
18

20 14 24 13 20 15 18 8 19
20 17

17 20 16 10 26 11 11 9 5 11 7 24 22 21 21 19 8 8 9 8
20

u
15

15 15 27 15 17

23 15
27
27 18 35 24 15 15 15 18

49 39 27 13 15 12 15 11 10 8 7

M f
17 36 20 8 11 29 16 15

17
15

18
15 14

14 18
8 7 7
13 17 31 31 6
12 12 15 15

20 13
27 20
10 13 27 37

23
15 20 20

26
18 17

49 9 7
15 15 14 14

10 13 12 26 24
46 20 17 10
25 16
10 17 9 19 12 10 49 31 37
15
14
24
14
14

14
21
13
25 26 17 9
24 9 8 45 19 46 16 9 7 19 11 9 7 8 11 8

49
17
14 21 13 21
20 15

15 15 28

27
20 15
28 30
18 27 44 23 6 6 17 26 39 25 30
15 15

30 45 6 9 6 12 12 12 11 18 10 39 26 11
17 16

15 13
16 18 35 17 21

23 48 26 20 15 13
15 15 13 13

9
43 12 13 33 29 30
16 20
15
6
19 6 9
10 11 27
26
49 45
18
35
15

13

13
28
16
12

29
16

16 12
15

15
29
15

27
18
18

18
18 26

26 13
6 10 6 14 8 22 36
26 15
25
26
38
16
17 14 12 9 41 15
19

19 26 21
13

13
17 14
13

29 17
19
19 11 10 19 12 11 10
9 7 12 10 24 11 19 14 11 17 26 13 26 17 18 8 14 26 16 26 9 8 14 8 19 14 8 16 19 9
26 16 15 12
20 14

16 14 25 15 23 16 11 13 20 11
15

12
15

23
28

19
19

36
20 14
40 18

16
16

26 17 24 34 13
20

18
16

27 17 23 36 21
18 22

18 10 14 10
14 25 11 11

18
20

14
14

26 16 24 18 11 13

40
18 16
22
11

11
16

24 22
36 22
22 20

20 22

28
25
19
26 19

19 27 20 24 26 13 23
23 19
26
16 40 34 20 31 16 11

35 49 21 23
16 26 19 36 24 36
21 12

11 17 25 40
12

23
14 23
28
18
18

18
50 41 18 26 38 41
27

24
22

42 45 42 15 26
26 19 36 22 28 20

18 36 23 19 11
18 57 15
22 20
19 27 22 20 22 20 29 23
26 18

17 11 18 12
24 11 20 50
28

20
20

29 29
18 18
12

12

21
29
12

21 21

22
19 12
23 19 13
14 19 47 30 40 27
29

22 30
20

18
11 17 8 15 12 16 13 9 6 6 12 17 16 13 17 17 9 9 5 9 5 6
18 12 21 16 21 16

12 8 15 11 40 15 14 15 13 24 9 10 17 34 16 16
31
13
13

13
16 16
27 15
13 13
18 9 30 10 30 18 10 10
18 12 24 13 19 13 28 15
6 12 17 8 15 13 17 13 8 9 6 6 13 13 13 17 16 15 18 16 9 6 8 5 10 5 6
20 15 18 12 21 16 21 16

13
27 13 27 16
12 24 11 29 32 19
13 7
27
22 8
15

9 7
13 15

12
25 15

9
14 15

10
9 26 14 18 20
20

16
18

27
24 41 23
17

9
16 15

18
13
16

9
16

24
16 26

32
17

25
17

21 25

25
10
27
10
17 12 21 41 18 26 13 9 12 9 15 10 10 10
29 17
7 8 6 10 5 8 8 8 11 17 13 26 18 23 14 9 6 11 27 10
17 23 10 12 23 11 34 11 7 6 9 7 8 6
32 13 27 15
13 13 29 13 13 29
16 16 16

7 15 45 12 22 26 14 34 26 9 30 8 29 17 29 16
14
12

12
12

21 14

14
31 27
12

14
14
12

20 16
14
32 29 19 12 9 11 8 13 9
10 6
12 12 12 19 12 36 15 26
23 13 20 38 19 5 10 5 12 10
27
31
13
30
13

29
13

15 29
30
14
14

15

27
32
15
18
30
14

28
13

15

30
14
27
13

15
31 18 11 9 11 20 11 16 25 32 11 30 18 12 13 7

15 13 27 15 17 29
12 14 17 17

15 46 23 24 30 32 21 12 29
16

16
12 20 14

14
32 29
17

16
23 16
19
37 31 22 15
20 13 17 12
25 23

25 20 13
13 15 12 13

17 12 21 13
20 30

14 11
12 13 13 15

7 11 11 11 16 26 16
19 36
41 9
18 10 19 41 23 32 18 37 25

23
20

30
25 20

20 30
19 45 48 25 9 7 12 9 7

27
30 14 30 14

13 15 28 14 15 30

11 7
19
10
6

7
8

9

6
12 32 11
23
12

11
38

17 16 26
36 23 31
27

22
24

42
47
5

6
10

10
5

6 18 11
29

19 31 22 45 31
27
30
13
14

15
31 19 30
30
14
14

15

20
13

28
25
21
31
13

15 28
20
13

22
25
13

15
25 21

22 25

25 20
20 25
11

27 45
12
14 9
6

7
20 9 19 38 24
30

18 13 18 46
32 18

20
10

25 12
8
13 6

8

=
20 20

33 13 13
20 17

14
17 24 29

36 30 34 15 48 11 27 20 29
23 19

19 29
37 7 8 7 8 12 11 12 10 6

10 14 26 10 10
20 20

9 11 13 7 21 26 45 23 40 11 18 25 12 45 37 20
20 17

17

29
25 29
20 17

17 25
22 21
11 8 11 7 8 9 22 28 13 10 25 14 38 12 8
30 22 21
21
12
12

12
29 29 20 21 10 16 13
18 21 29 29
18

18
18

26 20 28
18

21
17

26
30 20 15 20 11 11
17

18 20
16

24

11 19 29 24 26 9 11 14 29
20
20
28
22 20
20 22

22
22

36
20
23
12
18
17
19
18
18
50 13 19
27 19
19 26 18
13
23 18 34
15
47 12 22 10
18 21 12 15 23 20 18

10 25
33
10 9 18 12 9 9
10 21 23 25 27 16 27 13 18 12 11
20 12
26 23 20 13 9 20 6

29
17 12 15 17 20

29 9 26
23 13
23 24 17

34
22 8
18 17

9
13 15 19

8 10 7 10 9 8 7 6 7 30 19 18 18 25 23
19
13
13

15
21 24 14 20 23 19 18 20 18 12 27 13 10 16 10 6

19 13 21
18 18

14
18 28 23 46 30
15 42 19 41 27 21 23
14

13
13

21
40 20 17 11 17 11 24 20 24 30 48 19 6

32 11 16 18
21
11 15 14
20 11 20
50 13
16
20
24
14
40
20
36
19
14
19

28 14 20

20 12
25 16
26
16
16

18 16 8

15 22
20 14
34
20
36
20
17
26
24
24
16
27
16
19
19 10 15
18 14 19 8 15 19 13 20 17 20 12 20 15 13 15 8
13 20 16 20 11 17 8
22 20 11
27 19 20 22 22 24 11 15
20

21 35
18
19 26 22 36 16 22 16

24 23 10 12 13
26 14

26 25 24 16 18
40
9
14 20

11 18 15
11 11 22 18
25 14
14 20
13 8 22 10 11 11
19 15 22 7

10 19 15 9 13
10 11 13 23 9 23 11 16 13 15 18 16 13 15 21 17 11 8 8 19 9 17 10 7 18 10 6 8 6 7 7 7 8 7 6

9 16
29
17

16
18
20 18
20
18
17
20 19
17

11
24
8
26
16

16
15
16

18

15
8

20 13
40
8
17

11
8

8
36
19

23
12
8
19

28

15
15
23
15
25 14
14

16
20
12

15

16

26
8

8

20 17
17 20
20
19 26
43
11 16 10
20
30
17

21
8
21
22 13

13 23
6
19 11 26 8

25 22 21 21 19 9 8 13 24 15 34 20
24
17
18
20 20 22 14 20 22 19 20 19
24 17
17 20 17 10 29 12 11 15 17 8 11 7 9 5
20 15

15 15 28
20 15
28 30
10 18 39 26 11 7 27 18 46 24 26 17 30
15 15

30 45 39 25 6 6 12 12 12 11 9 6 6
18 16
17 21 16 17 35

20
13 13 15 15 13 12

9 12 26 21
17

14 17
13 29
12 20 8 11 13 12 30 30
34 20 16 9 11 10 45 35
15

13

12
27
18
13

29
18

18 13
15

15
29
15

28
16
16

16

23
27
15
49 26 18 19 6
9
6 36 22 38 17 25 10 6
14 6 8
15 15 27 18 15

20 15
27
8 10 7 18 27 16 29 39 27 15 15 17 15

49 35 23 8 12 15 15 11 13 8

27
36 20 17 18
15
17
15
18 14
14 15
11

31 31 13 17

12
18 17

15 14 13 20
24 15 21 14

25 47 24
15 15 14 14

37 17
23 14
21 14 21 13

26
17 10
19
46 7 49
14 14 13 14

7 9 19
9 11 8 12 13 48 26 24
17 20
10
43 10 12 25 49 31
37
15

15
15

27 20
17 26
15
25 16
12

13
12

27 10
10
19 9 19 45 46 16 24 9
9 8
20 20

26 17 26 13 26 14 26 16
10 8 19 14 16 19 9 12 10 24 11 10 7 11 18 26 13 26 19 14 15 26 16 26 17 18 8

23 49
19 16 22 20 15 15 22 18 8 6

31 19

16 27 16 18 30 17 35 25 19
12
27 12

12
27 9 11 9 11 8 12 7 8
13 16 18 18 12 13

19
12 13 21 17

24
17 19
25
22 11 10 19
23 21 23 12 27
27
17 19 19 18 15 16

40
15
12

9 14 9 15 9 13 33
19 19 10 21 22 17 18 17 16 15 15
6 6 6 10
17
6 13 13
8 17 6 6 10
10 10
19
9 17
18 24
18 13

13 19
12

14
18

17

12
14

19 15
17 11

11 17
15

15
20

9
17

12
12

18
15 12 15 12 15 12 15 12

9 11 19 11 12 16 18 14 12 18 14 14 18 13 14 18 10 13
18 10
10 9 15 11 23 8
17 12 17 14 16 12 17 13

8 8 6
6 6 8 9 14 9 16 14 19 14 14 28 15 12 18 15 9 13 24
14 13

13 21 23 27 23

42 31 22 17 10 38 26 42
18 18

16 23 18 26 40 24 40

8 19 11 15 27 25 38 22 40
28

19
19

36
18
21
18
25
18

16
16

26 16 25 19 34 31
23

24
19

36
44 41 18
7 6 9 29 22 28 29 28 20

15
18

13 11
18 40 31
22 21

7 7 19 20 30 31 21 29 21
12

11
11

21 20 28 20 12 14

8 8 10 8 26 11 18 24 28 22 25
18 16
28

20
20

28
36 22

22
22 20
20 22

28
25
19 15 15
26 19

19 27 17 19 47 23 37 23 21 16
24

21 35 49
17 16 26 19 36 13 22 18 27 13 18

25 40 23 36 21
15 13 26 16

10 16 15 22 16
20 14

14 25
34 20 13 11
26 19 36 22 23 16

9 13 10 18 10 14 20 22 20
19 27 22 20 22 17 23 25
20 14

25 12 17 40 23

16
17

23 19
14 25 19
20

14
14

26
23 26 24 14
30 14 30 13

14 15 29 14 15 28
11 11 19 11 32 12 30 18 6 12 36 12 19 26 15 38 19 29
30
13
13

15
31 27
30
14
13

15
31 32 18 10 5 12 5
16 16 16

11 13 10 19 18
21 12 20 14
16

9
14

23
16
14
12 12

14
29 12

14
12

14
28
9 12 16 7 8 11 9 13 7 12 22 15 45 14 26 26 34 31 29
32
13
13

13
29 19 29
27
13
15
13
32 30 9 29 17 8 10
30 14 30 14

13 15 27 14 15 30
6 6 30 32 18 8 10 12 38 27 24 28
30
13
14

15
29 30
30
14
14

15
31 31 19 12 11 10 5 5
20 13
28 25 22

20 23
13 15

18
20 13
28 21 25

13 18
13 15

7 9 9 18 46 12 25 7 11 17 26 16
23 40 31 31 19 31 25
22
21
25
25 20
20 25
22 45 45 27 10 6 6 11
20 12 23 17

27
16 14 12 14 17 17 16

16 14
29 16 19 29
15 46 22 37 15
7

13 32 31 30 32 21 12
27 13 27 15
23 24 28 15 13 29 15 17

14 21 9 19
9 13 20 11 14 12 19 11
25 47 36 23
21
13

17
12
13

15

12

13
17
12

20
13
12

13

13

15
30 20

23 25

25 48
7 7 12 11 16 42 18 32 19 45 37 20 41

34
7 7 16 26
19 36
30 23 30 20
18 9 10 23
20 25 20 25
20 20

13 13 14 57 33
20 17

17 25 29
7 8 11 8 14 10 25 12 38 40 23 12 45 25 20 29
20 17

17 25
37 11
20 20

10 11 8 7 10 9 10 6 13 11 8 12 10 12 9 11 7 14 44 31 37 21 26 18 11 27 15 48 37 20
23 19

19

29
29 29
20 17

17 24

31 13
26 27 15
34
15 14 17 16
9 10 13 24 11 40 15 15 10 30 10 10 30 18 13 13

16
13 13

18 16
11 34
12 18 17 21 16 21

8 6 6 9 17 8 15
17

11 17
11

13
16

12
12

18 5 9 5 6 9 16
17 12

12 19 17 9
17

13
13

19
9
27 13 27 16
25 15
29 32 20
16 16 26 16
24
14 15
12
15

11
13

13 15 15 21 18 17 16

13
15 20 12 18 16 21 16 21

10 14 26 18 26
32 17

9 13 9 13 10 15 10 21 23 20
24 12 17 21 25
12 21 41
13 26 41
17
7 7 9 10 9 16 27 18 9 24 9 27
9 18 8 17 12 17 13 17 13
25 25
18 15

6 6 6 8 9 8 15 12 18 13 13 24 13 13 5 8 5 6 10 16 13 19 16 9 15 28 6
10 10

29 18
7 7 6 8 6 8 7 8 5 8 8 8 6 10 18 14 24 18 11 25 14 10 11 12 23 11 34 9 11 27 6 10
18 23



1
3
·10

4 0.5

2
+2V
+5V 0
0
−5V
−2
−0.5
−4
2
−4
−2 0 ·103
0
2 −2 −1
4
·103


Eigenvalues



Deﬁnition

Mx = λx



Deﬁnition

M ∈ Rn×n

Mx = λx



Deﬁnition

M ∈ Rn×n

Mx = λx

eigenvector
x ∈ Cn , x = 0



Deﬁnition

M ∈ Rn×n eigenvalue
λ∈C

Mx = λx

eigenvector
x ∈ Cn , x = 0



Deﬁnition

M = M T ∈ Rn×n eigenvalue
λ∈R

Mx = λx

eigenvector
x ∈ Rn , x = 0



Application

Eigenvalue problems occur in many diﬀerent applications:
vibration analysis,
molecular and quantum dynamics
(e.g. Schr¨dinger or Kohn-Sham equations),
o
ﬁnite state of a Markov chain,
numerical mathematics (e.g. MOR, e M , convergence theory),
...



Deﬁnition

M = M T ∈ H(T , k) eigenvalue
λ∈R

Mx = λx

eigenvector
x ∈ Rn , x = 0



Preconditioned Inverse Iteration
[Knyazev, Neymeyr, et al.]

Deﬁnition
The function
x T Mx
µ(x) = µ(x, M) =
xT x
is called the Rayleigh quotient.




Deﬁnition
The function
x T Mx
µ(x) = µ(x, M) =
xT x

Minimize the Rayleigh quotient by a gradient method:
2
xi+1 := xi − α µ(xi ), µ(x) = T (Mx − xµ(x)),
x x




Residual r (x) = Mx − xµ(x).

2
x x




Deﬁnition
The function
x T Mx
µ(x) = µ(x, M) =
xT x

2
x x
+ preconditioning ⇒ update equation:
xi+1 := xi − T −1 (Mxi − xi µ(xi )).



[Knyazev, Neymeyr ’09]

xi+1 := xi − T −1 (Mxi − xi µ(xi ))

If
M symmetric positive deﬁnite and
T −1 approximates the inverse of M, so that

I − T −1 M M
≤ c < 1,



[Knyazev, Neymeyr ’09]

xi+1 := xi − T −1 (Mxi − xi µ(xi ))

If
M symmetric positive deﬁnite and
T −1 approximates the inverse of M, so that

I − T −1 M M
≤ c < 1,

then Preconditioned INVerse ITeration (PINVIT) converges.



BEM, smallest eigenvalues
dense matrix, but approximable by an H-matrix

104
PINVIT
103 O(ni (log2 ni ) CH )
H-Cholesky factor.
102 O(ni (log2 ni )2 CH )
CPU time in s

101

100

10−1

10−2
d = 4, c = 0.2, eig = 10−4
10−3
258 1026 4098 16386 65538



104
PINVIT
H-Cholesky factor.
CPU time in s

101 ARPACK / eigs

100

10−1

10−2
d = 4, c = 0.2, eig = 10−4
10−3
258 1026 4098 16386 65538


Concluding Remarks

H-arithmetic is cheap and applicable too many problems.
Computing eigenvalues of H-matrices in linear-polylogarithmic
complexity is possible.
See poster: “Slicing the Spectrum of Symmetric H -Matrices”.



Concluding Remarks

H-arithmetic is cheap and applicable too many problems.
Computing eigenvalues of H-matrices in linear-polylogarithmic
complexity is possible.
See poster: “Slicing the Spectrum of Symmetric H -Matrices”.

Thank you for your attention.


Appendix


104
PINVIT
H-Cholesky factor.
CPU time in s

101

100

10−1

10−2
d = 4, c = 0.2, eig = 10−4
10−3
258 1026 4098 16386 65538

Appendix


104
PINVIT
H-Cholesky factor.
CPU time in s

101
eig ∈ O(n3 )
100

10−1

10−2
d = 4, c = 0.2, eig = 10−4
10−3
258 1026 4098 16386 65538

Hierarchical Matrices: Concept, Application and Eigenvalues

More Related Content

Similar to Hierarchical Matrices: Concept, Application and Eigenvalues (20)

Recently uploaded (20)

Hierarchical Matrices: Concept, Application and Eigenvalues