metode iterasi Gauss seidel

Iterative Techniques in Matrix Algebra
Jacobi & Gauss-Seidel Iterative Techniques II
Numerical Analysis (9th Edition)
R L Burden & J D Faires
Beamer Presentation Slides
prepared by
John Carroll
Dublin City University
c
2011 Brooks/Cole, Cengage Learning

Gauss-Seidel Method Gauss-Seidel Algorithm Convergence Results Interpretation
Outline
1 The Gauss-Seidel Method
Numerical Analysis (Chapter 7) Jacobi & Gauss-Seidel Methods II R L Burden & J D Faires 2 / 38

Outline
2 The Gauss-Seidel Algorithm

Outline
3 Convergence Results for General Iteration Methods

Outline
4 Application to the Jacobi & Gauss-Seidel Methods

The Gauss-Seidel Method
Looking at the Jacobi Method
A possible improvement to the Jacobi Algorithm can be seen by
re-considering
x(k)
i =
1
aii
2
Xn
664
j=1
j6=i

−aijx(k−1)
j

+ bi
3
775
, for i = 1, 2, . . . , n
Numerical Analysis (Chapter 7) Jacobi Gauss-Seidel Methods II R L Burden J D Faires 4 / 38

re-considering
x(k)
i =
1
aii
2
Xn
664
j=1
j6=i

−aijx(k−1)
j

+ bi
3
775
, for i = 1, 2, . . . , n
The components of x(k−1) are used to compute all the
components x(k)
i of x(k).

re-considering
x(k)
i =
1
aii
2
Xn
664
j=1
j6=i

−aijx(k−1)
j

+ bi
3
775
, for i = 1, 2, . . . , n
The components of x(k−1) are used to compute all the
components x(k)
i of x(k).
But, for i 1, the components x(k)
1 , . . . , x(k)
i−1 of x(k) have already
been computed and are expected to be better approximations to
the actual solutions x1, . . . , xi−1 than are x(k−1)
1 , . . . , x(k−1)
i−1 .

Instead of using
x(k)
i =
1
aii
2
Xn
664
j=1
j6=i

−aijx(k−1)
j

+ bi
3
775
, for i = 1, 2, . . . , n
it seems reasonable, then, to compute x(k)
i using these most recently
calculated values.

Instead of using
x(k)
i =
1
aii
2
Xn
664
j=1
j6=i

−aijx(k−1)
j

+ bi
3
775
, for i = 1, 2, . . . , n
it seems reasonable, then, to compute x(k)
i using these most recently
calculated values.
The Gauss-Seidel Iterative Technique
x(k)
i =
1
aii
2
4−
Xi−1
j=1
(aijx(k)
j ) −
Xn
j=i+1
(aijx(k−1)
j ) + bi
3
5
for each i = 1, 2, . . . , n.

Example
Use the Gauss-Seidel iterative technique to find approximate solutions
to
10x1 − x2 + 2x3 = 6
−x1 + 11x2 − x3 + 3x4 = 25
2x1 − x2 + 10x3 − x4 = −11
3x2 − x3 + 8x4 = 15
,

Example
to
10x1 − x2 + 2x3 = 6
−x1 + 11x2 − x3 + 3x4 = 25
2x1 − x2 + 10x3 − x4 = −11
3x2 − x3 + 8x4 = 15
,
starting with x = (0, 0, 0, 0)t

Example
to
10x1 − x2 + 2x3 = 6
−x1 + 11x2 − x3 + 3x4 = 25
2x1 − x2 + 10x3 − x4 = −11
3x2 − x3 + 8x4 = 15
,
starting with x = (0, 0, 0, 0)t and iterating until
kx(k) − x(k−1)k1
kx(k)k1
10−3

Example
to
10x1 − x2 + 2x3 = 6
−x1 + 11x2 − x3 + 3x4 = 25
2x1 − x2 + 10x3 − x4 = −11
3x2 − x3 + 8x4 = 15
,
starting with x = (0, 0, 0, 0)t and iterating until
kx(k) − x(k−1)k1
kx(k)k1
10−3
Note: The solution x = (1, 2, −1, 1)t was approximated by Jacobi’s
method in an earlier example.

Solution (1/3)
For the Gauss-Seidel method we write the system, for each
k = 1, 2, . . . as
x(k)
1 =
1
10
x(k−1)
2 −
1
5
x(k−1)
3 +
3
5
x(k)
2 =
1
11
x(k)
1 +
1
11
x(k−1)
3 −
3
11
x(k−1)
4 +
25
11
x(k)
3 = −
1
5
x(k)
1 +
1
10
x(k)
2 +
1
10
x(k−1)
4 −
11
10
x(k)
4 = −
3
8
x(k)
2 +
1
8
x(k)
3 +
15
8

Solution (2/3)
When x(0) = (0, 0, 0, 0)t , we have
x(1) = (0.6000, 2.3272, −0.9873, 0.8789)t .

Solution (2/3)
When x(0) = (0, 0, 0, 0)t , we have
x(1) = (0.6000, 2.3272, −0.9873, 0.8789)t . Subsequent iterations give
the values in the following table:
k 0 1 2 3 4 5
x(k)
1 0.0000 0.6000 1.030 1.0065 1.0009 1.0001
x(k)
2 0.0000 2.3272 2.037 2.0036 2.0003 2.0000
x(k)
3 0.0000 −0.9873 −1.014 −1.0025 −1.0003 −1.0000
x(k)
4 0.0000 0.8789 0.984 0.9983 0.9999 1.0000

Solution (3/3)
Because
kx(5) − x(4)k1
kx(5)k1
=
0.0008
2.000 = 4 × 10−4
x(5) is accepted as a reasonable approximation to the solution.

Solution (3/3)
Because
kx(5) − x(4)k1
kx(5)k1
=
0.0008
2.000 = 4 × 10−4
x(5) is accepted as a reasonable approximation to the solution.
Note that, in an earlier example, Jacobi’s method required twice as
many iterations for the same accuracy.

The Gauss-Seidel Method: Matrix Form
Re-Writing the Equations
To write the Gauss-Seidel method in matrix form,

To write the Gauss-Seidel method in matrix form, multiply both sides of
x(k)
i =
1
aii
2
4−
Xi−1
j=1
(aijx(k)
j ) −
Xn
j=i+1
(aijx(k−1)
j ) + bi
3
5
by aii and collect all kth iterate terms,

To write the Gauss-Seidel method in matrix form, multiply both sides of
x(k)
i =
1
aii
2
4−
Xi−1
j=1
(aijx(k)
j ) −
Xn
j=i+1
(aijx(k−1)
j ) + bi
3
5
by aii and collect all kth iterate terms, to give
ai1x(k)
1 + ai2x(k)
2 + · · · + aiix(k)
i = −ai,i+1x(k−1)
i+1 − · · · − ainx(k−1)
n + bi
for each i = 1, 2, . . . , n.

Re-Writing the Equations (Cont’d)
Writing all n equations gives
(k)
1 = −a12x
a11x
(k−1)
2 − a13x
(k−1)
3 − · · · − a1nx
(k−1)
n + b1
(k)
1 + a22x
a21x
(k)
2 = −a23x
(k−1)
3 − · · · − a2nx
(k−1)
n + b2
...
an1x(k)
1 + an2x(k)
2 + · · · + annx(k)
n = bn

Writing all n equations gives
(k)
1 = −a12x
a11x
(k−1)
2 − a13x
(k−1)
3 − · · · − a1nx
(k−1)
n + b1
(k)
1 + a22x
a21x
(k)
2 = −a23x
(k−1)
3 − · · · − a2nx
(k−1)
n + b2
...
an1x(k)
1 + an2x(k)
2 + · · · + annx(k)
n = bn
With the definitions of D, L, and U given previously, we have the
Gauss-Seidel method represented by
(D − L)x(k) = Ux(k−1) + b

(D − L)x(k) = Ux(k−1) + b
Solving for x(k) finally gives
x(k) = (D − L)−1Ux(k−1) + (D − L)−1b, for each k = 1, 2, . . .

(D − L)x(k) = Ux(k−1) + b
Letting Tg = (D − L)−1U and cg = (D − L)−1b, gives the Gauss-Seidel
technique the form
x(k) = Tgx(k−1) + cg

(D − L)x(k) = Ux(k−1) + b
Letting Tg = (D − L)−1U and cg = (D − L)−1b, gives the Gauss-Seidel
technique the form
For the lower-triangular matrix D − L to be nonsingular, it is necessary
and sufficient that aii6= 0, for each i = 1, 2, . . . , n.

Outline
4 Application to the Jacobi Gauss-Seidel Methods

Gauss-Seidel Iterative Algorithm (1/2)
To solve Ax = b given an initial approximation x(0):

INPUT the number of equations and unknowns n;
the entries aij , 1 i, j n of the matrix A;
the entries bi , 1 i n of b;
the entries XOi , 1 i n of XO = x(0);
tolerance TOL;
maximum number of iterations N.

INPUT the number of equations and unknowns n;
the entries aij , 1 i, j n of the matrix A;
the entries bi , 1 i n of b;
the entries XOi , 1 i n of XO = x(0);
tolerance TOL;
maximum number of iterations N.
OUTPUT the approximate solution x1, . . . , xn or a message
that the number of iterations was exceeded.

Step 1 Set k = 1
Step 2 While (k N) do Steps 3–6:

Step 1 Set k = 1
Step 3 For i = 1, . . . , n
set xi =
1
aii
2
4−
Xi−1
j=1
aijxj −
Xn
j=i+1
aijXOj + bi
3
5

Step 1 Set k = 1
Step 3 For i = 1, . . . , n
set xi =
1
aii
2
4−
Xi−1
j=1
aijxj −
Xn
j=i+1
aijXOj + bi
3
5
Step 4 If ||x − XO|| TOL then OUTPUT (x1, . . . , xn)
(The procedure was successful)
STOP

Step 1 Set k = 1
Step 3 For i = 1, . . . , n
set xi =
1
aii
2
4−
Xi−1
j=1
aijxj −
Xn
j=i+1
aijXOj + bi
3
5
STOP
Step 5 Set k = k + 1

Step 1 Set k = 1
Step 3 For i = 1, . . . , n
set xi =
1
aii
2
4−
Xi−1
j=1
aijxj −
Xn
j=i+1
aijXOj + bi
3
5
STOP
Step 6 For i = 1, . . . , n set XOi = xi

Step 1 Set k = 1
Step 3 For i = 1, . . . , n
set xi =
1
aii
2
4−
Xi−1
j=1
aijxj −
Xn
j=i+1
aijXOj + bi
3
5
STOP
Step 6 For i = 1, . . . , n set XOi = xi
Step 7 OUTPUT (‘Maximum number of iterations exceeded’)
STOP (The procedure was unsuccessful)

Gauss-Seidel Iterative Algorithm
Comments on the Algorithm
Step 3 of the algorithm requires that aii6= 0, for each
i = 1, 2, . . . , n.

i = 1, 2, . . . , n. If one of the aii entries is 0 and the system is
nonsingular, a reordering of the equations can be performed so
that no aii = 0.

that no aii = 0.
To speed convergence, the equations should be arranged so that
aii is as large as possible.

that no aii = 0.
Another possible stopping criterion in Step 4 is to iterate until
kx(k) − x(k−1)k
kx(k)k
is smaller than some prescribed tolerance.

that no aii = 0.
Another possible stopping criterion in Step 4 is to iterate until
kx(k) − x(k−1)k
kx(k)k
is smaller than some prescribed tolerance.
For this purpose, any convenient norm can be used, the usual
being the l1 norm.

Outline

Convergence Results for General Iteration Methods
Introduction
To study the convergence of general iteration techniques, we need
to analyze the formula
x(k) = Tx(k−1) + c, for each k = 1, 2, . . .
where x(0) is arbitrary.
The following lemma and the earlier Theorem on convergent
matrices provide the key for this study.

Lemma
If the spectral radius satisfies (T) 1, then (I − T)−1 exists, and
(I − T)−1 = I + T + T2 + · · · =
1X
j=0
Tj

Lemma
(I − T)−1 = I + T + T2 + · · · =
1X
j=0
Tj
Proof (1/2)
Because Tx = x is true precisely when (I − T)x = (1 − )x, we
have as an eigenvalue of T precisely when 1 − is an
eigenvalue of I − T.

Lemma
(I − T)−1 = I + T + T2 + · · · =
1X
j=0
Tj
Proof (1/2)
But || (T) 1, so = 1 is not an eigenvalue of T, and 0
cannot be an eigenvalue of I − T.

Lemma
(I − T)−1 = I + T + T2 + · · · =
1X
j=0
Tj
Proof (1/2)
But || (T) 1, so = 1 is not an eigenvalue of T, and 0
cannot be an eigenvalue of I − T.
Hence, (I − T)−1 exists.

Proof (2/2)
Let
Sm = I + T + T2 + · · · + Tm

Proof (2/2)
Let
Sm = I + T + T2 + · · · + Tm
Then
(I −T)Sm = (1+T +T2+· · ·+Tm)−(T +T2+· · ·+Tm+1) = I −Tm+1

Proof (2/2)
Let
Sm = I + T + T2 + · · · + Tm
Then
and, since T is convergent, the Theorem on convergent matrices
implies that
lim
m!1
(I − T)Sm = lim
(I − Tm+1) = I
m!1

Proof (2/2)
Let
Sm = I + T + T2 + · · · + Tm
Then
and, since T is convergent, the Theorem on convergent matrices
implies that
lim
m!1
(I − T)Sm = lim
(I − Tm+1) = I
m!1
Thus, (I − T)−1 = limm!1 Sm = I + T + T2 + · · · =
P1j
=0 Tj

Theorem
For any x(0) 2 IRn, the sequence {x(k)}1k
=0 defined by
x(k) = Tx(k−1) + c, for each k 1
converges to the unique solution of
x = Tx + c
if and only if (T) 1.

Proof (1/5)
First assume that (T) 1.

Proof (1/5)
First assume that (T) 1. Then,
x(k) = Tx(k−1) + c

Proof (1/5)
x(k) = Tx(k−1) + c
= T(Tx(k−2) + c) + c

Proof (1/5)
x(k) = Tx(k−1) + c
= T(Tx(k−2) + c) + c
= T2x(k−2) + (T + I)c

Proof (1/5)
x(k) = Tx(k−1) + c
= T(Tx(k−2) + c) + c
= T2x(k−2) + (T + I)c
...
= Tkx(0) + (Tk−1 + · · · + T + I)c

Proof (1/5)
x(k) = Tx(k−1) + c
= T(Tx(k−2) + c) + c
= T2x(k−2) + (T + I)c
...
= Tkx(0) + (Tk−1 + · · · + T + I)c
Because (T) 1, the Theorem on convergent matrices implies that T
is convergent, and
lim
k!1
Tkx(0) = 0

Proof (2/5)
The previous lemma implies that
lim
k!1
x(k) = lim
k!1
Tkx(0) +
0
@
1X
j=0
Tj
1
Ac

Proof (2/5)
lim
k!1
x(k) = lim
k!1
Tkx(0) +
0
@
1X
j=0
Tj
1
Ac
= 0 + (I − T)−1c

Proof (2/5)
lim
k!1
x(k) = lim
k!1
Tkx(0) +
0
@
1X
j=0
Tj
1
Ac
= 0 + (I − T)−1c
= (I − T)−1c

Proof (2/5)
lim
k!1
x(k) = lim
k!1
Tkx(0) +
0
@
1X
j=0
Tj
1
Ac
= 0 + (I − T)−1c
= (I − T)−1c
Hence, the sequence {x(k)} converges to the vector x (I − T)−1c
and x = Tx + c.

Proof (3/5)
To prove the converse, we will show that for any z 2 IRn, we have
limk!1 Tkz = 0.

Proof (3/5)
limk!1 Tkz = 0.
Again, by the theorem on convergent matrices, this is equivalent
to (T) 1.

Proof (3/5)
limk!1 Tkz = 0.
to (T) 1.
Let z be an arbitrary vector, and x be the unique solution to
x = Tx + c.

Proof (3/5)
limk!1 Tkz = 0.
to (T) 1.
x = Tx + c.
Define x(0) = x − z, and, for k 1, x(k) = Tx(k−1) + c.

Proof (3/5)
limk!1 Tkz = 0.
to (T) 1.
x = Tx + c.
Define x(0) = x − z, and, for k 1, x(k) = Tx(k−1) + c.
Then {x(k)} converges to x.

Proof (4/5)
Also,
x − x(k) = (Tx + c) −

Tx(k−1) + c

= T

x − x(k−1)


Proof (4/5)
Also,
x − x(k) = (Tx + c) −

Tx(k−1) + c

= T

x − x(k−1)

so
x − x(k) = T

x − x(k−1)


Proof (4/5)
Also,
x − x(k) = (Tx + c) −

Tx(k−1) + c

= T

x − x(k−1)

so
x − x(k) = T

x − x(k−1)

= T2

x − x(k−2)


Proof (4/5)
Also,
x − x(k) = (Tx + c) −

Tx(k−1) + c

= T

x − x(k−1)

so
x − x(k) = T

x − x(k−1)

= T2

x − x(k−2)

=
...

Proof (4/5)
Also,
x − x(k) = (Tx + c) −

Tx(k−1) + c

= T

x − x(k−1)

so
x − x(k) = T

x − x(k−1)

= T2

x − x(k−2)

=
...
= Tk

x − x(0)


Proof (4/5)
Also,
x − x(k) = (Tx + c) −

Tx(k−1) + c

= T

x − x(k−1)

so
x − x(k) = T

x − x(k−1)

= T2

x − x(k−2)

=
...
= Tk

x − x(0)

= Tkz

Proof (5/5)
Hence
lim
k!1
Tkz = lim
k!1
Tk

x − x(0)


Proof (5/5)
Hence
lim
k!1
Tkz = lim
k!1
Tk

x − x(0)

= lim
k!1

x − x(k)


Proof (5/5)
Hence
lim
k!1
Tkz = lim
k!1
Tk

x − x(0)

= lim
k!1

x − x(k)

= 0

Proof (5/5)
Hence
lim
k!1
Tkz = lim
k!1
Tk

x − x(0)

= lim
k!1

x − x(k)

= 0
But z 2 IRn was arbitrary, so by the theorem on convergent
matrices, T is convergent and (T) 1.

Corollary
kTk 1 for any natural matrix norm and c is a given vector, then the
sequence {x(k)}1k
=0 defined by
x(k) = Tx(k−1) + c
converges, for any x(0) 2 IRn, to a vector x 2 IRn, with x = Tx + c, and
the following error bounds hold:

Corollary
sequence {x(k)}1k
=0 defined by
x(k) = Tx(k−1) + c
(i) kx − x(k)k kTkkkx(0) − xk

Corollary
sequence {x(k)}1k
=0 defined by
x(k) = Tx(k−1) + c
(ii) kx − x(k)k kTkk
1−kTkkx(1) − x(0)k

Corollary
sequence {x(k)}1k
=0 defined by
x(k) = Tx(k−1) + c
(ii) kx − x(k)k kTkk
1−kTkkx(1) − x(0)k
The proof of the following corollary is similar to that for the Corollary to
the Fixed-Point Theorem for a single nonlinear equation.

Outline

Convergence of the Jacobi Gauss-Seidel Methods
Using the Matrix Formulations
We have seen that the Jacobi and Gauss-Seidel iterative techniques
can be written
x(k) = Tjx(k−1) + cj and

can be written
using the matrices
Tj = D−1(L + U) and Tg = (D − L)−1U
respectively.

can be written
using the matrices
Tj = D−1(L + U) and Tg = (D − L)−1U
respectively. If (Tj ) or (Tg) is less than 1, then the corresponding
sequence {x(k)}1k
=0 will converge to the solution x of Ax = b.

Example
For example, the Jacobi method has
x(k) = D−1(L + U)x(k−1) + D−1b,

Example
x(k) = D−1(L + U)x(k−1) + D−1b,
and, if {x(k)}1k
=0 converges to x,

Example
x(k) = D−1(L + U)x(k−1) + D−1b,
and, if {x(k)}1k
=0 converges to x, then
x = D−1(L + U)x + D−1b

Example
x(k) = D−1(L + U)x(k−1) + D−1b,
and, if {x(k)}1k
x = D−1(L + U)x + D−1b
This implies that
Dx = (L + U)x + b and (D − L − U)x = b

Example
x(k) = D−1(L + U)x(k−1) + D−1b,
and, if {x(k)}1k
x = D−1(L + U)x + D−1b
This implies that
Dx = (L + U)x + b and (D − L − U)x = b
Since D − L − U = A, the solution x satisfies Ax = b.

The following are easily verified sufficiency conditions for convergence
of the Jacobi and Gauss-Seidel methods.

The following are easily verified sufficiency conditions for convergence
of the Jacobi and Gauss-Seidel methods.
Theorem
If A is strictly diagonally dominant, then for any choice of x(0), both the
Jacobi and Gauss-Seidel methods give sequences {x(k)}1k
=0 that
converge to the unique solution of Ax = b.

Is Gauss-Seidel better than Jacobi?

No general results exist to tell which of the two techniques, Jacobi
or Gauss-Seidel, will be most successful for an arbitrary linear
system.

No general results exist to tell which of the two techniques, Jacobi
or Gauss-Seidel, will be most successful for an arbitrary linear
system.
In special cases, however, the answer is known, as is
demonstrated in the following theorem.

(Stein-Rosenberg) Theorem
If aij 0, for each i6= j and aii 0, for each i = 1, 2, . . . , n, then one
and only one of the following statements holds:
(i) 0 (Tg) (Tj ) 1
(ii) 1 (Tj ) (Tg)
(iii) (Tj ) = (Tg) = 0
(iv) (Tj ) = (Tg) = 1

(Stein-Rosenberg) Theorem
If aij 0, for each i6= j and aii 0, for each i = 1, 2, . . . , n, then one
and only one of the following statements holds:
(i) 0 (Tg) (Tj ) 1
(ii) 1 (Tj ) (Tg)
(iii) (Tj ) = (Tg) = 0
(iv) (Tj ) = (Tg) = 1
For the proof of this result, see pp. 120–127. of
Young, D. M., Iterative solution of large linear systems, Academic
Press, New York, 1971, 570 pp.

Two Comments on the Thoerem
For the special case described in the theorem, we see from part
(i), namely
0 (Tg) (Tj ) 1

(i), namely
0 (Tg) (Tj ) 1
that when one method gives convergence, then both give
convergence,

(i), namely
0 (Tg) (Tj ) 1
convergence, and the Gauss-Seidel method converges faster than
the Jacobi method.

(i), namely
0 (Tg) (Tj ) 1
convergence, and the Gauss-Seidel method converges faster than
the Jacobi method.
Part (ii), namely
1 (Tj ) (Tg)
indicates that when one method diverges then both diverge, and
the divergence is more pronounced for the Gauss-Seidel method.

Eigenvalues Eigenvectors: Convergent Matrices
Theorem
The following statements are equivalent.
(i) A is a convergent matrix.
(ii) limn!1 kAnk = 0, for some natural norm.
(iii) limn!1 kAnk = 0, for all natural norms.
(iv) (A) 1.
(v) limn!1 Anx = 0, for every x.
The proof of this theorem can be found on p. 14 of Issacson, E. and H.
B. Keller, Analysis of Numerical Methods, John Wiley Sons, New
York, 1966, 541 pp.
Return to General Iteration Methods — Introduction
Return to General Iteration Methods — Lemma
Return to General Iteration Methods — Theorem

Fixed-Point Theorem
Let g 2 C[a, b] be such that g(x) 2 [a, b], for all x in [a, b]. Suppose, in
addition, that g0 exists on (a, b) and that a constant 0 k 1 exists
with
|g0(x)| k, for all x 2 (a, b).
Then for any number p0 in [a, b], the sequence defined by
pn = g(pn−1), n 1
converges to the unique fixed point p in [a, b].
Return to the Corrollary to the Fixed-Point Theorem

Functional (Fixed-Point) Iteration
Corrollary to the Fixed-Point Convergence Result
If g satisfies the hypothesis of the Fixed-Point Theorem then
|pn − p|
kn
1 − k
|p1 − p0|
Return to the Corollary to the Convergence Theorem for General Iterative Methods

metode iterasi Gauss seidel

More Related Content

What's hot (20)

Similar to metode iterasi Gauss seidel (20)

Recently uploaded (20)

metode iterasi Gauss seidel