![Contents ix
6.3.3 Optimization with differentiable, non-degenerate
inequality constraints . . . . . . . . . . . . . . . . 97
6.4 The method of projections in continuous time . . . . . . 98
6.5 Maxentropic approaches . . . . . . . . . . . . . . . . . . 99
6.5.1 Linear systems with band constraints . . . . . . . 100
6.5.2 Linear system with Euclidean norm constraints . 102
6.5.3 Linear systems with non-Euclidean norm
constraints . . . . . . . . . . . . . . . . . . . . . . 104
6.5.4 Linear systems with solutions in unbounded
convex sets . . . . . . . . . . . . . . . . . . . . . 105
6.5.5 Linear equations without constraints . . . . . . . 109
6.6 Linear systems with measurement noise . . . . . . . . . . 112
6.7 Bibliographical comments and references . . . . . . . . . 113
7. Some simple numerical examples and moment
problems 115
7.1 The density of the Earth . . . . . . . . . . . . . . . . . . 115
7.1.1 Solution by the standard L2[0, 1] techniques . . . 116
7.1.2 Piecewise approximations in L2([0, 1]) . . . . . . 117
7.1.3 Linear programming approach . . . . . . . . . . . 118
7.1.4 Maxentropic reconstructions: Influence of a priori
data . . . . . . . . . . . . . . . . . . . . . . . . . 120
7.1.5 Maxentropic reconstructions: Effect of the noise . 122
7.2 A test case . . . . . . . . . . . . . . . . . . . . . . . . . . 125
7.2.1 Standard L2[0, 1] technique . . . . . . . . . . . . 126
7.2.2 Discretized L2[0, 1] approach . . . . . . . . . . . 127
7.2.3 Maxentropic reconstructions: Influence of a priori
data . . . . . . . . . . . . . . . . . . . . . . . . . 128
7.2.4 Reconstruction by means of cubic splines . . . . 131
7.2.5 Fourier versus cubic splines . . . . . . . . . . . . 135
7.3 Standard maxentropic reconstruction . . . . . . . . . . . 141
7.3.1 Existence and stability . . . . . . . . . . . . . . . 144
7.3.2 Some convergence issues . . . . . . . . . . . . . . 146
7.4 Some remarks on moment problems . . . . . . . . . . . . 146
7.4.1 Some remarks about the Hamburger and Stieltjes
moment problems . . . . . . . . . . . . . . . . . . 149
7.5 Moment problems in Hilbert spaces . . . . . . . . . . . . 152
7.6 Reconstruction of transition probabilities . . . . . . . . . 154](https://image.slidesharecdn.com/1162355-250529084159-1be53f4c/85/Linear-Inverse-Problems-The-Maximum-Entropy-Connection-Series-On-Advances-In-Mathematics-For-Applied-Sciences-83-Harcdr-Henryk-Gzyl-16-320.jpg)
![List of Figures
2.1 Basic scheme. . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.2 Basic geometry of the CAT process. . . . . . . . . . . . . . . . 17
2.3 Tomographic image formation. . . . . . . . . . . . . . . . . . . 18
2.4 Actual CAT setup. . . . . . . . . . . . . . . . . . . . . . . . . 19
6.1 C ∩ A−1
y = ∅. . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
6.2 C ∩ A−1
y = ∅. x∗
= x0. . . . . . . . . . . . . . . . . . . . . . 94
6.3 C ∩ A−1
y x∗
= x0. . . . . . . . . . . . . . . . . . . . . . . . 95
7.1 Standard L2 solution to (7.1). . . . . . . . . . . . . . . . . . . 116
7.2 Standard L2 with penalization. . . . . . . . . . . . . . . . . . 117
7.3 Reconstruction with λ = 10−7
, ε = 6.0063 × 10−5
. . . . . . . . 118
7.4 Penalized least square reconstruction. Constrained a = 0,
b = 1; parameters λ = 10−5
, 10−7
. Reconstruction errors
ε = 5.23 × 10−3
, 6.39 × 10−5
. . . . . . . . . . . . . . . . . . . 119
7.5 Penalized least square reconstruction. Constrained a = 0.1,
b = 0.9; parameters λ = 10−5
, 10−7
, ε = 7.13 × 10−5
. . . . . . 119
7.6 Reconstructing using linear programming. . . . . . . . . . . . 120
7.7 Maxentropic reconstruction with uniform a priori on different
[a, b]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
7.8 Maxentropic reconstruction with uniform a priori on different
[a, b]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
7.9 A priori distribution of Bernoulli type masses p = q = 1
2 and
different intervals. . . . . . . . . . . . . . . . . . . . . . . . . . 122
7.10 A priori distribution of Bernoulli type on a fixed interval but
different masses at the ends. . . . . . . . . . . . . . . . . . . . 123
7.11 Reconstructions with uniform a priori distribution on [0, 1],
fixed σ1 = σ2 = 0.01 and varying T . . . . . . . . . . . . . . . . 124
xv](https://image.slidesharecdn.com/1162355-250529084159-1be53f4c/85/Linear-Inverse-Problems-The-Maximum-Entropy-Connection-Series-On-Advances-In-Mathematics-For-Applied-Sciences-83-Harcdr-Henryk-Gzyl-22-320.jpg)
![xvi Linear Inverse Problems: The Maxentropic Connection
7.12 Reconstructions with uniform a priori distribution on [0, 4],
varying σs and T s. . . . . . . . . . . . . . . . . . . . . . . . . 124
7.13 Reconstructions with fixed Bernoulli distribution on [0, 10] with
p = q = 1
2 , T = 1 and σ1 = σ2 varying. . . . . . . . . . . . . . 125
7.14 Reconstructions under Bernoulli a priori distribution on
[0, 10] with p = 1 = 1
2 , fixed σ1 = σ2 = 0.1, but T varying
as indicated. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
7.15 Reconstructions from first list of moments. Different λs and its
respectively error. . . . . . . . . . . . . . . . . . . . . . . . . . 127
7.16 Reconstructions from second list of moments. Different λs and
its respectively error. . . . . . . . . . . . . . . . . . . . . . . . 128
7.17 Reconstructions for first list moments and several λ. . . . . . 129
7.18 Reconstructions for second list of moments and several λ. . . 129
7.19 Reconstructions for first list of moments and several λ, pro-
jected onto constraint space. . . . . . . . . . . . . . . . . . . . 130
7.20 Reconstructions for second list of moments and several λ, pro-
jected onto constrained space. . . . . . . . . . . . . . . . . . . 130
7.21 Maxentropic reconstructions for different constraints and uni-
form a priori measure. . . . . . . . . . . . . . . . . . . . . . . 131
7.22 Maxentropic reconstruction for different constrained and
Bernoulli a priori measure with p = q = 1
2 . . . . . . . . . . . . 132
7.23 Maxentropic reconstruction for [a, b] = [−2, 2] and p = 0.5, 0.2,
0.8 with q = 1 − p. . . . . . . . . . . . . . . . . . . . . . . . . 132
7.24 Maxentropic reconstruction with cubic splines from m = 2,
w = 1 with error ε = 1.9830 × 10−10
. . . . . . . . . . . . . . . 133
7.25 Maxentropic reconstruction with cubic splines from m = 2,
w = π with error ε = 1.8340 × 10−8
. . . . . . . . . . . . . . . 134
7.26 Maxentropic reconstruction with cubic splines from m = 4,
w = π with error ε = 4.5339 × 10−8
. . . . . . . . . . . . . . . 134
7.27 Maxentropic reconstruction with cubic splines from m = 4,
w = π with error ε = 7.2104 × 10−9
. . . . . . . . . . . . . . . 135
7.28 Maxentropic reconstruction with cubic splines from m = 4,
w = 2π with error ε = 5.2992 × 10−8
. . . . . . . . . . . . . . . 135
7.29 Maxentropic reconstruction with cubic splines from m = 7,
w = 4π with error ε = 6.2902 × 10−6
. . . . . . . . . . . . . . . 136
7.30 Maxentropic reconstruction with cubic splines from m = 10,
w = 4π with error ε = 9.9747 × 10−6
. . . . . . . . . . . . . . . 136](https://image.slidesharecdn.com/1162355-250529084159-1be53f4c/85/Linear-Inverse-Problems-The-Maximum-Entropy-Connection-Series-On-Advances-In-Mathematics-For-Applied-Sciences-83-Harcdr-Henryk-Gzyl-23-320.jpg)
![List of Figures xvii
7.31 Maxentropic reconstruction with cubic splines from m = 4, in
[−5, 5], w = π, σ = 0.1, T1 = 0.5, T2 = 1, T3 = 1.5 and T4 = 2
with error ε1 = 0.05, ε2 = 0.10, ε3 = 0.20 and ε4 = 0.20. . . . 137
7.32 Maxentropic reconstruction with cubic splines from m = 4, in
[−5, 5], w = π, σ = 0.01, T1 = 1.5, T2 = 2, T3 = 3, T4 = 4
and T5 = 5 with the respective error ε1 = 0.020, ε2 = 0.020,
ε3 = 0.030, ε4 = 0.040 and ε5 = 0.050. . . . . . . . . . . . . . 137
7.33 Maxentropic reconstruction with cubic splines from m = 8, in
[−5, 5], w = π, σ = 0.1, T1 = 0.5, T2 = 2, T3 = 3 and T4 = 4
with the respective error ε1 = 0.20, ε2 = 0.20, ε3 = 0.30 and
ε4 = 0.40. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
7.34 Maxentropic reconstruction with cubic splines from m = 8, in
[−5, 5], w = π, σ = 0.01, T1 = 0.5, T2 = 1, T3 = 1.5 and T4 = 2
with the respective error ε1 = 0.0131, ε2 = 0.0135, ε3 = 0.020
and ε4 = 0.020. . . . . . . . . . . . . . . . . . . . . . . . . . . 138
7.35 Maxentropic reconstruction with cubic splines from m = 8, in
[−1, 1], w = 2π, σ = 0.01, T1 = 0.5, T2 = 1, T3 = 2 and T4 = 3
with the respective error ε1 = 0.0118, ε2 = 0.01, ε3 = 0.02 and
ε4 = 0.03. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
7.36 Maxentropic reconstruction with cubic splines from m = 8, in
[−2, 2], w = 2π, σ = 0.01, T1 = 0.5, T2 = 1, T3 = 1.5 and
T4 = 4 with the respective error ε1 = 0.005, ε2 = 0.010 and
ε3 = 0.020. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
7.37 Maxentropic reconstruction with cubic splines and method No.
2, in [−25, 25], from m = 12, w = 2π, σ = 0.01, µ0 = 0.01,
µ1 = 0, µ2 = 0.005 and µ3 = 0.02 with the respective error
ε1 = 0.0274, ε2 = 0.0326, and ε3 = 0.745. . . . . . . . . . . . . 140
7.38 Maxentropic reconstruction with cubic splines and method No.
2, in [−80, 80], from m = 12, w = 4π, σ = 0.01, µ0 = 0.01,
µ1 = 0, µ2 = 0.005 and µ3 = 0.02 with the respective error
ε1 = 0.0246, ε2 = 0.030, and ε3 = 0.0734. . . . . . . . . . . . . 140
7.39 Reconstructions from m = 8, the respective L1 errors are σ1 =
0.5272 and σ2 = 0.6786 and the reconstruction error is ε1 =
6.1374 × 10−8
. . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
7.40 Reconstructions from m = 14, the respective L1 errors are
σ1 = 0.3811 and σ2 = 0.5654 and the reconstruction error is
ε1 = 8.2644 × 10−8
. . . . . . . . . . . . . . . . . . . . . . . . . 141](https://image.slidesharecdn.com/1162355-250529084159-1be53f4c/85/Linear-Inverse-Problems-The-Maximum-Entropy-Connection-Series-On-Advances-In-Mathematics-For-Applied-Sciences-83-Harcdr-Henryk-Gzyl-24-320.jpg)
![xviii Linear Inverse Problems: The Maxentropic Connection
7.41 Reconstructions from m = 6, σ = 0.1, T1 = 0.5, T2 = 1 and
T3 = 1.5 the respective L1 errors are δ1 = 0.6088, δ2 = 0.6269,
δ3 = 0.6502, and δ4 = 0.6340, and the reconstruction errors for
maximum entropy method are respectively ε1 = 0.05, ε2 = 0.1,
ε3 = 0.15. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
7.42 Reconstructions from m = 6, σ = 0.01, T1 = 0.25, T2 = 1 and
T3 = 3 the respective L1 errors are δ1 = 0.5852, δ2 = 0.5895,
δ3 = 0.5999, and δ4 = 0.6340, and the reconstruction errors
for maximum entropy method are respectively ε1 = 0.00255,
ε2 = 0.01, ε3 = 0.03. . . . . . . . . . . . . . . . . . . . . . . . 142
7.43 Original function and three reconstructions using standard ME
applied to 7, 8 or 9 givens moments. . . . . . . . . . . . . . . 143
7.44 Graph of φ∗
reconstructed from four different prices. . . . . . 164
9.1 Difference between data and reconstruction, using uniform a
priori measure. . . . . . . . . . . . . . . . . . . . . . . . . . . 206
9.2 Difference between data and reconstruction, using Bernoulli a
priori measure, with p = 0.85, q = 0.15. . . . . . . . . . . . . 206
9.3 Difference between data and reconstruction. . . . . . . . . . . 208
9.4 Maxentropic approach to a linear programming problem, with
p = 0.75, q = 0.25, γ = 50.2199. The reconstruction error
ε = 1.7831 × 10−11
. . . . . . . . . . . . . . . . . . . . . . . . . 210
10.1 Reconstruction of exp(−t) sin(2πt) using sine, Legendre and
Laguerre bases. . . . . . . . . . . . . . . . . . . . . . . . . . . 231
10.2 Reconstruction of exp(−t) sin(8πt) using sine, Legendre and
Laguerre bases. . . . . . . . . . . . . . . . . . . . . . . . . . . 232
10.3 Reconstruction of sin(2πt) using sine, Legendre and Laguerre
bases. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232
10.4 Reconstruction of sin(8πt) using sine, Legendre and Laguerre
bases. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233
10.5 Reconstruction of N [1 + sin(2πt)] exp(−t) and
N [1 + sin(8πt)] exp(−t) using standard maximum entropy. . . 233
10.6 Reconstruction of N [1 + sin(2πt)] and N [1 + sin(8πt)] using
standard maximum entropy. . . . . . . . . . . . . . . . . . . . 234
10.7 Maxentropic reconstruction with first order splines from m =
5 (0,2,4,6,8) β = 1, t0 = 1, σ1 = σ2 = 0.6931, for different
intervals of reconstruction I1 = [0, 1], I2 = [−1, 1] with errors
ε1 = 1.4438 × 10−6
, ε2 = 1.7652 × 10−6
. . . . . . . . . . . . . 234](https://image.slidesharecdn.com/1162355-250529084159-1be53f4c/85/Linear-Inverse-Problems-The-Maximum-Entropy-Connection-Series-On-Advances-In-Mathematics-For-Applied-Sciences-83-Harcdr-Henryk-Gzyl-25-320.jpg)
![List of Figures xix
10.8 Maxentropic reconstruction with first order splines from m =
5 (0,2,4,6,8) β = 1, t0 = 1, σ1 = σ2 = 0.6931, for different
intervals of reconstruction I1 = [0, 1.5], I2 = [−2, 2] with errors
ε1 = 1.2994 × 10−4
, ε2 = 1.4573 × 10−6
. . . . . . . . . . . . . 235
10.9 Maxentropic reconstruction with first order splines from m =
10 (0,1,2,3,4,5,6,7,8,9), ω = 1, σ = 0.6931 with error ε =
1.7567 × 10−5
. . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
10.10 Multiplying by exp(βt) the functions showed in figure 10.2. . . 236
10.11 Maxentropic reconstructions with first order splines from
m = 15 (0, 2, 4, ..., 26, 28), ω = 2π, with different σ’s values.
Respectively values of σ’s and errors are listed: σ1 = 0.009,
ε1 = 3.28 × 10−6
; σ2 = 0.09, ε2 = 0.0014; σ3 = 0.9,
ε3 = 0.0025. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237
10.12 Maxentropic reconstructions with first order splines from
m = 15 (0,2,4,...,26,28), ω = 8π, with different σ’s values. Re-
spectively values of σ’s and errors are listed: σ1 = 0.009,
ε1 = 1.22 × 10−7
; σ2 = 0.09, ε2 = 4.68 × 10−5
; σ3 = 0.9,
ε3 = 6.25 × 10−5
. . . . . . . . . . . . . . . . . . . . . . . . . . 237
12.1 The original picture. . . . . . . . . . . . . . . . . . . . . . . . 250
12.2 The latitude (LAT) is equal to the height of the pole P above
the horizon. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252
12.3 Here we define variables for the analysis. We identified some
of the stars in the picture. The South Pole is identified with
a letter P. The star pointed with the arrow is the most weak
start that the author could identify on the original photo. . . 254
12.4 Southern Hemisphere’s Map, used to determine the photo’s
date. The date is displayed in the external bound (Norton,
1978). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256](https://image.slidesharecdn.com/1162355-250529084159-1be53f4c/85/Linear-Inverse-Problems-The-Maximum-Entropy-Connection-Series-On-Advances-In-Mathematics-For-Applied-Sciences-83-Harcdr-Henryk-Gzyl-26-320.jpg)
![Introduction 3
with smallest energy. In other instances one may be forced to shrug and
say: well it works and I don’t have anything else.
The basic heuristic behind the MEM, developed in Chapter 5, consists
of considering the set C of possible reconstructions as values of a random
variable X. Instead of devising methods to find an explicit solution to the
given equation, we search for a probability distribution P on C such that
AEP [X] satisfies either (1.1) or (1.2). Here EP [X] =
XdP denotes the
mean value of X with respect to P.
The problem is how to choose one among all possible P’s that do the
job. It is here that the concept of entropy enters: in analogy with (1.3),
the entropy S(P) will be a concave functional, defined on the set of all
probability measures which is a convex set. Thus, the problem becomes
one of maximization of a concave functional on some convex set. Hence the
name of the method.
To be fair, we have to ask why the specific functional S(P) chosen in
Chapter 5 works. Despite the effort in making things appear “natural” (not
yet in a functional sense), all we can say is that the procedure works.
Originally, the variational method was proposed by Jaynes to lay the
foundations of statistical physics, where it provided an astounding bridge
between micro- and macro-physics. The list of problems in which the
method works is still growing. See Section 6.5 of Chapter 6 and take a look
at references [12]-[27] for an interesting list of a variety of applications. Or
look at the journal “Entropy” for more.
Now, since the: Why does it work? is tied to assembling a whole frame-
work in which it does, we could as well as ourselves: Why that framework?
To avoid longer regressions, the best answer so far is that it works.
It will be up to you, dear reader to help explaining why. Or at least, to
have fun applying it.
Chapter 6 is devoted to the finite dimensional problems arising from the
discretization of continuous problems or the truncation of infinitely many
dimensional ones, and related issues. Some basic formulae are obtained
there. We devote Chapter 7 to present some numerical examples.
In Chapter 8 we carry out the program outlined in Chapter 5 in a truly
infinite dimensional setup. In particular, we shall see that some regular-
ization methods in Hilbert space can be obtained from the maxentropic
approach.
We shall devote Chapter 9 to review some results about reconstructing
a function in the plane when its integral along a few lines is known. We end
the chapter examining the reconstruction of the entries in a table from their](https://image.slidesharecdn.com/1162355-250529084159-1be53f4c/85/Linear-Inverse-Problems-The-Maximum-Entropy-Connection-Series-On-Advances-In-Mathematics-For-Applied-Sciences-83-Harcdr-Henryk-Gzyl-31-320.jpg)
![A collection of linear inverse problems 11
We shall have more to say about both approaches. To finish note that
a conformal mapping transforming the right-hand-complex half-plane onto
the unit circle allows us to relate the second approach to the famous:
Pick-Nevanlina interpolation problem: Given two finite sets of
complex numbers z1, ..., zn and ˜
f1, ..., ˜
fn; all of absolute value less than 1,
find a holomorphic function h : U → U on the unit disk such that:
h(zi) = ˜
fi i = 1, ..., n.
2.5 Fourier reconstruction from partial data
Let f(t) be a real-valued, function defined on an interval [0, T ]. It is known
that
T
0
exp(iωt)f(t)dt = ˆ
f(ω) (2.19)
for ω in the finite set {±ω1, ±ω2, ..., ±ωn} . The problem is to find f(t).
Sometimes one may want to think of ˆ
f(ω) as being the Fourier transform
of a measure dF(t) not necessarily absolutely continuous with respect to
dt. For example:
dF(t) =
pnδ(t − tn)dt
where pn are “weights” and tn are the points at which the weights are
concentrated.
This type of problem appears when one studies second order stationary
processes. Here one determines correlations R(k) from the data and one
searches for a (spectral) measure dF(t) on [0, 2π] such that:
R(k) =
[0,2π]
exp(ikλ)dF(λ), |k| ≤ N. (2.20)
Another problem in which one knows a Fourier transform occurs when
reconstructing velocity profiles in stratified models of the Earth. We refer
the reader to the references quoted at the end of the chapter. The problem
we are interested in is to find a(x) of compact support in [0, ∞) such that:
â(ω) =
∞
0
exp(iωx)a(x)dx (2.21)
is known. Here the interest lies in finding a(x)’s satisfying a1 ≤ a(x) ≤ a2.
These bounds can be justified since the speed of propagation of sound is](https://image.slidesharecdn.com/1162355-250529084159-1be53f4c/85/Linear-Inverse-Problems-The-Maximum-Entropy-Connection-Series-On-Advances-In-Mathematics-For-Applied-Sciences-83-Harcdr-Henryk-Gzyl-39-320.jpg)
![12 Linear Inverse Problems: The Maxentropic Connection
bounded below (by the speed of propagation of sound in the air) and above
(by the speed of propagation in some very rigid material).
Different types of constraints appear when one reconstructs probability
densities ρ(r) from their Fourier transforms. Here one knows F̂(p) such
that:
F̂(p) =
R5
exp(ipr)ρ(r)dr (2.22)
and one wants ρ(r) ≥ 0 such that
ρ(r)dr = 1. Of course, if F̂(p) is known
for all p in R3
, one would just invert the Fourier transform. But usually
F̂(p) is known only for finitely many values of p, and measurement errors
must be dealt with on top of it.
A classical example relating the issues of the choice of norms and the
summation of Fourier series is the following. Consider the mapping:
A : C[0, 1] → l2 defined by f(t) → cn =
1
2π
1
0
exp(2iπnt)f(t)dt.
Let f(t) ∈ C[0, 1] and let
cn exp(−2πint) be its Fourier representa-
tion.
Let {dn} ∈ l2 be given by dn = cn + ε
|n| , thus
d − c =
(dn − cn)2
1
2
= ε
π2
6
which can be made as small as desired, but if g(t) =
dn exp(−2iπnt)
then
f − g∞ = ε
1
n exp(−2πint)
∞
.
Since at t = 1 we obtain the harmonic series, f − g∞ = ∞. If you
(rightly) feel we seem to be cheating, just consider a sequence dN
n = cn + ε
n
for |n| ≤ N, and zero otherwise. Then
dN
n
is as close as you want to {cn}
and gN
(t) is well defined, but
gN
− f
∞
is as large as you want.
The moral is, even though A is continuous, A−1
is not.
2.6 More on the non-continuity of the inverse
A classical example of non-continuity of inverses is the following. On C[0, 1]
provided with its usual l∞ norm define T : C[0, 1] → C[0, 1] by:
(T x)(t) =
t
0
x(s)ds.](https://image.slidesharecdn.com/1162355-250529084159-1be53f4c/85/Linear-Inverse-Problems-The-Maximum-Entropy-Connection-Series-On-Advances-In-Mathematics-For-Applied-Sciences-83-Harcdr-Henryk-Gzyl-40-320.jpg)
![A collection of linear inverse problems 13
Certainly the range of T is the class C1
[0, 1] of continuously differen-
tiable functions on (0, 1).
If y1(t) is continuously differentiable, let y2(t) = y1(t) + A sin(ωt). Note
that y1 − y2∞ = |A| but x2 − x1∞ = y
2 − y
1∞ = |Aω| which can be
made as large as desired keeping |A| as small as desired as well.
But note that if we consider T : C1
[0, 1] → C[0, 1] and on C1
[0, 1] we
put the norm:
y1 − y2∗
= y1 − y2∞ + y
1 − y
2∞
then T has a continuous inverse T −1
(when restricted to C1
[0, 1] !).
Comment: If on C[0, 1] we define the σ-algebra F that makes the
coordinate mapping Xt : C[0, 1] → R, Xt(ω) = ω(t) measurable, and on
(C[0, 1], F) we define the usual Wiener measure, then C1
[0, 1] has measure
zero in C[0, 1] !
2.7 Transportation problems and reconstruction from
marginals
The simplest transportation problem consists of shipping gods from m
sources, in amounts si from the i-th source. Goods are to be received
at n destinations, in required amounts dj at the j-th one.
If there is a cost cij involved in the shipping from origin i to destination
j, the simple transportation problem consists of finding:
x∗
ij = inf
m,n
i,j=1,1
cijxij : xij ≥ 0,
n
j=1
xij = si,
m
i=1
xij = dj
.
This problem is usually solved in a few seconds by many software pack-
ages. Actually it usually takes longer to translate a given problem into that
set up, to key it in at the keyboard, than what it takes the PC to provide
the solution.
What is important for us here is the issue of the existence of feasible
solutions, i.e., given: {si : i = 1, 2, ..., m} and {dj : j = 1, 2, ..., n} find xij ≥
0, or aij ≤ xij ≤ bij for preassigned aij, bij, such that:
m
i=1
xij = dj,
n
j=1
xij = si
and consistency requires that
dj =
si.](https://image.slidesharecdn.com/1162355-250529084159-1be53f4c/85/Linear-Inverse-Problems-The-Maximum-Entropy-Connection-Series-On-Advances-In-Mathematics-For-Applied-Sciences-83-Harcdr-Henryk-Gzyl-41-320.jpg)
![A collection of linear inverse problems 15
Actually, since the class:
℘(m, n) = {ρ measure on (Z, Γ) : (2.23) holds}
is a convex set, trivially non-empty for m ⊗ n belongs to it, any lower
semicontinuous, convex function F : ℘(m, n) → R is likely to provide us
with candidates solving (2.23) when some compactness is available.
A variation on the theme of (2.23) and usually called the problem of
tomographic reconstruction consists of finding a positive function ρ on some
compact K of a metric space from the knowledge of:
ρ(ξ)µi(dξ) = di (2.24)
where the measures µi(dξ) are usually concentrated on sets of measure zero
with respect to a given measure m(dξ) on (K, β), β denoting the Borel sets
in K.
2.8 CAT
CAT stands for either computer axial tomography or computer aided to-
mography. Either name describes a complex process involves obtaining
X-ray images of an object.
The technique consists of a combination of hardware-software by means
of which an image of a series of consecutive slices (tomos) of an object,
usually part of a human body, are obtained. The image consists of an
X-ray plaque of each given section (slice). Different aspects of the his-
toric development of the tomographic process are contained in [10]-[13].
In [11] you will find a brief account of the story from the point of view
of the research-development-commercialization process, whereas the other
two emphasize the connection with the evolution of the development of the
mathematics of the inverse problem that is needed to present the images of
the slices.
Let us begin by describing the phenomenological model of light ab-
sorption by tissue. Everything is contained in the following simple, one-
dimensional model: consider a slab of material, of linear density ρ(x) ex-
tending from 0 to L. Let the radiation incident at x = 0 be of intensity I0
and denote by I(x) the intensity of the radiation reaching point x. Thus
the radiation reaching x + dx is
I(x + dx) = I(x) − λρ(x)I(x)dx](https://image.slidesharecdn.com/1162355-250529084159-1be53f4c/85/Linear-Inverse-Problems-The-Maximum-Entropy-Connection-Series-On-Advances-In-Mathematics-For-Applied-Sciences-83-Harcdr-Henryk-Gzyl-43-320.jpg)
![16 Linear Inverse Problems: The Maxentropic Connection
from which we obtain that the radiation at x = L is
I(L) = I0 exp
−
L
0
λρ(x)dx
. (2.25)
We shall follow [10] for the description of conventional tomography and
to present the reader with one more set of references to the subject. Ac-
cording to [13] it seems that the basics were patented by the Frenchman
Bocade around 1921. Consider two parallel planes, separated by a distance
L.
0 L
V1
V2
P1 P2
Fig. 2.1 Basic scheme.
Consider two points 1 and 2 on each plane. After some time ∆ they
will be at 1
and 2
respectively, with 11
= V1∆ and 22
= V2∆.
Using this as starting point, we leave it to the reader to prove that there
is a plane Pf which stays fixed. Write V2 = αV1 and determine the position
of the plane in terms of α.
Imagine now a source of X-rays on plane 1 and a photographic plate
on plane 2. Since the motion of the planes is much, much slower than
the speed of light, we might imagine that situation depicted in figure 2.2
holds instantaneously, and in a “more realistic” schematic depiction of an
X-raying process we will have something like figure 1 in [10].
Clearly, the details in the fixed plane get more (negatively) illuminated
than objects in non-fixed planes, so they will appear less blurred in the ex-
posed plate. If this process is repeated for different relative source-plate po-](https://image.slidesharecdn.com/1162355-250529084159-1be53f4c/85/Linear-Inverse-Problems-The-Maximum-Entropy-Connection-Series-On-Advances-In-Mathematics-For-Applied-Sciences-83-Harcdr-Henryk-Gzyl-44-320.jpg)
![20 Linear Inverse Problems: The Maxentropic Connection
Comment: It is left to the reader to convince himself/herself that
(2.31) describes marginalization of ρ along successively rotated axes. Thus
standard reconstructions from marginals fall within the class of problems
described above. We will examine some aspects of these problems in Chap-
ter 9.
2.9 Abstract spline interpolation
The classical interpolation problem consists of finding a σ ∈ C2
[0, 1] such
that σ(ti) = zi for i = 1, 2, ..., n. The procedure consists in minimizing:
F(σ) =
1
0
(σ
(t))2
dt (2.32)
and the result is contained in:
Theorem 2.1. There is a unique σ ∈ C2
[0, 1] minimizing (2.32) such that:
a) σ(ti) = zi.
b) σ is a cubic polynomial in [ti, ti+1] , i = 1, 2, ..., n − 1.
c) σ
(t1) = σ
(tn) = 0.
d) σ is linear on [0, t1] and [tn, 1] .
Instead of the interpolation conditions σ(ti) = zi for i = 1, 2, ..., n ; one
could search for a function σ satisfying:
1
0
σ(t)Pl(t)ω(t)dt = zi (2.33)
where ω(t) is some weight function and the {Pi(t), i ≥ 1} are the orthogo-
nal polynomials with respect to ω(t), i.e., they are orthogonal in 2([0, 1], ω).
The problem can be abstracted into the following setup: Let X, Y, Z be
Hilbert spaces. Let A : X → Y and B : X → Z be two bounded, linear
surjections. Let y0 ∈ Y and the problem consists of finding x∗
0 ∈ X such
that
x∗
0 = arg inf {BxZ : Ax = y0} . (2.34)
The existence and uniqueness of x∗
0 is covered by:
Theorem 2.2. If
a) B(Ker(A)) is closed in Z.](https://image.slidesharecdn.com/1162355-250529084159-1be53f4c/85/Linear-Inverse-Problems-The-Maximum-Entropy-Connection-Series-On-Advances-In-Mathematics-For-Applied-Sciences-83-Harcdr-Henryk-Gzyl-48-320.jpg)
![A collection of linear inverse problems 21
b) Ker(B) ∩ Ker(A) = {0X}. (0X denotes the zero element of X.)
c) y0 ∈ Y.
Then there is a unique x0 such that Ax0 = y0 and
B(x0)Z = inf {BxZ : Ax = y} .
Proof. A being surjective, implies the existence of x1 ∈ X such that
A−1
{y0} = x1 + Ker(A).
Thus B(A−1
{x0}) = B(x1) + B(Ker(A)).
From the continuity of A and B we obtain the closedness of B(Ker(A))
and hence, B(A−1
{x0}), which being a translate of a subspace is convex.
Let z0 denote the projection of 0Z onto B(A−1
{x0}), thus
z0 ∈ B(x1) + B(A−1
{x0})
which means that there exists x ∈ Ker(A) such that z0 = B(x1) + B(x) =
B(x1 + x) or there exists x ∈ Ker(A) such that x1 + x ∈ B−1
{z0}. But,
A(x + x1) = A(x1) + A(x) = y0 + 0X = y0
or x + x1 ∈ A−1
{y0} ∩ B−1
{z}, i.e., this set is not empty. Now let x1, x2
be two different elements in
A−1
{y0} ∩ B−1
{z}.
Then,
B(x1 − x2) = 0Z, A(x1 − x2) = 0Y
plus assumption b) implies that x1 = x2 or A−1
{y0} ∩ B−1
{z} is a single-
ton, which we denote by x0.
2.10 Bibliographical comments and references
Generic comment: References will be cited as [n] or [m-n]. The first is a
reference n at the end of the present chapter, whereas [m-n] denotes the
nth reference in the list at the end of Chapter m.
A couple of new (as of 1998) books in which finite dimensional inverse
problems are treated in full are: [1] and [2]. A nice collection of inverse
problems and the basic theory to deal with them is exposed in [2] and in [3].
Even more theory and applications, both linear and nonlinear appear in [4],
[5] and [6]. Two related references on the trigonometric moment problem
are [7], [7-7] and [7-8]. On the transportation problem take a look at any](https://image.slidesharecdn.com/1162355-250529084159-1be53f4c/85/Linear-Inverse-Problems-The-Maximum-Entropy-Connection-Series-On-Advances-In-Mathematics-For-Applied-Sciences-83-Harcdr-Henryk-Gzyl-49-320.jpg)
![22 Linear Inverse Problems: The Maxentropic Connection
book on Operations Research. For information about tomography, its his-
tory, and further references, consult with [8]-[13], and with many of the
papers described in Chapter 9.
References [14]-[16] contain historical remarks and description of the
evolution of attempts to solve the inverse problem of CAT and related
imaging problems in various fields as well as a list of further references.
For related work on transportation problems consider references [9-18]
and [9-27] and for a nice and short expose on abstract splines take a look
at [17] from which we took material for Section 9.10.
References
[1] Mansen, P. C. “Rank-Deficient and Discrete Ill-Posed Problems”.
SIAM, Philadelphia, 1998.
[2] Bertero, M. and Bocacci, P. “Introduction to Inverse Problems in
Imaging”. IOP Publishing, Bristol, 1998.
[3] Parker, R. “Geophysical Inverse Theory”. Princeton Univ. Press,
Princeton 1994.
[4] Ramm, A. G. “Scattering by Obstacles”. Kluwer Acad. Pub., Dor-
drecht, 1986.
[5] Ramm, A. G. “Multidimensional Inverse Problems”. Longman Scien-
tific, 1992.
[6] Laurentiev, M. M. “Some Improperly Posed Problems of Mathematical
Physics”. Springer-Verlag. 1967.
[7] Romanov, V. G. “Inverse Problems of Mathematical Physics”. VNU
Science Press, Utrecht, 1987.
[8] Orphanoudakis, S. and Strohben, J. “Mathematical model of conven-
tional tomography”. Med. Phys., Vol. 3 (1976), pp. 224-232.
[9] Adams, J. L. “Flying Buttresses, Entropy and O. Rings”. Harvard
Univ. Press, Cambridge, 1991.
[10] Shepp, L. A. and Kruskal, J. B. “Computerized Tomography: the new
medical X-ray technology”. Ann. Math. Monthly, Vol. 85 (1978) pp.
420-439.
[11] Panton, D. “Mathematical reconstruction techniques in computer axial
tomography”. Math. Scientist, Vol. 6 (1981), pp. 87-102.
[12] Fuchs, J., Mast, K., Hermann, A. and Lackner, K. “Two-dimensional
reconstruction of the radiation power density in ASDEX upgrade”.
21st. E.P.S. Conf. on Controlled Fusion and Plasma Physics. Joffrin,
E., Platz, P. and Scott, P. (eds), Publ. E.P.S., Paris, 1994.](https://image.slidesharecdn.com/1162355-250529084159-1be53f4c/85/Linear-Inverse-Problems-The-Maximum-Entropy-Connection-Series-On-Advances-In-Mathematics-For-Applied-Sciences-83-Harcdr-Henryk-Gzyl-50-320.jpg)
![A collection of linear inverse problems 23
[13] Friedman, A. “Image reconstruction in oil refinery” in Mathematics in
Industrial Problems, IMA Volumes in Mathematics and its Applica-
tions, Vol. 16, Springer-Verlag, 1988.
[14] Hounsfield, G. “Historical notes on computerized axial tomography”.
The J. of the Canadian Assos. of Radiologists, Vol. 27 (1976), pp.135-
191.
[15] Brooks, R and Di Chiro, G. “Principles of computer assisted tomog-
raphy (CAT) in radiographic and radioisotopic imaging”. Phys. Med.
Biol., Vol. 21 (1976), pp. 689-732.
[16] Gordon, R. and Herman, G. “Three dimensional reconstruction from
projections: a review of algorithms”. Inter. J. Cytology, Vol. 39
(1974), pp. 111-151.
[17] Champion, R., Lenard, T. C. and Mills, T. M. “An introduction to
abstract splines”, Math. Scientist. Vol.21, (1996), pp. 8-26.](https://image.slidesharecdn.com/1162355-250529084159-1be53f4c/85/Linear-Inverse-Problems-The-Maximum-Entropy-Connection-Series-On-Advances-In-Mathematics-For-Applied-Sciences-83-Harcdr-Henryk-Gzyl-51-320.jpg)
![The basics about linear inverse problems 27
worse, the {xn} can be chosen so that xn → ∞. In particular if A is 1:1,
the inverse A−1
: W1 = A(V ) ⊂ W → V is unbounded.
Proof. The operator  : V̂ → Y induced by A on the factor space
V̂ = V/K, defined by Â[x] = Ax is compact and 1:1. If Â−1
: W1 → V̂ ,
then Â−1
cannot be bounded.
If it were, I = Â−1
 and V̂ → V̂ would be compact which is impossible
if V̂ is infinite dimensional.
Since Â−1
is unbounded, there exists a sequence {xn : n ≥ 1} in V and
a corresponding sequence {[xn] : n ≥ 1} in V̂ such that Axn → 0 but
[xn] = 1. Let ξn ∈ K be such that xn + ξn ≥ a 1 and put x
n =
xn + ξn
Axn
, Ax
n → 0 but x
n ≥
a
Axn
→ ∞ as n → ∞.
Comments: To avoid contradiction with the comments above the
statement of the Theorem, either
i) V̂ is finite dimensional (and Â−1
is bounded) or
ii) V̂ is infinite dimensional and W1 is of first category in W. It can be
represented as union of a sequence of nowhere dense sets.
Besides the issues described above, there are two more classes of issues
that are involved in solving (3.1).
The first class is related to the fact that both, the model A and the
data y, depend on actual measurements and our problem does not merely
consist in solving (3.1) but in giving meaning to the problem.
(P) Find x ∈ V1 such that Ax = y where A ∈ N(A0, δ) and y ∈ B(y0,ε)
Here y ∈ B(y0, ε) is clear: it denotes the ball of radius ε around y0 in
the . . .W distance. The indeterminacy in A may be due to indeterminacy
in some physical parameters, and thus N(Ao, δ) denotes some appropriate
neighborhood of A0. Tomographic reconstruction, see [9-24] is a typical
situation of this kind.
The other issue compounding the difficulties is related to the fact that
in applications, both x and y lie in infinite dimensional spaces, and that
to describe B and By one has to resort to finite dimensional setups. To
be more precise, one has to be able to produce families of projections Pn :
V → Vn and Qn : W → Wn, where Vn and Wn are of appropriate finite
dimension, n, and such that x − PnxV → 0 for every x, y − QnyW → 0](https://image.slidesharecdn.com/1162355-250529084159-1be53f4c/85/Linear-Inverse-Problems-The-Maximum-Entropy-Connection-Series-On-Advances-In-Mathematics-For-Applied-Sciences-83-Harcdr-Henryk-Gzyl-54-320.jpg)
![was well received by all good men, and all gave him treasure, and in
Scotland also, and they sent in all a great sum in gold and silver by
him to Jerusalem, and there went with him and after him so great a
number as never before since the days of Pope Urban.”[5] Grants of
lands, as well as of money, were at the same time made to Hugh de
Payens and his brethren, some of which were shortly afterwards
confirmed by King Stephen on his accession to the throne, (A. D.
1135.) Among these is a grant of the manor of Bistelesham made to
the Templars by Count Robert de Ferrara, and a grant of the church
of Langeforde in Bedfordshire made by Simon de Wahull, and Sibylla
his wife, and Walter their son.
Hugh de Payens, before his departure, placed a Knight Templar at
the head of the order in this country, who was called the Prior of the
Temple, and was the procurator and vicegerent of the Master. It was
his duty to manage the estates granted to the fraternity, and to
transmit the revenues to Jerusalem. He was also delegated with the
power of admitting members into the order, subject to the control
and direction of the Master, and was to provide means of transport
for such newly-admitted brethren to the far east, to enable them to
fulfil the duties of their profession. As the houses of the Temple
increased in number in England, sub-priors came to be appointed,
and the superior of the order in this country was then called the
Grand Prior, and afterwards Master of the Temple.
An astonishing enthusiasm was excited throughout Christendom in
behalf of the Templars; princes and nobles, sovereigns and their
subjects, vied with each other in heaping gifts and benefits upon
them, and scarce a will of importance was made without an article in
it in their favour. Many illustrious persons on their deathbeds took
the vows, that they might be buried in the habit of the order; and
sovereign princes, quitting the government of their kingdoms,
enrolled themselves amongst the holy fraternity, and bequeathed
even their dominions to the Master and the brethren of the Temple.
St. Bernard, at the request of Hugh de Payens, took up his powerful
pen in their behalf. In a famous discourse “In praise of the New](https://image.slidesharecdn.com/1162355-250529084159-1be53f4c/85/Linear-Inverse-Problems-The-Maximum-Entropy-Connection-Series-On-Advances-In-Mathematics-For-Applied-Sciences-83-Harcdr-Henryk-Gzyl-60-320.jpg)
![Burgundian, (son-in-law of Anselm, Archbishop of Canterbury,) who,
after the death of his wife, had taken the vows and the habit of the
Templars.[6] At this period the fierce religious and military
enthusiasm of the Mussulmen had been again aroused by the
warlike Zinghis, and his son Noureddin, two of the most famous
chieftains of the age. The one was named Emod-ed-deen, “Pillar of
religion;” and the other Nour-ed-deen, “Light of Religion,” vulgarly,
Noureddin. The Templars were worsted by overpowering numbers.
The latin kingdom of Jerusalem was shaken to its foundations, and
the oriental clergy in trepidation and alarm sent urgent letters to the
Pope for assistance.
The Lord Robert, Master of the Temple, had at this period (A. D.
1146) been succeeded by Everard des Barres, Prior of France, who
convened a general chapter of the order at Paris, which was
attended by Pope Eugenius the Third, Louis the Seventh, king of
France, and many prelates, princes, and nobles, from all parts of
Christendom. The second crusade was there arranged, and the
Templars, with the sanction of the Pope, assumed the blood-red
cross, the symbol of martyrdom, as the distinguishing badge of the
order, which was appointed to be worn on their habits and mantles
on the left side of the breast over the heart, whence they came
afterwards to be known by the name of the Red Friars and the Red
Cross Knights. At this famous assembly various donations were
made to the Templars, to enable them to provide more effectually
for the defence of the Holy Land. Bernard Baliol, through love of God
and for the good of his soul, granted them his estate of Wedelee, in
Hertfordshire, which afterwards formed part of the preceptory of
Temple Dynnesley. This grant is expressed to be made at the
chapter held at Easter, in Paris, in the presence of the Pope, the king
of France, several archbishops, and one hundred and thirty Knights
Templars clad in white mantles.[7]
Brother Everard des Barres, the newly-elected Master of the Temple,
having collected together all the brethren from the western
provinces, joined the second crusade to Palestine. During the march](https://image.slidesharecdn.com/1162355-250529084159-1be53f4c/85/Linear-Inverse-Problems-The-Maximum-Entropy-Connection-Series-On-Advances-In-Mathematics-For-Applied-Sciences-83-Harcdr-Henryk-Gzyl-66-320.jpg)
![through Asia Minor, the rear of the christian army was protected by
the Templars, who greatly signalized themselves on every occasion.
Odo of Deuil, or Diagolum, the chaplain of King Louis, and his
constant attendant upon this expedition, informs us that the king
loved to see the frugality and simplicity of the Templars, and to
imitate it; he praised their union and disinterestedness, admired
above all things the attention they paid to their accoutrements, and
their care in husbanding and preserving their equipage and
munitions of war, and proposed them as a model to the rest of the
army.[8]
Conrad, emperor of Germany, had preceded King Louis at the head
of a powerful army, which was cut to pieces by the infidels in the
north of Asia; he fled to Constantinople, embarked on board some
merchant vessels, and arrived with only a few attendants at
Jerusalem, where he was received and entertained by the Templars,
and was lodged in the Temple in the Holy City. Shortly afterwards
King Louis arrived, accompanied by the new Master of the Temple,
Everard des Barres; and the Templars now unfolded for the first time
the red-cross banner in the field of battle. This was a white standard
made of woollen stuff, having in the centre of it the blood-red cross
granted by Pope Eugenius. The two monarchs, Louis and Conrad,
took the field, supported by the Templars, and laid siege to the
magnificent city of Damascus, “the Queen of Syria,” which was
defended by the great Noureddin, “Light of religion,” and his brother
Saif-eddin, “Sword of the faith.”
The services rendered by the Templars are thus gratefully recorded
in the following letter sent by Louis, the French king, to his minister
and vicegerent, the famous Suger, abbot of St. Denis: “I cannot
imagine how we could have subsisted for even the smallest space of
time in these parts, had it not been for their (the Templars’) support
and assistance, which have never failed me from the first day I set
foot in these lands up to the time of my despatching this letter—a
succour ably afforded and generously persevered in. I therefore
earnestly beseech you, that as these brothers of the Temple have](https://image.slidesharecdn.com/1162355-250529084159-1be53f4c/85/Linear-Inverse-Problems-The-Maximum-Entropy-Connection-Series-On-Advances-In-Mathematics-For-Applied-Sciences-83-Harcdr-Henryk-Gzyl-67-320.jpg)
![hitherto been blessed with the love of God, so now they may be
gladdened and sustained by our love and favour. I have to inform
you that they have lent me a considerable sum of money, which
must be repaid to them quickly, that their house may not suffer, and
that I may keep my word....”[9]
Among the English nobility who enlisted in the second crusade were
the two renowned warriors, Roger de Mowbray and William de
Warrenne. Roger de Mowbray was one of the most powerful and
warlike of the barons of England, and was one of the victorious
leaders at the famous battle of the standard: he marched with King
Louis to Palestine; fought under the banners of the Temple against
the infidels, and, smitten with admiration of the piety and valour of
the holy warriors of the order, he gave them, on his return to
England, many valuable estates and possessions. Among these were
the manors of Kileby and Witheley, divers lands in the isle of
Axholme, the town of Balshall in the county of Warwick, and various
places in Yorkshire: and so munificent were his donations, that the
Templars conceded to him and to his heirs various special privileges.
About the same period, Stephen, King of England, granted and
confirmed “to God and the blessed Virgin Mary, and to the brethren
of the Knighthood of the Temple of Solomon at Jerusalem, all the
manor of Cressynge, with the advowson of the church of the same
manor, and also the manors of Egle and Witham.” Queen Matilda,
likewise, granted them the manor of Covele or Cowley in
Oxfordshire, two mills in the same county, common of pasture in
Shotover forest, and the church of Stretton in Rutland. Ralph de
Hastings and William de Hastings also gave to the Templars, in the
same reign, (A. D. 1152,) lands at Hurst and Wyxham in Yorkshire,
afterwards formed into the preceptory of Temple Hurst. William
Asheby granted them the estate whereon the house and church of
Temple Bruere were afterwards erected; and the order continued
rapidly to increase in power and wealth in England and in all parts of
Europe, through the charitable donations of pious Christians.[10]](https://image.slidesharecdn.com/1162355-250529084159-1be53f4c/85/Linear-Inverse-Problems-The-Maximum-Entropy-Connection-Series-On-Advances-In-Mathematics-For-Applied-Sciences-83-Harcdr-Henryk-Gzyl-68-320.jpg)
![After the miserable failure of the second crusade, brother Everard
des Barres, the Master of the Temple, returned to Paris, with his
friend and patron Louis, the French king; and the Templars, deprived
of their chief, were now left, alone and unaided, to withstand the
victorious career of the fanatical Mussulmen. Their miserable
situation is pourtrayed in a melancholy letter from the treasurer of
the order, written to the Master, Everard des Barres, during his
sojourn at the court of the king of France, informing him of the
slaughter of the prince of Antioch and all his nobility. “We conjure
you,” says he, “to bring with you from beyond sea all our knights
and serving brothers capable of bearing arms. Perchance, alas! with
all your diligence, you may not find one of us alive. Use, therefore,
all imaginable celerity; pray forget not the necessities of our house:
they are such that no tongue can express them. It is also of the last
importance to announce to the Pope, to the king of France, and to
all the princes and prelates of Europe, the approaching desolation of
the Holy Land, to the intent that they succour us in person, or send
us subsidies.”
The Master of the Temple, however, instead of proceeding to
Palestine, abdicated his authority, and entered into the monastery of
Clairvaux, where he devoted the remainder of his days to the most
rigorous penance and mortification. He was succeeded (A. D. 1151)
by Bernard de Tremelay, a nobleman of an illustrious family in
Burgundy, in France, and a valiant and experienced soldier.[11]
Shortly after his accession to power, the infidels crossed the Jordan,
and advanced within sight of Jerusalem. Their banners waved on the
summit of the Mount of Olives, and the warlike sound of their kettle-
drums and trumpets was heard within the sacred precincts of the
holy city. They encamped on the mount over against the Temple;
and had the satisfaction of regarding from a distance the Beit Allah,
or Temple of the Lord, their holy house of prayer; but in a night
attack they were defeated with terrible slaughter, and were pursued
all the way to the Jordan, five thousand of their number being left
dead on the plain.](https://image.slidesharecdn.com/1162355-250529084159-1be53f4c/85/Linear-Inverse-Problems-The-Maximum-Entropy-Connection-Series-On-Advances-In-Mathematics-For-Applied-Sciences-83-Harcdr-Henryk-Gzyl-69-320.jpg)
![On the 20th of April, A. D. 1153, the Templars lost their great patron
Saint Bernard, who died in the sixty-third year of his age. On his
deathbed he wrote three letters in behalf of the order. The first was
addressed to the patriarch of Antioch, exhorting him to protect and
encourage the Templars, a thing which the holy abbot assures him
will prove most acceptable to God and man. The second was written
to Melesinda, queen of Jerusalem, praising her majesty for the
favour shown by her to the brethren of the order; and the third,
addressed to Brother André de Montbard, a Knight Templar, conveys
the affectionate salutations of St. Bernard to the Master and
brethren, to whose prayers he recommends himself.
The same year the Master of the Temple perished at the head of his
knights whilst attempting to carry the important city of Ascalon by
storm. Passing through a breach made in the walls, he penetrated
into the centre of the town, and was there surrounded and
overpowered. The dead bodies of the Master and his ill-fated knights
were exposed in triumph from the walls; and, according to the
testimony of an eye-witness, not a single Templar escaped.
De Tremelay was succeeded (A. D. 1154) by Brother Bertrand de
Blanquefort, a knight of a noble family of Guienne, called by William
of Tyre a pious and God-fearing man. On Tuesday, June 19, A. D.
1156, the Templars were drawn into an ambuscade whilst marching
with Baldwin, king of Jerusalem, near Tiberias, three hundred of the
brethren were slain on the field of battle, and eighty-seven fell into
the hands of the enemy, among whom was Bertrand de Blanquefort
himself, and Brother Odo, marshal of the kingdom. Shortly
afterwards, a small band of them captured a large detachment of
Saracens; and in a night attack on the camp of Noureddin, they
compelled that famous chieftain to fly, without arms and half-naked,
from the field of battle. In this last affair the name of Robert Mansel,
an Englishman, and Gilbert de Lacy, preceptor of the Temple of
Tripoli, are honourably mentioned.[12]](https://image.slidesharecdn.com/1162355-250529084159-1be53f4c/85/Linear-Inverse-Problems-The-Maximum-Entropy-Connection-Series-On-Advances-In-Mathematics-For-Applied-Sciences-83-Harcdr-Henryk-Gzyl-70-320.jpg)
![The fiery zeal and warlike enthusiasm of the Templars were
equalled, if not surpassed, by the stern fanaticism and religious
ardour of the followers of Mahomet. “Noureddin fought,” says his
oriental biographer, “like the meanest of his soldiers, saying, ‘Alas! it
is now a long time that I have been seeking martyrdom without
being able to obtain it.’ The Imaum Koteb-ed-din, hearing him on
one occasion utter these words, exclaimed, ‘In the name of God do
not put your life in danger, do not thus expose Islam and the
Moslems. Thou art their stay and support, and if (but God preserve
us therefrom) thou shouldest be slain, we are all undone.’ ‘Ah!
Koteb-ed-deen,’ said he, ‘what hast thou said, who can save Islam
and our country, but that great God who has no equal?’ ‘What,’ said
he, on another occasion, ‘do we not look to the security of our
houses against robbers and plunderers, and shall we not defend
RELIGION?’”[13] Like the Templars, Noureddin fought constantly with
spiritual and with carnal weapons. He resisted the world and its
temptations, by fasting and prayer, and by the daily exercise of the
moral and religious duties and virtues inculcated in the Koran. He
fought with the sword against the foes of Islam, and employed his
whole energies, to the last hour of his life in the enthusiastic and
fanatic struggle for the recovery of Jerusalem.[14] In his camp, all
profane and frivolous conversation was severely prohibited; the
exercises of religion were assiduously practised, and the intervals of
action were employed in prayer, meditation, and the study of the
Koran. “The sword,” says the prophet Mahomet, in that remarkable
book, “is the key of heaven and of hell; a drop of blood shed in the
cause of God, a night spent in arms, is of more avail than two
months of fasting and of prayer. Whosoever falls in battle, his sins
are forgiven him. At the day of judgment his wounds will be
resplendent as vermillion, and odoriferous as musk, and the loss of
limbs shall be supplied by the wings of angels and cherubims.”
Among the many instances of the fanatical ardour of the Moslem
warriors, are the following, extracted from the history of Abu
Abdollah Alwakidi, Cadi of Bagdad. “Methinks,” said a valiant Saracen](https://image.slidesharecdn.com/1162355-250529084159-1be53f4c/85/Linear-Inverse-Problems-The-Maximum-Entropy-Connection-Series-On-Advances-In-Mathematics-For-Applied-Sciences-83-Harcdr-Henryk-Gzyl-71-320.jpg)
![youth, in the heat of battle—“methinks I see the black-eyed girls
looking upon me, one of whom, should she appear in this world, all
mankind would die for love of her; and I see in the hand of one of
them a handkerchief of green silk, and a cap made of precious
stones, and she beckons me, and calls out, Come hither quickly, for I
love thee.” With these words, charging the Christian host, he made
havoc wherever he went, until at last he was struck down by a
javelin. “It is not,” said another dying Arabian warrior, when he
embraced for the last time his sister and mother—“it is not the
fading pleasure of this world that has prompted me to devote my life
in the cause of RELIGION, I seek the favour of God and his APOSTLE,
and I have heard from one of the companions of the prophet, that
the spirits of the martyrs will be lodged in the crops of green birds
who taste the fruits and drink of the waters of paradise. Farewell:
we shall meet again among the groves and fountains which God has
prepared for his elect.”[15]
The Master of the Temple, Brother Bertrand de Blanquefort, was
liberated from captivity at the instance of Manuel Comnenus,
Emperor of Constantinople. After his release, he wrote several letters
to Louis VII., king of France, describing the condition and prospects
of the Holy Land: the increasing power and boldness of the infidels;
and the ruin and desolation caused by a dreadful earthquake, which
had overthrown numerous castles, prostrated the walls and defences
of several towns, and swallowed up the dwellings of the inhabitants.
“The persecutors of the church,” says he, “hasten to avail
themselves of our misfortunes; they gather themselves together
from the ends of the earth, and come forth as one man against the
sanctuary of God.”
It was during his mastership, that Geoffrey, the Knight Templar, and
Hugh of Cæsarea, were sent on an embassy into Egypt, and had
their famous interview with the Caliph. They were introduced into
the palace of the Fatimites through a series of gloomy passages and
glittering porticos, amid the warbling of birds and the murmur of
fountains; the scene was enriched by a display of costly furniture](https://image.slidesharecdn.com/1162355-250529084159-1be53f4c/85/Linear-Inverse-Problems-The-Maximum-Entropy-Connection-Series-On-Advances-In-Mathematics-For-Applied-Sciences-83-Harcdr-Henryk-Gzyl-72-320.jpg)
![and rare animals; and the long order of unfolding doors was guarded
by black soldiers and domestic eunuchs. The sanctuary of the
presence chamber was veiled with a curtain, and the vizier who
conducted the ambassadors laid aside his scimitar, and prostrated
himself three times on the ground; the veil was then removed, and
they saw the Commander of the Faithful.[16]
Brother Bertrand de Blanquefort, in his letters to the king of France,
gives an account of the military operations undertaken by the order
of the Temple in Egypt, and of the capture of the populous and
important city of Belbeis, the ancient Pelusium.[17] During the
absence of the Master with the greater part of the fraternity on that
expedition, the sultan Noureddin invaded Palestine; he defeated with
terrible slaughter the serving brethren and Turcopoles, or light horse
of the order, who remained to defend the country, and sixty of the
knights who commanded them were left dead on the plain. Amalric,
king of Jerusalem, the successor of Baldwin the Third, in a letter “to
his dear friend and father,” Louis the Seventh, king of France,
beseeches the good offices of that monarch in behalf of all the
devout Christians of the Holy Land; “but above all,” says he, “we
earnestly entreat your Majesty constantly to extend to the utmost
your favour and regard to the Brothers of the Temple, who
continually render up their lives for God and the faith, and through
whom we do the little that we are able to effect, for in them indeed,
after God, is placed the entire reliance of all those in the eastern
regions who tread in the right path.”[18] The Master, Brother
Bertrand de Blanquefort, was succeeded, (A. D. 1167,) by Philip of
Naplous, the first Master of the Temple who had been born in
Palestine. He had been Lord of the fortresses of Krak and Montreal
in Arabia Petræa, and took the vows and the habit of the order of
the Temple after the death of his wife.[19]
We must now pause to take a glance at the rise of another great
religio-military institution which, from henceforth, takes a leading
part in the defence of the Latin kingdom. In the eleventh century,](https://image.slidesharecdn.com/1162355-250529084159-1be53f4c/85/Linear-Inverse-Problems-The-Maximum-Entropy-Connection-Series-On-Advances-In-Mathematics-For-Applied-Sciences-83-Harcdr-Henryk-Gzyl-73-320.jpg)
![According to De Vertot, the senior Hospitallers were greatly averse
to the military projects of their chief: “They urged,” says he, “that
they were a religious order, and that the church had not put arms
into their hands to make conquests;” but the younger and more
ardent of the brethren, burning to exchange the monotonous life of
the cloister for the enterprise and activity of the camp, received the
proposals of their superior with enthusiasm, and a majority of the
chapter decided in favour of the plans and projects of their
Guardian. They authorised him to borrow money of the Florentine
and Genoese merchants, to take hired soldiers into the pay of the
order, and to organize the Hospitallers as a great military society.
It was in the first year of the government of Philip of Naplous (A. D.
1168) that the king of Jerusalem and the Hospitallers marched forth
upon their memorable and unfortunate expedition. The Egyptians
were taken completely by surprise; the city of Belbeis was carried by
assault, and the defenceless inhabitants were barbarously
massacred. The cruelty and the injustice of the Christians, however,
speedily met with condign punishment. The king of Jerusalem was
driven back into Palestine; Belbeis was abandoned with precipitation;
and the Hospitallers fled before the infidels in sorrow and
disappointment to Jerusalem. There they vented their indignation
and chagrin upon the unfortunate Gilbert d’Assalit, their superior,
who had got the order into debt to the extent of 100,000 pieces of
gold; they compelled him to resign his authority, and the unfortunate
guardian of the hospital fled from Palestine to England, and was
drowned in the Channel. From this period, however, the character of
the order of the Hospital of St. John was entirely changed: the
Hospitallers appear henceforth as a great military body; their
superior styles himself Master, and leads in person the brethren into
the field of battle. Attendance upon the poor and the sick still
continued, indeed, one of the duties of the fraternity, but it must
have been feebly exercised amid the clash of arms and the
excitement of war.[20]](https://image.slidesharecdn.com/1162355-250529084159-1be53f4c/85/Linear-Inverse-Problems-The-Maximum-Entropy-Connection-Series-On-Advances-In-Mathematics-For-Applied-Sciences-83-Harcdr-Henryk-Gzyl-76-320.jpg)
![The Grand Master of the Temple, Philip of Naplous, resigned his
authority after a short government of three years, and was
succeeded (A. D. 1170) by Brother Odo de St. Amand, a proud and
fiery warrior, of undaunted courage and resolution; having,
according to William, Archbishop of Tyre, the fear neither of God nor
of man before his eyes.[21] It was during his Grand Mastership (A. D.
1172) that the Knight Templar Walter du Mesnil slew an envoy or
minister of the assassins. These were an odious religious sect,
settled in the fastnesses of the mountains above Tripoli, and
supposed to be descended from the Ismaelians of Persia. They
devoted their souls and bodies in blind obedience to a chief who is
called by the writers of the Crusades “the old man of the mountain,”
and were employed by him in the most extensive system of murder
and assassination known in the history of the world. Both Christian
and Moslem writers enumerate with horror the many illustrious
victims that fell beneath their daggers. They assumed all shapes and
disguises for the furtherance of their deadly designs, and carried, in
general, no arms except a small poniard concealed in the folds of
their dress, called in the Persian tongue hassissin, whence these
wretches were called assassins, their chief the prince of the
assassins; and the word itself, in all its odious import, has passed
into most European languages.[22]
Raimond, son of the count of Tripoli, had been slain by these
fanatics whilst kneeling at the foot of the altar in the church of the
Blessed Virgin at Carchusa or Tortosa; the Templars flew to arms to
avenge his death; they penetrated into the fastnesses and
strongholds of “the mountain chief,” and at last compelled him to
purchase peace by the payment of an annual tribute of two
thousand crowns into the treasury of the order. In the ninth year of
Amalric’s reign, Sinan Ben Suleiman, imaun of the assassins, sent a
trusty counsellor to Jerusalem, offering, in the name of himself and
his people, to embrace the christian religion, provided the Templars
would release them from the tribute money. The proposition was
favourably received; the envoy was honourably entertained for some](https://image.slidesharecdn.com/1162355-250529084159-1be53f4c/85/Linear-Inverse-Problems-The-Maximum-Entropy-Connection-Series-On-Advances-In-Mathematics-For-Applied-Sciences-83-Harcdr-Henryk-Gzyl-77-320.jpg)
Linear Inverse Problems The Maximum Entropy Connection Series On Advances In Mathematics For Applied Sciences 83 Harcdr Henryk Gzyl
- 1. Linear Inverse Problems The Maximum Entropy Connection Series On Advances In Mathematics For Applied Sciences 83 Harcdr Henryk Gzyl download https://ebookbell.com/product/linear-inverse-problems-the- maximum-entropy-connection-series-on-advances-in-mathematics-for- applied-sciences-83-harcdr-henryk-gzyl-2324710 Explore and download more ebooks at ebookbell.com
- 2. Here are some recommended products that we believe you will be interested in. You can click the link to download. Bayesian Nonlinear Statistical Inverse Problems Richard Nickl https://ebookbell.com/product/bayesian-nonlinear-statistical-inverse- problems-richard-nickl-55550218 Inverse Linear Problems On A Hilbert Space And Their Krylov Solvability Noe Angelo Caruso https://ebookbell.com/product/inverse-linear-problems-on-a-hilbert- space-and-their-krylov-solvability-noe-angelo-caruso-38376756 Linear And Nonlinear Inverse Problems With Practical Applications 20121115 Jennifer L Mueller https://ebookbell.com/product/linear-and-nonlinear-inverse-problems- with-practical-applications-20121115-jennifer-l-mueller-42218546 Seismic Imaging And Inversion Volume 1 Application Of Linear Inverse Theory Stolt Drh https://ebookbell.com/product/seismic-imaging-and-inversion- volume-1-application-of-linear-inverse-theory-stolt-drh-4584290
- 3. The Linear Sampling Method In Inverse Electromagnetic Scattering Cbmsnsf Regional Conference Series In Applied Mathematics Fioralba Cakoni https://ebookbell.com/product/the-linear-sampling-method-in-inverse- electromagnetic-scattering-cbmsnsf-regional-conference-series-in- applied-mathematics-fioralba-cakoni-2226046 Generalized Inverses Of Linear Transformations Stephen L Campbell https://ebookbell.com/product/generalized-inverses-of-linear- transformations-stephen-l-campbell-1627310 Linear Algebra 5th Edition Stephen H Friedberg Arnold J Insel https://ebookbell.com/product/linear-algebra-5th-edition-stephen-h- friedberg-arnold-j-insel-44875288 Linear Algebra From The Beginnings To The Jordan Normal Forms Toshitsune Miyake https://ebookbell.com/product/linear-algebra-from-the-beginnings-to- the-jordan-normal-forms-toshitsune-miyake-45796752 Linear Algebra For Everyone Gilbert Strang https://ebookbell.com/product/linear-algebra-for-everyone-gilbert- strang-46073830
- 6. LINEAR INVERSE PROBLEMS The Maximum Entropy Connection With CD-ROM
- 7. Series on Advances in Mathematics for Applied Sciences Editorial Board M. A. J. Chaplain Department of Mathematics University of Dundee Dundee DD1 4HN Scotland C. M. Dafermos Lefschetz Center for Dynamical Systems Brown University Providence, RI 02912 USA J. Felcman Department of Numerical Mathema ics Faculty of Mathematics and Physics Charles University in Prague Sokolovska 83 18675 Praha 8 The Czech Republic M. A. Herrero Departamento de Matematica Aplicada Facultad de Matemáticas Universidad Complutense Ciudad Universitaria s/n 28040 Madrid Spain S. Kawashima Department of Applied Sciences Engineering Faculty Kyushu University 36 Fukuoka 812 Japan M. Lachowicz Department of Mathematics University of Warsaw Ul. Banacha 2 PL-02097 Warsaw Poland N. Bellomo Editor-in-Charge Department of Mathematics Politecnico di Torino Corso Duca degli Abruzzi 24 10129 Torino Italy E-mail: nicola.bellomo@polito.it F. Brezzi Editor-in-Charge IMATI - CNR Via Ferrata 5 27100 Pavia Italy E-mail: brezzi@imati.cnr.it S. Lenhart Mathematics Department University of Tennessee Knoxville, TN 37996–1300 USA P. L. Lions University Paris XI-Dauphine Place du Marechal de Lattre de Tassigny Paris Cedex 16 France B. Perthame Laboratoire J.-L. Lions Université P. et M. Curie (Paris 6) BC 187 4, Place Jussieu F-75252 Paris cedex 05, France K. R. Rajagopal Department of Mechanical Engrg. Texas A&M University College Station, TX 77843-3123 USA R. Russo Dipartimento di Matematica II University Napoli Via Vivaldi 43 81100 Caserta Italy
- 8. Series on Advances in Mathematics for Applied Sciences Aims and Scope This Series reports on new developments in mathematical research relating to methods, qualitative and numerical analysis, mathematical modeling in the applied and the technological sciences. Contributions re- lated to constitutive theories, fluid dynamics, kinetic and transport theories, solid mechanics, system theory and mathematical methods for the applications are welcomed. This Series includes books, lecture notes, proceedings, collections of research papers. Monograph col- lections on specialized topics of current interest are particularly encouraged. Both the proceedings and monograph collections will generally be edited by a Guest editor. High quality, novelty of the content and potential for the applications to modern problems in applied science will be the guidelines for the selection of the content of this series. Instructions for Authors Submission of proposals should be addressed to the editors-in-charge or to any member of the editorial board. In the latter, the authors should also notify the proposal to one of the editors-in-charge. Acceptance of books and lecture notes will generally be based on the description of the general content and scope of the book or lecture notes as well as on sample of the parts judged to be more significantly by the authors. Acceptance of proceedings will be based on relevance of the topics and of the lecturers contributing to the volume. Acceptance of monograph collections will be based on relevance of the subject and of the authors contribu ing to the volume. Authors are urged, in order to avoid re-typing, not to begin the final prepara ion of the text until they received the publisher’s guidelines. They will receive from World Scientific the instructions for preparing camera-ready manuscript.
- 9. SERIES ON ADVANCES IN MATHEMATICS FOR APPLIED SCIENCES Published*: Vol. 69 Applied and Industrial Mathematics in Italy eds. M. Primicerio, R. Spigler and V. Valente Vol. 70 Multigroup Equations for the Description of the Particle Transport in Semiconductors by M. Galler Vol. 71 Dissipative Phase Transitions eds. P. Colli, N. Kenmochi and J. Sprekels Vol. 72 Advanced Mathematical and Computational Tools in Metrology VII eds. P. Ciarlini et al. Vol. 73 Introduction to Computational Neurobiology and Clustering by B. Tirozzi, D. Bianchi and E. Ferraro Vol. 74 Wavelet and Wave Analysis as Applied to Materials with Micro or Nanostructure by C. Cattani and J. Rushchitsky Vol. 75 Applied and Industrial Mathematics in Italy II eds. V. Cutello et al. Vol. 76 Geometric Control and Nonsmooth Analysis eds. F. Ancona et al. Vol. 77 Continuum Thermodynamics by K. Wilmanski Vol. 78 Advanced Mathematical and Computational Tools in Metrology and Testing eds. F. Pavese et al. Vol. 79 From Genetics to Mathematics eds. M. Lachowicz and J. Mi“kisz Vol. 80 Inelasticity of Materials: An Engineering Approach and a Practical Guide by A. R. Srinivasa and S. M. Srinivasan Vol. 81 Stability Criteria for Fluid Flows by A. Georgescu and L. Palese Vol. 82 Applied and Industrial Mathematics in Italy III eds. E. De Bernardis, R. Spigler and V. Valente Vol. 83 Linear Inverse Problems: The Maximum Entropy Connection by H. Gzyl and Y. Velásquez *To view the complete list of the published volumes in the series, please visit: http://www.worldscibooks.com/series/samas_series.shtml
- 10. NEW JERSEY • LONDON • SINGAPORE • BEIJING • SHANGHAI • HONG KONG • TAIPEI • CHENNAI World Scientific Series on Advances in Mathematics for Applied Sciences – Vol. 83 LINEAR INVERSE PROBLEMS The Maximum Entropy Connection With CD-ROM Henryk Gzyl IESA, Venezuela Yurayh Velásquez Universidad Metropolitana, Venezuela
- 11. Library of Congress Cataloging-in-Publication Data Gzyl, Henryk, Linear inverse problems : the maximum entropy connection (with CD-ROM) / by Henryk Gzyl & Yurayh Velásquez. p. cm. -- (Series on advances in mathematics for applied sciences ; v. 83) Includes bibliographical references. ISBN-13: 978-981-4338-77-6 (hardcover : alk. paper) ISBN-10: 981-4338-77-X (hardcover : alk. paper) 1. Inverse problems (Differential equations) 2. Maximum entropy method. I. Velásquez, Yurayh. II. Title. QA378.5.G98 2011 515'.357--dc22 2010047244 British Library Cataloguing-in-Publication Data A catalogue record for this book is available from the British Library. Cover illustration by Stefan Gzyl For photocopying of material in this volume, please pay a copying fee through the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, USA. In this case permission to photocopy is not required from the publisher. All rights reserved. This book, or parts thereof, may not be reproduced in any form or by any means, electronic or mechanical, including photocopying, recording or any information storage and retrieval system now known or to be invented, without written permission from the Publisher. Copyright © 2011 by World Scientific Publishing Co. Pte. Ltd. Published by World Scientific Publishing Co. Pte. Ltd. 5 Toh Tuck Link, Singapore 596224 USA office 27 Warren Street, Suite 401-402, Hackensack, NJ 07601 UK office 57 Shelton Street, Covent Garden, London WC2H 9HE Printed in Singapore.
- 12. Preface These lecture notes were originally prepared as backup material for a course on Inverse Problems and Maximum Entropy taught at the Venezuelan School of Mathematics. The event takes place annually in the city of Mérida, a university town in the Venezuelan Andean mountains. The at- tendance, mainly graduate students are exposed material that goes a bit beyond the standard courses. The course had three aims. On one hand, to present some basic results about linear inverse problems and how to solve them. On the other, to de- velop the method of maximum entropy in the mean, and to apply it to study linear inverse problems. This would show the weaknesses and strengths of both approaches. The other aim was to acquaint the participants with the use of the software that is provided along with the book. This consists of interactive screens on which the data of typical problems can be uploaded, and a solution is provided The present notes eliminate many mistakes and misprints that plague the original version, and hopefully not many new ones crept in the new material that we added for this version. The material was reorganized slightly, new applications were added, but no real effort was undertaken to update the enormous literature on applications of the maximum entropy method. It is a pleasure to thank the team of TeX experts at World Scientific for their help in bringing the original latex manuscript to its present form. v
- 13. “You can’t always get what you want but if you try, sometimes you may find that you get what you need” (M. Jagger & K. Richards) “Cuando se me termina el azul, uso el rojo” (P. Picasso) “No consigo comprender el significado de la palabra ‘investigación’ en la pintura moderna, a mi manera de ver, buscar no significa nada en pintura. Lo que cuenta es encontrar” (P. Picasso) “Desde cuando es el autor de un libro quien mejor lo comprende” (M. de Unamuno) “For every problem there is a solution that is simple, elegant and wrong” (H. L. Menken) The Roman jurists ruled: “Concerning evildoers, mathematicians, and the like” that: “to learn the art of geometry and to take part in public exercises, an art as damnable as mathematics, are forbidden” (downloaded from the web)
- 14. Contents Preface v List of Figures xv List of Tables xxi 1. Introduction 1 2. A collection of linear inverse problems 5 2.1 A battle horse for numerical computations . . . . . . . . 5 2.2 Linear equations with errors in the data . . . . . . . . . 6 2.3 Linear equations with convex constraints . . . . . . . . . 8 2.4 Inversion of Laplace transforms from finite number of data points . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 2.5 Fourier reconstruction from partial data . . . . . . . . . 11 2.6 More on the non-continuity of the inverse . . . . . . . . . 12 2.7 Transportation problems and reconstruction from marginals . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 2.8 CAT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 2.9 Abstract spline interpolation . . . . . . . . . . . . . . . . 20 2.10 Bibliographical comments and references . . . . . . . . . 21 3. The basics about linear inverse problems 25 3.1 Problem statements . . . . . . . . . . . . . . . . . . . . . 25 3.2 Quasi solutions and variational methods . . . . . . . . . 30 3.3 Regularization and approximate solutions . . . . . . . . 31 vii
- 15. viii Linear Inverse Problems: The Maxentropic Connection 3.4 Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . 35 3.5 Bibliographical comments and references . . . . . . . . . 36 4. Regularization in Hilbert spaces: Deterministic and stochastic approaches 37 4.1 Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 4.2 Tikhonov’s regularization scheme . . . . . . . . . . . . . 40 4.3 Spectral cutoffs . . . . . . . . . . . . . . . . . . . . . . . 44 4.4 Gaussian regularization of inverse problems . . . . . . . 46 4.5 Bayesian methods . . . . . . . . . . . . . . . . . . . . . . 48 4.6 The method of maximum likelihood . . . . . . . . . . . . 49 4.7 Bibliographical comments and references . . . . . . . . . 51 5. Maxentropic approach to linear inverse problems 53 5.1 Heuristic preliminaries . . . . . . . . . . . . . . . . . . . 53 5.2 Some properties of the entropy functionals . . . . . . . . 58 5.3 The direct approach to the entropic maximization problem . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 5.4 A more detailed analysis . . . . . . . . . . . . . . . . . . 62 5.5 Convergence of maxentropic estimates . . . . . . . . . . 64 5.6 Maxentropic reconstruction in the presence of noise . . . 67 5.7 Maxentropic reconstruction of signal and noise . . . . . . 70 5.8 Maximum entropy according to Dacunha-Castelle and Gamboa. Comparison with Jaynes’ classical approach . . 72 5.8.1 Basic results . . . . . . . . . . . . . . . . . . . . . 72 5.8.2 Jaynes’ and Dacunha and Gamboa’s approaches 77 5.9 MEM under translation . . . . . . . . . . . . . . . . . . . 79 5.10 Maxent reconstructions under increase of data . . . . . . 80 5.11 Bibliographical comments and references . . . . . . . . . 82 6. Finite dimensional problems 87 6.1 Two classical methods of solution . . . . . . . . . . . . . 87 6.2 Continuous time iteration schemes . . . . . . . . . . . . . 90 6.3 Incorporation of convex constraints . . . . . . . . . . . . 91 6.3.1 Basics and comments . . . . . . . . . . . . . . . . 91 6.3.2 Optimization with differentiable non-degenerate equality constraints . . . . . . . . . . . . . . . . . 95
- 16. Contents ix 6.3.3 Optimization with differentiable, non-degenerate inequality constraints . . . . . . . . . . . . . . . . 97 6.4 The method of projections in continuous time . . . . . . 98 6.5 Maxentropic approaches . . . . . . . . . . . . . . . . . . 99 6.5.1 Linear systems with band constraints . . . . . . . 100 6.5.2 Linear system with Euclidean norm constraints . 102 6.5.3 Linear systems with non-Euclidean norm constraints . . . . . . . . . . . . . . . . . . . . . . 104 6.5.4 Linear systems with solutions in unbounded convex sets . . . . . . . . . . . . . . . . . . . . . 105 6.5.5 Linear equations without constraints . . . . . . . 109 6.6 Linear systems with measurement noise . . . . . . . . . . 112 6.7 Bibliographical comments and references . . . . . . . . . 113 7. Some simple numerical examples and moment problems 115 7.1 The density of the Earth . . . . . . . . . . . . . . . . . . 115 7.1.1 Solution by the standard L2[0, 1] techniques . . . 116 7.1.2 Piecewise approximations in L2([0, 1]) . . . . . . 117 7.1.3 Linear programming approach . . . . . . . . . . . 118 7.1.4 Maxentropic reconstructions: Influence of a priori data . . . . . . . . . . . . . . . . . . . . . . . . . 120 7.1.5 Maxentropic reconstructions: Effect of the noise . 122 7.2 A test case . . . . . . . . . . . . . . . . . . . . . . . . . . 125 7.2.1 Standard L2[0, 1] technique . . . . . . . . . . . . 126 7.2.2 Discretized L2[0, 1] approach . . . . . . . . . . . 127 7.2.3 Maxentropic reconstructions: Influence of a priori data . . . . . . . . . . . . . . . . . . . . . . . . . 128 7.2.4 Reconstruction by means of cubic splines . . . . 131 7.2.5 Fourier versus cubic splines . . . . . . . . . . . . 135 7.3 Standard maxentropic reconstruction . . . . . . . . . . . 141 7.3.1 Existence and stability . . . . . . . . . . . . . . . 144 7.3.2 Some convergence issues . . . . . . . . . . . . . . 146 7.4 Some remarks on moment problems . . . . . . . . . . . . 146 7.4.1 Some remarks about the Hamburger and Stieltjes moment problems . . . . . . . . . . . . . . . . . . 149 7.5 Moment problems in Hilbert spaces . . . . . . . . . . . . 152 7.6 Reconstruction of transition probabilities . . . . . . . . . 154
- 17. x Linear Inverse Problems: The Maxentropic Connection 7.7 Probabilistic approach to Hausdorff’s moment problem . 156 7.8 The very basics about cubic splines . . . . . . . . . . . . 158 7.9 Determination of risk measures from market price of risk 159 7.9.1 Basic aspects of the problem . . . . . . . . . . . 159 7.9.2 Problem statement . . . . . . . . . . . . . . . . . 161 7.9.3 The maxentropic solution . . . . . . . . . . . . . 162 7.9.4 Description of numerical results . . . . . . . . . . 163 7.10 Bibliographical comments and references . . . . . . . . . 164 8. Some infinite dimensional problems 169 8.1 A simple integral equation . . . . . . . . . . . . . . . . . 169 8.1.1 The random function approach . . . . . . . . . . 170 8.1.2 The random measure approach: Gaussian measures . . . . . . . . . . . . . . . . . . . . . . . 173 8.1.3 The random measure approach: Compound Poisson measures . . . . . . . . . . . . . . . . . . 174 8.1.4 The random measure approach: Gaussian fields . 176 8.1.5 Closing remarks . . . . . . . . . . . . . . . . . . . 177 8.2 A simple example: Inversion of a Fourier transform given a few coefficients . . . . . . . . . . . . . . . . . . . . . . 178 8.3 Maxentropic regularization for problems in Hilbert spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179 8.3.1 Gaussian measures . . . . . . . . . . . . . . . . . 179 8.3.2 Exponential measures . . . . . . . . . . . . . . . 182 8.3.3 Degenerate measures in Hilbert spaces and spectral cut off regularization . . . . . . . . . . . 183 8.3.4 Conclusions . . . . . . . . . . . . . . . . . . . . . 184 8.4 Bibliographical comments and references . . . . . . . . . 184 9. Tomography, reconstruction from marginals and transportation problems 185 9.1 Generalities . . . . . . . . . . . . . . . . . . . . . . . . . 185 9.2 Reconstruction from marginals . . . . . . . . . . . . . . . 187 9.3 A curious impossibility result and its counterpart . . . . 188 9.3.1 The bad news . . . . . . . . . . . . . . . . . . . . 188 9.3.2 The good news . . . . . . . . . . . . . . . . . . . 190 9.4 The Hilbert space set up for the tomographic problem . 192 9.4.1 More on nonuniquenes of reconstructions . . . . . 194
- 18. Contents xi 9.5 The Russian Twist . . . . . . . . . . . . . . . . . . . . . 194 9.6 Why does it work . . . . . . . . . . . . . . . . . . . . . . 195 9.7 Reconstructions using (classical) entropic, penalized methods in Hilbert space . . . . . . . . . . . . . . . . . . 198 9.8 Some maxentropic computations . . . . . . . . . . . . . . 201 9.9 Maxentropic approach to reconstruction from marginals in the discrete case . . . . . . . . . . . . . . . . . . . . . 203 9.9.1 Reconstruction from marginals by maximum entropy on the mean . . . . . . . . . . . . . . . . 204 9.9.2 Reconstruction from marginals using the standard maximum entropy method . . . . . . . . . . . . . 207 9.10 Transportation and linear programming problems . . . . 209 9.11 Bibliographical comments and references . . . . . . . . . 211 10. Numerical inversion of Laplace transforms 215 10.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . 215 10.2 Basics about Laplace transforms . . . . . . . . . . . . . . 216 10.3 The inverse Laplace transform is not continuous . . . . . 218 10.4 A method of inversion . . . . . . . . . . . . . . . . . . . 218 10.4.1 Expansion in sine functions . . . . . . . . . . . . 219 10.4.2 Expansion in Legendre polynomials . . . . . . . . 220 10.4.3 Expansion in Laguerre polynomials . . . . . . . . 221 10.5 From Laplace transforms to moment problems . . . . . . 222 10.6 Standard maxentropic approach to the Laplace inversion problem . . . . . . . . . . . . . . . . . . . . . . . . . . . 223 10.7 Maxentropic approach in function space: The Gaussian case . . . . . . . . . . . . . . . . . . . . . 225 10.8 Maxentropic linear splines . . . . . . . . . . . . . . . . . 227 10.9 Connection with the complex interpolation problem . . . 229 10.10 Numerical examples . . . . . . . . . . . . . . . . . . . . . 230 10.11 Bibliographical comments and references . . . . . . . . . 236 11. Maxentropic characterization of probability distributions 241 11.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . 241 11.2 Example 1 . . . . . . . . . . . . . . . . . . . . . . . . . . 243 11.3 Example 2 . . . . . . . . . . . . . . . . . . . . . . . . . . 244 11.4 Example 3 . . . . . . . . . . . . . . . . . . . . . . . . . . 245
- 19. xii Linear Inverse Problems: The Maxentropic Connection 11.5 Example 4 . . . . . . . . . . . . . . . . . . . . . . . . . . 245 11.6 Example 5 . . . . . . . . . . . . . . . . . . . . . . . . . . 246 11.7 Example 6 . . . . . . . . . . . . . . . . . . . . . . . . . . 246 12. Is an image worth a thousand words? 249 12.1 Problem setup . . . . . . . . . . . . . . . . . . . . . . . . 249 12.1.1 List of questions for you to answer . . . . . . . . 251 12.2 Answers to the questions . . . . . . . . . . . . . . . . . . 251 12.2.1 Introductory comments . . . . . . . . . . . . . . 251 12.2.2 Answers . . . . . . . . . . . . . . . . . . . . . . . 251 12.3 Bibliographical comments and references . . . . . . . . . 258 Appendix A Basic topology 261 Appendix B Basic measure theory and probability 265 B.1 Some results from measure theory and integration . . . . 265 B.2 Some probabilistic jargon . . . . . . . . . . . . . . . . . . 272 B.3 Brief description of the Kolmogorov extension theorem . 275 B.4 Basic facts about Gaussian process in Hilbert spaces . . 276 Appendix C Banach spaces 279 C.1 Basic stuff . . . . . . . . . . . . . . . . . . . . . . . . . . 279 C.2 Continuous linear operator on Banach spaces . . . . . . 281 C.3 Duality in Banach spaces . . . . . . . . . . . . . . . . . . 283 C.4 Operators on Hilbert spaces. Singular values decompositions . . . . . . . . . . . . . . . . . . . . . . . 289 C.5 Some convexity theory . . . . . . . . . . . . . . . . . . . 290 Appendix D Further properties of entropy functionals 293 D.1 Properties of entropy functionals . . . . . . . . . . . . . 293 D.2 A probabilistic connection . . . . . . . . . . . . . . . . . 297 D.3 Extended definition of entropy . . . . . . . . . . . . . . . 301 D.4 Exponetial families and geometry in the space of probabilities . . . . . . . . . . . . . . . . . . . . . . . . . 302 D.4.1 The geometry on the set of positive vectors . . . 304 D.4.2 Lifting curves from G+ to G and parallel transport . . . . . . . . . . . . . . . . . . . . . . 306 D.4.3 From geodesics to Kullback’s divergence . . . . . 307
- 20. Contents xiii D.4.4 Coordinates on P . . . . . . . . . . . . . . . . . . 308 D.5 Bibliographical comments and references . . . . . . . . . 310 Appendix E Software user guide 313 E.1 Installation procedure . . . . . . . . . . . . . . . . . . . . 313 E.2 Quick start guide . . . . . . . . . . . . . . . . . . . . . . 316 E.2.1 Moment problems with MEM . . . . . . . . . . . 317 E.2.2 Moment problems with SME . . . . . . . . . . . 318 E.2.3 Moment problems with Quadratic Programming 318 E.2.4 Transition probabilities problem with MEM . . . 319 E.2.5 Transition probabilities problem with SME . . . 320 E.2.6 Transition probabilities problem with Quadratic Programming . . . . . . . . . . . . . . . . . . . . 320 E.2.7 Reconstruction from Marginals with MEM . . . . 320 E.2.8 Reconstruction from Marginals with SME . . . . 321 E.2.9 Reconstruction from Marginals with Quadratic Programming . . . . . . . . . . . . . . . . . . . . 321 E.2.10 A generic problem in the form Ax = y, with MEM . . . . . . . . . . . . . . . . . . . . . 322 E.2.11 A generic problem in the form Ax = y, with SME . . . . . . . . . . . . . . . . . . . . . . 323 E.2.12 A generic problem in the form Ax = y, with Quadratic Programming . . . . . . . . . . . . . . 323 E.2.13 The results windows . . . . . . . . . . . . . . . . 323 E.2.14 Messages that will appear . . . . . . . . . . . . . 324 E.2.15 Comments . . . . . . . . . . . . . . . . . . . . . . 326
- 21. This page intentionally left blank This page intentionally left blank
- 22. List of Figures 2.1 Basic scheme. . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 2.2 Basic geometry of the CAT process. . . . . . . . . . . . . . . . 17 2.3 Tomographic image formation. . . . . . . . . . . . . . . . . . . 18 2.4 Actual CAT setup. . . . . . . . . . . . . . . . . . . . . . . . . 19 6.1 C ∩ A−1 y = ∅. . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 6.2 C ∩ A−1 y = ∅. x∗ = x0. . . . . . . . . . . . . . . . . . . . . . 94 6.3 C ∩ A−1 y x∗ = x0. . . . . . . . . . . . . . . . . . . . . . . . 95 7.1 Standard L2 solution to (7.1). . . . . . . . . . . . . . . . . . . 116 7.2 Standard L2 with penalization. . . . . . . . . . . . . . . . . . 117 7.3 Reconstruction with λ = 10−7 , ε = 6.0063 × 10−5 . . . . . . . . 118 7.4 Penalized least square reconstruction. Constrained a = 0, b = 1; parameters λ = 10−5 , 10−7 . Reconstruction errors ε = 5.23 × 10−3 , 6.39 × 10−5 . . . . . . . . . . . . . . . . . . . 119 7.5 Penalized least square reconstruction. Constrained a = 0.1, b = 0.9; parameters λ = 10−5 , 10−7 , ε = 7.13 × 10−5 . . . . . . 119 7.6 Reconstructing using linear programming. . . . . . . . . . . . 120 7.7 Maxentropic reconstruction with uniform a priori on different [a, b]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 7.8 Maxentropic reconstruction with uniform a priori on different [a, b]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 7.9 A priori distribution of Bernoulli type masses p = q = 1 2 and different intervals. . . . . . . . . . . . . . . . . . . . . . . . . . 122 7.10 A priori distribution of Bernoulli type on a fixed interval but different masses at the ends. . . . . . . . . . . . . . . . . . . . 123 7.11 Reconstructions with uniform a priori distribution on [0, 1], fixed σ1 = σ2 = 0.01 and varying T . . . . . . . . . . . . . . . . 124 xv
- 23. xvi Linear Inverse Problems: The Maxentropic Connection 7.12 Reconstructions with uniform a priori distribution on [0, 4], varying σs and T s. . . . . . . . . . . . . . . . . . . . . . . . . 124 7.13 Reconstructions with fixed Bernoulli distribution on [0, 10] with p = q = 1 2 , T = 1 and σ1 = σ2 varying. . . . . . . . . . . . . . 125 7.14 Reconstructions under Bernoulli a priori distribution on [0, 10] with p = 1 = 1 2 , fixed σ1 = σ2 = 0.1, but T varying as indicated. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125 7.15 Reconstructions from first list of moments. Different λs and its respectively error. . . . . . . . . . . . . . . . . . . . . . . . . . 127 7.16 Reconstructions from second list of moments. Different λs and its respectively error. . . . . . . . . . . . . . . . . . . . . . . . 128 7.17 Reconstructions for first list moments and several λ. . . . . . 129 7.18 Reconstructions for second list of moments and several λ. . . 129 7.19 Reconstructions for first list of moments and several λ, pro- jected onto constraint space. . . . . . . . . . . . . . . . . . . . 130 7.20 Reconstructions for second list of moments and several λ, pro- jected onto constrained space. . . . . . . . . . . . . . . . . . . 130 7.21 Maxentropic reconstructions for different constraints and uni- form a priori measure. . . . . . . . . . . . . . . . . . . . . . . 131 7.22 Maxentropic reconstruction for different constrained and Bernoulli a priori measure with p = q = 1 2 . . . . . . . . . . . . 132 7.23 Maxentropic reconstruction for [a, b] = [−2, 2] and p = 0.5, 0.2, 0.8 with q = 1 − p. . . . . . . . . . . . . . . . . . . . . . . . . 132 7.24 Maxentropic reconstruction with cubic splines from m = 2, w = 1 with error ε = 1.9830 × 10−10 . . . . . . . . . . . . . . . 133 7.25 Maxentropic reconstruction with cubic splines from m = 2, w = π with error ε = 1.8340 × 10−8 . . . . . . . . . . . . . . . 134 7.26 Maxentropic reconstruction with cubic splines from m = 4, w = π with error ε = 4.5339 × 10−8 . . . . . . . . . . . . . . . 134 7.27 Maxentropic reconstruction with cubic splines from m = 4, w = π with error ε = 7.2104 × 10−9 . . . . . . . . . . . . . . . 135 7.28 Maxentropic reconstruction with cubic splines from m = 4, w = 2π with error ε = 5.2992 × 10−8 . . . . . . . . . . . . . . . 135 7.29 Maxentropic reconstruction with cubic splines from m = 7, w = 4π with error ε = 6.2902 × 10−6 . . . . . . . . . . . . . . . 136 7.30 Maxentropic reconstruction with cubic splines from m = 10, w = 4π with error ε = 9.9747 × 10−6 . . . . . . . . . . . . . . . 136
- 24. List of Figures xvii 7.31 Maxentropic reconstruction with cubic splines from m = 4, in [−5, 5], w = π, σ = 0.1, T1 = 0.5, T2 = 1, T3 = 1.5 and T4 = 2 with error ε1 = 0.05, ε2 = 0.10, ε3 = 0.20 and ε4 = 0.20. . . . 137 7.32 Maxentropic reconstruction with cubic splines from m = 4, in [−5, 5], w = π, σ = 0.01, T1 = 1.5, T2 = 2, T3 = 3, T4 = 4 and T5 = 5 with the respective error ε1 = 0.020, ε2 = 0.020, ε3 = 0.030, ε4 = 0.040 and ε5 = 0.050. . . . . . . . . . . . . . 137 7.33 Maxentropic reconstruction with cubic splines from m = 8, in [−5, 5], w = π, σ = 0.1, T1 = 0.5, T2 = 2, T3 = 3 and T4 = 4 with the respective error ε1 = 0.20, ε2 = 0.20, ε3 = 0.30 and ε4 = 0.40. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138 7.34 Maxentropic reconstruction with cubic splines from m = 8, in [−5, 5], w = π, σ = 0.01, T1 = 0.5, T2 = 1, T3 = 1.5 and T4 = 2 with the respective error ε1 = 0.0131, ε2 = 0.0135, ε3 = 0.020 and ε4 = 0.020. . . . . . . . . . . . . . . . . . . . . . . . . . . 138 7.35 Maxentropic reconstruction with cubic splines from m = 8, in [−1, 1], w = 2π, σ = 0.01, T1 = 0.5, T2 = 1, T3 = 2 and T4 = 3 with the respective error ε1 = 0.0118, ε2 = 0.01, ε3 = 0.02 and ε4 = 0.03. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 7.36 Maxentropic reconstruction with cubic splines from m = 8, in [−2, 2], w = 2π, σ = 0.01, T1 = 0.5, T2 = 1, T3 = 1.5 and T4 = 4 with the respective error ε1 = 0.005, ε2 = 0.010 and ε3 = 0.020. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 7.37 Maxentropic reconstruction with cubic splines and method No. 2, in [−25, 25], from m = 12, w = 2π, σ = 0.01, µ0 = 0.01, µ1 = 0, µ2 = 0.005 and µ3 = 0.02 with the respective error ε1 = 0.0274, ε2 = 0.0326, and ε3 = 0.745. . . . . . . . . . . . . 140 7.38 Maxentropic reconstruction with cubic splines and method No. 2, in [−80, 80], from m = 12, w = 4π, σ = 0.01, µ0 = 0.01, µ1 = 0, µ2 = 0.005 and µ3 = 0.02 with the respective error ε1 = 0.0246, ε2 = 0.030, and ε3 = 0.0734. . . . . . . . . . . . . 140 7.39 Reconstructions from m = 8, the respective L1 errors are σ1 = 0.5272 and σ2 = 0.6786 and the reconstruction error is ε1 = 6.1374 × 10−8 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141 7.40 Reconstructions from m = 14, the respective L1 errors are σ1 = 0.3811 and σ2 = 0.5654 and the reconstruction error is ε1 = 8.2644 × 10−8 . . . . . . . . . . . . . . . . . . . . . . . . . 141
- 25. xviii Linear Inverse Problems: The Maxentropic Connection 7.41 Reconstructions from m = 6, σ = 0.1, T1 = 0.5, T2 = 1 and T3 = 1.5 the respective L1 errors are δ1 = 0.6088, δ2 = 0.6269, δ3 = 0.6502, and δ4 = 0.6340, and the reconstruction errors for maximum entropy method are respectively ε1 = 0.05, ε2 = 0.1, ε3 = 0.15. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142 7.42 Reconstructions from m = 6, σ = 0.01, T1 = 0.25, T2 = 1 and T3 = 3 the respective L1 errors are δ1 = 0.5852, δ2 = 0.5895, δ3 = 0.5999, and δ4 = 0.6340, and the reconstruction errors for maximum entropy method are respectively ε1 = 0.00255, ε2 = 0.01, ε3 = 0.03. . . . . . . . . . . . . . . . . . . . . . . . 142 7.43 Original function and three reconstructions using standard ME applied to 7, 8 or 9 givens moments. . . . . . . . . . . . . . . 143 7.44 Graph of φ∗ reconstructed from four different prices. . . . . . 164 9.1 Difference between data and reconstruction, using uniform a priori measure. . . . . . . . . . . . . . . . . . . . . . . . . . . 206 9.2 Difference between data and reconstruction, using Bernoulli a priori measure, with p = 0.85, q = 0.15. . . . . . . . . . . . . 206 9.3 Difference between data and reconstruction. . . . . . . . . . . 208 9.4 Maxentropic approach to a linear programming problem, with p = 0.75, q = 0.25, γ = 50.2199. The reconstruction error ε = 1.7831 × 10−11 . . . . . . . . . . . . . . . . . . . . . . . . . 210 10.1 Reconstruction of exp(−t) sin(2πt) using sine, Legendre and Laguerre bases. . . . . . . . . . . . . . . . . . . . . . . . . . . 231 10.2 Reconstruction of exp(−t) sin(8πt) using sine, Legendre and Laguerre bases. . . . . . . . . . . . . . . . . . . . . . . . . . . 232 10.3 Reconstruction of sin(2πt) using sine, Legendre and Laguerre bases. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232 10.4 Reconstruction of sin(8πt) using sine, Legendre and Laguerre bases. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233 10.5 Reconstruction of N [1 + sin(2πt)] exp(−t) and N [1 + sin(8πt)] exp(−t) using standard maximum entropy. . . 233 10.6 Reconstruction of N [1 + sin(2πt)] and N [1 + sin(8πt)] using standard maximum entropy. . . . . . . . . . . . . . . . . . . . 234 10.7 Maxentropic reconstruction with first order splines from m = 5 (0,2,4,6,8) β = 1, t0 = 1, σ1 = σ2 = 0.6931, for different intervals of reconstruction I1 = [0, 1], I2 = [−1, 1] with errors ε1 = 1.4438 × 10−6 , ε2 = 1.7652 × 10−6 . . . . . . . . . . . . . 234
- 26. List of Figures xix 10.8 Maxentropic reconstruction with first order splines from m = 5 (0,2,4,6,8) β = 1, t0 = 1, σ1 = σ2 = 0.6931, for different intervals of reconstruction I1 = [0, 1.5], I2 = [−2, 2] with errors ε1 = 1.2994 × 10−4 , ε2 = 1.4573 × 10−6 . . . . . . . . . . . . . 235 10.9 Maxentropic reconstruction with first order splines from m = 10 (0,1,2,3,4,5,6,7,8,9), ω = 1, σ = 0.6931 with error ε = 1.7567 × 10−5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235 10.10 Multiplying by exp(βt) the functions showed in figure 10.2. . . 236 10.11 Maxentropic reconstructions with first order splines from m = 15 (0, 2, 4, ..., 26, 28), ω = 2π, with different σ’s values. Respectively values of σ’s and errors are listed: σ1 = 0.009, ε1 = 3.28 × 10−6 ; σ2 = 0.09, ε2 = 0.0014; σ3 = 0.9, ε3 = 0.0025. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237 10.12 Maxentropic reconstructions with first order splines from m = 15 (0,2,4,...,26,28), ω = 8π, with different σ’s values. Re- spectively values of σ’s and errors are listed: σ1 = 0.009, ε1 = 1.22 × 10−7 ; σ2 = 0.09, ε2 = 4.68 × 10−5 ; σ3 = 0.9, ε3 = 6.25 × 10−5 . . . . . . . . . . . . . . . . . . . . . . . . . . 237 12.1 The original picture. . . . . . . . . . . . . . . . . . . . . . . . 250 12.2 The latitude (LAT) is equal to the height of the pole P above the horizon. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252 12.3 Here we define variables for the analysis. We identified some of the stars in the picture. The South Pole is identified with a letter P. The star pointed with the arrow is the most weak start that the author could identify on the original photo. . . 254 12.4 Southern Hemisphere’s Map, used to determine the photo’s date. The date is displayed in the external bound (Norton, 1978). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256
- 27. This page intentionally left blank This page intentionally left blank
- 28. List of Tables 7.1 List of moments. . . . . . . . . . . . . . . . . . . . . . . . . . 126 7.2 Transition matrix obtained with maxentropic reconstruction, ε = 2.3842 × 10−10 . . . . . . . . . . . . . . . . . . . . . . . . . 155 7.3 Transition matrix obtained with maxentropic reconstruction, ε = 1.1230 × 10−12 . . . . . . . . . . . . . . . . . . . . . . . . . 156 7.4 Error of reconstruction risk price. . . . . . . . . . . . . . . . . 163 9.1 Data matrix. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204 9.2 Reconstruction using uniform a priori measure. ε = 2.1544 × 10−9 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205 9.3 Reconstruction using Bernoulli a priori measure, with p = 0.85, q = 0.15. ε = 1.6444 × 10−9 . . . . . . . . . . . . . . . . . 205 9.4 Reconstruction using standard maximum entropy. . . . . . . . 208 xxi
- 29. Chapter 1 Introduction In this volume we examine some basic aspects of linear inverse problems, examine some standard methods of solutions, and present the method of maximum entropy in the mean (MEM for short) as a powerful alternative method to solve constrained linear inverse problems. We shall see that the method takes care of the constraints in a natural way, and competes reason- ably with the standard methods. As we shall eventually see below, MEM includes some standard methods as particular cases. We shall illustrate the range of applications with examples from many different fields. The generic, and most general form of a linear inverse problem with convex constraints consists of solving for x in Ax ∈ BM (y, T), x ∈ C ⊂ V (1.1) where V and W are given Banach spaces, A : V → W is a linear bounded operator, C is some specified convex set in V. These are the convex con- straints imposed on the solution. Usually the data vector y is known up to some experimental error, time and again it does not even belong to A(V ) and we have to be flexible as to what we admit as solution. The right-hand side of (1.1) suggests that we will admit any x ∈ C, such that Ax is inside some ball, not necessarily in the same norm as in W, but certainly related to it and to the experimental errors, the radius T of which measures the “tolerance” with which we accept solutions. When the data is certain, we can be strict and intolerant, set T = 0 to obtain, instead of (1.1): Ax = y, x ∈ C ⊂ V. (1.2) So if linear inverse problems basically consist of solving a linear equation, why are they not simply called linear equations? Probably because of the 1
- 30. 2 Linear Inverse Problems: The Maxentropic Connection interpretation of (1.2) or (1.1). Quite often x is interpreted as an input or as an initial condition, the datum y as an output or as actual observed state of the system and A as the transfer operator describing the evolution of the system. Thus, finding the input or stimulus given the output or response is an inverse problem. Or you could follow Keller in saying: computing y from A and x is called a direct problem, the solving for x in Ax = y is called the inverse problem. The difficulties in solving (1.1) or (1.2) are of two kinds: First, it usually happens that A is neither injective nor surjective. The lack of surjectivity is easy to deal with by restricting the range of A. When A is not injective, then non-uniqueness is present and infinitely many solu- tions exist. The problem is to devise algorithms that produce “meaningful” solutions. Here meaningful is tied to the interpretation of the solution, and is not an obviously mathematizable concept. The second issue, important from the practical point of view, is that the “inversion” or “reconstruction” or more simply, the solution algorithms, should be robust, that is, small changes in the datum vector y should produce small changes in the reconstructed x. This amounts to saying that if y → x = B(y) is to satisfy (1.2) say, then B must be continuous in y. Different aspects of these issues dealing with either functional analysis and/or linear algebra on one hand, and with the computational side of these problems have been extensively studied. Many of the references to Chapter 2, 3, 4 or 5 are but entrance gates to these different aspects. One of the aims of these notes is to briefly review some basic notions related to solving (1.1) or (1.2). We carry out this in Chapter 3 and 4. Chapter 2 is devoted to listing some standard linear inverse problems. A standard way of approaching (1.2) is to convert it in a variational problem that exploits the metric nature of the spaces V and W: one searches for points x in C, which minimize a penalized error functional F(x) = Ax − yW + λ xV (1.3) defined on the constraint set C. In this fashion the machinery of convex analysis can be brought in to bear on theoretical (and practical) aspects of the problem. Even though the interpretation of (1.3) is rather direct, we must comment on the role of the penalization parameter λ: Among those x’s ∈ C yielding similar reconstruction error Ax − yW , we have to choose one with a norm as small as possible. Why? Well in some cases xV may have an interpretation like “energy” or cost, and it makes sense to choose among the many solutions the one
- 31. Introduction 3 with smallest energy. In other instances one may be forced to shrug and say: well it works and I don’t have anything else. The basic heuristic behind the MEM, developed in Chapter 5, consists of considering the set C of possible reconstructions as values of a random variable X. Instead of devising methods to find an explicit solution to the given equation, we search for a probability distribution P on C such that AEP [X] satisfies either (1.1) or (1.2). Here EP [X] = XdP denotes the mean value of X with respect to P. The problem is how to choose one among all possible P’s that do the job. It is here that the concept of entropy enters: in analogy with (1.3), the entropy S(P) will be a concave functional, defined on the set of all probability measures which is a convex set. Thus, the problem becomes one of maximization of a concave functional on some convex set. Hence the name of the method. To be fair, we have to ask why the specific functional S(P) chosen in Chapter 5 works. Despite the effort in making things appear “natural” (not yet in a functional sense), all we can say is that the procedure works. Originally, the variational method was proposed by Jaynes to lay the foundations of statistical physics, where it provided an astounding bridge between micro- and macro-physics. The list of problems in which the method works is still growing. See Section 6.5 of Chapter 6 and take a look at references [12]-[27] for an interesting list of a variety of applications. Or look at the journal “Entropy” for more. Now, since the: Why does it work? is tied to assembling a whole frame- work in which it does, we could as well as ourselves: Why that framework? To avoid longer regressions, the best answer so far is that it works. It will be up to you, dear reader to help explaining why. Or at least, to have fun applying it. Chapter 6 is devoted to the finite dimensional problems arising from the discretization of continuous problems or the truncation of infinitely many dimensional ones, and related issues. Some basic formulae are obtained there. We devote Chapter 7 to present some numerical examples. In Chapter 8 we carry out the program outlined in Chapter 5 in a truly infinite dimensional setup. In particular, we shall see that some regular- ization methods in Hilbert space can be obtained from the maxentropic approach. We shall devote Chapter 9 to review some results about reconstructing a function in the plane when its integral along a few lines is known. We end the chapter examining the reconstruction of the entries in a table from their
- 32. 4 Linear Inverse Problems: The Maxentropic Connection row and column sums, using two maxentropic approaches: the classical maximum entropy method and the method of maximum entropy in the mean. After that, instead of solving a transportation problem, we approach maxentropically a linear programming problem, arising in Chapter 7 when reconstructing a function from its moments. In Chapter 10 we review some basic stuff about Laplace transforms as well as some of the work done to deal with a vexing inverse problem in applied mathematics: that of reconstructing a function from a few values of its Laplace transform. We try a few maxentropic shots at it as well. The material in Chapter 11 is just for fun. It provides a maxentropic characterization of some non-exponential families, but it involves a depar- ture from the traditional or standard maximum entropy method. We provided a few appendices, where some basics on topology, mea- sure and probability theory, (very) elementary stuff on Banach spaces and convexity is gathered. It is really a commented list of definitions for those unfamiliar with that terminology. To finish we would like to thank Aldo Tagliani for contributing material to Chapters 7 and 10 as well as Michael Hazewinkel for making the resources of the nice library of the CWI in Amsterdam available to us. Special thanks go to Ignacio Ferrı́n for writing the last chapter. There he presents a “detectivesque inverse” problem consisting of interpreting a picture.
- 33. Chapter 2 A collection of linear inverse problems In this chapter we collect some examples of linear inverse problems. The comments about each item are to reflect typical issues described in the next chapters. Some of these we shall solve by the method of maximum entropy in the mean. Others we solve by the method of maximum entropy in the mean and at least one other method. This is just to compare solutions, and to emphasize the need of para-mathematical or meta-mathematical criteria of model building and problem solving. 2.1 A battle horse for numerical computations The problem consists of solving Ax = y (2.1) where A is an n × m matrix, y is a given vector in Rn and x is an unknown vector in Rm with n = m is so ubiquitous that we should just state it, describe some variants on the problem as well as some methods of solution. Even when m = n, A−1 may no exists and the system may not have a unique solution or no solution at all. Suppose to begin with that m ≥ n so that Ker(A) is not {0} . Two standard ways of choosing one among the infinitely many solutions depend on whether (A∗ A)−1 or (AA∗ )−1 exists. In the first case multiply both sides of (2.1) with A∗ and then solve for x in (A∗ A)x = A∗ y obtaining x = (A∗ A)−1 A∗ y. In the second case, and when Ker(A) = {0} but AA∗ is invertible, one way of choosing among the infinitely many possible solutions, is to consider 5
- 34. 6 Linear Inverse Problems: The Maxentropic Connection the x0 that solves the variational problem: inf 1 2 Ax − y2 . (2.2) We can apply either the standard Lagrange multiplier method or duality techniques to obtain: x0 = A∗ (AA∗ )−1 y. When, for example either A∗ A is not invertible, or if it is invertible, its eigenvalues are so small that possible measurement errors in y are amplified too much, one recurs to searching for x0 realizing x0 = arg inf λ 2 x2 + Ax − y2 : x ∈ Rm . (2.3) Here λ is a penalization parameter by means of which we assign “dif- ferent weights” to the two terms in (2.3). An easy computation yields x0 = (λI + A∗ A) −1 A∗ y (2.4) as the solution to (2.3). Here the role of λ becomes clear: if the eigenvalues of the symmetric matrix are too small, we push them up with λ. If they are all large, we take λ to be small. 2.2 Linear equations with errors in the data This time, instead of (2.1), we are supposed to have an equation like: Ax = y0 + ε = y where y0 is supposed to be a true value and ε is supposed to model a random error in the measurement of y0. Again suppose Ker(A) = {0}. Sometimes we may have enough mea- surements of y, in which the errors may add up to zero. Since the methods deployed in Section 2.1 yield a solution linear in y, when we average over solutions, we end up washing away the effect of the measurement errors. But usually one only has one value of y plus a mathematical or physical model for the error ε. In this case, a procedure that exploits both Euclidean Geometry and Gaussian distribution of errors suggests considering to solve for x in Ax ∈ BM (y0, T ) (2.5)
- 35. A collection of linear inverse problems 7 where BM (y0, T ) = η ∈ Rn : mij(yj − y0 j )(yi − y0 i ) ≤ T is the ball of radius T with center y0 in a (positive definite) metric mij. When the measurements of the components of y0 are independent of each other, it is natural to assume that M is diagonal matrix. Again, when Ker(A) = {0}, there are infinitely many solutions to (2.5), or solutions are unique up to addition of elements in Ker(A). To determine a solution, we do as in section 2.1, namely we look for x0 = arg inf 1 2 x2 : Ax − yM ≤ T . Actually, even if the drawing below is not exact, but it suggests the proof of the fact that we may replace the former characterization by: x0 = arg inf 1 2 x 2 : Ax − yM = T . (2.6) The drawing is: ||Ax − y||M ≤ T ||x||2 ≤ R2 x0 Norm of vector= R Tolerance zone Lemma 2.1. The infimum of 1 2 x2 : Ax − yM ≤ T is achieved at some x0 for which Ax0 − yM = T.
- 36. 8 Linear Inverse Problems: The Maxentropic Connection Proof. Assume that x0 realizes the minimum but Ax0 − yM = T1 T. The value of the tolerance at βx0, 0 β 1 is given by A(βx0) − y 2 = β(Ax0 − y) − (1 − β)y 2 = βT 2 1 + 2β(1 − β)(y, Ax0 − y) + (1 − β)2 y 2 . A simple continuity argument shows that as β gets closer to 1, we can keep the right-hand side of the last expression less than T2 , i.e. βx0 is within the tolerance zone, but its norm is less than that of x0, contrary to the assumption. The importance of this lemma is that to find x0 it suffices to apply Lagrange multipliers to (2.6). But this leads to a non-linear system. Let us drop this issue for the time being. 2.3 Linear equations with convex constraints In this example we consider the same basic setup as in the first one, but this time we have convex constraints on x of the type: x ∈ K, where K is a given convex set in Rm . Thus, we want to solve: Ax = y, x ∈ K (2.7) or to make it worse Find x ∈ K such that Ax − yM ≤ T. (2.8) Typical constraints are determined by convex sets like: K = {x ∈ Rm : ai xi bi} , (2.9) where −∞ ≤ ai bi ≤ +∞, or we may have K = x ∈ Rm : x − x0, B(x − x0) b2 , (2.10) where B is some symmetric, positive definite matrix. There are two cases in which it is easy to get variational solutions. To describe the first one, consider the problem of finding: x0 = arg inf λ 2 x 2 X + 1 2 Ax − y 2 Y : x ∈ K (2.11) where K is described by (2.10). Note that if we did not have the restriction x ∈ K, the following iterative procedure would lead to x0 : Define: xn+1 = xn − ε {λxn − A∗ (y − Axn)} , (2.12)
- 37. A collection of linear inverse problems 9 i.e., take steps of size ε again, the gradient of the convex function F(x) = λ 2 x 2 X + Ax − y 2 Y . If we define the projection operator P : Rm → K by: (Px)i = xi if ai xi bi ai if xi ≤ ai bi if bi ≤ xi for i = 1, 2, ..., m; then instead of (2.12), to find the x0 solving (2.11) we have xn+1 = (1 − λε)Pxn + εPA∗ (y − Axn). (2.13) This procedure already appears in some books about numerical recipes. The other variant appears when it makes sense to impose an L1 norm on Rm . We should consider x1 = m i=1 ωi |xi| . Note that |xi| = xi or −xi depending on whether xi 0 or xi 0. We shall then write x = u−v with ui and vi both ≥ 0 and recast problem (2.1) as Ãx̃ = A −A u v = y, x̃ ∈ R2m + (2.14) where the notational correspondences are obvious. The variational method for producing a solution to (2.14) becomes a standard linear programming problem: Find x̃0 = arg inf m i=1 ωiui + m i=1 ωivi : Ãx̃ = y, x̃ ∈ R2m + . (2.15) And it may be a physical requirement that instead of x̃ ∈ R2m + , we actually have a more general convex constraint as above, for example, in- stead of (2.15) we may have to find: x̃0 = arg inf 2m i=1 ωix̃i : Ãx̃ = y, ai ≤ x̃i ≤ bi . (2.16) Of course 0 ≤ ai bi +∞, as consistency requires.
- 38. 10 Linear Inverse Problems: The Maxentropic Connection 2.4 Inversion of Laplace transforms from finite number of data points This example and the next are typical of situations in which one has an integral transform like: ˜ f(s) = K(s, t)f(t)m(dt), (2.17) where f(t) is in some class of functions X defined on some measure space (E, E, m), and the transform maps X onto Y in a bijective function. Y is another class of functions on some other measure space (S, Σ, σ). But it may now happen that ˜ f(s) can be observed, measured or deter- mined only for a finite number of values of s. The problem is to solve (2.17) for f(t) when one is given ˜ f(s1), ..., ˜ f(sn). A tough, particular case of (2.17), consists of numerically inverting the Laplace transform of f(t) given finitely many values ˜ f(si) of the trans- formed function. . A candidate for X is for example the class of function defined on [0, ∞) that grow not faster than p(t) exp(α0t) for some fixed α0. Then ˜ f(s) = ∞ 0 exp(−st)f(t)dt (2.18) is well defined for complex s with Rs α0. Actually it is continuous for Rs ≥ α0, and analytic in Rs α0 if f(t) is integrable. Assume (2.18) is known only for finitely many numbers in Rs α0, and it is known that f(t) can be regarded to be in a smaller class; say linear combinations of products of exponentials times polynomials times elementary trigonometric functions (say sines and cosines). This happens when for example, f(t) is the response of a linear system. As started above, the problem belongs to the class of generalized mo- ment problems. There are two approaches to follow to solve for f(t) in (2.18) given ˜ f(s1), ..., ˜ f(sn). First approach: Just solve for f(t) given the finite data. Second approach: Try to find an ˜ fl(s) in a certain class such that ˜ fl(si) = ˜ f(si), i = 1, ..., n; then invert the Laplace transform assuming that ˜ fl(s) is the right Laplace transform of the unknown f.
- 39. A collection of linear inverse problems 11 We shall have more to say about both approaches. To finish note that a conformal mapping transforming the right-hand-complex half-plane onto the unit circle allows us to relate the second approach to the famous: Pick-Nevanlina interpolation problem: Given two finite sets of complex numbers z1, ..., zn and ˜ f1, ..., ˜ fn; all of absolute value less than 1, find a holomorphic function h : U → U on the unit disk such that: h(zi) = ˜ fi i = 1, ..., n. 2.5 Fourier reconstruction from partial data Let f(t) be a real-valued, function defined on an interval [0, T ]. It is known that T 0 exp(iωt)f(t)dt = ˆ f(ω) (2.19) for ω in the finite set {±ω1, ±ω2, ..., ±ωn} . The problem is to find f(t). Sometimes one may want to think of ˆ f(ω) as being the Fourier transform of a measure dF(t) not necessarily absolutely continuous with respect to dt. For example: dF(t) = pnδ(t − tn)dt where pn are “weights” and tn are the points at which the weights are concentrated. This type of problem appears when one studies second order stationary processes. Here one determines correlations R(k) from the data and one searches for a (spectral) measure dF(t) on [0, 2π] such that: R(k) = [0,2π] exp(ikλ)dF(λ), |k| ≤ N. (2.20) Another problem in which one knows a Fourier transform occurs when reconstructing velocity profiles in stratified models of the Earth. We refer the reader to the references quoted at the end of the chapter. The problem we are interested in is to find a(x) of compact support in [0, ∞) such that: â(ω) = ∞ 0 exp(iωx)a(x)dx (2.21) is known. Here the interest lies in finding a(x)’s satisfying a1 ≤ a(x) ≤ a2. These bounds can be justified since the speed of propagation of sound is
- 40. 12 Linear Inverse Problems: The Maxentropic Connection bounded below (by the speed of propagation of sound in the air) and above (by the speed of propagation in some very rigid material). Different types of constraints appear when one reconstructs probability densities ρ(r) from their Fourier transforms. Here one knows F̂(p) such that: F̂(p) = R5 exp(ipr)ρ(r)dr (2.22) and one wants ρ(r) ≥ 0 such that ρ(r)dr = 1. Of course, if F̂(p) is known for all p in R3 , one would just invert the Fourier transform. But usually F̂(p) is known only for finitely many values of p, and measurement errors must be dealt with on top of it. A classical example relating the issues of the choice of norms and the summation of Fourier series is the following. Consider the mapping: A : C[0, 1] → l2 defined by f(t) → cn = 1 2π 1 0 exp(2iπnt)f(t)dt. Let f(t) ∈ C[0, 1] and let cn exp(−2πint) be its Fourier representa- tion. Let {dn} ∈ l2 be given by dn = cn + ε |n| , thus d − c = (dn − cn)2 1 2 = ε π2 6 which can be made as small as desired, but if g(t) = dn exp(−2iπnt) then f − g∞ = ε 1 n exp(−2πint) ∞ . Since at t = 1 we obtain the harmonic series, f − g∞ = ∞. If you (rightly) feel we seem to be cheating, just consider a sequence dN n = cn + ε n for |n| ≤ N, and zero otherwise. Then dN n is as close as you want to {cn} and gN (t) is well defined, but gN − f ∞ is as large as you want. The moral is, even though A is continuous, A−1 is not. 2.6 More on the non-continuity of the inverse A classical example of non-continuity of inverses is the following. On C[0, 1] provided with its usual l∞ norm define T : C[0, 1] → C[0, 1] by: (T x)(t) = t 0 x(s)ds.
- 41. A collection of linear inverse problems 13 Certainly the range of T is the class C1 [0, 1] of continuously differen- tiable functions on (0, 1). If y1(t) is continuously differentiable, let y2(t) = y1(t) + A sin(ωt). Note that y1 − y2∞ = |A| but x2 − x1∞ = y 2 − y 1∞ = |Aω| which can be made as large as desired keeping |A| as small as desired as well. But note that if we consider T : C1 [0, 1] → C[0, 1] and on C1 [0, 1] we put the norm: y1 − y2∗ = y1 − y2∞ + y 1 − y 2∞ then T has a continuous inverse T −1 (when restricted to C1 [0, 1] !). Comment: If on C[0, 1] we define the σ-algebra F that makes the coordinate mapping Xt : C[0, 1] → R, Xt(ω) = ω(t) measurable, and on (C[0, 1], F) we define the usual Wiener measure, then C1 [0, 1] has measure zero in C[0, 1] ! 2.7 Transportation problems and reconstruction from marginals The simplest transportation problem consists of shipping gods from m sources, in amounts si from the i-th source. Goods are to be received at n destinations, in required amounts dj at the j-th one. If there is a cost cij involved in the shipping from origin i to destination j, the simple transportation problem consists of finding: x∗ ij = inf m,n i,j=1,1 cijxij : xij ≥ 0, n j=1 xij = si, m i=1 xij = dj . This problem is usually solved in a few seconds by many software pack- ages. Actually it usually takes longer to translate a given problem into that set up, to key it in at the keyboard, than what it takes the PC to provide the solution. What is important for us here is the issue of the existence of feasible solutions, i.e., given: {si : i = 1, 2, ..., m} and {dj : j = 1, 2, ..., n} find xij ≥ 0, or aij ≤ xij ≤ bij for preassigned aij, bij, such that: m i=1 xij = dj, n j=1 xij = si and consistency requires that dj = si.
- 42. 14 Linear Inverse Problems: The Maxentropic Connection To consider explicitly an example in which the non-uniqueness of the solution appears look at the empty 3 × 4 array: 20 10 10 10 7 8 15 which we have to fill up with positive numbers whose sums by rows and columns adds up to the indicated amounts. A started method of filling the array is called the northwest corner method, which consists of: i) Fill the most northwest (empty) box with a number not exceeding the smallest constraint. ii) If it equals the column (row) constraint, delete the column (or row) and update the corresponding column (row) contained the filled box. iii) Repeat with the remaining array. iv) In case of a tie, break it a piacere. If instead of considering the most northwest, we had taken the most northeast, southwest or southeast corners we would have ended with differ- ent reconstructions. Here are two of them: 10 5 3 2 20 0 0 5 5 10 0 2 0 8 10 10 7 8 15 0 0 5 15 20 0 7 3 0 10 10 0 0 0 10 10 7 8 15 By taking convex combinations one may generate infinitely many solu- tions. This problem is a particular case of a much more general problem. To state it consider two measure spaces (X, F, µ) and (Y, Ψ, u) and form Z = X × Y Γ = F ⊗ Ψ. The problem consists of finding a measure ρ on (Z, Γ) such that ρ(A × Y ) = m(A), ρ(X × B) = n(B) (2.23) for all A ∈ F, and B ∈ Ψ. And if possible, categorize all measures ρ on (Z, Γ) such that (2.23) holds! As in the linear programming problem, one hopes that looking for: ρ∗ = arg inf c(x, y)ρ(dx, dy) : (2.23) holds leads to an easier characterization of solutions to (2.23).
- 43. A collection of linear inverse problems 15 Actually, since the class: ℘(m, n) = {ρ measure on (Z, Γ) : (2.23) holds} is a convex set, trivially non-empty for m ⊗ n belongs to it, any lower semicontinuous, convex function F : ℘(m, n) → R is likely to provide us with candidates solving (2.23) when some compactness is available. A variation on the theme of (2.23) and usually called the problem of tomographic reconstruction consists of finding a positive function ρ on some compact K of a metric space from the knowledge of: ρ(ξ)µi(dξ) = di (2.24) where the measures µi(dξ) are usually concentrated on sets of measure zero with respect to a given measure m(dξ) on (K, β), β denoting the Borel sets in K. 2.8 CAT CAT stands for either computer axial tomography or computer aided to- mography. Either name describes a complex process involves obtaining X-ray images of an object. The technique consists of a combination of hardware-software by means of which an image of a series of consecutive slices (tomos) of an object, usually part of a human body, are obtained. The image consists of an X-ray plaque of each given section (slice). Different aspects of the his- toric development of the tomographic process are contained in [10]-[13]. In [11] you will find a brief account of the story from the point of view of the research-development-commercialization process, whereas the other two emphasize the connection with the evolution of the development of the mathematics of the inverse problem that is needed to present the images of the slices. Let us begin by describing the phenomenological model of light ab- sorption by tissue. Everything is contained in the following simple, one- dimensional model: consider a slab of material, of linear density ρ(x) ex- tending from 0 to L. Let the radiation incident at x = 0 be of intensity I0 and denote by I(x) the intensity of the radiation reaching point x. Thus the radiation reaching x + dx is I(x + dx) = I(x) − λρ(x)I(x)dx
- 44. 16 Linear Inverse Problems: The Maxentropic Connection from which we obtain that the radiation at x = L is I(L) = I0 exp − L 0 λρ(x)dx . (2.25) We shall follow [10] for the description of conventional tomography and to present the reader with one more set of references to the subject. Ac- cording to [13] it seems that the basics were patented by the Frenchman Bocade around 1921. Consider two parallel planes, separated by a distance L. 0 L V1 V2 P1 P2 Fig. 2.1 Basic scheme. Consider two points 1 and 2 on each plane. After some time ∆ they will be at 1 and 2 respectively, with 11 = V1∆ and 22 = V2∆. Using this as starting point, we leave it to the reader to prove that there is a plane Pf which stays fixed. Write V2 = αV1 and determine the position of the plane in terms of α. Imagine now a source of X-rays on plane 1 and a photographic plate on plane 2. Since the motion of the planes is much, much slower than the speed of light, we might imagine that situation depicted in figure 2.2 holds instantaneously, and in a “more realistic” schematic depiction of an X-raying process we will have something like figure 1 in [10]. Clearly, the details in the fixed plane get more (negatively) illuminated than objects in non-fixed planes, so they will appear less blurred in the ex- posed plate. If this process is repeated for different relative source-plate po-
- 45. A collection of linear inverse problems 17 P1 P2 1 1 2 2 V2∆ V1∆ Fig. 2.2 Basic geometry of the CAT process. sitions and for different source-plate speeds we obtain a sequence of images which are the basis for the inverse problem of tomographic reconstruction. For this we need to put in some detail into figure 2.3 with respect to figure 2.4, in which both the edges of the photographic plate and the center of the photographic plate are joined to the X-ray source by solid lines, denote by θ the angle of this last line and the horizontal axis. Let η be the vertical distance measured from the center of the plate, and let θ(η) be the angle between the ray from the source to η and the horizontal axis. If we assume that the object is located in the central area between the plate (which slides along the x = −L vertical axis) and the X-ray source (which slides along x = L vertical axis), and if we denote by γ the dotted line in figure 2.4, assuming that this configuration is held during a time δ during which the intensity of the radiation collected at η is δI(η, θ) = δI0(η, θ) exp − L −L λρ(x, x tan θ + L − x 2L η) dx cos θ . (2.26) Here y = x tan θ + L−x 2L η is the vertical coordinate of a point in the object whose horizontal coordinate is x. We are approximating θ(η) by θ and the length element along the line γ is approximated by dx cos θ . It is usually assumed that the error incurred in this approximation is negligible.
- 46. 18 Linear Inverse Problems: The Maxentropic Connection Fig. 2.3 Tomographic image formation. In (2.26) dI0(η, θ) would be the intensity recorded in the plate if there were no object in the path of the X-rays. If you do not find the approxima- tion in (2.26) satisfactory, the exponent should be replaced by the integral γ ρdl(θ) (2.27) of the density ρ(x, y) of the object along the ray γ(θ). For a given pair (V1, V2) of source-plate velocities, the central ray rotates in angle from θ0 to θ1, the total radiation collected at η in the plate is I(η) = θ1 θ0 dθ dI0 dθ (η, θ) exp −λ γ ρdl(θ) (2.28) I(η) ≈ θ1 θ0 dθC exp −λ L −L ρ(x, x tan θ + L − x 2L η) dx cos θ . (2.29) It would be reasonable to assume that for not too long an exposition time dI0(η,θ) dθ = C is a constant. Notice now that when θ1 −θ0 θ0, which has been brought about by the improved technology, we can rewrite (2.28) as ln dI(η) dI0(η) ∝ γ ρdl(θ). (2.30)
- 47. A collection of linear inverse problems 19 tomographic plane x-axis y-axis X-ray plaque x y Fig. 2.4 Actual CAT setup. If we consider the image to be formed by a parallel stream of X-rays impinging on a plate (actually a linear array of photosensitive detectors) lying along the line characterized by the unit vector. The X-ray image of the two-dimensional density ρ(x, y) at a point t along the plate is (Rθρ)(t) = ∞ −∞ ρ(t cos θ − s sin θ, t sin θ + s cos θ)ds. After this lengthy digression, we are ready to state what is commonly known as the tomographic reconstruction problem. Let V be a given Banach space of functions defined in a unit disk in R2 (or in some bounded set in Rn ). For a given family of lines with unit normals ηi = (− sin θi, cosθi) (or hyperplanes with normals ηi) for i = 1, 2, ..., N define the projection operators (Rθ(i)ρ)(x) = ∞ −∞ ρ(x + sηi)ds (2.31) where x denotes a point in the line (or in the hyperplane). The problem consists in finding ρ ∈ V from the knowledge of (2.31).
- 48. 20 Linear Inverse Problems: The Maxentropic Connection Comment: It is left to the reader to convince himself/herself that (2.31) describes marginalization of ρ along successively rotated axes. Thus standard reconstructions from marginals fall within the class of problems described above. We will examine some aspects of these problems in Chap- ter 9. 2.9 Abstract spline interpolation The classical interpolation problem consists of finding a σ ∈ C2 [0, 1] such that σ(ti) = zi for i = 1, 2, ..., n. The procedure consists in minimizing: F(σ) = 1 0 (σ (t))2 dt (2.32) and the result is contained in: Theorem 2.1. There is a unique σ ∈ C2 [0, 1] minimizing (2.32) such that: a) σ(ti) = zi. b) σ is a cubic polynomial in [ti, ti+1] , i = 1, 2, ..., n − 1. c) σ (t1) = σ (tn) = 0. d) σ is linear on [0, t1] and [tn, 1] . Instead of the interpolation conditions σ(ti) = zi for i = 1, 2, ..., n ; one could search for a function σ satisfying: 1 0 σ(t)Pl(t)ω(t)dt = zi (2.33) where ω(t) is some weight function and the {Pi(t), i ≥ 1} are the orthogo- nal polynomials with respect to ω(t), i.e., they are orthogonal in 2([0, 1], ω). The problem can be abstracted into the following setup: Let X, Y, Z be Hilbert spaces. Let A : X → Y and B : X → Z be two bounded, linear surjections. Let y0 ∈ Y and the problem consists of finding x∗ 0 ∈ X such that x∗ 0 = arg inf {BxZ : Ax = y0} . (2.34) The existence and uniqueness of x∗ 0 is covered by: Theorem 2.2. If a) B(Ker(A)) is closed in Z.
- 49. A collection of linear inverse problems 21 b) Ker(B) ∩ Ker(A) = {0X}. (0X denotes the zero element of X.) c) y0 ∈ Y. Then there is a unique x0 such that Ax0 = y0 and B(x0)Z = inf {BxZ : Ax = y} . Proof. A being surjective, implies the existence of x1 ∈ X such that A−1 {y0} = x1 + Ker(A). Thus B(A−1 {x0}) = B(x1) + B(Ker(A)). From the continuity of A and B we obtain the closedness of B(Ker(A)) and hence, B(A−1 {x0}), which being a translate of a subspace is convex. Let z0 denote the projection of 0Z onto B(A−1 {x0}), thus z0 ∈ B(x1) + B(A−1 {x0}) which means that there exists x ∈ Ker(A) such that z0 = B(x1) + B(x) = B(x1 + x) or there exists x ∈ Ker(A) such that x1 + x ∈ B−1 {z0}. But, A(x + x1) = A(x1) + A(x) = y0 + 0X = y0 or x + x1 ∈ A−1 {y0} ∩ B−1 {z}, i.e., this set is not empty. Now let x1, x2 be two different elements in A−1 {y0} ∩ B−1 {z}. Then, B(x1 − x2) = 0Z, A(x1 − x2) = 0Y plus assumption b) implies that x1 = x2 or A−1 {y0} ∩ B−1 {z} is a single- ton, which we denote by x0. 2.10 Bibliographical comments and references Generic comment: References will be cited as [n] or [m-n]. The first is a reference n at the end of the present chapter, whereas [m-n] denotes the nth reference in the list at the end of Chapter m. A couple of new (as of 1998) books in which finite dimensional inverse problems are treated in full are: [1] and [2]. A nice collection of inverse problems and the basic theory to deal with them is exposed in [2] and in [3]. Even more theory and applications, both linear and nonlinear appear in [4], [5] and [6]. Two related references on the trigonometric moment problem are [7], [7-7] and [7-8]. On the transportation problem take a look at any
- 50. 22 Linear Inverse Problems: The Maxentropic Connection book on Operations Research. For information about tomography, its his- tory, and further references, consult with [8]-[13], and with many of the papers described in Chapter 9. References [14]-[16] contain historical remarks and description of the evolution of attempts to solve the inverse problem of CAT and related imaging problems in various fields as well as a list of further references. For related work on transportation problems consider references [9-18] and [9-27] and for a nice and short expose on abstract splines take a look at [17] from which we took material for Section 9.10. References [1] Mansen, P. C. “Rank-Deficient and Discrete Ill-Posed Problems”. SIAM, Philadelphia, 1998. [2] Bertero, M. and Bocacci, P. “Introduction to Inverse Problems in Imaging”. IOP Publishing, Bristol, 1998. [3] Parker, R. “Geophysical Inverse Theory”. Princeton Univ. Press, Princeton 1994. [4] Ramm, A. G. “Scattering by Obstacles”. Kluwer Acad. Pub., Dor- drecht, 1986. [5] Ramm, A. G. “Multidimensional Inverse Problems”. Longman Scien- tific, 1992. [6] Laurentiev, M. M. “Some Improperly Posed Problems of Mathematical Physics”. Springer-Verlag. 1967. [7] Romanov, V. G. “Inverse Problems of Mathematical Physics”. VNU Science Press, Utrecht, 1987. [8] Orphanoudakis, S. and Strohben, J. “Mathematical model of conven- tional tomography”. Med. Phys., Vol. 3 (1976), pp. 224-232. [9] Adams, J. L. “Flying Buttresses, Entropy and O. Rings”. Harvard Univ. Press, Cambridge, 1991. [10] Shepp, L. A. and Kruskal, J. B. “Computerized Tomography: the new medical X-ray technology”. Ann. Math. Monthly, Vol. 85 (1978) pp. 420-439. [11] Panton, D. “Mathematical reconstruction techniques in computer axial tomography”. Math. Scientist, Vol. 6 (1981), pp. 87-102. [12] Fuchs, J., Mast, K., Hermann, A. and Lackner, K. “Two-dimensional reconstruction of the radiation power density in ASDEX upgrade”. 21st. E.P.S. Conf. on Controlled Fusion and Plasma Physics. Joffrin, E., Platz, P. and Scott, P. (eds), Publ. E.P.S., Paris, 1994.
- 51. A collection of linear inverse problems 23 [13] Friedman, A. “Image reconstruction in oil refinery” in Mathematics in Industrial Problems, IMA Volumes in Mathematics and its Applica- tions, Vol. 16, Springer-Verlag, 1988. [14] Hounsfield, G. “Historical notes on computerized axial tomography”. The J. of the Canadian Assos. of Radiologists, Vol. 27 (1976), pp.135- 191. [15] Brooks, R and Di Chiro, G. “Principles of computer assisted tomog- raphy (CAT) in radiographic and radioisotopic imaging”. Phys. Med. Biol., Vol. 21 (1976), pp. 689-732. [16] Gordon, R. and Herman, G. “Three dimensional reconstruction from projections: a review of algorithms”. Inter. J. Cytology, Vol. 39 (1974), pp. 111-151. [17] Champion, R., Lenard, T. C. and Mills, T. M. “An introduction to abstract splines”, Math. Scientist. Vol.21, (1996), pp. 8-26.
- 52. Chapter 3 The basics about linear inverse problems 3.1 Problem statements Much of what we shall say below would apply to linear as well as to non- linear problems but, since the maxentropic methods we shall develop in the next chapters are designed to be applied to linear problems, we shall concentrate on those. This chapter is to present some basic issues in inverse problem theory. Let V and W be two Banach spaces with norms . . .V and . . .W respectively. The subscripts shall be dropped when no ambiguity arises. Let A : V1 ⊂ V → W be a linear operator defined on a linear subset V1 of V. We do not assume that V1 is the largest domain on which A may be defined. In actual modeling, the points of V1 are the possible “inputs” or “sources” or “impulses”. The operator A models some physical effect or propagation and the points of W represent “data” or “observations” or “outputs” of the system. In the equation: Ax = y (3.1) the computation of y when A and x are known is called a direct problem, and solving for x when A and y are known is called an inverse problem. Exercise 3.1. Formulate the problem of determining A in (3.1), when y is given for a countable (usually finite) class of x’s. From the purely analytical point of view, the following problems may arise when solving (3.1) for x : i) The inverse operator A−1 is not defined. 25
- 53. 26 Linear Inverse Problems: The Maxentropic Connection ii) A−1 {y} has more than one element for each y ∈ R(A). iii) A−1 may not be continuous. Item (i) means that A is not surjective. But even if it were, the map A, may not be 1 : 1 so there are many inverses. But even if it were, item (ii) means that A may not be 1 : 1 so there are (infinitely) many inverses. But even if the operator A is a bijection, its inverse may not be con- tinuous. This means that small errors in the measured data may produce large indetermination in the possible inputs. To describe the solution to the inverse problem we may introduce the following Definition 3.1. An operator B : W1 ⊂ W → V1 ⊂ V solves (3.1) when- ever: A(By) = y. (3.2) Comments: The class W1 should be thought of as the “available” data and the fact that By ∈ V1 is just a consistency requirement. With this notation, cases (i), (ii) and (iii) can be rephrased as: a) There may exist no B such that (3.2) holds. b) There exist many B’s such that (3.2) holds. c) Given a B such that (3.2) holds, B is not continuous. Definition 3.2. We say that problem (3.1) is well posed when, there exists a unique B : R(A) → V which is continuous and (3.2) holds. Comment: It is clear that the continuity of B is relative to the topology induced on R(A) by the topology on W associated with . . .W . The notion of well posedness may be relativized to pairs V1, W1 without problem. It is easy to define an inverse when A is bijective. When it is not, and K = Ker(A) is known, then  : V̂ = V/K → W1 = Im(A) is a bijection. When A is bounded, W1 is a closed subspace of W and a Banach space on its own. As a consequence of the open mapping theorem, see Appendix C, Â−1 exists and is continuous. Theorem 3.1. Let A : V → W be compact and suppose that the fac- tor space V̂ = V/K has infinite dimension. Then there exists a sequence {xn : n ≥ 1} ⊂ V such that Axn → 0, but {xn} does not converge. Even
- 54. The basics about linear inverse problems 27 worse, the {xn} can be chosen so that xn → ∞. In particular if A is 1:1, the inverse A−1 : W1 = A(V ) ⊂ W → V is unbounded. Proof. The operator  : V̂ → Y induced by A on the factor space V̂ = V/K, defined by Â[x] = Ax is compact and 1:1. If Â−1 : W1 → V̂ , then Â−1 cannot be bounded. If it were, I = Â−1  and V̂ → V̂ would be compact which is impossible if V̂ is infinite dimensional. Since Â−1 is unbounded, there exists a sequence {xn : n ≥ 1} in V and a corresponding sequence {[xn] : n ≥ 1} in V̂ such that Axn → 0 but [xn] = 1. Let ξn ∈ K be such that xn + ξn ≥ a 1 and put x n = xn + ξn Axn , Ax n → 0 but x n ≥ a Axn → ∞ as n → ∞. Comments: To avoid contradiction with the comments above the statement of the Theorem, either i) V̂ is finite dimensional (and Â−1 is bounded) or ii) V̂ is infinite dimensional and W1 is of first category in W. It can be represented as union of a sequence of nowhere dense sets. Besides the issues described above, there are two more classes of issues that are involved in solving (3.1). The first class is related to the fact that both, the model A and the data y, depend on actual measurements and our problem does not merely consist in solving (3.1) but in giving meaning to the problem. (P) Find x ∈ V1 such that Ax = y where A ∈ N(A0, δ) and y ∈ B(y0,ε) Here y ∈ B(y0, ε) is clear: it denotes the ball of radius ε around y0 in the . . .W distance. The indeterminacy in A may be due to indeterminacy in some physical parameters, and thus N(Ao, δ) denotes some appropriate neighborhood of A0. Tomographic reconstruction, see [9-24] is a typical situation of this kind. The other issue compounding the difficulties is related to the fact that in applications, both x and y lie in infinite dimensional spaces, and that to describe B and By one has to resort to finite dimensional setups. To be more precise, one has to be able to produce families of projections Pn : V → Vn and Qn : W → Wn, where Vn and Wn are of appropriate finite dimension, n, and such that x − PnxV → 0 for every x, y − QnyW → 0
- 55. Exploring the Variety of Random Documents with Different Content
- 56. “XX. ... To all the professed knights, both in winter and summer, we give, if they can be procured, WHITE GARMENTS, that those who have cast behind them a dark life may know that they are to commend themselves to their Creator by a pure and white life. For what is whiteness but perfect chastity, and chastity is the security of the soul and the health of the body. And unless every knight shall continue chaste, he shall not come to perpetual rest, nor see God, as the apostle Paul witnesseth: Follow after peace with all men, and chastity, without which no man shall see God.... “XXI. ... Let all the esquires and retainers be clothed in black garments: but if such cannot be found, let them have what can be procured in the province where they live, so that they be of one colour, and such as is of a meaner character, viz. brown. “XXII. It is granted to none to wear WHITE habits, or to have WHITE mantles, excepting the above-named knights of Christ. “XXXVII. We will not that gold or silver, which is the mark of private wealth, should ever be seen on your bridles, breastplates, or spurs, nor should it be permitted to any brother to buy such. If, indeed, such like furniture shall have been charitably bestowed upon you, the gold and silver must be so coloured, that its splendour and beauty may not impart to the wearer an appearance of arrogance beyond his fellows. “XLI. It is in no wise lawful for any of the brothers to receive letters from his parents, or from any man, or to send letters, without the license of the Master, or of the procurator. After the brother shall have had leave, they must be read in the presence of the Master, if it so pleaseth him. If, indeed, anything whatever shall have been directed to him from his parents, let him not presume to receive it until information has been first given to the Master. But in this regulation the Master and the procurators of the houses are not included.
- 57. “XLII. We forbid, and we resolutely condemn, all tales related by any brother, of the follies and irregularities of which he hath been guilty in the world, or in military matters, either with his brother or with any other man. It shall not be permitted him to speak with his brother of the irregularities of other men, nor of the delights of the flesh with miserable women; and if by chance he should hear another discoursing of such things, he shall make him silent, or with the swift foot of obedience he shall depart from him as soon as he is able, and shall lend not the ear of the heart to the vender of idle tales. “XLIII. If any gift shall be made to a brother, let it be taken to the Master or the treasurer. If, indeed, his friend or his parent will consent to make the gift only on condition that he useth it himself, he must not receive it until permission hath been obtained from the Master. And whosoever shall have received a present, let it not grieve him if it be given to another. Yea, let him know assuredly, that if he be angry at it, he striveth against God. “XLVI. We are all of opinion that none of you should dare to follow the sport of catching one bird with another: for it is not agreeable unto religion for you to be addicted unto worldly delights, but rather willingly to hear the precepts of the Lord, constantly to kneel down to prayer, and daily to confess your sins before God with sighs and tears. Let no brother, for the above especial reason, presume to go forth with a man following such diversions with a hawk, or with any other bird. “XLVII. Forasmuch as it becometh all religion to behave decently and humbly without laughter, and to speak sparingly but sensibly, and not in a loud tone, we specially command and direct every professed brother that he venture not to shoot in the woods either with a long- bow or a cross-bow; and for the same reason, that he venture not to accompany another who shall do the like, except it be for the purpose of protecting him from the perfidious infidel; neither shall he dare to halloo, or to talk to a dog, nor shall he spur his horse with a desire of securing the game.
- 58. “LI. Under Divine Providence, as we do believe, this new kind of religion was introduced by you in the holy places, that is to say, the union of WARFARE with RELIGION, so that religion, being armed, maketh her way by the sword, and smiteth the enemy without sin. Therefore we do rightly adjudge, since ye are called Knights of the Temple, that for your renowned merit, and especial gift of godliness, ye ought to have lands and men, and possess husbandmen and justly govern them, and the customary services ought to be specially rendered unto you. “LV. We permit you to have married brothers in this manner, if such should seek to participate in the benefit of your fraternity; let both the man and his wife grant, from and after their death, their respective portions of property, and whatever more they acquire in after life, to the unity of the common chapter; and, in the interim, let them exercise an honest life, and labour to do good to the brethren: but they are not permitted to appear in the white habit and white mantle. If the husband dies first, he must leave his portion of the patrimony to the brethren, and the wife shall have her maintenance out of the residue, and let her depart therewith; for we consider it most improper that such women should remain in one and the same house with the brethren who have promised chastity unto God. “LVI. It is moreover exceedingly dangerous to join sisters with you in your holy profession, for the ancient enemy hath drawn many away from the right path to paradise through the society of women: therefore, dear brothers, that the flower of righteousness may always flourish amongst you, let this custom from henceforth be utterly done away with. “LXIV. The brothers who are journeying through different provinces should observe the rule, so far as they are able, in their meat and drink, and let them attend to it in other matters, and live irreproachably, that they may get a good name out of doors. Let them not tarnish their religious purpose either by word or deed; let them afford to all with whom they may be associated, an example of wisdom, and a perseverance in all good works. Let him with whom
- 59. they lodge be a man of the best repute, and, if it be possible, let not the house of the host on that night be without a light, lest the dark enemy (from whom God preserve us) should find some opportunity. “LXVIII. Care must be taken that no brother, powerful or weak, strong or feeble, desirous of exalting himself, becoming proud by degrees, or defending his own fault, remain unchastened. If he showeth a disposition to amend, let a stricter system of correction be added: but if by godly admonition and earnest reasoning he will not be amended, but will go on more and more lifting himself up with pride, then let him be cast out of the holy flock in obedience to the apostle, Take away evil from among you. It is necessary that from the society of the Faithful Brothers the dying sheep be removed. But let the Master, who ought to hold the staff and the rod in his hand, that is to say, the staff that he may support the infirmities of the weak, and the rod that he may with the zeal of rectitude strike down the vices of delinquents; let him study, with the counsel of the patriarch and with spiritual circumspection, to act so that, as blessed Maximus saith, The sinner be not encouraged by easy lenity, nor hardened in his iniquity by immoderate severity. Lastly. We hold it dangerous to all religion to gaze too much on the countenance of women; and therefore no brother shall presume to kiss neither widow, nor virgin, nor mother, nor sister, nor aunt, nor any other woman. Let the knighthood of Christ shun feminine kisses, through which men have very often been drawn into danger, so that each, with a pure conscience and secure life, may be able to walk everlastingly in the sight of God.” After the confirmation by a Papal bull of the rules and statutes of the order, Hugh de Payens proceeded to France, and from thence he came to England, and the following account is given of his arrival, in the Saxon chronicle. “This same year, (A. D. 1128,) Hugh of the Temple came from Jerusalem to the king in Normandy, and the king received him with much honour, and gave him much treasure in gold and silver, and afterwards he sent him into England, and there he
- 60. was well received by all good men, and all gave him treasure, and in Scotland also, and they sent in all a great sum in gold and silver by him to Jerusalem, and there went with him and after him so great a number as never before since the days of Pope Urban.”[5] Grants of lands, as well as of money, were at the same time made to Hugh de Payens and his brethren, some of which were shortly afterwards confirmed by King Stephen on his accession to the throne, (A. D. 1135.) Among these is a grant of the manor of Bistelesham made to the Templars by Count Robert de Ferrara, and a grant of the church of Langeforde in Bedfordshire made by Simon de Wahull, and Sibylla his wife, and Walter their son. Hugh de Payens, before his departure, placed a Knight Templar at the head of the order in this country, who was called the Prior of the Temple, and was the procurator and vicegerent of the Master. It was his duty to manage the estates granted to the fraternity, and to transmit the revenues to Jerusalem. He was also delegated with the power of admitting members into the order, subject to the control and direction of the Master, and was to provide means of transport for such newly-admitted brethren to the far east, to enable them to fulfil the duties of their profession. As the houses of the Temple increased in number in England, sub-priors came to be appointed, and the superior of the order in this country was then called the Grand Prior, and afterwards Master of the Temple. An astonishing enthusiasm was excited throughout Christendom in behalf of the Templars; princes and nobles, sovereigns and their subjects, vied with each other in heaping gifts and benefits upon them, and scarce a will of importance was made without an article in it in their favour. Many illustrious persons on their deathbeds took the vows, that they might be buried in the habit of the order; and sovereign princes, quitting the government of their kingdoms, enrolled themselves amongst the holy fraternity, and bequeathed even their dominions to the Master and the brethren of the Temple. St. Bernard, at the request of Hugh de Payens, took up his powerful pen in their behalf. In a famous discourse “In praise of the New
- 61. Chivalry,” the holy abbot sets forth, in eloquent and enthusiastic terms, the spiritual advantages and blessings enjoyed by the military friars of the Temple over all other warriors. He draws a curious picture of the relative situations and circumstances of the secular soldiery and the soldiery of Christ, and shows how different in the sight of God are the bloodshed and slaughter perpetrated by the one, from that committed by the other. Addressing himself to the secular soldiers he says “Ye cover your horses with silken trappings, and I know not how much fine cloth hangs pendent from your coats of mail. Ye paint your spears, shields, and saddles; your bridles and spurs are adorned on all sides with gold, and silver, and gems, and with all this pomp, with a shameful fury and a reckless insensibility, ye rush on to death. Are these military ensigns, or are they not rather the garnishments of women? Can it happen that the sharp- pointed sword of the enemy will respect gold, will it spare gems, will it be unable to penetrate the silken garment? Lastly, as ye yourselves have often experienced, three things are indispensably necessary to the success of the soldier; he must be bold, active, and circumspect; quick in running, prompt in striking; ye, however, to the disgust of the eye, nourish your hair after the manner of women, ye gather around your footsteps long and flowing vestures, ye bury up your delicate and tender hands in ample and wide-spreading sleeves. Among you, indeed, nought provoketh war or awakeneth strife, but either an irrational impulse of anger, or an insane lust of glory, or the covetous desire of possessing another man’s lands and possessions. In such causes it is neither safe to slay nor to be slain. “And now we will briefly display the mode of life of the Knights of Christ, such as it is in the field and in the convent, by which means it will be made plainly manifest to what extent the soldiery of God and the soldiery of the WORLD differ from one another.... The soldiers of Christ live together in common in an agreeable but frugal manner, without wives, and without children; and that nothing may be wanting to evangelical perfection, they dwell together without separate property of any kind, in one house, under one rule, careful to preserve the unity of the spirit in the bond of peace. You may say,
- 62. that to the whole multitude there is but one heart and one soul, as each one in no respect followeth after his own will or desire, but is diligent to do the will of the Master. They are never idle nor rambling abroad, but when they are not in the field, that they may not eat their bread in idleness, they are fitting and repairing their armour and their clothing, or employing themselves in such occupations as the will of the Master requireth, or their common necessities render expedient. Among them there is no distinction of persons; respect is paid to the best and most virtuous, not the most noble. They participate in each other’s honour, they bear one another’s burthens, that they may fulfil the law of Christ. An insolent expression, a useless undertaking, immoderate laughter, the least murmur or whispering, if found out, passeth not without severe rebuke. They detest cards and dice, they shun the sports of the field, and take no delight in that ludicrous catching of birds, (hawking,) which men are wont to indulge in. Jesters, and soothsayers, and storytellers, scurrilous songs, shows and games, they contemptuously despise and abominate as vanities and mad follies. They cut their hair, knowing that, according to the apostle, it is not seemly in a man to have long hair. They are never combed, seldom washed, but appear rather with rough neglected hair, foul with dust, and with skins browned by the sun and their coats of mail. Moreover, on the approach of battle they fortify themselves with faith within, and with steel without, and not with gold, so that armed and not adorned, they may strike terror into the enemy, rather than awaken his lust of plunder. They strive earnestly to possess strong and swift horses, but not garnished with ornaments or decked with trappings, thinking of battle and of victory, and not of pomp and show, and studying to inspire fear rather than admiration.... “There is a Temple at Jerusalem in which they dwell together, unequal, it is true, as a building, to that ancient and most famous one of Solomon, but not inferior in glory. For truly, the entire magnificence of that consisted in corrupt things, in gold and silver, in carved stone, and in a variety of woods; but the whole beauty of this resteth in the adornment of an agreeable conversation, in the godly
- 63. devotion of its inmates, and their beautifully-ordered mode of life. That was admired for its various external beauties, this is venerated for its different virtues and sacred actions, as becomes the sanctity of the house of God, who delighteth not so much in polished marbles as in well-ordered behaviour, and regardeth pure minds more than gilded walls. The face likewise of this Temple is adorned with arms, not with gems, and the wall, instead of the ancient golden chapiters, is covered around with pendent shields. Instead of the ancient candelabra, censers, and lavers, the house is on all sides furnished with bridles, saddles, and lances, all which plainly demonstrate that the soldiers burn with the same zeal for the house of God, as that which formerly animated their great leader, when, vehemently enraged, he entered into the Temple, and with that most sacred hand, armed not with steel, but with a scourge which he had made of small thongs, drove out the merchants, poured out the changers’ money, and overthrew the tables of them that sold doves; most indignantly condemning the pollution of the house of prayer, by the making of it a place of merchandize.” St. Bernard then congratulates Jerusalem on the advent of the soldiers of Christ, “Be joyful, O Jerusalem,” says he, in the words of the prophet Isaiah, “and know that the time of thy visitation hath arrived. Arise now, shake thyself from the dust, c., c. Hail, O holy city, hallowed by the tabernacle of the Most High! Hail, city of the great King, wherein so many wonderful and welcome miracles have been perpetually displayed. Hail, mistress of the nations, princess of provinces, possession of patriarchs, mother of the prophets and apostles, initiatress of the faith, glory of the christian people, whom God hath on that account always from the beginning permitted to be visited with affliction, that thou mightest thus be the occasion of virtue as well as of salvation to brave men. Hail, land of promise, which, formerly flowing only with milk and honey for thy possessors, now stretchest forth the food of life, and the means of salvation to the entire world. Most excellent and happy land, I say, which, receiving the celestial grain from the recess of the paternal heart, in that most fruitful bosom of thine, has produced such rich harvests of
- 64. martyrs from the heavenly seed, and whose fertile soil has no less manifoldly engendered fruit a thirtieth, sixtieth, and a hundredfold in the remaining race of all the faithful throughout the entire world. Whence most agreeably satiated, and most abundantly crammed with the great store of thy pleasantness, those who have seen thee diffuse around them in every place the remembrance of thy abundant sweetness, and tell of the magnificence of thy glory to the very end of the earth to those who have not seen thee, and relate the wonderful things that are done in thee. “Glorious things are spoken concerning thee, city of God!”
- 65. CHAPTER II. Hugh de Payens returns to Palestine—His death—Robert de Craon made Master—The second Crusade—The Templars assume the Red Cross—Lands, manors, and churches granted them in England—Bernard de Tremelay made Master—He is slain by the Infidels—Bertrand de Blanquefort made Master—He is taken prisoner, and sent in chains to Aleppo—the Pope confers vast privileges upon the Templars—The knights, priests, and serving brethren of the order—Their religious and military enthusiasm— Their war banner called Beauseant—Rise of the rival religio- military order of the Hospital of St. John—Contests between Saladin and the Templars—Imprisonment and death of the Grand Master—The new Master and the Patriarch go to England for succour—Consecration of the Temple church at London. “We heard the tecbir, so the Arabs call Their shout of onset, when with loud appeal They challenge heaven, as if commanding conquest.” Hugh de Payens, having now laid in Europe the foundations of the great monastic and military institution of the Temple, which was destined shortly to spread its ramifications to the remotest quarters of Christendom, returned to Palestine at the head of a valiant band of newly-elected Templars, drawn principally from England and France. On their arrival at Jerusalem they were received with great distinction by the king, the clergy, and the barons of the Latin kingdom. Hugh de Payens died, however, shortly after his return, and was succeeded (A. D. 1136) by the Lord Robert, surnamed the
- 66. Burgundian, (son-in-law of Anselm, Archbishop of Canterbury,) who, after the death of his wife, had taken the vows and the habit of the Templars.[6] At this period the fierce religious and military enthusiasm of the Mussulmen had been again aroused by the warlike Zinghis, and his son Noureddin, two of the most famous chieftains of the age. The one was named Emod-ed-deen, “Pillar of religion;” and the other Nour-ed-deen, “Light of Religion,” vulgarly, Noureddin. The Templars were worsted by overpowering numbers. The latin kingdom of Jerusalem was shaken to its foundations, and the oriental clergy in trepidation and alarm sent urgent letters to the Pope for assistance. The Lord Robert, Master of the Temple, had at this period (A. D. 1146) been succeeded by Everard des Barres, Prior of France, who convened a general chapter of the order at Paris, which was attended by Pope Eugenius the Third, Louis the Seventh, king of France, and many prelates, princes, and nobles, from all parts of Christendom. The second crusade was there arranged, and the Templars, with the sanction of the Pope, assumed the blood-red cross, the symbol of martyrdom, as the distinguishing badge of the order, which was appointed to be worn on their habits and mantles on the left side of the breast over the heart, whence they came afterwards to be known by the name of the Red Friars and the Red Cross Knights. At this famous assembly various donations were made to the Templars, to enable them to provide more effectually for the defence of the Holy Land. Bernard Baliol, through love of God and for the good of his soul, granted them his estate of Wedelee, in Hertfordshire, which afterwards formed part of the preceptory of Temple Dynnesley. This grant is expressed to be made at the chapter held at Easter, in Paris, in the presence of the Pope, the king of France, several archbishops, and one hundred and thirty Knights Templars clad in white mantles.[7] Brother Everard des Barres, the newly-elected Master of the Temple, having collected together all the brethren from the western provinces, joined the second crusade to Palestine. During the march
- 67. through Asia Minor, the rear of the christian army was protected by the Templars, who greatly signalized themselves on every occasion. Odo of Deuil, or Diagolum, the chaplain of King Louis, and his constant attendant upon this expedition, informs us that the king loved to see the frugality and simplicity of the Templars, and to imitate it; he praised their union and disinterestedness, admired above all things the attention they paid to their accoutrements, and their care in husbanding and preserving their equipage and munitions of war, and proposed them as a model to the rest of the army.[8] Conrad, emperor of Germany, had preceded King Louis at the head of a powerful army, which was cut to pieces by the infidels in the north of Asia; he fled to Constantinople, embarked on board some merchant vessels, and arrived with only a few attendants at Jerusalem, where he was received and entertained by the Templars, and was lodged in the Temple in the Holy City. Shortly afterwards King Louis arrived, accompanied by the new Master of the Temple, Everard des Barres; and the Templars now unfolded for the first time the red-cross banner in the field of battle. This was a white standard made of woollen stuff, having in the centre of it the blood-red cross granted by Pope Eugenius. The two monarchs, Louis and Conrad, took the field, supported by the Templars, and laid siege to the magnificent city of Damascus, “the Queen of Syria,” which was defended by the great Noureddin, “Light of religion,” and his brother Saif-eddin, “Sword of the faith.” The services rendered by the Templars are thus gratefully recorded in the following letter sent by Louis, the French king, to his minister and vicegerent, the famous Suger, abbot of St. Denis: “I cannot imagine how we could have subsisted for even the smallest space of time in these parts, had it not been for their (the Templars’) support and assistance, which have never failed me from the first day I set foot in these lands up to the time of my despatching this letter—a succour ably afforded and generously persevered in. I therefore earnestly beseech you, that as these brothers of the Temple have
- 68. hitherto been blessed with the love of God, so now they may be gladdened and sustained by our love and favour. I have to inform you that they have lent me a considerable sum of money, which must be repaid to them quickly, that their house may not suffer, and that I may keep my word....”[9] Among the English nobility who enlisted in the second crusade were the two renowned warriors, Roger de Mowbray and William de Warrenne. Roger de Mowbray was one of the most powerful and warlike of the barons of England, and was one of the victorious leaders at the famous battle of the standard: he marched with King Louis to Palestine; fought under the banners of the Temple against the infidels, and, smitten with admiration of the piety and valour of the holy warriors of the order, he gave them, on his return to England, many valuable estates and possessions. Among these were the manors of Kileby and Witheley, divers lands in the isle of Axholme, the town of Balshall in the county of Warwick, and various places in Yorkshire: and so munificent were his donations, that the Templars conceded to him and to his heirs various special privileges. About the same period, Stephen, King of England, granted and confirmed “to God and the blessed Virgin Mary, and to the brethren of the Knighthood of the Temple of Solomon at Jerusalem, all the manor of Cressynge, with the advowson of the church of the same manor, and also the manors of Egle and Witham.” Queen Matilda, likewise, granted them the manor of Covele or Cowley in Oxfordshire, two mills in the same county, common of pasture in Shotover forest, and the church of Stretton in Rutland. Ralph de Hastings and William de Hastings also gave to the Templars, in the same reign, (A. D. 1152,) lands at Hurst and Wyxham in Yorkshire, afterwards formed into the preceptory of Temple Hurst. William Asheby granted them the estate whereon the house and church of Temple Bruere were afterwards erected; and the order continued rapidly to increase in power and wealth in England and in all parts of Europe, through the charitable donations of pious Christians.[10]
- 69. After the miserable failure of the second crusade, brother Everard des Barres, the Master of the Temple, returned to Paris, with his friend and patron Louis, the French king; and the Templars, deprived of their chief, were now left, alone and unaided, to withstand the victorious career of the fanatical Mussulmen. Their miserable situation is pourtrayed in a melancholy letter from the treasurer of the order, written to the Master, Everard des Barres, during his sojourn at the court of the king of France, informing him of the slaughter of the prince of Antioch and all his nobility. “We conjure you,” says he, “to bring with you from beyond sea all our knights and serving brothers capable of bearing arms. Perchance, alas! with all your diligence, you may not find one of us alive. Use, therefore, all imaginable celerity; pray forget not the necessities of our house: they are such that no tongue can express them. It is also of the last importance to announce to the Pope, to the king of France, and to all the princes and prelates of Europe, the approaching desolation of the Holy Land, to the intent that they succour us in person, or send us subsidies.” The Master of the Temple, however, instead of proceeding to Palestine, abdicated his authority, and entered into the monastery of Clairvaux, where he devoted the remainder of his days to the most rigorous penance and mortification. He was succeeded (A. D. 1151) by Bernard de Tremelay, a nobleman of an illustrious family in Burgundy, in France, and a valiant and experienced soldier.[11] Shortly after his accession to power, the infidels crossed the Jordan, and advanced within sight of Jerusalem. Their banners waved on the summit of the Mount of Olives, and the warlike sound of their kettle- drums and trumpets was heard within the sacred precincts of the holy city. They encamped on the mount over against the Temple; and had the satisfaction of regarding from a distance the Beit Allah, or Temple of the Lord, their holy house of prayer; but in a night attack they were defeated with terrible slaughter, and were pursued all the way to the Jordan, five thousand of their number being left dead on the plain.
- 70. On the 20th of April, A. D. 1153, the Templars lost their great patron Saint Bernard, who died in the sixty-third year of his age. On his deathbed he wrote three letters in behalf of the order. The first was addressed to the patriarch of Antioch, exhorting him to protect and encourage the Templars, a thing which the holy abbot assures him will prove most acceptable to God and man. The second was written to Melesinda, queen of Jerusalem, praising her majesty for the favour shown by her to the brethren of the order; and the third, addressed to Brother André de Montbard, a Knight Templar, conveys the affectionate salutations of St. Bernard to the Master and brethren, to whose prayers he recommends himself. The same year the Master of the Temple perished at the head of his knights whilst attempting to carry the important city of Ascalon by storm. Passing through a breach made in the walls, he penetrated into the centre of the town, and was there surrounded and overpowered. The dead bodies of the Master and his ill-fated knights were exposed in triumph from the walls; and, according to the testimony of an eye-witness, not a single Templar escaped. De Tremelay was succeeded (A. D. 1154) by Brother Bertrand de Blanquefort, a knight of a noble family of Guienne, called by William of Tyre a pious and God-fearing man. On Tuesday, June 19, A. D. 1156, the Templars were drawn into an ambuscade whilst marching with Baldwin, king of Jerusalem, near Tiberias, three hundred of the brethren were slain on the field of battle, and eighty-seven fell into the hands of the enemy, among whom was Bertrand de Blanquefort himself, and Brother Odo, marshal of the kingdom. Shortly afterwards, a small band of them captured a large detachment of Saracens; and in a night attack on the camp of Noureddin, they compelled that famous chieftain to fly, without arms and half-naked, from the field of battle. In this last affair the name of Robert Mansel, an Englishman, and Gilbert de Lacy, preceptor of the Temple of Tripoli, are honourably mentioned.[12]
- 71. The fiery zeal and warlike enthusiasm of the Templars were equalled, if not surpassed, by the stern fanaticism and religious ardour of the followers of Mahomet. “Noureddin fought,” says his oriental biographer, “like the meanest of his soldiers, saying, ‘Alas! it is now a long time that I have been seeking martyrdom without being able to obtain it.’ The Imaum Koteb-ed-din, hearing him on one occasion utter these words, exclaimed, ‘In the name of God do not put your life in danger, do not thus expose Islam and the Moslems. Thou art their stay and support, and if (but God preserve us therefrom) thou shouldest be slain, we are all undone.’ ‘Ah! Koteb-ed-deen,’ said he, ‘what hast thou said, who can save Islam and our country, but that great God who has no equal?’ ‘What,’ said he, on another occasion, ‘do we not look to the security of our houses against robbers and plunderers, and shall we not defend RELIGION?’”[13] Like the Templars, Noureddin fought constantly with spiritual and with carnal weapons. He resisted the world and its temptations, by fasting and prayer, and by the daily exercise of the moral and religious duties and virtues inculcated in the Koran. He fought with the sword against the foes of Islam, and employed his whole energies, to the last hour of his life in the enthusiastic and fanatic struggle for the recovery of Jerusalem.[14] In his camp, all profane and frivolous conversation was severely prohibited; the exercises of religion were assiduously practised, and the intervals of action were employed in prayer, meditation, and the study of the Koran. “The sword,” says the prophet Mahomet, in that remarkable book, “is the key of heaven and of hell; a drop of blood shed in the cause of God, a night spent in arms, is of more avail than two months of fasting and of prayer. Whosoever falls in battle, his sins are forgiven him. At the day of judgment his wounds will be resplendent as vermillion, and odoriferous as musk, and the loss of limbs shall be supplied by the wings of angels and cherubims.” Among the many instances of the fanatical ardour of the Moslem warriors, are the following, extracted from the history of Abu Abdollah Alwakidi, Cadi of Bagdad. “Methinks,” said a valiant Saracen
- 72. youth, in the heat of battle—“methinks I see the black-eyed girls looking upon me, one of whom, should she appear in this world, all mankind would die for love of her; and I see in the hand of one of them a handkerchief of green silk, and a cap made of precious stones, and she beckons me, and calls out, Come hither quickly, for I love thee.” With these words, charging the Christian host, he made havoc wherever he went, until at last he was struck down by a javelin. “It is not,” said another dying Arabian warrior, when he embraced for the last time his sister and mother—“it is not the fading pleasure of this world that has prompted me to devote my life in the cause of RELIGION, I seek the favour of God and his APOSTLE, and I have heard from one of the companions of the prophet, that the spirits of the martyrs will be lodged in the crops of green birds who taste the fruits and drink of the waters of paradise. Farewell: we shall meet again among the groves and fountains which God has prepared for his elect.”[15] The Master of the Temple, Brother Bertrand de Blanquefort, was liberated from captivity at the instance of Manuel Comnenus, Emperor of Constantinople. After his release, he wrote several letters to Louis VII., king of France, describing the condition and prospects of the Holy Land: the increasing power and boldness of the infidels; and the ruin and desolation caused by a dreadful earthquake, which had overthrown numerous castles, prostrated the walls and defences of several towns, and swallowed up the dwellings of the inhabitants. “The persecutors of the church,” says he, “hasten to avail themselves of our misfortunes; they gather themselves together from the ends of the earth, and come forth as one man against the sanctuary of God.” It was during his mastership, that Geoffrey, the Knight Templar, and Hugh of Cæsarea, were sent on an embassy into Egypt, and had their famous interview with the Caliph. They were introduced into the palace of the Fatimites through a series of gloomy passages and glittering porticos, amid the warbling of birds and the murmur of fountains; the scene was enriched by a display of costly furniture
- 73. and rare animals; and the long order of unfolding doors was guarded by black soldiers and domestic eunuchs. The sanctuary of the presence chamber was veiled with a curtain, and the vizier who conducted the ambassadors laid aside his scimitar, and prostrated himself three times on the ground; the veil was then removed, and they saw the Commander of the Faithful.[16] Brother Bertrand de Blanquefort, in his letters to the king of France, gives an account of the military operations undertaken by the order of the Temple in Egypt, and of the capture of the populous and important city of Belbeis, the ancient Pelusium.[17] During the absence of the Master with the greater part of the fraternity on that expedition, the sultan Noureddin invaded Palestine; he defeated with terrible slaughter the serving brethren and Turcopoles, or light horse of the order, who remained to defend the country, and sixty of the knights who commanded them were left dead on the plain. Amalric, king of Jerusalem, the successor of Baldwin the Third, in a letter “to his dear friend and father,” Louis the Seventh, king of France, beseeches the good offices of that monarch in behalf of all the devout Christians of the Holy Land; “but above all,” says he, “we earnestly entreat your Majesty constantly to extend to the utmost your favour and regard to the Brothers of the Temple, who continually render up their lives for God and the faith, and through whom we do the little that we are able to effect, for in them indeed, after God, is placed the entire reliance of all those in the eastern regions who tread in the right path.”[18] The Master, Brother Bertrand de Blanquefort, was succeeded, (A. D. 1167,) by Philip of Naplous, the first Master of the Temple who had been born in Palestine. He had been Lord of the fortresses of Krak and Montreal in Arabia Petræa, and took the vows and the habit of the order of the Temple after the death of his wife.[19] We must now pause to take a glance at the rise of another great religio-military institution which, from henceforth, takes a leading part in the defence of the Latin kingdom. In the eleventh century,
- 74. when pilgrimages to Jerusalem had greatly increased, some Italian merchants of Amalfi, who carried on a lucrative trade with Palestine, purchased of the Caliph Monstasserbillah, a piece of ground in the Christian quarter of the Holy City, near the church of the Resurrection, whereon two hospitals were constructed, the one being appropriated for the reception of male pilgrims, and the other for females. Several pious and charitable Christians, chiefly from Europe, devoted themselves in these hospitals to constant attendance upon the sick and destitute. Two chapels were erected, the one annexed to the female establishment being dedicated to St. Mary Magdalene, and the other to St. John the Eleemosynary, a canonized patriarch of Alexandria, remarkable for his exceeding charity. The pious and kind-hearted people who here attended upon the sick pilgrims, clothed the naked and fed the hungry, were called “The Hospitallers of St. John.” On the conquest of Jerusalem by the Crusaders, these charitable persons were naturally regarded with the greatest esteem and reverence by their fellow-christians from the west; many of the soldiers of the cross, smitten with their piety and zeal, desired to participate in their good offices, and the Hospitallers, animated by the religious enthusiasm of the day, determined to renounce the world, and devote the remainder of their lives to pious duties and constant attendance upon the sick. They took the customary monastic vows of obedience, chastity, and poverty, and assumed as their distinguishing habit a black mantle with a white cross on the breast. Various lands and possessions were granted them by the lords and princes of the Crusade, both in Palestine and in Europe, and the order of the hospital of St. John speedily became a great and powerful institution. Gerard, a native of Provence, was at this period at the head of the society, with the title of “Guardian of the Poor.” He was succeeded (A. D. 1118) by Raymond Dupuy, a knight of Dauphiné, who drew up a series of rules for the direction and government of his brethren. In these rules no traces are discoverable of the military spirit which afterwards animated the order of the Hospital of St. John. The first authentic notice of an intention on the part of the Hospitallers to
- 75. occupy themselves with military matters, occurs in the bull of Pope Innocent the Second, dated A. D. 1130. This bull is addressed to the archbishops, bishops, and clergy of the church universal, and informs them that the Hospitallers then retained, at their own expense, a body of horsemen and foot soldiers, to defend the pilgrims in going to and returning from the holy places; the pope observes that the funds of the hospital were insufficient to enable them effectually to fulfil the pious and holy task, and he exhorts the archbishops, bishops, and clergy, to minister to the necessities of the order out of their abundant property. The Hospitallers consequently at this period had resolved to add the task of protecting to that of tending and relieving pilgrims. After the accession (A. D. 1168) of Gilbert d’Assalit to the guardianship of the Hospital—a man described by De Vertot as “bold and enterprising, and of an extravagant genius”—a military spirit was infused into the Hospitallers, which speedily predominated over their pious and charitable zeal in attending upon the poor and the sick. Gilbert d’Assalit was the friend and confidant of Amalric, king of Jerusalem, and planned with that monarch a wicked invasion of Egypt in defiance of treaties. The Master of the Temple being consulted concerning the expedition, flatly refused to have anything to do with it, or to allow a single brother of the order of the Temple to accompany the king in arms: “For it appeared a hard matter to the Templars,” says William of Tyre, “to wage war without cause, in defiance of treaties, and against all honour and conscience, upon a friendly nation, preserving faith with us, and relying on our own faith.” Gilbert d’Assalit consequently determined to obtain for the king from his own brethren that aid which the Templars denied; and to tempt the Hospitallers to arm themselves generally as a great military society, in imitation of the Templars, and join the expedition to Egypt, Gilbert d’Assalit was authorised to promise them in the name of the king, the possession of the wealthy and important city of Belbeis, the ancient Pelusium, in perpetual sovereignty.
- 76. According to De Vertot, the senior Hospitallers were greatly averse to the military projects of their chief: “They urged,” says he, “that they were a religious order, and that the church had not put arms into their hands to make conquests;” but the younger and more ardent of the brethren, burning to exchange the monotonous life of the cloister for the enterprise and activity of the camp, received the proposals of their superior with enthusiasm, and a majority of the chapter decided in favour of the plans and projects of their Guardian. They authorised him to borrow money of the Florentine and Genoese merchants, to take hired soldiers into the pay of the order, and to organize the Hospitallers as a great military society. It was in the first year of the government of Philip of Naplous (A. D. 1168) that the king of Jerusalem and the Hospitallers marched forth upon their memorable and unfortunate expedition. The Egyptians were taken completely by surprise; the city of Belbeis was carried by assault, and the defenceless inhabitants were barbarously massacred. The cruelty and the injustice of the Christians, however, speedily met with condign punishment. The king of Jerusalem was driven back into Palestine; Belbeis was abandoned with precipitation; and the Hospitallers fled before the infidels in sorrow and disappointment to Jerusalem. There they vented their indignation and chagrin upon the unfortunate Gilbert d’Assalit, their superior, who had got the order into debt to the extent of 100,000 pieces of gold; they compelled him to resign his authority, and the unfortunate guardian of the hospital fled from Palestine to England, and was drowned in the Channel. From this period, however, the character of the order of the Hospital of St. John was entirely changed: the Hospitallers appear henceforth as a great military body; their superior styles himself Master, and leads in person the brethren into the field of battle. Attendance upon the poor and the sick still continued, indeed, one of the duties of the fraternity, but it must have been feebly exercised amid the clash of arms and the excitement of war.[20]
- 77. The Grand Master of the Temple, Philip of Naplous, resigned his authority after a short government of three years, and was succeeded (A. D. 1170) by Brother Odo de St. Amand, a proud and fiery warrior, of undaunted courage and resolution; having, according to William, Archbishop of Tyre, the fear neither of God nor of man before his eyes.[21] It was during his Grand Mastership (A. D. 1172) that the Knight Templar Walter du Mesnil slew an envoy or minister of the assassins. These were an odious religious sect, settled in the fastnesses of the mountains above Tripoli, and supposed to be descended from the Ismaelians of Persia. They devoted their souls and bodies in blind obedience to a chief who is called by the writers of the Crusades “the old man of the mountain,” and were employed by him in the most extensive system of murder and assassination known in the history of the world. Both Christian and Moslem writers enumerate with horror the many illustrious victims that fell beneath their daggers. They assumed all shapes and disguises for the furtherance of their deadly designs, and carried, in general, no arms except a small poniard concealed in the folds of their dress, called in the Persian tongue hassissin, whence these wretches were called assassins, their chief the prince of the assassins; and the word itself, in all its odious import, has passed into most European languages.[22] Raimond, son of the count of Tripoli, had been slain by these fanatics whilst kneeling at the foot of the altar in the church of the Blessed Virgin at Carchusa or Tortosa; the Templars flew to arms to avenge his death; they penetrated into the fastnesses and strongholds of “the mountain chief,” and at last compelled him to purchase peace by the payment of an annual tribute of two thousand crowns into the treasury of the order. In the ninth year of Amalric’s reign, Sinan Ben Suleiman, imaun of the assassins, sent a trusty counsellor to Jerusalem, offering, in the name of himself and his people, to embrace the christian religion, provided the Templars would release them from the tribute money. The proposition was favourably received; the envoy was honourably entertained for some
- 78. Welcome to our website – the perfect destination for book lovers and knowledge seekers. We believe that every book holds a new world, offering opportunities for learning, discovery, and personal growth. That’s why we are dedicated to bringing you a diverse collection of books, ranging from classic literature and specialized publications to self-development guides and children's books. More than just a book-buying platform, we strive to be a bridge connecting you with timeless cultural and intellectual values. With an elegant, user-friendly interface and a smart search system, you can quickly find the books that best suit your interests. Additionally, our special promotions and home delivery services help you save time and fully enjoy the joy of reading. Join us on a journey of knowledge exploration, passion nurturing, and personal growth every day! ebookbell.com