![published by the press syndicate of the university of cambridge
The Pitt Building, Trumpington Street, Cambridge, United Kingdom
cambridge university press
The Edinburgh Building, Cambridge CB2 2RU, UK
40 West 20th Street, New York, NY 10011–4211, USA
477 Williamstown Road, Port Melbourne, VIC 3207, Australia
Ruiz de Alarcón 13, 28014 Madrid, Spain
Dock House, The Waterfront, Cape Town 8001, South Africa
http://www.cambridge.org
c
Cambridge University Press 2004
This book is in copyright. Subject to statutory exception
and to the provisions of relevant collective licensing agreements,
no reproduction of any part may take place without
the written permission of Cambridge University Press.
First published 2004
Printed in the United Kingdom at the University Press, Cambridge
Typeface Computer Modern 10/13pt System L
A
TEX 2ε [author]
A catalogue record for this book is available from the British Library
Library of Congress Cataloguing in Publication data available
ISBN 0 521 54253 7 paperback](https://image.slidesharecdn.com/1182935-250519121249-87b2d8c4/85/Foundations-Of-Computational-Mathematics-Minneapolis-2002-Felipe-Cucker-9-320.jpg)
![2 R. DeVore
there are many other stories of the type I put forward that are waiting
to be told.
The first of the three examples that I will discuss is that of image
compression. This subject has grown rapidly over the last decade with
an important infusion of ideas from mathematics especially the theories
of wavelets and nonlinear approximation. The main topic to be addressed
here is how we can decide which algorithms for compression are optimal.
A somewhat related topic will concern Analog to Digital (A/D) con-
version of signals. This is an area that is important in consumer electron-
ics. The story here centers around trying to understand why engineers
do A/D conversion in the way they do, which by the way is very coun-
terintuitive to what a first mathematical analysis would suggest.
Finally, I discuss adaptive methods for solving PDEs. This is an ex-
tremely important area of numerical computation in our quest to solve
large problems to higher and higher resolution. The question to be an-
swered is how can we know when an adaptive method is optimal in its
performance.
1.2 Image compression
Digital signal processing has revolutionized the storage and transmission
of audio and video signals as well as still images, in consumer electronics
and in more scientific settings (such as medical imaging). The main
advantage of digital signal processing is its robustness: although all the
operations have to be implemented with, of necessity, not quite ideal
hardware, the a priori knowledge that all correct outcomes must lie in a
very restricted set of well separated numbers makes it possible to recover
them by rounding off appropriately.
Every day, millions of digitized images are created, stored, and trans-
mitted over the Internet or using other mediums. A grey scale image
is an array (matrix) of pixel values. It is already important to have a
mathematical model for what these pixel values represent. We shall view
the pixel array as arising in the following fashion. We have a light inten-
sity function f defined on a continuum Ω. For simplicity we assume that
Ω := [0, 1]2
and that f takes values in [0, 1) (the latter can be achieved
by simple renormalization). Digitization corresponds to two operations:
averaging and quantization. We take a tiling of Ω into squares Q and
associate to each square Q the average intensity
fQ :=
1
|Q|
I
f(x) dx,](https://image.slidesharecdn.com/1182935-250519121249-87b2d8c4/85/Foundations-Of-Computational-Mathematics-Minneapolis-2002-Felipe-Cucker-15-320.jpg)
![1. Some Fundamental Issues 3
where |Q| denotes the Lebesgue measure of Q. The pixel values pQ are
derived from the numbers fQ ∈ [0, 1) by quantization. We write fQ in
its binary expansion
fQ =
∞
j=1
bj(fQ)2−j
and define the pixel value pQ :=
m
j=1 bj(fQ)2−j
. Typical choices of m
are m = 8 (one byte per pixel) or m = 16. The array I := I(f) := (pQ)
of pixel values is a digitization of f. The accuracy at which I resolves f
depends on the fineness of the tiling and the accuracy of the quantization
(i.e. size of m). We do not really know f. We only see it through the
digitized image I(f). In practice, the pixel values pQ are corrupted by
noise but we shall ignore this in our discussion since we are aiming in a
different direction.
We see that a digitized image in its raw form is described by mN bits
where N is the number of squares in the tiling. Lossy compression seeks
to significantly reduce this number of bits used to represent f at the
expense of some loss in the fidelity of resolution. Hopefully, this loss of
fidelity is not perceptible. There are two parts to a lossy compression
scheme. The encoder assigns to each pixel array I a bitstream B(I). A
decoder gives the recipe for changing any given bitstream B back into a
pixel array. After encoding and then decoding the resulting pixel values
p̄Q will generally not be the same as the original pQ. Some fidelity is lost.
One can imagine, given the practical importance of the compression
problem, that there are a ton of encoding/decoding schemes. How can
one decide from this myriad of choices which is the best? Engineers have
introduced a number called the PSNR (Peak Signal to Noise Ratio)
which measures the performance of a given encoding/decoding on a
given digitized image I. It is not necessary to give its precise definition
here but simply mention that it measures the least squares distortion
((#I)−1
Q[pQ − p̄Q]2
)1/2
as a function of the number of bits. Here
#(I) is the number of pixels. A new encoding scheme is tested by its
performance (PSNR) on a few test images – the Lena image being the
most widely used.
Now, there is a fundamental question here. Should the quality of a
compression algorithm be determined by its PSNR performance on a
few test images? Given a collection of 2k
images, we can encode them
all with k bits per image by simply enumerating them in binary. So on
a mathematical level this test of performance is quite unsatisfactory.
What has any of this to do with “foundations of computational
mathematics”. Well, we cannot have a decidable competition among](https://image.slidesharecdn.com/1182935-250519121249-87b2d8c4/85/Foundations-Of-Computational-Mathematics-Minneapolis-2002-Felipe-Cucker-16-320.jpg)
![1. Some Fundamental Issues 5
There are two main models for images: the stochastic and the deter-
ministic. Stochastic models are deeply embedded in the engineering and
information theory communities influenced in a large part by Shannon’s
theory for optimal encoding. Deterministic models take the view we have
presented of an image as a function defined on a continuum. We have
begun by assuming only that an image is a bounded function. This is
too broad of a class of functions to serve as the description of the images
we wish to compress. Images have more structure.
One deterministic view of an image function f is that it is a sum
of fundamental components corresponding to edges, texture, and noise.
For example the famous model of Mumford and Shah [17] views the
image as a sum f = u + v of a component u of Bounded Variation
(BV) and a component v in L2. The component u is not an arbitrary
BV function but rather has gradient given by a measure supported on a
one dimensional set (corresponding to the edges in the image) and an L1
part (corresponding to smooth regions in the image). The L2 component
v captures deviations from this model.
There are many variants of the Mumford–Shah model. These are beau-
tifully described in the lecture notes of Meyer [14] – a must read. We
wish to pick up on only one point of Meyer’s exposition. Even when one
settles on the functional nature of the two components u and v in the
image, there are infinitely many ways to write f = u + v depending on
how much energy one wishes to put in each of these components. This
is completely analogous to K-functional decompositions used in proving
theorems on interpolation of operators. One needs to look at this totality
of all such decompositions to truly understand f. For example, consider
the case where we simply look for decompositions of f = u + v where
u ∈ BV and v ∈ L2. We can give a quantitative description of these
decompositions through the K-functional
K(f, t) := K(f, t; L2, BV) := inf
f=u+v
vL2
+ t|u|BV, t 0, (1.1)
where the | · |BV is the BV seminorm. For any fixed t 0, the optimal
decomposition in (1.1) tries to balance the two terms. Thus for t small
it puts more energy into the BV component and less into the L2 com-
ponent. The rate of decrease in K(f, t) as t → 0 tells how nice f is with
respect to this model.
The role of the K-functional is to distinguish between images.
Certainly some images are more complex than others and more apt to
be more difficult to compress. The rate at which a K-functional tends to](https://image.slidesharecdn.com/1182935-250519121249-87b2d8c4/85/Foundations-Of-Computational-Mathematics-Minneapolis-2002-Felipe-Cucker-18-320.jpg)
![1. Some Fundamental Issues 9
these Besov spaces by Bs
q (Lp(Ω)) where p is the Lp space in which we are
measuring smoothness. The parameter s 0 gives the order of smooth-
ness much like the number of derivatives. The parameter 0 q ≤ ∞ is
a fine tuning parameter which makes subtle distinctions between these
spaces. We do not make a precise description of these spaces at this
juncture but we shall give a description of these spaces in a moment
using wavelet bases.
The reader is probably by now quite familiar with wavelet bases. We
shall limit ourselves to a few remarks which will serve to describe our
notation. When working on the domain R, a wavelet basis is given by
the shifted dilates ψλ := ψ(2j
· −k), λ = (j, k), of one fixed function
ψ. When moving to Rd
, one needs the shifted dilates of a collection ψe
of 2d
− 1 functions; the parameter e is usually indexed on the set E of
nonzero vertices of the unit cube [0, 1]d
. Thus the wavelets are indexed
by three parameters λ = (j, k, e) indicated frequency (j), location (k)
and type (e). When working on a finite domain, two adjustments need to
be made. The first is that the range of j is from j0 ≤ j ∞. The coarsest
level j = j0 corresponds to scaling functions; all other j correspond to
the actual wavelets. For notational convenience, we shall take j0 = 0 in
what follows. The second adjustment is that near the boundary some
massaging has to be made in defining ψλ.
Thus, a wavelet basis on a finite domain Ω in Rd
is a collection Ψ =
{ψλ : λ ∈ J } of functions ψλ. The indices λ encode scale, spatial location
and the type of the wavelet ψλ. We will denote by |λ| the scale associated
with ψλ. We shall only consider compactly supported wavelets, i.e., the
supports of the wavelets scale as follows
Sλ := supp ψλ, c02−|λ|
≤ diam Sλ ≤ C02−|λ|
, (1.12)
with c0 and C0 0 absolute constants. The index set J has the follow-
ing structure J = Jφ ∪ Jψ where Jφ is finite and indexes the scaling
functions on the fixed coarsest level 0. Jψ indexes the “true wavelets”
ψλ with |λ| 0. From compactness of the supports we know that at
each level, the set Jj := {λ ∈ J : |λ| = j} is finite. In fact, one has
#Jj ∼ 2jd
with constants depending on the underlying domain.
There is a natural tree structure associated to wavelet bases. A node
in this tree corresponds to all λ = (j, k, e), e ∈ E, with j, k fixed. In the
case the domain is Rd
, each such node has 2d
children corresponding
to the indices (j + 1, 2(k + e)) where e ∈ {0, 1}d
. In other words, the
children all occur on the next dyadic level. In the case of Haar functions,
the supports of the wavelets corresponding to children are contained](https://image.slidesharecdn.com/1182935-250519121249-87b2d8c4/85/Foundations-Of-Computational-Mathematics-Minneapolis-2002-Felipe-Cucker-22-320.jpg)
![1. Some Fundamental Issues 11
1.2.5 Near optimal encoding in Lp, 1 ≤ p ≤ ∞
Let us fix the metric of interest to us to be one of the Lp norms with
1 ≤ p ≤ ∞. If we derive metrics that seem better measures of distortion
for images then we would try to repeat the exercise of this section for
these metrics.
We seek an encoder E with the following properties. First the encoder
should be applicable to any function in Lp; such an encoder is said to
be universal. We would like the encoder to give an infinite bitstream;
giving more bits from the infinite bitsream would give a finer resolution
of f. Such an encoder is called progressive. Finally, we would like the
encoder to be near optimal for the compact sets that are unit balls of
the Besov spaces in the following sense. If Enf denotes the first n bits
of Ef, then we would like that for each such compact set K we have
DnEnf − fLp(Ω) ≤ CKn(K), f ∈ K, n = 1, 2, . . . , (1.16)
with CK a constant depending only on K. This means that for these
compact sets the encoder performs (save for the constant CK) as well as
any encoder.
It is quite amazing that there is a simple construction of encoders
with these three desirable properties using wavelet decompositions. To
describe these encoders, it is useful to keep in mind the case when the
metric is L2 and the wavelet basis is an orthonormal system. The general
case follows the same principles but the proofs are not as transparent.
Suppose then that X = L2(Ω) and f ∈ X with fX ≤ 1 has an
orthogonal wavelet expansion f =
λ∈J fλψλ. To start the encoding,
we would like to choose a few terms from this wavelet expansion which
best represent f. The best choice we can make is to choose the largest
terms since the remainder is always the sum of squares of the remaining
coefficients.
Suppose η 0 and Λη := Λη(f) := {λ : |fλ| ≥ η} is the set of coeffi-
cients obtained by thresholding the wavelet coefficients at the threshold
η. Note that Λη = ∅ when η 1. We would like to encode the infor-
mation about the set Λη and the coefficients fλ, λ ∈ Λη. There are two
issues to overcome. The first is that it is necessary to encode the posi-
tions of the indices in Λη. At first glance, these positions could occur
anywhere which would cost possibly an arbitrarily large bit budget to
encode them. But it turns out that for the sets K on which we want
optimality of the encoding, namely K a unit ball of a Besov space, these
positions align themselves at low frequencies. In fact, it can be proved [5]
that whenever f ∈ K one can find a tree T which contains Λη and is of](https://image.slidesharecdn.com/1182935-250519121249-87b2d8c4/85/Foundations-Of-Computational-Mathematics-Minneapolis-2002-Felipe-Cucker-24-320.jpg)
![12 R. DeVore
comparable size to Λη. This motivates us to define for each f ∈ L2(Ω),
Tη = Tη(f) as the smallest tree which contains Λη(f). What we gain in
going to a tree structure (which could be avoided at the expense of more
complications and less elegance) is that it is easy to encode the positions
of a tree using at most 2#(Tη) bits. Indeed, one simply encodes the tree
from its roots by assigning a bit 1 if the child of a current member λ of
Tη is in Tη and zero otherwise; see [6, 5] for details.
The second issue to overcome is how to encode the coefficients fλ for
λ ∈ Tη. To encode just one real number we need an infinite number
of bits. The way around this is the idea of quantization. In the present
context, the (scalar) quantization is easy. Any real number y with |y| ≤ 1
has a binary representation
y = (−1)s(y)
∞
i=0
bi(y)2−i
(1.17)
with the bi(y) ∈ {0, 1} and the sign bit s(y) defined as 0 if y 0 and 1
otherwise. Receiving the bits s(f), b0(f), . . . , bm(f), we can approximate
y by the partial sum ȳ = (−1)s(y)
m
i=0 bi(y)2−i
with accuracy
|y − ȳ| ≤ 2−m
. (1.18)
We apply this quantization to the coefficients fλ, λ ∈ Tη. How should
we choose m? Well in keeping with the strategy for thresholding, we
would only want the residual y − ȳ to be under the threshold η. Thus,
if η = 2−k
, we would choose the quantization so that m = k.
There is only one other thing to note before we define our encoding.
If η is a current threshold level and η
η is a new threshold then the
tree Tη is a growing of the tree Tη. Keeping this in mind, let us take
thresholds ηk = 2−k
, k = 0, 1, . . . and obtain the corresponding trees
Tk
:= Tηk
, k = 0, 1, . . . . To each f ∈ L2(Ω), we assign the following
bitstream:
P0(f), S0(f), B0(f), P1(f), S1(f), B1(f), B1,0(f), . . . (1.19)
Here, P0(f) denotes the bits needed to encode the positions in the tree
T0
(f). The bits S0(f) are the sign bits of the coefficients corresponding
to indices in T0
(f). The set B0(f) gives the first bits b0(fλ) of the
coefficients fλ corresponding to the λ ∈ T0
. When we advance k to the
value 1, we assign in P1 the bits needed to encode the new positions, i.e.
the positions in T1
T0
. Then S1(f) are the sign bits for the coefficients
corresponding to these new positions. The bits B1(f) are the b1 bits
(which correspond to 2−1
in the binary expansion) of the coefficients](https://image.slidesharecdn.com/1182935-250519121249-87b2d8c4/85/Foundations-Of-Computational-Mathematics-Minneapolis-2002-Felipe-Cucker-25-320.jpg)
![1. Some Fundamental Issues 13
corresponding to these new positions. Note that each new coefficient has
absolute value ≤ 1/2 so the bit b0 = 0 for these coefficients. The set
B1,0(f) gives the second bit (i.e. the b1 bit) in the binary expansion of
the fλ, λ ∈ T0
. The reason we add these bits is so that each coefficient,
whether it is from T0
or T1
is resolved to the same accuracy (i.e. accuracy
1/2 at this stage). The process continues as we increase k. We send
position bits to identify the new positions, a sign bit and a lead bit for
each coefficient corresponding to a new position, and then one bit from
the binary expansion of all the old positions.
We denote by E the mapping which takes a given f ∈ L2(Ω) into the
bitstream (1.19). For each k ≥ 0, we let Ek be the encoder obtained
from E by truncating at stage k. Thus Ek(f) is the finite bitstream
P0(f), S0(f), B0(f), . . . , Pk(f), Sk(f), Bk(f), Bk,0(f), . . . , Bk,k−1(f)
(1.20)
Let us say a few words about the decoding of such a bitstream. When a
receiver obtains a bitstream of the form (1.20), he knows the first bits will
give the positions of a tree of wavelet indices. From the form of the tree
encoding, he will know when the bits of P0 have ended; see [5]. At this
stage he knows the number of elements in the tree T0
. He therefore knows
the next #(T0
) bits will give the signs of the corresponding coefficients
and the following #(T0
) bits will be the binary bits for position 0 in the
binary expansion of each of these coefficients. Then, the process repeats
itself at level 1.
The encoder E has all of the properties we want. It is universal, i.e.
defined for each f ∈ L2(Ω). It is progressive: as we receive more bits
we get a finer resolution of f. Finally it is near optimal in the following
sense. If K is the unit ball of any of the Besov spaces which are compactly
embedded into L2(Ω), then for any k the encoder Ek is near optimal in
the sense of (1.16).
While we have discussed the encoder E in the context of measuring
distortion in L2, the same ideas apply when distortion is measured in
Lp for any 1 ≤ p ≤ ∞. The only alteration necessary is to work with
the wavelet decomposition with wavelets normalized in Lp(Ω) which of
course alters the coefficients as well.
1.3 Analog to Digital (A/D) conversion
As we have previously observed, the digital format is preferred for rep-
resenting signals because of its robustness. However, many signals are](https://image.slidesharecdn.com/1182935-250519121249-87b2d8c4/85/Foundations-Of-Computational-Mathematics-Minneapolis-2002-Felipe-Cucker-26-320.jpg)
![14 R. DeVore
not digital but rather analog in nature; audio signals, for instance, cor-
respond to functions f(t), modeling rapid pressure oscillations, which
depend on the “continuous” time t (i.e. t ranges over R or an interval
in R, and not over a discrete set), and the range of f typically also fills
an interval in R. For this reason, the first step in any digital processing
of such signals must consist in a conversion of the analog signal to the
digital world, usually abbreviated as A/D conversion. Note that at the
end of the chain, after the signal has been processed, stored, retrieved,
transmitted, . . . , all in digital form, it needs to be reconverted to an
analog signal that can be understood by a human hearing system; we
thus need a D/A conversion there.
There are many proposed algorithms for A/D conversion. As in our
discussion the last section, we would like to understand how we could
decide which of these algorithms is optimal for encoding/decoding. As in
that case, we have two initial issues: determine the metric to be used to
measure distortion and the class of signals that are to be encoded. The
metric issue is quite similar to that for images except now the human
visual system is replaced by the human auditory system. In fact such
considerations definitely play a role in the design of good algorithms
but as of yet we are aware of no mathematical metric which is used to
model the auditory system in the mathematical analysis of the encoding
problem. The two metrics usually utilized in distortion analysis are the
L2(R) and L∞(R) norms.
Concerning model classes for auditory signals, it is customary to model
audio signals by bandlimited functions, i.e. functions f ∈ L2
(R) for which
the Fourier transform
ˆ
f(ξ) =
1
√
2π
∞
−∞
f(t)e−iξt
dt
vanishes outside an interval |ξ| ≤ Ω. The bandlimited model is justified
by the observation that for the audio signals of interest to us, observed
over realistic intervals time intervals [−T, T], χ|ξ|Ω(χ|t|≤T f)∧
2 is neg-
ligible compared with χ|ξ|≤Ω(χ|t|≤T f)∧
2 for Ω 2π · 20, 000 Hz.
So in proceeding further, let us agree that we shall use either the L2
or L∞ metric and the class of functions we shall consider are those that
are bounded and bandlimited. We define S to be the collection of all
functions in L2 ∩ L∞ whose L2 norm is ≤ 1 and L∞ norm is ≤ a 1
with a 0 fixed and whose Fourier transform vanishes outside of [−π, π].
The choice of a and π are arbitrary; indeed given any f ∈ L2 ∩L∞ which
is bandlimited, we can dilate f and then multiply by a constant to arrive](https://image.slidesharecdn.com/1182935-250519121249-87b2d8c4/85/Foundations-Of-Computational-Mathematics-Minneapolis-2002-Felipe-Cucker-27-320.jpg)
![1. Some Fundamental Issues 15
at an element of S. Thus any encoders derived for S can easily be applied
to general f.
The class S has a lot of structure. There is a well-known sampling
theorem that says that any function f ∈ S is completely determined by
its values on Z. Indeed, we can recover f from the formula
f(t) =
n∈Z
f(n)
sin(t − n)
(t − n)
=
n∈Z
f(n)sinc(t − n) . (1.21)
This formula is usually referred to as the Shannon-Whitaker formula.
The sampling rate of 1 (called the Nyquist rate) arises because ˆ
f vanishes
outside of [−π, π]. The sinc functions appearing on the right side of
(1.21) form an orthonormal system in L2. Changing the support interval
from [−π, π] to [−Aπ, Aπ] would correspond to Nyquist sampling rate
of 1/A.
The proof of (1.21) is simple and instructive. We can write
ˆ
f = F · χ (1.22)
where F :=
k∈Z f(·+2kπ) is the periodization of ˆ
f and χ is the charac-
teristic function of [−π, π]. The Fourier coefficients of F are F̂(n) = f(n)
and so F =
n∈Z f(n)einω
. Substituting that into (1.22) and inverting
the Fourier transform we obtain (1.21) because the inverse Fourier trans-
form of χ is the sinc function.
With the formula (1.21) in hand, it seems that our quest for an optimal
encoder is a no brainer. We should simply quantize the Nyquist samples
f(n). Given a real number y ∈ (−1, 1), we can write as before
y = (−1)s(y)
∞
i=1
bi(y)2−i
(1.23)
where (−1)s(y)
, s(y) ∈ {0, 1}, is the sign of y and the bi are the binary
bits of |y|. If we pick a number m 0, the quantized values
ȳ = (−1)s(y)
m
i=1
bi(y)2−i
(1.24)
can be described by m bits and |y − ȳ| ≤ 2−m
. If we apply this to
the samples f(n), n ∈ Z, we have an encoding of f that uses m bits
per Nyquist sample. Encoders built on this simple idea are called Pulse
Code Modulation (PCM). However, they are not the encoders of choice
in A/D conversion. Our excursion into this topic in this section is to](https://image.slidesharecdn.com/1182935-250519121249-87b2d8c4/85/Foundations-Of-Computational-Mathematics-Minneapolis-2002-Felipe-Cucker-28-320.jpg)
![16 R. DeVore
understand why this is the case. Can we explain mathematically why
engineers do not prefer PCM and better yet to explain the advantages
of what they do prefer.
To begin the story, we have to dig a bit deeper into what we really
mean by an encoding of a signal. The formula (1.21) requires an infinite
number of samples to recover f and therefore apparently an infinite
number of bits. Of course, we cannot compute, store, or transmit an
infinite bitstream. But fortunately, we only want to recover f on a finite
time interval which we shall take to be [0, T]. Even then, the contribution
of samples far away from [0, T] is large. Indeed, if we incur a fixed error
δ in representing each sample, then the total error on [0, T] is possibly
infinite since the sinc functions decay so slowly:
n∈Z |sinc(t−n)| = ∞.
There is a way around this by sampling the function f at a slightly
higher rate. Let λ 1 and let gλ be a C∞
function such that ĝλ = 1 on
[−π, π] and ĝλ vanishes outside of [−λπ, λπ]. Returning to our derivation
of (1.21), using ĝλ in place of χ, we obtain the representation
f(t) =
1
λ
n∈Z
f
n
λ
gλ t −
n
λ
. (1.25)
Because g is smooth with fast decay, this series now converges absolutely
and uniformly; moreover if the f n
λ
is replaced by
fn = f n
λ
+ εn in
(1.25), with |εn| ε, then the difference between the approximation
f(x) and f(x) can be bounded uniformly:
|f(t) −
f(t)| ≤ ε
1
λ
n∈Z
g t −
n
λ
≤ εCg (1.26)
where Cg = λ−1
g
L1 + gL1 does not depend on T. Oversampling
thus buys the freedom of using reconstruction formulas, like (1.25), that
weigh the different samples in a much more localized way than (1.21)
(only the f n
λ
with
t − n
λ
“small” contribute significantly). In prac-
tice, it is customary to sample audio signals at a rate that is about 10 or
20% higher than the Nyquist rate; for high quality audio, a traditional
sampling rate is 44,000 Hz.
One can show that the above idea of sampling slightly higher than the
Nyquist rate and then quantizing the samples using binary expansion
peforms at the Kolmogorov entropy rate for the class S (at least when
T is large). This was first proved in [13] and later repeated in [10]. We
spare the details and refer the reader to either of these two papers. So it
seems now that the matter of encoding is closed by using such a modified
PCM encoding. But I have surprising news: engineers do not prefer this](https://image.slidesharecdn.com/1182935-250519121249-87b2d8c4/85/Foundations-Of-Computational-Mathematics-Minneapolis-2002-Felipe-Cucker-29-320.jpg)
![1. Some Fundamental Issues 17
method in practice. In fact they prefer another class of encoders known
as Sigma Delta Modulation which we shall describe below. The mystery
we still want to uncover is why they prefer these methods.
To explain the Sigma-Delta story, we return to the idea of oversam-
pling which is at the heart of these encoders. We have seen the benefits
of slight oversampling: sampling slightly higher than the Nyquist rate
and giving several bits for each sample performs at Kolmogorov entropy
rates. Sigma-Delta encoders go to the other extreme. They sample at a
rate λ 1 which is very large but then allot only one bit to each sample.
Thus, to each sample f(n
λ ) they assign a single bit qλ
n ∈ {−1, 1}. On the
surface this is very counter intuitive. If we think of that one bit as giving
an approximation to the sample f(n
λ ) then we cannot do very well. In
fact the best we can do is give the sign of the sample. But the Sigma-
Delta encoders do not do this. Rather, they make their bit assignment
to f(n
λ ) based on the past samples f(m
λ ), m n, and the bits that have
already been assigned to them.
Let us describe this in more detail by considering the simplest of
these encoders. We introduce an auxiliary sequence (un)n∈Z (sometimes
described as giving the “internal state” of the Sigma-Delta encoder)
iteratively defined by
un = un−1 + f
n
λ
− qλ
n
qλ
n = sign un−1 + f
n
λ
,
(1.27)
and with an “initial condition” u0 = 0. The qλ
n are the single bit we
assign to each sample. In circuit implementation, the range of n in (1.27)
is n ≥ 1. However, for theoretical reasons, we view (1.27) as defining
the un and qn for all n. At first glance, this means the un are defined
implicitly for n 0. However, it is possible to write un and qn directly
in terms of un+1 and fn+1 when n 0; see [9].
The role of the auxiliary sequence (un) is to track the difference be-
tween the running sums of the f(n
λ ) and those of the qλ
n. It is easy to
see that the choice for qλ
n used in (1.27) keeps the difference of these
running sums to be ≤ 1. For this, one proves simply by induction that
the |un| ≤ 1, for all n ∈ Z.
Of course, we need to describe how we decode the bit stream qλ
n. For
this we use (1.25) with f(n
λ ) replaced by qλ
n:
¯
f :=
1
λ
n∈Z
qλ
ngλ(n −
n
λ
). (1.28)](https://image.slidesharecdn.com/1182935-250519121249-87b2d8c4/85/Foundations-Of-Computational-Mathematics-Minneapolis-2002-Felipe-Cucker-30-320.jpg)
![18 R. DeVore
At this point, we have no information about the accuracy at which ¯
f
represents f. However, simple estimates are available using summation
by parts. For any t ∈ R, we have
f(t) −
1
λ
n
qλ
ngλ t −
n
λ
=
1
λ
n
f
n
λ
− qλ
n gλ t −
n
λ
=
1
λ
n
un gλ t −
n
λ
− gλ t −
n + 1
λ
≤
1
λ
n
gλ t −
n
λ
− gλ t −
n + 1
λ
≤
1
λ
n
t− n
λ
t− n+1
λ
|g
λ(y)|dy =
1
λ
g
λL1 ≤
C
λ
.
where we have used the fact that |un| ≤ 1, for all n ∈ Z.
There is good news and bad news in the last estimate. The good news
is that we see that ¯
f approximates f better and better as the sampling
rate λ increases. The bad news is that this decay O(1
λ ) is far inferior
to the exponential rate provided by PCM. Indeed, for an investment of
m bits per Nyquist sample PCM provides distortion O(2−m
) whereas
for an investment of λ bits per Nyquist sample, Sigma-Delta provides
distortion O(1/λ). So at this stage, we still have no clue why Sigma-Delta
Modulators are preferred in practice.
It is possible to improve the rate distortion for Sigma-Delta encoding
by using higher order methods. In [9], a family of such encoders were
constructed. The encoder of order k is proven to give rate distortion
O(λ−k
). If one allows k to depend on λ, one can derive error bounds
of the order O(2−(log λ)2
) – still far short of the exponential decay of
PCM. The pursuit of higher performing encoders has led to a series of
interesting questions concerning the optimal distortion possible using
single bit encoders. The best known bound (e−.07λ
) for such encoders
was given by Güntürk [12], Calderbank and Daubechies [4] show that
it is not possible to obtain the rate (2−λ
) of PCM. In any case, none
of these new methods are used in practical encoding and they do not
explain the penchant for the classical Sigma-Delta methods.
A couple of years ago, Ingrid Daubechies, Sinan Güntürk, and I were
guests of Vinay Vaishampayan at Shannon Labs of ATT for a one](https://image.slidesharecdn.com/1182935-250519121249-87b2d8c4/85/Foundations-Of-Computational-Mathematics-Minneapolis-2002-Felipe-Cucker-31-320.jpg)
![1. Some Fundamental Issues 19
month think tank directed at understanding the preferences for Sigma-
Delta Modulation. Shannon Labs is an oasis for Digital Signal Processing
and its circuit implementation. We were fortunate to have many lunch
with experts in Sigma-Delta methods asking them for their intuition
why these methods are preferred in practice. This would be followed
by an afternoon to put a mathematical justification behind their ideas.
Usually, these exercises ended in futility but what became eventually
clear is that the circuit implementation of Sigma-Delta Modulation is at
the heart of the matter.
The hardware implementation of encoders such as PCM or Sigma-
Delta Modulation requires building circuits for the various mathematical
operations and the application of quantizers Q such as the sign function
used in (1.27). Our mathematical analysis thus far has assumed that
these operations are made exactly. Of course, this is far from the case in
circuit implementation.
Let us suppose for example that the sign function used in (1.27) is
replaced at iteration n by a non-ideal quantizer
Qn(x) = sign(x) for |x| ≥ τ
|Qn(x)| ≤ 1 for |x| τ.
(1.29)
Thus, the quantizer Qn gives the right value of sign when |x| ≥ τ but may
not when |x| τ. Here τ would depend on the precision of the circuit
which of course is related to the dollar investment we want to make
in manufacturing such circuits. Note that we allow the quantization to
vary with each application but require the overall precision τ. When we
implement the Sigma-Delta recursion with this circuitry, we would not
compute the qλ
n of (1.27) but rather a new sequence q̄n given by
ūn = ūn−1 + f n
λ
− q̄n
q̄n = Qn ūn−1 + f n
λ
,
(1.30)
Thus, the error analysis given above is sort of irrelevant and needs to
be replaced by one involving the q̄n. Recall that our analysis above rested
on showing the state variables un are uniformly bounded. It turns out
that in the scenario the new state variables ūn are then still bounded,
uniformly, independently of the detailed behavior of Qn, as long as (1.29)
is satisfied. Namely, we have
Remark 1.1 Let f ∈ S, let ūn, q̄n be as defined in (1.30), and let Qn
satisfy (1.29) for all n. Then |ūn| ≤ 1 + τ for all n ≥ 0.
We refer the reader to [9] for the simple proof. Note that the remark
holds regardless of how large τ is; even τ 1 is allowed.](https://image.slidesharecdn.com/1182935-250519121249-87b2d8c4/85/Foundations-Of-Computational-Mathematics-Minneapolis-2002-Felipe-Cucker-32-320.jpg)
![20 R. DeVore
We now use the inaccurate bits q̄n to calculate ¯
f: The same summation
by parts argument that derived (1.29) can be applied to derive the new
error estimate:
|f(t) − ¯
f(t)| ≤
(1 + τ)g
λL1
λ
. (1.31)
Thus, except for the fact that the constants increase slightly, the
bounds on the accuracy of the encoder does not change. The precision
that can be attained is not limited by the circuit imperfection: by choos-
ing λ sufficiently large, the approximation error can be made arbitrarily
small.
The same is definitely not true for the binary expansion-type schemes
such as PCM. To see this, let us see how we quantize to obtain the
binary bits of a number y ∈ (−1, 1). To find the sign bit of y, we use the
quantizer Q as before. But to find the remaining bits, we would use
Q1(z) :=
0, z ≤ 1
1, z 1.
(1.32)
Once the sign bit b0 is found then, we define u1 := 2b0y = 2|y|. The
bit b1 is given by b1 := Q1(u1). Then the remaining bits are computed
recursively as follows: if ui and bi have been defined, we let
ui+1 := 2(ui − bi) (1.33)
and
bi+1 := Q1(ui+1). (1.34)
In circuit implementations, the quantization would not be exact. Sup-
pose for example, we use an imprecise quantizer (1.29) to find the sign
bit of y. Then, taking a y ∈ (−τ, τ), we may have the sign bit of y in-
correct. Therefore, ȳ and y do not even agree in sign so |ȳ − y| could be
as large as τ no matter how the remaining bits are computed. The mis-
take made by the imperfect quantizer cannot be recovered by computing
more bits, in contrast to the self-correcting property of the Sigma-Delta
scheme.
So there it is! This is a definite advantage in Sigma-Delta Modulation
when compared with PCM. In order to obtain good precision overall with
the binary quantizer, one must therefore impose very strict requirements
on τ, which would make such quantizers very expensive in practice (or
even impossible if τ is too small). On the other hand [10], Sigma-Delta
encoders are robust under such imperfections of the quantizer, allowing
for good precision even if cheap quantizers are used (corresponding to](https://image.slidesharecdn.com/1182935-250519121249-87b2d8c4/85/Foundations-Of-Computational-Mathematics-Minneapolis-2002-Felipe-Cucker-33-320.jpg)
![1. Some Fundamental Issues 21
less stringent restrictions on τ). It is our understanding that it is this
feature that makes Sigma-Delta schemes so successful in practice.
We have shown again where the understanding and formulation of fun-
damental questions in computation is vital to understanding numerical
methods. In this case of A/D conversion, it not only gives an understand-
ing of the advantages of the current state of the art encoders, it also leads
us to a myriad of questions at the heart of the matter [9, 10, 12]. We
shall pick up on just one of these.
We have seen that oversampling and one bit quantization allow error
correction in the encoding but the bit distortion rate in the encoding is
not very good. On the other hand, PCM has excellent distortion rate
(exponential) but no error correction. It is natural to ask whether we
can have the best of both worlds: exponential decay in distortion and
error correction. The key to answering this question lies in the world of
redundancy. We have seen the effect of the redundancy in the Sigma-
Delta Modulation which allowed for error correction. It turns out that
other types of redundancy can be utilized in PCM encoders. The essen-
tial idea is to replace the binary representation of a real number y by a
redundant representation.
Let 1 β 2 and γ := 1/β. Then each y ∈ [0, 1] has a representation
y =
∞
i=1
biγi
(1.35)
with
bi ∈ {0, 1}. (1.36)
In fact there are many such representations. The main observation that
we shall utilize below is that no matter what bits bi, i = 1, . . . , m, have
been assigned, then, as long as
y −
γm+1
1 − γ
≤
m
i=1
biγi
≤ y, (1.37)
there is a bit assignment (bk)km, which, when used with the previously
assigned bits, will exactly recover y.
We shall use this observation in an analogous fashion to the algorithm
for finding the binary bits of real numbers, with the added feature of
quantization error correction.
These encoders have a certain offset parameter µ whose purpose is
to make sure that even when there is an imprecise implementation
of the encoder, the bits assigned will satisfy (1.37); as shown below,](https://image.slidesharecdn.com/1182935-250519121249-87b2d8c4/85/Foundations-Of-Computational-Mathematics-Minneapolis-2002-Felipe-Cucker-34-320.jpg)
![22 R. DeVore
introducing µ corresponds to carrying out the decision to set a bit to 1
only when the input is well past its minimum threshold. We let Q1 be
the quantizer of (1.32).
The beta-encoder with offset µ. Let µ 0 and 1 β 2. For
y ∈ [0, 1], we define u1 := βy and b1 := Q1(u1 −µ). In general, if ui and
bi have been defined, we let
ui+1 := β(ui − bi), bi+1 := Q1(ui+1 − µ). (1.38)
It then follows that
y −
m
i=1
biγi
= y −
m
i=1
γi
(ui − γui+1)
= y − γu1 + γm+1
um+1 ≤ γm+1
ul∞ , (1.39)
showing that we have exponential precision in our reconstruction, pro-
vided the |ui| are uniformly bounded. It is easy to prove [10] that we
do indeed have such a uniform bound. Let’s analyze the error correcting
abilities of these encoders when the quantization is imprecise.
Suppose that in place of the quantizer Q1, we use at each iteration in
the beta-encoder the imprecise quantizer
Q̃1(z) :=
0, z ≤ 1 − τ
1, z 1 + τ
∈ {0, 1}, z ∈ (−τ, τ).
(1.40)
In place of the bits bi(y), we shall obtain inaccurate bits b̃i(y) which are
defined recursively by ũ1 := βy, b̃1 := Q̃1(ũ1 − µ) and more generally,
ũi+1 := β(ũi − b̃i), b̃i+1 := Q̃1(ũi+1 − µ). (1.41)
Theorem 1.1 Let δ 0 and y ∈ [0, 1). Suppose that in the beta-encoding
of y, the quantizer Q̃1 is used in place of Q1 at each occurrence, with
the values of τ possibly varying but always satisfying |τ| ≤ δ. If µ ≥ δ
and β satisfies
1 β ≤
2 + µ + δ
1 + µ + δ
, (1.42)
then for each m ≥ 1, ỹm :=
m
k=1 b̃kγk
satisfies
|y − ỹm| ≤ Cγm
, m = 1, 2, . . . , (1.43)
with C = 1 + µ + δ.
The simple proof of this theorem is given in [10].](https://image.slidesharecdn.com/1182935-250519121249-87b2d8c4/85/Foundations-Of-Computational-Mathematics-Minneapolis-2002-Felipe-Cucker-35-320.jpg)
![1. Some Fundamental Issues 25
(1.45) is to find u ∈ H1
0 (Ω) such that
a(u, w) = (f, w), w ∈ H1
0 (Ω), (1.46)
where a(y, w) := (∇y, ∇w), with (y, w) = (y, w)Ω :=
Ω
ywdx, is a
quadratic form on H1
0 (Ω). Here f can be any distribution in H−1
(Ω).
We use the notation
|||w|||2
:= a(w, w) = ∇w2
L2(Ω) (1.47)
to denote the energy norm which is the natural norm in which to measure
the performance of numerical methods. By Poincaré’s inequality there
exists a constant cΩ, depending on Ω, such that for any w ∈ H1
0 (Ω),
cΩwH1(Ω) ≤ |||w||| ≤ wH1(Ω), (1.48)
where w2
H1(Ω) = w2
L2(Ω) + ∇w2
L2(Ω).
1.4.2 Newest vertex subdivision
The numerical methods we shall discuss are those which approximate the
solution u of (1.46) by piecewise linear functions on triangular partitions
of the polygonal domain Ω. The partitions are generated by refining
triangles according to a fixed rule. While the following discussion applies
to a variety of refinement rules we shall specify the method of subdivision
to be newest vertex bisection since this will allow us a nice comparison
when we discuss adaptive methods. The book of Verfürth [18] and the
research article of Mitchell [15] describe this subdivision method and
give some of its properties. The article [1] gives finer results on newest
vertex bisection.
Let P0 = {∆} be an initial partition of Ω into a set of triangular cells.
To each edge associated to this partition we assign a label of 0 or 1. We
do this labelling in such a way that exactly one edge of any triangle has
a label 0 and all other edges have a label 1 (such a labelling is always
possible although it is nontrivial to prove; see [1]). For any triangle ∆
of P0, the vertex v(∆) opposite the side of this triangle labelled as 0 is
called its newest vertex. The edge in ∆ opposite to v(∆) will be denoted
by E(∆).
We have described how to assign to any triangle ∆ ∈ P0 a newest
vertex. Other triangles will arise by subdivision. We now describe the
rule for subdividing triangles and how to assign a newest vertex to each
triangle that arises by subdivision. If ∆ is such a triangle and if v(∆) is](https://image.slidesharecdn.com/1182935-250519121249-87b2d8c4/85/Foundations-Of-Computational-Mathematics-Minneapolis-2002-Felipe-Cucker-38-320.jpg)
![30 R. DeVore
into in this discussion. The reader may consult the the paper of Morin,
Nochetto, and Siebert [16] for a discussion relevant to the present prob-
lem of Laplace’s equation in two space dimensions. We mention only one
important fact. The local error estimators allow one to compute accu-
rate bounds for the global error |||u − un||| from the knowledge of f and
un. This is done through the residual ∆(u − un) = f − ∆(un).
1.4.5 Judging the performance of adaptive methods
The main question we wish to address is how can we evaluate the per-
formance of adaptive algorithms. Is there such a thing as an optimal
or near optimal adaptive algorithm? This question is in the same spirit
as the questions on image compression and A/D conversion. But there
is a significant difference. In the other cases, the function to be en-
coded/approximated was completely known to us. In the PDE case, we
only know u through our numerical approximations.
To begin the discussion, let us understand what approximants are
available to us when we use adaptive methods. We have noted earlier
that a partition P generated by an adaptive method can be associated to
finite trees T(P). As before, given any admissible partition P, we define
SP to be the space of piecewise linear functions which vanish on the
boundary of Ω and are subordinate to P. For any function w ∈ H1
0 (Ω)
and any such partition P, we define
E(w, SP ) := inf
S∈SP
|||w − S||| = |||u − uP |||, (1.52)
where uP is the Galerkin approximation to u. This is the smallest error
we can achieve by approximating u in the energy norm by the elements
of SP .
How should we measure the complexity of an adaptively generated
partition P. The most reasonable measure would be the number of com-
putations used to create P. This turns out to be closely related to the
number of subdivisions n = N(T(P)) used in the creation of P. As we
shall see later, for certain specific adaptive algorithms, we can bound the
number of computations used to create P and to compute uP by Cn.
With these remarks in mind, we define
σn(u) := inf
P ∈Pn
E(w, SP ) (1.53)
which is the best error we can possibly achieve in approximating w by
using n subdivisions. It is unreasonable to expect any adaptive algorithm](https://image.slidesharecdn.com/1182935-250519121249-87b2d8c4/85/Foundations-Of-Computational-Mathematics-Minneapolis-2002-Felipe-Cucker-43-320.jpg)
![1. Some Fundamental Issues 31
to perform exactly the same as σn(w). However, we may expect the same
asymptotic behavior. To quantify this, we introduce for any s 0, the
class As
of functions w ∈ H1
0 (Ω) such that
σn(w) ≤ Mn−s
, n = 1, 2, . . . . (1.54)
The smallest M for which (1.54) is satisfied is the norm in As
:
wAs := sup
n≥0
ns
σn(w). (1.55)
We have a similar measure of approximation when we restrict ourselves
to admissible partitions. Namely,
σa
n(w) := inf
P ∈Pa
n
E(w, SP )H1(Ω) (1.56)
now measures the best nonlinear approximation error obtained from ad-
missible partitions and Ȧs
:= Ȧs
(H1
0 (Ω)) consists of all w which satisfy
σa
n(w) ≤ Mn−s
, n = 1, 2, . . . . (1.57)
The smallest M for which (1.57) holds serves to define the norm wȦs .
It is shown in [1] that the two spaces As
are the same (see the discussion
in §1.4.7); so we do not make a distinction between them in going further.
1.4.6 The role of regularity theorems
It is of interest to know which functions are in As
. In [2], it is shown
that for a certain range of s (depending on the fact we are analyzing
approximation by piecewise linear functions), we have
As/2
≈ Bs
p(Lp(Ω)),
1
p
=
s − 1
2
+
1
2
. (1.58)
The nebulous notation (≈) used in this context means that there are
certain embeddings between Besov spaces and the space As
. We do not
want to get into the details (see [2]) but only work on this heuristic level.
The spaces in (1.58) lie on the Sobolev embedding line for H1
(Ω) so they
are very weak regularity conditions.
We can obtain a priori information on the possible performance of
adaptive methods by knowing regularity theorems for the above scale
of Besov spaces. For each u, let sNL := sNL(u) be the maximum of all
of the s such that u ∈ Bs
p(Lp(Ω)), 1
p = s−1
2 + 1
2 . The larger the value of
sNL, the better performance we can hope for an adaptive algorithm. In
particular, we would like to know if sNL is larger than the corresponding](https://image.slidesharecdn.com/1182935-250519121249-87b2d8c4/85/Foundations-Of-Computational-Mathematics-Minneapolis-2002-Felipe-Cucker-44-320.jpg)
![32 R. DeVore
value sL for standard Finite Element Methods as discussed in §1.4.3. We
emphasize that the above scale of Besov spaces that occur in describing
the approximation rates for adaptive approximation are quite different
from the scale of Besov spaces appearing in the linear case of standard
Finite Element Methods. For a given s 0 both require smoothness
of order s but this smoothness is measured in different ways. In the
standard case, the smoothness is relative to L2 but in the adaptive case
the smoothness is relative to an Lp which depends on s and gets smaller
as s gets larger. The Besov space in the adaptive case is much larger
than in the standard case and therefore more likely to contain u.
Let us take a simple example. We assume that the right side f is a
function in L2(Ω) and we ask what is the approximation error we can
expect to receive if we invest n triangular cells to the approximation. As
we have already described in §1.4.3, from this assumption on f we can
always conclude that f ∈ Bs
∞(L2(Ω) for s = 3/2 and depending on the
corners in the domain s could even be larger but never bigger than s = 2.
Thus, 3/2 ≤ sL ≤ 2. This means that in general we cannot expect more
than an approximation rate O(2
−n(sL−1)
2 ) when using standard finite
element methods with 2n
triangular cells; see (1.51).
What is the situation for sNL? To determine sNL we need new reg-
ularity theorems that infer regularity in the new scale of Besov spaces
Bs
p(Lp(Ω)), with 1
p
= s−1
2
+ 1
2
. Regularity theorems of this type were
proved in [8, 7]. For example, in the case of f ∈ L2(Ω), it is shown that
sNL ≥ 2 (see [8]). This means that for an investment of n triangular
cells we should always obtain an error like O(n−1/2
). This shows that
potentially an adaptive method can perform much better than a stan-
dard Finite Element Method. If the standard method gives an error
using a partition with n triangular cells, we might be able to obtain
an error as small as 2
using the same number of cells in an adaptive
method.
1.4.7 The performance of adaptive methods
The above analysis is all well and good but it provides us no information
about specific numerical methods and does not even answer the question
of whether such numerical methods converge. In fact, it was not until
very recently that truly adaptive numerical methods were constructed
and proven to converge (Dörfler [11], and Morin, Nochetto, Siebert [16])
and this convergence is only established for simple problems and special
methods of subdivision. We are much more ambitious and would like](https://image.slidesharecdn.com/1182935-250519121249-87b2d8c4/85/Foundations-Of-Computational-Mathematics-Minneapolis-2002-Felipe-Cucker-45-320.jpg)
![1. Some Fundamental Issues 33
to construct adaptive methods which perform at the optimal theoretical
approximation rate.
Suppose we have an adaptive algorithm in hand that we have some-
how constructed. Given an error tolerance 0, we run the adaptive
algorithm until we achieve an error . We shall say that an algorithm
is near optimal for s provided that whenever u ∈ As
the number N of
subdivisions used by the adaptive algorithm used to produce this error
satisfies
N ≤ C|u|As −1/s
, 0 1. (1.59)
This means that the adaptive algorithm performs at the optimal rate on
the class As
. The question of course is whether we can find an adaptive
algorithm with this near optimal performance.
An adaptive algorithm with near optimal performance for solving
(1.45) was constructed in [1]. Not only does this algorithm have op-
timality in terms of the number of subdivisions used but it is also shown
that the number of computations N(comp) needed to find uPn
satisfies
N(comp) ≤ C−1/s
, 0 1. (1.60)
We shall not describe the algorithm of [1] in detail but instead give
a brief overview of the algorithm and then pick up on details for the
two core results that allow the proof of near optimality. The algorithm
proceeds by setting error tolerances k := 2−k
and generating partitions
Pk, k = 1, 2, . . . , in which the error is guaranteed to be k. We stop at
the first integer n where the error (which we can measure a posteriori)
is ≤ . For sure, when
2−n
≤ (1.61)
this is the case. But the error bound may actually be attained for an
earlier value of k.
The construction of Pk from Pk−1 consists of several subiterations
taking an admissible partition Pk−1,j to a new admissible partition
Pk−1,j+1. Here, Pk−1,0 = Pk−1 and Pk is then found by coarsening a
partition Pk−1,K where K is a fixed integer. Each subiteration (send-
ing Pk−1,j to Pk−1,j+1) consists of the general strategy of marking cells,
subdividing, further markings and subdividing to remove hanging nodes.
The markings are made using the local error estimators found in [16] and
marking enough cells so as to capture a fraction of the global error (this
is known as bulk chasing). It was shown in [16] that such an iteration](https://image.slidesharecdn.com/1182935-250519121249-87b2d8c4/85/Foundations-Of-Computational-Mathematics-Minneapolis-2002-Felipe-Cucker-46-320.jpg)
![34 R. DeVore
reduces the error in the energy norm. The role of K (which is a fixed
integer) is to guarantee that after the coarsening step we have
|||u − uPk
||| ≤ 2−1
|||u − uPk−1
||| ≤ k = 2−k
. (1.62)
Of course, (1.62) guarantees the algorithm converges but gives little
information about convergence rates because we have said nothing about
how many subdivisions were needed to construct Pk. However, a deeper
analysis of the algorithm shows that it is near optimal, in the sense
described above, for the range 0 s 1/2 (we cannot expect to exceed
this range because we are using piecewise linear functions to approximate
in the H1
(Ω) norm).
The proof of near optimality rests on two fundamental results. The
first of these concerns the number of subdivisions necessary to remove
hanging nodes at a given iteration of the algorithm. It is necessary to
have such an estimate to be sure that the number of triangular cells does
not explode when the hanging nodes are removed. Suppose that we have
a sequence of partitions P̃k, k = 1, . . . , m, where each P̃k+1 is obtained
from P̃k by marking a set M̃k, subdividing these cells, and then apply a
further set of markings M̃
k and subdivisions to remove hanging nodes.
The fundamental result proved in [1] shows that
#(P̃m) ≤ #(P̃0) + C2(#(M̃0) + · · · + #(M̃m−1)). (1.63)
This means that the number of markings used to remove hanging nodes
is bounded by a multiple of the number of original markings. This is used
in the adaptive finite element method to show, among other things, that
#(Pk) ≤ C#(Pk−1) (1.64)
with C an absolute constant.
The proof of (1.63) does not proceed, as one might expect, by an
induction step which bound #(M̃
k) by C#(M̃k). Rather one needs to
keep track of the entire history in the creation of new cells, tracing this
history back to earlier cells that were subdivided; see [1] for details.
The second core result that is needed to keep control on the number
of triangular cells and arrive at an estimate like (1.16) is that of coars-
ening. Without this step, we would not arrive (1.59). The coarsening
step proceeds as follows. After constructing the set Pk,K, we compute a
Galerkin approximation vk := uPk,K
to u from SPk,K
which satisfies
|||u − vk||| ≤ γk+1 (1.65)
with γ a small fixed positive constant. The choice of γ influences the](https://image.slidesharecdn.com/1182935-250519121249-87b2d8c4/85/Foundations-Of-Computational-Mathematics-Minneapolis-2002-Felipe-Cucker-47-320.jpg)
![1. Some Fundamental Issues 35
choice of K above. The function vk is known to us and the coarsening
algorithm uses information about this function to create a smaller par-
tition Pk+1 ⊂ Pk,K such that vk can be approximated by an element of
SPk+1
to accuracy αk where again 0 α 1 is a fixed constant that is
chosen appropriate to the discussion that follows. The property gained
by the coarsening step is that Pk+1 satisfies
#(Pk+1) ≤ C0|u|As
−1/s
k (1.66)
with C0 an absolute constant. It is then shown that the Galerkin ap-
proximation uk+1 = uPk+1
satisfies
|||u − uk+1||| ≤
k
2
= k+1. (1.67)
This estimate together with the control on the size of the partition Pk+1
given in (1.66) combine to show that the algorithm is near optimal.
It is vital in the coarsening algorithm to keep a control on the number
of computations needed to find Pk+1. This is in fact the deeper aspect of
the coarsening algorithm. Normally to find a near optimal tree approx-
imation with n nodes, one might expect to have to do exponential in n
computations since there are that many trees with n nodes. It is shown
in [3] that the number of computations can be limited to O(n) by using
a clever adaptive strategy.
References
[1] Binev, P., Dahmen, W. and DeVore, R. (2002). Adaptive finite element
methods with convergence rates, IMI Preprint Series 12 (University of
South Carolina).
[2] Binev, P., Dahmen, W., DeVore, R. and Petrushev, P. (2002). Approxi-
mation Classes for Adaptive Methods, Serdica Math. J. 28, 391–416.
[3] Binev, P. and DeVore, R. (2002). Fast computation in tree approximation,
IMI Preprint Series 11 (University of South Carolina).
[4] Calderbank, R. and Daubechies, I. (2002). The Pros and Cons of Democ-
racy IEEE Transactions in Information Theory 48, 1721–1725.
[5] Cohen, A., Dahmen, W., Daubechies, I. and DeVore, R. (2001). Tree
approximation and encoding, ACHA 11, 192–226.
[6] Cohen, A., Daubechies, I., Guleryuz, O. and Orchard, M. (2002). On the
importance of combining non-linear approximation with coding strategies,
IEEE Transactions in Information Theory 48, 1895–1921.
[7] Dahlke, S. (1999). Besov regularity for elliptic boundary value problems
on polygonal domains, Appl. Math. Lett. 12, 31–36.
[8] Dahlke, S. and DeVore, R. (1997). Besov regularity for elliptic boundary
value problems, Comm. Partial Differential Equations 22, 1–16.](https://image.slidesharecdn.com/1182935-250519121249-87b2d8c4/85/Foundations-Of-Computational-Mathematics-Minneapolis-2002-Felipe-Cucker-48-320.jpg)
![36 R. DeVore
[9] Daubechies, I. and DeVore, R. Reconstructing a bandlimited function
from very coarsely quantized data: A family of stable sigma-delta modu-
lators of arbitrary order, Annals of Mathematics, to appear.
[10] Daubechies, I., DeVore, R., Gunturk, C.S. and Vaishampayan, V.
Exponential Precision in A/D Conversion with an Imperfect Quantizer,
preprint.
[11] Dörfler, W. (1996). A convergent adaptive algorithm for Poisson’s equa-
tion, SIAM J. Numer. Anal. 33, 1106–1124.
[12] Güntürk, S. One-bit Sigma-Delta quantization with exponential accuracy,
preprint.
[13] Kolmogorov, A.N. and Tikhomirov, V.M. (1959). -entropy and -capacity
of sets in function spaces, Usephi 14, 3–86. (Also in AMS Translations,
Series 2 17 (1961), 277–364).
[14] Meyer, Y. (2001). Oscillating pattern in image processing and in some
nonlinear evolution equations, The Fifteenth Dean Jaqueline B. Lewis
Memorial Lectures.
[15] Mitchell, W.F. (1989). A comparison of adaptive refinement techniques
for elliptic problems, ACM Transaction on Math. Software 15, 326–347.
[16] Morin, P., Nochetto, R. and Siebert, K. (2000). Data Oscillation and
Convergence of Adaptive FEM, SIAM J. Numer. Anal. 38, 466–488.
[17] Mumford, D. and Shah, J. (1985). Boundary detection by minimizing
functionals, Proc. IEEE Conf. Comp. Vis Pattern Recognition.
[18] Verfürth, R. (1996). A Review of A Posteriori Error Estimation and
Adaptive Mesh-Refinement Techniques (Wiley-Teubner, Chichester).](https://image.slidesharecdn.com/1182935-250519121249-87b2d8c4/85/Foundations-Of-Computational-Mathematics-Minneapolis-2002-Felipe-Cucker-49-320.jpg)
![38 H. Edelsbrunner and J. Harer
example oceanography where researchers study the distribution of vari-
ous attributes of water, with the goal to shed light on the ocean dynamics
and gain insight into global climate changes [4]. One such attribute is
temperature, another is salinity, an important indicator of water density.
The temperature distribution is often studied within a layer of constant
salinity, because water tends to mix along but not between these layers.
Mathematically, we may think of temperature and salinity as two con-
tinuous functions on a common portion of three-dimensional space. A
layer is determined by a level surface of the salinity function, and we are
interested in the temperature function restricted to that surface. This
is a continuous function on a two-dimensional domain, whose critical
points are generically minima, saddles, and maxima. In this paper, we
study the paths these critical points take when the salinity value varies.
As it turns out, these paths are also the paths the critical points of the
salinity function take if we restrict it to the level surfaces of the tem-
perature function. More generally, we study the relationship between
continuous functions defined on a common manifold by analyzing the
critical points within level set restrictions.
Sometimes it is useful to make up auxiliary functions to study the
properties of given ones. Consider for example a function that varies
with time, such as the gravitational potential generated by the sun,
planets, and moons in our solar system [18]. At the critical points of
that potential, the gravitational forces are at an equilibrium. The plan-
ets and moons move relative to each other and the sun, which implies
that the critical points move, appear, and disappear. To study such a
time-varying function, we introduce another, whose value at any point
in space-time is the time. The paths of the critical points of the gravita-
tional potential are then the Jacobi set of the two functions defined on
a common portion of space-time.
Results. The main object of study in this paper is the Jacobi set J =
J(f0, f1, . . . , fk) of k + 1 ≤ d Morse functions on a common d-manifold.
By definition, this is the set of critical points of f0 restricted to the
intersection of the level sets of f1 to fk. We observe that J is symmetric
in the k + 1 functions because it is the set of points at which the k + 1
gradient vectors are linearly dependent. In the simplest non-trivial case,
we have two Morse functions on a common 2-manifold. In this case, the
Jacobi set J = J(f, g) = J(g, f) is generically a collection of pairwise
disjoint smooth curves that are free of any self-intersections. Fig. 2.1
illustrates the concept for two Morse functions on the two-dimensional](https://image.slidesharecdn.com/1182935-250519121249-87b2d8c4/85/Foundations-Of-Computational-Mathematics-Minneapolis-2002-Felipe-Cucker-51-320.jpg)
![2. Jacobi Sets 39
Fig. 2.1. The two partially bold and partially dotted longitudinal circles form
the Jacobi set of f, g : M → R, where M is the torus and f and g map a point
x ∈ M to the Cartesian coordinates of its orthogonal projection on a plane
parallel to the longitudes.
torus. The Jacobi set of a generic collection of k + 1 Morse functions is
a submanifold of dimension k, provided d 2k − 2. The first time this
inequality fails is for d = k + 1 = 4. In this case, the Jacobi set is a
3-manifold except at a discrete set of points.
We describe an algorithm that computes an approximation of the
Jacobi set for k +1 piecewise linear functions that are interpolated from
values given at the vertices. In the absence of smoothness, it simulates
genericity and differentiability and computes J as a subcomplex of the
triangulation. The algorithm is combinatorial (as opposed to numerical)
in nature and reduces the computations to testing the criticality of the
k-simplices in the triangulation. Whether or not a simplex is considered
critical depends on its local topology. By using Betti numbers to express
that topology, we get an algorithm that works for triangulated manifolds
and runs in time that is polynomial in the number of simplices. Assuming
the links have sizes bounded from above by a constant, the running time
is proportional to the number of simplices.
Related prior work. The work in this paper fits within the general
area of singularities of smooth mappings, which was pioneered by Hassler
Whitney about half a century ago; see e.g. [19]. In this body of work,
the fold of a mapping from a d-dimensional manifold Md
to a (k + 1)-
dimensional manifold Nk+1
is the image of the set of points at which
the matrix of partial derivatives has less than full rank. In this paper,
we consider (k + 1)-tuple of Morse functions, which are special smooth
mappings in which the range is the (k +1)-dimensional Euclidean space,
Rk+1
. The fold of such a special mapping is the image of the Jacobi
set. Our restriction to Euclidean space is deliberate as it furnishes the
framework needed for our algorithm.](https://image.slidesharecdn.com/1182935-250519121249-87b2d8c4/85/Foundations-Of-Computational-Mathematics-Minneapolis-2002-Felipe-Cucker-52-320.jpg)
![40 H. Edelsbrunner and J. Harer
Whitney considered the case of a mapping between surfaces and stud-
ied mappings where d is small relative to k. A classic theorem in differ-
ential topology is the Whitney Embedding Theorem which states that
a closed, orientable manifold of dimension n can always be embedded in
R2n
. Thom extended the work of Whitney, studying the spaces of jets
(two functions have the same k-jet at p if their partial derivatives of
order k or less are equal). The Thom Transversality Theorem, together
with the Thom-Boardman stratification of C∞
functions on M, give
characterizations of the singularities of generic functions [11]. Mather
studied singularities from a more algebro-geometric point of view and
proved the equivalence of several definitions of stability for a map (a
concept not used in this work). In particular he provided the appropri-
ate framework to reconstruct a map from its restrictions to the strata
of the Thom-Boardman stratification.
In a completely different context, Nicola Wolpert used Jacobi curves
to develop exact and efficient algorithms for intersecting quadratic sur-
faces in R3
[20]. She does so by reducing the problem to computing the
arrangement of the intersection curves projected to R2
. Any such curve
can be written as the zero set of a smooth function R2
→ R, and any
pair defines another curve, namely the Jacobi set of the two functions.
Wolpert refers to them as Jacobi curves and uses them to resolve tan-
gential and almost tangential intersections. To be consistent with her
terminology, we decided to modify that name and to refer to our more
general concept as Jacobi sets.
Outline. §2.2 reviews background material from differential and combi-
natorial topology. §2.3 introduces the Jacobi set of two Morse functions.
§2.4 describes an algorithm that computes this set for piecewise linear
data. §2.5 generalizes the results to three or more Morse functions. §2.6
discusses a small selection of applications. §2.7 concludes the paper.
2.2 Background
This paper contains results for smooth and for piecewise linear functions.
We need background from Morse theory [15, 16] for smooth functions
and from combinatorial and algebraic topology [1, 17] for designing an
algorithm that works on piecewise linear data.
Morse functions. Let M be a smooth and compact d-manifold without
boundary. The differential of a smooth map f : M → R at a point x of](https://image.slidesharecdn.com/1182935-250519121249-87b2d8c4/85/Foundations-Of-Computational-Mathematics-Minneapolis-2002-Felipe-Cucker-53-320.jpg)
![2. Jacobi Sets 41
the manifold is a linear map dfx : TMx → R mapping the tangent space
at x to R. A point x ∈ M is critical if dfx is the zero map; otherwise, it
is regular. Let , be a Riemannian metric, i.e. an inner product in the
tangent spaces that varies smoothly on M. Since each vector in TMx
is the tangent vector to a curve c in M through x, the gradient ∇f
can be defined by the formula ∂c/∂t, ∇f = ∂(f ◦ c)/∂t, for every c. It
is always possible to choose coordinates xi so that the tangent vectors
∂
∂xi
(x) are orthonormal with respect to the Riemannian metric. For such
coordinates, the gradient is given by the familiar formula
∇f(x) =
∂f
∂x1
(x),
∂f
∂x2
(x), . . . ,
∂f
∂xd
(x)
.
We compute in local coordinates the Hessian of f:
Hf (x) =
∂2
f
∂x2
1
(x) . . . ∂2
f
∂xd∂x1
(x)
.
.
.
...
.
.
.
∂2
f
∂x1∂xd
(x) . . . ∂2
f
∂x2
d
(x)
,
which is a symmetric bi-linear form on the tangent space TMx. A critical
point p is non-degenerate if the Hessian is non-singular at p. The Morse
Lemma states that near a non-degenerate critical point p, it is possible
to choose local coordinates so that the function takes the form
f(x1, . . . , xd) = f(p) ± x2
1 ± . . . ± x2
d.
The number of minus signs is the index of p; it equals the number of
negative eigenvalues of Hf (p). The existence of these local coordinates
implies that non-degenerate critical points are isolated. The function f
is a called a Morse function if
(i) all its critical points are non-degenerate, and
(ii) f(p) = f(q) for all critical points p = q.
Transversality and stratification. Let F : X → Y be a smooth map
between two manifolds, and let U ⊆ Y be a smooth submanifold. The
map F is transversal to U if for every y ∈ U and every x ∈ F−1
(y), we
have dFx(TXx)+TUy = TYy. In words, the basis vectors of the image of
the tangent space of X at x under the derivative together with the basis
vectors of the tangent space of U at y = F(x) span the tangent space of
Y at y. The Transversality Theorem of differential topology asserts that
if F is transversal to U then F−1
(U) is a smooth submanifold of X and
the co-dimension of F−1
(U) in X is the same as that of U in Y [12].](https://image.slidesharecdn.com/1182935-250519121249-87b2d8c4/85/Foundations-Of-Computational-Mathematics-Minneapolis-2002-Felipe-Cucker-54-320.jpg)
Foundations Of Computational Mathematics Minneapolis 2002 Felipe Cucker
- 1. Foundations Of Computational Mathematics Minneapolis 2002 Felipe Cucker download https://ebookbell.com/product/foundations-of-computational- mathematics-minneapolis-2002-felipe-cucker-2365870 Explore and download more ebooks at ebookbell.com
- 2. Here are some recommended products that we believe you will be interested in. You can click the link to download. Foundations Of Computational Mathematics Santander 2005 Luis M Pardo https://ebookbell.com/product/foundations-of-computational- mathematics-santander-2005-luis-m-pardo-2379334 Foundations Of Computational Mathematics Hong Kong 2008 1st Edition Felipe Cucker https://ebookbell.com/product/foundations-of-computational- mathematics-hong-kong-2008-1st-edition-felipe-cucker-2502210 Foundations Of Computational Mathematics Proceedings Of The Smalefest 2000 Hong Kong 1317 2000 Stephen Smale Felipe Cucker J Maurice Rojas Eds https://ebookbell.com/product/foundations-of-computational- mathematics-proceedings-of-the-smalefest-2000-hong- kong-1317-2000-stephen-smale-felipe-cucker-j-maurice-rojas-eds-2632266 Handbook Of Numerical Analysis Special Volume Foundations Of Computational Mathematics Special F Cucker https://ebookbell.com/product/handbook-of-numerical-analysis-special- volume-foundations-of-computational-mathematics-special-f- cucker-893646
- 3. Logical Foundations Of Mathematics And Computational Complexity A Gentle Introduction 1st Edition Pavel Pudlk Auth https://ebookbell.com/product/logical-foundations-of-mathematics-and- computational-complexity-a-gentle-introduction-1st-edition-pavel- pudlk-auth-4242090 The Digital And The Real World Computational Foundations Of Mathematics Science Technology And Philosophy Klaus Mainzer https://ebookbell.com/product/the-digital-and-the-real-world- computational-foundations-of-mathematics-science-technology-and- philosophy-klaus-mainzer-57433232 Mathematical Foundations Of Computational Engineering A Handbook Prof Dr Peter Jan Pahl https://ebookbell.com/product/mathematical-foundations-of- computational-engineering-a-handbook-prof-dr-peter-jan-pahl-4187620 Mathematical Foundations Of Computational Electromagnetism 1st Ed Franck Assous https://ebookbell.com/product/mathematical-foundations-of- computational-electromagnetism-1st-ed-franck-assous-7147502 Multimodal Brain Image Analysis And Mathematical Foundations Of Computational Anatomy 4th International Workshop Mbia 2019 And 7th International Workshop Mfca 2019 Held In Conjunction With Miccai 2019 Shenzhen China October 17 2019 Proceedings 1st Ed 2019 Dajiang Zhu https://ebookbell.com/product/multimodal-brain-image-analysis-and- mathematical-foundations-of-computational-anatomy-4th-international- workshop-mbia-2019-and-7th-international-workshop-mfca-2019-held-in- conjunction-with-miccai-2019-shenzhen-china- october-17-2019-proceedings-1st-ed-2019-dajiang-zhu-10801144
- 6. This page intentionally left blank
- 7. London Mathematical Society Lecture Note Series Managing Editor: Professor N.J. Hitchin, Mathematical Institute, 24–29 St. Giles, Oxford OX1 3DP, UK All the titles listed below can be obtained from good booksellers or from Cambridge University Press. For a complete series listing visit http://publishing.cambridge.org/stm/mathematics/lmsn/ 283. Nonlinear elasticity, R. Ogden & Y. Fu (eds) 284. Foundations of computational mathematics, R. DeVore, A. Iserles & E. Suli (eds) 285. Rational points on curves over finite fields, H. Niederreiter & C. Xing 286. Clifford algebras and spinors, 2nd edn, P. Lounesto 287. Topics on Riemann surfaces and Fuchsian groups, E. Bujalance, A.F. Costa & E. Martinez (eds) 288. Surveys in combinatorics, 2001, J.W.P. Hirschfeld (ed) 289. Aspects of Sobolev-type inequalities, L. Saloffe-Coste 290. Quantum groups and Lie theory, A. Pressley 291. Tits buildings and the model theory of groups, K. Tent 292. A quantum groups primer, S. Majid 293. Second order partial differential equations in Hilbert spaces, G. da Prato & J. Zabczyk 294. Introduction to operator space theory, G. Pisier 295. Geometry and integrability, L. Mason & Y. Nutku (eds) 296. Lectures on invariant theory, I. Dolgachev 297. The homotopy theory of simply-connected 4-manifolds, H.J. Baues 298. Higher operads, higher categories, T. Leinster 299. Kleinian groups and hyperbolic 3-manifolds, Y. Komori, V. Markovic & C. Series (eds) 300. Introduction to Möbius differential geometry, U. Hertrich-Jeromin 301. Stable modules and the D(2)-problem, F.A.E. Johnson 302. Discrete and continuous nonlinear Schrödinger systems, M. Ablowitz, B. Prinari & D. Trubatch 303. Number theory and algebraic geometry M. Reid & A. Skorobogatov 304. Groups St Andrews 2001 in Oxford vol. 1, C.M. Campbell, E.F. Robertson & G.C. Smith (eds) 305. Groups St Andrews 2001 in Oxford vol. 2, C.M. Campbell, E.F. Robertson & G.C. Smith (eds) 306. Peyresq lectures on geometric mechanics and symmetry, J. Montaldi & T. Ratiu (eds) 307. Surveys in combinatorics, 2003, C.D. Wensley (ed) 308. Topology, geometry and quantum field theory, U.L. Tillmann (ed) 309. Corings and comodules, T. Brzezinski & R. Wisbauer 310. Topics in dynamics and ergodic theory, S. Bezuglyi & S. Kolyada (eds) 311. Groups: topological, combinatorial and arithmetic aspects, T.W. Müller (ed)
- 8. London Mathematical Society Lecture Note Series: 312 Foundations of Computational Mathematics: Minneapolis, 2002 Edited by Felipe Cucker The City University of Hong Kong Ron DeVore University of South Carolina Peter Olver University of Minnesota Endre Süli University of Oxford
- 9. published by the press syndicate of the university of cambridge The Pitt Building, Trumpington Street, Cambridge, United Kingdom cambridge university press The Edinburgh Building, Cambridge CB2 2RU, UK 40 West 20th Street, New York, NY 10011–4211, USA 477 Williamstown Road, Port Melbourne, VIC 3207, Australia Ruiz de Alarcón 13, 28014 Madrid, Spain Dock House, The Waterfront, Cape Town 8001, South Africa http://www.cambridge.org c Cambridge University Press 2004 This book is in copyright. Subject to statutory exception and to the provisions of relevant collective licensing agreements, no reproduction of any part may take place without the written permission of Cambridge University Press. First published 2004 Printed in the United Kingdom at the University Press, Cambridge Typeface Computer Modern 10/13pt System L A TEX 2ε [author] A catalogue record for this book is available from the British Library Library of Congress Cataloguing in Publication data available ISBN 0 521 54253 7 paperback
- 10. Contents Preface page vii 1 Some Fundamental Issues R. DeVore 1 2 Jacobi Sets H. Edelsbrunner and J. Harer 37 3 Approximation of boundary element operators S. Börm and W. Hackbusch 58 4 Quantum Complexity S. Heinrich 76 5 Straight-line Programs T. Krick 96 6 Numerical Solution of Structured Problems T. Apel, V. Mehrmann, and D. Watkins 137 7 Detecting Infeasibility M.J. Todd 157 8 Maple Packages and Java Applets I.M. Anderson 193 v
- 12. Preface The Society for the Foundations of Computational Mathematics sup- ports fundamental research in computational mathematics and its ap- plications, interpreted in the broadest sense. As part of its endeavour to promote research across a wide spectrum of subjects concerned with computation, the Society regularly organises conferences and specialist workshops which bring together leading researchers working in diverse fields that impinge on various aspects of computation. Major conferences of the Society were held in Park City (1995), Rio de Janeiro (1997), Oxford (1999), and Minneapolis (2002). The next FoCM conference will take place at the University of Santander in Spain in July 2005. More information about FoCM is avail- able from the website http://www.focm.net. The conference in Minneapolis on 5-14 August 2002 was attended by several hundred scientists. Workshops were held in eighteen fields which included: the foundations of the numerical solution of partial differential equations, geometric integration and computational mechanics, learning theory, optimization, special functions, approximation theory, computa- tional algebraic geometry, computational number theory, multiresolution and adaptivity, numerical linear algebra, quantum computing, compu- tational dynamics, geometrical modelling and animation, image and sig- nal processing, stochastic computation, symbolic analysis, complexity and information-based complexity theory. In addition to the workshops, eighteen plenary lectures, concerned with a broad spectrum of topics connected to computational mathematics, were delivered by some of the world’s foremost researchers. This volume is a collection of articles, based on the plenary talks presented at FoCM 2002. The topics covered in the lectures — ranging from the applications of computational math- ematics in geometry and algebra to optimization theory, from quantum vii
- 13. viii Preface complexity to the numerical solution of partial differential equations, from numerical linear algebra to Morse theory — reflect the breadth of research within computational mathematics as well as the richness and fertility of interactions between seemingly unrelated branches of pure and applied mathematics. We hope that the volume will be of interest to researchers in the field of computational mathematics but also to non-experts who wish to gain insight into the state of the art in this active and significant field. Like previous FoCM conferences, the Minneapolis gathering proved itself as a unique meeting point of researchers in computational math- ematics and of theoreticians in mathematics and in computer sciences. While presenting plenary talks by foremost world authorities and main- taining the highest technical level in the workshops, the emphasis, like in Park City, Rio de Janeiro and Oxford, was on multidisciplinary in- teraction across subjects and disciplines, in an informal and friendly atmosphere. It is only fair to say that for many of us the opportunity of meeting colleagues from different subject-areas and identifying the wide- ranging, and often surprising, common denominator to our research was a real journey of discovery. We wish to express our gratitude to the local organisers and adminis- trative staff of our hosts, the Institute of Mathematics and Its Applica- tions and the Department of Mathematics at the University of Minnesota at Minneapolis, for making FoCM 2002 such a success. We also wish to thank the National Science Foundation, the Digital Technology Center in Minneapolis, IBM, the Office of Naval Research, the Number Theory Foundation and the American Institute of Mathematics for their gen- erous sponsorship and support. Above all, however, we wish to express our gratitude to all participants of FoCM 2002 for attending the meeting and making it such an exciting, productive and scientifically stimulating event.
- 14. 1 Some Fundamental Issues in Computational Mathematics Ronald DeVore Department of Mathematics University of South Carolina Columbia, SC 29208 Email: devore@math.sc.edu Abstract We enter a discussion as to what constitutes the ‘foundations of compu- tational mathematics’. While not giving a definition, we give examples from image/signal processing and numerical computation where foun- dational issues have helped to ‘correctly’ formulate problems and guide their solution. 1.1 The question While past chair of the organization Foundations of Computational Mathematics (FOCM), I was frequently asked what is the meaning of ‘foundations of computational mathematics’. Most people understand what computational mathematics is. So the question really centers around the meaning of ‘foundations’ in this context. Even though I have thought about this quite a while, I would not dare to try to give a precise definition of foundations – I am sure it would be picked apart. However, I would like in this presentation to give some examples where the ad- herence to fundamental questions has helped to shape the formulation of computational issues and more importantly contributed to their so- lution. The examples I choose in signal/image processing and numerical methods for PDEs are of course related to my own research. I am sure 0 This work has been supported by the Office of Naval Research Contract Nr. N0014- 91-J1343, the Army Research Office Contract Nr. DAAD 19-02-1-0028, and the National Science Foundation Grant DMS0221642. 1
- 15. 2 R. DeVore there are many other stories of the type I put forward that are waiting to be told. The first of the three examples that I will discuss is that of image compression. This subject has grown rapidly over the last decade with an important infusion of ideas from mathematics especially the theories of wavelets and nonlinear approximation. The main topic to be addressed here is how we can decide which algorithms for compression are optimal. A somewhat related topic will concern Analog to Digital (A/D) con- version of signals. This is an area that is important in consumer electron- ics. The story here centers around trying to understand why engineers do A/D conversion in the way they do, which by the way is very coun- terintuitive to what a first mathematical analysis would suggest. Finally, I discuss adaptive methods for solving PDEs. This is an ex- tremely important area of numerical computation in our quest to solve large problems to higher and higher resolution. The question to be an- swered is how can we know when an adaptive method is optimal in its performance. 1.2 Image compression Digital signal processing has revolutionized the storage and transmission of audio and video signals as well as still images, in consumer electronics and in more scientific settings (such as medical imaging). The main advantage of digital signal processing is its robustness: although all the operations have to be implemented with, of necessity, not quite ideal hardware, the a priori knowledge that all correct outcomes must lie in a very restricted set of well separated numbers makes it possible to recover them by rounding off appropriately. Every day, millions of digitized images are created, stored, and trans- mitted over the Internet or using other mediums. A grey scale image is an array (matrix) of pixel values. It is already important to have a mathematical model for what these pixel values represent. We shall view the pixel array as arising in the following fashion. We have a light inten- sity function f defined on a continuum Ω. For simplicity we assume that Ω := [0, 1]2 and that f takes values in [0, 1) (the latter can be achieved by simple renormalization). Digitization corresponds to two operations: averaging and quantization. We take a tiling of Ω into squares Q and associate to each square Q the average intensity fQ := 1 |Q| I f(x) dx,
- 16. 1. Some Fundamental Issues 3 where |Q| denotes the Lebesgue measure of Q. The pixel values pQ are derived from the numbers fQ ∈ [0, 1) by quantization. We write fQ in its binary expansion fQ = ∞ j=1 bj(fQ)2−j and define the pixel value pQ := m j=1 bj(fQ)2−j . Typical choices of m are m = 8 (one byte per pixel) or m = 16. The array I := I(f) := (pQ) of pixel values is a digitization of f. The accuracy at which I resolves f depends on the fineness of the tiling and the accuracy of the quantization (i.e. size of m). We do not really know f. We only see it through the digitized image I(f). In practice, the pixel values pQ are corrupted by noise but we shall ignore this in our discussion since we are aiming in a different direction. We see that a digitized image in its raw form is described by mN bits where N is the number of squares in the tiling. Lossy compression seeks to significantly reduce this number of bits used to represent f at the expense of some loss in the fidelity of resolution. Hopefully, this loss of fidelity is not perceptible. There are two parts to a lossy compression scheme. The encoder assigns to each pixel array I a bitstream B(I). A decoder gives the recipe for changing any given bitstream B back into a pixel array. After encoding and then decoding the resulting pixel values p̄Q will generally not be the same as the original pQ. Some fidelity is lost. One can imagine, given the practical importance of the compression problem, that there are a ton of encoding/decoding schemes. How can one decide from this myriad of choices which is the best? Engineers have introduced a number called the PSNR (Peak Signal to Noise Ratio) which measures the performance of a given encoding/decoding on a given digitized image I. It is not necessary to give its precise definition here but simply mention that it measures the least squares distortion ((#I)−1 Q[pQ − p̄Q]2 )1/2 as a function of the number of bits. Here #(I) is the number of pixels. A new encoding scheme is tested by its performance (PSNR) on a few test images – the Lena image being the most widely used. Now, there is a fundamental question here. Should the quality of a compression algorithm be determined by its PSNR performance on a few test images? Given a collection of 2k images, we can encode them all with k bits per image by simply enumerating them in binary. So on a mathematical level this test of performance is quite unsatisfactory. What has any of this to do with “foundations of computational mathematics”. Well, we cannot have a decidable competition among
- 17. 4 R. DeVore compression algorithms without a clear and precise formulation of the compression problem. This is a foundations question that rests on two issues that we must clarify. The first is the metric we are going to use to compare two images (for example, the original and the compressed image). The second is the class of images we wish to compress. We shall briefly discuss these issues. 1.2.1 The metric We have already mentioned the PSNR which is based on the 2 metric. In our view of images as functions, this corresponds to the L2(Ω) func- tion metric. Is this the obvious choice? By no means. This choice seems to be more a matter of convenience and tradition. It is easy to solve optimization problems in the L2 metric. Certainly the choice of metric must depend on the intended applica- tion. In some targeted applications such as feature extraction and image registration, the least squares metric is clearly not appropriate and is better replaced by metrics such as L∞ or maximum gradient. Most compression is directed at producing visually pleasing images which cannot be distinguished from the original by the human eye. Thus, we can speak about the metric of the human visual system. The prob- lem is that this vague notion is useless mathematically. Our goal would be to derive a mathematical metric which is a good model for the hu- man visual system. There are some mathematical models for human vision which may be useful in directing our pursuit but little is agreed upon. So, at this stage, we are left with using simple mathematical metrics such as the Lp(Ω) norms, 0 p ≤ ∞, or certain smoothness norms. The point we wish to make here is not so much as to which metric is better but rather that any serious mathematical comparison of compres- sion algorithms must at the outset agree on the metric used to measure distortion. Once this is decided we can go further. 1.2.2 Model classes of images Once we have chosen a mathematical metric in which to measure distance between images, the question turns to describing the class of images that we wish to compress. This is a subject that spurs many interesting debates. We will touch on this only briefly and in a very prejudicial way.
- 18. 1. Some Fundamental Issues 5 There are two main models for images: the stochastic and the deter- ministic. Stochastic models are deeply embedded in the engineering and information theory communities influenced in a large part by Shannon’s theory for optimal encoding. Deterministic models take the view we have presented of an image as a function defined on a continuum. We have begun by assuming only that an image is a bounded function. This is too broad of a class of functions to serve as the description of the images we wish to compress. Images have more structure. One deterministic view of an image function f is that it is a sum of fundamental components corresponding to edges, texture, and noise. For example the famous model of Mumford and Shah [17] views the image as a sum f = u + v of a component u of Bounded Variation (BV) and a component v in L2. The component u is not an arbitrary BV function but rather has gradient given by a measure supported on a one dimensional set (corresponding to the edges in the image) and an L1 part (corresponding to smooth regions in the image). The L2 component v captures deviations from this model. There are many variants of the Mumford–Shah model. These are beau- tifully described in the lecture notes of Meyer [14] – a must read. We wish to pick up on only one point of Meyer’s exposition. Even when one settles on the functional nature of the two components u and v in the image, there are infinitely many ways to write f = u + v depending on how much energy one wishes to put in each of these components. This is completely analogous to K-functional decompositions used in proving theorems on interpolation of operators. One needs to look at this totality of all such decompositions to truly understand f. For example, consider the case where we simply look for decompositions of f = u + v where u ∈ BV and v ∈ L2. We can give a quantitative description of these decompositions through the K-functional K(f, t) := K(f, t; L2, BV) := inf f=u+v vL2 + t|u|BV, t 0, (1.1) where the | · |BV is the BV seminorm. For any fixed t 0, the optimal decomposition in (1.1) tries to balance the two terms. Thus for t small it puts more energy into the BV component and less into the L2 com- ponent. The rate of decrease in K(f, t) as t → 0 tells how nice f is with respect to this model. The role of the K-functional is to distinguish between images. Certainly some images are more complex than others and more apt to be more difficult to compress. The rate at which a K-functional tends to
- 19. 6 R. DeVore 0 as t → 0 measures this complexity. Thus, we can use the K-functional to separate images into classes Kα which are compact sets in our chosen metric. When classical metrics such as Lp norms are used, then these classes correspond to finite balls in smoothness spaces. In other words, using appropriate K-functionals, we can obtain a strata of image classes Kα reflecting the complexity of images. 1.2.3 Optimal encoding and Kolmogorov entropy Suppose now that we have decided on the metric to be used to measure the distortion between two images and suppose we also have our model classes Kα for the classification of images. We shall assume that the metric is given by a quasi-norm · := · X on a topological linear space X. Each of the sets K = Kα is assumed to be a compact subset in the topology given by · . Recall that an encoder E for K is a mapping that sends each f ∈ K into a bitstream B(f) := BE(f). Associated to E is a decoder D which takes any bitstream B and associates to it an element DB from X. Thus given f ∈ K, ¯ f := DEf = D(BE(f)) is the compressed image given by this encoding-decoding pair. This means that the distortion in the performance of this encoding on a given f is dE(f) := f − ¯ f = f − DEf. (1.2) Of course, we are interested in the performance of this encoding not on just one element f ∈ K but on the entire class. This leads us to define the distortion for the class K by dE(K) := sup f∈K dE(f). (1.3) This distortion also depends on the decoder which we do not indi- cate. (One could become more specific here by always choosing for the given encoder E and set K the best decoder in the sense of minimiz- ing the distortion (1.2).) To measure the complexity of the encoding we use #(E) := #(E(K)) := sup f∈K #(B(f)) (1.4) which is the maximum number of bits that E assigns to any of the elements of K. We are interested in a competition among encoders/decoders to deter- mine the optimal possible encoding of these classes. Suppose that we are
- 20. 1. Some Fundamental Issues 7 given a bit budget n; this means we are willing to allocate a maximum of n bits in the encoding of any of the elements of K. Then, dn(K) := inf #(E)≤n dE(K) (1.5) is the minimal distortion that can be obtained for the class K with this bit budget n. There is a mathematical description, called Kolmogorov entropy, that completely determines the optimal performance that is possible for an encoding of a given class K. Since K is compact in · , for any given there is a collection of balls B(fi, ), i = 1, . . . , N, of radius centered at fi ∈ X, such that K ⊂ N i=1 B(fi, ). (1.6) The smallest number N(K) of balls that provide such a cover is called the covering number of K. The Kolmogorov entropy of K (in the topol- ogy of X) is then given by H(K) := log N(K) (1.7) where here and later log always refers to the logarithm to the base 2. We fix K and think of H(K) is a function of . It gives a measure of the massivity of K. The slower H(K) tends to infinity as → 0 the more thin is the set K. We can reverse the roles of and the entropy H(K). Namely, given a positive integer n, let n(K) := inf{ : H(K) ≤ n}. (1.8) The n(K) are called the entropy numbers of K; they tend to zero as n → ∞. The faster they tend to zero, the smaller the set K. Notice that an asymptotic behavior H(K) = O(−1/α ) is equivalent to n(K) = O(n−α ). The two notions of optimal distortion and entropy numbers are iden- tical: n(K) = dn(K). (1.9) The proof is easy. Suppose E is an optimal encoder using n bits (if no such optimal encoder exists one modifies the following argument slightly). For each bitstream B = B(f), f ∈ K, let fB := D(B) which is an element of X. Then, taking = dn(K), we have f ∈ B(fB, ). Since
- 21. 8 R. DeVore there are at most 2n distinct bitstreams B(f), f ∈ K, we obtain that H ≤ n and hence n(K) ≤ = dn(K). We can reverse this inequality as follows. Suppose that n is given and = n(K). We assume that H(K) ≤ n. (We actually only know Hρ(K) ≤ n for all ρ so that our assumption is not necessarily valid but we can easily modify the argument when n(K) is not attained.) Let B(fi, ), i = 1, . . . , H(K), be a minimal covering for K with balls of radius . We associate to each i the binary bits in the binary representation of i. We define the encoder E as follows. If f ∈ K, we choose a ball B(fi, ) that contains f (this is possible because these balls cover K) and we assign to f the bits in the binary representation of i. The decoder takes the bitstream, calculates the integer i which has these bits in its binary expansion, and assigns the decoded element to be the center of the ball Bi. This encoding has distortion ≤ = n(K) and so we have dn(K) ≤ n(K). The above discussion shows that the construction of an optimal en- coder with distortion for the set K is the same as finding a minimal covering for K by balls of radius . Unfortunately, such coverings are usually impossible to find. For this reason, and others illuminated be- low, this approach is not very practical for encoding. On the other hand, it gives us a benchmark for the performance of encoders. If we could find an encoder which is nearly optimal for all the classes K of interest to us, then we could rest assured that we have done the job in the context in which we have framed the problem. We shall discuss in the next section how one could construct an encoder with these properties for a large collection of compact sets in standard metrics like Lp, 1 ≤ p ≤ ∞. 1.2.4 Wavelet bases and compact subsets of Lp A set K is compact in Lp provided that the modulus of smoothness ω(f, t)p := sup |h|≤t ∆h(f, ·)Lp(Ω), t 0 (1.10) for all of the elements f ∈ K have a continuous majorant ωK sup f∈K ω(f, t)p ≤ ωK(t) (1.11) where ωK(0) = 0. The rate at which ωK tends to zero at 0 measures the compactness of K. Thus the natural compact sets in Lp are described by common smoothness conditions. This leads to the Sobolev and Besov smoothness spaces. For example, the Besov spaces are defined by con- ditions on the higher order moduli of smoothness of f ∈ Lp. We denote
- 22. 1. Some Fundamental Issues 9 these Besov spaces by Bs q (Lp(Ω)) where p is the Lp space in which we are measuring smoothness. The parameter s 0 gives the order of smooth- ness much like the number of derivatives. The parameter 0 q ≤ ∞ is a fine tuning parameter which makes subtle distinctions between these spaces. We do not make a precise description of these spaces at this juncture but we shall give a description of these spaces in a moment using wavelet bases. The reader is probably by now quite familiar with wavelet bases. We shall limit ourselves to a few remarks which will serve to describe our notation. When working on the domain R, a wavelet basis is given by the shifted dilates ψλ := ψ(2j · −k), λ = (j, k), of one fixed function ψ. When moving to Rd , one needs the shifted dilates of a collection ψe of 2d − 1 functions; the parameter e is usually indexed on the set E of nonzero vertices of the unit cube [0, 1]d . Thus the wavelets are indexed by three parameters λ = (j, k, e) indicated frequency (j), location (k) and type (e). When working on a finite domain, two adjustments need to be made. The first is that the range of j is from j0 ≤ j ∞. The coarsest level j = j0 corresponds to scaling functions; all other j correspond to the actual wavelets. For notational convenience, we shall take j0 = 0 in what follows. The second adjustment is that near the boundary some massaging has to be made in defining ψλ. Thus, a wavelet basis on a finite domain Ω in Rd is a collection Ψ = {ψλ : λ ∈ J } of functions ψλ. The indices λ encode scale, spatial location and the type of the wavelet ψλ. We will denote by |λ| the scale associated with ψλ. We shall only consider compactly supported wavelets, i.e., the supports of the wavelets scale as follows Sλ := supp ψλ, c02−|λ| ≤ diam Sλ ≤ C02−|λ| , (1.12) with c0 and C0 0 absolute constants. The index set J has the follow- ing structure J = Jφ ∪ Jψ where Jφ is finite and indexes the scaling functions on the fixed coarsest level 0. Jψ indexes the “true wavelets” ψλ with |λ| 0. From compactness of the supports we know that at each level, the set Jj := {λ ∈ J : |λ| = j} is finite. In fact, one has #Jj ∼ 2jd with constants depending on the underlying domain. There is a natural tree structure associated to wavelet bases. A node in this tree corresponds to all λ = (j, k, e), e ∈ E, with j, k fixed. In the case the domain is Rd , each such node has 2d children corresponding to the indices (j + 1, 2(k + e)) where e ∈ {0, 1}d . In other words, the children all occur on the next dyadic level. In the case of Haar functions, the supports of the wavelets corresponding to children are contained
- 23. 10 R. DeVore in those corresponding to a given parent. This is modified on domains because only some of the indices are used on the domain. Wavelet bases have many remarkable properties. The first that we want to pick up on is that Ψ is an unconditional basis for many function spaces X. Consider first the case that X = L2(Ω). Then every f ∈ L2(Ω) has a unique expansion f = fλψλ and there exist some constants c and C independent of f such that c(fλ)λ∈J 2 ≤ λ∈J fλψλL2(Ω) ≤ C(fλ)λ∈J 2 . (1.13) In the case of Lp spaces, p = 2, the norm fLp(Ω) is not so direct and must be made through the square function. However, if we normalize the basis in Lp, ψλLp(Ω) = 1, then the space Bp of functions f = λ∈J fλψλ satisfying fBp := (fλ)p (1.14) is very close to Lp(Ω) and can be used as a poor man’s substitute in many instances. By the way, Bp is an example of a Besov space Bp = B0 p(Lp) where the smoothness order is zero. Besov spaces in general have a simple description in terms of wavelet coefficients. If f = λ∈J fλψλ with the ψλ normalized in Lp, ψλLp(Ω) = 1, then hBs q (Lp(Ω)) := ∞ j=0 2jsq |λ|=j |fλ|p q/p 1/q , 0 q ∞, supj≥0 2js |λ|=j |fλ|p 1/p , q = ∞. (1.15) Suppose that we fix 1 ≤ p ≤ ∞ and agree to measure the distortion of images in the Lp(Ω) norm. Which of the Besov spaces are embedded in Lp(Ω) and which are compactly embedded? This is easily answered by the Sobolev embedding theorem. The unit ball of the Besov space Bs q (Lτ (Ω)) is a compact subset if and only if 1 τ s d − 1 p . Notice that this condition does not depend on q. When 1 τ = s d − 1 p (the so-called critical line in the Sobolev embedding) then the Besov space Bs q (Lτ (Ω)) is embedded in Lp(Ω) for small enough q but these embeddings are not compact.
- 24. 1. Some Fundamental Issues 11 1.2.5 Near optimal encoding in Lp, 1 ≤ p ≤ ∞ Let us fix the metric of interest to us to be one of the Lp norms with 1 ≤ p ≤ ∞. If we derive metrics that seem better measures of distortion for images then we would try to repeat the exercise of this section for these metrics. We seek an encoder E with the following properties. First the encoder should be applicable to any function in Lp; such an encoder is said to be universal. We would like the encoder to give an infinite bitstream; giving more bits from the infinite bitsream would give a finer resolution of f. Such an encoder is called progressive. Finally, we would like the encoder to be near optimal for the compact sets that are unit balls of the Besov spaces in the following sense. If Enf denotes the first n bits of Ef, then we would like that for each such compact set K we have DnEnf − fLp(Ω) ≤ CKn(K), f ∈ K, n = 1, 2, . . . , (1.16) with CK a constant depending only on K. This means that for these compact sets the encoder performs (save for the constant CK) as well as any encoder. It is quite amazing that there is a simple construction of encoders with these three desirable properties using wavelet decompositions. To describe these encoders, it is useful to keep in mind the case when the metric is L2 and the wavelet basis is an orthonormal system. The general case follows the same principles but the proofs are not as transparent. Suppose then that X = L2(Ω) and f ∈ X with fX ≤ 1 has an orthogonal wavelet expansion f = λ∈J fλψλ. To start the encoding, we would like to choose a few terms from this wavelet expansion which best represent f. The best choice we can make is to choose the largest terms since the remainder is always the sum of squares of the remaining coefficients. Suppose η 0 and Λη := Λη(f) := {λ : |fλ| ≥ η} is the set of coeffi- cients obtained by thresholding the wavelet coefficients at the threshold η. Note that Λη = ∅ when η 1. We would like to encode the infor- mation about the set Λη and the coefficients fλ, λ ∈ Λη. There are two issues to overcome. The first is that it is necessary to encode the posi- tions of the indices in Λη. At first glance, these positions could occur anywhere which would cost possibly an arbitrarily large bit budget to encode them. But it turns out that for the sets K on which we want optimality of the encoding, namely K a unit ball of a Besov space, these positions align themselves at low frequencies. In fact, it can be proved [5] that whenever f ∈ K one can find a tree T which contains Λη and is of
- 25. 12 R. DeVore comparable size to Λη. This motivates us to define for each f ∈ L2(Ω), Tη = Tη(f) as the smallest tree which contains Λη(f). What we gain in going to a tree structure (which could be avoided at the expense of more complications and less elegance) is that it is easy to encode the positions of a tree using at most 2#(Tη) bits. Indeed, one simply encodes the tree from its roots by assigning a bit 1 if the child of a current member λ of Tη is in Tη and zero otherwise; see [6, 5] for details. The second issue to overcome is how to encode the coefficients fλ for λ ∈ Tη. To encode just one real number we need an infinite number of bits. The way around this is the idea of quantization. In the present context, the (scalar) quantization is easy. Any real number y with |y| ≤ 1 has a binary representation y = (−1)s(y) ∞ i=0 bi(y)2−i (1.17) with the bi(y) ∈ {0, 1} and the sign bit s(y) defined as 0 if y 0 and 1 otherwise. Receiving the bits s(f), b0(f), . . . , bm(f), we can approximate y by the partial sum ȳ = (−1)s(y) m i=0 bi(y)2−i with accuracy |y − ȳ| ≤ 2−m . (1.18) We apply this quantization to the coefficients fλ, λ ∈ Tη. How should we choose m? Well in keeping with the strategy for thresholding, we would only want the residual y − ȳ to be under the threshold η. Thus, if η = 2−k , we would choose the quantization so that m = k. There is only one other thing to note before we define our encoding. If η is a current threshold level and η η is a new threshold then the tree Tη is a growing of the tree Tη. Keeping this in mind, let us take thresholds ηk = 2−k , k = 0, 1, . . . and obtain the corresponding trees Tk := Tηk , k = 0, 1, . . . . To each f ∈ L2(Ω), we assign the following bitstream: P0(f), S0(f), B0(f), P1(f), S1(f), B1(f), B1,0(f), . . . (1.19) Here, P0(f) denotes the bits needed to encode the positions in the tree T0 (f). The bits S0(f) are the sign bits of the coefficients corresponding to indices in T0 (f). The set B0(f) gives the first bits b0(fλ) of the coefficients fλ corresponding to the λ ∈ T0 . When we advance k to the value 1, we assign in P1 the bits needed to encode the new positions, i.e. the positions in T1 T0 . Then S1(f) are the sign bits for the coefficients corresponding to these new positions. The bits B1(f) are the b1 bits (which correspond to 2−1 in the binary expansion) of the coefficients
- 26. 1. Some Fundamental Issues 13 corresponding to these new positions. Note that each new coefficient has absolute value ≤ 1/2 so the bit b0 = 0 for these coefficients. The set B1,0(f) gives the second bit (i.e. the b1 bit) in the binary expansion of the fλ, λ ∈ T0 . The reason we add these bits is so that each coefficient, whether it is from T0 or T1 is resolved to the same accuracy (i.e. accuracy 1/2 at this stage). The process continues as we increase k. We send position bits to identify the new positions, a sign bit and a lead bit for each coefficient corresponding to a new position, and then one bit from the binary expansion of all the old positions. We denote by E the mapping which takes a given f ∈ L2(Ω) into the bitstream (1.19). For each k ≥ 0, we let Ek be the encoder obtained from E by truncating at stage k. Thus Ek(f) is the finite bitstream P0(f), S0(f), B0(f), . . . , Pk(f), Sk(f), Bk(f), Bk,0(f), . . . , Bk,k−1(f) (1.20) Let us say a few words about the decoding of such a bitstream. When a receiver obtains a bitstream of the form (1.20), he knows the first bits will give the positions of a tree of wavelet indices. From the form of the tree encoding, he will know when the bits of P0 have ended; see [5]. At this stage he knows the number of elements in the tree T0 . He therefore knows the next #(T0 ) bits will give the signs of the corresponding coefficients and the following #(T0 ) bits will be the binary bits for position 0 in the binary expansion of each of these coefficients. Then, the process repeats itself at level 1. The encoder E has all of the properties we want. It is universal, i.e. defined for each f ∈ L2(Ω). It is progressive: as we receive more bits we get a finer resolution of f. Finally it is near optimal in the following sense. If K is the unit ball of any of the Besov spaces which are compactly embedded into L2(Ω), then for any k the encoder Ek is near optimal in the sense of (1.16). While we have discussed the encoder E in the context of measuring distortion in L2, the same ideas apply when distortion is measured in Lp for any 1 ≤ p ≤ ∞. The only alteration necessary is to work with the wavelet decomposition with wavelets normalized in Lp(Ω) which of course alters the coefficients as well. 1.3 Analog to Digital (A/D) conversion As we have previously observed, the digital format is preferred for rep- resenting signals because of its robustness. However, many signals are
- 27. 14 R. DeVore not digital but rather analog in nature; audio signals, for instance, cor- respond to functions f(t), modeling rapid pressure oscillations, which depend on the “continuous” time t (i.e. t ranges over R or an interval in R, and not over a discrete set), and the range of f typically also fills an interval in R. For this reason, the first step in any digital processing of such signals must consist in a conversion of the analog signal to the digital world, usually abbreviated as A/D conversion. Note that at the end of the chain, after the signal has been processed, stored, retrieved, transmitted, . . . , all in digital form, it needs to be reconverted to an analog signal that can be understood by a human hearing system; we thus need a D/A conversion there. There are many proposed algorithms for A/D conversion. As in our discussion the last section, we would like to understand how we could decide which of these algorithms is optimal for encoding/decoding. As in that case, we have two initial issues: determine the metric to be used to measure distortion and the class of signals that are to be encoded. The metric issue is quite similar to that for images except now the human visual system is replaced by the human auditory system. In fact such considerations definitely play a role in the design of good algorithms but as of yet we are aware of no mathematical metric which is used to model the auditory system in the mathematical analysis of the encoding problem. The two metrics usually utilized in distortion analysis are the L2(R) and L∞(R) norms. Concerning model classes for auditory signals, it is customary to model audio signals by bandlimited functions, i.e. functions f ∈ L2 (R) for which the Fourier transform ˆ f(ξ) = 1 √ 2π ∞ −∞ f(t)e−iξt dt vanishes outside an interval |ξ| ≤ Ω. The bandlimited model is justified by the observation that for the audio signals of interest to us, observed over realistic intervals time intervals [−T, T], χ|ξ|Ω(χ|t|≤T f)∧ 2 is neg- ligible compared with χ|ξ|≤Ω(χ|t|≤T f)∧ 2 for Ω 2π · 20, 000 Hz. So in proceeding further, let us agree that we shall use either the L2 or L∞ metric and the class of functions we shall consider are those that are bounded and bandlimited. We define S to be the collection of all functions in L2 ∩ L∞ whose L2 norm is ≤ 1 and L∞ norm is ≤ a 1 with a 0 fixed and whose Fourier transform vanishes outside of [−π, π]. The choice of a and π are arbitrary; indeed given any f ∈ L2 ∩L∞ which is bandlimited, we can dilate f and then multiply by a constant to arrive
- 28. 1. Some Fundamental Issues 15 at an element of S. Thus any encoders derived for S can easily be applied to general f. The class S has a lot of structure. There is a well-known sampling theorem that says that any function f ∈ S is completely determined by its values on Z. Indeed, we can recover f from the formula f(t) = n∈Z f(n) sin(t − n) (t − n) = n∈Z f(n)sinc(t − n) . (1.21) This formula is usually referred to as the Shannon-Whitaker formula. The sampling rate of 1 (called the Nyquist rate) arises because ˆ f vanishes outside of [−π, π]. The sinc functions appearing on the right side of (1.21) form an orthonormal system in L2. Changing the support interval from [−π, π] to [−Aπ, Aπ] would correspond to Nyquist sampling rate of 1/A. The proof of (1.21) is simple and instructive. We can write ˆ f = F · χ (1.22) where F := k∈Z f(·+2kπ) is the periodization of ˆ f and χ is the charac- teristic function of [−π, π]. The Fourier coefficients of F are F̂(n) = f(n) and so F = n∈Z f(n)einω . Substituting that into (1.22) and inverting the Fourier transform we obtain (1.21) because the inverse Fourier trans- form of χ is the sinc function. With the formula (1.21) in hand, it seems that our quest for an optimal encoder is a no brainer. We should simply quantize the Nyquist samples f(n). Given a real number y ∈ (−1, 1), we can write as before y = (−1)s(y) ∞ i=1 bi(y)2−i (1.23) where (−1)s(y) , s(y) ∈ {0, 1}, is the sign of y and the bi are the binary bits of |y|. If we pick a number m 0, the quantized values ȳ = (−1)s(y) m i=1 bi(y)2−i (1.24) can be described by m bits and |y − ȳ| ≤ 2−m . If we apply this to the samples f(n), n ∈ Z, we have an encoding of f that uses m bits per Nyquist sample. Encoders built on this simple idea are called Pulse Code Modulation (PCM). However, they are not the encoders of choice in A/D conversion. Our excursion into this topic in this section is to
- 29. 16 R. DeVore understand why this is the case. Can we explain mathematically why engineers do not prefer PCM and better yet to explain the advantages of what they do prefer. To begin the story, we have to dig a bit deeper into what we really mean by an encoding of a signal. The formula (1.21) requires an infinite number of samples to recover f and therefore apparently an infinite number of bits. Of course, we cannot compute, store, or transmit an infinite bitstream. But fortunately, we only want to recover f on a finite time interval which we shall take to be [0, T]. Even then, the contribution of samples far away from [0, T] is large. Indeed, if we incur a fixed error δ in representing each sample, then the total error on [0, T] is possibly infinite since the sinc functions decay so slowly: n∈Z |sinc(t−n)| = ∞. There is a way around this by sampling the function f at a slightly higher rate. Let λ 1 and let gλ be a C∞ function such that ĝλ = 1 on [−π, π] and ĝλ vanishes outside of [−λπ, λπ]. Returning to our derivation of (1.21), using ĝλ in place of χ, we obtain the representation f(t) = 1 λ n∈Z f n λ gλ t − n λ . (1.25) Because g is smooth with fast decay, this series now converges absolutely and uniformly; moreover if the f n λ is replaced by fn = f n λ + εn in (1.25), with |εn| ε, then the difference between the approximation f(x) and f(x) can be bounded uniformly: |f(t) − f(t)| ≤ ε 1 λ n∈Z g t − n λ ≤ εCg (1.26) where Cg = λ−1 g L1 + gL1 does not depend on T. Oversampling thus buys the freedom of using reconstruction formulas, like (1.25), that weigh the different samples in a much more localized way than (1.21) (only the f n λ with t − n λ “small” contribute significantly). In prac- tice, it is customary to sample audio signals at a rate that is about 10 or 20% higher than the Nyquist rate; for high quality audio, a traditional sampling rate is 44,000 Hz. One can show that the above idea of sampling slightly higher than the Nyquist rate and then quantizing the samples using binary expansion peforms at the Kolmogorov entropy rate for the class S (at least when T is large). This was first proved in [13] and later repeated in [10]. We spare the details and refer the reader to either of these two papers. So it seems now that the matter of encoding is closed by using such a modified PCM encoding. But I have surprising news: engineers do not prefer this
- 30. 1. Some Fundamental Issues 17 method in practice. In fact they prefer another class of encoders known as Sigma Delta Modulation which we shall describe below. The mystery we still want to uncover is why they prefer these methods. To explain the Sigma-Delta story, we return to the idea of oversam- pling which is at the heart of these encoders. We have seen the benefits of slight oversampling: sampling slightly higher than the Nyquist rate and giving several bits for each sample performs at Kolmogorov entropy rates. Sigma-Delta encoders go to the other extreme. They sample at a rate λ 1 which is very large but then allot only one bit to each sample. Thus, to each sample f(n λ ) they assign a single bit qλ n ∈ {−1, 1}. On the surface this is very counter intuitive. If we think of that one bit as giving an approximation to the sample f(n λ ) then we cannot do very well. In fact the best we can do is give the sign of the sample. But the Sigma- Delta encoders do not do this. Rather, they make their bit assignment to f(n λ ) based on the past samples f(m λ ), m n, and the bits that have already been assigned to them. Let us describe this in more detail by considering the simplest of these encoders. We introduce an auxiliary sequence (un)n∈Z (sometimes described as giving the “internal state” of the Sigma-Delta encoder) iteratively defined by un = un−1 + f n λ − qλ n qλ n = sign un−1 + f n λ , (1.27) and with an “initial condition” u0 = 0. The qλ n are the single bit we assign to each sample. In circuit implementation, the range of n in (1.27) is n ≥ 1. However, for theoretical reasons, we view (1.27) as defining the un and qn for all n. At first glance, this means the un are defined implicitly for n 0. However, it is possible to write un and qn directly in terms of un+1 and fn+1 when n 0; see [9]. The role of the auxiliary sequence (un) is to track the difference be- tween the running sums of the f(n λ ) and those of the qλ n. It is easy to see that the choice for qλ n used in (1.27) keeps the difference of these running sums to be ≤ 1. For this, one proves simply by induction that the |un| ≤ 1, for all n ∈ Z. Of course, we need to describe how we decode the bit stream qλ n. For this we use (1.25) with f(n λ ) replaced by qλ n: ¯ f := 1 λ n∈Z qλ ngλ(n − n λ ). (1.28)
- 31. 18 R. DeVore At this point, we have no information about the accuracy at which ¯ f represents f. However, simple estimates are available using summation by parts. For any t ∈ R, we have f(t) − 1 λ n qλ ngλ t − n λ = 1 λ n f n λ − qλ n gλ t − n λ = 1 λ n un gλ t − n λ − gλ t − n + 1 λ ≤ 1 λ n gλ t − n λ − gλ t − n + 1 λ ≤ 1 λ n t− n λ t− n+1 λ |g λ(y)|dy = 1 λ g λL1 ≤ C λ . where we have used the fact that |un| ≤ 1, for all n ∈ Z. There is good news and bad news in the last estimate. The good news is that we see that ¯ f approximates f better and better as the sampling rate λ increases. The bad news is that this decay O(1 λ ) is far inferior to the exponential rate provided by PCM. Indeed, for an investment of m bits per Nyquist sample PCM provides distortion O(2−m ) whereas for an investment of λ bits per Nyquist sample, Sigma-Delta provides distortion O(1/λ). So at this stage, we still have no clue why Sigma-Delta Modulators are preferred in practice. It is possible to improve the rate distortion for Sigma-Delta encoding by using higher order methods. In [9], a family of such encoders were constructed. The encoder of order k is proven to give rate distortion O(λ−k ). If one allows k to depend on λ, one can derive error bounds of the order O(2−(log λ)2 ) – still far short of the exponential decay of PCM. The pursuit of higher performing encoders has led to a series of interesting questions concerning the optimal distortion possible using single bit encoders. The best known bound (e−.07λ ) for such encoders was given by Güntürk [12], Calderbank and Daubechies [4] show that it is not possible to obtain the rate (2−λ ) of PCM. In any case, none of these new methods are used in practical encoding and they do not explain the penchant for the classical Sigma-Delta methods. A couple of years ago, Ingrid Daubechies, Sinan Güntürk, and I were guests of Vinay Vaishampayan at Shannon Labs of ATT for a one
- 32. 1. Some Fundamental Issues 19 month think tank directed at understanding the preferences for Sigma- Delta Modulation. Shannon Labs is an oasis for Digital Signal Processing and its circuit implementation. We were fortunate to have many lunch with experts in Sigma-Delta methods asking them for their intuition why these methods are preferred in practice. This would be followed by an afternoon to put a mathematical justification behind their ideas. Usually, these exercises ended in futility but what became eventually clear is that the circuit implementation of Sigma-Delta Modulation is at the heart of the matter. The hardware implementation of encoders such as PCM or Sigma- Delta Modulation requires building circuits for the various mathematical operations and the application of quantizers Q such as the sign function used in (1.27). Our mathematical analysis thus far has assumed that these operations are made exactly. Of course, this is far from the case in circuit implementation. Let us suppose for example that the sign function used in (1.27) is replaced at iteration n by a non-ideal quantizer Qn(x) = sign(x) for |x| ≥ τ |Qn(x)| ≤ 1 for |x| τ. (1.29) Thus, the quantizer Qn gives the right value of sign when |x| ≥ τ but may not when |x| τ. Here τ would depend on the precision of the circuit which of course is related to the dollar investment we want to make in manufacturing such circuits. Note that we allow the quantization to vary with each application but require the overall precision τ. When we implement the Sigma-Delta recursion with this circuitry, we would not compute the qλ n of (1.27) but rather a new sequence q̄n given by ūn = ūn−1 + f n λ − q̄n q̄n = Qn ūn−1 + f n λ , (1.30) Thus, the error analysis given above is sort of irrelevant and needs to be replaced by one involving the q̄n. Recall that our analysis above rested on showing the state variables un are uniformly bounded. It turns out that in the scenario the new state variables ūn are then still bounded, uniformly, independently of the detailed behavior of Qn, as long as (1.29) is satisfied. Namely, we have Remark 1.1 Let f ∈ S, let ūn, q̄n be as defined in (1.30), and let Qn satisfy (1.29) for all n. Then |ūn| ≤ 1 + τ for all n ≥ 0. We refer the reader to [9] for the simple proof. Note that the remark holds regardless of how large τ is; even τ 1 is allowed.
- 33. 20 R. DeVore We now use the inaccurate bits q̄n to calculate ¯ f: The same summation by parts argument that derived (1.29) can be applied to derive the new error estimate: |f(t) − ¯ f(t)| ≤ (1 + τ)g λL1 λ . (1.31) Thus, except for the fact that the constants increase slightly, the bounds on the accuracy of the encoder does not change. The precision that can be attained is not limited by the circuit imperfection: by choos- ing λ sufficiently large, the approximation error can be made arbitrarily small. The same is definitely not true for the binary expansion-type schemes such as PCM. To see this, let us see how we quantize to obtain the binary bits of a number y ∈ (−1, 1). To find the sign bit of y, we use the quantizer Q as before. But to find the remaining bits, we would use Q1(z) := 0, z ≤ 1 1, z 1. (1.32) Once the sign bit b0 is found then, we define u1 := 2b0y = 2|y|. The bit b1 is given by b1 := Q1(u1). Then the remaining bits are computed recursively as follows: if ui and bi have been defined, we let ui+1 := 2(ui − bi) (1.33) and bi+1 := Q1(ui+1). (1.34) In circuit implementations, the quantization would not be exact. Sup- pose for example, we use an imprecise quantizer (1.29) to find the sign bit of y. Then, taking a y ∈ (−τ, τ), we may have the sign bit of y in- correct. Therefore, ȳ and y do not even agree in sign so |ȳ − y| could be as large as τ no matter how the remaining bits are computed. The mis- take made by the imperfect quantizer cannot be recovered by computing more bits, in contrast to the self-correcting property of the Sigma-Delta scheme. So there it is! This is a definite advantage in Sigma-Delta Modulation when compared with PCM. In order to obtain good precision overall with the binary quantizer, one must therefore impose very strict requirements on τ, which would make such quantizers very expensive in practice (or even impossible if τ is too small). On the other hand [10], Sigma-Delta encoders are robust under such imperfections of the quantizer, allowing for good precision even if cheap quantizers are used (corresponding to
- 34. 1. Some Fundamental Issues 21 less stringent restrictions on τ). It is our understanding that it is this feature that makes Sigma-Delta schemes so successful in practice. We have shown again where the understanding and formulation of fun- damental questions in computation is vital to understanding numerical methods. In this case of A/D conversion, it not only gives an understand- ing of the advantages of the current state of the art encoders, it also leads us to a myriad of questions at the heart of the matter [9, 10, 12]. We shall pick up on just one of these. We have seen that oversampling and one bit quantization allow error correction in the encoding but the bit distortion rate in the encoding is not very good. On the other hand, PCM has excellent distortion rate (exponential) but no error correction. It is natural to ask whether we can have the best of both worlds: exponential decay in distortion and error correction. The key to answering this question lies in the world of redundancy. We have seen the effect of the redundancy in the Sigma- Delta Modulation which allowed for error correction. It turns out that other types of redundancy can be utilized in PCM encoders. The essen- tial idea is to replace the binary representation of a real number y by a redundant representation. Let 1 β 2 and γ := 1/β. Then each y ∈ [0, 1] has a representation y = ∞ i=1 biγi (1.35) with bi ∈ {0, 1}. (1.36) In fact there are many such representations. The main observation that we shall utilize below is that no matter what bits bi, i = 1, . . . , m, have been assigned, then, as long as y − γm+1 1 − γ ≤ m i=1 biγi ≤ y, (1.37) there is a bit assignment (bk)km, which, when used with the previously assigned bits, will exactly recover y. We shall use this observation in an analogous fashion to the algorithm for finding the binary bits of real numbers, with the added feature of quantization error correction. These encoders have a certain offset parameter µ whose purpose is to make sure that even when there is an imprecise implementation of the encoder, the bits assigned will satisfy (1.37); as shown below,
- 35. 22 R. DeVore introducing µ corresponds to carrying out the decision to set a bit to 1 only when the input is well past its minimum threshold. We let Q1 be the quantizer of (1.32). The beta-encoder with offset µ. Let µ 0 and 1 β 2. For y ∈ [0, 1], we define u1 := βy and b1 := Q1(u1 −µ). In general, if ui and bi have been defined, we let ui+1 := β(ui − bi), bi+1 := Q1(ui+1 − µ). (1.38) It then follows that y − m i=1 biγi = y − m i=1 γi (ui − γui+1) = y − γu1 + γm+1 um+1 ≤ γm+1 ul∞ , (1.39) showing that we have exponential precision in our reconstruction, pro- vided the |ui| are uniformly bounded. It is easy to prove [10] that we do indeed have such a uniform bound. Let’s analyze the error correcting abilities of these encoders when the quantization is imprecise. Suppose that in place of the quantizer Q1, we use at each iteration in the beta-encoder the imprecise quantizer Q̃1(z) := 0, z ≤ 1 − τ 1, z 1 + τ ∈ {0, 1}, z ∈ (−τ, τ). (1.40) In place of the bits bi(y), we shall obtain inaccurate bits b̃i(y) which are defined recursively by ũ1 := βy, b̃1 := Q̃1(ũ1 − µ) and more generally, ũi+1 := β(ũi − b̃i), b̃i+1 := Q̃1(ũi+1 − µ). (1.41) Theorem 1.1 Let δ 0 and y ∈ [0, 1). Suppose that in the beta-encoding of y, the quantizer Q̃1 is used in place of Q1 at each occurrence, with the values of τ possibly varying but always satisfying |τ| ≤ δ. If µ ≥ δ and β satisfies 1 β ≤ 2 + µ + δ 1 + µ + δ , (1.42) then for each m ≥ 1, ỹm := m k=1 b̃kγk satisfies |y − ỹm| ≤ Cγm , m = 1, 2, . . . , (1.43) with C = 1 + µ + δ. The simple proof of this theorem is given in [10].
- 36. 1. Some Fundamental Issues 23 The beta encoder can be used in place of binary encoder in a PCM type encoding. We sample at slightly higher than Nyquist rate, i.e., λ is slightly larger than one. For each sample f(n/λ), we use the beta encoder to determine m bits in the beta expansion of this sample. The corresponding bitstream will therefore assign slightly more than m bits per Nyquist sample. Decoding these bits gives an approximation ¯ fn to f(n/λ). Even if the quantization is not exact, we will have the accuracy |f(n/λ) − ¯ fn| ≤ Cβm . Therefore, the signal ¯ f reconstructed using these bits will have exponen- tial accuracy as well: |f(t) − ¯ f(t)| ≤ Cβm , t ∈ R. (1.44) 1.4 Adaptive methods for PDEs The third and last topic we wish to engage concerns the numerical computation of solutions to PDEs. Given such an equation, how can we decide if a given numerical method is best possible? We shall see that there are three intertwining ingredients here: approximation the- ory, regularity theorems for PDEs, and analysis of the given numerical method. Any numerical method can be viewed as a form of approximation. In classical Finite Element Methods (FEM), the approximation tool is piecewise polynomials subordinate to partitions of the domain for the PDE. The role of approximation theory is to tell us what we can expect as a best performance using such an approximation tool. It assumes that the solution is known to us in its analysis of performance and therefore does not apply directly to our unknown solution. The usual form of an approximation theorem is to characterize precisely which functions are approximated with a specified order by the approximation tool. For example, if n is the number of parameters used in the approximation, then the typical theorem says that a function can be (best) approxi- mated with an error O(n−s ) if and only if f is in a certain smoothness space As . To use such approximation theorems as a gauge of the performance of the numerical method, we need to know in which smoothness space As the solution lies. That is, we want to know the largest s such that the solution u is in As . This is the role of regularity theorems for PDEs. The correct form for these theorems is a statement that says whenever the forcing data of the problem has certain properties then the solution lies in As for certain values of s.
- 37. 24 R. DeVore The approximation and regularity theory tells us the optimal perfor- mance we can expect for our numerical method. However, they do not give us an actual numerical method since they usually use full informa- tion about u which is not available to us. So the third leg is to construct numerical methods which perform at this optimal performance. This entails not only the construction of the numerical method but also a rigorous analysis to establish its convergent rates. We shall illustrate this trifecta by considering a very simple problem: the solution of Laplace’s equation on a polygonal domain Ω in R2 . Here we shall consider numerical methods of two distinct types. The first proceeds by specifying in advance the numerical method to be used. In the case of Finite Element Methods this means that a sequence (Pn) of triangulations of Ω are prescribed in advance. Typically, Pn+1 is a uniform refinement of Pn. The solution u of the PDE is approximated by a piecewise polynomial subordinate to the partition Pn. When higher accuracy in the approximation is needed then n is increased. There is a second class of numerical methods which does not set the full numerical scheme in advance but rather makes decisions on the run. In the case of Finite Element Methods, after an approximation to u has been made, the method examines the residual to decide what the new triangulation should be. Typically, some triangular cells are refined and others are not. These methods are called adaptive methods and we shall be interested in answering two question about their performance. The first is whether they provide any advantage over their non-adaptive counterparts? The second is how can we decide when an adaptive method is optimal? 1.4.1 Elliptic problems We shall restrict our discussion to the Poisson problem −∆u = f in Ω, u = 0 on ∂Ω, (1.45) where Ω is a polygonal domain in R2 and ∂Ω is its boundary. Our goal is to draw out the type of questions that should be asked when evaluating a numerical method for such elliptic problems. In order to be able to work with the least smoothness requirements on approximations to u it is best to formulate (1.45), not in classical terms, but rather in its weak formulation. For this we introduce the Sobolev space H1 0 (Ω) of functions which vanish on the boundary ∂Ω of Ω and have weak derivatives of first order in L2(Ω). The weak formulation of
- 38. 1. Some Fundamental Issues 25 (1.45) is to find u ∈ H1 0 (Ω) such that a(u, w) = (f, w), w ∈ H1 0 (Ω), (1.46) where a(y, w) := (∇y, ∇w), with (y, w) = (y, w)Ω := Ω ywdx, is a quadratic form on H1 0 (Ω). Here f can be any distribution in H−1 (Ω). We use the notation |||w|||2 := a(w, w) = ∇w2 L2(Ω) (1.47) to denote the energy norm which is the natural norm in which to measure the performance of numerical methods. By Poincaré’s inequality there exists a constant cΩ, depending on Ω, such that for any w ∈ H1 0 (Ω), cΩwH1(Ω) ≤ |||w||| ≤ wH1(Ω), (1.48) where w2 H1(Ω) = w2 L2(Ω) + ∇w2 L2(Ω). 1.4.2 Newest vertex subdivision The numerical methods we shall discuss are those which approximate the solution u of (1.46) by piecewise linear functions on triangular partitions of the polygonal domain Ω. The partitions are generated by refining triangles according to a fixed rule. While the following discussion applies to a variety of refinement rules we shall specify the method of subdivision to be newest vertex bisection since this will allow us a nice comparison when we discuss adaptive methods. The book of Verfürth [18] and the research article of Mitchell [15] describe this subdivision method and give some of its properties. The article [1] gives finer results on newest vertex bisection. Let P0 = {∆} be an initial partition of Ω into a set of triangular cells. To each edge associated to this partition we assign a label of 0 or 1. We do this labelling in such a way that exactly one edge of any triangle has a label 0 and all other edges have a label 1 (such a labelling is always possible although it is nontrivial to prove; see [1]). For any triangle ∆ of P0, the vertex v(∆) opposite the side of this triangle labelled as 0 is called its newest vertex. The edge in ∆ opposite to v(∆) will be denoted by E(∆). We have described how to assign to any triangle ∆ ∈ P0 a newest vertex. Other triangles will arise by subdivision. We now describe the rule for subdividing triangles and how to assign a newest vertex to each triangle that arises by subdivision. If ∆ is such a triangle and if v(∆) is
- 39. 26 R. DeVore the newest vertex that has already been assigned then the subdivision of ∆ consists of splitting ∆ into two new triangles by inserting the diagonal that connects the newest vertex to the bisecting point of the edge E(∆) opposite the newest vertex. Thus the cell produces two new cells and their newest vertex (assigned to each new triangular cell) is by definition the midpoint of E(∆). The partitions which arise when using newest vertex bisection satisfy a uniform minimal angle condition. This is established by showing that all triangles that arise in newest vertex bisection can be classified into a set of similarity classes depending only on the initial partition P0 (see Mitchell). We can represent newest vertex bisection by an infinite binary tree T∗ (which we call the master tree). The master tree T∗ consists of all triangular cells which can be obtained by a sequence of subdivisions. The roots of the master tree are the triangular cells in P0. When a cell ∆ is subdivided, it produces two new cells which are called the children of ∆ and ∆ is their parent. It is very important to note that, no matter how a cell arises in a subdivision process, its associated newest vertex is unique and only depends on the initial assignment of the newest vertices in P0. This means that the children of ∆ are uniquely determined and do not depend on how ∆ arose in the subdivision process, i.e., it does not depend on the preceding sequence of subdivisions. The reason for this is that any subdivision only assigns newest vertices for the new triangular cells produced by the subdivision and does not alter any previous assignment. It follows that T∗ is unique and does not depend at all on the order of subdivisions. The generation of a triangular cell ∆ is the number g(∆) of ancestors it has in the master tree. Thus cells in P0 have generation 0, their children have generation 1 and so on. The generation of a cell is also the number of subdivisions necessary to create this cell from its corresponding root cell in P0. There is a simple way to keep track of the newest vertices for triangular cells that arise in newest vertex bisection by giving a rule that labels any edges that arise from the subdivision process. There will be two main properties of this labelling. The first is that each triangular cell will have sides with labels (i, i, i − 1) for some positive integer i. The second is that the newest vertex for this cell will be the vertex opposite the side with lowest label. Certainly the edges in P0 have such a labelling as we have just shown. Suppose that a triangular cell ∆ ∈ P0 has sides which have been labelled (i, i, i−1) and the newest vertex for this cell is the one opposite
- 40. 1. Some Fundamental Issues 27 the side labelled i−1. When this cell is subdivided (using newest vertex bisection) the side labelled i − 1 is bisected and we label each of the two new sides i + 1. We also label the bisector by i + 1, i.e. the new edge connecting the newest vertex of ∆ with the midpoint of the edge E(∆) labelled by i − 1. Thus each new triangle now has sides labelled (i, i + 1, i + 1) with the newest vertex opposite the side with the lowest label. We note the important fact that if a cell has label (i + 1, i + 1, i) then it is of generation i (i.e. it has been obtained from a cell in P0 by i subdivisions). Therefore, specifying that the generation of the cell is i is the same as specifying that its label is (i + 1, i + 1, i). A subtree T ⊂ T∗ is a collection of triangular cells ∆ ∈ T∗ with the following two properties: (i) whenever ∆ ∈ T then its sibling is also in the tree; (ii) when ∆ ⊂ ∆ are both in the tree then each triangular cell ¯ ∆ ∈ T∗ with ∆ ⊂ ¯ ∆ ⊂ ∆ is also in T. The roots of T are all the cells ∆ ∈ T whose parents are not in T. We say that T is proper if it has the same roots as T∗, i.e., it contains all ∆ ∈ P0. If T ⊂ T∗ is a finite subtree, we say ∆ ∈ T is a leaf of T if T contains none of the children of ∆. We denote by L(T) the collection of all leaves of T. For a proper subtree T, we define N(T) to be the number of subdivisions made to produce T. Any partition P = Pn which is obtained by the application of an adaptive procedure based on newest vertex bisection can be associated to a proper subtree T = T(P) of T∗ consisting of all triangular cells that were created during the algorithm, i.e. all of the cells in P0, . . . , Pn. The set of leaves L(T) form the final partition P = Pn. We shall say that T = T(P) and P are admissible if P has no hanging nodes. We denote the class of all proper trees by T and all admissible trees by T a . We also let Tn be the set of all proper trees T with N(T) = n and by T a n the corresponding class of admissible trees from Tn. We denote by P the class of all partitions P that can be generated by newest vertex bisection and by Pa the set of all admissible partitions. Similarly, Pn and Pa n are the subclasses of those partitions that are obtained from P0 by using n subdivisions. There is a precise identification between Pn and Tn. Any P ∈ Pn can be given by a tree, i.e. P = P(T) for some T ∈ Tn. Conversely any T ∈ Tn determines a P = P(T) in Pn. The same can be said about admissible partitions and trees. 1.4.3 Standard finite element methods As mentioned before, in this type of numerical approximation, a sequence of partitions P0, . . . , Pn, . . . is set in advance. In our case of newest vertex
- 41. 28 R. DeVore bisection, we start with P0 and refine every triangular cell in P0 (using newest vertex bisection) to obtain P1, and continue in this way, refining every cell in a given Pn to obtain the next partition Pn+1. The partition Pn has the following properties. The triangular cells in Pn satisfy a minimal angle condition. Each angle is greater than a fixed positive constant c0 which is independent of n (it depends only on the initial partition P0). The partition Pn has no hanging nodes; this follows from the fact that every triangular cell is refined in moving from Pn to Pn+1. It follows that all triangular cells in Pn have comparable size. Their area is proportional to 2−n and their diameter is proportional to 2−n/2 . Let Sn := SPn be the space of continuous piecewise linear functions subordinate to the partition Pn which vanish on the boundary of Ω. These spaces are nested: Sn ⊂ Sn+1, n = 0, 1, . . . . To numerically solve (1.45) we consider the Galerkin approximation to u from Sn. It is deter- mined by the system of equations a(un, w) = (f, w), w ∈ Sn. (1.49) The solution un to this system minimizes the approximation in the en- ergy norm: |||u − un||| = inf S∈Sn |||u − S|||. (1.50) In other words, given that we decided to measure the error in the energy norm and given that we have decided to use elements from Sn for the approximation, there is no question that we have found in un the best approximation to u that we can possibly obtain. The role of approximation theory and regularity theory in this context is to explain what rate of approximation we can expect to obtain for a given right hand side f. The approximation theory says that we obtain an approximation rate |||u − un||| = O(2−ns/2 ), n = 1, 2, . . . , (1.51) if and only if u is in the Besov space Bs+1 ∞ (L2(Ω)). That is u should have s orders of smoothness more than the H1 smoothness. Given the solution u to (1.45), we define sL = sL(u) to be the maximum of all s 1 such that u ∈ Bs ∞(L2(Ω)). Here the subscript L indicates we are analyzing a linear approximation method; we will give a similar analysis for nonlinear (adaptive) methods. Regularity theorems would tell us sufficient conditions on the right hand sides f for u to have a given Besov smoothness. In other words, they would give information on sL(u). For example, if we know that
- 42. 1. Some Fundamental Issues 29 f ∈ L2(Ω) then u will at worst be in B 3/2 ∞ (L2(Ω). The worst behavior occurs for general Lipschitz domains. In our case of a polygonal domain the smoothness one can obtain depends on the angles corresponding to the boundary. In other words, for f ∈ L2(Ω), the best we can say is that sL ≥ 3/2. The point we wish to drive home here is that we know everything we could possibly want to know about our numerical method. We know it is the best method for approximating the solution by the approximation tools (piecewise linear functions) in the chosen metric (energy norm). We also know conditions on f sufficient to guarantee a given approximation rate and these sufficient conditions cannot be improved. 1.4.4 Adaptive finite element methods We want to contrast the satisfying situation of the previous section with that for adaptive methods. Such methods do not set down the partitions Pn in advance but rather generate them in an adaptive fashion that depends very much on the solution u and the previous computations. We begin this part of the discussion by giving the form of a typical adaptive algorithm. The starting point is, as before, the partition P0. Given that the par- tition Pk has already been constructed, the algorithm computes the Galerkin solution uk from SPk . It examines uk = uPk and makes a deci- sion whether or not to subdivide a given cell in Pk. In other words, the algorithm marks certain triangular cells ∆ ∈ Pk for subdivision. We shall denote by Mk the collection of marked cells in Pk. These marked cells are subdivided using the newest vertex bisection. This process, however, creates hanging nodes. To remove the hanging nodes, a certain collection M k of additional cells are subdivided. The result after these two sets of subdivisions is the partition Pk+1. The partitions Pn, n = 1, 2, . . . , have no hanging nodes and as noted earlier satisfy a minimal angle condition independent of n. One of the keys to the success of such adaptive algorithms is the mark- ing strategy, i.e. how to effectively choose which cells in Pk to subdivide. The idea is to choose only cells where the error u − uk is large (these would correspond to where the solution u is not smooth). Unfortunately, we do not know u (we are only seeing u through the computations uk) so we do not know this local error. One uses in the place of the actual local error what are called local error estimators. The subject of local error estimators is a long and important one that we shall not enter
- 43. 30 R. DeVore into in this discussion. The reader may consult the the paper of Morin, Nochetto, and Siebert [16] for a discussion relevant to the present prob- lem of Laplace’s equation in two space dimensions. We mention only one important fact. The local error estimators allow one to compute accu- rate bounds for the global error |||u − un||| from the knowledge of f and un. This is done through the residual ∆(u − un) = f − ∆(un). 1.4.5 Judging the performance of adaptive methods The main question we wish to address is how can we evaluate the per- formance of adaptive algorithms. Is there such a thing as an optimal or near optimal adaptive algorithm? This question is in the same spirit as the questions on image compression and A/D conversion. But there is a significant difference. In the other cases, the function to be en- coded/approximated was completely known to us. In the PDE case, we only know u through our numerical approximations. To begin the discussion, let us understand what approximants are available to us when we use adaptive methods. We have noted earlier that a partition P generated by an adaptive method can be associated to finite trees T(P). As before, given any admissible partition P, we define SP to be the space of piecewise linear functions which vanish on the boundary of Ω and are subordinate to P. For any function w ∈ H1 0 (Ω) and any such partition P, we define E(w, SP ) := inf S∈SP |||w − S||| = |||u − uP |||, (1.52) where uP is the Galerkin approximation to u. This is the smallest error we can achieve by approximating u in the energy norm by the elements of SP . How should we measure the complexity of an adaptively generated partition P. The most reasonable measure would be the number of com- putations used to create P. This turns out to be closely related to the number of subdivisions n = N(T(P)) used in the creation of P. As we shall see later, for certain specific adaptive algorithms, we can bound the number of computations used to create P and to compute uP by Cn. With these remarks in mind, we define σn(u) := inf P ∈Pn E(w, SP ) (1.53) which is the best error we can possibly achieve in approximating w by using n subdivisions. It is unreasonable to expect any adaptive algorithm
- 44. 1. Some Fundamental Issues 31 to perform exactly the same as σn(w). However, we may expect the same asymptotic behavior. To quantify this, we introduce for any s 0, the class As of functions w ∈ H1 0 (Ω) such that σn(w) ≤ Mn−s , n = 1, 2, . . . . (1.54) The smallest M for which (1.54) is satisfied is the norm in As : wAs := sup n≥0 ns σn(w). (1.55) We have a similar measure of approximation when we restrict ourselves to admissible partitions. Namely, σa n(w) := inf P ∈Pa n E(w, SP )H1(Ω) (1.56) now measures the best nonlinear approximation error obtained from ad- missible partitions and Ȧs := Ȧs (H1 0 (Ω)) consists of all w which satisfy σa n(w) ≤ Mn−s , n = 1, 2, . . . . (1.57) The smallest M for which (1.57) holds serves to define the norm wȦs . It is shown in [1] that the two spaces As are the same (see the discussion in §1.4.7); so we do not make a distinction between them in going further. 1.4.6 The role of regularity theorems It is of interest to know which functions are in As . In [2], it is shown that for a certain range of s (depending on the fact we are analyzing approximation by piecewise linear functions), we have As/2 ≈ Bs p(Lp(Ω)), 1 p = s − 1 2 + 1 2 . (1.58) The nebulous notation (≈) used in this context means that there are certain embeddings between Besov spaces and the space As . We do not want to get into the details (see [2]) but only work on this heuristic level. The spaces in (1.58) lie on the Sobolev embedding line for H1 (Ω) so they are very weak regularity conditions. We can obtain a priori information on the possible performance of adaptive methods by knowing regularity theorems for the above scale of Besov spaces. For each u, let sNL := sNL(u) be the maximum of all of the s such that u ∈ Bs p(Lp(Ω)), 1 p = s−1 2 + 1 2 . The larger the value of sNL, the better performance we can hope for an adaptive algorithm. In particular, we would like to know if sNL is larger than the corresponding
- 45. 32 R. DeVore value sL for standard Finite Element Methods as discussed in §1.4.3. We emphasize that the above scale of Besov spaces that occur in describing the approximation rates for adaptive approximation are quite different from the scale of Besov spaces appearing in the linear case of standard Finite Element Methods. For a given s 0 both require smoothness of order s but this smoothness is measured in different ways. In the standard case, the smoothness is relative to L2 but in the adaptive case the smoothness is relative to an Lp which depends on s and gets smaller as s gets larger. The Besov space in the adaptive case is much larger than in the standard case and therefore more likely to contain u. Let us take a simple example. We assume that the right side f is a function in L2(Ω) and we ask what is the approximation error we can expect to receive if we invest n triangular cells to the approximation. As we have already described in §1.4.3, from this assumption on f we can always conclude that f ∈ Bs ∞(L2(Ω) for s = 3/2 and depending on the corners in the domain s could even be larger but never bigger than s = 2. Thus, 3/2 ≤ sL ≤ 2. This means that in general we cannot expect more than an approximation rate O(2 −n(sL−1) 2 ) when using standard finite element methods with 2n triangular cells; see (1.51). What is the situation for sNL? To determine sNL we need new reg- ularity theorems that infer regularity in the new scale of Besov spaces Bs p(Lp(Ω)), with 1 p = s−1 2 + 1 2 . Regularity theorems of this type were proved in [8, 7]. For example, in the case of f ∈ L2(Ω), it is shown that sNL ≥ 2 (see [8]). This means that for an investment of n triangular cells we should always obtain an error like O(n−1/2 ). This shows that potentially an adaptive method can perform much better than a stan- dard Finite Element Method. If the standard method gives an error using a partition with n triangular cells, we might be able to obtain an error as small as 2 using the same number of cells in an adaptive method. 1.4.7 The performance of adaptive methods The above analysis is all well and good but it provides us no information about specific numerical methods and does not even answer the question of whether such numerical methods converge. In fact, it was not until very recently that truly adaptive numerical methods were constructed and proven to converge (Dörfler [11], and Morin, Nochetto, Siebert [16]) and this convergence is only established for simple problems and special methods of subdivision. We are much more ambitious and would like
- 46. 1. Some Fundamental Issues 33 to construct adaptive methods which perform at the optimal theoretical approximation rate. Suppose we have an adaptive algorithm in hand that we have some- how constructed. Given an error tolerance 0, we run the adaptive algorithm until we achieve an error . We shall say that an algorithm is near optimal for s provided that whenever u ∈ As the number N of subdivisions used by the adaptive algorithm used to produce this error satisfies N ≤ C|u|As −1/s , 0 1. (1.59) This means that the adaptive algorithm performs at the optimal rate on the class As . The question of course is whether we can find an adaptive algorithm with this near optimal performance. An adaptive algorithm with near optimal performance for solving (1.45) was constructed in [1]. Not only does this algorithm have op- timality in terms of the number of subdivisions used but it is also shown that the number of computations N(comp) needed to find uPn satisfies N(comp) ≤ C−1/s , 0 1. (1.60) We shall not describe the algorithm of [1] in detail but instead give a brief overview of the algorithm and then pick up on details for the two core results that allow the proof of near optimality. The algorithm proceeds by setting error tolerances k := 2−k and generating partitions Pk, k = 1, 2, . . . , in which the error is guaranteed to be k. We stop at the first integer n where the error (which we can measure a posteriori) is ≤ . For sure, when 2−n ≤ (1.61) this is the case. But the error bound may actually be attained for an earlier value of k. The construction of Pk from Pk−1 consists of several subiterations taking an admissible partition Pk−1,j to a new admissible partition Pk−1,j+1. Here, Pk−1,0 = Pk−1 and Pk is then found by coarsening a partition Pk−1,K where K is a fixed integer. Each subiteration (send- ing Pk−1,j to Pk−1,j+1) consists of the general strategy of marking cells, subdividing, further markings and subdividing to remove hanging nodes. The markings are made using the local error estimators found in [16] and marking enough cells so as to capture a fraction of the global error (this is known as bulk chasing). It was shown in [16] that such an iteration
- 47. 34 R. DeVore reduces the error in the energy norm. The role of K (which is a fixed integer) is to guarantee that after the coarsening step we have |||u − uPk ||| ≤ 2−1 |||u − uPk−1 ||| ≤ k = 2−k . (1.62) Of course, (1.62) guarantees the algorithm converges but gives little information about convergence rates because we have said nothing about how many subdivisions were needed to construct Pk. However, a deeper analysis of the algorithm shows that it is near optimal, in the sense described above, for the range 0 s 1/2 (we cannot expect to exceed this range because we are using piecewise linear functions to approximate in the H1 (Ω) norm). The proof of near optimality rests on two fundamental results. The first of these concerns the number of subdivisions necessary to remove hanging nodes at a given iteration of the algorithm. It is necessary to have such an estimate to be sure that the number of triangular cells does not explode when the hanging nodes are removed. Suppose that we have a sequence of partitions P̃k, k = 1, . . . , m, where each P̃k+1 is obtained from P̃k by marking a set M̃k, subdividing these cells, and then apply a further set of markings M̃ k and subdivisions to remove hanging nodes. The fundamental result proved in [1] shows that #(P̃m) ≤ #(P̃0) + C2(#(M̃0) + · · · + #(M̃m−1)). (1.63) This means that the number of markings used to remove hanging nodes is bounded by a multiple of the number of original markings. This is used in the adaptive finite element method to show, among other things, that #(Pk) ≤ C#(Pk−1) (1.64) with C an absolute constant. The proof of (1.63) does not proceed, as one might expect, by an induction step which bound #(M̃ k) by C#(M̃k). Rather one needs to keep track of the entire history in the creation of new cells, tracing this history back to earlier cells that were subdivided; see [1] for details. The second core result that is needed to keep control on the number of triangular cells and arrive at an estimate like (1.16) is that of coars- ening. Without this step, we would not arrive (1.59). The coarsening step proceeds as follows. After constructing the set Pk,K, we compute a Galerkin approximation vk := uPk,K to u from SPk,K which satisfies |||u − vk||| ≤ γk+1 (1.65) with γ a small fixed positive constant. The choice of γ influences the
- 48. 1. Some Fundamental Issues 35 choice of K above. The function vk is known to us and the coarsening algorithm uses information about this function to create a smaller par- tition Pk+1 ⊂ Pk,K such that vk can be approximated by an element of SPk+1 to accuracy αk where again 0 α 1 is a fixed constant that is chosen appropriate to the discussion that follows. The property gained by the coarsening step is that Pk+1 satisfies #(Pk+1) ≤ C0|u|As −1/s k (1.66) with C0 an absolute constant. It is then shown that the Galerkin ap- proximation uk+1 = uPk+1 satisfies |||u − uk+1||| ≤ k 2 = k+1. (1.67) This estimate together with the control on the size of the partition Pk+1 given in (1.66) combine to show that the algorithm is near optimal. It is vital in the coarsening algorithm to keep a control on the number of computations needed to find Pk+1. This is in fact the deeper aspect of the coarsening algorithm. Normally to find a near optimal tree approx- imation with n nodes, one might expect to have to do exponential in n computations since there are that many trees with n nodes. It is shown in [3] that the number of computations can be limited to O(n) by using a clever adaptive strategy. References [1] Binev, P., Dahmen, W. and DeVore, R. (2002). Adaptive finite element methods with convergence rates, IMI Preprint Series 12 (University of South Carolina). [2] Binev, P., Dahmen, W., DeVore, R. and Petrushev, P. (2002). Approxi- mation Classes for Adaptive Methods, Serdica Math. J. 28, 391–416. [3] Binev, P. and DeVore, R. (2002). Fast computation in tree approximation, IMI Preprint Series 11 (University of South Carolina). [4] Calderbank, R. and Daubechies, I. (2002). The Pros and Cons of Democ- racy IEEE Transactions in Information Theory 48, 1721–1725. [5] Cohen, A., Dahmen, W., Daubechies, I. and DeVore, R. (2001). Tree approximation and encoding, ACHA 11, 192–226. [6] Cohen, A., Daubechies, I., Guleryuz, O. and Orchard, M. (2002). On the importance of combining non-linear approximation with coding strategies, IEEE Transactions in Information Theory 48, 1895–1921. [7] Dahlke, S. (1999). Besov regularity for elliptic boundary value problems on polygonal domains, Appl. Math. Lett. 12, 31–36. [8] Dahlke, S. and DeVore, R. (1997). Besov regularity for elliptic boundary value problems, Comm. Partial Differential Equations 22, 1–16.
- 49. 36 R. DeVore [9] Daubechies, I. and DeVore, R. Reconstructing a bandlimited function from very coarsely quantized data: A family of stable sigma-delta modu- lators of arbitrary order, Annals of Mathematics, to appear. [10] Daubechies, I., DeVore, R., Gunturk, C.S. and Vaishampayan, V. Exponential Precision in A/D Conversion with an Imperfect Quantizer, preprint. [11] Dörfler, W. (1996). A convergent adaptive algorithm for Poisson’s equa- tion, SIAM J. Numer. Anal. 33, 1106–1124. [12] Güntürk, S. One-bit Sigma-Delta quantization with exponential accuracy, preprint. [13] Kolmogorov, A.N. and Tikhomirov, V.M. (1959). -entropy and -capacity of sets in function spaces, Usephi 14, 3–86. (Also in AMS Translations, Series 2 17 (1961), 277–364). [14] Meyer, Y. (2001). Oscillating pattern in image processing and in some nonlinear evolution equations, The Fifteenth Dean Jaqueline B. Lewis Memorial Lectures. [15] Mitchell, W.F. (1989). A comparison of adaptive refinement techniques for elliptic problems, ACM Transaction on Math. Software 15, 326–347. [16] Morin, P., Nochetto, R. and Siebert, K. (2000). Data Oscillation and Convergence of Adaptive FEM, SIAM J. Numer. Anal. 38, 466–488. [17] Mumford, D. and Shah, J. (1985). Boundary detection by minimizing functionals, Proc. IEEE Conf. Comp. Vis Pattern Recognition. [18] Verfürth, R. (1996). A Review of A Posteriori Error Estimation and Adaptive Mesh-Refinement Techniques (Wiley-Teubner, Chichester).
- 50. 2 Jacobi Sets of Multiple Morse Functions Herbert Edelsbrunner Department of Computer Science and Mathematics Duke University, Durham, and Raindrop Geomagic Research Triangle Park North Carolina John Harer Department of Computer Science and Mathematics Duke University, Durham North Carolina Abstract The Jacobi set of two Morse functions defined on a common d-manifold is the set of critical points of the restrictions of one function to the level sets of the other function. Equivalently, it is the set of points where the gra- dients of the functions are parallel. For a generic pair of Morse functions, the Jacobi set is a smoothly embedded 1-manifold. We give a polynomial- time algorithm that computes the piecewise linear analog of the Jacobi set for functions specified at the vertices of a triangulation, and we gen- eralize all results to more than two but at most d Morse functions. 2.1 Introduction This paper is a mathematical and algorithmic study of multiple Morse functions, and in particular of their Jacobi sets. As we will see, this set is related to the Lagrange multiplier method in multi-variable calculus of which our algorithm may be viewed as a discrete analog. Motivation. Natural phenomena are frequently modeled using contin- uous functions, and having two or more such functions defined on the same domain is a fairly common scenario in the sciences. Consider for 0 Research of the first author is partially supported by NSF under grants EIA-99- 72879 and CCR-00-86013. Research of the second author is partially supported by NSF under grant DMS-01-07621. 37
- 51. 38 H. Edelsbrunner and J. Harer example oceanography where researchers study the distribution of vari- ous attributes of water, with the goal to shed light on the ocean dynamics and gain insight into global climate changes [4]. One such attribute is temperature, another is salinity, an important indicator of water density. The temperature distribution is often studied within a layer of constant salinity, because water tends to mix along but not between these layers. Mathematically, we may think of temperature and salinity as two con- tinuous functions on a common portion of three-dimensional space. A layer is determined by a level surface of the salinity function, and we are interested in the temperature function restricted to that surface. This is a continuous function on a two-dimensional domain, whose critical points are generically minima, saddles, and maxima. In this paper, we study the paths these critical points take when the salinity value varies. As it turns out, these paths are also the paths the critical points of the salinity function take if we restrict it to the level surfaces of the tem- perature function. More generally, we study the relationship between continuous functions defined on a common manifold by analyzing the critical points within level set restrictions. Sometimes it is useful to make up auxiliary functions to study the properties of given ones. Consider for example a function that varies with time, such as the gravitational potential generated by the sun, planets, and moons in our solar system [18]. At the critical points of that potential, the gravitational forces are at an equilibrium. The plan- ets and moons move relative to each other and the sun, which implies that the critical points move, appear, and disappear. To study such a time-varying function, we introduce another, whose value at any point in space-time is the time. The paths of the critical points of the gravita- tional potential are then the Jacobi set of the two functions defined on a common portion of space-time. Results. The main object of study in this paper is the Jacobi set J = J(f0, f1, . . . , fk) of k + 1 ≤ d Morse functions on a common d-manifold. By definition, this is the set of critical points of f0 restricted to the intersection of the level sets of f1 to fk. We observe that J is symmetric in the k + 1 functions because it is the set of points at which the k + 1 gradient vectors are linearly dependent. In the simplest non-trivial case, we have two Morse functions on a common 2-manifold. In this case, the Jacobi set J = J(f, g) = J(g, f) is generically a collection of pairwise disjoint smooth curves that are free of any self-intersections. Fig. 2.1 illustrates the concept for two Morse functions on the two-dimensional
- 52. 2. Jacobi Sets 39 Fig. 2.1. The two partially bold and partially dotted longitudinal circles form the Jacobi set of f, g : M → R, where M is the torus and f and g map a point x ∈ M to the Cartesian coordinates of its orthogonal projection on a plane parallel to the longitudes. torus. The Jacobi set of a generic collection of k + 1 Morse functions is a submanifold of dimension k, provided d 2k − 2. The first time this inequality fails is for d = k + 1 = 4. In this case, the Jacobi set is a 3-manifold except at a discrete set of points. We describe an algorithm that computes an approximation of the Jacobi set for k +1 piecewise linear functions that are interpolated from values given at the vertices. In the absence of smoothness, it simulates genericity and differentiability and computes J as a subcomplex of the triangulation. The algorithm is combinatorial (as opposed to numerical) in nature and reduces the computations to testing the criticality of the k-simplices in the triangulation. Whether or not a simplex is considered critical depends on its local topology. By using Betti numbers to express that topology, we get an algorithm that works for triangulated manifolds and runs in time that is polynomial in the number of simplices. Assuming the links have sizes bounded from above by a constant, the running time is proportional to the number of simplices. Related prior work. The work in this paper fits within the general area of singularities of smooth mappings, which was pioneered by Hassler Whitney about half a century ago; see e.g. [19]. In this body of work, the fold of a mapping from a d-dimensional manifold Md to a (k + 1)- dimensional manifold Nk+1 is the image of the set of points at which the matrix of partial derivatives has less than full rank. In this paper, we consider (k + 1)-tuple of Morse functions, which are special smooth mappings in which the range is the (k +1)-dimensional Euclidean space, Rk+1 . The fold of such a special mapping is the image of the Jacobi set. Our restriction to Euclidean space is deliberate as it furnishes the framework needed for our algorithm.
- 53. 40 H. Edelsbrunner and J. Harer Whitney considered the case of a mapping between surfaces and stud- ied mappings where d is small relative to k. A classic theorem in differ- ential topology is the Whitney Embedding Theorem which states that a closed, orientable manifold of dimension n can always be embedded in R2n . Thom extended the work of Whitney, studying the spaces of jets (two functions have the same k-jet at p if their partial derivatives of order k or less are equal). The Thom Transversality Theorem, together with the Thom-Boardman stratification of C∞ functions on M, give characterizations of the singularities of generic functions [11]. Mather studied singularities from a more algebro-geometric point of view and proved the equivalence of several definitions of stability for a map (a concept not used in this work). In particular he provided the appropri- ate framework to reconstruct a map from its restrictions to the strata of the Thom-Boardman stratification. In a completely different context, Nicola Wolpert used Jacobi curves to develop exact and efficient algorithms for intersecting quadratic sur- faces in R3 [20]. She does so by reducing the problem to computing the arrangement of the intersection curves projected to R2 . Any such curve can be written as the zero set of a smooth function R2 → R, and any pair defines another curve, namely the Jacobi set of the two functions. Wolpert refers to them as Jacobi curves and uses them to resolve tan- gential and almost tangential intersections. To be consistent with her terminology, we decided to modify that name and to refer to our more general concept as Jacobi sets. Outline. §2.2 reviews background material from differential and combi- natorial topology. §2.3 introduces the Jacobi set of two Morse functions. §2.4 describes an algorithm that computes this set for piecewise linear data. §2.5 generalizes the results to three or more Morse functions. §2.6 discusses a small selection of applications. §2.7 concludes the paper. 2.2 Background This paper contains results for smooth and for piecewise linear functions. We need background from Morse theory [15, 16] for smooth functions and from combinatorial and algebraic topology [1, 17] for designing an algorithm that works on piecewise linear data. Morse functions. Let M be a smooth and compact d-manifold without boundary. The differential of a smooth map f : M → R at a point x of
- 54. 2. Jacobi Sets 41 the manifold is a linear map dfx : TMx → R mapping the tangent space at x to R. A point x ∈ M is critical if dfx is the zero map; otherwise, it is regular. Let , be a Riemannian metric, i.e. an inner product in the tangent spaces that varies smoothly on M. Since each vector in TMx is the tangent vector to a curve c in M through x, the gradient ∇f can be defined by the formula ∂c/∂t, ∇f = ∂(f ◦ c)/∂t, for every c. It is always possible to choose coordinates xi so that the tangent vectors ∂ ∂xi (x) are orthonormal with respect to the Riemannian metric. For such coordinates, the gradient is given by the familiar formula ∇f(x) = ∂f ∂x1 (x), ∂f ∂x2 (x), . . . , ∂f ∂xd (x) . We compute in local coordinates the Hessian of f: Hf (x) = ∂2 f ∂x2 1 (x) . . . ∂2 f ∂xd∂x1 (x) . . . ... . . . ∂2 f ∂x1∂xd (x) . . . ∂2 f ∂x2 d (x) , which is a symmetric bi-linear form on the tangent space TMx. A critical point p is non-degenerate if the Hessian is non-singular at p. The Morse Lemma states that near a non-degenerate critical point p, it is possible to choose local coordinates so that the function takes the form f(x1, . . . , xd) = f(p) ± x2 1 ± . . . ± x2 d. The number of minus signs is the index of p; it equals the number of negative eigenvalues of Hf (p). The existence of these local coordinates implies that non-degenerate critical points are isolated. The function f is a called a Morse function if (i) all its critical points are non-degenerate, and (ii) f(p) = f(q) for all critical points p = q. Transversality and stratification. Let F : X → Y be a smooth map between two manifolds, and let U ⊆ Y be a smooth submanifold. The map F is transversal to U if for every y ∈ U and every x ∈ F−1 (y), we have dFx(TXx)+TUy = TYy. In words, the basis vectors of the image of the tangent space of X at x under the derivative together with the basis vectors of the tangent space of U at y = F(x) span the tangent space of Y at y. The Transversality Theorem of differential topology asserts that if F is transversal to U then F−1 (U) is a smooth submanifold of X and the co-dimension of F−1 (U) in X is the same as that of U in Y [12].
- 55. Other documents randomly have different content
- 56. PART II
- 57. I—THE PEACE CONFERENCE AT WORK A Vivid Account from the Inside of the Machinery Which Produced the Peace Treaty. How the Crises with Japan, Italy and Belgium Were Averted By THOMAS W. LAMONT Financial and Economic Adviser at Paris to the American Commission to Negotiate Peace When we finally gain an historic perspective of the work of the Peace Conference we shall realize that, instead of being unduly delayed, it was accomplished in an astonishingly brief period. The Treaty of Vienna, back in 1815, took eleven months, and the factors to be dealt with were nothing like so numerous nor so complex. The Paris Conference occupied only about six months, and the earlier weeks were largely given over to questions relating to the renewal of the Armistice, rather than to the actual framing of the Peace Treaty. The Treaty text itself—aside from the League of Nations Covenant—was whipped through in a little over three months; for the active work of the Commissions which were to draft the various chapters did not get under way until February 1st; and the Treaty was presented to the German delegates at Versailles on May 7th. COVENANTS OPENLY ARRIVED AT No adequate history of the Peace Conference can be written until years have elapsed— until it is possible, as it is not now possible, to make public a multitude of intimate details. Hundreds of important documents were woven into the completed text of the Treaty. Such documents must eventually be made available to the chroniclers of history, who must finally have access to the official records, so that in course of time they can acquaint the world with the details of those momentous conferences which were held
- 58. among the Chiefs of State, where the ultimate decisions settling every important question were made. There have been complaints that the covenants of the Treaty were not as President Wilson had promised, openly arrived at. In point of fact, as far as lay within the bounds of possibility, the covenants of the Treaty were openly arrived at, inasmuch as their essence was made public just as soon as an understanding upon them had been reached, and in many cases, long before the final agreement. Nothing was held back which the public had any legitimate interest in knowing. It would, of course, have been quite out of the question for the Chiefs of State to discuss in public all the highly delicate and complex situations which were bound to, and which did, arise at Paris. Every man of strong character and powerful conviction has a view of his own upon any given subject, and naturally maintains that view with vigor and tenacity—even at times, if he be bitterly opposed—with acrimony. To take a familiar instance, it is an open secret that M. Clemenceau's first solution of the question of the Saar Basin did not at all suit President Wilson. Not unnaturally, M. Clemenceau simply wanted in effect to annex the Saar Basin, on the grounds that the Germans had destroyed the coal mines of Northern France. Mr. Wilson was in entire accord—to this extent, that France should, until her coal mines had been repaired, enjoy the entire output of the Saar coal fields; but to have France permanently annex the Basin was contrary to his profoundest convictions, as expressed in the well-known Fourteen Points. In the course of the discussion between M. Clemenceau and Mr. Wilson, their ideas at the start being so divergent, vigorous views were undoubtedly expressed; quite possibly tart language was used, at any rate by the French Premier, who was feeling all the distress of German frightfulness and war weariness. But to what possible good end could the detail of such intimate conversations have been made public? I allude to the possible conversations on the Saar Basin not as an historical fact, but as an example of what might have taken place, and very likely did take place; and if such temporary disagreements existed on that question, undoubtedly, among so many Chiefs of State as were gathered together at Paris, they existed on others. But in all cases amicable and cordial agreements were finally reached. Whenever agreements were even in sight, the press was informed; so that, when the Treaty of Peace and the summary of it finally came out, there were no surprises for the public. Every covenant, every clause, had been already foreshadowed and accurately pictured. THE BIG THREE
- 59. Naturally, the question is often asked: Who were the peacemakers at Paris? Were they two or three powerful Chiefs of State? The answer is both Yes and No. The final decision on every important matter lay in the hands of the so-called Big Four, and after Premier Orlando's defection and return to Italy, it narrowed down to the Big Triumvirate, Messrs. Wilson, Lloyd George, and Clemenceau. Yet while they made the final decisions, these were almost invariably based upon reports and opinions expressed to this trio, or to the quartet, by their advisers and experts. The actual text of the Treaty was, of course, written by the technicians, and there is hardly a phrase in the whole of it that can claim as its original author any one of the Chiefs of State. In every true sense, then, the Treaty of Peace has been the product, not of three men, not even of three-score, nor of three hundred, but of thousands; for quite aside from the official delegations at Paris, which comprised several hundred persons, we must remember that the data and the various suggested solutions on most of the questions had been canvassed at home for each delegation by large groups of office and technical experts. Of course it sounds well to say that the Treaty was written by three men: the picture of those few Chiefs of State sitting in conference day after day is dramatic in the extreme. That is, I must confess, the picture which comes back oftenest to my mind. I see them today, as I saw them for months at Paris, sitting in that large but cosy salon in the house allotted to President Wilson on the Place des États Unis; for, by common consent, it was there that the Supreme Council finally held all its meetings. It is in that theatre, with the three or four Chiefs of State taking the leading rôles, that we saw the other characters in the great drama moving slowly on the stage, playing their parts, and then disappearing into the wings. Today it might be Paderewski, pleading with all his earnestness and sincerity, to have Danzig allotted to the sovereignty of Poland. To-morrow it might be Hymans, the Belgian Secretary for Foreign Affairs, begging that there should be a prompt realization of those pledges to Belgium, which Belgium felt had been made by all the Allies; or it might even be word brought by special aeroplane from the King of the Belgians at Brussels, with fresh and important instructions to his delegation in the matter of Reparation. Or it might be a group of the representatives of those newer nationalities, Czechoslovakia, Rumania, Jugoslavia, arguing some burning question of boundary rights. Or it might be the British shipping experts, maintaining that the captured German ships should be restored to the various Allies upon a basis dividing the ships pro rata to the losses sustained by submarines, and contending against the American claim that the United States should have all the German ships finding lodgment in American harbors. Or it might be Herbert Hoover, that brilliant American, come to describe to the Big Four starvation conditions in Vienna, and to emphasize his belief that, enemy or no enemy, those conditions must be relieved or Bolshevism would march into Austria and directly on west until it reached France—and beyond.
- 60. THE PLACE OF MEETING The stage for this world drama was originally set at the Ministry of War, behind the Chamber of Deputies and across the Seine; and here Premier Clemenceau—who, it will be remembered, was Minister of War as well as President of the Council of French Ministers—was the presiding genius. But eventually, as the result of an interesting trend of circumstances, the all important conferences took place at President Wilson's house. Copyright Walter Adams Sons David Lloyd George Ray Standard Baker, who attended the Peace Conference, wrote in his book, What Wilson Did at Paris: Lloyd George personally was one of the most charming and amiable figures at Paris, full of Celtic quicksilver, a torrential talker in the conference, but no one was ever quite sure, having heard him express an unalterable determination on one day, that he would not be unalterably determined some other way the day following. Click for a larger image.
- 61. The original theatre of operations at the War Ministry had been so large, and there was such an enormous chorus brought into play, that progress was interminably slow. There were usually present all five of the plenipotentiaries of each of the five great powers, including Japan, and very frequently Marshal Foch as well. His presence automatically commanded the attendance of the chief military experts of the other delegates. With the innumerable secretaries who had to attend the plenipotentiaries, with the interpreters and whatnot, the Supreme Council came to look like a legislative chamber, in the midst of which sat Clemenceau, presiding with his usual incisiveness. At such meetings progress could be made only upon rather formal matters which had been threshed out beforehand. When it came to a point of great delicacy, where the discussions could be only on a most intimate basis, it became quite impossible to carry on. Nobody would feel like speaking out in meeting and calling the other fellow names—as was necessary at times in order to clear the atmosphere—if there were half a hundred other people around, to hear those names, and promptly to babble them to an expectant throng outside. So finally the Supreme Council was boiled down to the four Chiefs of State, including Japan's representative on any questions not strictly confined to Western Europe; and the small Council began to meet alternately at Clemenceau's office in the War Ministry, at Mr. Lloyd George's house, and at Mr. Wilson's, which was just around the corner from the British Premier's. Then in March, shortly after President Wilson's second coming from the United States, he fell ill with the grippe. After a rather severe attack he was able to get on his feet again and to do business, but was warned by his vigilant friend and physician, Admiral Grayson, to keep within doors for a time. Mr. Lloyd George, M. Clemenceau, and Signor Orlando were glad to accommodate themselves to Mr. Wilson's necessities, and formed the habit of meeting regularly at his house. His large salon was much better adapted for these conferences than the room at Mr. Lloyd George's. So there it was they met during all the final weeks of the Conference, leading up to the very end. A DESCRIPTION OF THE COUNCIL CHAMBER In the middle of the salon, facing the row of windows looking out upon the Place, was a large yet most inviting fireplace. On the left of this, a little removed from it, President Wilson usually ensconced himself on a small sofa, where he made room for some one member of his delegation, whom, for the particular subject under discussion, he desired to have most available. On the other side of the fireplace sat Mr. Lloyd George in a rather high, old-fashioned chair of carved Italian maple, and at his left sat his experts. Opposite the fireplace, to the right of it, and about half-way across the room, sat M. Clemenceau, with such of his Ministers as he needed, and then between him and President Wilson was Signor Orlando with the Italians. This made a semi-circle around the fireplace, and
- 62. whenever Viscount Chinda of the Japanese delegation was present, the circle was usually enlarged so as to give him a seat in the middle of it. Behind this first semi-circle was a second one, made up of secretaries and various technical experts, but the conference was always a limited one, and was not allowed to grow so large as to become unwieldy. Directly in front of the fireplace, almost scorching his coat-tails, sat Professor Mantou, the official interpreter for the Big Four. Mantou is a Frenchman, Professor of French in the University of London, so he had a perfect mastery of both French and English, with a good working knowledge of Italian. Mantou was quite an extraordinary character, and the most vivid interpreter I have ever heard, or rather seen; for at times he entered into the spirit of the discussions more vigorously than the original actors. M. Clemenceau, for instance, might make a quiet, moderate statement, in French, of course; and when it became Mantou's time to interpret it into English, he would enliven and embellish it with his own unique gestures. The Secretary of the Council was Sir Maurice Hankey, a British Army officer of great skill and tact, who had a marvelous aptitude for keeping everything straight, for taking perfectly adequate, and yet not too voluminous minutes, for seeing that no topic was left in the air without further reference, and in the last analysis, for holding the Chiefs of State with their noses to the grindstone. He knew French and Italian well, and was a distinct asset to the Council. I note that, in the honors and money-grants disbursed by Parliament to Marshal Haig, Admiral Beatty and others, Hankey received £25,000. Everybody who worked with him at Paris will be glad of this just recognition. I have described this Council Chamber in the President's house rather minutely because, as I have said, it formed the stage for all of the momentous decisions which went to make up the final peace settlement. At these conferences there was no formal presiding officer, but to President Wilson was usually accorded the courtesy of acting as moderator. HOW THE TREATY WAS COMPOUNDED What, then, is the Treaty? The answer is that it is a human document, a compound of all the qualities possessed by human beings at their best—and at their worst. People might expect a Treaty of Peace to be a formal, legal, mechanical sort of document; and undoubtedly an effort was made by some of the drafting lawyers, who bound all the different clauses together, to throw the Treaty into the mold of formality. But all the same, it is a compound quivering with human passion—virtue, entreaty, fear, sometimes rage, and above all, I believe, justice.
- 63. The reason fear enters into the Treaty must be manifest. Take, for instance, the case of France. France had lived under the German menace for half a century. Finally the sword of Damocles had fallen, and almost one-sixth of beautiful France had been laid waste. Her farms, her factories, her villages, had been destroyed; her women ravished and led captive; her children made homeless; her men folk killed. Do we realize that almost 60 per cent. of all the French soldiers under thirty-one years of age were killed in the war? Is it any wonder France could not believe that the German menace was gone forever, and that the world would never again allow German autocracy to overwhelm her? She could not believe it, and for that reason she felt it essential that the terms of the Treaty should be so severe as to leave Germany stripped for generations of any power to wage aggression against beautiful France. If her Allies pointed out that to cripple Germany economically was to make it impossible for Germany to repair the frightful damage she had wrought in France, France would in effect reply that this might be so, but never again could she endure such a menace as had threatened her eastern border for the previous half century. If certain of the Treaty clauses appear to some minds as unduly severe, it must be remembered that the Allies, little more than France, could bear the thought of letting Germany off so easily that within a few years she might again prepare for war. There was fear, too, on the part of those new nations, which had been largely split off from the effete and outworn Austro-Hungarian Empire, that in some way their ancient oppressors would once more gain sway over them. And, every nation, great and small, was overshadowed with the constant terror of Bolshevism,—that dread specter which seemed to be stalking, with long strides, from eastern Europe west towards the Atlantic. Unless peace were hastened that evil might overtake all the Allies. Such apprehensions as these, far more than imperial ambition or greed, were factors in the Treaty decisions. Judgments that might take many months in the ripening could not with safety be awaited. THE PROTECTION DEMANDED BY FRANCE France, I say, was thoroughly shocked at the frightful fate which had come upon so great a portion of her land and population. She seemed to have real fear that out of the ground, or from the sky, or from the waters of the earth, at the waving of the devil's wand, there would spring into being a fresh German army, ready to overwhelm her. It was this fear that led France to ask for a special Treaty by which England and America would pledge themselves to come to her aid in case of Germany's unprovoked attack against her. Those Americans who object to this have no conception of the real terror in France which led her to entreat her two most powerful Allies to make such a special treaty with her. France maintained, and with some reason, that during the formative
- 64. period of the League of Nations, before it might become an effective instrument, if she did not have the psychological and practical protection of England and America, she must look to her own defense, and the only real defense she could conceive was to make the Rhine her eastern boundary. This suggestion of Marshal Foch, based upon sound military concept, was rejected by President Wilson and Mr. Lloyd George on the theory that it would mean the annexation of German territory, would change Germany's ancient boundary line of the Rhine, and inevitably lead to future trouble. Copyright Underwood Underwood President Poincaré With the Swiss President, M. Gustave Ador, Driving to the Peace Conference in Paris. Click for a larger image. Very well, in effect answered M. Clemenceau, we see your point, but if you will not allow us to fix this natural boundary for defense, then we must beg you to guarantee us by treaty your coöperation against German aggression. That coöperation you will never be called upon to render with military force, because if Germany knows you are pledged to come to our defense, that very fact will act as a complete deterrent to any aggression. This was the sound reasoning which led President Wilson and Mr. Lloyd George to agree to submit respectively to Congress and Parliament this special French Treaty; this is the reasoning which ought to lead Congress, as it has led Parliament, to ratify the French Treaty promptly. My belief is that after five years, this special Treaty will be abrogated by
- 65. mutual consent, because by that time the League of Nations will be built up into such an effective instrument for the prevention of future wars, any special treaties will be deemed unnecessary. THE LEAGUE OF NATIONS COVENANT If, in the foregoing paragraphs, I have given some idea as to how the Treaty of Peace was compounded, how it was made up of a mixture of virtue, selfishness, fear and justice, then perhaps I can proceed to describe briefly how the document was actually evolved. First, then, we deal with the drafting of the League of Nations Covenant: The world has come to regard President Wilson as the special promoter and sponsor for the League of Nations. It is perfectly true that Mr. Wilson went to Paris with a fixed determination, above all else, to bring about some definite arrangement which would tend to prevent future wars. It is also true, however, that English statesmen had, for an even longer time than President Wilson, been giving this same subject earnest thought and study. Some of the more enlightened French statesmen, like Leon Bourgeois, had also been sketching out plans for a League of Free Nations. In England Viscount Grey of Falloden, England's really great Minister of Foreign Affairs for almost a decade prior to the war, the man who did everything that human intelligence and wisdom could devise to prevent the war, and now happily named as British Ambassador to the United States, had long worked for a League of Nations. Lord Robert Cecil, a worthy son of a noble father, was another British statesman who had given his mind to the same subject. General Smuts of South Africa, recently made Premier in succession to the late General Botha, was another. So that President Wilson, Colonel House, and the other delegates, upon their arrival in Paris, found themselves in a not uncongenial atmosphere. To be sure, on the part of Clemenceau and of course of the militarists, there was great scepticism. Nevertheless the French joined in, and early in January the Covenant for the League of Nations began to evolve. It was built up step by step, President Wilson taking a most active part in the work. Finally the Covenant was adopted in a preliminary way and made public late in February. It was subject to amendment, and those who drafted the document welcomed amendments and urged that they be offered. An especial effort was made to secure suggestions from various Republican statesmen. No amendments, so far as I have been able to learn, were offered by any of the Republican Senators, but ex-President Taft suggested certain changes, some of which were adopted. President Lowell of Harvard contributed one or two which were taken over almost verbatim. Ex-Senator Elihu Root
- 66. also made valuable suggestions, some of which were utilized in the final drafting of the Covenant, made public early in April. ESSENCE AND SPIRIT OF THE LEAGUE Roughly, as the situation developed, the purpose of the League of Nations became two- fold. The initial purpose, of course, was to set up the machinery for a body, representative of the nations, keeping in such close contact and guided by such general principles as would tend to make it impossible for one nation to begin war upon another. Elsewhere in this volume ex-Attorney General Wickersham has described in detail the clauses of the Covenant; but even in this brief allusion it is proper to set down the essence and spirit of the League. It is this: No two peoples, if they come to know each other and each other's motives sufficiently well, and if by certain machinery they are maintained in close personal and ideal contact, can conceivably fly at each other's throats. Now no machinery can be devised that will absolutely prevent war, but a carrying out of the spirit and principles set forth in the present Covenant ought to make war well-nigh impossible. The machinery that was thus set up at Paris was deemed at the time to be of course imperfect and subject to constant improvement. The second purpose of the League was to act as the binder, and in a way, the administrative force of the present existing Treaty. That is to say, we found as time went on there were many situations so complex that human wisdom could not devise an immediate formula for their solution. Hence, it became necessary for the Peace Conference to establish certain machinery which, if necessary, should function over a series of years, and thus work out permanently the problems involved. Therefore, as it fell out, there were established under the Treaty, almost a score of Commissions, most of them to act under the general supervision of a League of Nations. Here, then, is another great function that the League of Nations is immediately called upon to fulfil. WORK OF THE COMMISSIONS With the Covenant of the League of Nations more or less complete, the next business of the Conference was the setting up of the Treaty proper. The method for this work was roughly as follows: About the first of February there was appointed a large number of special Commissions, made up of members of the various delegations. These Commissions, which were each to treat of separate topics, having arrived at a solution of the special subject, were then to draft their reports in such language that they could readily be embodied in the final Treaty of Peace itself. Thus, for instance, there was appointed a Commission on Reparations, a Commission on Economic Phases of the
- 67. Treaty, a Commission on Finance, a Commission on Boundaries, a Commission on Military and Naval Armament, a Commission on German Colonies, a Commission on the Saar Basin Coal Fields, a Commission on Inland Waterways, and so on to the number of perhaps twenty. These Commissions immediately organized, and if the subject were particularly complex and many-sided, resolved themselves into sub-commissions. These sub-commissions in turn organized, each with its chairman and vice-chairman, its secretariat, and its interpreters, together with experts called into attendance. DELAYS TO THE TREATY The sittings of all these Commissions began, as I say, about February 1st, and at that time the plan was that the work of the Commissions should be concluded in the form of a report to the Supreme Council six weeks later, or about March 15th. The plan, further, was for the Supreme Council to pass upon these various reports, amend them if need be, and then have them drafted in such form as together would go to make up the Treaty, which, under this scheme, would be presented to the Germans on or about April 1st. The Germans would presumably sign within a fortnight, and we should all be going home about April 15th. As a matter of fact, the Germans signed the Treaty at Versailles at three o'clock on the afternoon of June 28th, two and one-half months later than the time originally planned. This delay was, however, not at all unreasonable, if one stops to consider the number of questions involved, their magnitude, and the difficulty of dealing with them promptly. In the first place, each Commission was supposed to present the Supreme Council a unanimous report. The Council had ruled that the Commissions should not report by majority vote, for if in any given instance the majority overruled the minority, the minority might have such bitter complaint that there would be left in the situation the seed for future trouble. Therefore the Council determined that in the case of divergence of opinion in the same Commission, the two or more groups in the Commission should make separate reports to the Council, each having its own day in court. The Council would act as judges of the last resort, and no delegation would go away feeling that it had not had ample opportunity to present its case. Inevitable and sharp differences of opinion did arise, so that at least half the reports, I should say, as presented to the Big Four had to be thrashed out there in considerable detail. The second handicap to rapid progress, of course, lay in the composition of the various Commissions. Each of the large five powers had to be represented on each Commission, and in most instances smaller powers also demanded representation. On some of the important Commissions the larger powers had two or more delegates sitting. Owing to
- 68. the fact that Paris was full of influenza, each delegate had to have his alternate so as to keep the ball rolling. When they first met these delegates were not well acquainted with each other. They did not know how to get along together. It took weeks for them to shake down, so as to understand each other's methods and points of view; so as to be prepared to make the necessary give and take, certain meetings of views which are always essential where people are gathered from the four corners of the earth with a single aim, but with vastly different ideas for attaining it. Copyright by Underwood Underwood Where the Peace Treaty Was Signed This was the table and chair at which the delegates sat and signed the peace document. Click for a larger image. POLITICS AT THE CONFERENCE Still another difficulty was the question of politics which could not be eliminated. It is easy enough to say, cut out politics, but in any international gathering it is never possible to do it. I must say right here, however, that—as it seemed to me—the American delegation well-nigh attained that ideal, and be it to President Wilson's credit, I never once saw him throughout the length of the conference, play politics. But some of the other delegations naturally felt that at home there was a list'ning senate to applaud or to condemn, and many of these delegates, being members of their respective parliaments or ministries, naturally had their ear to the ground for the effect that their course at Paris was producing. Then if, at the sittings of a Commission, one delegate made a particularly
- 69. eloquent speech, his fellow delegate might feel it incumbent upon him to make another equally long. Some of the delegates deemed it their duty to make an extended speech every day and seemed to feel that they were lacking in patriotism if they failed each morning to cover several pages of the record with their views. THE DIFFICULTY OF LANGUAGE Then the final difficulty, uniting with the other troubles to prevent rapid progress, was that of language. The Paris Conference was, of course, a regular Tower of Babel. There were two official languages—French and English. Each delegation used the language with which its delegates were most familiar, and every word uttered by those delegates had to be translated into the vernacular of the others. Not only did this interpretation consume a vast amount of time, but of course it frequently proved most unsatisfactory. Both the English and French languages are so idiomatic that the finer shades of meaning can never be well transmuted from one to the other. Hence, frequent and sometimes serious mistakes arose. For instance, a Serbian delegate who knew not a word of English would misunderstand something said by the British delegate, poorly translated into French. As the Serbian delegate's knowledge of French was also very limited he could not readily understand. So he would fly into a towering rage, and for an hour a heated argument would volley back and forth. Perhaps, at the end of that time, some cool- headed delegate (frequently an American), would point out that neither of the honorable delegates had any conception of what the other had said, and at bottom their views were precisely similar. Each of the competitors would then listen to reason, the situation would clear up, and things move on more happily. I use here as an example a Serbian delegate, not that the Serbian delegates were more prone to passion than anybody else. We were all fighting like mad to make peace. We realized that though fundamentally we all had the same aim, yet as to methods our views were so divergent, that when we entered into conference at ten o'clock in the morning we should probably have one continuous struggle, with interludes for luncheon and dinner, until perhaps late in the evening. These struggles never ceased altogether, but as we got to know one another better, they of course let up materially, and we got on amicably and effectively. THE COMMISSION ON REPARATIONS No sketch of the Peace Conference, even one as cursory and superficial as this, could give any idea of the picture without a more detailed reference to the workings of some particular Commission that played an important part in the building up of the Peace
- 70. Treaty. Hence I may be permitted to mention the Commission on Reparations. All things considered, this was perhaps the most important Commission at work. The original Commission on Reparations was divided into three sub-commissions. Commission Number One was to determine upon what principles reparation should be demanded from Germany, that is to say, what items of damage should be included. In addition to physical damage inflicted by Germany upon the Allies, by reason of her aggression on land or sea, and from the air, should the cost of pensions for dead French soldiers be claimed? Was the entire cost of the war as waged by England, for instance, to be included as a charge against Germany? In other words, just what categories should be adopted in order to define Germany's liability? This Commission Number One sat for weeks, and it was only towards the very end that it succeeded in establishing the categories. At the start there was a sharp divergence of opinion among the various delegations. The American delegation pointed out that under President Wilson's Fourteen Points costs of war would have to be excluded. The British delegation maintained otherwise. The French thought the costs of war ought to be included, but deemed the matter academic, inasmuch as Germany could never pay the total war costs. And so the argument ran. Sub-commission Number Two on reparations had for its object to determine what Germany's capacity to pay was, and what the proper method of payment should be. Sub- commission Number Three was to devise sanctions or guarantees by which the Allies should be assured of receiving the payments finally determined upon. For weeks I was active upon Sub-commission Number Two, and in fact was charged with the duty of drawing up the initial report covering the question of Germany's capacity to pay. Early in the deliberations of this Sub-commission it became apparent that its work was of momentous import, for whatever the Sub-commission determined as Germany's capacity to pay, undoubtedly that sum would be fixed as what Germany should be obligated to pay. Theoretically, as the French had pointed out, it did not make a great difference what categories of damage were included, because Germany would probably be unable to pay even the extent of material damage she had wrought. It was equally evident that she would be compelled by the Allies to pay to the utmost extent of her capacity. Therefore Sub-commission Number Two was in effect, naming the amount of the German indemnity. AN ESTIMATE OF GERMANY'S CAPACITY TO PAY
- 71. This knowledge rendered the work of the delegates on Sub-commission Number Two considerably more difficult. To estimate Germany's capacity to pay over a series of years was by no means a purely scientific matter. No banker, or economist, or financier, whatever his experience, could look far enough into the future to be able to say what Germany could or could not pay, in ten, twenty, or thirty years. The initial estimate made by one of the delegations, as representing Germany's capacity to pay, was one thousand million of francs. Another estimate was twenty-four billion sterling, about one hundred twenty billion of dollars. Now Germany's entire wealth was estimated at not over eighty billion dollars, so it was inconceivable how it could be possible, even over a series of years, for Germany to pick up her entire commonwealth and transfer it to the Allies. Most of Germany's property consists of the soil, railroads, factories, dwellings, and none of those things can be transported, none can be made available for the payment of reparation. Hence the question arose as to how much liquid wealth Germany could export year after year and still maintain her own economic life. This was the estimate upon which the British, French and American delegations wrangled pleasantly for weeks. Whenever we reached too tense a point, tea and toast was served, with jam to sweeten the atmosphere a bit, and then we would start afresh. As a matter of fact, as we encouraged newspaper reporters to surmise, we had nearly arrived upon a basis of agreement for demanding a fixed sum from Germany. That sum would not have exceeded forty or forty-five billion dollars, with interest added. The American delegation believed it to be far sounder economically to name a fixed sum and thus limit Germany's liability, so that all nations could address themselves to a definite end and arrange their fiscal and taxation policies accordingly. But both Mr. Lloyd George and M. Clemenceau urged that public opinion in both their countries would not acquiesce in any sum that fell far below previous expectations; that, therefore, inasmuch as it was difficult anyway to arrive at once upon the exact amount of damage caused, it would be wiser to leave the amount of reparation open, to be determined by a commission which should examine into the damage sustained, and fix the total amount within two years. America's material interest in the question was so limited that President Wilson finally did not oppose Mr. Lloyd George's and M. Clemenceau's judgment. This, in brief, is the history of the Reparation clauses in the Treaty. As I have already said, if we realize that in almost every one of the other chapters similar complex courses of procedure had to be followed, we shall not be surprised at the time which the Treaty took for drafting. THE ITALIAN CRISIS
- 72. The world is already familiar with the several crises which arose during the course of the Peace Conference. The so-called Fiume crisis, when the Italian delegation walked out and returned to Rome, was regarded as the most serious. I am not sure it was, although it was generally so considered. I believe most of Italy's warmest friends maintain that her action in going home was a mistake. The question of putting Fiume under Italian sovereignty was not covered nor even touched upon in the Treaty of London. In face, the question of Fiume arose long after the Peace Conference was under way. Signor Orlando, the Italian Premier, was accused of fostering Italian feeling on Fiume and of fanning it into flame. I believe there is no truth in this. At any rate, if the Italians had been wise, they would have prevented the matter of Fiume from becoming such a cause celèbre. I think that by judicious work they could have prevented it. Then, too, probably the difficulty would have been lessened if President Wilson's statement to the Italian people had previously met Signor Orlando's approval. Mr. Wilson made his statement with the best will in the world, with the intent to allay and not inflame Italian public opinion. It should have been possible to coördinate his idealism with Signor Orlando's position. Later on the Italian delegation returned to Paris, realizing that the question of Fiume, which was formerly an Austrian port, did not bear one way or another upon the Treaty with Germany. But the Italians had lost a certain tactical position which was important to them, and in my judgment the move cost Italy much more than the whole question of Fiume amounted to. Copyright Underwood Underwood Awaiting the Decision of the German Peace Delegates.
- 73. President Clemenceau is shown standing. Next to him from right to left are: President Wilson, Secretary of State Robert Lansing, Commissioner Henry White, Colonel House, Gen. Tasker H. Bliss, Stephen Pichon, French Minister of Foreign Affairs; Louis Klotz, French Minister of Finance, and André Tardieu, French High Commissioner. From Clemenceau, left to right: Premier Lloyd George, Bonar Law and A. J. Balfour. Click for a larger image. THE QUESTION OF SHANTUNG The Shantung crisis was another serious one. It was so realized at the time by the conferees at Paris. The Japanese delegation considered that it had already suffered one or two rebuffs. Their clause to embody race equality in the League of Nations Covenant had not been accepted. They, as the leading Far Eastern Power, were being urged to take an active part in the organization and development of the League of Nations, yet they could see nothing for Japan in the idea except a chance to help the other fellow. It was at this time that the Treaty clause was being drafted covering the disposition of German rights in the Far East, including those on the Shantung Peninsula. It will be remembered that at the outbreak of the war Germany, by reason of treaty rights with China, had possession of Kiauchau, upon the neck of the Shantung Peninsula. Back in 1916, at a time when the war was going badly, after Japan had driven the Germans out of the Far East and had prevented German submarines from getting a base there to prey upon British troop ships from Australia, Japan had demanded from England and France that she become the inheritor of whatever rights Germany had in Shantung. England and France readily granted this request, as America probably would have done if she had been in the war at the time. Later on, according to the record, China confirmed Japan in these rights. President Wilson's idea, however, was China for Chinamen; therefore Shantung should be turned over to China. This was a proper point of view. It was a great pity that it could not be made to prevail. The difficulty, however, was two-fold: first, the agreement which I have just cited between England and France on one hand, and Japan on the other; second, Japan's statement to President Wilson that if he began his League of Nations by forcing England and France to break a solemn agreement with Japan, then Japan would have no use for such a faithless confederation and would promptly withdraw. At the same time, however, Japan reiterated that her inheritance of Shantung was largely a formal matter, and that if the Allies gave her that recognition, she would feel in honor bound to withdraw from Shantung in the near future. This statement, made repeatedly
- 74. by the Japanese delegates to President Wilson, finally led him to refrain from forcing Great Britain and France to break their agreement, as he might perhaps otherwise have done. The climax, of course, came when Japan gave her ultimatum and said that unless she had her rights she would retire from the Conference. DEMANDS OF BELGIUM Then came the third and last crisis—the Belgians threatened to withdraw and go home. They had, as they claimed, been promised by their Allies, as well as by their enemies, including specifically Germany, that their country, trampled over and devastated in order to defend France and England from attack, was to be fully restored and reimbursed for its expenditures. Early in the Conference Colonel House projected a plan to Mr. Balfour of the British delegation and Mr. Klotz of the French delegation, granting Belgium a priority of $500,000,000 on the German reparation, this sum being sufficient to set Belgium well on her way to recovery. There was, however, great delay in getting the final assent to this priority. The American delegation worked hard to bring it about and to push the plan on every occasion, but it still hung fire. The Belgian delegation, finally becoming alarmed, insisted on formally taking up the question with the Council of Four. The Belgian delegation, under the leadership of Mr. Hymans, Minister of Foreign Affairs, made two chief demands, one for the priority already mentioned, and one for reimbursement for what the war had cost her. To this latter item there was vigorous objection on the ground that it was inadmissible to provide for Belgium's costs of war and not for those of England, France, Italy and the other Allies. As a compromise to meet the situation, a formula was finally proposed in a phrase to the effect that Germany was to be obligated especially to reimburse Belgium for all the sums borrowed by Belgium from the Allies as a necessary consequence of the violation of the Treaty of 1839. Inasmuch as all such sums borrowed by Belgium were used for the prosecution of the war, this phrase was simply a euphemism for granting to Belgium the war costs which she had demanded. But it was finally agreed to on all hands, and the crisis was averted.
- 75. Copyright by Press Illustrating Service The George Washington It was on this ship that President and Mrs. Wilson made their two trips across the Atlantic and back during the Peace Conference. Click for a larger image. THE TREATY PRESENTED TO THE GERMANS AT VERSAILLES The Treaty in its final form was presented to the Germans at Versailles May 7th. The Germans were hoping they would be permitted to discuss certain phases of the Treaty in person with the Allied delegates, and in fact repeatedly requested the opportunity. Some of us believed such conversations might be advantageous if they were held; not between the chiefs of the Allied states and the heads of the German delegation, but between technical experts on both sides. Mr. Wilson favored this view, as tending to enlighten the Germans on certain phases of the Treaty, which from their written communications it was evident they did not understand. We thought that some weeks of delay might possibly be averted by sitting around the table with the Germans, distasteful as that task might be, and holding a kind of miniature peace conference. This suggestion, however, was strongly opposed by M. Clemenceau, although it was favored by some of his ministers. In fact, some of the latter, as well as many of the British, were for a time convinced that the terms of the Treaty were such that Germany would never sign them. Again and again Clemenceau was urged to give way on this point, but he sturdily opposed the view and
- 76. declared positively that he knew the German character; that the only way to secure a German signature to the Treaty was to insist upon purely formal and written communications. Clemenceau had his way, and then began the laying of a good many wagers as to whether the Germans would sign. This was after the original German delegation, or at least the chiefs of it, had returned to Berlin and declared that they would not come back again to Versailles. My own opinion was, that after making as great a kick as possible the Germans would undoubtedly sign. The logic of the situation was all for their signing, the reasoning being this: If the Treaty were a just Treaty, then they ought to sign any way; if it were an unjust Treaty, then, even if signed, it would eventually fall of its own weight, and the Germans would run no risk in signing it. I felt that the German psychology of the situation would be acute enough to see these points and to lead to a signature. GERMANY SIGNS THE TREATY This proved to be the case, and on Saturday, the 21st of June, after questionings and misgivings, we finally got the word that the Germans were to sign. I shall never forget the moment that the news came. Some of us were in session with the Council of Four at the President's house. Mr. Wilson sat on the right of the fireplace, Mr. Lloyd George on the left, and M. Clemenceau in the middle. Mr. Orlando was in Italy but his foreign minister, Baron Sonnino, was there in his place. The afternoon was a tense one, for the time was growing short and the Germans had, as I say, not yet signified their intention of signing the treaty. In the mind of every one of us there lurked the question as to the terrible steps that would have to be taken in the event the Germans refused to sign. Late in the afternoon an orderly slipped into the room and whispered into M. Clemenceau's ear. He struggled to his feet, marched up to President Wilson and Mr. Lloyd George, and, drawing himself up, said in solemn tones, I have the honor to announce to you that the Germans will sign the treaty. And then a moment later the cannon boomed forth to the expectant populace the news that the Germans would sign, and M. Clemenceau, turning to me, breathed: Ah, that is the sound that I have been waiting to hear for forty-eight years.
- 77. II—WILSON'S FOURTEEN POINTS An Attempt to Raise International Morality to the Level of Private Morality On January 8, 1918, President Wilson outlined the fourteen points on the basis of which the Allies should make peace. This program was the startling climax of a whole series of peace proposals which had kept coming from both camps of belligerents, from neutrals, Socialists, and the Pope. It is without doubt one of the greatest and most inspiring State documents in the history of the world. It struck a vital and telling blow at the basic causes of modern wars. For that reason it electrified into complete unity the masses of the Allied countries. Liberal, radical and pacifist opponents of the war rallied around it as the last great hope of civilization. Its most important effect was to give a democratic basis to the weary and disillusioned masses of the Central Powers who were longing for peace. It was on the basis of the fourteen points that the enemy surrendered. THE WILSON PROGRAM We entered this war because violations of right had occurred which touched us to the quick and made the life of our own people impossible unless they were corrected and the world secured once for all against their recurrence. What we demand in this war, therefore, is nothing peculiar to ourselves. It is that the world be made fit and safe to live in; and particularly that it be made safe for every peace-loving nation which, like our own, wishes to live its own life, determine its own institutions, be assured of justice and fair dealings by the other peoples of the world, as against force and selfish aggression. All the peoples of the world are in effect partners in this interest and for our own part we see very clearly that unless justice be done to others it will not be done to us. The programme of the world's peace, therefore, is our programme, and that programme, the only possible programme, as we see it, is this: I. Open covenants of peace, openly arrived at, after which there shall be no private international understandings of any kind, but diplomacy shall proceed always frankly and in the public view.
- 78. II. Absolute freedom of navigation upon the seas, outside territorial waters, alike in peace and in war, except as the seas may be closed in whole or in part by international action for the enforcement of international covenants. III. The removal, as far as possible, of all economic barriers and the establishment of an equality of trade conditions among all the nations consenting to the peace and associating themselves for its maintenance. IV. Adequate guarantees given and taken that national armaments will be reduced to the lowest point consistent with domestic safety. V. A free, open minded, and absolutely impartial adjustment of all colonial claims, based upon a strict observance of the principle that in determining all such questions of sovereignty the interests of the populations concerned must have equal weight with the equitable claims of the government whose title is to be determined. Copyright by Underwood Underwood Paris Crowds Greeting President Wilson A general holiday was declared to welcome the President of the United States. This photograph was taken in the Place dé la Concorde. Click for a larger image. VI. The evacuation of Russian territory and such a settlement of all questions affecting Russia as will secure the best and freest coöperation of the other nations of the world in obtaining for her an unhampered and unembarrassed opportunity for the independent
- 79. determination of her own political development and national policy and assure her of a sincere welcome into the society of free nations under institutions of her own choosing; and, more than a welcome, assistance also of every kind that she may need and may herself desire. The treatment accorded Russia by her sister nations in the months to come will be the acid test of their good will, of their comprehension of her needs as distinguished from their own interests, and of their intelligent and unselfish sympathy. VII. Belgium, the whole world will agree, must be evacuated and restored, without any attempt to limit the sovereignty which she enjoys in common with all other free nations. No other single act will serve as this will serve to restore confidence among the nations in the laws which they have themselves set and demanded for the government of their relations with one another. Without this healing act the whole structure and validity of international law is forever impaired. VIII. All French territory should be freed and the invaded portions restored, and the wrong done France by Prussia in 1871 in the matter of Alsace-Lorraine, which has unsettled the peace of the world for nearly fifty years, should be righted, in order that peace may once more be made secure in the interest of all. IX. A readjustment of the frontiers of Italy should be effected along clearly recognizable lines of nationality. X. The peoples of Austria-Hungary, whose place among the nations we wish to see safeguarded and assured, should be accorded the freest opportunity of autonomous development. XI. Rumania, Serbia and Montenegro should be evacuated; occupied territories restored; Serbia accorded free and secure access to the sea, and the relations of the several Balkan States to one another determined by friendly counsel along historically established lines of allegiance and nationality, and international guarantees of the political and economic independence and territorial integrity of the several Balkan States should be entered into. XII. The Turkish portions of the present Ottoman Empire should be assured a secure sovereignty, but the other nationalities which are now under Turkish rule should be assured an undoubted security of life and an absolutely unmolested opportunity of autonomous development, and the Dardanelles should be permanently opened as a free passage to the ships and commerce of all nations under international guarantees.
- 80. XIII. An independent Polish State should be erected which should include the territories inhabited by indisputably Polish population, which should be assured a free and secure access to the sea and whose political and economic independence and territorial integrity should be guaranteed by international covenant. XIV. A general association of nations must be formed under specific covenants for the purpose of affording mutual guarantees of political independence and territorial integrity to great and small States alike.
- 81. III—HOW THE PEACE TREATY WAS SIGNED A Description of the Historic Ceremony in the Hall of Mirrors at the Palace of Versailles, June 8, 1919 (Reprinted from the New York Times.) No nobler and more eloquent setting could have been found for this greatest of all modern events, the signing of the Peace of Versailles, after five years of terrific struggle on whose outcome the fate of the whole world had hung, than the palace of the greatest of French Kings on the hillcrest of the Paris suburb that gave its name to the treaty. To reach it, says the correspondent of The New York Times, the plenipotentiaries and distinguished guests from all parts of the world motored to Versailles that day, and drove down the magnificent tree-lined Avenue du Château, then across the huge square—the famous Place d'Armes of Versailles—and up through the gates and over the cobblestones of the Court of Honor to the entrance, where officers of the Republican Guard, whose creation dates back to the French Revolution, in picturesque uniform, were drawn up to receive them. All day the crowd had been gathering. It was a cloudy day; not till noon did the sky clear. By noon eleven regiments of French cavalry and infantry had taken position along the approaches to the palace, while within the court on either side solid lines of infantry in horizon blue were drawn up at attention. Hours before the time set for the ceremony an endless stream of automobiles began moving out of Paris up the cannon-lined hill of the Champs Elysées, past the massive Arc de Triomphe, bulking somberly against the leaden sky, and out through the Bois de Boulogne. This whole thoroughfare was kept clear by pickets, dragoons, and mounted gendarmes. In the meantime thousands of Parisians were packing regular and special trains on all the lines leading to Versailles, and contending with residents of the town for
- 82. places in the vast park where the famous fountains would rise in white fleur-de-lis to mark the end of the ceremony. A MEMORABLE SCENE Past the line of gendarmes thrown across the approaches to the square reserved for ticket holders, the crowd surged in a compact and irresistible wave, while hundreds of the more fortunate ones took up positions in the high windows of every wing of the palace. Up the broad boulevard of the Avenue de Paris the endless chain of motor cars rolled between rows of French soldiers; and a guard of honor at the end of the big court presented arms to the plenipotentiaries and delegates as they drove through to the entrance, which for the Allied delegates only was by the marble stairway to the Queen's Apartments and the Hall of Peace, giving access to the Hall of Mirrors. A separate route of entry was prescribed for the Germans, an arrangement which angered and disconcerted them when they discovered it, through the park and up the marble stairway through the ground floor. The delegates and plenipotentiaries began to arrive shortly after 2 p. m., their automobiles rolling between double lines of infantry with bayonets fixed—it was estimated that there were 20,000 soldiers altogether guarding the route—that held back the cheering throngs. The scene from the Court of Honor was impressive. The Place d'Armes was a lake of white faces, dappled everywhere by the bright colors of flags and fringed with the horizon blue of troops whose bayonets flamed silverly as the sun emerged for a moment from behind heavy clouds. At least a dozen airplanes wheeled and curvetted above. Up that triumphal passage, leading for a full quarter of a mile from the wings of the palace to the entrance to the Hall of Mirrors, representatives of the victorious nations passed in flag-decked limousines—hundreds, one after another, without intermission, for fifty minutes. Just inside the golden gates, which were flung wide, they passed the big bronze statue of Louis XIV., the Sun-King, on horseback, flanked by statues of the Princes and Governors, Admirals and Generals who had made Louis the Grand Monarque of France. And on the façade of the twin, temple-like structures on either side of the great statue they could read as they passed an inscription symbolic of the historic ceremony just about to occur: To All the Glories of France. NOTABILITIES ARRIVE
- 83. One of the earliest to arrive was Marshal Foch, amid a torrent of cheering, which burst out even louder a few moments later when the massive head of Premier Clemenceau was seen through the windows of a French military car. To these and other leaders, including President Wilson, General Pershing, and Premier Lloyd George, the troops drawn up all around the courtyard presented arms. After Clemenceau the unique procession continued, diplomats, soldiers, Princes of India in gorgeous turbans and swarthy faces, dapper Japanese in immaculate Western dress, Admirals, aviators, Arabs; one caught a glimpse of the bright colors of French, British, and Colonial uniforms. British Tommies and American doughboys also dashed up on crowded camions, representing the blood and sweat of the hard-fought victory; they got an enthusiastic reception. It was 2:45 when Mr. Balfour, bowing and smiling, heralded the arrival of the British delegates. Mr. Lloyd George was just behind him, for once wearing the conventional high hat instead of his usual felt. At 2:50 came President Wilson in a black limousine with his flag, a white eagle on a dark blue ground; he received a hearty welcome. By 3 o'clock the last contingent had arrived, and the broad ribbon road stretched empty between the lines of troops from the gates of the palace courtyard. The Germans had already entered; to avoid any unpleasant incident they had been quietly conveyed from their lodgings at the Hotel des Reservoirs Annex through the park. THE SCENE INSIDE The final scene in the great drama was enacted in the magnificent Hall of Mirrors. Versailles contains no more splendid chamber than this royal hall, whose three hundred mirrors gleam from every wall, whose vaulted and frescoed ceiling looms dark and high, in whose vastness the footfalls of the passer re-echo over marble floors and die away reverberatingly. It was no mere matter of convenience or accident that the Germans were brought to sign the Peace Treaty in this hall. For this same hall, which saw the German peace delegates of 1919, representing a beaten and prostrate Germany, affix their signatures to the Allied terms of peace, had witnessed in the year 1871 a very different ceremony. It was in the Hall of Mirrors that the German Empire was born. Forty-nine years ago, on a January morning, while the forts of beleaguered Paris were firing their last defiant shots, in that mirror-gleaming hall was inaugurated the reign of that German Empire the virtual end of which, so far as the concept held by its originators is concerned, was signalized in Versailles in the same spot on Saturday, June 28. And in 1871 President Thiers had signed there the crushing terms of defeat imposed by a victorious and ruthless Germany.
- 84. In anticipation of the present ceremony carpets had been laid and the ornamental table, with its eighteenth century gilt and bronze decorations, had been placed in position on the daïs where the plenipotentiaries were seated. Fronting the chair of M. Clemenceau was placed a small table, on which the diplomatic instruments were laid. It was to this table that each representative was called, in alphabetical order by countries, to sign his name to the treaty and affix to it his Governmental seal. The four hundred or more invited guests were given places in the left wing of the Hall of Mirrors, while the right wing was occupied by about the same number of press representatives. Sixty seats were allotted to the French press alone. Besides the military guards outside the palace, the grand stairway up which the delegates came to enter the hall was controlled by the Republican Guards in their most brilliant gala uniform. THE PEACE TABLE The peace table—a huge hollow rectangle with its open side facing the windows in the hall—was spread with tawny yellow coverings blending with the rich browns, blues, and yellows of the antique hangings and rugs; these, and the mellow tints of the historical paintings, depicting scenes from France's ancient wars, in the arched roof of the long hall, lent bright dashes of color to an otherwise austere scene. Against the sombre background also stood out the brilliant uniforms of a few French guards, in red plumed silver helmets and red, white, and blue uniforms, and a group of Allied Generals, including General Pershing, who wore the scarlet sash of the Legion of Honor. But all the diplomats and members of the parties who attended the ceremony of signing wore conventional civilian clothes. All gold lace and pageantry was eschewed, the fanciful garb of the Middle Ages was completely absent as representative of traditions and practices sternly condemned in the great bound treaty-volume of Japanese paper, covered with seals and printed in French and English, which was signed by twenty-seven nations that afternoon. As a contrast with the Franco-German peace session of 1871, held in the same hall, there were present some grizzled French veterans of the Franco-Prussian war. They took the place of the Prussian guardsmen of the previous ceremony, and gazed with a species of grim satisfaction at the disciples of Bismarck, who sat this time in the seats of the lowly, while the white marble statue of Minerva looked stonily on. ENTRANCE OF CHIEF ACTORS
- 85. Copyright Underwood Underwood Henry White Former Ambassador to France and Italy and one of the United States delegates to the Peace Conference. The ceremony of signing was marked only by three minor incidents: a protest by the German delegation at the eleventh hour over the provision of separate entrance, the filing of a document of protest by General Jan Smuts of the South African delegation, and the deliberate absence of the Chinese delegates from the ceremony, due to dissatisfaction over the concessions granted to Japan in Shantung. The treaty was deposited on the table at 2:10 p.m. by William Martin of the French Foreign Office; it was inclosed in a stamped leather case, and bulked large. Because of the size of the volume and the fragile seals it bore, the plan to present it for signing to Premier Clemenceau, President Wilson, and Premier Lloyd George had been given up. A box of old- fashioned goose quills, sharpened by the expert pen pointer of the French Foreign Office, was placed on each of the three tables for the use of plenipotentiaries who desired to observe the conventional formalities. Secretary Lansing, meanwhile, had been the first of the American delegation to arrive in the palace—at 1:45 p.m. Premier Clemenceau entered at 2:20. Three detachments each consisting of fifteen private soldiers—from the American, British, and French forces—just before 3 o'clock and took their places in the embrasures of the windows overlooking the château park, a few feet from Marshal Foch, who was seated with the French delegation at the peace table. Marshal Foch was present only as a spectator, and did not participate in the signing. These forty-five soldiers of the three main belligerent nations were present as the real artisans of peace and stood within the inclosure reserved for plenipotentiaries and high officials of the conference as a visible sign of their rôle in bringing into being a new
- 86. Europe. These men had been selected from those who bore honorable wounds. Premier Clemenceau stepped up to the poilus of the French detachment and shook the hand of each, expressing his pleasure at seeing them, and his regrets for the suffering they had endured for France. PRESIDENT WILSON ENTERS Delegates of the minor powers made their way with difficulty through the crowd to their places at the table. Officers and civilians lined the walls and filled the aisles. President Wilson entered the Hall of Mirrors at 2:50. All the Allied delegates were then seated, except the Chinese representatives, who were conspicuous by their absence. The difficulty of seeing well militated against demonstrations on the arrival of prominent statesmen. The crowd refused to be seated and thronged toward the center of the hall, which is so long that a good view was impossible from any distance, even with the aid of opera glasses. German correspondents were ushered into the hall just before 3 o'clock and took standing room in a window at the rear of the correspondents' section. At 3 o'clock a hush fell over the hall. There were a few moments of disorder while the officials and the crowd took their places. At 3:07 the German delegates, Dr. Hermann Müller, German Secretary for Foreign Affairs, and Dr. Johannes Bell, Colonial Secretary, were shown into the hall; with heads held high they took their seats. The other delegates remained seated, according to a prearranged plan reminiscent of the discourtesy displayed by von Brockdorff-Rantzau, who at the ceremony of delivery of the peace treaty on May 7th, had refused to rise to read his address to the Allied delegates. The seats of the German delegates touched elbows with the Japanese on the right and the Brazilians on the left. They were thus on the side nearest the entrance, and the program required them to depart by a separate exit before the other delegates at the close of the ceremony. Delegates from Ecuador, Peru, and Liberia faced them across the narrow table. THE GERMANS SIGN M. Clemenceau, as President of the Peace Conference, opened the ceremony. Rising, he made the following brief address, amid dead silence:
- 87. The session is open. The allied and associated powers on one side and the German Reich on the other side have come to an agreement on the conditions of peace. The text has been completed, drafted, and the President of the Conference has stated in writing that the text that is about to be signed now is identical with the 200 copies that have been delivered to the German delegation. The signatures will be given now, and they amount to a solemn undertaking faithfully and loyally to execute the conditions embodied by this treaty of peace. I now invite the delegates of the German Reich to sign the treaty.' There was a tense pause for a moment. Then in response to M. Clemenceau's bidding the German delegates rose without a word, and, escorted by William Martin, master of ceremonies, moved to the signatory table, where they placed upon the treaty the sign-manuals which German Government leaders had declared over and over again, with emphasis and anger, would never be appended to this treaty. They also signed a protocol covering changes in the documents, and the Polish undertaking. All three documents were similarly signed by the Allied delegates who followed. WILSON SIGNS NEXT When the German delegates regained their seats after signing, President Wilson immediately rose and, followed by the other American plenipotentiaries, moved around the sides of the horseshoe to the signature tables. It was thus President Wilson, and not M. Clemenceau, who was first of the Allied delegates to sign. This, however, was purely what may be called an alphabetical
- 88. Welcome to our website – the perfect destination for book lovers and knowledge seekers. We believe that every book holds a new world, offering opportunities for learning, discovery, and personal growth. That’s why we are dedicated to bringing you a diverse collection of books, ranging from classic literature and specialized publications to self-development guides and children's books. More than just a book-buying platform, we strive to be a bridge connecting you with timeless cultural and intellectual values. With an elegant, user-friendly interface and a smart search system, you can quickly find the books that best suit your interests. Additionally, our special promotions and home delivery services help you save time and fully enjoy the joy of reading. Join us on a journey of knowledge exploration, passion nurturing, and personal growth every day! ebookbell.com