![2 | Preliminaries
The angle between two vectors x, y ∈ Rp
{0} is the value θ ∈ [0, π] given by
cos θ :=
hx, yi
kxk kyk
.
The open (closed) ball and the sphere centered at x ∈ Rp
with radius ε > 0 are given,
respectively, by:
B(x, ε) := {x ∈ Rp
| kx − xk < ε},
D(x, ε) := {x ∈ Rp
| kx − xk ≤ ε},
and
S(x, ε) := {x ∈ Rp
| kx − xk = ε}.
One says that a subset A ⊂ Rp
is bounded if it is contained in an open ball centered
in the origin i.e., if there exists M > 0 such that A ⊂ B(0, M).
A neighborhood of an element x ∈ Rp
is a subset of Rp
which contains an open
ball centered in x. We denote by V(x) the class of all neighborhoods of x. Let us sum-
marize some facts:
– A subset of Rp
is open if it is empty or it is neighborhood for all of its points.
– A subset of Rp
is closed if its complement with respect to Rp
is open.
– An element a is an interior point of the set A ⊂ Rp
if A is a neighborhood of a. We
denote by int A the interior of A (i.e., the set of all interior points of A).
– An element a is an accumulation point (or a limit point) of A ⊂ Rp
if every neigh-
borhood of a has at least one element in common with the set A which is different
from a. We denote by A0
the set of all limit points of A. If a ∈ A A0
, one says that
a is an isolated point of A.
– An element a is an adherent point (or a closure point) of A ⊂ Rp
if every neigh-
borhood of a has at least one element in common with the set A. We will use the
notations cl A and A to denote the closure of A (i.e., the set of all the adherent
points of A).
– A subset of Rp
is compact if it is bounded and closed.
We denote by bd A the set cl A int A = cl A ∩ cl(Rp
A) and we call it the boundary of
A.
Proposition 1.1.1. (i) A subset of Rp
is open if and only if it coincides with its interior.
(ii) A subset of Rp
is closed if and only if it coincides with its closure.
Definition 1.1.2. One says that a function f : N → Rp
is a sequence of elements from
Rp
.
The value of the function f in n ∈ N, f(n), is denoted by xn (or yn, zn, ...), and the
sequence defined by f is denoted by (xn) (respectively, by (yn), (zn), ...).
Definition 1.1.3. A sequence is bounded if the set of its terms is bounded.](https://image.slidesharecdn.com/25523077-250512184110-75773370/85/An-Introduction-To-Nonlinear-Optimization-Theory-Marius-Durea-Radu-Strugariu-16-320.jpg)
![4 | Preliminaries
∀ε > 0, ∃nε ∈ N, ∀n ≥ nε, ∀p ∈ N : kxn+p − xnk < ε.
Theorem 1.1.15 (Cauchy). The space Rp
is complete, i.e., a sequence from Rp
is con-
vergent if and only if it is a Cauchy sequence.
The next results are specific to the case of real sequences.
Definition 1.1.16. One says that a sequence (xn) of real numbers is increasing (strictly
increasing, decreasing, strictly decreasing) if for every n ∈ N, xn+1 ≥ xn (xn+1 > xn,
xn+1 ≤ xn, xn+1 < xn). If (xn) is either increasing or decreasing, then it is called monotone.
Let R := R ∪ {−∞, +∞} be the set of extended real numbers. A neighborhood of +∞
is a subset of R which contains an interval of the form (x, +∞], where x ∈ R. The
neighborhoods of −∞ are defined in a similar manner.
Definition 1.1.17. (i) One says that the sequence (xn) ⊂ R has the limit equal to +∞ if
∀V ∈ V(+∞), ∃nV ∈ N, ∀n ≥ nV : xn ∈ V.
(ii) One says that the sequence (xn) ⊂ R has the limit equal to −∞ if
∀V ∈ V(−∞), ∃nV ∈ N, ∀n ≥ nV : xn ∈ V.
Proposition 1.1.18. (i) A sequence (xn) ⊂ R has the limit equal to +∞ if and only if
∀A > 0, ∃nA ∈ N, ∀n ≥ nA : xn > A.
(ii) A sequence (xn) ⊂ R has the limit equal to −∞ if and only if
∀A > 0, ∃nA ∈ N, ∀n ≥ nA : xn < −A.
Proposition 1.1.19. Let (xn), (yn), (zn) be sequences of real numbers, x, y ∈ R and n0 ∈
N. Then:
(i) (Passing to the limit in inequalities) if xn → x, yn → y and xn ≤ yn for every
n ≥ n0, then x ≤ y;
(ii) (The boundedness criterion) if |xn − x| ≤ yn for every n ≥ n0, and yn → 0, then
xn → x;
(iii) if xn ≥ yn for every n ≥ n0, and yn → +∞, then xn → +∞;
(iv) if xn ≥ yn for every n ≥ n0, and xn → −∞, then yn → −∞;
(v) if (xn) is bounded and yn → 0, then xnyn → 0;
(vi) if xn ≤ yn ≤ zn for every n ≥ n0, and xn → x, zn → x, then yn → x;
(vii) xn → 0 ⇔ |xn| → 0 ⇔ x2
n → 0.
We now present some fundamental results in the theory of real sequences.](https://image.slidesharecdn.com/25523077-250512184110-75773370/85/An-Introduction-To-Nonlinear-Optimization-Theory-Marius-Durea-Radu-Strugariu-18-320.jpg)
![6 | Preliminaries
Theorem 1.2.3. Let f : A → Rq
, A ⊂ Rp
, l ∈ Rq
and a ∈ A0
. If the function f has the
limit l at a, then this limit is unique.
Remark 1.2.4. If there exist two sequences x0
n
, x00
n
⊂ A {a} , x0
n → a, x00
n → a
such that f(x0
n) → l0
, f(x00
n ) → l00
and l0
≠ l00
, then the limit of the function f at a ∈ A0
does not exist.
Theorem 1.2.5. Let f : A → Rq
, A ⊂ Rp
, f = (f1, f2, ..., fq) and a ∈ A0
. Then f has
the limit l = (l1, l2, ..., lq) ∈ Rq
at a if and only if there exists lim
x→a
fi (x) = li, for every
i ∈ 1, q.
Definition 1.2.6. Let a ∈ R, A ⊂ R and denote As = A ∩(−∞, a], Ad = A ∩[a, ∞). One
says that the element a is a left (right) accumulation point for A, if it is an accumulation
point for As (Ad, respectively). We will denote the set of left (right) accumulation points
of A by A0
s (A0
d, respectively).
Definition 1.2.7. Let f : A → Rq
, A ⊂ R and a be a left (right) accumulation point of
A. One says that the element l ∈ Rq
is the left-hand (right-hand) limit of the function f
in a if for every neighborhood V ∈ V (l) there exists U ∈ V(a), such that if x ∈ U ∩ As
(x ∈ U ∩ Ad, respectively), x ≠ a, then f (x) ∈ V. In this case we will write lim
x→a,xa
f (x) =
l, or lim
x→a−
f (x) = l, or lim
x↑a
f (x) = l ( lim
x→a,xa
f (x) = l, or lim
x→a+
f (x) = l, or lim
x↓a
f (x) = l,
respectively).
Theorem 1.2.8. Let I ⊂ R be an open interval, f : I → Rq
, and a ∈ I. Then there exists
lim
x→a
f (x) = l if and only if the left-hand and the right-hand limits of f at a exist and they
are equal. In this case, all three limits are equal:
lim
x→a−
f (x) = lim
x→a+
f (x) = l.
A well-known result says that the monotone real functions admit lateral limits at every
accumulation point of their domains.
Theorem 1.2.9 (The boundedness criterion). Let f : A → Rq
, g : A → R, A ⊂ Rp
and a ∈ A0
. If there exist l ∈ Rq
and U ∈ V (a) such that kf (x) − lk ≤ |g (x)| for every
x ∈ U {a} , and lim
x→a
g (x) = 0, then there exists lim
x→a
f (x) = l.
Theorem 1.2.10. Let f , g : A ⊂ Rp
→ Rq
, and a ∈ A0
. If lim
x→a
f (x) = 0 and there exists
U ∈ V (a) such that g is bounded on U, then there exists the limit lim
x→a
f (x) g (x) = 0.
Theorem 1.2.11. Let f : A ⊂ Rp
→ Rq
, and a ∈ A0
. If there exists lim
x→a
f (x) = l, l 0
(l 0), then there exists U ∈ V (a) such that for every x ∈ U ∩A, x ≠ a, one has f (x) 0
(respectively, f (x) 0).](https://image.slidesharecdn.com/25523077-250512184110-75773370/85/An-Introduction-To-Nonlinear-Optimization-Theory-Marius-Durea-Radu-Strugariu-20-320.jpg)
![12 | Preliminaries
this case the differential is denoted by ∇xf(a, b). If f is differentiable at (a, b), then f
is differentiable with respect to x and y at a and b, respectively, and
∇xf(a, b) = ∇f(a, b)(·, 0), ∇yf(a, b) = ∇f(a, b)(0, ·).
In the general case, one says that f : D ⊂ Rp
→ Rq
is twice Fréchet differentiable
at a ∈ int D if f is Fréchet differentiable on a neighborhood V ⊂ D of a and ∇f :
V → L(Rp
, Rq
) is Fréchet differentiable at a, i.e., there exists a functional denoted by
∇2
f(a), from the space L2
(Rp
, Rq
) := L(Rp
, L(Rp
, Rq
)), and α : D − {a} → L(Rp
, Rq
),
such that limh→0 α(h) = α(0) = 0 and for every h ∈ D − {a}, one has
∇f(a + h) = ∇f(a) + ∇2
f(a)(h, ·) + khk α(h).
Recall that the space L2
(Rp
, Rq
) mentioned above can be identified with the space
of bilinear maps from Rp
× Rp
to Rq
.
One says that f is of class C2
on the open set D if it is twice Fréchet differentiable
on D and ∇2
f : D → L2
(Rp
, Rq
) is continuous.
Theorem 1.3.3. Let f : D ⊂ Rp
→ Rq
and a ∈ int D. If f is twice Fréchet differentiable
at a, then ∇2
f(a) is a symmetric bilinear map.
In the case when q = 1, the map ∇2
f(a) is defined by the symmetric square matrix
H(a) =
∂2
f
∂xi∂xj
(a)
i,j∈1,p
, which is called the Hessian matrix of f at a. Moreover,
D
H(a)ut
t
, v
E
= ∇2
f(a)(u, v) for every u, v ∈ Rp
, i.e.,
∇2
f(a)(u, v) =
n
X
i,j=1
∂2
f
∂xi∂xj
(a)uivj.
If a, b ∈ Rp
, one defines the closed and the open line segments between a and b
as follows:
[a, b] := {αa + (1 − α)b | α ∈ [0, 1]},
(a, b) := {αa + (1 − α)b | α ∈ (0, 1)}.
Theorem 1.3.4 (Lagrange and Taylor Theorems). Let U ⊂ Rp
be an open set, f : U →
R and a, b ∈ U with [a, b] ⊂ U. If f is of class C1
on U, then there exists c ∈ (a, b) such
that
f(b) = f(a) + ∇f(c)(b − a).
If f is of class C2
on U, then there exists c ∈ (a, b) such that
f(b) = f(a) + ∇f(a)(b − a) +
1
2
∇2
f(c)(b − a, b − a).](https://image.slidesharecdn.com/25523077-250512184110-75773370/85/An-Introduction-To-Nonlinear-Optimization-Theory-Marius-Durea-Radu-Strugariu-26-320.jpg)
![Differentiability | 13
Theorem 1.3.5 (Implicit Function Theorem). Let D ⊂ Rp
× Rq
be an open set, h : D →
Rq
be a function and x ∈ Rp
, y ∈ Rq
be such that:
(i) h(x, y) = 0;
(ii) the function h is of class C1
on D;
(iii) ∇yh(x, y) is invertible.
Then there exist two neighborhoods U and V of x and y, respectively, and a unique
continuous function φ : U → V such that:
(a) h(x, φ(x)) = 0 for every x ∈ U;
(b) if (x, y) ∈ U × V and h(x, y) = 0, then y = φ(x);
(c) φ is differentiable on U and
∇φ(x) = −[∇yh(x, φ(x))]−1
∇xh(x, φ(x)), ∀x ∈ U.
Some fundamental results from the theory of differentiability of the real functions are
briefly given at the end of this section.
In the case p = q = 1 one can apply Proposition 1.3.2. It is also sensible to speak of
the existence of the derivative at points of the domain which are accumulation points
of it: consider the limit from the relation (1.3.1) at accumulation points. Moreover, as in
the case of the lateral limits, one can speak about the left and right-hand derivatives,
by considering the lateral limits in the expression from relation (1.3.1). When they ex-
ist, we will call these limits the left, and the right-hand derivatives of the function f at
a and we will denote them by f0
−(a) and f0
+(a), respectively.
Definition 1.3.6. Let A ⊂ R and f : A → R. One says that a ∈ A is a local minimum
(maximum) point for f if there exists a neighborhood V of a such that f(a) ≤ f(x) (re-
spectively, f(a) ≥ f(x)), for every x ∈ A ∩ V. One says that a point is a local extremum if
it is a local minimum or a local maximum.
Theorem 1.3.7 (Fermat Theorem). Let I ⊂ R be an interval and a ∈ int I. If f : I → R
is derivable at a, and a is a local extremum point for f , then f0
(a) = 0.
Theorem 1.3.8 (Rolle Theorem). Let a, b ∈ R, a b, and f : [a, b] → R be a function
which is continuous on [a, b], derivable on (a, b), and satisfies f(a) = f(b). Then there
exists c ∈ (a, b) such that f0
(c) = 0.
Theorem 1.3.9 (Lagrange Theorem). Let a, b ∈ R, a b, and f : [a, b] → R be a
function which is continuous on [a, b], derivable on (a, b). Then there exists c ∈ (a, b)
such that f(b) − f(a) = f0
(c)(b − a).
Proposition 1.3.10. Let I ⊂ R be an interval and f : I → R be derivable on I.
(i) If f0
(x) = 0 for every x ∈ I, then f is constant on I.](https://image.slidesharecdn.com/25523077-250512184110-75773370/85/An-Introduction-To-Nonlinear-Optimization-Theory-Marius-Durea-Radu-Strugariu-27-320.jpg)
![The Riemann Integral | 15
1.4 The Riemann Integral
At the end of this chapter we discuss the main aspects concerning the Riemann inte-
gral. Let a, b ∈ R, a b.
Definition 1.4.1. (i) A partition of the interval [a, b] is a finite set of real numbers
x0, x1, ..., xn (n ∈ N*
), denoted by ∆, such that
a = x0 x1 ... xn−1 xn = b.
(ii) The norm of the partition ∆ is the number
k∆k := max{xi − xi−1 | i ∈ 1, n}.
(iii) A tagged partition of the interval [a, b] is a partition ∆, together with a finite set
of real numbers Ξ := {ξi | i ∈ 1, n}, such that ξi ∈ [xi−1, xi] for any i ∈ 1, n. The set Ξ
is called the intermediate points system associated to ∆.
(iv) Let f : [a, b] → R be a function. The Riemann sum associated to a tagged
partition of the interval [a, b] is
S(f , ∆, Ξ) :=
n
X
i=1
f(ξi)(xi − xi−1).
Definition 1.4.2. Let f : [a, b] → R be a function. One says that f is Riemann integrable
on [a, b] if there exists I ∈ R such that for every ε 0, there exists δ 0 such that for
any partition ∆ of the interval [a, b] with the property k∆k δ, and for any intermediate
points system Ξ associated to ∆, the next inequality holds:
S(f , ∆, Ξ) − I ε.
The real number I from the previous definition, which is unique, is called the Riemann
integral of f an [a, b] and is denoted by
b
Z
a
f(x)dx.
Theorem 1.4.3. Any function which is Riemann integrable on [a, b] is bounded on
[a, b].
Definition 1.4.4. Let f : [a, b] → R be a function. One says that a function F : [a, b] →
R is an antiderivative (or, equivalently, a primitive integral, or an indefinite integral) of
f on [a, b] if F is derivable on [a, b] and F0
(x) = f(x) for any x ∈ [a, b].
If an antiderivative exists for a given function, then infinitely many antiderivatives
exist for that function and the difference of any two such antiderivatives is a constant.
The next result is sometimes called the fundamental theorem of calculus.](https://image.slidesharecdn.com/25523077-250512184110-75773370/85/An-Introduction-To-Nonlinear-Optimization-Theory-Marius-Durea-Radu-Strugariu-29-320.jpg)
![16 | Preliminaries
Theorem 1.4.5 (Leibniz-Newton). If f : [a, b] → R is Riemann integrable on the inter-
val [a, b] and it admits an antiderivative F on [a, b], then
b
Z
a
f(x)dx = F(b) − F(a).
Continuous functions satisfy both hypotheses of the preceding theorem.
Theorem 1.4.6. If f : [a, b] → R is continuous on [a, b], then f is Riemann integrable
on [a, b] and it admits antiderivatives on [a, b].
Theorem 1.4.7. If f : [a, b] → R is bounded and has a finite set of discontinuity points,
then f is Riemann integrable on [a, b]. Every function which is monotone on [a, b] is
Riemann integrable on [a, b].
We present now the main properties of the Riemann integral.
Theorem 1.4.8. (i) If f , g : [a, b] → R are Riemann integrable on [a, b], and α, β ∈ R,
then αf + βg is Riemann integrable on [a, b] and
b
Z
a
(αf(x) + βg(x))dx = α
b
Z
a
f(x)dx + β
b
Z
a
g(x)dx.
(ii) If f : [a, b] → R is Riemann integrable on [a, b], and m ≤ f(x) ≤ M for every
x ∈ [a, b] (m, M ∈ R), then
m(b − a) ≤
b
Z
a
f(x)dx ≤ M(b − a).
In particular, if f(x) ≥ 0 for every x ∈ [a, b], then
b
Z
a
f(x)dx ≥ 0,
and if f , g : [a, b] → R are Riemann integrable and f(x) ≤ g(x) for every x ∈ [a, b], then
b
Z
a
f(x)dx ≤
b
Z
a
g(x)dx.
(iii) If f : [a, b] → R is Riemann integrable on [a, b], then |f| is Riemann integrable
on [a, b].
(iv) If f , g : [a, b] → R are Riemann integrable [a, b], then f · g is Riemann inte-
grable on [a, b].](https://image.slidesharecdn.com/25523077-250512184110-75773370/85/An-Introduction-To-Nonlinear-Optimization-Theory-Marius-Durea-Radu-Strugariu-30-320.jpg)
![The Riemann Integral | 17
Theorem 1.4.9. (i) If f : [a, b] → R is Riemann integrable on [a, b], then f is Riemann
integrable on every subinterval of [a, b].
(ii) If c ∈ (a, b) and f is Riemann integrable on [a, c] and on [c, b], then f is Rie-
mann integrable on [a, b] and
b
Z
a
f(x)dx =
c
Z
a
f(x)dx +
b
Z
c
f(x)dx.
Theorem 1.4.10. Let f : [a, b] → R be a function, and f*
: [a, b] → R be another
function which coincides with f on [a, b], except on a finite set of points. If f*
is Riemann
integrable on [a, b], then f is Riemann integrable on [a, b] and
b
Z
a
f(x)dx =
b
Z
a
f*
(x)dx.
Theorem 1.4.11 (integration by parts). If f , g : [ a, b ] → R are C1
functions, then
b
Z
a
f(x)g0
(x) dx = f(x)g(x)| b
a −
b
Z
a
f0
(x)g(x) dx.
Theorem 1.4.12 (change of variable). Let φ : [a, b] → [c, d] be a C1
function, and let
f : [c, d] → R be a continuous function. Then
b
Z
a
f(φ(t)) · φ0
(t) dt =
φ(b)
Z
φ(a)
f(x) dx. (1.4.1)
We end this section by the next multidimensional variant of Taylor Theorem. In what
follows, the equality is understood on components (i.e., for every function fi : Rp
→
R, i = 1, p, where f = (f1, ..., fp)).
Theorem 1.4.13. Suppose f : Rp
→ Rp
is continuously differentiable on some convex
open set D and that x, x + y ∈ D. Then there is t ∈ (0, 1) such that
f(x + y) = f(x) +
1
Z
0
∇f(x + ty)(y) dt.](https://image.slidesharecdn.com/25523077-250512184110-75773370/85/An-Introduction-To-Nonlinear-Optimization-Theory-Marius-Durea-Radu-Strugariu-31-320.jpg)
![© 2014 Marius Durea and Radu Strugariu
This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivs 3.0 License.
2 Nonlinear Analysis Fundamentals
In this chapter we study convex sets and functions, convex cones, the Bouligand
tangent cone to a set at a point, and semicontinuous functions. We prove fundamental
results, which will be the main investigation tools in subsequent chapters. The Farkas
Lemma and its consequences will be decisive for establishing optimality conditions
in Karush-Kuhn-Tucker form, while the Banach fixed point Principle will be used for
discussing some optimization algorithms. We also study some important inequalities
related to the study of convex functions.
2.1 Convex Sets and Cones
Definition 2.1.1. One says that a nonempty set D ⊂ Rp
is convex if for every x, y ∈ D,
[x, y] = {αx + (1 − α)y | α ∈ [0, 1]} ⊂ D.
In other words, D is convex if and only if together with two points a1, a2, it contains
the whole segment [a1, a2]. It is sufficient to take α ∈ (0, 1). By mathematical in-
duction one can show that if D is convex, then for every n ∈ N*
, x1, x2, ..., xn ∈ D,
α1, α2, ..., αn ∈ [0, 1] with
Pn
i=1 αi = 1 :
n
X
i=1
αixi ∈ D.
A sum like the one above is called a convex combination of the elements (xi). In R, the
convex sets are intervals.
One of the main objects for our study is defined next.
Definition 2.1.2. A nonempty subset K ⊂ Rp
is a cone if the next relation holds:
∀y ∈ K, ∀λ ∈ R+ := [0, ∞) : λy ∈ K.
According to the definition, every cone contains the origin.
Proposition 2.1.3. A cone C is convex if and only if C + C = C.
Proof Suppose first that C is a convex cone. As 0 ∈ C, it is clear that C ⊂ C + C. Let
u ∈ C + C. Then there exist c1, c2 ∈ C such that c1 + c2 = u. But, using the properties
of C and the obvious relation
u = 2
2−1
c1 + 2−1
c2
,](https://image.slidesharecdn.com/25523077-250512184110-75773370/85/An-Introduction-To-Nonlinear-Optimization-Theory-Marius-Durea-Radu-Strugariu-32-320.jpg)
![20 | Nonlinear Analysis Fundamentals
which is the desired conclusion.
(iii) If x ∈ A, then ax := x is the unique element having the previously introduced
property. Let x ∉ A. As d(x, A) is a real number, there exists r 0 such that A1 :=
A ∩ D (x, r) ≠ ∅. Since A1 is a compact set and the function g : A1 → R, g (y) =
d (x, y) is continuous, according to Weierstrass Theorem, g attains its minimum on
A1, i.e., there exists ax ∈ A1 with g (ax) = infy∈A1
g (y) = d(x, A1). Now, one can check
that d (x, A1) = d (x, A) and the first conclusion follows. Suppose that, moreover, A
is convex. If x ∈ A, there is nothing to prove. Take x ∉ A. Consider a1, a2 ∈ A with
d (x, A) = kx − a1k = kx − a2k . Using the parallelogram law we know:
(x − a1) + (x − a2)
2
+ (x − a1) − (x − a2)
2
= 2 kx − a1k2
+ 2 kx − a2k2
,
i.e.,
k2x − a1 − a2k2
+ ka2 − a1k2
= 4d2
(x, A),
and dividing by 4 one gets
x −
a1 + a2
2
2
+ 4−1
ka2 − a1k2
= d2
(x, A).
Since A is convex, 2−1
(a1+a2) ∈ A, hence x −
a1 + a2
2
2
≥ d2
(x, A). This relation and
the previous equality show that ka2 − a1k = 0, hence a1 = a2. The proof of uniqueness
is now complete. Let us prove now that ax verifies the relation hx − ax, u − axi ≤ 0 for
any u ∈ A. For this, take u ∈ A. Then for every α ∈ (0, 1], one has
v = αu + (1 − α)ax ∈ A.
Hence,
kx − axk ≤ x − αu − (1 − α)ax = x − ax − α(u − ax) ,
and, consequently,
kx − axk2
≤ kx − axk2
− 2α hx − ax, u − axi + α2
ku − axk2
.
After reducing terms and dividing by α 0, we can see that
0 ≤ −2 hx − ax, u − axi + α ku − axk2
.
If we let α → 0, the desired inequality follows. For the converse, if an element a ∈ A
satisfies hx − a, u − ai ≤ 0 for any u ∈ A, then for every v ∈ A one has
kx − ak2
− kx − vk2
= 2 hx − a, v − ai − ka − vk2
≤ 0,
hence a coincides with ax. The proof is now complete.
In the case when A is closed, then for x ∈ Rp
one denotes the projection set of x
on A by
prA x :=
a ∈ A | d(x, A) = kx − ak .](https://image.slidesharecdn.com/25523077-250512184110-75773370/85/An-Introduction-To-Nonlinear-Optimization-Theory-Marius-Durea-Radu-Strugariu-34-320.jpg)
![Convex Sets and Cones | 21
If, moreover, A is convex, then, according to the above theorem, this set consists of
only one element, which we still denote by prA x, and we call this the projection of x
on A.
Let S ⊂ Rp
be a nonempty set. The polar of S is the set
S−
:= {u ∈ Rp
| hu, xi ≤ 0, ∀x ∈ S}.
It is easy to observe that S−
is a closed convex cone and that, in general, S ⊂ (S−
)−
. If
we consider the reverse inclusion, the next result follows.
Theorem 2.1.6. Let C ⊂ Rp
be a closed convex cone. Then C = (C−
)−
.
Proof Consider z ∈ (C−
)−
and z = prC z. We will prove that z = z. From the last part of
Theorem 2.1.5, for any c ∈ C, one has
hz − z, c − zi ≤ 0.
As 0 ∈ C and 2z ∈ C, we deduce
− hz, z − zi ≤ 0, hz, z − zi ≤ 0
hence
hz − z, ci ≤ 0
for every c ∈ C, i.e., z − z ∈ C−
. As z ∈ (C−
)−
, one gets
hz, z − zi ≤ 0.
But
kz − zk2
= hz − z, zi − hz − z, zi ≤ 0
which means that z = z, so z ∈ C. This establishes the theorem.
Example 2.1.7. 1. Consider S = {(x, 0) ∈ R2
| x ≥ 0}. One can observe that S−
=
{(x, y) ∈ R2
| x ≤ 0}. Obviously, (S−
)−
= S−
.
2. The polar of R2
+ := {(x, y) ∈ R2
| x, y ≥ 0} is R2
− := {(x, y) ∈ R2
| x, y ≤ 0}. The
polar of the set S = {(x, 0) ∈ R2
| x ≥ 0} ∪ {(0, y) ∈ R2
| y ≥ 0} is also R2
−. From this
example one can see that, in general, S−
1 = S−
2 does not imply S1 = S2.
The next result, which has an algebraic character, it was obtained by the Hungarian
mathematician Julius Farkas in 1902.
Theorem 2.1.8 (Farkas’ Lemma). Let n ∈ N*
, (φi)i∈1,n ⊂ L(Rp
, R) and φ ∈ L(Rp
, R).
Then
∀x ∈ Rp
: [φ1(x) ≤ 0, . . . , φn(x) ≤ 0] ⇒ φ(x) ≤ 0 (2.1.1)
if and only if there exists (αi)i∈1,n ⊂ [0, ∞) such that φ =
Pn
i=1 αiφi.](https://image.slidesharecdn.com/25523077-250512184110-75773370/85/An-Introduction-To-Nonlinear-Optimization-Theory-Marius-Durea-Radu-Strugariu-35-320.jpg)
![22 | Nonlinear Analysis Fundamentals
Proof The converse implication is obvious. We prove the other one by induction for
n ≥ 1. Define the proposition P(n) which says that for every φ, φ1, . . . , φn ∈ L(Rp
, R)
satisfying (2.1.1), there exist (αi)i∈1,n ⊂ [0, ∞) such that φ =
Pn
i=1 αiφi.
Let us prove that P(1) is true. Indeed, let φ, φ1 ∈ L(Rp
, R) such that
φ1(x) ≤ 0 ⇒ φ(x) ≤ 0.
If φ = 0 then, obviously, φ = 0φ1. Suppose φ ≠ 0. Then, by the assumption:
φ1(x) = 0 ⇔ [φ1(x) ≤ 0, φ1(−x) ≤ 0] ⇒ [φ(x) ≤ 0, φ(−x) ≤ 0] ⇔ φ(x) = 0,
hence Ker φ1 ⊂ Ker φ. Since φ ≠ 0, one has φ1 ≠ 0, so there exists x1 ∈ Rp
with
φ1(x) = −1. Also by the assumption, φ(x) ≤ 0. Take x ∈ Rp
arbitrarily. Then it is easy
to verify that
x + φ1(x)x ∈ Ker φ1,
hence
x + φ1(x)x ∈ Ker φ,
i.e.,
φ x + φ1(x)x
= 0,
which proves that
φ(x) = −φ(x)φ1(x).
Notice that x was arbitrarily chosen, so the desired relation is proved for α1 := −φ(x) ≥
0.
Suppose now that P(n) is true for a fixed n ≥ 1 and we will try to prove that P(n+1)
is true.
Take φ, φ1, . . . , φn, φn+1 ∈ L(Rp
, R) such that
∀x ∈ Rp
: [φ1(x) ≤ 0, . . . , φn(x) ≤ 0, φn+1(x) ≤ 0] ⇒ φ(x) ≤ 0. (2.1.2)
If
∀x ∈ Rp
: [φ1(x) ≤ 0, . . . , φn(x) ≤ 0] ⇒ φ(x) ≤ 0, (2.1.3)
then, from P(n), there exist (αi)i∈1,n ⊂ [0, ∞) such that φ =
Pn
i=1 αiφi; take αn+1 := 0
and the conclusion follows.
Suppose relation (2.1.3) is not satisfied. Then there exists x ∈ Rp
such that φ(x) 0
and φi(x) ≤ 0 for any i ∈ 1, n. As (2.1.2) holds, φn+1(x) 0; we may suppose (by
multiplying by the appropriate positive scalar) that φn+1(x) = 1. But
φn+1(x − φn+1(x)x) = 0, ∀x ∈ Rp
,
and from (2.1.2) we deduce
∀x ∈ Rp
: [φ1(x − φn+1(x)x) ≤ 0, . . . , φn(x − φn+1(x)x) ≤ 0] ⇒ φ(x − φn+1(x)x) ≤ 0.
(2.1.4)](https://image.slidesharecdn.com/25523077-250512184110-75773370/85/An-Introduction-To-Nonlinear-Optimization-Theory-Marius-Durea-Radu-Strugariu-36-320.jpg)
![Convex Sets and Cones | 23
Take φ0
i := φi − φi(x)φn+1 for i ∈ 1, n and φ0
:= φ − φ(x)φn+1, and then relation (2.1.4)
becomes
∀x ∈ X : [φ0
1(x) ≤ 0, . . . , φ0
n(x) ≤ 0] ⇒ φ0
(x) ≤ 0.
As P(n) is true, there exist (αi)i∈1,n ⊂ [0, ∞) such that φ0
=
Pn
i=1 αiφ0
i . We deduce that
φ − φ(x)φn+1 =
n
X
i=1
αi
φi − φi(x)φn+1
,
hence φ =
Pn+1
i=1 αiφi, where αn+1 = φ(x) −
Pn
i=1 αiφi(x) ≥ 0 from the choice of x and
from the fact that αi ≥ 0 for any i ∈ 1, n). The proof is now complete.
Throughout this book we shall use several different concepts of tangent vectors to
a set at a point. We introduce now one of these concepts.
Definition 2.1.9. Let M ⊂ Rp
be a nonempty set and x ∈ cl M. One says that a
vector u ∈ Rp
is tangent in the sense of Bouligand to the set M at x if there exist
(tn) ⊂ (0, ∞), tn → 0 and (un) → u such that for any n ∈ N, one has
x + tnun ∈ M.
It is sufficient that the above inclusion holds for every n ∈ N sufficiently large.
Theorem 2.1.10. The set, denoted by TB(M, x), which contains all the tangent vectors
to the set M at x is a closed cone, which we call the Bouligand tangent cone (or the
contingent cone) to the set M at the point x.
Proof Let us prove first that 0 ∈ TB(M, x). If x ∈ M, then the assertion trivially
follows, because it is sufficient to take (un) constantly equal to 0. If x ∉ M, then
there exists (xn)n∈N ⊂ M such that xn → x, and we consider tn :=
p
kxn − xk and
un :=
p
kxn − xk
−1
(xn − x) for every n ∈ N. Since tn → 0 and un → 0, one obtains
the conclusion.
Consider now u ∈ TB(M, x) and λ 0. According to the definition, there exist
(tn) ⊂ (0, ∞), tn → 0 and (un) → u such that for every n ∈ N,
x + tnun ∈ M.
This is equivalent to
x +
tn
λ
(λun) ∈ M.
As
tn
λ
→ 0 and (λun) → λu, one deduces that λu ∈ TB(M, x), hence TB(M, x) is
a cone. We prove that the closure of TB(M, x) is contained in TB(M, x). Take (un) ⊂](https://image.slidesharecdn.com/25523077-250512184110-75773370/85/An-Introduction-To-Nonlinear-Optimization-Theory-Marius-Durea-Radu-Strugariu-37-320.jpg)
![Convex Sets and Cones | 25
hence u0
n :=
vkn
n − x
tn
n→∞
→ u. But
x + tnu0
n = vkn
n ∈ M,
hence u ∈ TB(M, x). The second part of the conclusion easily follows: if x ∈ int M,
then for every u ∈ Rp
and every (tn) ⊂ (0, ∞), tn → 0, one has x + tnun ∈ M for any
n sufficiently large. This shows, in particular, that u ∈ TB(M, x), and the conclusion
follows.
In general, the Bouligand tangent cone is not convex and the relation TB(M, x) =
Rp
can be satisfied, even if x ∉ int M.
Example 2.1.13. 1. Consider the set M ⊂ R2
, M = {(x, y) | x ≥ 0, y = 0} ∪ {(x, y) | x =
0, y ≥ 0}. Then TB(M, (0, 0)) = M is not a convex set.
2. Let set M represent the plane domain bounded by the curve (the cardioid) which
has the parametric representation
(
x = −2 cos t + cos 2t + 1
y = 2 sin t − sin 2t
, t ∈ [0, 2π].
Then TB(M, (0, 0)) = R2
, but (0, 0) ∉ int M.
Figure 2.1: The cardioid.](https://image.slidesharecdn.com/25523077-250512184110-75773370/85/An-Introduction-To-Nonlinear-Optimization-Theory-Marius-Durea-Radu-Strugariu-39-320.jpg)
![30 | Nonlinear Analysis Fundamentals
subsequence which converges to a nonzero element, which must be from A∞
). Let us
observe also that if A is a cone, then A∞
= TB(A, 0) = cl A.
The next result concerns decomposition.
Theorem 2.1.18. Let A ⊂ Rp
be a nonempty closed set.
(i) If 0 ∉ A, then cl cone A = cone A ∪ A∞
.
(ii) If 0 ∈ A, then cl cone A = cone A ∪ A∞
∪ TB(A, 0).
Proof (i) Suppose that 0 ∉ A. It is clear that cone A ⊂ cl cone A. From the def-
inition of A∞
, one also has that A∞
⊂ cl cone A. For the reverse inclusion, take
(un) ⊂ cone A, un → u. We must prove that u ∈ cone A ∪ A∞
. If u = 0, then the
relation u ∈ cone A is obvious. Suppose that u ≠ 0, then for every n ∈ N, there ex-
ist tn ≥ 0 and an ∈ A such that un = tnan. If (an) is unbounded, one can pass to a
subsequence (ank ), where kank k → ∞. Hence tnk → 0, and u ∈ A∞
. Suppose (an) is
bounded. Since 0 ∉ A and A is closed, there exists γ 0 such that kank ≥ γ for every
n. One can deduce that (tn) is bounded, so it converges (on a subsequence (tnk ), even-
tually) to a number t ≥ 0. If t = 0, then unk → 0 = u (a situation which is excluded at
this point of the proof). Accordingly, t 0 and
ank − t−1
u = t−1
ktank − uk = t−1
tnk ank − u + (t − tnk )ank
≤ t−1
ktnk ank − uk + |t − tnk | t−1
kank k → 0.
Hence ank → t−1
u and since A is closed, u ∈ cone A. The proof of this part is complete.
(ii)Theinclusion cone A∪A∞
∪TB(A, 0) ⊂ cl cone A isobvious.Take u ∈ cl cone A.
In the above it is possible as well that (an) converges to 0. Then tn → ∞, hence u ∈
TB(A, 0), which completes the proof.
We finish with the following characterization result:
Corollary 2.1.19. Let A ⊂ Rp
be a nonempty closed set.
(i) If 0 ∉ A, then cone A is closed if and only if A∞
⊂ cone A.
(ii) If 0 ∈ A, then cone A is closed if and only if A∞
∪ TB(A, 0) ⊂ cone A.
2.2 Convex Functions
2.2.1 General Results
In this section we present the special class of convex functions. These functions are
defined on convex sets.
Definition 2.2.1. Let D ⊂ Rp
be a convex set. One says that a function f : D → R is
convex if
f(λx + (1 − λ)y) ≤ λf(x) + (1 − λ)f(y), ∀x, y ∈ D, ∀λ ∈ [0, 1]. (2.2.1)](https://image.slidesharecdn.com/25523077-250512184110-75773370/85/An-Introduction-To-Nonlinear-Optimization-Theory-Marius-Durea-Radu-Strugariu-44-320.jpg)
![Convex Functions | 31
It is clear that in the above definition it is sufficient to take λ ∈ (0, 1).
As said before, in R the convex sets are exactly the intervals. In this framework, the
convexity has the following geometric meaning: for every two points x, y ∈ D, x y,
the graph of the restriction of f to the [x, y] interval lies below the line segment joining
the points (x, f(x)) and (y, f(y)). This can be written as follows: for every u ∈ [x, y],
f(u) ≤ f(x) +
f(y) − f(x)
y − x
(u − x), (2.2.2)
inequality which can be deduced from (2.2.1) by replacing λ with the value given by
the relation u = λx +(1− λ)y. Therefore, (2.2.1) and (2.2.2) are equivalent (for functions
defined on R).
Definition 2.2.2. Let D ⊂ Rp
be a convex set. One says that a function f : D → R is
concave if −f is convex.
All of the properties of concave functions can be easily deduced from the similar prop-
erties of the convex functions, so in what follows we will consider only the later case.
We first deduce some general properties of the convex functions.
Proposition 2.2.3. Let D ⊂ Rp
be a convex set and f : D → R. The following relations
are equivalent:
(i) f is convex;
(ii) the epigraph of f ,
epi f := {(x, t) ∈ D × R | f(x) ≤ t},
is a convex subset of Rp
× R;
(iii) for any x, y ∈ D, define
Ix,y := {t ∈ R | tx + (1 − t)y ∈ D};
then the function φx,y : Ix,y→ R, φx,y(t) = f(tx + (1 − t)y) is convex.
Proof We prove first the implication from (i) to (ii). Take λ ∈ (0, 1) and (x, t), (y, s) ∈
epi f . By the convexity of D, one knows that λx +(1− λ)y ∈ D, and using the convexity
of f , one can say:
f(λx + (1 − λ)y) ≤ λf(x) + (1 − λ)f(y) ≤ λt + (1 − λ)s,
i.e., λx + (1 − λ)y, λt + (1 − λ)s
∈ epi f . Therefore, epi f is a convex set.
We prove now the converse implication. Take x, y ∈ D and λ ∈ [0, 1]. Then
(x, f(x)), (y, f(y)) ∈ epi f and by assumption, λ(x, f(x)) + (1 − λ)(y, f(y)) ∈ epi f , hence
f(λx + (1 − λ)y) ≤ λf(x) + (1 − λ)f(y),
which shows that f is a convex function.](https://image.slidesharecdn.com/25523077-250512184110-75773370/85/An-Introduction-To-Nonlinear-Optimization-Theory-Marius-Durea-Radu-Strugariu-45-320.jpg)
![32 | Nonlinear Analysis Fundamentals
We prove now the equivalence between (i) and (iii). Observe first that Ix,y is an
interval which contains [0, 1]. Suppose that f is convex and take u, v ∈ Ix,y, λ ∈ [0, 1].
One knows that:
φx,y(λu + (1 − λ)v) = f([λu + (1 − λ)v]x + [1 − λu − (1 − λ)v]y)
= f(λ(ux + (1 − u)y) + (1 − λ)(vx + (1 − v)y))
≤ λf(ux + (1 − u)y) + (1 − λ)f(vx + (1 − v)y)
= λφx,y(u) + (1 − λ)φx,y(v).
For the converse implication, take x, y ∈ D and t ∈ [0, 1]. Then φx,y is convex, hence
for every λ ∈ [0, 1], u, v ∈ Ix,y
φx,y(λu + (1 − λ)v) ≤ λφx,y(u) + (1 − λ)φx,y(v)
= λf(ux + (1 − u)y) + (1 − λ)f(vx + (1 − v)y).
By taking u = 1, v = 0, λ = t we deduce that
φx,y(t) ≤ tf(x) + (1 − t)f(y),
hence f is convex.
Theorem 2.2.4. Let D ⊂ Rp
be a convex set and f : D → R be a convex function. Then
f is continuous at every interior point of D.
Proof Take x ∈ int D. A translation permits us to consider the case x = 0. We prove
first that f is bounded on a neighborhood of 0. If we denote by (ei)i∈1,p the canonical
base of Rp
, then there exists an a 0 such that aei and −aei are in D for any i ∈ 1, p.
Under these conditions, the set
V :=
(
x ∈ Rp
| x =
p
X
i=1
xiei, |xi|
a
p
, ∀i ∈ 1, p
)
is a neighborhood of 0 contained in D. For x ∈ V, there exist (xi)i∈1,p with |xi| a
p ,
i ∈ 1, p and x =
Pp
i=1 xiei. Suppose first that xi ≠ 0 for any i ∈ 1, p. One has
f(x) = f
p
X
i=1
xiei
!
= f
p
X
i=1
|xi|
a
a
xi
|xi|
ei + 1 −
p
X
i=1
|xi|
a
!
0
!
≤
p
X
i=1
|xi|
a
f
a
xi
|xi|
ei
+ 1 −
p
X
i=1
|xi|
a
!
f(0)
≤ max{f (aei) , f(−aei) | i ∈ 1, p} + f(0) .
We can now observe that if there are indices i for which xi = 0, then these can be
excluded from the above calculations, and the estimation holds.](https://image.slidesharecdn.com/25523077-250512184110-75773370/85/An-Introduction-To-Nonlinear-Optimization-Theory-Marius-Durea-Radu-Strugariu-46-320.jpg)
![Convex Functions | 35
Proof In the case of real functions of one variable, the equivalence between the mono-
tonicity of f0
and the sign of f00
is sufficient to prove that f is convex if and only if f0
is
increasing on I. If f is convex, the monotonicity of the derivative follows from Propo-
sition 2.2.6. Conversely, suppose that f0
is increasing and we prove that f is convex.
Take a, b ∈ I. Define g : [a, b] → R given by
g(x) = f(x) − f(a) − (x − a)
f(b) − f(a)
b − a
.
Obviously, g(a) = g(b) = 0, and
g0
(x) = f0
(x) −
f(b) − f(a)
b − a
.
The function f satisfies the conditions of Lagrange Theorem on [a, b], hence there
exists c ∈ (a, b) such that
f(b) − f(a)
b − a
= f0
(c).
Consequently, g0
(x) = f0
(x) − f0
(c). From the monotonicity of f0
, we deduce that g is
decreasing on (a, c) and increasing on (c, b), and since g(a) = g(b) = 0, we know that
g is negative on the whole interval [a, b]. Take x ∈ (a, b). Then there exists λ ∈ (0, 1)
such that
x = λa + (1 − λ)b.
By replacing x in the expression of g and taking into account that g(x) ≤ 0, we deduce
that
f(λa + (1 − λ)b) − f(a) − (1 − λ)(b − a)
f(b) − f(a)
b − a
≤ 0,
relation which reduces to the definition of the convexity.
Example 2.2.8. Based on the above result, one deduces the convexity of the following
functions: f : R → R, f(x) = ax + b, with a, b ∈ R; f : (0, ∞) → R, f(x) = − ln x;
f : (0, ∞) → R, f(x) = x ln x; f : (0, ∞) → R, f(x) = xa
, a ≥ 1; f : R → R, f(x) = ex
;
f : ( − 1, 1) → R, f(x) = −
√
1 − x2; f : (0, π) → R, f(x) = sin−1
x.
Another example is given by the next result.
Proposition 2.2.9. Let D ⊂ Rp
be a nonempty convex set. Then the function dD : Rp
→
R given by dD(x) = d(x, D) is convex.
Proof Take x, y ∈ Rp
and α ∈ [0, 1]. For any ε 0, there exist dx,ε, dy,ε ∈ D such that
kdx,ε − xk dD(x) + ε
kdy,ε − yk dD(y) + ε.](https://image.slidesharecdn.com/25523077-250512184110-75773370/85/An-Introduction-To-Nonlinear-Optimization-Theory-Marius-Durea-Radu-Strugariu-49-320.jpg)
![36 | Nonlinear Analysis Fundamentals
Using the convexity of D, one knows:
dD(αx + (1 − α)y) ≤ αx + (1 − α)y − (αdx,ε + (1 − α)dy,ε)
≤ α kdx,ε − xk + (1 − α) kdy,ε − yk
αdD(x) + (1 − α)dD(y) + ε.
As ε is arbitrarily chosen, we may pass to the limit for ε → 0 and the conclusion
follows.
We now characterize differentiable convex functions in the general case.
Theorem 2.2.10. Let D ⊂ Rp
be an open convex set and f : D → R be a function.
(i) If f is differentiable on D, then f is convex if and only if for any x, y ∈ D,
f(y) ≥ f(x) + ∇f(x)(y − x).
(ii) If f is twice differentiable on D, then f is convex if and only if for every x ∈ D
and y ∈ Rp
, one has
∇2
f(x)(y, y) ≥ 0.
Proof (i) Consider first the case when p = 1. Fix x ∈ D with y ≠ x and take λ ∈ (0, 1].
Since f is convex, we get
f (x + λ (y − x)) = f ((1 − λ) x + λy)
≤ (1 − λ) f(x) + λf(y) = f(x) + λ (f (y) − f (x)) .
Consequently,
f (x + λ (y − x)) − f (x)
λ (y − x)
· (y − x) ≤ f (y) − f (x) .
Passing to the limit for λ → 0, one gets f0
(x) (y − x) ≤ f (y) − f (x).
We pass now to the general case. For x ∈ D, consider the function φy,x from Propo-
sition 2.2.3, which we already know that is convex. Moreover, φy,x is differentiable on
the open interval Iy,x, and
φ0
y,x(t) = ∇f(ty + (1 − t)x)(y − x).
According to the preceding step,
φy,x(1) ≥ φy,x(0) + φ0
y,x(0)
which means that
f(y) ≥ f(x) + ∇f(x)(y − x).](https://image.slidesharecdn.com/25523077-250512184110-75773370/85/An-Introduction-To-Nonlinear-Optimization-Theory-Marius-Durea-Radu-Strugariu-50-320.jpg)
![Convex Functions | 37
Conversely, fix x, y ∈ D and λ ∈ [0, 1]. Therefore, by assumption,
f (x) ≥ f λx + (1 − λ)y
+ (1 − λ)∇f λx + (1 − λ)y
(x − y)
and
f (y) ≥ f λx + (1 − λ)y
+ λ∇f λx + (1 − λ)y
(y − x) .
Multiplying the first inequality by λ, the second one by (1 − λ), and summing up
the new inequalities, one obtains
λf (x) + (1 − λ)f (y) ≥ f λx + (1 − λ)y
,
which proves that f is convex.
(ii) The case p = 1 is proved in Theorem 2.2.7. Now, in order to pass to the general
case, take x ∈ D, y ∈ Rp
. Suppose f is convex. Since D is open, there exists an α 0
such that u := x + αy ∈ D. According to the assumption, φu,x is convex, and taking
into account the case that we have already studied, φ00
u,x(t) ≥ 0 for any t ∈ Iu,x. For
t = 0, one deduces that
0 ≤ φ00
u,x(0) = ∇2
f(x)(u − x, u − x),
and the conclusion follows. Conversely, for x, y ∈ D and t ∈ Ix,y, φ00
x,y(t) ≥ 0. From the
case p = 1, we get that φx,y is convex, hence f is convex. The proof is now complete.
From these results, one may observe that some properties of the convex functions
have a global character, an aspect which will persist in subsequent sections.
At the end of this subsection, we will discuss a property which is stronger than
convexity.
Definition 2.2.11. Let D ⊂ Rp
be a convex set. One says that a function f : D → R is
strictly convex if
f(λx + (1 − λ)y) λf(x) + (1 − λ)f(y), ∀x, y ∈ D, x ≠ y, ∀λ ∈ (0, 1).
Definition 2.2.12. Let D ⊂ Rp
be a convex set. One says that a function f : D → R is
strictly concave if −f is strictly convex.
Every strictly convex function is convex, but the converse is false. To see this, consider
a convex function which is constant on an interval. Again, the properties of the strictly
concave functions easily follow from the corresponding ones of the strictly convex
functions.
By the use of very similar arguments as in the proofs of preceding results, one can
deduce the next characterizations.](https://image.slidesharecdn.com/25523077-250512184110-75773370/85/An-Introduction-To-Nonlinear-Optimization-Theory-Marius-Durea-Radu-Strugariu-51-320.jpg)
![Convex Functions | 39
Theorem 2.2.17. Let I be a nondegenerate interval (i.e., not a singleton set) and f : I →
R be convex. Then either f is monotone on int I, or there exists x ∈ int I such that f is
decreasing on I ∩ (−∞, x], and increasing on I ∩ [x, ∞).
Proof Because of relation (2.2.2), it is sufficient to restrict our attention to the case
when I is open, i.e., I = int I. Suppose that f is not monotone on I. Then there exist
a, b, c ∈ I, a b c such that f(a) f(b) f(c) or f(a) f(b) f(c). The second
situation cannot hold, since in that case, using (2.2.2), one would have
f(b) ≤ f(a) +
f(c) − f(a)
c − a
(b − a) =
f(a)(c − b) + f(c)(b − a)
c − a
f(b).
Hence, f(a) f(b) f(c). As f is continuous on [a, c], its minimum on this interval
must be attained at a point x (from the Weierstrass Theorem). Take x ∈ I ∩ (−∞, a).
According to Proposition 2.2.5,
f(x) − f(x)
x − x
≤
f(a) − f(x)
a − x
,
i.e.,
(x − a)f(x) ≥ (x − a)f(x) + (x − x)f(a) ≥ (x − a)f(x),
hence f(x) ≤ f(x). Similarly, one can prove that f(x) ≤ f(x) for x ∈ I ∩ (c, ∞). It follows
that f(x) = inf f(I). We next prove that f is decreasing on I ∩ (−∞, x). Take u, v ∈
I ∩ (−∞, x), u v. On one hand,
f(v) − f(u)
v − u
≤
f(x) − f(u)
x − u
,
and on the other hand,
f(x) − f(u)
x − u
=
f(u) − f(x)
u − x
≤
f(v) − f(x)
v − x
≤ 0,
hence f(v) − f(u) ≤ 0, which is exactly what we wanted to prove. Similarly, one can
show that f is increasing on I ∩ (x, +∞). The continuity of f on I finalizes the proof.
As we saw before, a convex function defined on an interval can have discontinu-
ities only at the extremities of the interval. The previous theorem allows us to consider
a different function in those eventual discontinuity points, without losing the convex-
ity. In this way one obtains the next consequence.
Corollary 2.2.18. Let a, b ∈ R, a b and f : [a, b] → R be a convex function. Then
there exist limx→a+ f(x) and limx→b− f(x), and the function
f(x) =
limx→a+ f(x), x = a
f(x), x ∈ (a, b)
limx→b− f(x), x = b
is convex and continuous on [a, b].](https://image.slidesharecdn.com/25523077-250512184110-75773370/85/An-Introduction-To-Nonlinear-Optimization-Theory-Marius-Durea-Radu-Strugariu-53-320.jpg)
![40 | Nonlinear Analysis Fundamentals
From the proof of Theorem 2.2.17 one can also obtain the next result, which will be
restated in a more general framework in the next chapter.
Corollary 2.2.19. Let I ⊂ R be a nondegenerate interval. If f : I → R is a convex and
non-monotone function, then it has a global minimum on int I.
The next proposition, sometimes called the Jensen inequality, follows by applying the
definitions and the mathematical induction principle.
Proposition 2.2.20. Let D ⊂ Rp
be a convex set and f : D → R. If the function f is
convex, then
f(λ1x1 + ... + λmxm) ≤ λ1f(x1) + ... + λmf(xm)
for any m ∈ N*
, x1, ..., xm ∈ D, λ1, ..., λm ≥ 0, λ1 + ... + λm = 1. The inequality is
strict if the function f is strictly convex, at least two of the points (xk) are different and
the corresponding scalars (λk) are strictly positive.
Actually, for the convexity of a continuous function it is sufficient that the inequality
from the definition is satisfied for λ = 2−1
.
Theorem 2.2.21. Let I ⊂ R be an interval and f : I → R be a continuous function. The
function f is convex if and only if
f
x + y
2
≤
f(x) + f(y)
2
, ∀x, y ∈ I. (2.2.3)
Proof The necessity of the condition (2.2.3) is obvious. Let us prove that it is also suf-
ficient. Suppose, by contradiction, that f is not convex, which means (2.2.2) is not
satisfied. Then there exist x, y ∈ I, x y and u ∈ (x, y) such that
f(u) f(x) +
f(y) − f(x)
y − x
(u − x). (2.2.4)
Observe by the relation (2.2.4) that the cases u = x and u = y cannot hold. Consider
then g : [x, y] → R,
g(t) = f(t) − f(x) −
f(y) − f(x)
y − x
(t − x),
which is continuous and satisfies the relations g(x) = g(y) = 0. By (2.2.4) and Weier-
strass’ Theorem, there exists z ∈ (x, y) such that g(z) = supt∈[x,y] g(t) 0. Denote
by
w := inf{z ∈ (x, y) | g(z) = sup
t∈[x,y]
g(t)}.
By the continuity of g, it follows that g(w) = supt∈[a,b] g(t) 0, and hence w ∈ (x, y).
Consequently, there exists h 0 such that w + h, w − h ∈ (x, y). But w = 2−1
(w + h) +](https://image.slidesharecdn.com/25523077-250512184110-75773370/85/An-Introduction-To-Nonlinear-Optimization-Theory-Marius-Durea-Radu-Strugariu-54-320.jpg)
An Introduction To Nonlinear Optimization Theory Marius Durea Radu Strugariu
- 1. An Introduction To Nonlinear Optimization Theory Marius Durea Radu Strugariu download https://ebookbell.com/product/an-introduction-to-nonlinear- optimization-theory-marius-durea-radu-strugariu-51046154 Explore and download more ebooks at ebookbell.com
- 2. Here are some recommended products that we believe you will be interested in. You can click the link to download. An Introduction To Nonlinear Analysis Schechter M https://ebookbell.com/product/an-introduction-to-nonlinear-analysis- schechter-m-2047344 An Introduction To Nonlinear Partial Differential Equations 2nd Edition J David Logan https://ebookbell.com/product/an-introduction-to-nonlinear-partial- differential-equations-2nd-edition-j-david-logan-2249850 An Introduction To Nonlinear Functional Analysis And Elliptic Problems 1st Edition Antonio Ambrosetti https://ebookbell.com/product/an-introduction-to-nonlinear-functional- analysis-and-elliptic-problems-1st-edition-antonio-ambrosetti-2543308 An Introduction To Nonlinear Finite Element Analysis With Applications To Heat Transfer Fluid Mechanics And Solid Mechanics 2nd Edition Reddy https://ebookbell.com/product/an-introduction-to-nonlinear-finite- element-analysis-with-applications-to-heat-transfer-fluid-mechanics- and-solid-mechanics-2nd-edition-reddy-34081414
- 3. An Introduction To Nonlinear Analysis Theory 1st Edition Zdzisaw Denkowski https://ebookbell.com/product/an-introduction-to-nonlinear-analysis- theory-1st-edition-zdzisaw-denkowski-4592578 An Introduction To Nonlinear Analysis And Fixed Point Theory 1st Ed 2018 Hemant Kumar Pathak https://ebookbell.com/product/an-introduction-to-nonlinear-analysis- and-fixed-point-theory-1st-ed-2018-hemant-kumar-pathak-7043362 An Introduction To Nonlinear Finite Element Analysis Illustrated Edition J N Reddy https://ebookbell.com/product/an-introduction-to-nonlinear-finite- element-analysis-illustrated-edition-j-n-reddy-981524 Complex Dynamics And Morphogenesis An Introduction To Nonlinear Science 1st Edition Chaouqi Misbah Auth https://ebookbell.com/product/complex-dynamics-and-morphogenesis-an- introduction-to-nonlinear-science-1st-edition-chaouqi-misbah- auth-5741688 Optima And Equilibria An Introduction To Nonlinear Analysis 2 Ed Aubin https://ebookbell.com/product/optima-and-equilibria-an-introduction- to-nonlinear-analysis-2-ed-aubin-11640254
- 5. Marius Durea and Radu Strugariu An Introduction to Nonlinear Optimization Theory
- 7. Marius Durea and Radu Strugariu An Introduction to Nonlinear Optimization Theory | Managing Editor: Aleksandra Nowacka-Leverton Associate Editor: Vicentiu Radulescu Language Editor: Nick Rogers
- 8. Published by De Gruyter Open Ltd, Warsaw/Berlin Part of Walter de Gruyter GmbH, Berlin/Munich/Boston This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivs 3.0 license, which means that the text may be used for non-commercial purposes, provided credit is given to the author. For details go to http://creativecommons.org/licenses/by-nc-nd/3.0/. Copyright © 2014 Marius Durea and Radu Strugariu, published by de Gruyter Open ISBN 978-3-11-042603-8 e-ISBN 978-3-11-042604-5 Bibliographic information published by the Deutsche Nationalbibliothek The Deutsche Nationalbibliothek lists this publication in the Deutsche Nationalbibliografie; detailed bibliographic data are available in the Internet at http://dnb.dnb.de. Managing Editor: Aleksandra Nowacka-Leverton Associate Editor: Vicentiu Radulescu Language Editor: Nick Rogers www.degruyteropen.com
- 9. Contents Preface | vii 1 Preliminaries | 1 1.1 Rp Space | 1 1.2 Limits of Functions and Continuity | 5 1.3 Differentiability | 10 1.4 The Riemann Integral | 15 2 Nonlinear Analysis Fundamentals | 18 2.1 Convex Sets and Cones | 18 2.2 Convex Functions | 30 2.2.1 General Results | 30 2.2.2 Convex Functions of One Variable | 38 2.2.3 Inequalities | 42 2.3 Banach Fixed Point Principle | 52 2.3.1 Contractions and Fixed Points | 53 2.3.2 The Case of One Variable Functions | 62 2.4 Graves Theorem | 71 2.5 Semicontinuous Functions | 73 3 The Study of Smooth Optimization Problems | 78 3.1 General Optimality Conditions | 78 3.2 Functional Restrictions | 91 3.2.1 Fritz John Optimality Conditions | 92 3.2.2 Karush-Kuhn-Tucker Conditions | 95 3.2.3 Qualification Conditions | 102 3.3 Second-order Conditions | 109 3.4 Motivations for Scientific Computations | 113 4 Convex Nonsmooth Optimization | 117 4.1 Further Properties and Separation of Convex Sets | 117 4.2 The Subdifferential of a Convex Function | 120 4.3 Optimality Conditions | 125 5 Lipschitz Nonsmooth Optimization | 131 5.1 Clarke Generalized Calculus | 131 5.1.1 Clarke Subdifferential | 131 5.1.2 Clarke Tangent and Normal Cones | 148 5.1.3 Optimality Conditions in Lipschitz Optimization | 156 5.2 Mordukhovich Generalized Calculus | 159
- 10. vi | Contents 5.2.1 Fréchet and Mordukhovich Normal Cones | 160 5.2.2 Fréchet and Mordukhovich Subdifferentials | 167 5.2.3 The Extremal Principle | 177 5.2.4 Calculus Rules | 181 5.2.5 Optimality Conditions | 193 6 Basic Algorithms | 196 6.1 Algorithms for Nonlinear Equations | 197 6.1.1 Picard’s Algorithm | 197 6.1.2 Newton’s Method | 203 6.2 Algorithms for Optimization Problems | 206 6.2.1 The Case of Unconstrained Problems | 206 6.2.2 The Case of Constraint Problems | 213 6.3 Scientific Calculus Implementations | 223 7 Exercises and Problems, and their Solutions | 240 7.1 Analysis of Real Functions of One Variable | 240 7.2 Nonlinear Analysis | 252 7.3 Smooth Optimization | 263 7.4 Nonsmooth Optimization | 291 Bibliography | 313 List of Notations | 315 Index | 317
- 11. © 2014 Marius Durea and Radu Strugariu This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivs 3.0 License. Preface This book aims to provide a thorough introduction to smooth and nonsmooth (convex and nonconvex) optimization theory on finite dimensional normed vector spaces (Rp spaces with p ∈ N {0}). We present several important achievements of nonlinear analysis that motivate optimization problems and offer deeper insights into the further developments. Many of the results in this book hold in more general normed vector spaces, but the fundamental ideas of the theory are similar in every setting. We chose the framework of finite dimensional vector spaces since it offers the possibility to simplify some of the proofs and permits an intuitive understanding of the main ideas. This book is intended to support courses of optimization and/or nonlinear anal- ysis for undergraduate students, but we hope that graduate students in pure and ap- plied mathematics, researchers and engineers could also benefit from it. We base our hopes on the following five facts: – This book is largely self-contained: the prerequisites are the main facts of the clas- sical differential calculus for functions of several real variables and linear algebra. They are recalled in the first chapter, so that this book can be then used without other references. – We give necessary (and sometimes sufficient) optimality conditions in several cases of regularity for each problems’ data: in the case of smooth functions, con- vex nonsmooth functions, and locally Lipschitz nonconvex functions in order to cover a large part of optimization theory. – We present many deep results of nonlinear analysis under natural assumptions and proofs. – Basic theoretical algorithms and their effective implementation in Matlab, to- gether with the results of these numerical simulations are presented, and this shows both the power and practical applicability of the theory. – An extended chapter of problems and their solutions gives the reader the possi- bility to solidify theoretical facts and to have a better understanding on various aspects of the main results in this book. It should be clearly stated that we do not claim any originality in this monograph, but the selection and the organization of the material reflects our point of view on optimization theory. Based on our scientific and teaching criteria, the material of this book is organized into seven chapters which we briefly describe here. The first chapter is introductory and fixes the general framework, the notations and the prerequisites. The second chapter contains several concepts and results of nonlinear analysis which are essential to the rest of the book. Convex sets and functions, cones, the Bouli-
- 12. viii | Preface gand tangent cone to a set at a point are studied, and we give complete proofs of fun- damental results, among which Farkas Lemma, Banach Principle of fixed point and Graves Theorem. The third chapter is the first one fully dedicated to optimization problems; it presents in detail the main aspects of the theory for the case of smooth data. We present general necessary and sufficient optimality conditions of first and second- order for problems with differentiable cost functions and with geometrical or smooth functional constraints. We arrive at the famous Karush-Kuhn-Tucker optimality con- ditions and we investigate several qualification conditions needed in this celebrated result. The fourth chapter concerns the case of convex nonsmooth optimization prob- lems. We introduce here, in compensation for the missing differentiability, the concept of the subgradient and we deduce, in this setting, necessary optimality conditions in Fritz John and Karush-Kuhn-Tucker forms. The fifth chapter generalizes the theory. We work with functions that are neither differentiable nor convex, but are locally Lipschitz. This is a good setting to present Clarke and Mordukhovich generalized differentiation calculus, which finally allows us to arrive, once again, at optimality conditions with similar formulations as in the previous two chapters. The sixth chapter is dedicated to the presentation of some basic algorithms for smooth optimization problems. We show Matlab code that accurately approximate the solutions of some optimization problems or related nonlinear equations. The seventh chapter contains more than one hundred exercises and problems which are organized according to main themes of the book: nonlinear analysis, smooth optimization, nonsmooth optimization. In our presentation we used several important monographs as follows: for the- oretical expositions we mainly used (Zălinescu 1998; Pachpatte 2005; Nocedal and Wright 2006; Niculescu and Persson 2006; Rădulescu et al. 2009; Mordukhovich 2006; Hiriart-Urruty 2008; Clarke 2013; Cârjă 2003;), while for examples, problems and ex- ercises we used (Pedregal 2004; Nocedal and Write, 2006; Isaacson and Keller 1966; Hiriart-Urruty 2009; Hestenes 1975; Forsgren et al. 2002; Clarke, 1983) and (Bazaraa et al. 2006). Finally, for the Matlab numerical simulations we used (Quarteroni and Saleri 2006). In the case of the Ekeland Variational Principle, which was obtained in 1974 in the framework of general metric spaces, and whose original proof was based on an iteration procedure, the simpler proof for the case of finite dimensional vector spaces we present here was obtained in 1983 by J.-B. Hiriart-Urruty in (Hiriart-Urruty 1983). The very simple and natural proof of Farkas Lemma that is given in this book is based on the paper of D. Bartl (Bartl 2012) and on a personal communication to the authors from C. Zălinescu. The Graves theorem is taken from (Cârjă 2003).
- 13. | ix Of course, the main reference for convex analysis is the celebrated R. T. Rockafel- lar monograph (Rockafellar 1970), but we also used the books (Niculescu and Pers- son 2006; Zălinescu 1998) and (Zălinescu 2002). For the section concerning the fixed points for function of real variable, we used several problems presented in (Radulescu et al. 2009). For the section dedicated to the generalized Clarke calculus, we used the mono- graphs (Clarke 1983; Clarke 2013; Rockafellar and Wets 1998). Theorem 5.1.32 is taken from (Rockafellar 1985). For the second part of Chapter 5, dedicated to Mordukhovich calculus, we mainly used (Mordukhovich 2006). The calculus rules for the Fréchet subdifferential of difference of functions, as well as the chain rule for the Fréchet sub- differential were taken from (Mordukhovich et al. 2006). Many of the optimization problems given as exercises are taken from (Hiriart- Urruty 2008) and (Pedregal 2004), but (Hiriart-Urruty 2008) was used as well for some other theoretical examples such as the second problem from Section 3.4 or the Kan- torovich inequality. The rather complicated proof of the fact that the Mangasarian Fro- movitz condition is a qualification condition is taken from (Nocedal and Write 2006) which is used as well for presenting the sufficient optimality conditions of second order in Section 3.3. The Hardy and Carleman inequalities correspond to material in (Pachpatte 2005). Chapter 6 is dedicated to numerical algorithms. We used the monographs: (Isaac- son and Keller 1966) (for the convergence of Picard iterations and the Aitken methods), (Nocedal and Write 2006) (for the Newton and the SQP methods). For the presentation of the barrier method, we used (Forsgren et al. 2002). Acknowledgements: We would like to thank Professor Vicenţiu Rădulescu, who kindly showed us the opportunity to write this book. Then, our thanks are addressed to dr. Aleksandra Nowacka-Leverton, Managing Editor to De Gruyter Open, for her sup- port during the preparation of the manuscript, and to the Technical Department of De Gruyter Open, for their professional contribution to the final form of the monograph. We also take this opportunity to thank our families for their endless patience during the many days (including weekends) of work on this book. 01.10.2014 Marius Durea Iaşi, Romania Radu Strugariu
- 15. © 2014 Marius Durea and Radu Strugariu This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivs 3.0 License. 1 Preliminaries The aim of this chapter is to introduce the notations, notions and results which will be useful in subsequent chapters. The results will be given without proof as they refer to basic notions of mathematical analysis, differential calculus and linear alge- bra. 1.1 Rp Space Let N be the set of natural numbers and N* := N {0} . Take p ∈ N* . We denote by R the set of real numbers and we introduce the set Rp := {(x1 , x2 , ..., xp ) | xi ∈ R, ∀i ∈ 1, p}. This set can be organized as a p dimensional real vector space, with respect to the stan- dard operations defined as follows: for every x = (x1 , x2 , ..., xp ), y = (y1 , y2 , ..., yp ) ∈ Rp and every a ∈ R x + y := (x1 + y1 , x2 + y2 , ..., xp + yp ) ∈ Rp , ax := (ax1 , ax2 , ..., axp ) ∈ Rp . Recall that the canonical base in Rp is the set {e1, ..., ep} , where for any i ∈ 1, p, ei := (0, ..., 1, ..., 0), and 1 is placed on the i−th coordinate. In some situations, when there is no risk of confusion, we will use the notation with subscript indices of the components. We will extend these operations also for sets: if A, B ⊂ Rp are nonempty, α ∈ R {0} and C ⊂ R is nonempty, one defines A + B := {a + b | a ∈ A, b ∈ B}, αA := {αa | a ∈ A}, CA := {αa | α ∈ C, a ∈ A}, A − B := A + (−1)B. One can consider an element x of Rp as a matrix of dimension 1 × p. The corre- sponding transposed matrix will be denoted by xt . Also, one defines the usual scalar product of two vectors x, y ∈ Rp by hx, yi := p X i=1 xi yi = xyt . Moreover, Rp can be seen as a normed vector space (in particular, as a metric space) endowed with the Euclidean norm k·k : Rp → R+ given by kxk := p hx, xi = v u u t p X i=1 (xi)2. It is easy to prove that for every x, y ∈ Rp , the next relation (the parallelogram law) holds kx + yk2 + kx − yk2 = 2 kxk2 + 2 kyk2 .
- 16. 2 | Preliminaries The angle between two vectors x, y ∈ Rp {0} is the value θ ∈ [0, π] given by cos θ := hx, yi kxk kyk . The open (closed) ball and the sphere centered at x ∈ Rp with radius ε > 0 are given, respectively, by: B(x, ε) := {x ∈ Rp | kx − xk < ε}, D(x, ε) := {x ∈ Rp | kx − xk ≤ ε}, and S(x, ε) := {x ∈ Rp | kx − xk = ε}. One says that a subset A ⊂ Rp is bounded if it is contained in an open ball centered in the origin i.e., if there exists M > 0 such that A ⊂ B(0, M). A neighborhood of an element x ∈ Rp is a subset of Rp which contains an open ball centered in x. We denote by V(x) the class of all neighborhoods of x. Let us sum- marize some facts: – A subset of Rp is open if it is empty or it is neighborhood for all of its points. – A subset of Rp is closed if its complement with respect to Rp is open. – An element a is an interior point of the set A ⊂ Rp if A is a neighborhood of a. We denote by int A the interior of A (i.e., the set of all interior points of A). – An element a is an accumulation point (or a limit point) of A ⊂ Rp if every neigh- borhood of a has at least one element in common with the set A which is different from a. We denote by A0 the set of all limit points of A. If a ∈ A A0 , one says that a is an isolated point of A. – An element a is an adherent point (or a closure point) of A ⊂ Rp if every neigh- borhood of a has at least one element in common with the set A. We will use the notations cl A and A to denote the closure of A (i.e., the set of all the adherent points of A). – A subset of Rp is compact if it is bounded and closed. We denote by bd A the set cl A int A = cl A ∩ cl(Rp A) and we call it the boundary of A. Proposition 1.1.1. (i) A subset of Rp is open if and only if it coincides with its interior. (ii) A subset of Rp is closed if and only if it coincides with its closure. Definition 1.1.2. One says that a function f : N → Rp is a sequence of elements from Rp . The value of the function f in n ∈ N, f(n), is denoted by xn (or yn, zn, ...), and the sequence defined by f is denoted by (xn) (respectively, by (yn), (zn), ...). Definition 1.1.3. A sequence is bounded if the set of its terms is bounded.
- 17. Rp Space | 3 Definition 1.1.4. One says that (yk) is a subsequence of (xn) if for every k ∈ N, one has yk = xnk , where by (nk) one denotes a strictly increasing sequence of natural numbers (i.e., nk < nk+1 for every k ∈ N). Definition 1.1.5. One says that a sequence (xn) ⊂ Rp is convergent (or converges) if there exists x ∈ Rp such that ∀ V ∈ V(x), ∃nV ∈ N, ∀n ≥ nV : xn ∈ V. The element x is called the limit of (xn). If it exists, the limit of a sequence is unique. We will use the notations xn → x, lim n→∞ xn = x or, simplified, lim xn = x to formalize the previous definition. Proposition 1.1.6. A sequence (xn) is convergent to x ∈ Rp if and only if ∀ε > 0, ∃nε ∈ N, ∀n ≥ nε : kxn − xk < ε. Proposition 1.1.7. The sequence (xn) ⊂ Rp converges to x ∈ Rp if and only if the coordinate sequences (xi n) converge (in R) to xi for every i ∈ 1, p. Proposition 1.1.8. A sequence is convergent to x ∈ Rp if and only if all of its subse- quences are convergent to x. Proposition 1.1.9. Every convergent sequence is bounded. Proposition 1.1.10 (Characterization of the closure points using sequences). Con- sider A ⊂ Rp . A point x ∈ Rp is a closure point of A if and only if there exists a sequence (xn) ⊂ A such that xn → x. Proposition 1.1.11. The set A ⊂ Rp is closed if and only if every convergent sequence from A has its limit in A. Proposition 1.1.12. The set A ⊂ Rp is compact if and only if every sequence from A has a subsequence which converges to a point of A. Theorem 1.1.13 (Cesàro Lemma). Every bounded sequence contains a convergent sub- sequence. Definition 1.1.14. One says that (xn) ⊂ Rp is a Cauchy sequence or a fundamental sequence if ∀ε > 0, ∃nε ∈ N, ∀n, m ≥ nε : kxn − xmk < ε. The above definition can be reformulated as follows: (xn) is a Cauchy sequence if:
- 18. 4 | Preliminaries ∀ε > 0, ∃nε ∈ N, ∀n ≥ nε, ∀p ∈ N : kxn+p − xnk < ε. Theorem 1.1.15 (Cauchy). The space Rp is complete, i.e., a sequence from Rp is con- vergent if and only if it is a Cauchy sequence. The next results are specific to the case of real sequences. Definition 1.1.16. One says that a sequence (xn) of real numbers is increasing (strictly increasing, decreasing, strictly decreasing) if for every n ∈ N, xn+1 ≥ xn (xn+1 > xn, xn+1 ≤ xn, xn+1 < xn). If (xn) is either increasing or decreasing, then it is called monotone. Let R := R ∪ {−∞, +∞} be the set of extended real numbers. A neighborhood of +∞ is a subset of R which contains an interval of the form (x, +∞], where x ∈ R. The neighborhoods of −∞ are defined in a similar manner. Definition 1.1.17. (i) One says that the sequence (xn) ⊂ R has the limit equal to +∞ if ∀V ∈ V(+∞), ∃nV ∈ N, ∀n ≥ nV : xn ∈ V. (ii) One says that the sequence (xn) ⊂ R has the limit equal to −∞ if ∀V ∈ V(−∞), ∃nV ∈ N, ∀n ≥ nV : xn ∈ V. Proposition 1.1.18. (i) A sequence (xn) ⊂ R has the limit equal to +∞ if and only if ∀A > 0, ∃nA ∈ N, ∀n ≥ nA : xn > A. (ii) A sequence (xn) ⊂ R has the limit equal to −∞ if and only if ∀A > 0, ∃nA ∈ N, ∀n ≥ nA : xn < −A. Proposition 1.1.19. Let (xn), (yn), (zn) be sequences of real numbers, x, y ∈ R and n0 ∈ N. Then: (i) (Passing to the limit in inequalities) if xn → x, yn → y and xn ≤ yn for every n ≥ n0, then x ≤ y; (ii) (The boundedness criterion) if |xn − x| ≤ yn for every n ≥ n0, and yn → 0, then xn → x; (iii) if xn ≥ yn for every n ≥ n0, and yn → +∞, then xn → +∞; (iv) if xn ≥ yn for every n ≥ n0, and xn → −∞, then yn → −∞; (v) if (xn) is bounded and yn → 0, then xnyn → 0; (vi) if xn ≤ yn ≤ zn for every n ≥ n0, and xn → x, zn → x, then yn → x; (vii) xn → 0 ⇔ |xn| → 0 ⇔ x2 n → 0. We now present some fundamental results in the theory of real sequences.
- 19. Limits of Functions and Continuity | 5 Theorem 1.1.20. Every monotone real sequence has its limit in R. Moreover, if the se- quence is bounded, then it is convergent, as follows: if it is increasing, then its limit is the supremum of the set of its terms, and if it is decreasing, the limit is the infimum of the set of its terms. If it is unbounded, then its limit is either +∞ if the sequence is increasing, or −∞ if the sequence is decreasing. Theorem 1.1.21 (Weierstrass theorem for sequences). If (xn) is a bounded and mono- tone sequence of real numbers, then (xn) is convergent. Definition 1.1.22. Let (xn)n≥0 be a sequence of real numbers. An element x ∈ R is called a limit point of (xn) if there exists a subsequence (xnk ) of (xn) such that x = lim k→∞ xnk . We finalize this section with two useful convergence criteria. Proposition 1.1.23. Let (xn) be a sequence of strictly positive real numbers such that there exists lim xn+1 xn = x. If x < 1, then xn → 0, and if x > 1, then xn → +∞. Proposition 1.1.24 (Stolz-Cesàro Criterion). Let (xn) and (yn) be real sequences such that (yn) is strictly increasing and its limit is equal to +∞. If there exists lim xn+1 − xn yn+1 − yn = x ∈ R, then lim xn yn exists and is equal to x. 1.2 Limits of Functions and Continuity In this section, we expand on some issues related to the concepts of limit and conti- nuity for functions. Let p, q ∈ N* . Definition 1.2.1. Let f : A → Rq , A ⊂ Rp and a ∈ A0 . One says that the element l ∈ Rq is the limit of the function f at a, if for every V ∈ V (l) , there exists U ∈ V (a) such that if x ∈ U ∩ A, x ≠ a, then f (x) ∈ V. We will denote this situation by lim x→a f (x) = l. Theorem 1.2.2. Let f : A → Rq , A ⊂ Rp and a ∈ A0 . The next assertions are equiva- lent: (i) lim x→a f (x) = l; (ii) for every B (l, ε) ⊂ Rq , there exists B (a, δ) ⊂ Rp such that if x ∈ B (a, δ) ∩ A, x ≠ a, then f (x) ∈ B (l, ε) ; (iii) for every ε > 0, there exists δ > 0, such that if kx − ak < δ, x ∈ A, x ≠ a, then kf (x) − lk < ε; (iv) for every ε > 0, there exists δ > 0, such that if |xi − ai| < δ for every i ∈ 1, p, where x = (x1, x2, .., xp) ∈ A, a = (a1, a2, .., ap) , x ≠ a, then kf (x) − lk < ε; (v) for every sequence (xn) ⊂ A {a} , xn → a implies that f (xn) → l.
- 20. 6 | Preliminaries Theorem 1.2.3. Let f : A → Rq , A ⊂ Rp , l ∈ Rq and a ∈ A0 . If the function f has the limit l at a, then this limit is unique. Remark 1.2.4. If there exist two sequences x0 n , x00 n ⊂ A {a} , x0 n → a, x00 n → a such that f(x0 n) → l0 , f(x00 n ) → l00 and l0 ≠ l00 , then the limit of the function f at a ∈ A0 does not exist. Theorem 1.2.5. Let f : A → Rq , A ⊂ Rp , f = (f1, f2, ..., fq) and a ∈ A0 . Then f has the limit l = (l1, l2, ..., lq) ∈ Rq at a if and only if there exists lim x→a fi (x) = li, for every i ∈ 1, q. Definition 1.2.6. Let a ∈ R, A ⊂ R and denote As = A ∩(−∞, a], Ad = A ∩[a, ∞). One says that the element a is a left (right) accumulation point for A, if it is an accumulation point for As (Ad, respectively). We will denote the set of left (right) accumulation points of A by A0 s (A0 d, respectively). Definition 1.2.7. Let f : A → Rq , A ⊂ R and a be a left (right) accumulation point of A. One says that the element l ∈ Rq is the left-hand (right-hand) limit of the function f in a if for every neighborhood V ∈ V (l) there exists U ∈ V(a), such that if x ∈ U ∩ As (x ∈ U ∩ Ad, respectively), x ≠ a, then f (x) ∈ V. In this case we will write lim x→a,xa f (x) = l, or lim x→a− f (x) = l, or lim x↑a f (x) = l ( lim x→a,xa f (x) = l, or lim x→a+ f (x) = l, or lim x↓a f (x) = l, respectively). Theorem 1.2.8. Let I ⊂ R be an open interval, f : I → Rq , and a ∈ I. Then there exists lim x→a f (x) = l if and only if the left-hand and the right-hand limits of f at a exist and they are equal. In this case, all three limits are equal: lim x→a− f (x) = lim x→a+ f (x) = l. A well-known result says that the monotone real functions admit lateral limits at every accumulation point of their domains. Theorem 1.2.9 (The boundedness criterion). Let f : A → Rq , g : A → R, A ⊂ Rp and a ∈ A0 . If there exist l ∈ Rq and U ∈ V (a) such that kf (x) − lk ≤ |g (x)| for every x ∈ U {a} , and lim x→a g (x) = 0, then there exists lim x→a f (x) = l. Theorem 1.2.10. Let f , g : A ⊂ Rp → Rq , and a ∈ A0 . If lim x→a f (x) = 0 and there exists U ∈ V (a) such that g is bounded on U, then there exists the limit lim x→a f (x) g (x) = 0. Theorem 1.2.11. Let f : A ⊂ Rp → Rq , and a ∈ A0 . If there exists lim x→a f (x) = l, l 0 (l 0), then there exists U ∈ V (a) such that for every x ∈ U ∩A, x ≠ a, one has f (x) 0 (respectively, f (x) 0).
- 21. Limits of Functions and Continuity | 7 Theorem 1.2.12. Let f : A ⊂ Rp → Rq , and a ∈ A0 . If there exists lim x→a f (x) = l, then there exists U ∈ V (a) such that f is bounded on U (i.e., there exists M 0 such that for every x ∈ U, one has kf (x)k ≤ M). Definition 1.2.13. Let f : A ⊂ Rp → R and a ∈ A0 . One says that the function f has the limit equal to +∞ (respectively, −∞) at a, if for every V ∈ V (+∞) (respectively, V ∈ V (−∞)), there exists U ∈ V (a) such that for every x ∈ U ∩ A, x ≠ a, one has f (x) ∈ V. In this case, we will write lim x→a f (x) = +∞ (respectively, lim x→a f (x) = −∞). Theorem 1.2.14. Let f : A ⊂ Rp → R and a ∈ A0 . Then there exists lim x→a f (x) = +∞ (respectively, lim x→a f (x) = −∞) if and only if for every ε 0, there exists δ 0, such that if kx − ak δ, x ∈ A, x ≠ a, one has f (x) ε (respectively, f (x) −ε). Definition 1.2.15. Let f : A ⊂ R → Rq , such that +∞ (respectively, −∞) is an accumu- lation point of A. One says that the element l ∈ Rq is the limit of f at +∞ (respectively, −∞), if for every V ∈ V (l) , there exists U ∈ V (+∞) (respectively, U ∈ V (−∞)) such that for every x ∈ U ∩ A, one has f (x) ∈ V. In this case, we will write lim x→+∞ f (x) = l (respectively, lim x→−∞ f (x) = l). Theorem 1.2.16. Let f : A ⊂ R → Rq , such that +∞ (respectively, −∞) is an accumu- lation point of A. Then there exists lim x→+∞ f (x) = l (respectively, lim x→−∞ f (x) = l) if and only if for every ε 0, there exists δ 0, such that if x δ (respectively, x −δ), x ∈ A, one has kf (x) − lk ε. Definition 1.2.17. Let f : A ⊂ Rp → Rq , and a ∈ A. One says that the function f is continuous at a if for every V ∈ V(f(a)), there exists U ∈ V(a) such that for every x ∈ U ∩ A, one has f(x) ∈ V. If the function f is not continuous at a, one says that f is discontinuous at a, or that a is a discontinuity point of the function f. Theorem 1.2.18. Let f : A ⊂ Rp → Rq , and a ∈ A0 ∩ A. The function f is continuous at a if and only if lim x→a f(x) = f(a). If a is an isolated point of A, then f is continuous at a. Theorem 1.2.19. Let f : A ⊂ Rp → Rq , and a ∈ A. The next assertions are equivalent: (i) f is continuous at a; (ii) (ε − δ characterization) for every ε 0, there exists δ 0, such that if kx − ak δ, x ∈ A, then f(x) − f(a) ε; (iii) (sequential characterization) for every (xn) ⊂ A, xn → a, one has f(xn) → f(a).
- 22. 8 | Preliminaries Theorem 1.2.20. The image of a compact set through a continuous function is a com- pact set. Theorem 1.2.21 (Weierstrass Theorem). Let K be a compact subset of Rp . If f : K → R is a continuous function, then f is bounded and it attains its extreme values on the set K (i.e., there exist a, b ∈ K, such that sup x∈K f(x) = f(a) and inf x∈K f(x) = f(b)). Definition 1.2.22. Let f : D ⊂ Rp → Rq . One says that the function f is uniformly continuous on the set D if for every ε 0, there exists δ 0, such that for every x0 , x00 ∈ D with x0 − x00 δ, one has f(x0 ) − f(x00 ) ε. Remark 1.2.23. Every function which is uniformly continuous on D is continuous on D, i.e., it is continuous at every point of D. Theorem 1.2.24 (Cantor Theorem). Every function which is continuous on a compact set K ⊂ Rp and takes values in Rq is uniformly continuous on K. Definition 1.2.25. Let L ≥ 0 be a real number. One says that a function f : A ⊂ Rp → Rq is Lipschitz on A with modulus L, or L−Lipschitz on A, if f(x) − f(y) ≤ L kx − yk, for every x, y ∈ A. Proposition 1.2.26. Every Lipschitz function on A ⊂ Rp is uniformly continuous on A. Theorem 1.2.27. Let I ⊂ R be an interval. If f : I → R is injective and continuous, then f is strictly monotone on I. Definition 1.2.28. Let I ⊂ R be an interval. One says that the function f : I → R has the Darboux property if for every a, b ∈ I, a b and every λ ∈ (f(a), f(b)) or λ ∈ (f(b), f(a)), there exists cλ ∈ (a, b) such that f(cλ) = λ. Theorem 1.2.29. Let I ⊂ R be an interval. If the function f : I → R has the Darboux property and there exist a, b ∈ I, a b, such that f(a)f(b) 0, then the equation f(x) = 0 has at least one solution in (a, b). Theorem 1.2.30. Let I ⊂ R be an interval. The function f : I → R has the Darboux property if and only if for every interval J ⊂ I, f (J) is an interval. Theorem 1.2.31. Let I ⊂ R be an interval. If f : I → R is continuous, then f has the Darboux property.
- 23. Limits of Functions and Continuity | 9 Recall that every linear operator T : Rp → Rq is continuous. For such a map, one uses the constant kTk := inf{M 0 | kTxk ≤ M kxk , ∀x ∈ Rp } = sup kTxk | x ∈ D(0, 1) . The mapping T 7→ kTk satisfies the axioms of a norm, therefore it is called the norm of the operator T. Consequently, the set of linear operators from Rp to Rq is a real normed vector space, with respect to the usual algebraic operations and to the norm previously defined. This space is denoted by L(Rp , Rq ) and can be isomorphically identified with Rpq . Every operator T ∈ L(Rp , Rq ) can be naturally associated with a q × p matrix, denoted by AT = (aji)j∈1,q,i∈1,p, as follows: if (ei)i∈1,p and (e0 i)i∈1,q are the canonical bases of the spaces Rp and Rq , respectively, then (aji)j∈1,q,i∈1,p are the coordinates of the expressions of the images of the elements (ei)i∈1,p through T with respect to the basis (e0 i)i∈1,q, i.e., T(ei) = q X j=1 ajie0 j , ∀i ∈ 1, p. Consequently, T 7→ AT is an isomorphism of linear spaces between L(Rp , Rq ) and the space of real q × p matrices. Also, for every x ∈ Rp : T(x) = (AT xt )t . Moreover, for every x ∈ Rp and y ∈ Rq , one has that D (AT xt )t , y E = D x, (At T yt )t E . If A is a q×p matrix, then the linear operator associated with A is surjective if and only if the map associated with At is injective. Recall also that if T : Rp → Rq is a linear operator, then its kernel, Ker(T) := {x ∈ Rp | T(x) = 0}, is a linear subspace of Rp , and its image, Im(T) := {T(x) | x ∈ Rp }, is a linear subspace of Rq . Moreover, p = dim(Ker(T)) + dim(Im(T)), where by dim we denote the algebraic dimension. From the theory of linear algebra, one knows that if A is a symmetric square ma- trix of order p, then its eigenvalues are real and, moreover, there exists an orthogonal
- 24. 10 | Preliminaries matrix B (i.e., BBt = Bt B = I) such that Bt AB is the diagonal matrix having the eigen- values on its main diagonal. Recall that, as usual, I denotes the identity matrix. One says that a matrix A as above is positive semidefinite if (Axt )t , x ≥ 0 for every x ∈ Rp , and positive definite if (Axt )t , x 0 for every x ∈ Rp {0}. Actually, A is positive definite if and only if it is positive semidefinite and invertible. We end this section by mentioning the celebrated result of Hahn-Banach. Recall that a function f : Rp → R is called sublinear if it is positive homogeneous (i.e., f(αx) = αf(x) for all α ≥ 0 and x ∈ Rp ) and subadditive (i.e., f(x + y) ≤ f(x) + f(y) for all x, y ∈ Rp ). Theorem 1.2.32 (Hahn-Banach). Let X be a linear subspace of Rp , χ : Rp → R be a sublinear function, and φ0 : X → R be a linear function. If φ0(x) ≤ χ(x) for every x ∈ X, then there exists a linear function φ : Rp → R such that φ X = φ0 and φ(x) ≤ χ(x) for every x ∈ Rp . 1.3 Differentiability Definition 1.3.1. Let f : D ⊂ Rp → Rq and a ∈ int D. One says that f is Fréchet differentiable (or, simply, differentiable) at a if there exists a linear operator denoted by ∇f(a) : Rp → Rq such that lim h→0 f(a + h) − f(a) − ∇f(a)(h) khk = lim x→a f(x) − f(a) − ∇f(a)(x − a) kx − ak = 0. The map ∇f(a) is called the Fréchet differential of the function f at a. The previous relation is equivalent to the following conditions: ∀ε 0, ∃δ 0, ∀x ∈ B(a, δ) : f(x) − f(a) − ∇f(a)(x − a) ≤ ε kx − ak ; ∃α : D − {a} → Rq : lim h→0 α(h) = α(0) = 0, f(a + h) = f(a) + ∇f(a)(h) + khk α(h), ∀h ∈ D − {a}. One says that f : D ⊂ Rp → Rq is of class C1 on the open set D if f is Fréchet differentiable on D and ∇f is continuous on D. Obviously, f can be written as f = (f1, f2, ..., fq), where fi : Rp → R, i ∈ 1, q and, in general, the map ∇f(a) ∈ L(Rp , Rq ) will be identified to the q × p matrix ∂f1 ∂x1 (a) ∂f1 ∂x2 (a) · · · ∂f1 ∂xp (a) ∂f2 ∂x1 (a) ∂f2 ∂x2 (a) · · · ∂f2 ∂xp (a) . . . . . . ... . . . ∂fq ∂x1 (a) ∂fq ∂x2 (a) · · · ∂fq ∂xp (a) ,
- 25. Differentiability | 11 called the Jacobian matrix of f at the point a, where ∂fi ∂xj (a) is the partial derivative of the function fi with respect to the variable xj at a. We will subsequently refer several times to the Jacobian matrix instead of the dif- ferential. Based on a general result, if f : D ⊂ Rp → Rp and a ∈ int D, ∇f(a) is an isomorphism of Rp if and only if the Jacobian matrix of f at a is invertible. The next calculus rules hold. – Let f : Rp → Rq be an affine function, i.e., it takes the form f(x) := g(x) + u for every x ∈ Rp , where g : Rp → Rq is linear, and u ∈ Rq . Then for every x ∈ Rp , ∇f(x) = g. – Let f : Rp → R be of the form f(x) = 1 2 (Axt )t , x +hb, xi , where A is a symmetric square matrix of order p, and b ∈ Rp . Then for every x ∈ Rp , ∇f(x) = (Axt )t + b. – Let D ⊂ Rp , E ⊂ Rq , x ∈ int D, y ∈ int E and f , g : D → Rq , φ : D → R, h : E → Rk . – If f , g are differentiable at x, and α, β ∈ R, then the function αf + βg is differ- entiable at x and ∇(αf + βg)(x) = α∇f(x) + β∇g(x). – If f , φ are differentiable at x, then φf is differentiable at x at ∇(φf)(x) = φ(x)∇f(x) + f(x)∇φ(x), where f(x)∇φ(x) (x) := ∇φ(x)(x) · f(x). – (Chain rule) If f(D) ⊂ E, y = f(x), f is differentiable at x and h is differentiable at y, then h ◦ f is differentiable at x and ∇(h ◦ f)(x) = ∇h(y) ◦ ∇f(x). A casewhich deservesspecial attention is p = 1. In thiscase one saysthat f is derivable at a if there exists lim h→0 f(a + h) − f(a) h ∈ Rq . (1.3.1) One denotes this limit by f0 (a) and it is called the derivative of f at a. Proposition 1.3.2. Let f : D ⊂ R → Rq and a ∈ int D. The next assertions are equiva- lent: (i) f is derivable at a; (ii) f is Fréchet differentiable at a. In every one of these cases, ∇f(a)(x) = xf0 (a) for every x ∈ R. Let r ∈ N* . If f : D ⊂ Rp × Rq → Rr , and (a, b) ∈ int D is fixed, one defines D1 := {x ∈ Rp | (x, b) ∈ D} and f1 : D1 → Rr , f1(x) := f(x, b). One says that f is Fréchet differentiable with respect to x at a if f1 is Fréchet differentiable at a, and in
- 26. 12 | Preliminaries this case the differential is denoted by ∇xf(a, b). If f is differentiable at (a, b), then f is differentiable with respect to x and y at a and b, respectively, and ∇xf(a, b) = ∇f(a, b)(·, 0), ∇yf(a, b) = ∇f(a, b)(0, ·). In the general case, one says that f : D ⊂ Rp → Rq is twice Fréchet differentiable at a ∈ int D if f is Fréchet differentiable on a neighborhood V ⊂ D of a and ∇f : V → L(Rp , Rq ) is Fréchet differentiable at a, i.e., there exists a functional denoted by ∇2 f(a), from the space L2 (Rp , Rq ) := L(Rp , L(Rp , Rq )), and α : D − {a} → L(Rp , Rq ), such that limh→0 α(h) = α(0) = 0 and for every h ∈ D − {a}, one has ∇f(a + h) = ∇f(a) + ∇2 f(a)(h, ·) + khk α(h). Recall that the space L2 (Rp , Rq ) mentioned above can be identified with the space of bilinear maps from Rp × Rp to Rq . One says that f is of class C2 on the open set D if it is twice Fréchet differentiable on D and ∇2 f : D → L2 (Rp , Rq ) is continuous. Theorem 1.3.3. Let f : D ⊂ Rp → Rq and a ∈ int D. If f is twice Fréchet differentiable at a, then ∇2 f(a) is a symmetric bilinear map. In the case when q = 1, the map ∇2 f(a) is defined by the symmetric square matrix H(a) = ∂2 f ∂xi∂xj (a) i,j∈1,p , which is called the Hessian matrix of f at a. Moreover, D H(a)ut t , v E = ∇2 f(a)(u, v) for every u, v ∈ Rp , i.e., ∇2 f(a)(u, v) = n X i,j=1 ∂2 f ∂xi∂xj (a)uivj. If a, b ∈ Rp , one defines the closed and the open line segments between a and b as follows: [a, b] := {αa + (1 − α)b | α ∈ [0, 1]}, (a, b) := {αa + (1 − α)b | α ∈ (0, 1)}. Theorem 1.3.4 (Lagrange and Taylor Theorems). Let U ⊂ Rp be an open set, f : U → R and a, b ∈ U with [a, b] ⊂ U. If f is of class C1 on U, then there exists c ∈ (a, b) such that f(b) = f(a) + ∇f(c)(b − a). If f is of class C2 on U, then there exists c ∈ (a, b) such that f(b) = f(a) + ∇f(a)(b − a) + 1 2 ∇2 f(c)(b − a, b − a).
- 27. Differentiability | 13 Theorem 1.3.5 (Implicit Function Theorem). Let D ⊂ Rp × Rq be an open set, h : D → Rq be a function and x ∈ Rp , y ∈ Rq be such that: (i) h(x, y) = 0; (ii) the function h is of class C1 on D; (iii) ∇yh(x, y) is invertible. Then there exist two neighborhoods U and V of x and y, respectively, and a unique continuous function φ : U → V such that: (a) h(x, φ(x)) = 0 for every x ∈ U; (b) if (x, y) ∈ U × V and h(x, y) = 0, then y = φ(x); (c) φ is differentiable on U and ∇φ(x) = −[∇yh(x, φ(x))]−1 ∇xh(x, φ(x)), ∀x ∈ U. Some fundamental results from the theory of differentiability of the real functions are briefly given at the end of this section. In the case p = q = 1 one can apply Proposition 1.3.2. It is also sensible to speak of the existence of the derivative at points of the domain which are accumulation points of it: consider the limit from the relation (1.3.1) at accumulation points. Moreover, as in the case of the lateral limits, one can speak about the left and right-hand derivatives, by considering the lateral limits in the expression from relation (1.3.1). When they ex- ist, we will call these limits the left, and the right-hand derivatives of the function f at a and we will denote them by f0 −(a) and f0 +(a), respectively. Definition 1.3.6. Let A ⊂ R and f : A → R. One says that a ∈ A is a local minimum (maximum) point for f if there exists a neighborhood V of a such that f(a) ≤ f(x) (re- spectively, f(a) ≥ f(x)), for every x ∈ A ∩ V. One says that a point is a local extremum if it is a local minimum or a local maximum. Theorem 1.3.7 (Fermat Theorem). Let I ⊂ R be an interval and a ∈ int I. If f : I → R is derivable at a, and a is a local extremum point for f , then f0 (a) = 0. Theorem 1.3.8 (Rolle Theorem). Let a, b ∈ R, a b, and f : [a, b] → R be a function which is continuous on [a, b], derivable on (a, b), and satisfies f(a) = f(b). Then there exists c ∈ (a, b) such that f0 (c) = 0. Theorem 1.3.9 (Lagrange Theorem). Let a, b ∈ R, a b, and f : [a, b] → R be a function which is continuous on [a, b], derivable on (a, b). Then there exists c ∈ (a, b) such that f(b) − f(a) = f0 (c)(b − a). Proposition 1.3.10. Let I ⊂ R be an interval and f : I → R be derivable on I. (i) If f0 (x) = 0 for every x ∈ I, then f is constant on I.
- 28. 14 | Preliminaries (ii) If f0 (x) 0 (respectively, if f0 (x) ≥ 0) for every x ∈ I, then f is strictly increasing (respectively, it is increasing) on I. (iii) If f0 (x) 0 (respectively, if f0 (x) ≤ 0), for every x ∈ I, then f is strictly decreasing (respectively, it is decreasing) on I. Theorem 1.3.11 (Rolle Sequence). Let I ⊂ R be an interval and f : I → R be a deriv- able function. If x1, x2 ∈ I, x1 x2 are consecutive roots of the derivative f0 (i.e., f0 (x1) = 0, f0 (x2) = 0 and f0 (x) ≠ 0 for any x ∈ (x1, x2)) then: (i) if f(x1)f(x2) 0, the equation f(x) = 0 has exactly one root in the inter- val (x1, x2); (ii) if f(x1)f(x2) 0, the equation f(x) = 0 has no roots in the interval (x1, x2); (iii) if f(x1) = 0 or f(x2) = 0, then x1 or x2 is a multiple root of the equation f(x) = 0 and this equation has no other root in the interval (x1, x2). Theorem 1.3.12 (Cauchy Rule). Let I ⊂ R be an interval and f , g : I → R, a ∈ I, which satisfy: (i) f(a) = g(a) = 0; (ii) f , g are derivable at a; (iii) g0 (a) ≠ 0. Then there exists V ∈ V(a) such that g(x) ≠ 0, for any x ∈ V {a} and lim x→a f(x) g(x) = f0 (a) g0(a) . Theorem 1.3.13 (L’Hôpital Rule). Let f , g : (a, b) → R, where −∞ ≤ a b ≤ ∞. If: (i) f , g are derivable on (a, b) with g0 ≠ 0 on (a, b); (ii) there exists lim x→a xa f0 (x) g0(x) = L ∈ R; (iii) lim x→a xa f(x) = lim x→a xa g(x) = 0 or (iii)’ lim x→a xa g(x) = ∞, then there exists lim x→a f(x) g(x) = L. Theorem 1.3.14. Let I ⊂ R be an open interval, f : I → R be a n−times derivable function at a ∈ I, (n ∈ N, n ≥ 2), such that f0 (a) = 0, f00 (a) = 0, ..., f(n−1) (a) = 0, f(n) (a) ≠ 0. (i) If n is even, then a is an extremum point, more precisely: a local maximum if f(n) (a) 0, and a local minimum if f(n) (a) 0. (ii) If n is odd, then a is not an extremum point.
- 29. The Riemann Integral | 15 1.4 The Riemann Integral At the end of this chapter we discuss the main aspects concerning the Riemann inte- gral. Let a, b ∈ R, a b. Definition 1.4.1. (i) A partition of the interval [a, b] is a finite set of real numbers x0, x1, ..., xn (n ∈ N* ), denoted by ∆, such that a = x0 x1 ... xn−1 xn = b. (ii) The norm of the partition ∆ is the number k∆k := max{xi − xi−1 | i ∈ 1, n}. (iii) A tagged partition of the interval [a, b] is a partition ∆, together with a finite set of real numbers Ξ := {ξi | i ∈ 1, n}, such that ξi ∈ [xi−1, xi] for any i ∈ 1, n. The set Ξ is called the intermediate points system associated to ∆. (iv) Let f : [a, b] → R be a function. The Riemann sum associated to a tagged partition of the interval [a, b] is S(f , ∆, Ξ) := n X i=1 f(ξi)(xi − xi−1). Definition 1.4.2. Let f : [a, b] → R be a function. One says that f is Riemann integrable on [a, b] if there exists I ∈ R such that for every ε 0, there exists δ 0 such that for any partition ∆ of the interval [a, b] with the property k∆k δ, and for any intermediate points system Ξ associated to ∆, the next inequality holds: S(f , ∆, Ξ) − I ε. The real number I from the previous definition, which is unique, is called the Riemann integral of f an [a, b] and is denoted by b Z a f(x)dx. Theorem 1.4.3. Any function which is Riemann integrable on [a, b] is bounded on [a, b]. Definition 1.4.4. Let f : [a, b] → R be a function. One says that a function F : [a, b] → R is an antiderivative (or, equivalently, a primitive integral, or an indefinite integral) of f on [a, b] if F is derivable on [a, b] and F0 (x) = f(x) for any x ∈ [a, b]. If an antiderivative exists for a given function, then infinitely many antiderivatives exist for that function and the difference of any two such antiderivatives is a constant. The next result is sometimes called the fundamental theorem of calculus.
- 30. 16 | Preliminaries Theorem 1.4.5 (Leibniz-Newton). If f : [a, b] → R is Riemann integrable on the inter- val [a, b] and it admits an antiderivative F on [a, b], then b Z a f(x)dx = F(b) − F(a). Continuous functions satisfy both hypotheses of the preceding theorem. Theorem 1.4.6. If f : [a, b] → R is continuous on [a, b], then f is Riemann integrable on [a, b] and it admits antiderivatives on [a, b]. Theorem 1.4.7. If f : [a, b] → R is bounded and has a finite set of discontinuity points, then f is Riemann integrable on [a, b]. Every function which is monotone on [a, b] is Riemann integrable on [a, b]. We present now the main properties of the Riemann integral. Theorem 1.4.8. (i) If f , g : [a, b] → R are Riemann integrable on [a, b], and α, β ∈ R, then αf + βg is Riemann integrable on [a, b] and b Z a (αf(x) + βg(x))dx = α b Z a f(x)dx + β b Z a g(x)dx. (ii) If f : [a, b] → R is Riemann integrable on [a, b], and m ≤ f(x) ≤ M for every x ∈ [a, b] (m, M ∈ R), then m(b − a) ≤ b Z a f(x)dx ≤ M(b − a). In particular, if f(x) ≥ 0 for every x ∈ [a, b], then b Z a f(x)dx ≥ 0, and if f , g : [a, b] → R are Riemann integrable and f(x) ≤ g(x) for every x ∈ [a, b], then b Z a f(x)dx ≤ b Z a g(x)dx. (iii) If f : [a, b] → R is Riemann integrable on [a, b], then |f| is Riemann integrable on [a, b]. (iv) If f , g : [a, b] → R are Riemann integrable [a, b], then f · g is Riemann inte- grable on [a, b].
- 31. The Riemann Integral | 17 Theorem 1.4.9. (i) If f : [a, b] → R is Riemann integrable on [a, b], then f is Riemann integrable on every subinterval of [a, b]. (ii) If c ∈ (a, b) and f is Riemann integrable on [a, c] and on [c, b], then f is Rie- mann integrable on [a, b] and b Z a f(x)dx = c Z a f(x)dx + b Z c f(x)dx. Theorem 1.4.10. Let f : [a, b] → R be a function, and f* : [a, b] → R be another function which coincides with f on [a, b], except on a finite set of points. If f* is Riemann integrable on [a, b], then f is Riemann integrable on [a, b] and b Z a f(x)dx = b Z a f* (x)dx. Theorem 1.4.11 (integration by parts). If f , g : [ a, b ] → R are C1 functions, then b Z a f(x)g0 (x) dx = f(x)g(x)| b a − b Z a f0 (x)g(x) dx. Theorem 1.4.12 (change of variable). Let φ : [a, b] → [c, d] be a C1 function, and let f : [c, d] → R be a continuous function. Then b Z a f(φ(t)) · φ0 (t) dt = φ(b) Z φ(a) f(x) dx. (1.4.1) We end this section by the next multidimensional variant of Taylor Theorem. In what follows, the equality is understood on components (i.e., for every function fi : Rp → R, i = 1, p, where f = (f1, ..., fp)). Theorem 1.4.13. Suppose f : Rp → Rp is continuously differentiable on some convex open set D and that x, x + y ∈ D. Then there is t ∈ (0, 1) such that f(x + y) = f(x) + 1 Z 0 ∇f(x + ty)(y) dt.
- 32. © 2014 Marius Durea and Radu Strugariu This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivs 3.0 License. 2 Nonlinear Analysis Fundamentals In this chapter we study convex sets and functions, convex cones, the Bouligand tangent cone to a set at a point, and semicontinuous functions. We prove fundamental results, which will be the main investigation tools in subsequent chapters. The Farkas Lemma and its consequences will be decisive for establishing optimality conditions in Karush-Kuhn-Tucker form, while the Banach fixed point Principle will be used for discussing some optimization algorithms. We also study some important inequalities related to the study of convex functions. 2.1 Convex Sets and Cones Definition 2.1.1. One says that a nonempty set D ⊂ Rp is convex if for every x, y ∈ D, [x, y] = {αx + (1 − α)y | α ∈ [0, 1]} ⊂ D. In other words, D is convex if and only if together with two points a1, a2, it contains the whole segment [a1, a2]. It is sufficient to take α ∈ (0, 1). By mathematical in- duction one can show that if D is convex, then for every n ∈ N* , x1, x2, ..., xn ∈ D, α1, α2, ..., αn ∈ [0, 1] with Pn i=1 αi = 1 : n X i=1 αixi ∈ D. A sum like the one above is called a convex combination of the elements (xi). In R, the convex sets are intervals. One of the main objects for our study is defined next. Definition 2.1.2. A nonempty subset K ⊂ Rp is a cone if the next relation holds: ∀y ∈ K, ∀λ ∈ R+ := [0, ∞) : λy ∈ K. According to the definition, every cone contains the origin. Proposition 2.1.3. A cone C is convex if and only if C + C = C. Proof Suppose first that C is a convex cone. As 0 ∈ C, it is clear that C ⊂ C + C. Let u ∈ C + C. Then there exist c1, c2 ∈ C such that c1 + c2 = u. But, using the properties of C and the obvious relation u = 2 2−1 c1 + 2−1 c2 ,
- 33. Convex Sets and Cones | 19 one can deduce that u ∈ C. For the converse implication, suppose that C is a cone which satisfies C + C = C. Fix α ∈ (0, 1) and c1, c2 ∈ C. Then, from the cone property of C, one knows that αc1, (1 − α)c2 ∈ C, therefore αc1 + (1 − α)c2 ∈ C + C = C, which ends the proof. Definition 2.1.4. Let A ⊂ Rp be a nonempty set and x ∈ Rp . One defines the distance from x to A by the relation: d(x, A) := inf{kx − ak | a ∈ A}. We also consider the function dA : Rp → R given by dA(x) := d(x, A). We now introduce some basic properties of the distance from a point to a (nonempty) set. Theorem 2.1.5. Let A ⊂ Rp , A ≠ ∅. Then: (i) d (x, A) = 0 if and only if x ∈ cl A. (ii) The function dA is 1−Lipschitz. (iii) If A is closed, then for every x ∈ Rp , there exists ax ∈ A such that d (x, A) = kx − axk . If, moreover, A is convex, then ax having the previous property is unique and it is characterized by the relations ( ax ∈ A hx − ax, u − axi ≤ 0, ∀u ∈ A. Proof (i) The following equivalences hold d (x, A) = 0 ⇔ inf a∈A kx − ak = 0 ⇔ ∃ (an) ⊂ A with lim n→∞ kx − ank = 0 ⇔ x ∈ A. (ii) For every x, y ∈ Rp and every a ∈ A, these relations hold: d (x, A) ≤ kx − ak ≤ kx − yk + ky − ak . As a is taken arbitrary from A, one deduces d (x, A) ≤ kx − yk + d (y, A) , i.e., d (x, A) − d (y, A) ≤ kx − yk . By reversing the roles of x and y, one has: |d (x, A) − d (y, A)| ≤ kx − yk ,
- 34. 20 | Nonlinear Analysis Fundamentals which is the desired conclusion. (iii) If x ∈ A, then ax := x is the unique element having the previously introduced property. Let x ∉ A. As d(x, A) is a real number, there exists r 0 such that A1 := A ∩ D (x, r) ≠ ∅. Since A1 is a compact set and the function g : A1 → R, g (y) = d (x, y) is continuous, according to Weierstrass Theorem, g attains its minimum on A1, i.e., there exists ax ∈ A1 with g (ax) = infy∈A1 g (y) = d(x, A1). Now, one can check that d (x, A1) = d (x, A) and the first conclusion follows. Suppose that, moreover, A is convex. If x ∈ A, there is nothing to prove. Take x ∉ A. Consider a1, a2 ∈ A with d (x, A) = kx − a1k = kx − a2k . Using the parallelogram law we know: (x − a1) + (x − a2) 2 + (x − a1) − (x − a2) 2 = 2 kx − a1k2 + 2 kx − a2k2 , i.e., k2x − a1 − a2k2 + ka2 − a1k2 = 4d2 (x, A), and dividing by 4 one gets x − a1 + a2 2 2 + 4−1 ka2 − a1k2 = d2 (x, A). Since A is convex, 2−1 (a1+a2) ∈ A, hence x − a1 + a2 2 2 ≥ d2 (x, A). This relation and the previous equality show that ka2 − a1k = 0, hence a1 = a2. The proof of uniqueness is now complete. Let us prove now that ax verifies the relation hx − ax, u − axi ≤ 0 for any u ∈ A. For this, take u ∈ A. Then for every α ∈ (0, 1], one has v = αu + (1 − α)ax ∈ A. Hence, kx − axk ≤ x − αu − (1 − α)ax = x − ax − α(u − ax) , and, consequently, kx − axk2 ≤ kx − axk2 − 2α hx − ax, u − axi + α2 ku − axk2 . After reducing terms and dividing by α 0, we can see that 0 ≤ −2 hx − ax, u − axi + α ku − axk2 . If we let α → 0, the desired inequality follows. For the converse, if an element a ∈ A satisfies hx − a, u − ai ≤ 0 for any u ∈ A, then for every v ∈ A one has kx − ak2 − kx − vk2 = 2 hx − a, v − ai − ka − vk2 ≤ 0, hence a coincides with ax. The proof is now complete. In the case when A is closed, then for x ∈ Rp one denotes the projection set of x on A by prA x := a ∈ A | d(x, A) = kx − ak .
- 35. Convex Sets and Cones | 21 If, moreover, A is convex, then, according to the above theorem, this set consists of only one element, which we still denote by prA x, and we call this the projection of x on A. Let S ⊂ Rp be a nonempty set. The polar of S is the set S− := {u ∈ Rp | hu, xi ≤ 0, ∀x ∈ S}. It is easy to observe that S− is a closed convex cone and that, in general, S ⊂ (S− )− . If we consider the reverse inclusion, the next result follows. Theorem 2.1.6. Let C ⊂ Rp be a closed convex cone. Then C = (C− )− . Proof Consider z ∈ (C− )− and z = prC z. We will prove that z = z. From the last part of Theorem 2.1.5, for any c ∈ C, one has hz − z, c − zi ≤ 0. As 0 ∈ C and 2z ∈ C, we deduce − hz, z − zi ≤ 0, hz, z − zi ≤ 0 hence hz − z, ci ≤ 0 for every c ∈ C, i.e., z − z ∈ C− . As z ∈ (C− )− , one gets hz, z − zi ≤ 0. But kz − zk2 = hz − z, zi − hz − z, zi ≤ 0 which means that z = z, so z ∈ C. This establishes the theorem. Example 2.1.7. 1. Consider S = {(x, 0) ∈ R2 | x ≥ 0}. One can observe that S− = {(x, y) ∈ R2 | x ≤ 0}. Obviously, (S− )− = S− . 2. The polar of R2 + := {(x, y) ∈ R2 | x, y ≥ 0} is R2 − := {(x, y) ∈ R2 | x, y ≤ 0}. The polar of the set S = {(x, 0) ∈ R2 | x ≥ 0} ∪ {(0, y) ∈ R2 | y ≥ 0} is also R2 −. From this example one can see that, in general, S− 1 = S− 2 does not imply S1 = S2. The next result, which has an algebraic character, it was obtained by the Hungarian mathematician Julius Farkas in 1902. Theorem 2.1.8 (Farkas’ Lemma). Let n ∈ N* , (φi)i∈1,n ⊂ L(Rp , R) and φ ∈ L(Rp , R). Then ∀x ∈ Rp : [φ1(x) ≤ 0, . . . , φn(x) ≤ 0] ⇒ φ(x) ≤ 0 (2.1.1) if and only if there exists (αi)i∈1,n ⊂ [0, ∞) such that φ = Pn i=1 αiφi.
- 36. 22 | Nonlinear Analysis Fundamentals Proof The converse implication is obvious. We prove the other one by induction for n ≥ 1. Define the proposition P(n) which says that for every φ, φ1, . . . , φn ∈ L(Rp , R) satisfying (2.1.1), there exist (αi)i∈1,n ⊂ [0, ∞) such that φ = Pn i=1 αiφi. Let us prove that P(1) is true. Indeed, let φ, φ1 ∈ L(Rp , R) such that φ1(x) ≤ 0 ⇒ φ(x) ≤ 0. If φ = 0 then, obviously, φ = 0φ1. Suppose φ ≠ 0. Then, by the assumption: φ1(x) = 0 ⇔ [φ1(x) ≤ 0, φ1(−x) ≤ 0] ⇒ [φ(x) ≤ 0, φ(−x) ≤ 0] ⇔ φ(x) = 0, hence Ker φ1 ⊂ Ker φ. Since φ ≠ 0, one has φ1 ≠ 0, so there exists x1 ∈ Rp with φ1(x) = −1. Also by the assumption, φ(x) ≤ 0. Take x ∈ Rp arbitrarily. Then it is easy to verify that x + φ1(x)x ∈ Ker φ1, hence x + φ1(x)x ∈ Ker φ, i.e., φ x + φ1(x)x = 0, which proves that φ(x) = −φ(x)φ1(x). Notice that x was arbitrarily chosen, so the desired relation is proved for α1 := −φ(x) ≥ 0. Suppose now that P(n) is true for a fixed n ≥ 1 and we will try to prove that P(n+1) is true. Take φ, φ1, . . . , φn, φn+1 ∈ L(Rp , R) such that ∀x ∈ Rp : [φ1(x) ≤ 0, . . . , φn(x) ≤ 0, φn+1(x) ≤ 0] ⇒ φ(x) ≤ 0. (2.1.2) If ∀x ∈ Rp : [φ1(x) ≤ 0, . . . , φn(x) ≤ 0] ⇒ φ(x) ≤ 0, (2.1.3) then, from P(n), there exist (αi)i∈1,n ⊂ [0, ∞) such that φ = Pn i=1 αiφi; take αn+1 := 0 and the conclusion follows. Suppose relation (2.1.3) is not satisfied. Then there exists x ∈ Rp such that φ(x) 0 and φi(x) ≤ 0 for any i ∈ 1, n. As (2.1.2) holds, φn+1(x) 0; we may suppose (by multiplying by the appropriate positive scalar) that φn+1(x) = 1. But φn+1(x − φn+1(x)x) = 0, ∀x ∈ Rp , and from (2.1.2) we deduce ∀x ∈ Rp : [φ1(x − φn+1(x)x) ≤ 0, . . . , φn(x − φn+1(x)x) ≤ 0] ⇒ φ(x − φn+1(x)x) ≤ 0. (2.1.4)
- 37. Convex Sets and Cones | 23 Take φ0 i := φi − φi(x)φn+1 for i ∈ 1, n and φ0 := φ − φ(x)φn+1, and then relation (2.1.4) becomes ∀x ∈ X : [φ0 1(x) ≤ 0, . . . , φ0 n(x) ≤ 0] ⇒ φ0 (x) ≤ 0. As P(n) is true, there exist (αi)i∈1,n ⊂ [0, ∞) such that φ0 = Pn i=1 αiφ0 i . We deduce that φ − φ(x)φn+1 = n X i=1 αi φi − φi(x)φn+1 , hence φ = Pn+1 i=1 αiφi, where αn+1 = φ(x) − Pn i=1 αiφi(x) ≥ 0 from the choice of x and from the fact that αi ≥ 0 for any i ∈ 1, n). The proof is now complete. Throughout this book we shall use several different concepts of tangent vectors to a set at a point. We introduce now one of these concepts. Definition 2.1.9. Let M ⊂ Rp be a nonempty set and x ∈ cl M. One says that a vector u ∈ Rp is tangent in the sense of Bouligand to the set M at x if there exist (tn) ⊂ (0, ∞), tn → 0 and (un) → u such that for any n ∈ N, one has x + tnun ∈ M. It is sufficient that the above inclusion holds for every n ∈ N sufficiently large. Theorem 2.1.10. The set, denoted by TB(M, x), which contains all the tangent vectors to the set M at x is a closed cone, which we call the Bouligand tangent cone (or the contingent cone) to the set M at the point x. Proof Let us prove first that 0 ∈ TB(M, x). If x ∈ M, then the assertion trivially follows, because it is sufficient to take (un) constantly equal to 0. If x ∉ M, then there exists (xn)n∈N ⊂ M such that xn → x, and we consider tn := p kxn − xk and un := p kxn − xk −1 (xn − x) for every n ∈ N. Since tn → 0 and un → 0, one obtains the conclusion. Consider now u ∈ TB(M, x) and λ 0. According to the definition, there exist (tn) ⊂ (0, ∞), tn → 0 and (un) → u such that for every n ∈ N, x + tnun ∈ M. This is equivalent to x + tn λ (λun) ∈ M. As tn λ → 0 and (λun) → λu, one deduces that λu ∈ TB(M, x), hence TB(M, x) is a cone. We prove that the closure of TB(M, x) is contained in TB(M, x). Take (un) ⊂
- 38. 24 | Nonlinear Analysis Fundamentals TB(M, x) and (un) → u. One must prove that u ∈ TB(M, x). For every n ∈ N, there exist (tk n)k ⊂ (0, ∞), tk n k→∞ → 0 and (uk n) k→∞ → un such that for every k ∈ N, x + tk nuk n ∈ M. By using a diagonalization procedure, for every n ∈ N* , there exists kn ∈ N such that the next relations hold: tkn n 1 n ukn n − un ≤ 1 n . It is easy to observe that the positive sequence (tkn n )n converges to 0, and using the inequality ukn n − u ≤ ukn n − un + kun − uk , one can deduce that (ukn n ) → u. Moreover, for every n ∈ N, x + tkn n ukn n ∈ M, hence u ∈ TB(M, x) and the proof is complete. Our first example is given next. Example 2.1.11. 1. Consider the ball M ⊂ R2 , M := {(x, y) ∈ R2 | (x − 1)2 + y2 ≤ 1}. Then TB(M, (0, 0)) = {(x, y) ∈ R2 | x ≥ 0}. 2. One can easily observe that if C ⊂ Rp is a closed cone, then TB(C, 0) = C. Proposition 2.1.12. If ∅ ≠ M ⊂ Rp and x ∈ cl M, then TB(M, x) = TB(cl M, x). If x ∈ int M, then TB(M, x) = Rp . Proof The inclusion TB(M, x) ⊂ TB(cl M, x) is obvious by the use of M ⊂ cl M. Take u ∈ TB(cl M, x). There exist (tn) ⊂ (0, ∞), tn → 0 and (un) → u such that for every n ∈ N, x + tnun ∈ cl M. Using the sequence characterization of the closure, for any fixed n, there exist (vk n)k ⊂ M such that vk n k → x + tnun. As above, for any fixed n, there exists kn ∈ N such that vkn n − (x + tnun) ≤ t2 n. Then, one can write vkn n − x tn − u ≤ vkn n − x tn − un + kun − uk ≤ tn + kun − uk ,
- 39. Convex Sets and Cones | 25 hence u0 n := vkn n − x tn n→∞ → u. But x + tnu0 n = vkn n ∈ M, hence u ∈ TB(M, x). The second part of the conclusion easily follows: if x ∈ int M, then for every u ∈ Rp and every (tn) ⊂ (0, ∞), tn → 0, one has x + tnun ∈ M for any n sufficiently large. This shows, in particular, that u ∈ TB(M, x), and the conclusion follows. In general, the Bouligand tangent cone is not convex and the relation TB(M, x) = Rp can be satisfied, even if x ∉ int M. Example 2.1.13. 1. Consider the set M ⊂ R2 , M = {(x, y) | x ≥ 0, y = 0} ∪ {(x, y) | x = 0, y ≥ 0}. Then TB(M, (0, 0)) = M is not a convex set. 2. Let set M represent the plane domain bounded by the curve (the cardioid) which has the parametric representation ( x = −2 cos t + cos 2t + 1 y = 2 sin t − sin 2t , t ∈ [0, 2π]. Then TB(M, (0, 0)) = R2 , but (0, 0) ∉ int M. Figure 2.1: The cardioid.
- 40. 26 | Nonlinear Analysis Fundamentals Proposition 2.1.14. Let A1, A2 ⊂ Rp be closed sets. Then the next relations hold: (i) if x ∈ A1 ∩ A2, then TB(A1 ∪ A2, x) = TB(A1, x) ∪ TB(A2, x); (ii) if x ∈ A1 ∩ A2, then TB(A1 ∩ A2, x) ⊂ TB(A1, x) ∩ TB(A2, x); (iii) if x ∈ bd A1, then TB(bd A1, x) = TB(A1, x) ∩ TB(Rp A1, x). Proof The first two relations easily follow, as well as the inclusion TB(bd A1, x) ⊂ TB(A1, x) ∩ TB(Rp A1, x) from (iii), which can be proved by using (ii) and Propo- sition 2.1.12. Let us prove now the other inclusion from (iii). Take u ∈ TB(A1, x) ∩ TB(Rp A1, x). According to the definition, there exist (tn), (t0 n) ⊂ (0, ∞), tn, t0 n → 0 and (un), (u0 n) → u such that for every n ∈ N, x + tnun ∈ A1, and x + t0 nu0 n ∈ Rp A1. If an infinite number of terms from the first or the second relation are on the boundary of A1, there is nothing to prove. Suppose next, without loss of generality, that for every n ∈ N, there exists λn ∈ (0, 1) such that λn (x + tnun) + (1 − λn)(x + t0 nu0 n) ∈ bd A1. Consider the sequences (t00 n ) := (λntn + (1 − λn)t0 n) ⊂ (0, ∞) (u00 n ) := tnλn t00 n un + t0 n(1 − λn) t00 n u0 n. It is clear that (t00 n ) → 0. On the other hand, u00 n − u ≤ kun − uk + u0 n − u , hence (u00 n ) → u. Since x + t00 n u00 n ∈ bd A1, one gets the desired conclusion. Denote by NB(M, x) the polar of TB(M, x) (i.e., NB(M, x) := TB(M, x)− ), and we call this set the Bouligand normal cone to M at x. If the set M is convex, then the Bouligand tangent and normal cones have a special form. Proposition 2.1.15. Let ∅ ≠ M ⊂ Rp be a convex set and x ∈ M. Then TB(M, x) = cl R+(M − x), and NB(M, x) = {u ∈ Rp | hu, c − xi ≤ 0, ∀c ∈ M}.
- 41. Convex Sets and Cones | 27 Proof Take c ∈ M and d := c − x. Consider (tk)k → 0. Then x + tkd = (1 − tk)x + tkc ∈ M, hence M − x ⊂ TB(M, x). Since TB(M, x) is a closed cone, one gets that cl R+(M − x) ⊂ TB(M, x). Take u ∈ TB(M, x). Then there exist (tk) ⊂ (0, ∞), tk → 0 and (uk) → u such that for every k ∈ N, xk: = x + tkuk ∈ M. Hence u = limk xk − x tk . But xk − x tk k ⊂ R+(M − x). One can deduce that TB(M, x) ⊂ cl R+(M − x). Recall that, by definition, NB(M, x) = TB(M, x)− = {u ∈ Rp | hu, vi ≤ 0, ∀v ∈ TB(M, x)}. Now, taking into account the particular form of TB(M, x), the conclusion follows. For a reason we make clear later on, in the case of convex sets, we do not use the subscript B in the notation of these cones. Example 2.1.16. We want to compute the tangent and normal cones, at different points, to the set M ⊂ Rp , M = ( x = (x1, x2, ..., xp) ∈ Rp | xi ≥ 0, ∀i ∈ 1, p, p X i=1 xi = 1 ) , which is called the unit simplex. This set is convex and closed. According to the previous result, for every x ∈ M, T(M, x) = cl R+(M − x) = cl u ∈ Rp | ∃α ≥ 0, x ∈ M, u = α(x − x) . Take u from the right-hand set. It is clear that, on one hand, Pp i=1 ui = 0, and, on the other hand, if xi = 0, then ui ≥ 0. Denote by I(x) := i ∈ 1, p | xi = 0 . It follows that T(M, x) ⊂ ( u ∈ Rp | p X i=1 ui = 0 and ui ≥ 0, ∀i ∈ I(x) ) . Let us now prove the reverse inclusion. It is easy to verify that the right-hand set is closed. Take u from this set. If u = 0, then, obviously, u ∈ T(M, x). If u ≠ 0, then we must prove that there exists α 0 such that x+αu ∈ M. On one hand, it is clear that Pp i=1(xi +αui) = 1 is satisfied for any α. If there is no i with ui 0, then it is also easy to observe that xi + αui ≥ 0, for any i ∈ 1, p, hence u ∈ T(M, x). Suppose that the set J of indices for which uj 0 is nonempty. Then J ⊂ 1, p I(x), hence xj 0 for any j ∈ J. One can choose the positive α such that α min{−u−1 j xj | j ∈ J},
- 42. 28 | Nonlinear Analysis Fundamentals and again one has xi + αui ≥ 0, for any i ∈ 1, p. Hence, u ∈ T(M, x), and the double inclusion follows. We prove next that N(M, x) = (a, a, ..., a) ∈ Rp | a ∈ R + v ∈ Rp | vi ≤ 0, ∀i ∈ I(x), vi = 0, i ∉ I(x) . For this, consider the elements a0 = (1, 1, ..., 1), a1 = −(1, 0, ..., 0), ..., an = −(0, 0, ..., 1) and observe that T(M, x) can be equivalently written as: T(M, x) = u ∈ Rp | ha0, ui ≤ 0, h−a0, ui ≤ 0, hai, ui ≤ 0, ∀i ∈ I(x) . The polar of this set is N(M, x) = αa0 − βa0 + X i∈I(x) αiai | α, β, αi ≥ 0, ∀i ∈ I(x) . Indeed, the fact that the right-hand set is contained in the normal cone is obvious, and the reverse inclusion follows from Farkas’ Lemma (Theorem 2.1.8). We now obtain the desired form of the normal cone. At the end of this section, we discuss the concepts of convex hull and conic hull of a set. Let A ⊂ Rp be a nonempty set. The convex hull of A is the set conv A = ( n X i=1 αixi | n ∈ N* , (αi)i∈1,n ⊂ [0, ∞), n X i=1 αi = 1, (xi)i∈1,n ⊂ A ) . It is not difficult to see that conv A is a convex set which contains A. One can easily verify that conv A is the smallest set (in the sense of inclusion) with these properties (see Problem 7.26). The conic hull of the set A is cone A := [0, ∞)A := {αx | α ≥ 0, x ∈ A} . In fact, cone A is the smallest cone which contains A. We give next two results concerning these sets. The first one refers to the structure of the set conv A and it is called the Carathéodory Theorem, after the name of the Greek mathematician Constantin Carathéodory, who proved this result in 1911 for compact sets. Theorem 2.1.17 (Carathéodory Theorem). Let A ⊂ Rp be a nonempty set. Then conv A = (p+1 X i=1 αixi | (αi)i∈1,p+1 ⊂ [0, ∞), p+1 X i=1 αi = 1, (xi)i∈1,p+1 ⊂ A ) .
- 43. Convex Sets and Cones | 29 Proof We must prove that every element from conv A can be written as a combination of at most p + 1 elements from A. Consider x ∈ conv A. According to the definition of conv A, x can be written as a convex combination of elements from A. Suppose, by means of contradiction, that the minimal number of elements from A which can form a convex combination equal to x is n p + 1. So there exist x1, x2, ..., xn ∈ A, α1, α2, ..., αn ∈ (0, 1) with Pn i=1 αi = 1 such that Pn i=1 αixi = x. Then the elements (xi − xn)i=1,n−1 are linearly dependent (their number is greater than the dimension p of the space), so there exist (λi)i=1,n−1, not all equal to 0, such that n−1 X i=1 λi(xi − xn) = 0, which means n−1 X i=1 λixi − n−1 X i=1 λi ! xn = 0. By denoting − Pn−1 i=1 λi = λn, one has Pn i=1 λi = 0 and Pn i=1 λixi = 0. Then for every t ∈ R, x = n X i=1 αixi + t n X i=1 λixi = n X i=1 (αi + tλi)xi and n X i=1 (αi + tλi) = 1. As Pn i=1 λi = 0 and there is at least one nonzero element, there exists at least one negative value among the numbers (λi)i∈1,n. Denote t := min{−αiλ−1 i | λi 0}. Then all the values (αi + tλi) are in the interval [0, ∞), and the corresponding value of the index which gives the minimum from above is zero, whence x is a convex combina- tion of less than n elements from A, contradicting the minimality of n. Therefore, the assumption that we made was false, and the conclusion follows. We now discuss the necessary conditions one needs in order that the conic hull of a set is closed. This does not happens automatically, as one can see from the example given by A ⊂ R2 , A := {(x, y) ∈ R2 | (x − 1)2 + y2 = 1}, for which cone A = {(x, y) ∈ R2 | x 0} ∪ {(0, 0)}. First, one defines for a nonempty set A ⊂ X the asymptotic cone of A as A∞ = {u ∈ X | ∃(tn) → 0, ∃(an) ⊂ A, tnan → u}. It is clear, by repeating the arguments from the case of the Bouligand tangent cone, that A∞ is a closed cone. If A is bounded, then A∞ = {0}, and the converse also holds (if A would contain an unbounded sequence (an), then an kank would also have a
- 44. 30 | Nonlinear Analysis Fundamentals subsequence which converges to a nonzero element, which must be from A∞ ). Let us observe also that if A is a cone, then A∞ = TB(A, 0) = cl A. The next result concerns decomposition. Theorem 2.1.18. Let A ⊂ Rp be a nonempty closed set. (i) If 0 ∉ A, then cl cone A = cone A ∪ A∞ . (ii) If 0 ∈ A, then cl cone A = cone A ∪ A∞ ∪ TB(A, 0). Proof (i) Suppose that 0 ∉ A. It is clear that cone A ⊂ cl cone A. From the def- inition of A∞ , one also has that A∞ ⊂ cl cone A. For the reverse inclusion, take (un) ⊂ cone A, un → u. We must prove that u ∈ cone A ∪ A∞ . If u = 0, then the relation u ∈ cone A is obvious. Suppose that u ≠ 0, then for every n ∈ N, there ex- ist tn ≥ 0 and an ∈ A such that un = tnan. If (an) is unbounded, one can pass to a subsequence (ank ), where kank k → ∞. Hence tnk → 0, and u ∈ A∞ . Suppose (an) is bounded. Since 0 ∉ A and A is closed, there exists γ 0 such that kank ≥ γ for every n. One can deduce that (tn) is bounded, so it converges (on a subsequence (tnk ), even- tually) to a number t ≥ 0. If t = 0, then unk → 0 = u (a situation which is excluded at this point of the proof). Accordingly, t 0 and ank − t−1 u = t−1 ktank − uk = t−1 tnk ank − u + (t − tnk )ank ≤ t−1 ktnk ank − uk + |t − tnk | t−1 kank k → 0. Hence ank → t−1 u and since A is closed, u ∈ cone A. The proof of this part is complete. (ii)Theinclusion cone A∪A∞ ∪TB(A, 0) ⊂ cl cone A isobvious.Take u ∈ cl cone A. In the above it is possible as well that (an) converges to 0. Then tn → ∞, hence u ∈ TB(A, 0), which completes the proof. We finish with the following characterization result: Corollary 2.1.19. Let A ⊂ Rp be a nonempty closed set. (i) If 0 ∉ A, then cone A is closed if and only if A∞ ⊂ cone A. (ii) If 0 ∈ A, then cone A is closed if and only if A∞ ∪ TB(A, 0) ⊂ cone A. 2.2 Convex Functions 2.2.1 General Results In this section we present the special class of convex functions. These functions are defined on convex sets. Definition 2.2.1. Let D ⊂ Rp be a convex set. One says that a function f : D → R is convex if f(λx + (1 − λ)y) ≤ λf(x) + (1 − λ)f(y), ∀x, y ∈ D, ∀λ ∈ [0, 1]. (2.2.1)
- 45. Convex Functions | 31 It is clear that in the above definition it is sufficient to take λ ∈ (0, 1). As said before, in R the convex sets are exactly the intervals. In this framework, the convexity has the following geometric meaning: for every two points x, y ∈ D, x y, the graph of the restriction of f to the [x, y] interval lies below the line segment joining the points (x, f(x)) and (y, f(y)). This can be written as follows: for every u ∈ [x, y], f(u) ≤ f(x) + f(y) − f(x) y − x (u − x), (2.2.2) inequality which can be deduced from (2.2.1) by replacing λ with the value given by the relation u = λx +(1− λ)y. Therefore, (2.2.1) and (2.2.2) are equivalent (for functions defined on R). Definition 2.2.2. Let D ⊂ Rp be a convex set. One says that a function f : D → R is concave if −f is convex. All of the properties of concave functions can be easily deduced from the similar prop- erties of the convex functions, so in what follows we will consider only the later case. We first deduce some general properties of the convex functions. Proposition 2.2.3. Let D ⊂ Rp be a convex set and f : D → R. The following relations are equivalent: (i) f is convex; (ii) the epigraph of f , epi f := {(x, t) ∈ D × R | f(x) ≤ t}, is a convex subset of Rp × R; (iii) for any x, y ∈ D, define Ix,y := {t ∈ R | tx + (1 − t)y ∈ D}; then the function φx,y : Ix,y→ R, φx,y(t) = f(tx + (1 − t)y) is convex. Proof We prove first the implication from (i) to (ii). Take λ ∈ (0, 1) and (x, t), (y, s) ∈ epi f . By the convexity of D, one knows that λx +(1− λ)y ∈ D, and using the convexity of f , one can say: f(λx + (1 − λ)y) ≤ λf(x) + (1 − λ)f(y) ≤ λt + (1 − λ)s, i.e., λx + (1 − λ)y, λt + (1 − λ)s ∈ epi f . Therefore, epi f is a convex set. We prove now the converse implication. Take x, y ∈ D and λ ∈ [0, 1]. Then (x, f(x)), (y, f(y)) ∈ epi f and by assumption, λ(x, f(x)) + (1 − λ)(y, f(y)) ∈ epi f , hence f(λx + (1 − λ)y) ≤ λf(x) + (1 − λ)f(y), which shows that f is a convex function.
- 46. 32 | Nonlinear Analysis Fundamentals We prove now the equivalence between (i) and (iii). Observe first that Ix,y is an interval which contains [0, 1]. Suppose that f is convex and take u, v ∈ Ix,y, λ ∈ [0, 1]. One knows that: φx,y(λu + (1 − λ)v) = f([λu + (1 − λ)v]x + [1 − λu − (1 − λ)v]y) = f(λ(ux + (1 − u)y) + (1 − λ)(vx + (1 − v)y)) ≤ λf(ux + (1 − u)y) + (1 − λ)f(vx + (1 − v)y) = λφx,y(u) + (1 − λ)φx,y(v). For the converse implication, take x, y ∈ D and t ∈ [0, 1]. Then φx,y is convex, hence for every λ ∈ [0, 1], u, v ∈ Ix,y φx,y(λu + (1 − λ)v) ≤ λφx,y(u) + (1 − λ)φx,y(v) = λf(ux + (1 − u)y) + (1 − λ)f(vx + (1 − v)y). By taking u = 1, v = 0, λ = t we deduce that φx,y(t) ≤ tf(x) + (1 − t)f(y), hence f is convex. Theorem 2.2.4. Let D ⊂ Rp be a convex set and f : D → R be a convex function. Then f is continuous at every interior point of D. Proof Take x ∈ int D. A translation permits us to consider the case x = 0. We prove first that f is bounded on a neighborhood of 0. If we denote by (ei)i∈1,p the canonical base of Rp , then there exists an a 0 such that aei and −aei are in D for any i ∈ 1, p. Under these conditions, the set V := ( x ∈ Rp | x = p X i=1 xiei, |xi| a p , ∀i ∈ 1, p ) is a neighborhood of 0 contained in D. For x ∈ V, there exist (xi)i∈1,p with |xi| a p , i ∈ 1, p and x = Pp i=1 xiei. Suppose first that xi ≠ 0 for any i ∈ 1, p. One has f(x) = f p X i=1 xiei ! = f p X i=1 |xi| a a xi |xi| ei + 1 − p X i=1 |xi| a ! 0 ! ≤ p X i=1 |xi| a f a xi |xi| ei + 1 − p X i=1 |xi| a ! f(0) ≤ max{f (aei) , f(−aei) | i ∈ 1, p} + f(0) . We can now observe that if there are indices i for which xi = 0, then these can be excluded from the above calculations, and the estimation holds.
- 47. Convex Functions | 33 Since the right-hand part is a constant (which we denote by M), the proof is fin- ished. Take ε ∈ (0, 1) and U a symmetric neighborhood of 0 such that ε−1 U ⊂ V. Then, for any x ∈ U, f(x) = f ε(ε−1 x) + (1 − ε)0 ≤ εf(ε−1 x) + (1 − ε)f(0) ≤ εM + (1 − ε)f(0), i.e., f(x) − f(0) ≤ εM − εf(0). From the fact that U is symmetric, one deduces that for every x ∈ U f(−x) ≤ εM + (1 − ε)f(0). Moreover, f(0) = f 1 2 x + 1 2 (−x) ≤ 1 2 f(x) + 1 2 f(−x) ≤ 1 2 f(x) + 1 2 εM + (1 − ε)f(0) , hence f(0) − f(x) ≤ εM − εf(0). This relation can be combined with the similar one from above, and gives f(x) − f(0) ≤ εM − εf(0). This inequality proves the continuity of f at 0. We want to emphasize now some characterizations of differentiable convex func- tions. Some preliminary results on convex functions defined on a real intervals are necessary. Proposition 2.2.5. Let I ⊂ R be an interval and f : I → R be a function. The next relations are equivalent: (i) f is convex; (ii) for every x1, x2, x3 ∈ I satisfying the relation x1 x2 x3 one has f(x2) − f(x1) x2 − x1 ≤ f(x3) − f(x1) x3 − x1 ≤ f(x3) − f(x2) x3 − x2 ; (iii) for every a ∈ int I, the function g : I {a} → R given by g(x) = f(x) − f(a) x − a is increasing. Proof We prove the (i) ⇒ (ii) implication. Take λ = x2 − x1 x3 − x1 ∈ (0, 1). Then the equality x2 = λx3 + (1 − λ)x1 holds and one must prove now that f(x2) − f(x1) λ(x3 − x1) ≤ f(x3) − f(x1) x3 − x1 ≤ f(x3) − f(x2) (1 − λ)(x3 − x1) .
- 48. 34 | Nonlinear Analysis Fundamentals After some calculations, one can show that: f(x2) ≤ λf(x3) + (1 − λ)f(x1). The proof of the implication (ii) ⇒ (i) follows the inverse path of the proof of (i) ⇒ (ii), hence (i) and (ii) are equivalent. In order to prove (ii) ⇒ (iii), we fix x1, x2 ∈ I {a} with x1 x2 and we find three situations. If x1 x2 a, then we apply (ii) for the triplet (x1, x2, a). If x1 a x2, then we apply (ii) for the triplet (x1, a, x2). Finally, if a x1 x2, then we apply (ii) for the triplet (a, x1, x2). We prove now (iii) ⇒ (i). Take x, y ∈ I with x y and λ ∈ (0, 1). Then x λx + (1 − λ)y y, and by applying (iii) with a = λx + (1 − λ)y, one deduces f(x) − f(λx + (1 − λ)y) x − λx − (1 − λ)y ≤ f(y) − f(λx + (1 − λ)y) y − λx − (1 − λ)y . After some calculations, the relation follows from the definition of convexity. The fact that this relation holds for any x, y ∈ I with x y and for any λ ∈ (0, 1) is sufficient to prove the desired assertion. The proof is complete. Proposition 2.2.6. Let I ⊂ R be an interval and f : I → R be a convex function. Then f admits lateral derivatives in every interior point of I and for every x, y ∈ int I with x y, one has f0 −(x) ≤ f0 +(x) ≤ f0 −(y) ≤ f0 +(y). Proof Fix a ∈ int I. Since the function g : I {a} → R given by g(x) = f(x) − f(a) x − a is increasing (see the previous result) one deduces that g admits finite lateral limits, which implies the existence of the lateral derivatives of f at a. Moreover, f0 −(a) ≤ f0 +(a). For x, y ∈ int I, x y and for any u, v ∈ (x, y), u ≤ v, by using again the argument given by the last conclusion of Proposition 2.2.5, one deduces that f(u) − f(x) u − x ≤ f(v) − f(x) v − x = f(x) − f(v) x − v ≤ f(y) − f(v) y − v = f(v) − f(y) v − y . Passing to the limit for u → x and v → y, one gets f0 +(x) ≤ f0 −(y). Here we characterize differentiable convex functions of one variable. Theorem 2.2.7. Let I be an open interval and f : I → R be a function. (i) If f is differentiable on I, then f is convex if and only if f0 is increasing on I. (ii) If f is twice differentiable on I, then f is convex if and only if f00 (x) ≥ 0 for every x ∈ I.
- 49. Convex Functions | 35 Proof In the case of real functions of one variable, the equivalence between the mono- tonicity of f0 and the sign of f00 is sufficient to prove that f is convex if and only if f0 is increasing on I. If f is convex, the monotonicity of the derivative follows from Propo- sition 2.2.6. Conversely, suppose that f0 is increasing and we prove that f is convex. Take a, b ∈ I. Define g : [a, b] → R given by g(x) = f(x) − f(a) − (x − a) f(b) − f(a) b − a . Obviously, g(a) = g(b) = 0, and g0 (x) = f0 (x) − f(b) − f(a) b − a . The function f satisfies the conditions of Lagrange Theorem on [a, b], hence there exists c ∈ (a, b) such that f(b) − f(a) b − a = f0 (c). Consequently, g0 (x) = f0 (x) − f0 (c). From the monotonicity of f0 , we deduce that g is decreasing on (a, c) and increasing on (c, b), and since g(a) = g(b) = 0, we know that g is negative on the whole interval [a, b]. Take x ∈ (a, b). Then there exists λ ∈ (0, 1) such that x = λa + (1 − λ)b. By replacing x in the expression of g and taking into account that g(x) ≤ 0, we deduce that f(λa + (1 − λ)b) − f(a) − (1 − λ)(b − a) f(b) − f(a) b − a ≤ 0, relation which reduces to the definition of the convexity. Example 2.2.8. Based on the above result, one deduces the convexity of the following functions: f : R → R, f(x) = ax + b, with a, b ∈ R; f : (0, ∞) → R, f(x) = − ln x; f : (0, ∞) → R, f(x) = x ln x; f : (0, ∞) → R, f(x) = xa , a ≥ 1; f : R → R, f(x) = ex ; f : ( − 1, 1) → R, f(x) = − √ 1 − x2; f : (0, π) → R, f(x) = sin−1 x. Another example is given by the next result. Proposition 2.2.9. Let D ⊂ Rp be a nonempty convex set. Then the function dD : Rp → R given by dD(x) = d(x, D) is convex. Proof Take x, y ∈ Rp and α ∈ [0, 1]. For any ε 0, there exist dx,ε, dy,ε ∈ D such that kdx,ε − xk dD(x) + ε kdy,ε − yk dD(y) + ε.
- 50. 36 | Nonlinear Analysis Fundamentals Using the convexity of D, one knows: dD(αx + (1 − α)y) ≤ αx + (1 − α)y − (αdx,ε + (1 − α)dy,ε) ≤ α kdx,ε − xk + (1 − α) kdy,ε − yk αdD(x) + (1 − α)dD(y) + ε. As ε is arbitrarily chosen, we may pass to the limit for ε → 0 and the conclusion follows. We now characterize differentiable convex functions in the general case. Theorem 2.2.10. Let D ⊂ Rp be an open convex set and f : D → R be a function. (i) If f is differentiable on D, then f is convex if and only if for any x, y ∈ D, f(y) ≥ f(x) + ∇f(x)(y − x). (ii) If f is twice differentiable on D, then f is convex if and only if for every x ∈ D and y ∈ Rp , one has ∇2 f(x)(y, y) ≥ 0. Proof (i) Consider first the case when p = 1. Fix x ∈ D with y ≠ x and take λ ∈ (0, 1]. Since f is convex, we get f (x + λ (y − x)) = f ((1 − λ) x + λy) ≤ (1 − λ) f(x) + λf(y) = f(x) + λ (f (y) − f (x)) . Consequently, f (x + λ (y − x)) − f (x) λ (y − x) · (y − x) ≤ f (y) − f (x) . Passing to the limit for λ → 0, one gets f0 (x) (y − x) ≤ f (y) − f (x). We pass now to the general case. For x ∈ D, consider the function φy,x from Propo- sition 2.2.3, which we already know that is convex. Moreover, φy,x is differentiable on the open interval Iy,x, and φ0 y,x(t) = ∇f(ty + (1 − t)x)(y − x). According to the preceding step, φy,x(1) ≥ φy,x(0) + φ0 y,x(0) which means that f(y) ≥ f(x) + ∇f(x)(y − x).
- 51. Convex Functions | 37 Conversely, fix x, y ∈ D and λ ∈ [0, 1]. Therefore, by assumption, f (x) ≥ f λx + (1 − λ)y + (1 − λ)∇f λx + (1 − λ)y (x − y) and f (y) ≥ f λx + (1 − λ)y + λ∇f λx + (1 − λ)y (y − x) . Multiplying the first inequality by λ, the second one by (1 − λ), and summing up the new inequalities, one obtains λf (x) + (1 − λ)f (y) ≥ f λx + (1 − λ)y , which proves that f is convex. (ii) The case p = 1 is proved in Theorem 2.2.7. Now, in order to pass to the general case, take x ∈ D, y ∈ Rp . Suppose f is convex. Since D is open, there exists an α 0 such that u := x + αy ∈ D. According to the assumption, φu,x is convex, and taking into account the case that we have already studied, φ00 u,x(t) ≥ 0 for any t ∈ Iu,x. For t = 0, one deduces that 0 ≤ φ00 u,x(0) = ∇2 f(x)(u − x, u − x), and the conclusion follows. Conversely, for x, y ∈ D and t ∈ Ix,y, φ00 x,y(t) ≥ 0. From the case p = 1, we get that φx,y is convex, hence f is convex. The proof is now complete. From these results, one may observe that some properties of the convex functions have a global character, an aspect which will persist in subsequent sections. At the end of this subsection, we will discuss a property which is stronger than convexity. Definition 2.2.11. Let D ⊂ Rp be a convex set. One says that a function f : D → R is strictly convex if f(λx + (1 − λ)y) λf(x) + (1 − λ)f(y), ∀x, y ∈ D, x ≠ y, ∀λ ∈ (0, 1). Definition 2.2.12. Let D ⊂ Rp be a convex set. One says that a function f : D → R is strictly concave if −f is strictly convex. Every strictly convex function is convex, but the converse is false. To see this, consider a convex function which is constant on an interval. Again, the properties of the strictly concave functions easily follow from the corresponding ones of the strictly convex functions. By the use of very similar arguments as in the proofs of preceding results, one can deduce the next characterizations.
- 52. 38 | Nonlinear Analysis Fundamentals Theorem 2.2.13. Let I be an open interval and f : I → R be a differentiable function. The next assertions are equivalent: (i) f is strictly convex; (ii) f (x) f (a) + f0 (a) (x − a) , for any x, a ∈ I, x ≠ a; (iii) f0 is strictly increasing. If, moreover, f is twice differentiable (on I), then one more equivalence holds : (iv) f00 (x) ≥ 0 for any t ∈ I and {x ∈ I | f00 (x) = 0} does not contain any proper interval. Example 2.2.14. Using this result, one gets the strict convexity of the following func- tions: f : (0, ∞) → R, f(x) = − ln x; f : (0, ∞) → R, f(x) = x ln x; f : (0, ∞) → R, f(x) = xa , a 1; f : R → R, f(x) = ex ; f : (0, ∞) → R, f(x) = (1 + xp ) 1 p , p 1. Theorem 2.2.15. Let D ⊂ Rp an open convex set and f : D → R be a differentiable function. The next assertions are equivalent: (i) f is strictly convex; (ii) f (x) f (a) + ∇f(a) (x − a) , for any x, a ∈ D, x ≠ a. If, moreover, f is twice differentiable (on D), then the preceding two items are implied by the relation: (iii) ∇2 f(x)(y, y) 0 for any x ∈ D and y ∈ Rp {0}. 2.2.2 Convex Functions of One Variable In this subsection we will focus on some properties and applications of convex func- tions defined on real intervals, even if some of the results hold in more general situa- tions. The class of convex functions is stable under several algebraic operations, an as- pect which makes it very useful. Here are some of these operations: Proposition 2.2.16. Let I, J ⊂ R be intervals. (i) Let n ∈ N* and f1, f2, ..., fn : I → R be convex functions, and λ1, λ2, ..., λn ≥ 0. Then Pn i=1 λifi is convex. If at least one of the functions is strictly convex, and the corresponding scalar is not zero, then Pn i=1 λifi is strictly convex. (ii) Let f : I → J be (strictly) convex, g : J → R be convex and (strictly) increasing. Then g ◦ f is (strictly) convex. (iii) Let f : I → J be strictly decreasing, (strictly) convex, and surjective. Then f−1 is (strictly) convex. The next result emphasizes some monotonicity properties of convex functions of one variable.
- 53. Convex Functions | 39 Theorem 2.2.17. Let I be a nondegenerate interval (i.e., not a singleton set) and f : I → R be convex. Then either f is monotone on int I, or there exists x ∈ int I such that f is decreasing on I ∩ (−∞, x], and increasing on I ∩ [x, ∞). Proof Because of relation (2.2.2), it is sufficient to restrict our attention to the case when I is open, i.e., I = int I. Suppose that f is not monotone on I. Then there exist a, b, c ∈ I, a b c such that f(a) f(b) f(c) or f(a) f(b) f(c). The second situation cannot hold, since in that case, using (2.2.2), one would have f(b) ≤ f(a) + f(c) − f(a) c − a (b − a) = f(a)(c − b) + f(c)(b − a) c − a f(b). Hence, f(a) f(b) f(c). As f is continuous on [a, c], its minimum on this interval must be attained at a point x (from the Weierstrass Theorem). Take x ∈ I ∩ (−∞, a). According to Proposition 2.2.5, f(x) − f(x) x − x ≤ f(a) − f(x) a − x , i.e., (x − a)f(x) ≥ (x − a)f(x) + (x − x)f(a) ≥ (x − a)f(x), hence f(x) ≤ f(x). Similarly, one can prove that f(x) ≤ f(x) for x ∈ I ∩ (c, ∞). It follows that f(x) = inf f(I). We next prove that f is decreasing on I ∩ (−∞, x). Take u, v ∈ I ∩ (−∞, x), u v. On one hand, f(v) − f(u) v − u ≤ f(x) − f(u) x − u , and on the other hand, f(x) − f(u) x − u = f(u) − f(x) u − x ≤ f(v) − f(x) v − x ≤ 0, hence f(v) − f(u) ≤ 0, which is exactly what we wanted to prove. Similarly, one can show that f is increasing on I ∩ (x, +∞). The continuity of f on I finalizes the proof. As we saw before, a convex function defined on an interval can have discontinu- ities only at the extremities of the interval. The previous theorem allows us to consider a different function in those eventual discontinuity points, without losing the convex- ity. In this way one obtains the next consequence. Corollary 2.2.18. Let a, b ∈ R, a b and f : [a, b] → R be a convex function. Then there exist limx→a+ f(x) and limx→b− f(x), and the function f(x) = limx→a+ f(x), x = a f(x), x ∈ (a, b) limx→b− f(x), x = b is convex and continuous on [a, b].
- 54. 40 | Nonlinear Analysis Fundamentals From the proof of Theorem 2.2.17 one can also obtain the next result, which will be restated in a more general framework in the next chapter. Corollary 2.2.19. Let I ⊂ R be a nondegenerate interval. If f : I → R is a convex and non-monotone function, then it has a global minimum on int I. The next proposition, sometimes called the Jensen inequality, follows by applying the definitions and the mathematical induction principle. Proposition 2.2.20. Let D ⊂ Rp be a convex set and f : D → R. If the function f is convex, then f(λ1x1 + ... + λmxm) ≤ λ1f(x1) + ... + λmf(xm) for any m ∈ N* , x1, ..., xm ∈ D, λ1, ..., λm ≥ 0, λ1 + ... + λm = 1. The inequality is strict if the function f is strictly convex, at least two of the points (xk) are different and the corresponding scalars (λk) are strictly positive. Actually, for the convexity of a continuous function it is sufficient that the inequality from the definition is satisfied for λ = 2−1 . Theorem 2.2.21. Let I ⊂ R be an interval and f : I → R be a continuous function. The function f is convex if and only if f x + y 2 ≤ f(x) + f(y) 2 , ∀x, y ∈ I. (2.2.3) Proof The necessity of the condition (2.2.3) is obvious. Let us prove that it is also suf- ficient. Suppose, by contradiction, that f is not convex, which means (2.2.2) is not satisfied. Then there exist x, y ∈ I, x y and u ∈ (x, y) such that f(u) f(x) + f(y) − f(x) y − x (u − x). (2.2.4) Observe by the relation (2.2.4) that the cases u = x and u = y cannot hold. Consider then g : [x, y] → R, g(t) = f(t) − f(x) − f(y) − f(x) y − x (t − x), which is continuous and satisfies the relations g(x) = g(y) = 0. By (2.2.4) and Weier- strass’ Theorem, there exists z ∈ (x, y) such that g(z) = supt∈[x,y] g(t) 0. Denote by w := inf{z ∈ (x, y) | g(z) = sup t∈[x,y] g(t)}. By the continuity of g, it follows that g(w) = supt∈[a,b] g(t) 0, and hence w ∈ (x, y). Consequently, there exists h 0 such that w + h, w − h ∈ (x, y). But w = 2−1 (w + h) +
- 55. Random documents with unrelated content Scribd suggests to you:
- 56. From these facts, few as they are, there can be little doubt that dyeing, and even calico-printing, had made considerable progress among the ancients; and this could not have taken place without a considerable knowledge of colouring matters, and of the mordants by which these colouring matters were fixed. These facts, however, were probably but imperfectly understood, and could not be the means of furnishing the ancients with any accurate chemical knowledge. VI.—SOAP. Soap, which constitutes so important and indispensable an article in the domestic economy of the moderns, was quite unknown to the ancient inhabitants of Asia, and even of Greece. No allusion to it occurs in the Old Testament. In Homer, we find Nausicaa, the daughter of the King of the Phæacians, using nothing but water to wash her nuptial garments: They seek the cisterns where Phæacian dames Wash their fair garments in the limped streams; Where gathering into depth from falling rills, The lucid wave a spacious bason fills. The mules unharness’d range beside the main, Or crop the verdant herbage of the plain. Then emulous the royal robes they lave, And plunge the vestures in the cleansing wave. Odyssey, vi. 1. 99. We find, in some of the comic poets, that the Greeks were in the habit of adding wood-ashes to water to make it a better detergent. Wood-ashes contain a certain portion of carbonate of potash, which of course would answer as a detergent; though, from
- 57. its caustic qualities, it would be injurious to the hands of the washerwomen. There is no evidence that carbonate of soda, the nitrum of the ancients, was ever used as a detergent; this is the more surprising, because we know from Pliny that it was employed in dyeing, and one cannot see how a solution of it could be employed by the dyers in their processes without discovering that it acted powerfully as a detergent. The word soap (sapo) occurs first in Pliny. He informs us that it was an invention of the Gauls, who employed it to render their hair shining; that it was a compound of wood-ashes and tallow, that there were two kinds of it, hard and soft (spissus et liquidus); and that the best kind was made of the ashes of the beech and the fat of goats. Among the Germans it was more employed by the men than the women.88 It is curious that no allusion whatever is made by Pliny to the use of soap as a detergent; shall we conclude from this that the most important of all the uses of soap was unknown to the ancients? It was employed by the ancients as a pomatum; and, during the early part of the government of the emperors, it was imported into Rome from Germany, as a pomatum for the young Roman beaus. Beckmann is of opinion that the Latin word sapo is derived from the old German word sepe, a word still employed by the common people of Scotland.89 It is well known that the state of soap depends upon the alkali employed in making it. Soda constitutes a hard soap, and potash a soft soap. The ancients being ignorant of the difference between the two alkalies, and using wood-ashes in the preparation of it, doubtless formed soft soap. The addition of some common salt, during the boiling of the soap, would convert the soft into hard soap. As Pliny informs us that the ancients were acquainted both with hard and soft soap, it is clear that they must have followed some such process.
- 58. VII.—STARCH. The manufacture of starch was known to the ancients. Pliny informs us that it was made from wheat and from siligo, which was probably a variety or sub-species of wheat. The invention of starch is ascribed by Pliny to the inhabitants of the island of Chio, where in his time the best starch was still made. Pliny’s description of the method employed by the ancients of making starch is tolerably exact. Next to the China starch that of Crete was most celebrated; and next to it was the Egyptian. The qualities of starch were judged of by the weight; the lightest being always reckoned the best. VIII.—BEER. That the ancients were acquainted with wine is universally known. This knowledge must have been nearly coeval with the origin of society; for we are informed in Genesis that Noah, after the flood, planted a vineyard, and made wine, and got intoxicated by drinking the liquid which he had manufactured.90 Beer also is a very old manufacture. It was in common use among the Egyptians in the time of Herodotus, who informs us that they made use of a kind of wine made from barley, because no vines grew in their country.91 Tacitus informs us, that in his time it was the drink of the Germans.92 Pliny informs us that it was made by the Gauls, and by other nations. He gives it the name of cerevisia or cervisia; the name obviously alluding to the grain from which it was made. But though the ancients seem acquainted with both wine and beer, there is no evidence of their having ever subjected these liquids to distillation, and of having collected the products. This would have furnished them with ardent spirits or alcohol, of which
- 59. there is every reason to believe they were entirely ignorant. Indeed, the method employed by Dioscorides to obtain mercury from cinnabar, is a sufficient proof that the true process of distillation was unknown to them. He mixed cinnabar with iron filings, put the mixture into a pot, to the top of which a cover of stoneware was luted. Heat was applied to the pot, and when the process was at an end, the mercury was found adhering to the inside of the cover. Had they been aware of the method of distilling the quicksilver ore into a receiver, this imperfect mode of collecting only a small portion of the quicksilver, separated from the cinnabar, would never have been practised. Besides, there is not the smallest allusion to ardent spirits, either in the writings of the poets, historians, naturalists, or medical men of ancient Greece; a circumstance not to be accounted for had ardent spirits been known, and applied even to one-tenth of the uses to which they are put by the moderns. IX.—STONEWARE. The manufacture of stoneware vessels was known at a very early period of society. Frequent allusions to the potter’s wheel occur in the Old Testament, showing that the manufacture must have been familiar to the Jewish nation. The porcelain of the Chinese boasts of a very high antiquity indeed. We cannot doubt that the processes of the ancients were similar to those of the moderns, though I am not aware of any tolerably accurate account of them in any ancient author whatever. Moulds of plaster of Paris were used by the ancients to take casts precisely as at present.93 The sand of Puzzoli was used by the Romans, as it is by the moderns, to form a mortar capable of hardening under water.
- 60. Pliny gives us some idea of the Roman bricks, which are known to have been of an excellent quality. There were three sizes of bricks used by the Romans. 1. Lydian, which were 1½ foot long and 1 foot broad. 2. Tetradoron, which was a square of 16 inches each side. 3. Pentadoron, which was a square, each side of which was 20 inches long. Doron signifies the palm of the hand: of course it was equivalent to 4 inches. X.—PRECIOUS STONES AND MINERALS. Pliny has given a pretty detailed description of the precious stones of the ancients; but it is not very easy to determine the specific minerals to which he alludes. 1. The description of the diamond is tolerably precise. It was found in Ethiopia, India, Arabia, and Macedonia. But the Macedonian diamond, as well as the adamas cyprius and siderites, were obviously not diamonds, but soft stones. 2. The emerald of the ancients (smaragdus) must have varied in its nature. It was a green, transparent, hard stone; and, as colour was the criterion by which the ancients distinguished minerals and divided them into species, it is obvious that very different minerals must have been confounded together, under the name of emerald. Sapphire, beryl, doubtless fluor spar when green, and probably even serpentine, nephrite, and some ores of copper, seem to have occasionally got the same name. There is no reason to believe that the emerald of the moderns was known before the discovery of
- 61. America. At least it has been only found in modern times in America. Some of the emeralds described by Pliny as losing their colour by exposure to the sun, must have been fluor spars. There is a remarkably deep and beautiful green fluor spar, met with some years ago in the county of Durham, in one of the Weredale mines that possesses this property. The emeralds of the ancients were of such a size (13½ feet, large enough to be cut into a pillar), that we can consider them in no other light than as a species of rock. 3. Topaz of the ancients had a green colour, which is never the case with the modern topaz. It was found in the island Topazios, in the Red Sea.94 It is generally supposed to have been the chrysolite of the moderns. But Pliny mentions a statue of it six feet long. Now chrysolite never occurs in such large masses. Bruce mentions a green substance in an emerald island in the Red Sea, not harder than glass. Might not this be the emerald of the ancients? 4. Calais, from the locality and colour was probably the Persian turquoise, as it is generally supposed to be. 5. Whether the prasius and chrysoprasius of Pliny were the modern stones to which these names are given, we have no means of determining. It is generally supposed that they are, and we have no evidence to the contrary. 6. The chrysolite of Pliny is supposed to be our topaz: but we have no other evidence of this than the opinion of M. Du Tems. 7. Asteria of Pliny is supposed by Saussure to be our sapphire. The lustre described by Pliny agrees with this opinion. The stone is said to have been very hard and colourless. 8. Opalus seems to have been our opal. It is called, Pliny says, pæderos by many, on account of its beauty. The Indians called it sangenon.
- 62. 9. Obsidian was the same as the mineral to which we give that name. It was so called because a Roman named Obsidianus first brought it from Egypt. I have a piece of obsidian, which the late Mr. Salt brought from the locality specified by Pliny, and which possesses all the characters of that mineral in its purest state. 10. Sarda was the name of carnelian, so called because it was first found near Sardis. The sardonyx was also another name for carnelian. 11. Onyx was a name sometimes given to a rock, gypsum; sometimes it was a light-coloured chalcedony. The Latin name for chalcedony was carchedonius, so called because Carthage was the place where this mineral was exposed to sale. The Greek name for Carthage was Καρχηδων (carchedon). 12. Carbunculus was the garnet; and anthrax was a name for another variety of the same mineral. 13. The oriental amethyst of Pliny was probably a sapphire. The fourth species of amethyst described by Pliny, seems to have been our amethyst. Pliny derives the name from α (a) and μυθη (mythe), wine, because it has not quite the colour of wine. But the common derivation is from α and μυθυω, to intoxicate, because it was used as an amulet to prevent intoxication. 14. The sapphire is described by Pliny as always opaque, and as unfit for engraving on. We do not know what it was. 15. The hyacinth of Pliny is equally unknown. From its name it was obviously of a blue colour. Our hyacinth has a reddish-brown colour, and a great deal of hardness and lustre. 16. The cyanus of Pliny may have been our cyanite. 17. Astrios agrees very well, as far as the description of Pliny goes, with the variety of felspar called adularia.
- 63. 18. Belioculus seems to have been our catseye. 19. Lychnites was a violet-coloured stone, which became electric by heat. Unless it was a blue tourmalin, I do not know what it could be. 20. The jasper of the ancients was probably the same as ours. 21. Molochites may have been our malachite. The name comes from the Greek word μολοχη, mallow, or marshmallow. 22. Pliny considers amber as the juice of a tree concreted into a solid form. The largest piece of it that he had ever seen weighed 13 lbs. Roman weight, which is nearly equivalent to 9¾ lbs. avoirdupois. Indian amber, of which he speaks, was probably copal, or some transparent resin. It may be dyed, he says, by means of anchusa and the fat of kids. 23. Lapis specularis was foliated sulphate of lime, or selenite. 24. Pyrites had the same meaning among the ancients that it has among the moderns; at least as far as iron pyrites or bisulphuret of iron is concerned. Pliny describes two kind of pyrites; namely, the white (arsenical pyrites), and the yellow (iron pyrites). It was used for striking fire with steel, in order to kindle tinder. Hence the name pyrites or firestone. 25. Gagates, from the account given of it by Pliny, was obviously pit-coal or jet. 26. Marble had the same meaning among the ancients that it has among the moderns. It was sawed by the ancients into slabs, and the action of the saw was facilitated by a sand brought for the purpose from Ethiopia and the isle of Naxos. It is obvious that this sand was powdered corundum, or emery.
- 64. 27. Creta was a name applied by the ancients not only to chalk, but to white clay. 28. Melinum was an oxide of iron. Pliny gives a list of one hundred and fifty-one species of stones in the order of the alphabet. Very few of the minerals contained in this list can be made out. He gives also a list of fifty-two species of stones, whose names are derived from a fancied resemblance which the stones are supposed to bear to certain parts of animals. Of these, also, very few can be made out. XI.—MISCELLANEOUS OBSERVATIONS. The ancients seem to have been ignorant of the nature and properties of air, and of all gaseous bodies. Pliny’s account of air consists of a single sentence: “Aër densatur nubibus; furit procellis.” “Air is condensed in clouds, it rages in storms.” Nor is his description of water much more complete, since it consists only of the following phrases: “Aquæ subeunt in imbres, rigescunt in grandines, tumescunt in fluctus, præcipitantur in torrentes.”95 “Water falls in showers, congeals in hail, swells in waves, and rushes down in torrents.” In the thirty-eighth chapter of the second book, indeed, he professes to treat of air; but the chapter contains merely an enumeration of meteorological phenomena, without once touching upon the nature and properties of air. Pliny, with all the philosophers of antiquity, admitted the existence of the four elements, fire, air, water, and earth; but though he enumerates these in the fifth chapter of his first book, he never attempts to explain their nature or properties. Earth, among the ancients, had two meanings, namely, the planet on which we live, and the soil upon which vegetables grow. These two meanings still exist in common language. The meaning afterwards given to the
- 65. term, earth, by the chemists, did not exist in the days of Pliny, or, at least, was unknown to him; a sufficient proof that chemistry, in his time, had made no progress as a science; for some notions respecting the properties and constituents of those supposed four elements must have constituted the very foundation of scientific chemistry. The ancients were acquainted with none of the acids which at present constitute so numerous a tribe, except vinegar, or acetic acid; and even this acid was not known to them in a state of purity. They knew none of the saline bases, except lime, soda, and potash, and these very imperfectly. Of course the whole tribe of salts was unknown to them, except a very few, which they found ready formed in the earth, or which they succeeded in forming by the action of vinegar on lead and copper. Hence all that extensive and most important branch of chemistry, consisting of the combinations of the acids and bases, on which scientific chemistry mainly depends, must have been unknown to them. Sulphur occurring native in large quantities, and being remarkable for its easy combustibility, and its disagreeable smell when burning, was known in the very earliest ages. Pliny describes four kinds of sulphur, differing from each other, probably, merely in their purity. These were 1. Sulphur vivum, or apyron. It was dug out of the earth solid, and was doubtless pure, or nearly so. It alone was used in medicine. 2. Gleba—used only by fullers. 3. Egula—used also by fullers. Pliny says, it renders woollen stuffs white and soft. It is obvious from this, that the ancients knew the method of bleaching flannel by the fumes of sulphur, as practised by the moderns. 4. The fourth kind was used only for sulphuring matches.
- 66. Sulphur, in Pliny’s time, was found native in the Æolian islands, and in Campania. It is curious that he never mentions Sicily, whence the great supply is drawn for modern manufacture. In medicine, it seems to have been only used externally by the ancients. It was considered as excellent for removing eruptions. It was used also for fumigating. The word alumen, which we translate alum, occurs often in Pliny; and is the same substance which the Greeks distinguished by the name of στυπτηρια (stypteria). It is described pretty minutely by Dioscorides, and also by Pliny. It was obviously a natural production, dug out of the earth, and consequently quite different from our alum, with which the ancients were unacquainted. Dioscorides says that it was found abundantly in Egypt; that it was of various kinds, but that the slaty variety was the best. He mentions also many other localities. He says that, for medical purposes, the most valued of all the varieties of alumen were the slaty, the round, and the liquid. The slaty alumen is very white, has an exceedingly astringent taste, a strong smell, is free from stony concretions, and gradually cracks and emits long capillary crystals from these rifts; on which account it is sometimes called trichites. This description obviously applies to a kind of slate-clay, which probably contained pyrites mixed with it of the decomposing kind. The capillary crystals were probably similar to those crystals at present called hair-salt by mineralogists, which exude pretty abundantly from the shale of the coal-beds, when it has been long exposed to the air. Hair-salt differs very much in its nature. Klaproth ascertained by analysis, that the hair-salt from the quicksilver-mines in Idria is sulphate of magnesia, mixed with a small quantity of sulphate of iron.96 The hair-salt from the abandoned coal-pits in the neighbourhood of Glasgow is a double salt, composed of sulphate of alumina, and sulphate of iron, in definite proportions; the composition being 1 atom protosulphate of iron, 1½atom sulphate of alumina,
- 67. 15 atoms water. I suspect strongly that the capillary crystals from the schistose alumen of Dioscorides were nearly of the same nature. From Pliny’s account of the uses to which alumen was applied, it is quite obvious that it must have varied very much in its nature. Alumen nigrum was used to strike a black colour, and must therefore have contained iron. It was doubtless an impure native sulphate of iron, similar to many native productions of the same nature still met with in various parts of the world, but not employed; their use having been superseded by various artificial salts, more definite in their nature, and consequently more certain in their application, and at the same time cheaper and more abundant than the native. The alumen employed as a mordant by the dyers, must have been a sulphate of alumina more or less pure; at least it must have been free from all sulphate of iron, which would have affected the colour of the cloth, and prevented the dyer from accomplishing his object.97 What the alumen rotundum was, is not easily conjectured. Dioscorides says, that it was sometimes made artificially; but that the artificial alumen rotundum was not much valued. The best, he says, was full of air-bubbles, nearly white, and of a very astringent taste. It had a slaty appearance, and was found in Egypt or the Island of Melos. The liquid alumen was limpid, milky, of an equal colour, free from hard concretions, and having a fiery shade of colour.98 In its nature, it was similar to the alumen candidum; it must therefore have consisted chiefly, at least, of sulphate of alumina. Bitumen and naphtha were known to the ancients, and used by them to give light instead of oil; they were employed also as external applications in cases of disease, and were considered as
- 68. having the same virtues as sulphur. It is said, that the word translated salt in the New Testament—“Ye are the salt of the earth: but if the salt have lost his savour, wherewith shall it be salted? It is henceforth good for nothing, but to be cast out, and to be trodden under foot of men”99—it is said, that the word salt in this passage refers to asphalt, or bitumen, which was used by the Jews in their sacrifices, and called salt by them. But I have not been able to find satisfactory evidence of the truth of this opinion. It is obvious from the context, that the word translated salt could not have had that meaning among the Jews; because salt never can be supposed to lose its savour. Bitumen, while liquid, has a strong taste and smell, which it loses gradually by exposure to the air, as it approaches more and more to a solid form. Asphalt was one of the great constituents of the Greek fire. A great bed of it still existing in Albania, supplied the Greeks with this substance. Concerning the nature of the Greek fire, it is clear that many exaggerated and even fabulous statements have been published. The obvious intention of the Greeks being, probably, to make their invention as much dreaded as possible by their enemies. Nitre was undoubtedly one of the most important of its constituents; though no allusion whatever is ever made. We do not know when nitrate of potash, the nitre of the moderns, became known in Europe. It was discovered in the east; and was undoubtedly known in China and India before the commencement of the Christian era. The property of nitre, as a supporter of combustion, could not have remained long unknown after the discovery of the salt. The first person who threw a piece of it upon a red-hot coal would observe it. Accordingly we find that its use in fireworks was known very early in China and India; though its prodigious expansive power, by which it propels bullets with so great and destructive velocity, is a European invention, posterior to the time of Roger Bacon. The word nitre ()רתנ had been applied by the ancients to carbonate of soda, a production of Egypt, where it is still formed from sea-water, by some unknown process of nature in the marshes
- 69. near Alexandria. This is evident, not merely from the account given of it by Dioscorides and Pliny; for the following passage, from the Old Testament, shows that it had the same meaning among the Jews: “As he that taketh away a garment in cold weather, is as vinegar upon nitre: so is he that singeth songs to a heavy heart.”100 Vinegar poured upon saltpetre produces no sensible effect whatever, but when poured upon carbonate of soda, it occasions an effervescence. When saltpetre came to be imported to Europe, it was natural to give it the same name as that applied to carbonate of soda, to which both in taste and appearance it bore some faint resemblance. Saltpetre possessing much more striking properties than carbonate of soda much more attention was drawn to it, and it gradually fixed upon itself the term nitre, at first applied to a different salt. When this change of nomenclature took place does not appear; but it was completed before the time of Roger Bacon, who always applies the term nitrum to our nitrate of potash and never to carbonate of soda. In the preceding history of the chemical facts known to the ancients, I have taken no notice of a well-known story related of Cleopatra. This magnificent and profligate queen boasted to Antony that she would herself consume a million of sistertii at a supper. Antony smiled at the proposal, and doubted the possibility of her performing it. Next evening a magnificent entertainment was provided, at which Antony, as usual, was present, and expressed his opinion that the cost of the feast, magnificent as it was, fell far short of the sum specified by the queen. She requested him to defer computing till the dessert was finished. A vessel filled with vinegar was placed before her, in which she threw two pearls, the finest in the world, and which were valued at ten millions of sistertii; these pearls were dissolved by the vinegar,101 and the liquid was immediately drunk by the queen. Thus she made good her boast, and destroyed the two finest pearls in the world.102 This story, supposing it true, shows that Cleopatra was aware that vinegar has the property of dissolving pearls. But not that she knew the nature
- 70. of these beautiful productions of nature. We now know that pearls consist essentially of carbonate of lime, and that the beauty is owing to the thin concentric laminæ, of which they are composed. Nor have I taken any notice of lime with which the ancients were well acquainted, and which they applied to most of the uses to which the moderns put it. Thus it constituted the base of the Roman mortar, which is known to have been excellent. They employed it also as a manure for the fields, as the moderns do. It was known to have a corrosive nature when taken internally; but was much employed by the ancients externally, and in various ways as an application to ulcers. Whether they knew its solubility in water does not appear; though, from the circumstance of its being used for making mortar, this fact could hardly escape them. These facts, though of great importance, could scarcely be applied to the rearing of a chemical structure, as the ancients could have no notion of the action of acids upon lime, or of the numerous salts which it is capable of forming. Phenomena which must have remained unknown till the discovery of the acids enabled experimenters to try their effects upon limestone and quicklime. Not even a conjecture appears in any ancient writer that I have looked into, about the difference between quicklime and limestone. This difference is so great that it must have been remarked by them, yet nobody seems ever to have thought of attempting to account for it. Even the method of burning or calcining lime is not described by Pliny; though there can be no doubt that the ancients were acquainted with it. Nor have I taken any notice of leather or the method of tanning it. There are so many allusions to leather and its uses by the ancient poets and historians, that the acquaintance of the ancients with it is put out of doubt. But so far as I know, there is no description of the process of tanning in any ancient author whatever.
- 71. CHAPTER III. CHEMISTRY OF THE ARABIANS. Hitherto I have spoken of Alchymy, or of the chemical manufactures of the ancients. The people to whom scientific chemistry owes its origin are the Arabians. Not that they prosecuted scientific chemistry themselves; but they were the first persons who attempted to form chemical medicines. This they did by mixing various bodies with each other, and applying heat to the mixture in various ways. This led to the discovery of some of the mineral acids. These they applied to the metals, c., and ascertained the effects produced upon that most important class of bodies. Thus the Arabians began those researches which led gradually to the formation of scientific chemistry. We must therefore endeavour to ascertain the chemical facts for which we are indebted to the Arabians. When Mahomet first delivered his dogmas to his countrymen they were not altogether barbarous. Possessed of a copious and expressive language, and inhabiting a burning climate, their imaginations were lively and their passions violent. Poetry and fiction were cultivated by them with ardour, and with considerable success. But science and inductive philosophy, had made little or no progress among them. The fatalism introduced by Mahomet, and the blind enthusiasm which he inculcated, rendered them furious bigots and determined enemies to every kind of intellectual improvement. The rapidity with which they overran Asia, Africa, and even a portion of Europe, is universally known. At that period the western world, was
- 72. sunk into extreme barbarism, and the Greeks, with whom the remains of civilization still lingered, were sadly degenerated from those sages who graced the classic ages. Bent to the earth under the most grinding but turbulent despotism that ever disgraced mankind, and having their understandings sealed up by the most subtle and absurd, and uncompromising superstition, all the energy of mind, all the powers of invention, all the industry and talent, which distinguished their ancestors, had completely forsaken them. Their writers aimed at nothing new or great, and were satisfied with repeating the scientific facts determined by their ancestors. The lamp of science fluttered in its socket, and was on the eve of being extinguished. Nothing good or great could be expected from such a state of society. It was, therefore, wisely determined by Providence that the Mussulman conquerors, should overrun the earth, sweep out those miserable governors, and free the wretched inhabitants from the trammels of despotism and superstition. As a despotism not less severe, and a superstition still more gloomy and uncompromising, was substituted in their place, it may seem at first sight, that the conquests of the Mahometans brought things into a worse state than they found them. But the listless inactivity, the almost deathlike torpor which had frozen the minds of mankind, were effectually roused. The Mussulmans displayed a degree of energy and activity which have few parallels in the history of the world: and after the conquests of the Mahometans were completed, and the Califs quietly seated upon the greatest and most powerful throne that the world had ever seen; after Almanzor, about the middle of the eighth century, had founded the city of Bagdad, and settled a permanent and flourishing peace, the arts and sciences, which usually accompany such a state of society, began to make their appearance. That calif founded an academy at Bagdad, which acquired much celebrity, and gradually raised itself above all the other academies in his dominions. A medical college was established there with powers to examine all those persons who intended to devote themselves to
- 73. the medical profession. So many professors and pupils flocked to this celebrated college, from all parts of the world, that at one time their number amounted to no fewer than six thousand. Public hospitals and laboratories were established to facilitate a knowledge of diseases, and to make the students acquainted with the method of preparing medicines. It was this last establishment which originated with the califs that gave a first beginning to the science of chemistry. In the thirteenth century the calif Mostanser re-established the academy and the medical college at Bagdad: for both had fallen into decay, and had been replaced by an infinite number of Jewish seminaries. Mostanser gave large salaries to the professors, collected a magnificent library, and established a new school of pharmacy. He was himself often present at the public lectures. The successor of Mostanser was the calif Haroun-Al-Raschid, the perpetual hero of the Arabian tales. He not only carried his love for the sciences further than his predecessors, but displayed a liberality and a tolerance for religious opinions, which was not quite consistent with Mahometan bigotry and superstition. He drew round him the Syrian Christians, who translated the Greek classics, rewarded them liberally, and appointed them instructors of his Mahometan subjects, especially in medicine and pharmacy. He protected the Christian school of Dschondisabour, founded by the Nestorian Christians, before the time of Mahomet, and still continuing in a flourishing state: always surrounded by literary men, he frequently condescended to take a part in their discussions, and not unfrequently, as might have been expected from his rank, came off victorious. The most enlightened of all the califs was Almamon, who has rendered his name immortal by his exertions in favour of the sciences. It was during his reign that the Arabian schools came to be thoroughly acquainted with Greek science; he procured the translation of a great number of important works. This conduct inflamed the religious zeal of the faithful, who devoted him to
- 74. destruction, and to the divine wrath, for favouring philosophy, and in that way diminishing the authority of the Koran. Almamon purchased the ancient classics, from all quarters, and recommended the care of doing so in a particular manner to his ambassadors at the court of the Greek emperors. To Leo, the philosopher, he made the most advantageous offers, to induce him to come to Bagdad; but that philosopher would not listen to his invitation. It was under the auspices of this enlightened prince, that the celebrated attempt was made to determine the size of the earth by measuring a degree of the meridian. The result of this attempt it does not belong to this work to relate. Almotassem and Motawakkel, who succeeded Almamon, followed his example, favoured the sciences, and extended their protection to men of science who were Christians. Motawakkel re- established the celebrated academy and library of Alexandria. But he acted with more severity than his predecessors with regard to the Christians, who may perhaps have abused the tolerance which they enjoyed. The other vicars of the prophet, in the different Mahometan states, followed the fine example set them by Almamon. Already in the eighth century the sovereigns of Mogreb and the western provinces of Africa showed themselves the zealous friends of the sciences. One of them called Abdallah-Ebn-Ibadschab rendered commerce and industry flourishing at Tunis. He himself cultivated poetry and drew numerous artists and men of science into his state. At Fez and in Morocco the sciences flourished, especially during the reign of the Edrisites, the last of whom, Jahiah, a prince possessed of genius, sweetness, and goodness, changed his court into an academy, and paid attention to those only who had distinguished themselves by their scientific knowledge. But Spain was the most fortunate of all the Mahometan states, and had arrived at such a degree of prosperity both in commerce, manufactures, population, and wealth, as is hardly to be credited.
- 75. The three Abdalrahmans and Alhakem carried, from the eighth to the tenth century, the country subject to the Calif of Cordova to the highest degree of splendour. They protected the sciences, and governed with so much mildness, that Spain was probably never so happy under the dominion of any Christian prince. Alhakem established at Cordova an academy, which for several ages was the most celebrated in the whole world. All the Christians of Western Europe repaired to this academy in search of information. It contained, in the tenth century, a library of 280,000 volumes. The catalogue of this library filled no less than forty-four volumes. Seville, Toledo, and Murcia, had likewise their schools of science and their libraries, which retained their celebrity as long as the dominion of the Moors lasted. In the twelfth century there were seventy public libraries in that part of Spain which belonged to the Mahometans. Cordova had produced one hundred and fifty authors, Almeria fifty- two, and Murcia sixty-two. The Mahometan states of the east continued also to favour the sciences. An emir of Irak, Adad-El-Daula by name, distinguished himself towards the end of the tenth century by the protection which he afforded to men of science. To him almost all the philosophers of the age dedicated their works. Another emir of Irak, Saif-Ed-Daula, established schools at Kufa and at Bussora, which soon acquired great celebrity. Abou-Mansor-Baharam, established a public library at Firuzabad in Curdistan, which at its very commencement contained 7000 volumes. In the thirteenth century there existed a celebrated school of medicine in Damascus. The calif Malek-Adel endowed it richly, and was often present at the lectures with a book under his arm. Had the progress of the sciences among the Arabians been proportional to the number of those who cultivated them, we might hail the Saracens as the saviours of literature during the dark and benighted ages of Christianity; but we must acknowledge with regret, that notwithstanding the enlightened views of the califs, notwithstanding the multiplicity of academies and libraries, and the
- 76. prodigious number of writers, the sciences received but little improvement from the Arabians. There are very few Arabian writers in whose works we find either philosophical ideas, successful researches, new facts, or great and new and important truths. How, indeed, could such things be expected from a people naturally hostile to mental exertion; professing a religion which stigmatizes all exercise of the judgment as a crime, and weighed down by the heavy yoke of despotism? It was the religion of the Arabians, and the despotism of their princes, that opposed the greatest obstacles to the progress of the sciences, even during the most flourishing period of their civilization.103 Fortunately chemistry was the branch of science least obnoxious to the religious prejudices of the Mahometans. It was in it, therefore, that the greatest improvements were made: of these improvements it will be requisite now to endeavour to give the reader some idea. Astrology and alchymy, they both derived from the Greeks: neither of them were inconsistent with the taste of the nation—neither of them were anathematized by the Mahometan creed, though Islamism prohibited magic and all the arts of divination. Alchymy may have suggested the chemical processes—but the Arabians applied them to the preparation of medicines, and thus opened a new and most copious source of investigation. The chemical writings of the Arabians which I have had an opportunity of seeing and perusing in a Latin dress, being ignorant of the original language in which they were written, are those of Geber and Avicenna. Geber, whose real name was Abou-Moussah-Dschafar-Al-Soli, was a Sabean of Harran, in Mesopotamia, and lived during the eighth century. Very little is known respecting the history of this writer, who must be considered as the patriarch of chemistry. Golius, professor of the oriental languages in the University of Leyden, made a present of Geber’s work in manuscript to the public library. He translated it into Latin, and published it in the same city in folio, and afterwards in quarto, under the title of “Lapis
- 77. Philosophorum.”104 It was translated into English by Richard Russel in 1678, under the title of, “The Works of Geber, the most famous Arabian Prince and Philosopher.”105 The works of Geber, so far as they appeared in Latin or English, consist of four tracts. The first is entitled, “Of the Investigation or Search of Perfection.” The second is entitled, “Of the Sum of Perfection, or of the perfect Magistery.” The third, “Of the Invention of Verity or Perfection.” And the last, “Of Furnaces, c.; with a Recapitulation of the Author’s Experiments.” The object of Geber’s work is to teach the method of making the philosopher’s stone, which he distinguishes usually by the name of medicine of the third class. The whole is in general written with so much plainness, that we can understand the nature of the substances which he employed, the processes which he followed, and the greater number of the products which he obtained. It is, therefore, a book of some importance, because it is the oldest chemical treatise in existence,106 and because it makes us acquainted with the processes followed by the Arabians, and the progress which they had made in chemical investigations. I shall therefore lay before the reader the most important facts contained in Geber’s work. 1. He considered all the metals as compounds of mercury and sulphur: this opinion did not originate with him. It is evident from what he says, that the same notion had been adopted by his predecessors—men whom he speaks of under the title of the ancients. 2. The metals with which he was acquainted were gold, silver, copper, iron, tin, and lead. These are usually distinguished by him under the names of Sol, Luna, Venus, Mars, Jupiter, and Saturn. Whether these names of the planets were applied to the metals by Geber, or only by his translators, I cannot say; but they were always employed by the alchymists, who never designated the metals by any other appellations.
- 78. 3. Gold and silver he considered as perfect metals; but the other four were imperfect metals. The difference between them depends, in his opinion, partly upon the proportions of mercury and sulphur in each, and partly upon the purity or impurity of the mercury and sulphur which enters into the composition of each. Gold, according to him, is created of the most subtile substance of mercury and of most clear fixture, and of a small substance of sulphur, clean and of pure redness, fixed, clear, and changed from its own nature, tinging that; and because there happens a diversity in the colours of that sulphur, the yellowness of gold must needs have a like diversity.107 His evidence that gold consisted chiefly of mercury, is the great ease with which mercury dissolves gold. For mercury, in his opinion, dissolves nothing that is not of its own nature. The lustre and splendour of gold is another proof of the great proportion of mercury which it contains. That it is a fixed substance, void of all burning sulphur, he thinks evident by every operation in the fire, for it is neither diminished nor inflamed. His other reasons are not so intelligible.108 Silver, like gold, is composed of much mercury and a little sulphur; but in the gold the sulphur is red; whereas the sulphur that goes to the formation of silver is white. The sulphur in silver is also clean, fixed, and clear. Silver has a purity short of that of gold, and a more gross inspissation. The proof of this is, that its parts are not so condensed, nor is it so fixed as gold; for it may be diminished by fire, which is not the case with gold.109 Iron is composed of earthy mercury and earthy sulphur, highly fixed, the latter in by far the greatest quantity. Sulphur, by the work of fixation, more easily destroys the easiness of liquefaction than mercury. Hence the reason why iron is not fusible, as is the case with the other metals.110
- 79. Sulphur not fixed melts sooner than mercury; but fixed sulphur opposes fusion. What contains more fixed sulphur, more slowly admits of fusion than what partakes of burning sulphur, which more easily and sooner flows.111 Copper is composed of sulphur unclean, gross and fixed as to its greater part; but as to its lesser part not fixed, red, and livid, in relation to the whole not overcoming nor overcome and of gross mercury.112 When copper is exposed to ignition, you may discern a sulphureous flame to arise from it, which is a sign of sulphur not fixed; and the loss of the quantity of it by exhalation through the frequent combustion of it, shows that it has fixed sulphur. This last being in abundance, occasions the slowness of its fusion and the hardness of its substance. That copper contains red and unclean sulphur, united to unclean mercury, is, he thinks, evident, from its sensible qualities.113 Tin consists of sulphur of small fixation, white with a whiteness not pure, not overcoming but overcome, mixed with mercury partly fixed and partly not fixed, white and impure.114 That this is the constitution of tin he thinks evident; for when calcined, it emits a sulphureous stench, which is a sign of sulphur not fixed: it yields no flame, not because the sulphur is fixed, but because it contains a great portion of mercury. In tin there is a twofold sulphur and also a twofold mercury. One sulphur is less fixed, because in calcining it gives out a stench as sulphur. The fixed sulphur continues in the tin after it is calcined. He thinks that the twofold mercury in tin is evident, from this, that before calcination it makes a crashing noise when bent, but after it has been thrice calcined, that crashing noise can no longer be perceived.115 Geber says, that if lead be washed with mercury, and after its washing melted in a fire not exceeding the fire of its fusion, a portion of the mercury will remain combined with the lead, and will give it the crashing noise and all the qualities
- 80. of tin. On the other hand, you may convert tin into lead. By manifold repetition of its calcination, and the administration of fire convenient for its reduction, it is turned into lead.116 Lead, in Geber’s opinion, differs from tin only in having a more unclean substance commixed of the two more gross substances, sulphur and mercury. The sulphur in it is burning and more adhesive to the substance of its own mercury, and it has more of the substance of fixed sulphur in its composition than tin has.117 Such are the opinions which Geber entertained respecting the composition of the metals. I have been induced to state them as nearly in his own words as possible, and to give the reasons which he has assigned for them, even when his facts were not quite correct, because I thought that this was the most likely way of conveying to the reader an accurate notion of the sentiments of this father of the alchymists, upon the very foundation of the whole doctrine of the transmutation of metals. He was of opinion that all the imperfect metals might be transformed into gold and silver, by altering the proportions of the mercury and sulphur of which they are composed, and by changing the nature of the mercury and sulphur so as to make them the same with the mercury and sulphur which constitute gold and silver. The substance capable of producing these important changes he calls sometimes the philosopher’s stone, but generally the medicine. He gives the method of preparing this important magistery, as he calls it. But it is not worth while to state his process, because he leaves out several particulars, in order to prevent the foolish from reaping any benefit from his writings, while at the same time those readers who possess the proper degree of sagacity will be able, by studying the different parts of his writings, to divine the nature of the steps which he omits, and thus profit by his researches and explanations. But it will be worth while to notice the most important of his processes, because this will enable us to judge of the state of chemistry in his time.
- 81. 4. In his book on furnaces, he gives a description of a furnace proper for calcining metals, and from the fourteenth chapter of the fourth part of the first book of his Sum of Perfection, it is obvious that the method of calcining or oxidizing iron, copper, tin, and lead, and also mercury and arsenic were familiarly known to him. He gives a description of a furnace for distilling, and a pretty minute account of the glass or stoneware, or metallic aludel and alembic, by means of which the process was conducted. He was in the habit of distilling by surrounding his aludel with hot ashes, to prevent it from being broken. He was acquainted also with the water-bath. These processes were familiar to him. The description of the distillation of many bodies occurs in his work; but there is not the least evidence that he was acquainted with ardent spirits. The term spirit occurs frequently in his writings, but it was applied to volatile bodies in general, and in particular to sulphur and white arsenic, which he considered as substances very similar in their properties. Mercury also he considered as a spirit. The method of distilling per descensum, as is practised in the smelting of zinc, was also known to him. He describes an apparatus for the purpose, and gives several examples of such distillations in his writings. He gives also a description of a furnace for melting metals, and mentions the vessels in which such processes were conducted. He was acquainted with crucibles; and even describes the mode of making cupels, nearly similar to those used at present. The process of cupellating gold and silver, and purifying them by means of lead, is given by him pretty minutely and accurately: he calls it cineritium, or at least that is the term used by his Latin translator. He was in the habit of dissolving salts in water and acetic acid, and even the metals in different menstrua. Of these menstrua he nowhere gives any account; but from our knowledge of the properties of the different metals, and from some processes which
- 82. he notices, it is easy to perceive what his solvents must have been; namely, the mineral acids which were known to him, and to which there is no allusion whatever in any preceding writer that I have had an opportunity of consulting. Whether Geber was the discoverer of these acids cannot be known, as he nowhere claims the discovery: indeed his object was to slur over these acids, as much as possible, that their existence, or at least their remarkable properties, might not be suspected by the uninitiated. It was this affectation of secrecy and mystery that has deprived the earliest chemists of that credit and reputation to which they would have been justly entitled, had their discoveries been made known to the public in a plain and intelligible manner. The mode of purifying liquids by filtration, and of separating precipitates from liquids by the same means, was known to Geber. He called the process distillation through a filter. Thus the greater number of chemical processes, such as they were practised almost to the end of the eighteenth century, were known to Geber. If we compare his works with those of Dioscorides and Pliny, we shall perceive the great progress which chemistry or rather pharmacy had made. It is more than probable that these improvements were made by the Arabian physicians, or at least by the physicians who filled the chairs in the medical schools, which were under the protection of the califs: for as no notice is taken of these processes by any of the Greek or Roman writers that have come down to us, and as we find them minutely described by the earliest chemical writers among the Arabians, we have no other alternative than to admit that they originated in the east. I shall now state the different chemical substances or preparations which were known to Geber, or which he describes the method of preparing in his works. 1. Common salt. This substance occurring in such abundance in the earth, and being indispensable as a seasoner of food, was
- 83. known from the earliest ages. But Geber describes the method which he adopted to free it from impurities. It was exposed to a red heat, then dissolved in water, filtered, crystallized by evaporation, and the crystals being exposed to a red heat, were put into a close vessel, and kept for use.118 Whether the identity of sal-gem (native salt) and common salt was known to Geber is nowhere said. Probably not, as he gives separate directions for purifying each. 2. Geber gives an account of the two fixed alkalies, potash and soda, and gives processes for obtaining them. Potash was obtained by burning cream of tartar in a crucible, dissolving the residue in water, filtering the solution, and evaporating to dryness.119 This would yield a pure carbonate of potash. Carbonate of soda he calls sagimen vitri, and salt of soda. He mentions plants which yield it when burnt, points out the method of purifying it, and even describes the method of rendering it caustic by means of quicklime.120 3. Saltpetre, or nitrate of potash, was known to him; and Geber is the first writer in whom we find an account of this salt. Nothing is said respecting its origin; but there can be little doubt that it came from India, where it was collected, and known long before Europeans were acquainted with it. The knowledge of this salt was probably one great cause of the superiority of the Arabians over Europeans in chemical knowledge; for it enabled them to procure nitric acid, by means of which they dissolved all the metals known in their time, and thus acquired a knowledge of various important saline compounds, which were of considerable importance. There is a process for preparing saltpetre artificially, in several of the Latin copies of Geber, though it does not appear in our English translation. The method was to dissolve sagimen vitri, or carbonate of soda, in aqua fortis, to filter and crystallize by evaporation.121 If this process be genuine, it is obvious that Geber must have been
- 84. Welcome to our website – the perfect destination for book lovers and knowledge seekers. We believe that every book holds a new world, offering opportunities for learning, discovery, and personal growth. That’s why we are dedicated to bringing you a diverse collection of books, ranging from classic literature and specialized publications to self-development guides and children's books. More than just a book-buying platform, we strive to be a bridge connecting you with timeless cultural and intellectual values. With an elegant, user-friendly interface and a smart search system, you can quickly find the books that best suit your interests. Additionally, our special promotions and home delivery services help you save time and fully enjoy the joy of reading. Join us on a journey of knowledge exploration, passion nurturing, and personal growth every day! ebookbell.com