![REGRESSION 5
However, sometimes we do not distinguish notationally a random variable and its
realization, and the notation of (1.4) is used also in the place of notation (1.3) to
denote a realization of the random vectors and not the random vectors themselves.
In regression analysis we typically want to estimate the conditional expectation
f(x) = E(YX = x),
and now we assume that the sequence (X, Yi),..., (Xn, Yn) consists of identically
distributed random variables, and {X,Y) has the same distribution as (X^Y*),
i = 1,..., n. Besides conditional expectation we could estimate conditional mode,
conditional variance, conditional quantile, and so on. Estimation of the conditional
centers of distribution are discussed in Section 1.1.2 and estimation of conditional
risk measures such as variance and quantiles are discussed in Section 1.1.4 and in
Section 1.1.6.
Fixed Design Regression In fixed design regression the data are a sequence
2/1 > • • • ,2/n,
where yi G R, i — 1,..., n. We assume that every observation yi is associated with
a fixed design point Xi G Rd
.
Now the design points are not chosen by a random mechanism, but they are
chosen by the conducter of the experiment. Typical examples could be time series
data, where Xi is the time when the observation yi is recorded, and spatial data, where
Xi is the location where the observation yi is made. Time series data are discussed in
Section 1.1.9.
We model the data as a sequence of random variables
Y,..., Yn.
In the fixed design regression we typically do not assume that the data are identically
distributed. For example, we may assume that
Yi = f(xi) + 6i, i — 1,..., n,
where Xi — i/n, f : [0,1] R is the function we want to estimate, and Eei — 0.
Now the data Y,..., Yn are not identically distributed, since the observations Yi
have different expectations.
1.1.2 Mean Regression
The regression function is typically defined as a conditional expectation. Besides
expectation and conditional expectation also median and conditional median can be
used to characterize the center of a distribution and thus to predict and explain with
the help of explanatory variables. We mention also the mode (maximum of the
density function) as a third characterization of the center of a distribution, although
the mode is typically not used in regression analysis.](https://image.slidesharecdn.com/2371206-250602123846-ce7ab0c9/85/Multivariate-Nonparametric-Regression-And-Visualization-With-R-And-Applications-To-Finance-1st-Edition-Jussi-Klemel-35-320.jpg)
![REGRESSION 7
(a) (b)
Figure 1.1 Mean regression, (a) A scatter plot of regression data, (b) A contour plot of
the estimated joint density of the explanatory variable and the response variable. The linear
regression function estimate is shown with red and the kernel regression estimate is shown
with blue.
density estimation with smoothing parameter h = 0.6. Linear regression is discussed
in Section 2.1, and kernel methods are discussed in Section 3.2. In the scatter plot we
have used histogram smoothing with 1002
bins, as explained in Section 6.1.1. This
example indicates that the daily returns are dependent random variables, although it
can be shown that they are nearly uncorrelated.
Median and Conditional Median The median can be defined in the case
of continuous distribution function of a random variable Y G R as the number
median(Y) G R satisfying
P(Y < median(Y)) = 0.5.
In general, covering also the case of discrete distributions, we can define the median
uniquely as the generalized inverse of the distribution function:
median(Y) = inf{y : P(Y <y)> 0.5}. (1.8)
The conditional median is defined using the conditional distribution of Y given X:
median (Y X = x) = inf{y : P(Y <y X = x)> 0.5}, x G Rd
. (1.9)
The sample median of observations Y,..., Yn G R can be defined as the median
of the empirical distribution. The empirical distribution is the discrete distribution
with the probability mass function P({Y;}) = 1/n for % — 1,..., n. Then,
median(Yi,..., Yn) = Y[ n / 2 ] + 1 , (1.10)
where Y^ < • • • < Y(n) is the ordered sample and [x] is the largest integer smaller
or equal to x.](https://image.slidesharecdn.com/2371206-250602123846-ce7ab0c9/85/Multivariate-Nonparametric-Regression-And-Visualization-With-R-And-Applications-To-Finance-1st-Edition-Jussi-Klemel-37-320.jpg)
![REGRESSION 9
whenxi > Oand E(Y X = x) > 0. The partial elasticity describes the approximate
percentage change of conditional expectation of Y when the value of X is changed
by one percent, when the values of the other variables are fixed.1
The partial
semielasticity of X is defined as
when E(Y | X = x) > 0. The partial semielasticity describes the approximate
percentage change of conditional expectation of Y when the value of X is changed
by 1 unit, when the values of the other variables are fixed.
We can use the visualization of partial effects as a tool to visualize regression
functions. In Section 7.4 we show how level set trees can be used to visualize the
mode structure of functions. The mode structure of a function means the number, the
largeness, and the location of the local maxima of a function. Analogously, level set
trees can be used to visualize the antimode structure of a function, where the antimode
structure means the number, the largeness, and the location of the local minima of
a function. Local maxima and minima are important characteristics of a regression
function. However, we need to know more about a regression function than just
the mode structure or antimode structure. Partial effects are a useful tool to convey
additional important information about a regression function. If the partial effect is
flat for each variable, then we know that the regression function is close to a linear
function. When we visualize the mode structure of the partial effect of variable Xi,
then we get information about whether a variable X is causing the expected value
of the response variable to increase in several locations (the number of local maxima
of the partial effect), how much an increase of the value of the variable X increases
the expected value of the response variable Y (the largeness of the local maxima of
the partial effect), and where the influence of the response variable X is the largest
(the location of the local maxima of the partial effect). Analogous conclusions can
be made by visualizing the antimode structure of the partial effect.
We present two methods for the estimation of partial effects. The first method is to
use the partial derivatives of a kernel regression function estimator, and this method
is presented in Section 3.2.9. The second method is to use a local linear estimator,
and this method is presented in Section 5.2.1.
1.1.4 Variance Regression
The mean regression gives information about the center of the conditional distribution,
and with the variance regression we get information about the dispersion and on the
s(xix2,...,xd) = ~— ogE(YX = x)
d
E(Y X = x)'
1
1
This interpretation follows from the approximation
log f(x + h)~ log/Or) « [/(* + h)~ /(*)]//(*),
which follows from the approximation log(rr) « x — 1, when x « 1.](https://image.slidesharecdn.com/2371206-250602123846-ce7ab0c9/85/Multivariate-Nonparametric-Regression-And-Visualization-With-R-And-Applications-To-Finance-1st-Edition-Jussi-Klemel-39-320.jpg)
![1 0 OVERVIEW OF REGRESSION AND CLASSIFICATION
heaviness of the tails of the conditional distribution. Variance is a classical measure
of dispersion and risk which is used for example in the Markowitz theory of portfolio
selection. Partial moments are risk measures that generalize the variance.
Variance and Conditional Variance The variance of random variable Y is
defined by
Var(Y) = E(Y - EY)2
= EY2
- (EY)2
. (1.12)
The standard deviation of Y is the square root of the variance of Y. The conditional
variance of random variable Y is equal to
Var(Y X = x) = E { [ Y - E ( Y X = x)}2
X = x} (1.13)
= E(Y2
X = x)- [E(Y | X = x)]2
. (1.14)
The conditional standard deviation of Y is the square root of the conditional variance.
The sample variance is defined by
i=1 i=l
where Y,..., Yn is a sample of random variables having identical distribution with
Y.
Conditional Variance Estimation Conditional variance Var(Y X = x) can
be constant not depending on x. Let us write
y = / P 0 + €,
where f(x) = E(Y | X = x) and e = Y - /(X), so that E{e X = x) = 0.
If Var(y | X = x) = E(e2
) is a constant not depending on x, we say that the
noise is homoskedastic. Otherwise the noise is heteroskedastic. If the noise is
heteroskedastic, it is of interest to estimate the conditional variance
Var(y X = x) = E(e2
X = x).
Estimation of the conditional variance can be reduced to the estimation of the
conditional expectation by using (1.13). First we estimate the conditional expectation
f(x) = E(Y | X = x) by f(x). Second we calculate the residuals
ii = Y i - f(Xi),
and estimate the conditional variance from the data (Xi, e2
),..., (Xn, e2
).
Estimation of the conditional variance can be reduced to the estimation of the
conditional expectation by using (1.14). First we estimate the conditional expec-
tation E(Y2
| X = x) using the regression data (Xi, Y2
),..., (Xn, Y2
). Sec-
ond we estimate the conditional expectation f(x) = E(Y | X = x) using data
(Xi, Yi),..., (Xn, Yn).](https://image.slidesharecdn.com/2371206-250602123846-ce7ab0c9/85/Multivariate-Nonparametric-Regression-And-Visualization-With-R-And-Applications-To-Finance-1st-Edition-Jussi-Klemel-40-320.jpg)
![12 OVERVIEW OF REGRESSION AND CLASSIFICATION
where it is assumed that x i , . . . G R, and x < • • • < xn. The estimator
was studied and modified in various ways in Rice (1984), Gasser, Sroka & Jennen-
Steinmetz (1986), Hall, Kay & Titterington (1990), Hall, Kay & Titterington (1991),
Thompson, Kay & Titterington (1991), and Munk, Bissantz, Wagner & Freitag
(2005).
Conditional Variance in a Time Series Setting In a time series setting, when
we observe Yt,t = 1,2,..., the conditional heteroskedasticity assumption is that
Yt = <jteu f = 0 , ± 1 , ± 2 , . . . , (1.16)
where et is an i.i.d. sequence, Eet = 0, Ee2
— 1, and crt is the volatility process.
The volatility process is a predictable random process, that is, at is measurable
with respect to the sigma-field generated by the variables Yt-i,Yt-2, — When
we assume that et is independent from Y^-i, Yt_2, • • then under the conditional
heteroskedasticity model,
Var(Y, | Ji_x) = Var(a,e, | = *t
2
Var(et I Tt-1) = ^2
Var(et) = a2
, (1.17)
where Tt-1 is the sigma-algebra generated by variables Yt -i,Yt -2, — In a con-
ditional heteroskedasticity model the main interest is in predicting the value of the
random variable of, which is thus related to estimating the conditional variance.
The statistical problem is to predict a2
using a finite number of past observations
Yi,..., Yt-. Special cases of conditional heteroskedasticity models are the ARCH
model discussed in Section 2.5.2 and the GARCH model discussed in Section 3.9.2.
Partial Moments The variance of random variable Y G R is defined as Var(Y) =
E(Y — EY)2
. The variance can be generalized to other centered moments
EY-EYk
,
for k = 1,2, The centered moments take a contribution both from the left and
the right tails of the distribution. When we are interested only in the left tail or in the
right tail (losses or gains ), then we can use the lower partial moments or the upper
partial moments. The upper partial moment is defined as
UPMr,fe(y) = E [ ( Y - r)fe
/[T,oo)(F)]
and the lower partial moment is defined as
LPMT,fc(F) = E [ ( t - y)f c
/( _0 0 ,r ] (F)],
where k = 0,1, 2,..., and r G R. In risk management r could be the target rate.
When Y has density f y , we can write
rOO PT
UPMr,k (Y)= / (y-T)k
fY(y)dy, LPMr,fc(y) = / (r - y)k
fY(y) dy.
J T J — OO](https://image.slidesharecdn.com/2371206-250602123846-ce7ab0c9/85/Multivariate-Nonparametric-Regression-And-Visualization-With-R-And-Applications-To-Finance-1st-Edition-Jussi-Klemel-42-320.jpg)
]. (1.18)
The square root of the lower semivariance can be used to replace the standard deviation
in the definition of the Sharpe ratio or in the Markowitz criterion. We can define
conditional versions of partial moments by changing the expectations to conditional
expectations.
1.1.5 Covariance and Correlation Regression
The covariance of random variables Y and Z is defined by
Cov(Y, Z) = E[(Y - EY)(Z - EZ)} = E(YZ) - EYEZ.
The sample covariance is defined by
1 n 1 n
Z) = - Y(YZ -?){Zi-Z) = - Y YiZi - YZ,
i= 1 i= 1
where Y,..., Yn and Z,..., Zn are samples of random variables having identi-
cal distributions with Y and Z,Y = n~l
J X i and
% = n
~l
I X l z
i- The
conditional covariance is obtained by changing the expectations to conditional ex-
pectations.
We have two methods of estimation of conditional covariance, analogously to two
methods of conditional variance estimation based on formulas (1.13) or (1.14). The
first method uses Cov(Y, Z) = E[(Y - EY){Z - EZ)} and the second method uses
Cov(F, Z) = E(YZ) - EYEZ.
The correlation is defined by
Cor(y, Z) = C o
^ z )
sd(y) sd(Z)'
where sd(Y) and sd(Z) are the standard deviations of Y and Z. The conditional
correlation is defined by
Cor(V, 2X = x) = .. ( U 9 )
sd(F | X = x) sd(Z | X = x)
where
sd(Y X = x) = v
/
Var(F X = x), sd(ZX = x) = vVar(Z X = x).](https://image.slidesharecdn.com/2371206-250602123846-ce7ab0c9/85/Multivariate-Nonparametric-Regression-And-Visualization-With-R-And-Applications-To-Finance-1st-Edition-Jussi-Klemel-43-320.jpg)
![1 4 OVERVIEW OF REGRESSION AND CLASSIFICATION
We can write
Cor(F, Z X = x)= Cov(y, ZX = x), (1.20)
where
~ _ Y
~ _ Z
Y =
sd(F X = x)' Z =
sd(Z X = x)'
Thus we have two approaches to the estimation of conditional correlation.
1. We can use (1.19). First we estimate the conditional covariance and the
conditional standard deviations. Second we use (1.19) to define the estimator
of the conditional correlation.
2. We can use (1.20). First we estimate the conditional standard deviations
by sdy(x) and sdz(x), and calculate the standardized observations Yi —
Yi/sdy(Xi) and Zi = Zi/sdz(Xi). Second we estimate the conditional
correlation using Yi, Zi), i — 1,..., ti.
A time series (Yt)tez is weakly stationary if EYt = EYt+h and EYtYt+h depends
only on h, for all t,h G Z. For a weakly stationary time series (Yt)tez, the
autocovariance function is defined by
7(ft) = cov(yt,yt+/l),
and the autocorrelation is defined by
p(h) = 7(*0/7(0),
where h = 0, ±1, —
A vector time series (Xt)tez, Xt G is weakly stationary if EXt = EXt+h
and EXtX't+h depends only on h9 for alH, h G Z. For a weakly stationary vector
time series (Xt)tez, the autocovariance function is defined by
T(h) = Cov(Xt, Xt+h) = E[(Xt - n)(Xt+h ~ m/], (1.21)
for h = 0, ± 1 , . . w h e r e i — EXt — EXt+h> Matrix T(h) is a d x d matrix which
is not symmetric. It holds that
T(h) = r(-h)'. (1.22)
1.1.6 Quantile Regression
A quantile generalizes the median. In quantile regression a conditional quantile is
estimated. Quantiles can be used to measure the value at risk (VaR). The expected
shortfall is a related measure of dispersion and risk.](https://image.slidesharecdn.com/2371206-250602123846-ce7ab0c9/85/Multivariate-Nonparametric-Regression-And-Visualization-With-R-And-Applications-To-Finance-1st-Edition-Jussi-Klemel-44-320.jpg)
![REGRESSION 1 5
Quantile and Conditional Quantile The pth quantile is defined as
QP(Y) = inf{y : P(Y < y) > p}, x G (1.23)
where 0 < p < 1. For p = 1 / 2 , QP(Y) is equal to median med(F), defined in (1.8).
In the case of a continuous distribution function we have
P(Y<Qp(Y))=p
and thus it holds that
Qp(Y)--=Fyp),
where Fy(y) — P(Y < y) is the distribution function of Y and Fy1
is the inverse
of Fy. The pth conditional quantile is defined replacing the distribution of Y with
the conditional distribution of Y given X
QP(Y X = x)= inf {y : P(Y < y X = x) > p}, x G (1.24)
where 0 < p < 1. Conditional quantile estimation has been considered in Koenker
(2005) and Koenker & Bassett (1978).
Estimation of a Quantile and a Conditional Quantile Estimation of quan-
tiles is closely related to the estimation of the distribution function. It is usually
possible to derive a method for the estimation of a quantile or a conditional quantile
if we have a method for the estimation of a distribution function or a conditional
distribution function.
Empirical Quantile Let us define the empirical distribution function, based on the
dta Y,... ,yn , as
1 n
— y e n .
2=1
Now we can define an estimate of the quantile by
Qp = inf{x : F(x) > p},
where 0 < p < 1. Now it holds that
0 < p < 1/n,
1/n <p< 2/n,
(1.25)
(1.26)
Qp —
Yt
(i)'
(1.27)
Y(n_ 1)5 1 — 2/n < p < 1 — 1 / n ,
Y(n), 1 — 1/n < p < 1,
where the ordered sample is denoted by Y^) < Y(2) < • • < ),,,. A third
description of the empirical estimator of the quantile is given by the following steps:
1. Order the sample from the smallest observation to the largest observation:
Y(i)<---<Y(ny
2. Let m = pri], where y is the the smallest integer > y.
3. Set Qp = y(m).](https://image.slidesharecdn.com/2371206-250602123846-ce7ab0c9/85/Multivariate-Nonparametric-Regression-And-Visualization-With-R-And-Applications-To-Finance-1st-Edition-Jussi-Klemel-45-320.jpg)
![REGRESSION 1 7
Expected Shortfall The expected shortfall is a measure of risk which aggregates
all quantiles in the right tail (or in the left tail). The expected shortfall for the right
tail is defined as
p Jp
ESP {Y) = - / Qu(Y)du, 0 < p < I.
JP
When Y has a continuous distribution function, then
ESP (Y) = E(YY> QP(Y)) = E (Y/[ Q p ( y ) > o o ) (Y)); (1.34)
see McNeil, Frey & Embrechts (2005, lemma 2.16). We have defined the loss in
(1.86) as the negative of the change in the value of the portfolio, and thus the risk
management wants to control the right tails of the loss distribution However, we can
define the expected shortfall for the left tail as
ESP(Y) = - [P
Qu(Y)du, 0 < p < l . (1.35)
P Jo
When Y has a continuous distribution function, then
ESP(Y) = E(YY < QP(Y)) = ± ^ / ( ^ ^ ( Y ) ) .
This expression shows that in the case of a continuous distribution function, pESp(F)
is equal to the expectation which is taken only over the left tail, when the left tail is
defined as the region which is to the left of a quantile of the distribution.2
The expected shortfall can be estimated from the data Y,..., Yn in the case where
the expected shortfall is given in (1.34) by using
ESP = - ]T Y(i
m z
—' v
rri *—' v 7
i=m
where Y^) < • • • < F(n) and m = |"(1 — p)n]. When the expected shortfall is given
by (1.35), then we define
^ rn
ESp = — } Y(i),
2=1
where m = pn.
Let us consider the location-scale model
Y = p + ere,
where p G R, o > 0, and e is a random variable with a continuous distribution. Now
ESp(Y) = /x + o-ESp(e).
2
Sometimes the expected shortfall for the left tail is defined as QP(Y) — EIYI^^^q (y)j (y)] and the
absolute shortfall is defined as —E[Y](https://image.slidesharecdn.com/2371206-250602123846-ce7ab0c9/85/Multivariate-Nonparametric-Regression-And-Visualization-With-R-And-Applications-To-Finance-1st-Edition-Jussi-Klemel-47-320.jpg)
![REGRESSION 1 9
1. When p(t) = t2
and Q is the class of all measurable functions Hd
R, then
/, defined by (1.36), is equal to the conditional expectation:
f ( x ) = E(Y X = x) = argmingegE(Y - g ( X ) f .
Indeed,
E(g(X) - Y)2
= E(g(X) - E(Y | X))2
+ E(E(Y X) - Y")2
, (1.37)
because E[(g{X) - E(Y X)){E{Y X) - Y)] = 0, and thus E(g{X) - Y)2
is minimized with respect to g : Hd
R by choosing g{x) = E{Y | X = x)?
Note also that the expectation EY is the best constant approximation of Y.
That is, if we choose Q as the class of constant functions
g = {g : Kd
R|0(x) = p for all x G R,/i E R},
then
EY - argmingegE(Y - g(X)f = argmin^RE(Y - p f . (1.38)
Indeed,
E(Y - p)2
= E(Y - EY)2
-f (EY - p)2
,
and this is minimized with respect to p G R by choosing p — EY.
2. When p(t) = t and Q is the class of all measurable functions Hd
—
» R, then
/ defined by (1.36) is the conditional median:
med(F X = x)= avgmmgegEY - g(X) (1.39)
where the conditional median is defined in (1.9). Equation (1.39) is proved in
the next item.
3. When p is defined as
PP{t)=tp-I(-oo,0)(t))} = {t
t%~1)
' d-40)
for 0 < p < 1 and Q is the class of all measurable functions, then the best
approximation is the conditional quantile. Figure 1.2 shows the loss function
in (1.40) with p = 0.5 (black line) and with p — 0.1 (red line). We show that
if the distribution function Fy is strictly monotonic, then
QP(Y) = argmine e n EP p (Y - 6). (1.41)
3
Note that the conditional expectation defined as f ( x ) = E(Y | X = x) is a real-valued function of x,
but E{X | Y) is a real-valued random variable which can be defined as E(X Y) = f ( X ) .](https://image.slidesharecdn.com/2371206-250602123846-ce7ab0c9/85/Multivariate-Nonparametric-Regression-And-Visualization-With-R-And-Applications-To-Finance-1st-Edition-Jussi-Klemel-49-320.jpg)
![2 2 OVERVIEW OF REGRESSION AND CLASSIFICATION
where Y G R is a scalar random variable and X G Hd
is a random vector. We have
Fy | x=Ay) = E [/(-ocdQO | X = x] (1.42)
and thus the estimation of the conditional distribution function can be considered
as a regression problem, where the conditional expectation of the random variable
I(-oo,y](Y) is estimated. The random variable /(_00^](F) takes only values 0
or 1. The unconditional distribution function can be estimated with the empirical
distribution function, which is defined for the data Y,..., Yn as
My) = - E h-oo,y] (Yi) = n~l
#{i : < 2/, i = 1,..., n}, (1.43)
n
z=i
where means the cardinality of set A. The conditional distribution function
estimation is considered in Section 3.7, where local averaging estimators are defined.
Conditional Density Conditional density function is defined as
when fx(x) > 0,
fvx=x(y) = , n .
1 n
otherwise,
for y G R, where fx,y : R d + 1
R is thejoint density of (X, Y) and f x : Rd
R
is the density of X. We mention three ways to estimate the conditional density.
First, we can replace the density of (X, Y) and the density of X with their
estimators fx,y and f x and define
f , x f x , y { x , y )
JYx=x{y) = —p——;—,
f x ( x )
for fx(x) > 0. This approach is close to the approach used in Section 3.6, where
local averaging estimators of the conditional density are defined.
Second, empirical risk minimization can be used in the estimation of the condi-
tional density, as explained in Section 5.1.3.
Third, sometimes it is reasonable to assume that the conditional density has the
form
fvX=x(y) = fg(x)(y), (1.44)
where fe, 0 G A C Kk
, is a family of density functions and g : Hd
—» A, where
k > 1. Then the estimation of the conditional density reduces to the estimation of
the "regression function" g. The mean regression is a special case of this approach
when the distribution of errors is known: Assume that
Y = f{}0 + e,
where e is independent of X, Ee = 0, and the density of e is denoted by fe. Then
fyx=x(y) = fe(y - f(x)),](https://image.slidesharecdn.com/2371206-250602123846-ce7ab0c9/85/Multivariate-Nonparametric-Regression-And-Visualization-With-R-And-Applications-To-Finance-1st-Edition-Jussi-Klemel-52-320.jpg)
![2 4 OVERVIEW OF REGRESSION AND CLASSIFICATION
be a c/-dimensional vector time series. Definition (1.45) generalizes to the setting of
vector time series. Define
- g(Zl+i), = (Zi,..., Z2-k+1), (1.47)
i = k,... ,T — 1, where g : Hd
—
» R is a function with real values. We define the
regression function, as previously, by
f ( x ) = E(YiXi = x), xeKdk
.
The regression function is now defined on the higher-dimensional space of dimension
kd.
We can predict and explain without autoregression parameter k and take into
account all the previous observations and not just the k last observations. However,
this approach does not fit into the standard regression approach. Let Z,..., Zt G R
be a scalar time series and define
Y{ = Z i + i , X i = ( Z i , . . . , Z),
i — ,... ,T — 1. The sequence of observations (Y, X i ) , . . . , (Yr-i> X T - I ) is not
a sequence of identically distributed random vectors. For example, the regression
function fi(x) = E(Yi | Xi = x), x G Hld
, is defined in a different space for each i.
Time—Space Prediction In time-space prediction the time parameter is taken
as the explanatory variable, in contrast to (1.45), where the previous observations in
the time series are taken as the explanatory variables. We denote
= Xi=i, i = 1,... ,T. (1.48)
The obtained regression model is a fixed design regression model, as described in
Section 1.1.1.
Time-space prediction can be used when the time series can be modeled as a
nonstationary time series of signal with additive noise:
Yi = & + (nei, i = 1,... ,T, (1.49)
where ^ G R is the deterministic signal, <Ji > 0 are nonrandom values, and the
noise ^ is stationary with mean zero and unit variance. For statistical estimation and
asymptotic analysis we can use a slightly different model
YZjT = + <t(U,t) ei,r, i = 1,... ,T, (1.50)
where t^T — i/T, /i : [0,1] —
> R, a : [0,1] (0, oo), and e^r is stationary with
mean zero and unit variance. Now it can be thought that the observations are coming
from a continuous time process Y(t), t G [0,1], and the sampled discrete time process
is obtained as Y^T = Y(i/T), i = 1,..., T. The asymptotics as T —
>
• oo is called
in-fill asymptotics, because points t^T are filling the interval [0,1] as T —> oo.](https://image.slidesharecdn.com/2371206-250602123846-ce7ab0c9/85/Multivariate-Nonparametric-Regression-And-Visualization-With-R-And-Applications-To-Finance-1st-Edition-Jussi-Klemel-54-320.jpg)
Multivariate Nonparametric Regression And Visualization With R And Applications To Finance 1st Edition Jussi Klemel
- 1. Multivariate Nonparametric Regression And Visualization With R And Applications To Finance 1st Edition Jussi Klemel download https://ebookbell.com/product/multivariate-nonparametric- regression-and-visualization-with-r-and-applications-to- finance-1st-edition-jussi-klemel-4742412 Explore and download more ebooks at ebookbell.com
- 2. Here are some recommended products that we believe you will be interested in. You can click the link to download. Advanced Linear Modeling Multivariate Time Series And Spatial Data Nonparametric Regression And Response Surface Maximization 2nd Edition Ronald Christensen Auth https://ebookbell.com/product/advanced-linear-modeling-multivariate- time-series-and-spatial-data-nonparametric-regression-and-response- surface-maximization-2nd-edition-ronald-christensen-auth-4271670 Multivariate Nonparametric Methods With R An Approach Based On Spatial Signs And Ranks 1st Edition Hannu Oja Auth https://ebookbell.com/product/multivariate-nonparametric-methods-with- r-an-approach-based-on-spatial-signs-and-ranks-1st-edition-hannu-oja- auth-1855414 Modern Nonparametric Robust And Multivariate Methods Festschrift In Honour Of Hannu Oja Klaus Nordhausen https://ebookbell.com/product/modern-nonparametric-robust-and- multivariate-methods-festschrift-in-honour-of-hannu-oja-klaus- nordhausen-5235926 Multivariate Statistical Modeling In Engineering And Management 1st Edition Jhareswar Maiti https://ebookbell.com/product/multivariate-statistical-modeling-in- engineering-and-management-1st-edition-jhareswar-maiti-46083382
- 3. Multivariate Data Analysis Fionn Murtagh Andre Heck https://ebookbell.com/product/multivariate-data-analysis-fionn- murtagh-andre-heck-47912096 Multivariate Reducedrank Regression Theory Methods And Applications 2nd Edition Gregory C Reinsel https://ebookbell.com/product/multivariate-reducedrank-regression- theory-methods-and-applications-2nd-edition-gregory-c-reinsel-48696422 Multivariate Frequency Analysis Of Hydrometeorological Variables A Copulabased Approach Fateh Chebana https://ebookbell.com/product/multivariate-frequency-analysis-of- hydrometeorological-variables-a-copulabased-approach-fateh- chebana-48775100 Multivariate Calculus Samiran Karmakar Sibdas Karmakar https://ebookbell.com/product/multivariate-calculus-samiran-karmakar- sibdas-karmakar-49224188 Multivariate Calculus Samiran Karmakar Sibdas Karmakar https://ebookbell.com/product/multivariate-calculus-samiran-karmakar- sibdas-karmakar-49492868
- 7. Multivariate Nonparametric Regression and Visualization
- 8. WILEY SERIES IN COMPUTATIONAL STATISTICS Consulting Editors: Paolo Giudici University of Pavia, Italy Geof H. Givens Colorado State University, USA Bani K. Mallick Texas A&M University, USA Wiley Series in Computational Statistics is comprised of practical guides and cut- ting edge research books on new developments in computational statistics. It fea- tures quality authors with a strong applications focus. The texts in the series provide detailed coverage of statistical concepts, methods and case studies in areas at the in- terface of statistics, computing, and numerics. With sound motivation and a wealth of practical examples, the books show in concrete terms how to select and to use appropriate ranges of statistical computing techniques in particular fields of study. Readers are assumed to have a basic under- standing of introductory terminology. The series concentrates on applications of computational methods in statistics to fields of bioinformatics, genomics, epidemiology, business, engineering, finance and applied statistics. Billard and Diday • Symbolic Data Analysis: Conceptual Statistics and Data Mining Bolstad • Understanding Computational Bayesian Statistics Dunne • A Statistical Approach to Neural Networks for Pattern Recognition Ntzoufras • Bayesian Modeling Using WinBUGS Klemela • Multivariate Nonparametric Regression and Visualization: With R and Applications to Finance
- 9. Multivariate Nonparametric Regression and Visualization With R and Applications to Finance JUSSI KLEMELA Department of Mathematical Sciences University of Ouiu Ouiu, Finland W I L E Y
- 10. Copyright © 2014 by John Wiley & Sons, Inc. All rights reserved. Published by John Wiley & Sons, Inc., Hoboken, New Jersey. Published simultaneously in Canada. No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 750-4470, or on the web at www.copyright.com. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., Ill River Street, Hoboken, NJ 07030, (201) 748-6011, fax (201) 748-6008, or online at http://www.wiley.com/go/permission. Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representation or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages. For general information on our other products and services please contact our Customer Care Department within the United States at (800) 762-2974, outside the United States at (317) 572-3993 or fax (317)572-4002. Wiley also publishes its books in a variety of electronic formats. Some content that appears in print, however, may not be available in electronic formats. For more information about Wiley products, visit our web site at www.wiley.com. Library of Congress Cataloging-in-Publication Data: Klemela, Jussi, 1965- Multivariate nonparametric regression and visualization : with R and applications to finance / Jussi Klemela. pages cm. — (Wiley series in computational statistics ; 699) Includes bibliographical references and index. ISBN 978-0-470-38442-8 (hardback) 1. Finance—Mathematical models. 2. Visualization. 3. Regression analysis. I. Title. HG176.5.K55 2014 519.5*36—dc23 2013042095 Printed in Singapore. 10 9 8 7 6 5 4 3 2 1
- 11. To my parents
- 13. CONTENTS IN BRIEF PART I METHODS OF REGRESSION AND CLASSIFICATION 1 Overview of Regression and Classification 3 2 Linear Methods and Extensions 77 3 Kernel Methods and Extensions 127 4 Semiparametric and Structural Models 229 5 Empirical Risk Minimization 241 PART II VISUALIZATION 6 Visualization of Data 277 7 Visualization of Functions 295 vii
- 15. CONTENTS Preface xvii Introduction xix 1.1 Estimation of Functionals of Conditional Distributions xx 1.2 Quantitative Finance xxi 1.3 Visualization xxi 1.4 Literature xxiii PART I METHODS OF REGRESSION AND CLASSIFICATION 1 Overview of Regression and Classification 3 1-1 Regression 3 1.1.1 Random Design and Fixed Design 4 1.1.2 Mean Regression 5 1.1.3 Partial Effects and Derivative Estimation 8 1.1.4 Variance Regression 9 1.1.5 Covariance and Correlation Regression 13 1.1.6 Quantile Regression 14 1.1.7 Approximation of the Response Variable 18 1.1.8 Conditional Distribution and Density 21 ix
- 16. x CONTENTS 1.1.9 Time Series Data 23 1.1.10 Stochastic Control 25 1.1.11 Instrumental Variables 26 1.2 Discrete Response Variable 29 1.2.1 Binary Response Models 29 1.2.2 Discrete Choice Models 31 1.2.3 Count Data 33 1.3 Parametric Family Regression 33 1.3.1 General Parametric Family 33 1.3.2 Exponential Family Regression 35 1.3.3 Copula Modeling 36 1.4 Classification 37 1.4.1 BayesRisk 38 1.4.2 Methods of Classification 39 1.5 Applications in Quantitative Finance 42 1.5.1 Risk Management 42 1.5.2 Variance Trading 44 1.5.3 Portfolio Selection 45 1.5.4 Option Pricing and Hedging 50 1.6 Data Examples 52 1.6.1 Time Series of S&P 500 Returns 52 1.6.2 Vector Time Series of S&P 500 and Nasdaq-100 Returns 53 1.7 Data Transformations 53 1.7.1 Data Sphering 54 1.7.2 Copula Transformation 55 1.7.3 Transformations of the Response Variable 56 1.8 Central Limit Theorems 58 1.8.1 Independent Observations 58 1.8.2 Dependent Observations 58 1.8.3 Estimation of the Asymptotic Variance 60 1.9 Measuring the Performance of Estimators 61 1.9.1 Performance of Regression Function Estimators 61 1.9.2 Performance of Conditional Variance Estimators 66 1.9.3 Performance of Conditional Covariance Estimators 68 1.9.4 Performance of Quantile Function Estimators 69 1.9.5 Performance of Estimators of Expected Shortfall 71 1.9.6 Performance of Classifiers 72 1.10 Confidence Sets 73
- 17. CONTENTS Xiii 1.10.1 Pointwise Confidence Intervals 73 1.10.2 Confidence Bands 75 1.11 Testing 75 2 Linear Methods and Extensions 77 2.1 Linear Regression 78 2.1.1 Least Squares Estimator 79 2.1.2 Generalized Method of Moments Estimator 81 2.1.3 Ridge Regression 84 2.1.4 Asymptotic Distributions for Linear Regression 87 2.1.5 Tests and Confidence Intervals for Linear Regression 90 2.1.6 Variable Selection 92 2.1.7 Applications of Linear Regression 94 2.2 Varying Coefficient Linear Regression 97 2.2.1 The Weighted Least Squares Estimator 97 2.2.2 Applications of Varying Coefficient Regression 98 2.3 Generalized Linear and Related Models 102 2.3.1 Generalized Linear Models 102 2.3.2 Binary Response Models 104 2.3.3 Growth Models 107 2.4 Series Estimators 107 2.4.1 Least Squares Series Estimator 107 2.4.2 Orthonormal Basis Estimator 108 2.4.3 Splines 110 2.5 Conditional Variance and ARCH Models 111 2.5.1 Least Squares Estimator 112 2.5.2 ARCH Model 113 2.6 Applications in Volatility and Quantile Estimation 116 2.6.1 Benchmarks for Quantile Estimation 116 2.6.2 Volatility and Quantiles with the LS Regression 118 2.6.3 Volatility with the Ridge Regression 121 2.6.4 Volatility and Quantiles with ARCH 122 2.7 Linear Classifiers 124 3 Kernel Methods and Extensions 127 3.1 Regressogram 129 3.2 Kernel Estimator 130 3.2.1 Definition of the Kernel Regression Estimator 130
- 18. xii CONTENTS 3.2.2 Comparison to the Regressogram 132 3.2.3 Gasser-Miiller and Priestley-Chao Estimators 134 3.2.4 Moving Averages 134 3.2.5 Locally Stationary Data 136 3.2.6 Curse of Dimensionality 140 3.2.7 Smoothing Parameter Selection 140 3.2.8 Effective Sample Size 142 3.2.9 Kernel Estimator of Partial Derivatives 145 3.2.10 Confidence Intervals in Kernel Regression 146 3.3 Nearest-Neighbor Estimator 147 3.4 Classification with Local Averaging 148 3.4.1 Kernel Classification 148 3.4.2 Nearest-Neighbor Classification 149 3.5 Median Smoothing 151 3.6 Conditional Density Estimation 152 3.6.1 Kernel Estimator of Conditional Density 152 3.6.2 Histogram Estimator of Conditional Density 156 3.6.3 Nearest-Neighbor Estimator of Conditional Density 157 3.7 Conditional Distribution Function Estimation 158 3.7.1 Local Averaging Estimator 159 3.7.2 Time-Space Smoothing 159 3.8 Conditional Quantile Estimation 160 3.9 Conditional Variance Estimation 162 3.9.1 State-Space Smoothing and Variance Estimation 162 3.9.2 GARCH and Variance Estimation 163 3.9.3 Moving Averages and Variance Estimation 172 3.10 Conditional Covariance Estimation 176 3.10.1 State-Space Smoothing and Covariance Estimation 178 3.10.2 GARCH and Covariance Estimation 178 3.10.3 Moving Averages and Covariance Estimation 181 3.11 Applications in Risk Management 181 3.11.1 Volatility Estimation 182 3.11.2 Covariance and Correlation Estimation 193 3.11.3 Quantile Estimation 198 3.12 Applications in Portfolio Selection 205 3.12.1 Portfolio Selection Using Regression Functions 205 3.12.2 Portfolio Selection Using Classification 215 3.12.3 Portfolio Selection Using Markowitz Criterion 223
- 19. CONTENTS Xiii 4 Semiparametric and Structural Models 229 4.1 Single-Index Model 230 4.1.1 Definition of the Single-Index Model 230 4.1.2 Estimators in the Single-Index Model 230 4.2 Additive Model 234 4.2.1 Definition of the Additive Model 234 4.2.2 Estimators in the Additive Model 235 4.3 Other Semiparametric Models 237 4.3.1 Partially Linear Model 237 4.3.2 Related Models 238 5 Empirical Risk Minimization 241 5.1 Empirical Risk 243 5.1.1 Conditional Expectation 243 5.1.2 Conditional Quantile 244 5.1.3 Conditional Density 245 5.2 Local Empirical Risk 247 5.2.1 Local Polynomial Estimators 247 5.2.2 Local Likelihood Estimators 255 5.3 Support Vector Machines 257 5.4 Stagewise Methods 259 5.4.1 Forward Stagewise Modeling 259 5.4.2 Stagewise Fitting of Additive Models 261 5.4.3 Projection Pursuit Regression 262 5.5 Adaptive Regressograms 264 5.5.1 Greedy Regressograms 264 5.5.2 CART 268 5.5.3 Dyadic CART 271 5.5.4 Bootstrap Aggregation 272 PART II VISUALIZATION 6 Visualization of Data 277 6.1 Scatter Plots 278 6.1.1 Two-Dimensional Scatter Plots 278 6.1.2 One-Dimensional Scatter Plots 278 6.1.3 Three- and Higher-Dimensional Scatter Plots 282 6.2 Histogram and Kernel Density Estimator 283
- 20. xiv CONTENTS 6.3 Dimension Reduction 284 6.3.1 Projection Pursuit 284 6.3.2 Multidimensional Scaling 286 6.4 Observations as Objects 288 6.4.1 Graphical Matrices 289 6.4.2 Parallel Coordinate Plots 290 6.4.3 Other Methods 293 7 Visualization of Functions 295 7.1 Slices 296 7.2 Partial Dependence Functions 298 7.3 Reconstruction of Sets 299 7.3.1 Estimation of Level Sets of a Function 300 7.3.2 Point Cloud Data 303 7.4 Level Set Trees 304 7.4.1 Definition and Illustrations 304 7.4.2 Calculation of Level Set Trees 308 7.4.3 Volume Function 313 7.4.4 Barycenter Plot 321 7.4.5 Level Set Trees in Regression Function Estimation 322 7.5 Unimodal Densities 325 7.5.1 Probability Content of Level Sets 327 7.5.2 Set Visualization 327 Appendix A: R Tutorial 329 A.1 Data Visualization 329 A. 1.1 QQ Plots 329 A. 1.2 Tail Plots 330 A. 1.3 Two-Dimensional Scatter Plots 330 A. 1.4 Three-Dimensional Scatter Plots 331 A.2 Linear Regression 331 A.3 Kernel Regression 332 A.3.1 One-Dimensional Kernel Regression 332 A.3.2 Moving Averages 333 A.3.3 Two-Dimensional Kernel Regression 334 A.3.4 Three- and Higher-Dimensional Kernel Regression 336 A.3.5 Kernel Estimator of Derivatives 338 A.3.6 Combined State- and Time-Space Smoothing 340 A.4 Local Linear Regression 341
- 21. CONTENTS Xiii A.4.1 One-Dimensional Local Linear Regression 341 A.4.2 Two-Dimensional Local Linear Regression 342 A.4.3 Three- and Higher-Dimensional Local Linear Regression 343 A.4.4 Local Linear Derivative Estimation 343 A.5 Additive Models: Backfitting 344 A.6 Single-Index Regression 345 A.6.1 Estimating the Index 346 A.6.2 Estimating the Link Function 346 A.6.3 Plotting the Single-Index Regression Function 346 A.7 Forward Stagewise Modeling 347 A.7.1 Stagewise Fitting of Additive Models 347 A.7.2 Projection Pursuit Regression 348 A.8 Quantile Regression 349 A.8.1 Linear Quantile Regression 349 A.8.2 Kernel Quantile Regression 349 References 351 Author Index 361 Topic Index 365
- 23. PREFACE The book is intended for students and researchers who want to learn to apply non- parametric and semiparametric methods and to use visualization tools related to these estimation methods. In particular, the book is intended for students and researchers in quantitative finance who want to apply statistical methods and for students and researchers of statistics who want to learn to apply statistical methods in quantitative finance. The book continues the themes of Klemela (2009), which studied density estimation. The current book focuses on regression function estimation. The book was written at the University of Ouiu, Department of Mathematical Sciences. I wish to acknowledge the support provided by the University of Ouiu and the Department of Mathematical Sciences. The web page of the book is http://cc.oulu.fi/~jklemela/regstruct/. Jussi KLEMELA Ouiu, Finland October 2013 xvii
- 25. INTRODUCTION We study regression analysis and classification, as well as estimation of conditional variances, quantiles, densities, and distribution functions. The focus of the book is on nonparametric methods. Nonparametric methods are flexible and able to adapt to various kinds of data, but they can suffer from the curse of dimensionality and from the lack of interpretability. Semiparametric methods are often able to cope with quite high-dimensional data and they are often easier to interpret, but they are less flexible and their use may lead to modeling errors. In addition to terms "nonparametric esti- mator" and "semiparametric estimator", we can use the term "structured estimator" to denote such estimators that arise, for example, in additive models. These estimators obey a structural restriction, whereas the term "semiparametric estimator" is used for estimators that have a parametric and a nonparametric component. Nonparametric, semiparametric, and structured methods are well established and widely applied. There are, nevertheless, areas where a further work is useful. We have included three such areas in this book: 1. Estimation of several functionals of a conditional distribution; not only esti- mation of the conditional expectation but also estimation of the conditional variance and conditional quantiles. 2. Quantitative finance as an area of application for nonparametric and semipara- metric methods. x i x
- 26. XX INTRODUCTION 3. Visualization tools in statistical learning. 1.1 ESTIMATION OF FUNCTIONALS OF CONDITIONAL DISTRIBUTIONS One of the main topics of the book are the kernel methods. Kernel methods are easy to implement and computationally feasible, and their definition is intuitive. For example, a kernel regression estimator is a local average of the values of the response variable. Local averaging is a general regression method. In addition to the kernel estimator, examples of local averaging include the nearest-neighbor estimator, the regressogram, and the orthogonal series estimator. We cover linear regression and generalized linear models. These models can be seen as starting points to many semiparametric and structured regression models. For example, the single index model, the additive model, and the varying coefficient linear regression model can be seen as generalizations of the linear regression model or the generalized linear model. Empirical risk minimization is a general approach to statistical estimation. The methods of empirical risk minimization can be used in regression function estimation, in classification, in quantile regression, and in the estimation of other functionals of the conditional distribution. The method of local empirical risk minimization is a method which can be seen as a generalization of the kernel regression. A regular regressogram is a special case of local averaging, but the empirical choice of the partition leads to a rich class of estimators. The choice of the parti- tion is made using empirical risk minimization. In the one- and two-dimensional cases a regressogram is usually less efficient than the kernel estimator, but in high- dimensional cases a regressogram can be useful. For example, a method to select the partition of a regressogram can be seen as a method of variable selection, if the chosen partition is such that it can be defined using only a subset of the variables. The estimators that are defined as a solution of an optimization problem, like the min- imizers of an empirical risk, need typically be calculated with numerical methods. Stagewise algorithms can also be taken as a definition of an estimator, even without giving an explicit minimization problem which they solve. A regression function is defined as the conditional expectation of the distribution of a response variable. The conditional expectation is useful in making predictions as well as in finding causal relationships. We cover also the estimation of the condi- tional variance and conditional quantiles. These are needed to give a more complete view of the conditional distribution. Also, the estimation of the conditional variance and conditional quantiles is needed in risk management, which is an important area of quantitative finance. The conditional variance can be estimated by estimating the conditional expectation of the squared random variable, whereas a conditional quantile is a special case of the conditional median. In the time series setting the stan- dard approaches for estimating the conditional variance are the ARCH and GARCH modeling, but we discuss nonparametric alternatives. The GARCH estimator is close
- 27. QUANTITATIVE FINANCE XXi to a moving average, whereas the ARCH estimator is related to linear state space modeling. In classification we are not interested in the estimation of functionals of a distribu- tion, but the aim is to construct classification rules. However, most of the regression function estimation methods have a counterpart in classification. 1.2 QUANTITATIVE FINANCE Risk management, portfolio selection, and option pricing can be identified as three important areas of quantitative finance. Parametric statistical methods have been dominating the statistical research in quantitative finance. In risk management, probability distributions have been modeled with the Pareto distribution or with distributions derived from the extreme value theory. In portfolio selection the multi- variate normal model has been used together with the Markowitz theory of portfolio selection. In option pricing the Black-Scholes model of stock prices has been widely applied. The Black-Scholes model has also been extended to more general parametric models for the process of stock prices. In risk management the p-quantile of a loss distribution has a direct interpretation as such threshold that the probability of the loss exceeding the threshold is less than p. Thus estimation of conditional quantiles is directly relevant for risk management. Unconditional quantile estimators do not take into account all available information, and thus in risk management it is useful to estimate conditional quantiles. The estimation of the conditional variance can be applied in the estimation of a conditional quantile, because in location-scale families the variance determines the quantiles. The estimation of conditional variance can be extended to the estimation of the conditional covariance or the conditional correlation. We apply nonparametric regression function estimation in portfolio selection. The portfolio is selected either with the maximization of a conditional expected utility or with the maximization of a Markowitz criterion. When the collection of allowed portfolio weights is a finite set, then also classification can be used in portfolio selection. The squared returns are much easier to predict than the returns themselves, and thus in quantitative finance the focus has been in the prediction of volatility. However, it can be shown that despite the weak predictability of the returns, portfolio selection can profit from statistical prediction. Option pricing can be formulated as a problem of stochastic control. We do not study the statistics of option pricing in detail, but give a basic framework for solving some option pricing problems nonparametrically. 1.3 VISUALIZATION Statistical visualization is often considered as a visualization of the raw data. The visualization of the raw data can be a part of the exploratory data analysis, a first step to model building, and a tool to generate hypotheses about the data-generating mechanism. However, we put emphasis on a different approach to visualization.
- 28. XXii INTRODUCTION In this approach, visualization tools are associated with statistical estimators or inference procedures. For example, we estimate first a regression function and then try to visualize and describe the properties of this regression function estimate. The distinction between the visualization of the raw data and the visualization of the estimator is not clear when nonparametric function estimation is used. In fact, nonparametric function estimation can be seen as a part of exploratory data analysis. The SiZer is an example of a tool that combines visualization and inference, see Chaudhuri & Marron (1999). This methodology combines formal testing for the existence of modes with the SiZer maps to find out whether a mode of a density estimate of a regression function estimate is really there. Semiparametric function estimates are often easier to visualize than nonparametric function estimates. For example, in a single index model the regression function estimate is a composition of a linear function and a univariate function. Thus in a single index model we need only to visualize the coefficients of the linear function and a one-dimensional function. The ease of visualization gives motivation to study semiparametric methods. CART, as presented in Breiman, Friedman, Olshen & Stone (1984), is an example of an estimation method whose popularity is not only due to its statistical properties but also because it is defined in terms of a binary tree that gives directly a visualization of the estimator. Even when it is possible to find estimators with better statistical properties than CART, the possibility to visualization gives motivation to use CART. Visualization of nonparametric function estimates, such as kernel estimates, is challenging. For the visualization of completely nonparametric estimates, we can use level set tree-based methods, as presented in Klemela (2009). Level set tree- based methods have found interest also in topological data analysis and in scientific visualization, and these methods have their origin in the concept of a Reeb graph, defined originally in Reeb (1946). In density estimation we are often interested in the mode structure of the density, defined as the number of local extremes, the largeness of the local extremes, and the location of the local extremes. The local extremes of a density function are related to the areas of concentration of the probability mass. In regression function estimation we are also interested in the mode structure. The local maxima of a regression function are related to the regions of the space of the explanatory variables where the response variable takes the largest values. The antimode structure is equally important to describe. The antimode structure means the number of local minima, the size of the local minima, and the location of the local minima. The local minima of a regression function are related to the areas of the space of the explanatory variables where the response variable takes the smallest values. The mode structure of a regression function does not give complete information about the properties of the regression function. In regression analysis we are inter- ested in the effects of the explanatory variables on the response variable and in the interaction between the explanatory variables. The effect of an explanatory variable can be formalized with the concept of a partial effect. The partial effect of an ex- planatory variable is the partial derivative of the regression function with respect to this variable. Nearly constant partial effects indicate that the regression function is
- 29. LITERATURE XXiM close to a linear function, since the partial derivatives of a linear function are con- stants. The local maxima of a partial effect correspond to the areas in the space of the explanatory variables where the increase of the expected value of the response variable, resulting from an increase of the value of the explanatory variable, is the largest. We can use level set trees of partial effects to visualize the mode structure and the antimode structure of the partial effects, and thus to visualize the effects and the interactions of the explanatory variables. 1.4 LITERATURE We mention some of the books that have been used in the preparation of this book. Hardle (1990) covers nonparametric regression with an emphasis on kernel regres- sion, discussing smoothing parameter selection, giving confidence bands, and provid- ing various econometric examples. Hastie, Tibshirani & Friedman (2001) describe high-dimensional linear and nonlinear classification and regression methods, giv- ing many examples from biometry and machine learning. Gyorfi, Kohler, Krzyzak & Walk (2002) cover asymptotic theory of kernel regression, nearest-neighbor re- gression, empirical risk minimization, and orthogonal series methods, and they also include a treatment of time series prediction. Ruppert, Wand & Carroll (2003) view nonparametric regression as an extension of parametric regression and treat them together. Hardle, Miiller, Sperlich & Werwatz (2004) explain single index models, generalized partial linear models, additive models, and several nonparametric regres- sion function estimators, giving econometric examples. Wooldridge (2005) provides an asymptotic theory of linear regression, including instrumental variables and panel data. Fan & Yao (2005) study nonlinear time series and use nonparametric function estimation in time series prediction and explanation. Wasserman (2005) provides information on nonparametric regression and density estimation with confidence intervals and bootstrap confidence intervals. Horowitz (2009) covers semiparamet- ric models and discusses the identifiability and asymptotic distributions. Spokoiny (2010) introduces local parametric methods into nonparametric estimation. Bouchaud & Potters (2003) have developed nonparametric techniques for financial analysis. Franke, Hardle & Hafner (2004) discuss statistical analysis of financial markets, with emphasis being on the parametric methods. Ruppert (2004) is a textbook suitable for statistics students interested in quantitative finance, and this book discusses statistical tools related to classical financial models. Malevergne & Sornette (2005) have analyzed financial data with nonparametric methods. Li & Racine (2007) consider various non- and semiparametric regression models presenting asymptotic distribution theory and the theory of smoothing parameter selection, directing towards econometric applications.
- 31. PARTI M E T H O D S O F R E G R E S S I O N A N D CLASSIFICATION
- 33. CHAPTER 1 OVERVIEW OF REGRESSION AND CLASSIFICATION 1.1 REGRESSION In regression analysis we are interested in prediction or in inferring causal rela- tionships. We try to predict the value of a response variable given the values of explanatory variables or try to deduce the causal influence of the explanatory vari- ables to the response variable. The inference of a causal relationship is important when we want to to change the values of an explanatory variable in order to get an op- timal value for the response variable. For example, we want to know the influence of education to the employment status of a worker in order to choose the best education. On the other hand, prediction is applied also in the cases when we are not able to, or do not wish to, change the values of the response variable. For example, in volatility prediction it is reasonable to use any variables that have a predictive relevance even if these variables do not have any causal relationship to volatility. Both in prediction and in estimation of causal influence, it is useful to estimate the conditional expectation E{YX = x) of the response variable Y £ R given the explanatory variables X £ Rd . The choice of the explanatory variables and the method of estimation can depend on the purpose Multivariate Nonparametric Regression and Visualization. By Jussi Klemela 3 Copyright © 2014 John Wiley & Sons, Inc.
- 34. 4 OVERVIEW OF REGRESSION AND CLASSIFICATION of the research. In prediction an explanatory variable can be any variable that has predictive relevance whereas in the estimation of a causal influence the explanatory variables are determined by the scientific theory about the causal relationship. For the purpose of causal inference, it is reasonable to choose an estimation method that can help to find the partial effect of a given explanatory variable to the response variable. The partial effect is defined in Section 1.1.3. In linear regression the regression function estimate is a linear function: f ( x ) =a + fiixi + • • • + pdxd. (1.1) A different type of linearity occurs, if the estimator can be written as n f(x) = ^ l i ( x ) Y i , (1.2) 2=1 for some sequence of weights h(x)1..., ln(x). In fact, for the linear regression estimate, representations (1.1) and (1.2) hold; see (2.11). In the case of local averaging estimators, like regressogram, kernel estimator, and nearest-neighbor estimator, we use the notation f(x) = Y17=i Pi(x ) the c a s e °f local averaging estimators the weights pi(x) satisfy the properties that pi(x) is close to zero when Xi is distant from x and that Pi(x) is large when Xi is near x. Local averaging is discussed in Section 3. There exists regression function estimates that cannot be written as in (1.2), like the orthogonal series estimators with hard thresholding; see (2.72). In addition to the estimation of the conditional expectation of the response variable given the explanatory variables, we can consider also the estimation of the conditional median of the response variable given the explanatory variables, or the estimation of other conditional quantiles of the response variable given the explanatory vari- ables, which is called quantile regression. Furthermore, we will consider estimation of the conditional variance, as well as estimation of the conditional density and the conditional distribution function of the response variable given the explanatory variables. In regression analysis the response variable can take any real value or any value in a given interval, but we consider also classification. In classification the response variable can take only a finite number of distinct values and the interest lies in the prediction of the values of the response variable. 1.1.1 Random Design and Fixed Design Random Design Regression In random design regression the data are a se- quence of n pairs {xi,yi),...,(xn,yn), (1.3) where X{ G Hd and yi G R for i = 1,..., n. Data are modeled as a realization of a sequence of n random vectors (1.4)
- 35. REGRESSION 5 However, sometimes we do not distinguish notationally a random variable and its realization, and the notation of (1.4) is used also in the place of notation (1.3) to denote a realization of the random vectors and not the random vectors themselves. In regression analysis we typically want to estimate the conditional expectation f(x) = E(YX = x), and now we assume that the sequence (X, Yi),..., (Xn, Yn) consists of identically distributed random variables, and {X,Y) has the same distribution as (X^Y*), i = 1,..., n. Besides conditional expectation we could estimate conditional mode, conditional variance, conditional quantile, and so on. Estimation of the conditional centers of distribution are discussed in Section 1.1.2 and estimation of conditional risk measures such as variance and quantiles are discussed in Section 1.1.4 and in Section 1.1.6. Fixed Design Regression In fixed design regression the data are a sequence 2/1 > • • • ,2/n, where yi G R, i — 1,..., n. We assume that every observation yi is associated with a fixed design point Xi G Rd . Now the design points are not chosen by a random mechanism, but they are chosen by the conducter of the experiment. Typical examples could be time series data, where Xi is the time when the observation yi is recorded, and spatial data, where Xi is the location where the observation yi is made. Time series data are discussed in Section 1.1.9. We model the data as a sequence of random variables Y,..., Yn. In the fixed design regression we typically do not assume that the data are identically distributed. For example, we may assume that Yi = f(xi) + 6i, i — 1,..., n, where Xi — i/n, f : [0,1] R is the function we want to estimate, and Eei — 0. Now the data Y,..., Yn are not identically distributed, since the observations Yi have different expectations. 1.1.2 Mean Regression The regression function is typically defined as a conditional expectation. Besides expectation and conditional expectation also median and conditional median can be used to characterize the center of a distribution and thus to predict and explain with the help of explanatory variables. We mention also the mode (maximum of the density function) as a third characterization of the center of a distribution, although the mode is typically not used in regression analysis.
- 36. 6 OVERVIEW OF REGRESSION AND CLASSIFICATION Expectation and Conditional Expectation When the data ..., (Xn, Yn) are a sequence of identically distributed random variables, we can use the data to estimate the regression function, defined as the conditional expectation of Y given X: f ( x ) = E(Y X = x), x G (1.5) where (X, Y) has the same distribution as (Xi, Yi), i = 1,..., n, and X G Rd , Y G R. The random variable Y is called the response variable, and the elements of random vector X are called the explanatory variables. The mean of random variable Y G R with a continuous distribution can be defined by / OO yfv(y)dy, (1.6) -oo where f y : R — » R is the density function of Y. The regression function has been defined in (1.5) as the conditional mean of Y, and the conditional expectation can be defined in terms of the conditional density as / oo yfyx=Ay)dy, -oo where the conditional density can be defined as , , x fx,v(x,y) fyx=x(y) = , / x , J / ^ R , (1.7) fx{x) when f x ( x ) > 0 and fY x=x(y) = 0 otherwise, where fx,Y ' R d + 1 — > R is the joint density of (X, Y) and f x • R d — > R is the density of X: fx(x)= / fx,v(x,y)dy, x G Hd . J R Figure 1.1 illustrates mean regression. Our data consist of the daily S&P 500 returns Rt = (St — St-i)/St-i, where St is the price of the index. There are about 16,000 observations. The S&P 500 index data are described more precisely in Section 1.6.1. We define the explanatory and the response variables as Xt lQ ge k 2=1 2 t-i ' Yt = loge | Rt Panel (a) shows the scatter plot of (Xt, Y*), and panel (b) shows the estimated density of (Xt,Yt) together with the estimated regression functions. The red line shows the linear regression function estimate, and the blue line shows a kernel regression estimate with smoothing parameter h = 0.4. The density is estimated using kernel
- 37. REGRESSION 7 (a) (b) Figure 1.1 Mean regression, (a) A scatter plot of regression data, (b) A contour plot of the estimated joint density of the explanatory variable and the response variable. The linear regression function estimate is shown with red and the kernel regression estimate is shown with blue. density estimation with smoothing parameter h = 0.6. Linear regression is discussed in Section 2.1, and kernel methods are discussed in Section 3.2. In the scatter plot we have used histogram smoothing with 1002 bins, as explained in Section 6.1.1. This example indicates that the daily returns are dependent random variables, although it can be shown that they are nearly uncorrelated. Median and Conditional Median The median can be defined in the case of continuous distribution function of a random variable Y G R as the number median(Y) G R satisfying P(Y < median(Y)) = 0.5. In general, covering also the case of discrete distributions, we can define the median uniquely as the generalized inverse of the distribution function: median(Y) = inf{y : P(Y <y)> 0.5}. (1.8) The conditional median is defined using the conditional distribution of Y given X: median (Y X = x) = inf{y : P(Y <y X = x)> 0.5}, x G Rd . (1.9) The sample median of observations Y,..., Yn G R can be defined as the median of the empirical distribution. The empirical distribution is the discrete distribution with the probability mass function P({Y;}) = 1/n for % — 1,..., n. Then, median(Yi,..., Yn) = Y[ n / 2 ] + 1 , (1.10) where Y^ < • • • < Y(n) is the ordered sample and [x] is the largest integer smaller or equal to x.
- 38. 8 OVERVIEW OF REGRESSION AND CLASSIFICATION Mode and Conditional Mode The mode is defined as an argument maximizing the density function of a random variable: mode(Y) = argmax^jxfy(y), (1.11) where f y : R R is the density function of Y. The density f y can have several local maxima, and the use of the mode seems to be interesting only in cases where the density function is unimodal (has one local maximum). The conditional mode is defined as an argument maximizing the conditional density: mode(y X = x)= argmaxyGR/y|X=a:(?/). 1.1.3 Partial Effects and Derivative Estimation Let us consider mean regression, where we are estimating the conditional expectation E(Y | X — x), where X = (Xi,..., Xd) is the vector of explanatory variables and we denote x = (x,..., Xd). The partial effect of the variable X is defined as the partial derivative d p(x i;x2,...,xd) = —— E(YX = x). ox i The partial effect describes how the conditional expectation of Y changes when the value of X is changed, when the values of the other variables are fixed. In general, the partial effect is a function of x that is different for each x2,..., xd. However, for the linear model E(Y | X = x) = a + /3f x we have p(x i;x2, ...,xd) = so that the partial effect is a constant which is the same for all x2, • • •, xd. Linear models are studied in Section 2.1. For the additive model E(Y X = x) = /i (x) + h fd(xd) we have p(x i;x2, ...,xd) = f'(x i), so that the partial effect is a function of x which is the same for all x2, •.., xd. Thus additive models provide easily interpretable partial effects. Additive models are studied in Section 4.2. For the single index model E(Y | X = x) = g(P'x) we have p(x i;x2l ...,xd)= g'(/3'x)l3i, so that the partial effect is a function of x which is different for each x2,..., xd. Single index models are studied in Section 4.1. The partial elasticity of X is defined as d e{xix2,-..,xd) = — ogE(YX = x) O logXi 9 E(Y | X = x) Xl dxx v 1 ; E(YX = Xy
- 39. REGRESSION 9 whenxi > Oand E(Y X = x) > 0. The partial elasticity describes the approximate percentage change of conditional expectation of Y when the value of X is changed by one percent, when the values of the other variables are fixed.1 The partial semielasticity of X is defined as when E(Y | X = x) > 0. The partial semielasticity describes the approximate percentage change of conditional expectation of Y when the value of X is changed by 1 unit, when the values of the other variables are fixed. We can use the visualization of partial effects as a tool to visualize regression functions. In Section 7.4 we show how level set trees can be used to visualize the mode structure of functions. The mode structure of a function means the number, the largeness, and the location of the local maxima of a function. Analogously, level set trees can be used to visualize the antimode structure of a function, where the antimode structure means the number, the largeness, and the location of the local minima of a function. Local maxima and minima are important characteristics of a regression function. However, we need to know more about a regression function than just the mode structure or antimode structure. Partial effects are a useful tool to convey additional important information about a regression function. If the partial effect is flat for each variable, then we know that the regression function is close to a linear function. When we visualize the mode structure of the partial effect of variable Xi, then we get information about whether a variable X is causing the expected value of the response variable to increase in several locations (the number of local maxima of the partial effect), how much an increase of the value of the variable X increases the expected value of the response variable Y (the largeness of the local maxima of the partial effect), and where the influence of the response variable X is the largest (the location of the local maxima of the partial effect). Analogous conclusions can be made by visualizing the antimode structure of the partial effect. We present two methods for the estimation of partial effects. The first method is to use the partial derivatives of a kernel regression function estimator, and this method is presented in Section 3.2.9. The second method is to use a local linear estimator, and this method is presented in Section 5.2.1. 1.1.4 Variance Regression The mean regression gives information about the center of the conditional distribution, and with the variance regression we get information about the dispersion and on the s(xix2,...,xd) = ~— ogE(YX = x) d E(Y X = x)' 1 1 This interpretation follows from the approximation log f(x + h)~ log/Or) « [/(* + h)~ /(*)]//(*), which follows from the approximation log(rr) « x — 1, when x « 1.
- 40. 1 0 OVERVIEW OF REGRESSION AND CLASSIFICATION heaviness of the tails of the conditional distribution. Variance is a classical measure of dispersion and risk which is used for example in the Markowitz theory of portfolio selection. Partial moments are risk measures that generalize the variance. Variance and Conditional Variance The variance of random variable Y is defined by Var(Y) = E(Y - EY)2 = EY2 - (EY)2 . (1.12) The standard deviation of Y is the square root of the variance of Y. The conditional variance of random variable Y is equal to Var(Y X = x) = E { [ Y - E ( Y X = x)}2 X = x} (1.13) = E(Y2 X = x)- [E(Y | X = x)]2 . (1.14) The conditional standard deviation of Y is the square root of the conditional variance. The sample variance is defined by i=1 i=l where Y,..., Yn is a sample of random variables having identical distribution with Y. Conditional Variance Estimation Conditional variance Var(Y X = x) can be constant not depending on x. Let us write y = / P 0 + €, where f(x) = E(Y | X = x) and e = Y - /(X), so that E{e X = x) = 0. If Var(y | X = x) = E(e2 ) is a constant not depending on x, we say that the noise is homoskedastic. Otherwise the noise is heteroskedastic. If the noise is heteroskedastic, it is of interest to estimate the conditional variance Var(y X = x) = E(e2 X = x). Estimation of the conditional variance can be reduced to the estimation of the conditional expectation by using (1.13). First we estimate the conditional expectation f(x) = E(Y | X = x) by f(x). Second we calculate the residuals ii = Y i - f(Xi), and estimate the conditional variance from the data (Xi, e2 ),..., (Xn, e2 ). Estimation of the conditional variance can be reduced to the estimation of the conditional expectation by using (1.14). First we estimate the conditional expec- tation E(Y2 | X = x) using the regression data (Xi, Y2 ),..., (Xn, Y2 ). Sec- ond we estimate the conditional expectation f(x) = E(Y | X = x) using data (Xi, Yi),..., (Xn, Yn).
- 41. REGRESSION 1 1 Theory of variance estimation is often given in the fixed design case, but the results can be extended to the random design regression by conditioning on the design variables. Let us write a heteroskedastic fixed design regression model Yi = f(xi) + cr{xi) ei, i = 1,..., n, (1.15) where x{ e Kd , f : R d R is the mean function, a : Hd R is the standard deviation function, and Ci are identically distributed with Eci = 0. Now we want to estimate both the function / and the function a. Wasserman (2005, Section 5.6) has proposed making the following transformation. Let Zi — log(F^ — f(xi))2 . Then we have ZI = log(cr2 (xi)) + log E2 . Let / be an estimate of / and define Zi = log(F^ — f(xi))2 . Let g(x) be an estimate of log cr2 (x), obtained using regression data (xi, ZI),..., (xn, ZN), and define a2 (x) = exp{^(x)}. A difference-based method for conditional variance estimation has been proposed. Let x < • • • < xn be univariate fixed design points. Now a2 (x) is estimated with 2~l g(x), where g is a regression function estimate obtained with the regression data i (Yi Yi_ i )2 ), i = 2,..., n. This approach has been used in Wang, Brown, Cai & Levine (2008). Variance Estimation with Homoskedastic Noise Let us consider the fixed design regression model Yi = f(Xi) + €i, Z = 1, ... ,71, where Xi E Rd , / : Hd — > R is the mean function, and Eei = 0. In the case of homoskedastic noise we should estimate a2 =f E(e2 ). Spokoiny (2002) showed that for twice differentiable regression functions / , the optimal rate for the estimation of a2 is n - 1 / 2 for d < 8 and otherwise the optimal rate is n~4 /d . We can first estimate the mean function / by / and then use 2=1 These types of estimators were studied by Miiller & Stadtmiiller (1987), Hall & Carroll (1989), Hall & Marron (1990), and Neumann (1994). Local polynomial estimators were studied by Ruppert, Wand, Hoist & Hossjer (1997), and Fan & Yao (1998). A difference-based estimator was studied by von Neumann (1941). He used the estimator
- 42. 12 OVERVIEW OF REGRESSION AND CLASSIFICATION where it is assumed that x i , . . . G R, and x < • • • < xn. The estimator was studied and modified in various ways in Rice (1984), Gasser, Sroka & Jennen- Steinmetz (1986), Hall, Kay & Titterington (1990), Hall, Kay & Titterington (1991), Thompson, Kay & Titterington (1991), and Munk, Bissantz, Wagner & Freitag (2005). Conditional Variance in a Time Series Setting In a time series setting, when we observe Yt,t = 1,2,..., the conditional heteroskedasticity assumption is that Yt = <jteu f = 0 , ± 1 , ± 2 , . . . , (1.16) where et is an i.i.d. sequence, Eet = 0, Ee2 — 1, and crt is the volatility process. The volatility process is a predictable random process, that is, at is measurable with respect to the sigma-field generated by the variables Yt-i,Yt-2, — When we assume that et is independent from Y^-i, Yt_2, • • then under the conditional heteroskedasticity model, Var(Y, | Ji_x) = Var(a,e, | = *t 2 Var(et I Tt-1) = ^2 Var(et) = a2 , (1.17) where Tt-1 is the sigma-algebra generated by variables Yt -i,Yt -2, — In a con- ditional heteroskedasticity model the main interest is in predicting the value of the random variable of, which is thus related to estimating the conditional variance. The statistical problem is to predict a2 using a finite number of past observations Yi,..., Yt-. Special cases of conditional heteroskedasticity models are the ARCH model discussed in Section 2.5.2 and the GARCH model discussed in Section 3.9.2. Partial Moments The variance of random variable Y G R is defined as Var(Y) = E(Y — EY)2 . The variance can be generalized to other centered moments EY-EYk , for k = 1,2, The centered moments take a contribution both from the left and the right tails of the distribution. When we are interested only in the left tail or in the right tail (losses or gains ), then we can use the lower partial moments or the upper partial moments. The upper partial moment is defined as UPMr,fe(y) = E [ ( Y - r)fe /[T,oo)(F)] and the lower partial moment is defined as LPMT,fc(F) = E [ ( t - y)f c /( _0 0 ,r ] (F)], where k = 0,1, 2,..., and r G R. In risk management r could be the target rate. When Y has density f y , we can write rOO PT UPMr,k (Y)= / (y-T)k fY(y)dy, LPMr,fc(y) = / (r - y)k fY(y) dy. J T J — OO
- 43. REGRESSION 1 3 For example, when k = 0, then UPMr,0(Y) = P(Y > r), LPMr,0(Y) = P(Y < r), so that the upper partial moment is equal to the probability that Y is greater or equal to r and the lower partial moment is equal to the probability that Y is smaller or equal to r. For k = 2 and r = EY the partial moments are called upper or lower semivariance of Y. The lower semivariance is defined as E[(Y-EY)2 I{_^ey](Y)]. (1.18) The square root of the lower semivariance can be used to replace the standard deviation in the definition of the Sharpe ratio or in the Markowitz criterion. We can define conditional versions of partial moments by changing the expectations to conditional expectations. 1.1.5 Covariance and Correlation Regression The covariance of random variables Y and Z is defined by Cov(Y, Z) = E[(Y - EY)(Z - EZ)} = E(YZ) - EYEZ. The sample covariance is defined by 1 n 1 n Z) = - Y(YZ -?){Zi-Z) = - Y YiZi - YZ, i= 1 i= 1 where Y,..., Yn and Z,..., Zn are samples of random variables having identi- cal distributions with Y and Z,Y = n~l J X i and % = n ~l I X l z i- The conditional covariance is obtained by changing the expectations to conditional ex- pectations. We have two methods of estimation of conditional covariance, analogously to two methods of conditional variance estimation based on formulas (1.13) or (1.14). The first method uses Cov(Y, Z) = E[(Y - EY){Z - EZ)} and the second method uses Cov(F, Z) = E(YZ) - EYEZ. The correlation is defined by Cor(y, Z) = C o ^ z ) sd(y) sd(Z)' where sd(Y) and sd(Z) are the standard deviations of Y and Z. The conditional correlation is defined by Cor(V, 2X = x) = .. ( U 9 ) sd(F | X = x) sd(Z | X = x) where sd(Y X = x) = v / Var(F X = x), sd(ZX = x) = vVar(Z X = x).
- 44. 1 4 OVERVIEW OF REGRESSION AND CLASSIFICATION We can write Cor(F, Z X = x)= Cov(y, ZX = x), (1.20) where ~ _ Y ~ _ Z Y = sd(F X = x)' Z = sd(Z X = x)' Thus we have two approaches to the estimation of conditional correlation. 1. We can use (1.19). First we estimate the conditional covariance and the conditional standard deviations. Second we use (1.19) to define the estimator of the conditional correlation. 2. We can use (1.20). First we estimate the conditional standard deviations by sdy(x) and sdz(x), and calculate the standardized observations Yi — Yi/sdy(Xi) and Zi = Zi/sdz(Xi). Second we estimate the conditional correlation using Yi, Zi), i — 1,..., ti. A time series (Yt)tez is weakly stationary if EYt = EYt+h and EYtYt+h depends only on h, for all t,h G Z. For a weakly stationary time series (Yt)tez, the autocovariance function is defined by 7(ft) = cov(yt,yt+/l), and the autocorrelation is defined by p(h) = 7(*0/7(0), where h = 0, ±1, — A vector time series (Xt)tez, Xt G is weakly stationary if EXt = EXt+h and EXtX't+h depends only on h9 for alH, h G Z. For a weakly stationary vector time series (Xt)tez, the autocovariance function is defined by T(h) = Cov(Xt, Xt+h) = E[(Xt - n)(Xt+h ~ m/], (1.21) for h = 0, ± 1 , . . w h e r e i — EXt — EXt+h> Matrix T(h) is a d x d matrix which is not symmetric. It holds that T(h) = r(-h)'. (1.22) 1.1.6 Quantile Regression A quantile generalizes the median. In quantile regression a conditional quantile is estimated. Quantiles can be used to measure the value at risk (VaR). The expected shortfall is a related measure of dispersion and risk.
- 45. REGRESSION 1 5 Quantile and Conditional Quantile The pth quantile is defined as QP(Y) = inf{y : P(Y < y) > p}, x G (1.23) where 0 < p < 1. For p = 1 / 2 , QP(Y) is equal to median med(F), defined in (1.8). In the case of a continuous distribution function we have P(Y<Qp(Y))=p and thus it holds that Qp(Y)--=Fyp), where Fy(y) — P(Y < y) is the distribution function of Y and Fy1 is the inverse of Fy. The pth conditional quantile is defined replacing the distribution of Y with the conditional distribution of Y given X QP(Y X = x)= inf {y : P(Y < y X = x) > p}, x G (1.24) where 0 < p < 1. Conditional quantile estimation has been considered in Koenker (2005) and Koenker & Bassett (1978). Estimation of a Quantile and a Conditional Quantile Estimation of quan- tiles is closely related to the estimation of the distribution function. It is usually possible to derive a method for the estimation of a quantile or a conditional quantile if we have a method for the estimation of a distribution function or a conditional distribution function. Empirical Quantile Let us define the empirical distribution function, based on the dta Y,... ,yn , as 1 n — y e n . 2=1 Now we can define an estimate of the quantile by Qp = inf{x : F(x) > p}, where 0 < p < 1. Now it holds that 0 < p < 1/n, 1/n <p< 2/n, (1.25) (1.26) Qp — Yt (i)' (1.27) Y(n_ 1)5 1 — 2/n < p < 1 — 1 / n , Y(n), 1 — 1/n < p < 1, where the ordered sample is denoted by Y^) < Y(2) < • • < ),,,. A third description of the empirical estimator of the quantile is given by the following steps: 1. Order the sample from the smallest observation to the largest observation: Y(i)<---<Y(ny 2. Let m = pri], where y is the the smallest integer > y. 3. Set Qp = y(m).
- 46. 1 6 OVERVIEW OF REGRESSION AND CLASSIFICATION Standard Deviation-Based Quantile Estimators We can also use an estimate of the standard deviation to derive an estimate for a quantile. Namely, consider the location-scale model Y = /i + CF 6, where p G R, cr > 0, and 6 is a random variable with a continuous distribution. Now where Fe is the distribution function of e. If e has a continuous distribution, then Fe is monotone increasing and the inverse function F~l exists. The pth quantile QP(Y) of Y satisfies P (Y < QP(Y)) = p, and we can solve this equation to get Qp(Y)=n + <rFc-1 (p). Thus, for a known Fe, we get from the estimates p of p and a of cr the estimate Qp{Y)=ii + dF-1 [P). (1.28) Standard Deviation-Based Conditional Quantile Estimators To get an estimate for a conditional quantile in the heteroskedastic fixed design model (1.15), we can use QP(Y X = x)= f(x) + &(x) F - 1 (p). (1.29) Similarly, in the conditional heteroskedasticity model (1.16) we can use Qp(YtFt_l)=&tF-p). (1.30) We apply in Section 2.5.1 and in Section 3.11.3 three quantile estimators which are based on the standard deviation estimates. 1. First estimator uses the standard normal distribution, which gives the quantile estimator where $ is the distribution function of the standard normal distribution. 2. Second estimator uses the ^-distribution, which gives the quantile estimator Qp(Yt | Tt-1) = at t~p), (1.32) where tv is the distribution function of the ^-distribution with v degrees of freedom. If X - tv, then Var(X) = vj{y - 2), so that t~l (p) is the p-quantile of the standardized ^-distribution, which has unit variance. 3. Third estimator uses the empirical quantiles of the residuals. Now = (1.33) where Qres (p) is the empirical quantile of the residuals Yt/at. Empirical quantiles were defined in (1.26). This estimator was suggested in Fan & Gu (2003).
- 47. REGRESSION 1 7 Expected Shortfall The expected shortfall is a measure of risk which aggregates all quantiles in the right tail (or in the left tail). The expected shortfall for the right tail is defined as p Jp ESP {Y) = - / Qu(Y)du, 0 < p < I. JP When Y has a continuous distribution function, then ESP (Y) = E(YY> QP(Y)) = E (Y/[ Q p ( y ) > o o ) (Y)); (1.34) see McNeil, Frey & Embrechts (2005, lemma 2.16). We have defined the loss in (1.86) as the negative of the change in the value of the portfolio, and thus the risk management wants to control the right tails of the loss distribution However, we can define the expected shortfall for the left tail as ESP(Y) = - [P Qu(Y)du, 0 < p < l . (1.35) P Jo When Y has a continuous distribution function, then ESP(Y) = E(YY < QP(Y)) = ± ^ / ( ^ ^ ( Y ) ) . This expression shows that in the case of a continuous distribution function, pESp(F) is equal to the expectation which is taken only over the left tail, when the left tail is defined as the region which is to the left of a quantile of the distribution.2 The expected shortfall can be estimated from the data Y,..., Yn in the case where the expected shortfall is given in (1.34) by using ESP = - ]T Y(i m z —' v rri *—' v 7 i=m where Y^) < • • • < F(n) and m = |"(1 — p)n]. When the expected shortfall is given by (1.35), then we define ^ rn ESp = — } Y(i), 2=1 where m = pn. Let us consider the location-scale model Y = p + ere, where p G R, o > 0, and e is a random variable with a continuous distribution. Now ESp(Y) = /x + o-ESp(e). 2 Sometimes the expected shortfall for the left tail is defined as QP(Y) — EIYI^^^q (y)j (y)] and the absolute shortfall is defined as —E[Y
- 48. 1 8 OVERVIEW OF REGRESSION AND CLASSIFICATION Thus the estimate for the expected shortfall can be obtained as ESp(y) = £ + <7ESp(e), where p is an estimate of p and a is an estimate of a. If e ~ N(0,1) and the expected shortfall is defined for the right tail as in (1.34), then ES.W . where 0 is the density function of the standard normal distribution and is the distribution function of the standard normal distribution. If e ~ tu, where tv is the t-distribution with v degrees of freedom, and the expected shortfall is defined for the right tail as in (1.34), then F<; , , gAK'iP)) ^ + (C1 (p))2 = T^p — ' where gy is the density function of the ^-distribution with v degrees of freedom and tv is the distribution function of the ^-distribution with v degrees of freedom. Expected shortfall is sometimes preferred to the quantiles on the grounds that the expected shortfall satisfies the axiom of subadditivity. Risk measure g is said to be subadditive if g{X + Y) < g(X) + g(Y), where X and Y are random variables interpreted as portfolio losses. Quantiles do not satisfy subadditivity like the expected shortfall. The other axioms of a coherent risk measure are the monotonicity: if Y >X, then g(Y) > g(X); the positive homogeneity: for A > 0, g(XY) = Ag(Y); and the translation invariance: for a G R, g(Y + a) — g(Y) + a. For more about coherent risk measures, see McNeil et al. (2005, Section 6.1). 1.1.7 Approximation of the Response Variable We have defined the regression function in (1.5) as the conditional expectation of the response variable. The conditional expectation can be viewed as an approxi- mation of response variable Y G R with the help of explanatory random variables X i , . . . , Xd G R. The approximation is a random variable f(Xi,..., Xd) G R, where / : Hd — > R is a fixed function. This viewpoint leads to generalizations. The best approximation of the response variable can be defined using various loss functions p : R — » R. The best approximation is f ( X i , . . . , Xd), where / is defined as / = argming e g Ep(Y - g(X)), X = (Xu ... ,Xd), (1.36) where Q is a suitable class of functions g : Hd — > > R. Since / is defined in terms of the unknown distribution of (X, F), we have to estimate / using statistical data available from the distribution of (X, F). Examples of Loss Functions We give examples of different choices of p and
- 49. REGRESSION 1 9 1. When p(t) = t2 and Q is the class of all measurable functions Hd R, then /, defined by (1.36), is equal to the conditional expectation: f ( x ) = E(Y X = x) = argmingegE(Y - g ( X ) f . Indeed, E(g(X) - Y)2 = E(g(X) - E(Y | X))2 + E(E(Y X) - Y")2 , (1.37) because E[(g{X) - E(Y X)){E{Y X) - Y)] = 0, and thus E(g{X) - Y)2 is minimized with respect to g : Hd R by choosing g{x) = E{Y | X = x)? Note also that the expectation EY is the best constant approximation of Y. That is, if we choose Q as the class of constant functions g = {g : Kd R|0(x) = p for all x G R,/i E R}, then EY - argmingegE(Y - g(X)f = argmin^RE(Y - p f . (1.38) Indeed, E(Y - p)2 = E(Y - EY)2 -f (EY - p)2 , and this is minimized with respect to p G R by choosing p — EY. 2. When p(t) = t and Q is the class of all measurable functions Hd — » R, then / defined by (1.36) is the conditional median: med(F X = x)= avgmmgegEY - g(X) (1.39) where the conditional median is defined in (1.9). Equation (1.39) is proved in the next item. 3. When p is defined as PP{t)=tp-I(-oo,0)(t))} = {t t%~1) ' d-40) for 0 < p < 1 and Q is the class of all measurable functions, then the best approximation is the conditional quantile. Figure 1.2 shows the loss function in (1.40) with p = 0.5 (black line) and with p — 0.1 (red line). We show that if the distribution function Fy is strictly monotonic, then QP(Y) = argmine e n EP p (Y - 6). (1.41) 3 Note that the conditional expectation defined as f ( x ) = E(Y | X = x) is a real-valued function of x, but E{X | Y) is a real-valued random variable which can be defined as E(X Y) = f ( X ) .
- 50. 20 OVERVIEW OF REGRESSION AND CLASSIFICATION Figure 1.2 Loss functions for quantile estimation. Loss function in (1.40) with p = 0.5 (black line) and with p = 0.1 (red line). To show (1.41), note that / 0 /'Co (y-0)dFY(y)+p (y-0)dFY(y) -oo J e and thus j^Epp(Y-0) = (l-p) J' dFY(y)-pf™ dFY(y) = FY(0)-p. Setting dEpp(Y - 0)/d0 = 0, we get (1.41), when Fy is strictly monotonic. We can prove similarly the case of conditional quantiles: QP(Y X = x)= argming e g Epp (Y - g(X)), where Q is the class of measurable functions Hd — > R. When p — 1/2, then Pp(t) = I t, and we have proved the result (1.39). Estimation Using Loss Function If a regression function can be characterized as a minimizer of a loss function, then we can use empirical risk minimization with this loss function to define an estimator for the regression function. Empirical risk minimization is discussed in Chapter 5.
- 51. REGRESSION 2 1 For example, conditional expectation f(x) = E(YX = x) can be estimated minimizing the sum of squared errors: n / = a r g m i n / e ^ (Y* - / ( X , ) ) 2 , 2=1 where T is a class of functions / : Rd — > R. For example, T could be the class of linear functions. Estimation of quantiles and conditional quantiles can also be done using empirical risk minimization. The estimator of the pih quantile is n QP(Y) = argmin0GR ^ pp(Yi - 0) 2 = 1 and the estimator of the pih conditional quantile f(x) = QP(Y | X = x) is n f = argmin/Gjr ^ pp(Yi - 2=1 where T is a class of functions / : Hd R. A further idea which we will discuss in Section 5.2 is to define an estimator for the conditional quantile using local empirical risk: n f(x) = argmmeen^2pi(x) pp{Yi - 0), where pi(x) > 0 and Pi(x ) = 1- These weights should have the property that Pi(x) is large when Xi is close to x and pi(x) is small when Xi is far away from x. 1.1.8 Conditional Distribution and Density Instead of estimating only conditional expectation, conditional variance, or con- ditional quantile, we can try to estimate the complete conditional distribution by estimating the conditional distribution function or the conditional density function. Conditional Distribution Function The distribution function of random vari- able Y G R is defined as4 FY(y) = P(Y <y), y € R. The conditional distribution function is defined as FYx=x{y) = P(Y<yX = x), ye R, x G Rd , 4 This definition can be extended to the multivariate case Y = (Yi,... , Y^) by Fy{y) = P(Yi < yu .. ., Yd < yd), y = (yu.. . ,yd) G
- 52. 2 2 OVERVIEW OF REGRESSION AND CLASSIFICATION where Y G R is a scalar random variable and X G Hd is a random vector. We have Fy | x=Ay) = E [/(-ocdQO | X = x] (1.42) and thus the estimation of the conditional distribution function can be considered as a regression problem, where the conditional expectation of the random variable I(-oo,y](Y) is estimated. The random variable /(_00^](F) takes only values 0 or 1. The unconditional distribution function can be estimated with the empirical distribution function, which is defined for the data Y,..., Yn as My) = - E h-oo,y] (Yi) = n~l #{i : < 2/, i = 1,..., n}, (1.43) n z=i where means the cardinality of set A. The conditional distribution function estimation is considered in Section 3.7, where local averaging estimators are defined. Conditional Density Conditional density function is defined as when fx(x) > 0, fvx=x(y) = , n . 1 n otherwise, for y G R, where fx,y : R d + 1 R is thejoint density of (X, Y) and f x : Rd R is the density of X. We mention three ways to estimate the conditional density. First, we can replace the density of (X, Y) and the density of X with their estimators fx,y and f x and define f , x f x , y { x , y ) JYx=x{y) = —p——;—, f x ( x ) for fx(x) > 0. This approach is close to the approach used in Section 3.6, where local averaging estimators of the conditional density are defined. Second, empirical risk minimization can be used in the estimation of the condi- tional density, as explained in Section 5.1.3. Third, sometimes it is reasonable to assume that the conditional density has the form fvX=x(y) = fg(x)(y), (1.44) where fe, 0 G A C Kk , is a family of density functions and g : Hd —» A, where k > 1. Then the estimation of the conditional density reduces to the estimation of the "regression function" g. The mean regression is a special case of this approach when the distribution of errors is known: Assume that Y = f{}0 + e, where e is independent of X, Ee = 0, and the density of e is denoted by fe. Then fyx=x(y) = fe(y - f(x)),
- 53. REGRESSION 2 3 which is a special case of (1.44), when we take fe(y) = fe(y — 0) and g(x) = f(x). The case of heteroskedastic variance is an other example: Now we assume that Y = f ( X ) + a(X)e, where e is independent of X, Ee = 0, and the density of e is denoted by fe. Then fYx=x(y) = ^ r 1 M ( y - f ( x ) ) / a ( x ) ) , which is a special case of (1.44), when we take 0 = (9, 62), fe(y) = — 0 i ) / 0 2 ) , and g(x) = (f(x),a(x)). This approach is used in parametric family regression, explained in Section 1.3.1. 1.1.9 Time Series Data Regression data are a sequence (X,Y{),..., (Xn,Yn) of identically distributed copies of (X, Y), where X E Hd is the explanatory variable and Y E R is the response variable, as we wrote in (1.4). However, we can use regression methods with time series data Z,..., Zt G R, where the observation Zt is made at time t, t = 1,..., T. In order to apply regression methods we identify the response variable and the explanatory variables. We consider two ways for the choice of the explanatory variables. In the first case the state space of the time series is used as the space of the explanatory variables, and in the second case the time space is used as the space of the explanatory variables. State—Space Prediction In the state-space prediction an autoregression param- eter k > 1 is chosen and we denote Yi = Zi+1, Xi = (Zi,..., Zi-k+1), (1-45) i = /C,...,T — 1. When the time series ZI,..., Zt is stationary, then the sequence (.Xi, Yi), i = /C,...,T — 1, consists of identically distributed random variables and we can denote by (X, Y) a random vector which is identically distributed as (Xi, Yi). We define the regression function, as previously, by f(x) = E(Y X = x), xe R* (1.46) We can estimate this regression function using data (Xi, Yi), i = k,...,T — 1. Estimator of the regression function / : Rfc — ^ R can be used to predict or explain the next outcome of the time series using k previous observations. For example, let f r be an estimator of the regression function at time T, constructed using data (Xi,Yi), i = k,.. .,T — 1. The prediction of the next outcome is fr(Xr), where Xt = (ZT,..., Z;R-FC+I). Let Z,..., Zt £ Rd
- 54. 2 4 OVERVIEW OF REGRESSION AND CLASSIFICATION be a c/-dimensional vector time series. Definition (1.45) generalizes to the setting of vector time series. Define - g(Zl+i), = (Zi,..., Z2-k+1), (1.47) i = k,... ,T — 1, where g : Hd — » R is a function with real values. We define the regression function, as previously, by f ( x ) = E(YiXi = x), xeKdk . The regression function is now defined on the higher-dimensional space of dimension kd. We can predict and explain without autoregression parameter k and take into account all the previous observations and not just the k last observations. However, this approach does not fit into the standard regression approach. Let Z,..., Zt G R be a scalar time series and define Y{ = Z i + i , X i = ( Z i , . . . , Z), i — ,... ,T — 1. The sequence of observations (Y, X i ) , . . . , (Yr-i> X T - I ) is not a sequence of identically distributed random vectors. For example, the regression function fi(x) = E(Yi | Xi = x), x G Hld , is defined in a different space for each i. Time—Space Prediction In time-space prediction the time parameter is taken as the explanatory variable, in contrast to (1.45), where the previous observations in the time series are taken as the explanatory variables. We denote = Xi=i, i = 1,... ,T. (1.48) The obtained regression model is a fixed design regression model, as described in Section 1.1.1. Time-space prediction can be used when the time series can be modeled as a nonstationary time series of signal with additive noise: Yi = & + (nei, i = 1,... ,T, (1.49) where ^ G R is the deterministic signal, <Ji > 0 are nonrandom values, and the noise ^ is stationary with mean zero and unit variance. For statistical estimation and asymptotic analysis we can use a slightly different model YZjT = + <t(U,t) ei,r, i = 1,... ,T, (1.50) where t^T — i/T, /i : [0,1] — > R, a : [0,1] (0, oo), and e^r is stationary with mean zero and unit variance. Now it can be thought that the observations are coming from a continuous time process Y(t), t G [0,1], and the sampled discrete time process is obtained as Y^T = Y(i/T), i = 1,..., T. The asymptotics as T — > • oo is called in-fill asymptotics, because points t^T are filling the interval [0,1] as T —> oo.
- 55. Another Random Document on Scribd Without Any Related Topics
- 56. and of violence, by which this divine sweetness is tainted, and this peace broken by suspicion, by hatred, and heat of blood." "The book says somewhere," said the Duke, turning over the leaves, "that, as the penitent thief rose from the cross to Paradise, so we, if we long after Christ with all the powers of our souls, shall, at the hour of death, rapidly soar aloft from our mortal remains, and then all fear of returning to earth and earthly desires will be at an end." "It must surely," said Inglesant after a pause, speaking more to himself than to the Duke, "be among the things most surprising to an angelic nature that observes mankind, that, shadows ourselves, standing upon the confines even of this shadowy land, and not knowing what, if aught, awaits us elsewhere, hatred or revenge or unkindness should be among the last passions that are overcome. When the veil is lifted, and we see things as they really are, nothing will so much amaze us as the blindness and perversity that marked our life among our fellow-men. Surely the lofty life is hard, as it seems hard to your Grace; but the very effort itself is gain." Inglesant left the presence of the Duke after his first interview impressed and softened, but troubled in his mind more than ever at the nature of the mission on which he was sent. Now that he had seen the Duke, and had been touched by his eager questions, and by the earnest searching look in the worn face, his conscience smote him at the thought of abusing his confidence, and of persuading him to adopt a course which Inglesant's own heart warned him might not in the end be conducive either to his own peace or to the welfare of his people, whose happiness he sincerely sought. He found that, in the antechambers and reception rooms of the palace, and even at
- 57. the Duke's own table, the principal subject of conversation was the expected cession of the dukedom to the Papal See; and that emissaries from Rome had preceded him, and had evidently received instructions announcing his arrival, and were prepared to welcome him as an important ally. On the other hand, there were not wanting those who openly or covertly opposed the cession, some of whom were said to be agents of the Grand Duke of Florence, who was heir to the Duchy of Umbria through his wife. These latter, whose opposition was more secret than open, sought every opportunity of winning Inglesant to their party, employing the usual arguments with which, since his coming into Italy, he had been so familiar. Many days passed in this manner, and Inglesant had repeated conferences with the Duke, during which he made great progress in his favour, and was himself won by his lofty, kindly, and trustful character. He had resided at Umbria a little less than a month, when he received instructions by a courier from Rome, by which he was informed that at the approaching festival of the Ascension a determined effort was to be made by the agents and friends of the Pope to bring the business to a conclusion. The Duke had promised to keep this festival, which is celebrated at Venice and in other parts of Italy with great solemnity, with unusual magnificence; and it was hoped that while his feelings were influenced and his religious instincts excited by the solemn and tender thoughts and imaginations which gather round the figure of the ascending Son of man, he might be induced to sign the deed of cession. Hitherto the Duke had not mentioned the subject to Inglesant, having found his conversation upon questions of the spiritual life and practice sufficient to occupy the time; but it was not probable that this
- 58. silence would continue much longer, and on the first day in Ascension week Inglesant was attending Vespers at one of the Churches in the town in considerable anxiety and trouble of mind. The sun had hardly set, and the fête in the garden was not yet begun, when, Vespers being over, he came out upon the river-side lined with stately houses which fronted the palace gardens towering in terraced walks and trellises of green hedges on the opposite bank. The sun, setting behind the wooded slopes, flooded this green hill- side with soft and dream-like light, and bathed the carved marble façade of the palace, rising above it with a rosy glimmer, in which the statues on its roof, and the fretted work of its balustrades, rested against the darkening blue of the evening sky. A reflex light, ethereal and wonderful, coming from the sky behind him, and the marble buildings and towers on which the sun's rays rested more fully than they did upon the palace, brooded over the river and the bridge with its rows of angelic forms, and, climbing the leafy slopes, as if to contrast its softer splendour with the light above, transfigured with colour the wreaths of vapour which rose from the river and hung about its wharves. The people were already crowding out of the city, and forcing their way across the bridge towards the palace, where the illuminations and the curious waterworks, upon which the young Duke had, during his short reign, expended much money, were to be exhibited as soon as the evening was sufficiently dark. The people were noisy and jostling, but as usual good-tempered and easily pleased. Few masques or masquerade dresses had appeared as yet, but almost every one was armed with a small trumpet, a drum, or a Samarcand cane, from which to shoot peas or comfits. At the corner
- 59. of the main street that opened on to the quay, however, some disturbing cause was evidently at work. The crowd was perplexed by two contending currents, the one consisting of those who were attempting to turn into the street from the wharf, in order to learn the cause of the confusion, the other, of those who were apparently being driven forcibly out of the street, towards the wharves and the bridge, by pressure from behind. Discordant cries and exclamations of anger and contempt rose above the struggling mass. Taking advantage of the current that swept him onward, Inglesant reached the steps of the Church of St. Felix, which stood at the corner of the two streets, immediately opposite the bridge and the ducal lions which flanked the approach. On reaching this commanding situation the cause of the tumult presented itself in the form of a small group of men, who were apparently dragging a prisoner with them, and had at this moment reached the corner of the wharf, not far from the steps of the Church, surrounded and urged on by a leaping, shouting, and excited crowd. Seen from the top of the broad marble bases that flanked the steps, the whole of the wide space, formed by the confluence of the streets, and over which the shadows were rapidly darkening, presented nothing but a sea of agitated and tossing heads, while, from the windows, the bridge, and even the distant marble terraced steps that led up to the palace, the crowd appeared curious, and conscious that something unusual was in progress. From the cries and aspect of the crowd, and of the men who dragged their prisoner along, it was evident that it was the intention of the people to throw the wretched man over the parapets of the bridge into the river below, and that to frustrate this intention not a
- 60. moment was to be lost. The pressure of the crowd, greater from the opposite direction than from the one in which Inglesant had come, fortunately swept the group almost to the foot of the steps. Near to Inglesant, and clinging to the carved bases of the half-columns that supported the façade of the Church, were two or three priests who had come out of the interior, attracted by the tumult. Availing himself of their support, Inglesant shouted to the captors of the unhappy man, in the name of the Church and of the Duke, to bring their prisoner up the steps. They probably would not have obeyed him, though they hesitated for a moment; but the surrounding crowd, attracted towards the Church by Inglesant's gestures, began to press upon it from all sides, as he had indeed foreseen would be the case, and finally, by their unconscious and involuntary motion, swept the prisoner and his captors up the steps to the side of the priests and of Inglesant. It was a singular scene. The rapidly advancing night had changed the golden haze of sunset to a sombre gloom, but lights began to appear in the houses all around, and paper lanterns showed themselves among the crowd. The cause of all this confusion was dragged by his persecutors up the steps, and placed upon the last of the flight, confronting the priests. His hair was disordered, his clothes nearly torn from his limbs, and his face and dress streaked with blood. Past the curtain across the entrance of the Church, which was partly drawn back by those inside, a flash of light shot across the marble platform, and shone upon the faces of the foremost of the crowd. This light shone full upon Inglesant, who stood, in striking contrast to the dishevelled figure that confronted him, dressed in a suit of black satin and silver, with a deep collar of Point-de-Venice lace. The priests stood a little
- 61. behind, apparently desirous to learn the nature of the prisoner's offence before they interfered; and the accusers therefore addressed themselves to Inglesant, who, indeed, was recognized by many as a friend of the Duke, and whom the priests especially had received instructions from Rome to support. The confusion in the crowd meanwhile increased rather than diminished; there seemed to be causes at work other than the slight one of the seizure by the mob of an unpopular man. The town was very full of strangers, and it struck Inglesant that the arrest of the man before him was merely an excuse, and was being used by some who had an object to gain by stirring up the people. He saw, at any rate, however this might be, a means of engaging the priests to assist him, should their aid be necessary in saving the man's life. That there was a passionate attachment among the people to a separate and independent government of their city and state, an affection towards the family of their hereditary dukes, and a dread and jealous dislike of the Pope's government and of the priests, he had reason to believe. It seemed to him that the people were about to break forth into some demonstration of this antipathy, which, if allowed to take place, and if taken advantage of, as it would be, by the neighbouring princes, would be most displeasing to the policy of Rome, if not entirely subversive of it. With these thoughts in his mind, as he stood for a moment silent on the marble platform, and saw before him, what is perhaps the most impressive of all sights, a vast assemblage of people in a state of violent and excited opposition, and reflected on the causes which he imagined agitated them,—causes which in his heart he, though enlisted on the opposite side, had difficulty in persuading himself were not
- 62. justifiable,—it came into his mind more powerfully than ever, that the moment foretold to him by Serenus de Cressy was at last indeed come. Surely it behoved him to look well to his steps, lest he should be found at last absolutely and unequivocally fighting against his conscience and his God; if, indeed, this looking well to their steps on such occasions, and not boldly choosing their side, had not been for many years the prevailing vice of his family, and to some extent the cause of his own spiritual failure. The two men who held the apparent cause of all this uproar were two mechanics of jovial aspect, who appeared to look upon the affair more in the light of a brutal practical joke (no worse in their eyes for its brutality), than as a very serious matter. To Inglesant's question what the man had done they answered that he had refused to kneel to the Blessed Sacrament, as it was being carried through the streets to some poor, dying soul, and upon being remonstrated with, had reviled not only the Sacrament itself, but the Virgin, the Holy Father, and the Italians generally, as Papistical asses, with no more sense than the Pantaleoni of their own comedies. The men gave this evidence in an insolent half-jesting manner, as though not sorry to utter such words safely in the presence of the priests. Inglesant, who kept his eyes fixed upon the prisoner, and noticed that he was rapidly recovering from the breathless and exhausted condition the ill-treatment he had met with had reduced him to, and was assuming a determined and somewhat noble aspect, abstained from questioning him, lest he should make his own case only the more desperate; but, turning to the priests, he rapidly explained his fears to them, and urged that the man should be immediately secured from the people, that he might be examined
- 63. by the Duke, and the result forwarded to Rome. The priests hesitated. Apart from the difficulty, they said, of taking the man out of the hands of his captors, such a course would be sure to exasperate the people still further, and bring on the very evil that he was desirous of averting. It would be better to let the mob work their will upon the man; it would at least occupy some time, and every moment was precious. In less than an hour the fireworks at the palace would begin, might indeed be hastened by a special messenger; and the fête once begun, they hoped all danger would be over. To this Inglesant answered that the man's arrest was evidently only an excuse for riot, and had probably already answered its purpose; that to confine the people's attention to it would be unfavourable to the intentions of those who were promoting a political tumult; and that the avowed cause of the man's seizure, and of the excitement of the mob, being disrespectful language towards the Holy Father, the tumult, if properly managed, might be made of service to the cause of Rome rather than the reverse. Without waiting for the effect of this somewhat obscure argument on the priests, Inglesant directed the men who held their prisoner to bring him into the Church. They were unwilling to do so, but the crowd below was so confused and tumultuous, one shouting one thing and one another, that it seemed impossible that, if they descended into it again, they would be allowed to retain their prey, and would not rather be overwhelmed in a common destruction with him. On the other hand, by obeying Inglesant, they at least kept possession of their prisoner, and could therefore scarcely fail of receiving some reward from the authorities. They therefore consented, and by a sudden movement they entered the Church, the
- 64. doors of which were immediately closed, after some few of the populace had managed to squeeze themselves in. A messenger was at once despatched to the palace to hasten the fireworks, and to request that a detachment of the Duke's guard should be sent into the Church by a back way. The darkness had by this time so much increased that few of the people were aware of what had taken place, and the ignorance of the crowd as to the cause of the tumult was so general that little disturbance took place among those who were shut out of the Church. They remained howling and hooting, it is true, for some time, and some went so far as to beat against the closed doors; but a rumour being spread among the crowd that the fireworks were immediately to begin, they grew tired of this unproductive occupation, and flocked almost to a man out of the square and wharves, and crowded across the bridge into the gardens. When the guard arrived, Inglesant claimed the man as the Duke's prisoner, to be examined before him in the morning. The curiosity of the Duke in all religious matters being well known, this seemed very reasonable to the officer of the guard, and the priests did not like to dispute it after the instructions they had received with regard to Inglesant's mission. The two artisans were propitiated by a considerable reward, and the prisoner was then transported by unfrequented ways to the palace, and shut up in a solitary apartment, whilst the rest of the world delighted itself at the palace fêtes. The garden festivities passed away amid general rejoicing and applause. The finest effect was produced at the conclusion, when the whole mass of water at the command of the engines, being
- 65. thrown into the air in thin fan-like jets, was illuminated by various coloured lights, producing the appearance of innumerable rainbows, through which the palace itself, the orangeries, the gardens, and terraces, and the crowds of delighted people, were seen illuminated and refracted in varied and ever-changing tints. Amid these sparkling colours strange birds passed to and fro, and angelic forms descended by unseen machinery and walked on the higher terraces, and as it were upon the flashing rainbows themselves. Delicious music from unseen instruments ravished the sense, and when the scene appeared complete and nothing further was expected, an orange grove in the centre of the whole apparently burst open, and displayed the stage of a theatre, upon which antic characters performed a pantomime, and one of the finest voices in Italy sang an ode in honour of the day, of the Duke, and of the Pope. CHAPTER VII. The Duke had engaged the next morning to be present at a theatrical representation of a religious character, somewhat of the nature of a miracle play, to be given in the courtyard of the "Hospital of Death," which adjoined to the Campo Santo of the city. Before accompanying his Highness, Inglesant had given orders to have the man, who had been the cause of so much excitement the evening before, brought into his apartment, that he might see whether or no his eccentricity made him sufficiently interesting to be presented to the Duke.
- 66. When the stranger was brought to the palace early in the morning, and having been found to be quite harmless, was entrusted by the guard to two servants to be brought into Inglesant's presence, he thought himself in a new world. Hitherto his acquaintance with Italian life had been that of a stranger and from the outside; he was now to see somewhat of the interior life of a people among whom the glories of the Renaissance still lingered, and to see it in one of the most wonderful of the Renaissance works, the ducal palace of Umbria. Born in the dull twilight of the north, and having spent most of his mature years amongst the green mezzotints of Germany, he was now transplanted into a land of light and colour, dazzling to a stranger so brought up. Reared in the sternest discipline, he found himself among a people to whom life was a fine art, and the cultivation of the present and its enjoyments the end of existence. From room to room, as he followed his guide, who pointed out from time to time such of the beauties of the place as he considered most worthy of notice, the stranger saw around what certainly might have intoxicated a less composed and determined brain. The highest efforts of the genius of the Renaissance had been expended upon this magnificent house. The birth of a new instinct, differing in some respects from any instincts of art which had preceded it, produced in this and other similar efforts original and wonderful results. The old Greek art entered with unsurpassable intensity into sympathy with human life; but it was of necessity original and creative, looking always forward and not back, and lacked the pathos and depth of feeling that accompanied that new birth of art which sought much of its inspiration among the tombs
- 67. and ruined grottoes, and most of its sympathetic power among the old well-springs of human feeling, read in the torn and faded memorials of past suffering and destruction. This new instinct of art abandoned itself without reserve to the pursuit of everything which mankind had ever beheld of the beautiful, or had felt of the pathetic or the sad, or had dreamed of the noble or the ideal. The genius of the Renaissance set itself to reproduce this enchanted world of form and colour, traversed by thoughts and spiritual existences mysterious and beautiful, and the home of beings who had found this form and colour and these mysterious thoughts blend into a human life delicious in its very sorrows, grotesque and incongruous in its beauty, alluring and attractive amid all its griefs and hardships; so much so indeed that, in the language of the old fables, the Gods themselves could not be restrained from throwing off their divine garments, and wandering up and down among the paths and the adventures of men. By grotesque and humorous delineation, by fanciful representation of human passion under strange and unexpected form, by the dumb ass speaking and grasshoppers playing upon flutes, was this world of intelligent life reproduced in the rooms and on the walls of the house through which the stranger walked for the first time. He probably thought that he saw little of it, yet the bizarre effect was burning itself into his brain. From the overhanging chimney-pieces antique masques and figures such as he had never seen, even in dreams, leered out upon him from arabesque carvings of foliage, or skulked behind trophies of war, of music, or of the arts of peace. The door and window frames seemed bowers of fruit and flowers, and forests of carved leaves wreathed the pilasters and
- 68. walls. But this was not all; with a perfection of design and an extraordinary power of fancy, this world of sylvan imagery was peopled by figures and stories of exquisite grace and sweetness, representing the most touching incidents of human life and history. Men and women; lovers and warriors in conflicts and dances and festivals, in sacrifices and games; children sporting among flowers; bereavement and death, husbandry and handicraft, hunters and beasts of chase. Again, among briony and jasmin and roses, or perched upon ears of corn and sheaves of maize, birds of every plumage confronted—so the grotesque genius willed—fish and sea monsters and shells and marine wonders of every kind. Upon the walls, relieved by panelling of wood, were paintings of landscapes and the ruined buildings of antiquity overgrown with moss, or of modern active life in markets and theatres, of churches and cities in the course of erection with the architects and scaffold poles, of the processions and marriages of princes, of the ruin of emperors and of kings. Below and beside these were credenzas and cabinets upon which luxury and art had lavished every costly device and material which the world conceived or yielded. Inlaid with precious woods, and glittering with costly jewels and marbles, they reproduced in these differing materials all those infinite designs which the carved walls had already wearied themselves to express. Plaques and vases from Castel Durante or Faience,—some of a strange pale colour, others brilliant with a grotesque combination of blue and yellow,—crowded the shelves. Passing through this long succession of rooms, the stranger reached at last a library, a noble apartment of great size, furnished with books in brilliant antique binding of gold and white vellum, and
- 69. otherwise ornamented with as much richness as the rest of the palace. Upon reading desks were open manuscripts and printed books richly illuminated. Connected with this apartment by open arches, was an anteroom or corridor, which again opened on a loggia, beyond the shady arches of which lay the palace gardens, long vistas of green walks, and reaches of blue sky, flecked and crossed by the spray of fountains. The decorations of the anteroom and loggia were more profuse and extravagant than any that the stranger had yet seen. There was a tradition that this portion of the palace had been finished last, and that when the workmen arrived at it the time for the completion of the whole was very nearly run out. The attention of all the great artists, hitherto engaged upon different parts of the entire palace, was concentrated upon this unfinished portion, and all their workmen and assistants were called to labour upon it alone. The work went on by night and day, not ceasing even to allow of sleep. Unlimited supplies of Greek wine were furnished to the workmen; and stimulated by excitement and the love of art, emulating each other, and half-intoxicated by the delicious wine, the work exceeded all previous productions. For wild boldness and luxuriance of fancy these rooms were probably unequalled in the world. In the anteroom facing the loggia the stranger found Inglesant conversing with an Italian who held rather a singular post in the ducal Court. He was standing before a cabinet of black oak, inlaid with representations of lutes and fifes, over which were strewn roses confined by coloured ribbons, and supporting vases of blue and yellow majolica, thrown into strong relief by the black wood. Above this cabinet was a painting representing some battle in which a
- 70. former Duke had won great honour; while on a grassy knoll in the foreground the huntsmen of Ganymede were standing with their eyes turned upward towards the bird of Zeus, who is carrying the youth away to the skies, emblematical of the alleged apotheosis of the ducal hero. Richly dressed in a fantastic suit of striped silk, and leaning against the cabinet in an attitude of listless repose, Inglesant was contemplating an object which he held in his hand, and which both he and his companion appeared to regard with intense interest. This was an antique statuette of a faun, holding its tail in its left hand, and turning its head and body to look at it,—an occupation of which, if we may trust the monuments of antiquity, this singular creature appears to have been fond. The Italian was of a striking figure, and was dressed somewhat more gaily than was customary with his countrymen; and the whole group was fully in unison with the spirit of the place and with the wealth of beauty and luxury of human life that pervaded the whole. The man who was standing by Inglesant's side, and who had the air of a connoisseur or virtuoso, was an Italian of some fifty years of age. His appearance, as has been said, was striking at first sight, but on longer acquaintance became very much more so. He was tall and had been dark, but his hair and beard were plentifully streaked with gray. His features were large and aquiline, and his face deeply furrowed and lined. His appearance would have been painfully worn, almost to ghastliness, but for a mocking and humorous expression which laughed from his eyes, his mouth, his nostrils, and every line and feature of his face. Whenever this expression subsided, and his countenance sank into repose, a look of wan sadness and even terror took its place, and the large black
- 71. eyes became fixed and intense in their gaze, as though some appalling object attracted their regard. This man had been born of a good but poor family, and had been educated by his relations with the expectation of his becoming an ecclesiastic, and he had even passed some time as a novice of some religious order. The tendency of his mind not leading him to the further pursuit of a religious life, he left his monastery, and addressed himself to live by his wits, among the families and households of princes. He had made himself very useful in arranging comedies and pageantries, and he had at one time belonged to one of those dramatic companies called "Zanni," who went about the country reciting and acting comedies. Combined with this talent he discovered great aptitude in the management of serious affairs, and was more than once, while apparently engaged entirely on theatrical performances, employed in secret State negotiations which could not so well be entrusted to an acknowledged and conspicuous agent. In this manner of life he might have continued; but having become involved in one of the contests which disturbed Italy, he received a dangerous wound in the head, and on rising from his sick bed in the Albergo in which he had been nursed, he was merely removed to another as a singular if not dangerous lunatic. The symptoms of his disease first manifested themselves in a very unpleasant familiarity with the secrets of those around him, and it was probably this feature of his complaint which led to his detention. As he improved in health, however, he ceased to indulge in any conversation which might give offence, but, assuming a sedate and agreeable manner, he conversed with all who came to him, calling them, although strangers and such as he had never before seen, by their proper
- 72. names, and talking to them pleasantly concerning their parents, relations, the coats-of-arms of their families, and such other harmless and agreeable matters. What brought him prominently into notice was the strangely prophetic spirit he manifested before, or at the moment of the occurrence of, more than one public event. He was taken from the hospital and examined by the Pope, and afterwards at several of the sovereign Courts of Italy. Thus, not long before the time when Inglesant met him in the ducal palace at Umbria, he was at Chambery assisting at the preparation of some festivals which the young Duke of Savoy was engaged in celebrating. One day, as he was seated at dinner with several of the Duke's servants, he suddenly started up from his seat, exclaiming that he saw the Duke de Nemours fall dead from his horse, killed by a pistol shot. The Duke, who was uncle to the young monarch of Savoy, was then in France, where he was one of the leaders of the party of the Fronde. Before many days were passed, however, the news reached Chambery of the fatal duel between this nobleman and the Duke of Beaufort, which occurred at the moment the Italian had thus announced it. These and other similar circumstances caused the man to be much talked of and sought after among the courts of Italy, where a belief in manifestations of the supernatural was scarcely less universal than in the previous age, when, according to an eye- witness, "the Pope would decide no question, would take no journey, hold no sitting of the Consistory, without first consulting the stars; nay, very few cardinals would transact an affair of any kind, were it but to buy a load of wood, except after consultation duly held with
- 73. some astrologer or wizard." The credit which the man gained, and the benefits he derived from this reputation, raised him many enemies, who did not scruple to assert that he was simply a clever knave, who was not even his own dupe. Setting on one side, however, the revelations of the distant and the unknown made by him, which seemed inexplicable except by supposing him possessed of some unusual spiritual faculty, there was in the man an amount of knowledge of the world and of men of all classes and ranks, combined with much learning and a humorous wit, which made his company well worth having for his conversation alone. It was not then surprising that he should be found at this juncture at the court of Umbria, where the peculiar idiosyncrasies of the aged Duke, and the interest attached to the intrigue for the session of the dukedom, had assembled a strange and heterogeneous company, and towards which at the moment all men's eyes in Italy were turned. "Yes, doubtless, it is an antique," the Italian was saying, "though in the last age many artists produced masques and figures so admirable as to be mistaken for antiques; witness that masque which Messire Georgio Vassari says he put in a chimney-piece of his house at Arezzo, which every one took to be an antique. I have seen such myself. This little fellow, however, I saw found in a vineyard near the Miserecordia—a place which I take to have been at some time or other the scene of some terrible event, such as a conflict or struggle or massacre; for though now it is quiet and serene enough, with the sunlight and the rustling leaves, and the splash of a fountain about which there is some good carving, I think of Fra Giovanni Agnolo,—for all this, I never walk there but I feel the
- 74. presence of fatal events, and a sense of dim figures engaged in conflict, and of faint and distant cries and groans." As he spoke these last words his eye rested upon the strange figure of the man so hardly rescued from death the night before, and he stopped. His manner changed, and his eyes assumed that expression of intense expectation of which we have spoken before. The appearance of the stranger, and the contrast it presented to the objects around, was indeed such as to make him almost seem an inhabitant of another world, and one of those phantasms of past conflict of which the Italian had just spoken. His clothes, which had originally been of the plainest texture, and most uncourtly make, were worn and ragged, and stained with damp and dirt. His form and features were gaunt and uncouth, and his gesture stiff and awkward; but, with all this, there was a certain steadiness and dignity about his manner, which threw an appearance of nobility over this rugged and unpleasing form. Contrasted with the dress and manner of the other men, he looked like some enthusiastic prophet, standing in the house of mirth and luxury, and predicting ruin and woe. At this moment a servant entered the room, bringing a sottocoppa of silver, upon which were two or three stiff necked glasses, called caraffas, containing different sorts of wine, and also water, and one or two more empty drinking-glasses, so that the visitor could please himself as to the strength and nature of his beverage. Inglesant offered this refreshment to the Italian, who filled himself a glass and drank, pledging Inglesant as he did so. The latter did not drink, but offered wine and cakes to the stranger, who refused or rather took no heed of these offers of politeness; he
- 75. remained silent, keeping his eyes fixed upon the face of the man who, but a few hours before, had saved him from a violent death. "I have had some feelings of this kind myself, in certain places," said Inglesant, in answer to the Italian's speech, "and very frequently in all places the sense of something vanishing, which in another moment I should have seen; it has seemed to me that, could I once see this thing, matters would be very different with me. Whether I ever shall or not I do not know." "Who can say?" replied the other. "We live and move amid a crowd of flitting objects unknown or dimly seen. The beings and powers of the unseen world throng around us. We call ourselves lords of our own actions and fate, but we are in reality the slaves of every atom of matter of which the world is made and we ourselves created. Among this phantasm of struggling forms and influences (like a man forcing his way through a crowd of masques who mock at him and retard his steps) we fight our way towards the light. Many of us are born with the seeds within us of that which makes such a fight hopeless from the first—the seeds of disease, of ignorance, of adverse circumstance, of stupidity; for even a dullard has had once or twice in his life glimpses of the light. So we go on. I was at Chambery once when a man came before the Duke in the palace garden to ask an alms. He was a worker in gold, a good artist, not unworthy of Cellini himself. His sight had failed him, and he could no longer work for bread to give to his children. He stood before the Prince and those who stood with him, among whom were a Cardinal and two or three nobles, with their pages and grooms, trying with his dim eyes to make out one from the other, which was noble and which was groom, and to see whether his suit was
- 76. rejected or allowed. Behind him, beyond the garden shade, the dazzling glitter stretched up to the white Alps. We are all the creatures of a day, and the puny afflictions of any man's life are not worth a serious thought; yet this man seemed to me so true an image of his kind, helpless and half-blind, yet struggling to work out some good for himself, that I felt a strange emotion of pity. They gave him alms—some more, some less. I was a fool, yet even now I think the man was no bad emblem of the life of each of us. We do not understand this enough. Will the time ever come when these things will be better known?" As the Italian spoke the stranger took his eyes off Inglesant and fixed them on the speaker with a startled expression, as though the tone of his discourse was unexpected to him. He scarcely waited for the other to finish before he broke in upon the conversation, speaking slowly and with intense earnestness, as though above all things desirous of being understood. He spoke a strange and uncouth Italian, full of rough northern idioms, yet the earnestness and dignity of his manner ensured him an audience, especially with two such men as those who stood before him. "Standing in a new world," he said, "and speaking as I speak, to men of another language, and of thoughts and habits distinct from mine, I see beneath the tinsel of earthly rank and splendour, and a luxury of life and of beauty, the very meaning of which is unknown to me, something of a common feeling, which assures me that the voice I utter will not be entirely strange, coming as it does from the common Father. I see around me a land given over to idolatry and sensual crime, as if the old Pagans were returned again to earth; and here around me I see the symbols of the Pagan worship and of
- 77. the Pagan sin, and I hear no other talk than that which would have befitted the Pagan revels and the Pagan darkness which overhung the world to come. Standing on the brink of a violent death, and able to utter few words that can be understood, I call, in these short moments which are given me, and in these few words which I have at command—I call upon all who will listen to me, that they leave those things which are behind, with all the filthy recollections of ages steeped in sin, and that they press forward towards the light,— the light of God in Jesus Christ." He stopped, probably for want of words to clothe his thoughts, and Inglesant replied,— "You may be assured from the events of last night, signore, that you are in no danger of violent death in this house, and that every means will be taken to protect you, until you have been found guilty of some crime. You must, however, know that no country can allow its customs and its religion to be outraged by strangers and aliens, and you cannot be surprised if such conduct is resented both by the governors of the country and by the ignorant populace, though these act from different motives. As to what you have said respecting the ornaments and symbols of this house, and of the converse in which you have found us engaged, it would seem that to a wise man these things might serve as an allegory, or at least as an image and representation of human life, and be, therefore, not without their uses." "I desire no representation nor image of a past world of iniquity," said the stranger, "I would I could say of a dead life, but the whole world lieth in wickedness until this day. This is why I travel through all lands, crying to all men that they repent and escape the
- 78. most righteous judgment of God, if haply there be yet time. These are those latter days in which our Saviour and Redeemer Jesus Christ, the Son of God, predicted that iniquity 'should be increased;' wherein, instead of serving God, all serve their own humours and affections, being rocked to sleep with the false and deceitful lullaby of effeminate pleasures and delights of the flesh, and know not that an horrible mischief and overthrow is awaiting them, that the pit of Hell yawns beneath them, and that for them is reserved the inevitable rigour of the eternal fire. Is it a time for chambering and wantonness, for soft raiment and dainty living, for reading of old play-books such as the one I see on the table, for building houses of cedar, painted with vermilion, and decked with all the loose and fantastic devices which a disordered and debauched intellect could itself conceive, or could borrow from Pagan tombs and haunts of devils, full of uncleanness and dead sins?" "You speak too harshly of these things," said Inglesant. "I see nothing in them but the instinct of humanity, differing in its outward aspect in different ages, but alike in its meaning and audible voice. This house is in itself a representation of the world of fancy and reality combined, of the material life of the animal mingled with those half-seen and fitful glimpses of the unknown life upon the verge of which we stand. This little fellow which I hold in my hand, speaks to me, in an indistinct and yet forcible voice, of that common sympathy—magical and hidden though it may be—by which the whole creation is linked together, and in which, as is taught in many an allegory and quaint device upon these walls, the Creator of us all has a kindly feeling for the basest and most inanimate. My imagination follows humanity through all the paths by which it has
- 79. reached the present moment, and the more memorials I can gather of its devious footsteps the more enlarged my view becomes of what its trials, its struggles, and its virtues were. All things that ever delighted it were in themselves the good blessings of God—the painter's and the player's art—action, apparel, agility, music. Without these life would be a desert; and as it seems to me, these things softened manners so as to allow Religion to be heard, who otherwise would not have been listened to in a savage world, and among a brutal people destitute of civility. As I trace these things backward for centuries, I live far beyond my natural term, and my mind is delighted with the pleasures of nations who were dust ages before I was born." "I am not concerned to dispute the vain pleasures of the children of this world," exclaimed the stranger with more warmth than he had hitherto shown. "Do you suppose that I myself am without the lusts and desires of life? Have I no eyes like other men, that I cannot take a carnal pleasure in that which is cunningly formed by the enemy to please the eye? Am not I warmed like other men? And is not soft clothing and dainty fare pleasing to me as to them? But I call on all men to rise above these things, which are transitory and visionary as a dream, and which you yourself have spoken of as magical and hidden, of which only fitful glimpses are obtained. You are pleasing yourself with fond and idle imaginations, the product of delicate living and unrestrained fancies; but in this the net of the devil is about your feet, and before you are aware you will find yourself ensnared for ever. These things are slowly but surely poisoning your spiritual life. I call upon you to leave these delusions, and come out into the clear atmosphere of God's truth; to tread the
- 80. life of painful self-denial, leaving that of the powerful and great of this world, and following a despised Saviour, who knew none of these things, and spent His time not in kings' houses gorgeously tricked out, but knew not where to lay His head. You speak to me of pleasures of the mind, of music, of the painter's art; do you think that last night, when beaten, crushed, and almost breathless, in the midst of a blood-thirsty and howling crowd, I was dimly conscious of help, and looking up I saw you in the glare of the lanterns, in your courtier's dress of lace and silver, calm, beneficent, powerful for good, you did not seem to my weak human nature, and my low needs and instincts, beautiful as an angel of light? Truly you did; yet I tell you, speaking by a nature and in a voice that is more unerring than mine, that, to the divine vision, of us two at that moment you were the one to be pitied,—you were the outcast, the tortured of demons, the bound hand and foot, whose portion is in this life, who, if this fleeting hour is left unheeded, will be tormented in the life to come." The Italian turned away his head to conceal a smile, and even to Inglesant, who was much better able to understand the man's meaning, this result of his interference to save his life appeared somewhat ludicrous. The Italian, however, probably thinking that Inglesant would be glad to be relieved from his strange visitor, seemed desirous of terminating the interview. "His Grace expects me," he said to Inglesant, "at the Casa di Morte this morning, and it is near the time for him to be there. I will therefore take my leave." "Ah! the Casa di Morte; yes, he will expect me there also," said Inglesant, with some slight appearance of reluctance. "I will follow
- 81. you anon." He moved from the indolent attitude he had kept till this moment before the sideboard, and exchanged with the Italian those formal gestures of leave-taking and politeness in which his nation were precise. When the Italian was gone Inglesant summoned a servant, and directed him to provide the stranger with an apartment, and to see that he wanted for nothing. He then turned to the fanatic, and requested him as a favour not to attempt to leave the palace until he had returned from the Duke. The stranger hesitated, but finally consented. "I owe you my life," he said,—"a life I value not at a straw's weight, but for which my Master may perchance have some use even yet. I am therefore in your debt, and I will give my word to remain quiet until you return; but this promise only extends to nightfall; should you be prevented by any chance from returning this day, I am free from my parole." Inglesant bowed. "I would," continued the man, looking upon his companion with a softened and even compassionate regard, "I would I could say more. I hear a secret voice, which tells me that you are even now walking in slippery places, and that your heart is not at ease." He stopped, and seemed to seek earnestly for some phrases or arguments which he might suppose likely to influence a courtier placed as he imagined Inglesant to be; but before he resumed, the latter excused himself on the ground of his attendance on the Duke, and, promising to see him again on his return, left the room. Inglesant found a carriage waiting to convey him to the "Hospital of Death," as the monastic house adjoining the public
- 82. Campo Santo was called. The religious performance had already begun. Passing through several sombre corridors and across a courtyard, he was ushered into the Duke's presence, who sat, surrounded by his Court and by the principal ecclesiastics of the city, in an open balcony or loggia. As Inglesant entered by a small door in the back of the gallery a most extraordinary sight met his eyes. Beyond the loggia was a small yard or burial-ground, and beyond this the Campo Santo stretching out into the far country. The whole of the yard immediately before the spectators was thronged by a multitude of persons, of all ages and ranks, apparently just risen from the tomb. Many were utterly without clothing, others were attired as kings, bishops, and even popes. Their attitudes and conduct corresponded with the characters in which they appeared, the ecclesiastics collecting in calm and sedate attitudes, while many of the rest, among whom kings and great men were not wanting, appeared in an extremity of anguish and fear. Beyond the sheltering walls which enclosed the court the dazzling heat brooded over the Campo Santo to the distant hills, and the funereal trees stood, black and sombre, against the glare of the yellow sky. At the moment of Inglesant's entrance it appeared that something had taken place of the nature of an excommunication, and the ecclesiastics in the gallery were, according to custom, casting candles and flaming torches, which the crowd of nude figures below were struggling and fighting to obtain. A wild yet solemn strain of music, that came apparently from the open graves, ascended through the fitful and half-stifled cries. The first sight that struck upon Inglesant's sense, as he entered the gallery from the dark corridors, was the lurid yellow light
- 83. beyond. The second was the wild confused crowd of leaping and struggling figures, in a strange and ghastly disarray, naked or decked as in mockery with the torn and disordered symbols of rank and wealth, rising as from the tomb, distracted and terror-stricken as at the last great assize. The third was the figure of the Duke turning to him, and the eyes of the priests and clergy fixed upon his face. The words that the fanatic had uttered had fallen upon a mind prepared to receive them, and upon a conscience already awakened to acknowledge their truth. A mysterious conviction laid hold upon his imagination that the moment had arrived in which he was bound to declare himself, and by every tie which the past had knotted round him to influence the Duke to pursue a line of conduct from which his conscience and his better judgment revolted. On the one hand, a half-aroused and uncertain conscience, on the other, circumstance, habit, interest, inclination, perplexed his thoughts. The conflict was uneven, the result hardly doubtful. The eyes of friends and enemies, of agents of the Holy See, of courtiers and priests, were upon him; the inquiring glance of the aged Duke seemed to penetrate into his soul. He advanced to the ducal chair, the solemn music that streamed up as from the grave, wavered and faltered as if consciousness and idea were nearly lost. Something of the old confusion overpowered his senses, the figures that surrounded him became shadowy and unreal, and the power of decision seemed no longer his own. Out of the haze of confused imagery and distracting thought which surrounded him, he heard with unspeakable amazement the Duke's words,—
- 84. "I have waited your coming, Mr. Inglesant, impatiently, for I have a commission to entrust you with, or rather my daughter, the Grand Duchess, has written urgently to me from Florence to request me to send you to her without a moment's delay. Family matters relating to some in whom she takes the greatest interest, and who are well known, she says, to yourself, are the causes which lead to this request." Inglesant was too bewildered to speak. He had believed himself quite unknown to the Grand Duchess, whom he had never seen, but as he had passed before her in the ducal receptions at Florence. Who could these be in whom she took so great an interest, and who were known to him? But the Duke went on, speaking with a certain melancholy in his tone. "I have wished, Mr. Inglesant," he said, "to mark in some way the regard I have conceived for you, and the obligation under which I conceive myself to remain. It may be that, in the course that events are taking, it will no longer in a few weeks be in my power to bestow favours upon any man. I desire, therefore, to do what I have purposed before you leave the presence. I have caused the necessary deeds to be prepared which bestow upon you a small fief in the Apennines, consisting of some farms and of the Villa-Castle of San Georgio, where I myself in former days have passed many happy hours." He stopped, and in a moment or two resumed abruptly, without finishing the sentence. "The revenue of the fief is not large, but its possession gives the title of Cavaliere to its owner, and its situation and the character of its neighbourhood make it a desirable and delightful abode. The
- 85. letters of naturalization which are necessary to enable you to hold this property have been made out, and nothing is wanting but your acceptance of the gift. I offer it you with no conditions and no request save that, as far as in you lies, you will be a faithful servant to the Grand Duchess when I am gone." The Duke paused for a moment, and then, turning slightly to his chaplain, he said, "The reverend fathers will tell you that this affair has not been decided upon without their knowledge, and that it has their full approval." These last words convinced Inglesant of the fact that had occurred. Although the Duke had said nothing on the subject, he felt certain that the deed of cession had been signed, and that for some reason or other he himself was considered by the clerical party to have been instrumental in obtaining this result, and to be deserving of reward accordingly. He had never, as we have seen, spoken to the Duke concerning the succession, and his position at the moment was certainly a peculiar one. Nothing was expected of him but that he should express his grateful thanks for the Duke's favour, and leave the presence. Surely, at that moment, no law of heaven or earth could require him to break through the observances of civility and usage, to enter upon a subject upon which he was not addressed, and to refuse acts of favour offered to him with every grace and delicacy of manner. Whatever might be the case with other men, he certainly was not one to whom such a course was possible. He expressed his gratitude with all the grace of manner of which he was capable, he assured the Duke of his readiness to start immediately for Florence, and he left the ducal presence before many minutes had passed away.
- 86. Welcome to our website – the perfect destination for book lovers and knowledge seekers. We believe that every book holds a new world, offering opportunities for learning, discovery, and personal growth. That’s why we are dedicated to bringing you a diverse collection of books, ranging from classic literature and specialized publications to self-development guides and children's books. More than just a book-buying platform, we strive to be a bridge connecting you with timeless cultural and intellectual values. With an elegant, user-friendly interface and a smart search system, you can quickly find the books that best suit your interests. Additionally, our special promotions and home delivery services help you save time and fully enjoy the joy of reading. Join us on a journey of knowledge exploration, passion nurturing, and personal growth every day! ebookbell.com