![Bibliography
Before listing Ching-Zong’s publications, we give a brief introduction of their back-
ground and divide them broadly into five groups, in which the papers are referred
to by their numbers in the subsequent list.
A. Least squares estimates in stochastic regression models
Ching-Zong’s work in this area began with papers [1], [2] and [3], in which the strong
consistency of least squares estimates is established in fixed-design linear regression
models. In particular, when the errors are square integrable martingale differences,
a necessary and sufficient condition for the strong consistency of least squares es-
timates is given. However, when the regressors are stochastic, this condition is too
weak to ensure consistency. Paper [6] is devoted to resolving this difficulty, and es-
tablishes strong consistency and asymptotic normality of least squares estimates in
stochastic regression models under mild assumptions on the stochastic regressors
and errors. These results can be applied to interval estimation of the regression
parameters and to recursive on-line identification and control schemes for linear
dynamic systems, as shown in [6]. Papers [7], [12] and [15] extend the results of [6]
and establish the asymptotic properties of least squares estimates in more general
settings.
B. Adaptive control and stochastic approximation
Papers [17] and [18] resolve the dilemma between the control objective and the
need of information for parameter estimation by occasional use of white-noise prob-
ing inputs and by a reparametrization of the model. Asymptotically efficient self-
tuning regulators are constructed in [18] by making use of certain basic properties of
adaptive predictors involving recursive least squares for the reparametrized model.
Paper [16] studies excitation properties of the designs generated by adaptive con-
trol schemes. Instead of using least squares, [13] uses stochastic approximation for
recursive estimation of the unknown parameters in adaptive control. Paper [20]
introduces a multivariate version of adaptive stochastic approximation and demon-
strates that it is asymptotically efficient from both the estimation and control points
of view, while [28] uses martingale transforms with non-atomic limits to analyze
stochastic approximation. Paper [23] introduces irreversibility constraints into the
classical multi-armed bandit problem in adaptive control.
C. Nonstationary time series
For a general autoregressive (AR) process, [9] proves for the first time that the
least squares estimate is strongly consistent regardless of whether the roots of the
characteristic polynomial lie inside, on, or outside the unit disk. Paper [22] shows
that in general unstable AR models, the limiting distribution of the least squares
estimate can be characterized as a function of stochastic integrals. The techniques
x](https://image.slidesharecdn.com/505560-250529211005-008cb0ac/85/Time-Series-And-Related-Topics-In-Memory-Of-Chingzong-Wei-Ims-Hwaichung-Ho-15-320.jpg)
![xi
developed in [22] and in the earlier paper [19] for deriving the asymptotic distribu-
tion soon became standard tools for analyzing unstable time series and led to many
important developments in econometric time series, including recent advances in
the analysis of cointegration processes.
D. Adaptive prediction and model selection
Paper [21] considers sequential prediction problems in stochastic regression models
with martingale difference errors, and gives an asymptotic expression for the cu-
mulative sum of squared prediction errors under mild conditions. Paper [27] shows
that Rissanen’s predictive least squares (PLS) criterion can be decomposed as a
sum of two terms; one measures the goodness of fit and the other penalizes the
complexity of the selected model. Using this decomposition, sufficient conditions
for PLS to be strongly consistent in stochastic regression models are given, and
the asymptotic equivalence between PLS and the Bayesian information criterion
(BIC) is established. Moreover, a new criterion, FIC, is introduced and shown to
share most asymptotic properties with PLS while removing some of the difficulties
encountered by PLS in finite-sample situations. In [38], the first complete proof of
an analogous property for Akaike’s information criterion (AIC) in determining the
order of a vector autoregressive model used to fit a weakly stationary time series is
given, while in [41], AIC is shown to be asymptotically efficient for same-realization
predictions. Closely related papers on model selection and adaptive prediction are
[39], [42] and [43].
E. Probability theory, stochastic processes and other topics
In [4] and [5], sufficient conditions are given for the law of the iterated logarithm to
hold for random subsequences, least squares estimates in linear regression models
and partial sums of linear processes. Papers [8] and [14] provide sufficient conditions
for a general linear process to be a convergence system, while [10] considers mar-
tingale difference sequences that satisfy a local Marcinkiewicz-Zygmund condition.
Papers [24], [25] and [26] resolve long-standing estimation problems in branching
processes with immigration. Paper [35] studies the asymptotic behavior of the resid-
ual empirical process in stochastic regression models. In [36], uniform convergence
of sample second moments is established for families of time series arrays, whose
modeling by multistep prediction or likelihood methods is considered in [40]. Paper
[11], [29], [30] and [33] investigate moment inequalities and their statistical applica-
tions. Density estimation, mixtures, weak convergence of recursions and sequential
analysis are considered in [31], [32], [34] and [37].
Publications of Ching-Zong Wei
[1] Strong consistency of least squares estimates in multiple regression. Proc. Nat.
Acad. Sci. USA 75 (1978), 3034–3036. (With T. L. Lai and H. Robbins.)
[2] Strong consistency of least squares estimates in multiple regression II. J. Mul-
tivariate Anal. 9 (1979), 343–462. (With T. L. Lai and H. Robbins.)
[3] Convergence systems and strong consistency of least squares estimates in re-
gression models. J. Multivariate Anal. 11 (1981), 319–333. (With G. J. Chen
and T. L. Lai.)](https://image.slidesharecdn.com/505560-250529211005-008cb0ac/85/Time-Series-And-Related-Topics-In-Memory-Of-Chingzong-Wei-Ims-Hwaichung-Ho-16-320.jpg)
![xii
[4] Iterated logarithm laws with random subsequences. Z. Warsch. verw. Gebiete
57 (1981), 235–251. (With Y. S. Chow, H. Teicher and K. F. Yu.)
[5] A law of the iterated logarithm for double arrays of independent random vari-
ables with applications to regression and series models. Ann. Probab. 10 (1982),
320–335. (With T. L. Lai.)
[6] Least squares estimates in stochastic regression models with applications to
identification and control of dynamic systems. Ann. Statist. 10 (1982), 154–
166. (With T. L. Lai.)
[7] Asymptotic properties of projections with applications to stochastic regression
problems. J. Multivariate Anal. 12 (1982), 346–370. (With T. L. Lai.)
[8] Lacunary systems and generalized linear processes. Stoch. Process. Appl. 14
(1983), 187–199. (With T. L. Lai.)
[9] Asymptotic properties of general autoregressive models and strong consistency
of least squares estimates of their parameters. J. Multivariate Anal. 13 (1982),
1–23. (With T. L. Lai.)
[10] A note on martingale difference sequences satisfying the local Marcinkiewicz-
Zygmund condition. Bull. Inst. Math. Acad. Sinica 11 (1983), 1–13. (With
T. L. Lai.)
[11] Moment inequalities with applications to regression and time series models. In
Inequalities in Statistics and Probability (Y. L. Tong, ed.), 165–172. Monograph
Series, Institute of Mathematical Statistics, 1984. (With T. L. Lai.)
[12] Asymptotic properties of multivariate weighted sums with application to sto-
chastic regression in linear dynamic systems. In Multivariate Analysis VI
(P. R. Krishnaiah, ed.), 373–393. North-Holland, Amsterdam, 1985. (With
T. L. Lai.)
[13] Adaptive control with the stochastic approximation algorithm: Geometry and
convergence. IEEE Trans. Auto. Contr. 30 (1985), 330–338. (With A. Becker
and P. R. Kumar.)
[14] Orthonormal Banach systems with applications to linear processes. Z. Warsch.
verw. Gebiete 70 (1985), 381–393. (With T. L. Lai.)
[15] Asymptotic properties of least squares estimates in stochastic regression mod-
els. Ann. Statist. 13 (1985), 1498–1508.
[16] On the concept of excitation in least squares identification and adaptive con-
trol. Stochastics 16 (1986), 227–254. (With T. L. Lai.)
[17] Extended least squares and their application to adaptive control and prediction
in linear systems. IEEE Trans. Auto Contr. 31 (1986), 898–906. (With T. L.
Lai.)
[18] Asymptotically efficient self-tuning regulators. SIAM J. Contr. Optimization
25 (1987), 466–481. (With T. L. Lai.)
[19] Asymptotic inference for nearly nonstationary AR(1) process. Ann. Statist.,
15 (1987), 1050–1063. (With N. H. Chan.)
[20] Multivariate adaptive stochastic approximation. Ann. Statist. 15 (1987), 1115–
1130.
[21] Adaptive prediction by least squares predictors in stochastic regression models
with applications to time series. Ann. Statist. 15 (1987), 1667–1682.
[22] Limiting distributions of least squares estimates of unstable autoregressive
processes. Ann. Statist. 16 (1988), 367–401. (With N. H. Chan.)](https://image.slidesharecdn.com/505560-250529211005-008cb0ac/85/Time-Series-And-Related-Topics-In-Memory-Of-Chingzong-Wei-Ims-Hwaichung-Ho-17-320.jpg)
![xiii
[23] Irreversible adaptive allocation rules. Ann. Statist. 17 (1989), 801–823. (With
I. Hu.)
[24] Some asymptotic results for the branching process with immigration. Stoch.
Process. Appl. 31 (1989), 261–282. (With J. Winnicki.)
[25] Estimation of the means in the branching process with immigration. Ann.
Statist. 18 (1990), 1757–1778. (With J. Winnicki.)
[26] Convergence rates for the critical branching process with immigration. Statist.
Sinica 1 (1991), 175–184.
[27] On predictive least squares principles. Ann. Statist. 20 (1992), 1–42.
[28] Martingale transforms with non-atomic limits and stochastic approximation.
Probab. Theory Related Fields 95 (1993), 103–114.
[29] Moment bounds for deriving time series CLT’s and model selection procedures.
Statist. Sinica 3 (1993), 453–480. (With D. F. Findley.)
[30] A lower bound for expectation of a convex functional. Statist. Probab. Letters
18 (1993), 191–194. (With M. H. Guo.)
[31] A regression point of view toward density estimation. J. Nonparametric Statist.
4 (1994), 191–201. (With C. K. Chu.)
[32] How to mix random variables. J. Chinese Statist. Asso. 32 (1994), 295–300.
[33] A moment inequality for products. J. Chinese Statist. Asso. 33 (1995), 429–
436. (With Y. S. Chow.)
[34] Weak convergence of recursion. Stoch. Process. Appl. 68 (1997), 65–82. (With
G. K. Basak and I. Hu.)
[35] On residual empirical processes of stochastic regression models with applica-
tions to time series. Ann. Statist. 27 (1999), 237–261. (With S. Lee.)
[36] Uniform convergence of sample second moments of families of time series
arrays. Ann. Statist. 29 (2001), 815–838. (With D. F. Findley and B. M.
Pötscher.)
[37] Comments on “Sequential Analysis: Some Classical Problems and New Chal-
lenges” by T. L. Lai. Statist. Sinica 11 (2001), 378–379.
[38] AIC, overfitting principles, and the boundness of moments of inverse matrices
for vector autoregressions and related models. J. Multivariate Anal. 83 (2002),
415–450. (With D. F. Findley.)
[39] On same-realization prediction in an infinite-order autoregressive process.
J. Multivariate Anal. 85 (2003), 130–155. (With C. K. Ing.)
[40] Modeling of time series arrays by multistep prediction or likelihood meth-
ods. J. Econometrics 118 (2004), 151–187. (With D. F. Findley and B. M.
Pötscher.)
[41] Order selection for the same-realization prediction in autoregressive processes.
Ann. Statist. 33 (2005), 2423–2474. (With C. K. Ing.)
[42] A maximal moment inequality for long range dependent time series with appli-
cations to estimation and model selection. Statist. Sinica 16 (2006), 721–740.
(With C. K. Ing.)
[43] Forecasting unstable processes. In Time Series and Related Topics (H. C. Ho,
C. K. Ing and T. L. Lai, eds.). Monograph Series, Institute of Mathematical
Statistics, 2006. (With J. L. Lin.)](https://image.slidesharecdn.com/505560-250529211005-008cb0ac/85/Time-Series-And-Related-Topics-In-Memory-Of-Chingzong-Wei-Ims-Hwaichung-Ho-18-320.jpg)
![2 F. J. Breidt et al.
where {Zt} is a sequence of independent and identically distributed random vari-
ables with mean 0 and variance σ2
, is one of the simplest models in time series.
The MA(1) model is invertible if and only if |θ0| 1, since in this case Zt can be
represented explicitly in terms of past values of the Xt, i.e.,
Zt =
∞
j=0
θj
0Xt−j.
Under this invertibility constraint, standard estimation procedures that produce
asymptotically normal estimates are readily available. For example, if θ̂ represents
the maximum likelihood estimator, found by maximizing the Gaussian likelihood
based on the data X1, . . . , Xn, then it is well known (see Brockwell and Davis [3]),
that
(1.2)
√
n(θ̂ − θ0)
d
→ N(0, 1 − θ2
0) .
From the form of the limiting variance in (1.2), the asymptotic behavior of θ̂, let
alone the scaling, is not immediately clear in the unit root case corresponding to
θ0 = 1.
In the Gaussian case, the parameters θ0 and σ2
are not identifiable without the
constraint |θ0| ≤ 1. In particular, the profile Gaussian log-likelihood, obtained by
concentrating out the variance parameter, satisfies
L(θ) = L(1/θ) .
It follows that θ = 1 is a critical value of the profile likelihood and hence there is
a positive probability that θ = 1 is indeed the maximum likelihood estimator. If
θ0 = 1, then it turns out that this probability does not vanish asymptotically (see
for example Anderson and Takemura [1], Tanaka [7], and Davis and Dunsmuir [6]).
This phenomenon is referred to as the pile-up effect. For the case that θ0 = 1 or is
near one in the sense that θ0 = 1 + γ/n, it was shown in Davis and Dunsmuir [6]
that
n(θ̂ − θ0)
d
→ ξγ,
where ξγ is random variable with a discrete component at 0, corresponding to the
asymptotic pile-up probability, and a continuous component on (−∞, 0).
The MA(1) with unit root (θ0 = 1) arises naturally in a variety of time series
applications. For example, if an underlying time series consists of a linear trend plus
white noise errors, then the differenced series is an MA(1) with a unit root. In such
cases, testing for a unit root of the differenced series is equivalent to testing the
adequacy of the trend plus noise model. The unit root problem also arises naturally
in a signal plus noise model in which the signal is modeled as a random walk. The
differenced series follows a MA(1) model and has a unit root if and only if the
random walk signal is in fact a constant.
For Gaussian likelihood estimation, the pile-up effect is directly attributable
to the non-identifiability of θ0 in the unconstrained parameter space. On the other
hand, if the data are non-Gaussian, then θ0 is identifiable (see Breidt and Davis [2]).
In this paper, we focus on the pile-up probability for estimates based on a Laplace
likelihood. Assuming a Laplace distribution for the noise, we derive an expression
for the joint likelihood of θ and zinit, where zinit is an augmented variable that
is treated as a parameter and the scale parameter σ is concentrated out of the
likelihood. If zinit is set equal to 0, then the resulting joint likelihood corresponds](https://image.slidesharecdn.com/505560-250529211005-008cb0ac/85/Time-Series-And-Related-Topics-In-Memory-Of-Chingzong-Wei-Ims-Hwaichung-Ho-21-320.jpg)
![4 F. J. Breidt et al.
It follows that maximizing the joint Laplace log-likelihood is equivalent to minimiz-
ing the following objective function,
n(θ, zinit) =
n
t=0 |zt|, if |θ| ≤ 1,
n
t=0 |zt||θ|, otherwise.
(2.2)
In order to study the asymptotic properties of the minimizer of n when the
model θ0 = 1, we follow Davis and Dunsmuir [6] by building the sample size into
the parameterization of θ. Specifically, we use
θ = 1 +
β
n
,
(2.3)
where β is any real number. Additionally, since we are also treating zinit as a
parameter, this term is reparameterized as
zinit = Z0 +
ασ
√
n
.
(2.4)
Under the (β, α) parameterization, minimizing n with respect to θ and zinit is
equivalent to minimizing the function,
Un(β, α) ≡
1
σ
[n(θ, zinit) − n(1, Z0)] ,
with respect to β and α. The following theorem describes the limiting behavior
of Un.
Theorem 2.1. For the model (1.1) with θ0 = 1, assume the noise sequence {Zt}
is IID with EZt = 0, E[ sign(Zt)] = 0 (i.e., median of Zt is zero), EZ4
t ∞ and
common probability density function fZ(z) = σ−1
f(z/σ), where σ 0 is the scale
parameter. We further assume that the density function fZ has been normalized so
that σ = E|Zt|. Then
Un(β, α)
fidi
→ U(β, α),
(2.5)
where
fidi
→ denotes convergence in distribution of finite dimensional distributions
and
U(β, α) =
1
0
β
s
0
eβ(s−t)
dS(t) + αeβs
dW(s)
+f(0)
1
0
β
s
0
eβ(s−t)
dS(t) + αeβs
2
ds,
(2.6)
for β ≤ 0, and
U(β, α) =
1
0
−β
1
s+
e−β(t−s)
dS(t) + αe−β(1−s)
dW(s)
+f(0)
1
0
−β
1
s
e−β(t−s)
dS(t) + αe−β(1−s)
2
ds,
(2.7)
for β 0, in which S(t) and W(t) are the limits of the following partial sums
Sn(t) =
1
√
n
[nt]
i=0
Zi/σ, Wn(t) =
1
√
n
[nt]
i=0
sign(Zi),
respectively.](https://image.slidesharecdn.com/505560-250529211005-008cb0ac/85/Time-Series-And-Related-Topics-In-Memory-Of-Chingzong-Wei-Ims-Hwaichung-Ho-23-320.jpg)
![Non-invertible MA(1) 5
Remark. The stochastic integrals in (2.6) and (2.7) refer to Itô integrals. The
double stochastic stochastic integral in the first term on the right side of (2.7) is
computed as
1
0
1
s+
e−β(t−s)
dS(t)dW(s) =
1
0
e−βt
dS(t)
1
0
eβs
dW(s)
−
1
0
s
0
e−β(t−s)
dS(t)dW(s) −
1
0
dS(t)dW(t),
where (see (2.15) below)
1
0
dS(t)dW(t) = E(Zisign(Zi))/σ = E|Zi|/σ = 1 .
Proof. We only prove the result (2.5) for a fixed (β, α); the extension to a finite
collection of (β, α)’s is relatively straightforward. First consider the case β ≤ 0. For
calculating the Laplace likelihood n(θ, zinit) based on model (1.1), the residuals are
solved by zt = Xt + θzt−1 for t = 1, 2, . . . , n with the initial value z0 = zinit. Since
Xt = Zt −Zt−1, all of the true innovations can be solved forward by Zt = Xt +Zt−1
for t = 1, 2, . . . , n with the initial Z0. Therefore, the centered term n(1, Z0) can be
written as
n(1, Z0) = |Z0| +
n
i=1
|Xi + Xi−1 + · · · + X1 + Z0| =
n
i=0
|Zi|.
For β ≤ 0, i.e., θ ≤ 1,
zi = Xi + θXi−1 + · · · + θi−1
X1 + θi
zinit
= (Zi − Zi−1) + θ(Zi−1 − Zi−2) + · · · + θi−1
(Z1 − Z0) + θi
zinit
= Zi − (1 − θ)Zi−1 − θ(1 − θ)Zi−2 − · · · − θi−1
(1 − θ)Z0 − θi
(Z0 − zinit),
which, under the true model θ = 1, implies
1
σ
[n(θ, zinit) − n(1, Z0)] =
1
σ
n
i=0
|zi| −
n
i=0
|Zi|
(2.8)
=
1
σ
n
i=0
(|Zi − yi| − |Zi|) ,
where y0 ≡ Z0 − zinit and
yi ≡ (1 − θ)
i−1
j=0
θi−1−j
Zj + θi
(Z0 − zinit),
for i = 1, 2, . . . , n. Using the identity
|Z − y| − |Z| = −y sign(Z) + 2(y − Z)
1{0Zy} − 1{yZ0}
(2.9)](https://image.slidesharecdn.com/505560-250529211005-008cb0ac/85/Time-Series-And-Related-Topics-In-Memory-Of-Chingzong-Wei-Ims-Hwaichung-Ho-24-320.jpg)
![6 F. J. Breidt et al.
for Z = 0, the equation (2.8) is expressed as two summations, the first of which is
−
n
i=0
yi
σ
sign(Zi) = (θ − 1)
n
i=1
i−1
j=0
θi−1−j Zj
σ
sign(Zi)
+
zinit − Z0
σ
n
i=0
θi
sign(Zi)
=
β
n
n
i=1
i−1
j=0
1 +
β
n
i−j−1
Zj
σ
sign(Zi)
+
α
√
n
n
i=0
1 +
β
n
i
sign(Zi)
(2.10)
= β
1
0
s−
0
1 +
β
n
−nt
dSn(t)
1 +
β
n
ns−1
dWn(s)
+ α
1
0
1 +
β
n
ns
dWn(s)
→ β
1
0
s
0
eβ(s−t)
dS(t)dW(s) + α
1
0
eβs
dW(s) ,
where the limit in (2.10) follows from a simple adaptation of Theorem 2.4 (ii) in
Chan and Wei [4].
To handle the second summation in computing Un(β, α), we approximate the
sum
n
i=0
2
yi − Zi
σ
1{0Ziyi} − 1{yiZi0}
by
n
i=0
2E
yi − Zi
σ
1{0Ziyi} − 1{yiZi0}
|Fi−1 ,
where Fi is the σ-field generated by {Zj : j = 0, 1, . . . , i}. First we establish conver-
gence of the latter sum and then show that the variance of the difference in sums
converges to zero. Since
max
1≤i≤n
|yi| → 0,
yi ∈ Fi−1, we have
2E
yi − Zi
σ
1{0Ziyi}|Fi−1 = 2
yi
0
yi − Z
σ
1
σ
f(
z
σ
)dz
≈ f(0)
yi
0
2
yi − z
σ
d
z
σ
= f(0)
yi
σ
2
,](https://image.slidesharecdn.com/505560-250529211005-008cb0ac/85/Time-Series-And-Related-Topics-In-Memory-Of-Chingzong-Wei-Ims-Hwaichung-Ho-25-320.jpg)
![8 F. J. Breidt et al.
variance of (2.12) also converges to zero. The variance of (2.12) is equal to
n
i=0
var (y∗
i − E (y∗
i |Fi−1)) + 2
ij
cov
y∗
i − E (y∗
i |Fi−1) , y∗
j − E
y∗
j |Fj−1
=
n
i=0
E [y∗
i − E (y∗
i |Fi−1)]
2
=
n
i=0
EE (y∗
i )2
− (E (y∗
i |Fi−1))
2
|Fi−1
!
=
n
i=0
E E
(y∗
i )2
|Fi−1
− (E (y∗
i |Fi−1))
2
!
(2.13)
≈
n
i=0
E
4
3
f(0)
yi
σ
3
− f(0)2
yi
σ
4
≈
4
3
f(0)E
n
i=0
yi
σ
3
− f(0)2
E
n
i=0
yi
σ
4
→ 0,
as n → ∞, where
cov
y∗
i − E (y∗
i |Fi−1) , y∗
j − E
y∗
j |Fj−1
= E [y∗
i − E (y∗
i |Fi−1)]
y∗
j − E
y∗
j |Fj−1
#
= EE
(y∗
i − E (y∗
i |Fi−1))
y∗
j − E
y∗
j |Fj−1
$
$
$
$Fj−1
= E
(y∗
i − E (y∗
i |Fi−1)) E
y∗
j − E
y∗
j |Fj−1
$
$
$
$Fj−1
= 0,
for i j, and
E (y∗
i |Fi−1) ≈ f(0)
yi
σ
2
,
E
(y∗
i )
2
|Fi−1
≈
4
3
f(0)
yi
σ
3
,
√
n
n
i=0
yi
σ
3
→ −
1
0
β
s
0
eβ(s−t)
dS(t) + αeβs
3
ds,
n
n
i=0
yi
σ
4
→
1
0
β
s
0
eβ(s−t)
dS(t) + αeβs
4
ds.
Based on (2.10), (2.11), and (2.13), the proof for β ≤ 0 is complete.
The proof for β ≥ 0 given in (2.7) is similar to that for β ≤ 0. For β ≥ 0,
i.e., θ ≥ 1, the residuals {zt} are solved backward by zt−1 = θ−1
(zt − Xt) for
t = n, n − 1, . . . , 1 with the initial zn ≡ zinit +
n
t=1 Xt. Solving these equations,
we have
zn−1−i = −θ−1
Xn−i + θ−1
Xn−i−1 + · · · + θ−i
Xn − θ−i
zn
,](https://image.slidesharecdn.com/505560-250529211005-008cb0ac/85/Time-Series-And-Related-Topics-In-Memory-Of-Chingzong-Wei-Ims-Hwaichung-Ho-27-320.jpg)
![Non-invertible MA(1) 9
for i = 0, 1, . . . , n − 1. Writing Xt = Zt − Zt−1, we obtain
−zn−1−iθ = Xn−i + θ−1
Xn−i−1 + · · · + θ−i
Xn − θ−i
zn
= (Zn−i − Zn−i−1) + θ−1
(Zn−i+1 − Zn−i) + · · ·
+ θ−i
(Zn − Zn−1) − θ−i
zn
= −Zn−i−1 + (1 − θ−1
)Zn−i + · · · + θ−(i−1)
(1 − θ−1
)Zn−1
+ θ−i
(Zn − zn)
= −Zn−i−1 + yn−i−1,
where
yn−1−i ≡
1 − θ−1
i
j=1
(θ−1
)i−j
Zn−j + θ−i
(Zn − zn)
=
1 − θ−1
i
j=1
(θ−1
)i−j
Zn−j + θ−i
n
i=1
Xi + Z0 −
n
i=1
Xi + zinit
=
1 − θ−1
i
j=1
(θ−1
)i−j
Zn−j + θ−i
(Z0 − zinit),
for i = 0, 1, . . . , n − 1 and yn ≡ Zn − zn = Z0 − zinit. Again, for θ ≥ 1, we have
1
σ
[n(θ, zinit) − n(1, Z0)] =
1
σ
n
i=0
(|Zi − yi| − |Zi|) ,
which has the same form as that for θ ≤ 1 but with different {yi}. Following a
similar derivation for θ ≤ 1, one can show that
−
n
i=1
yi
σ
sign(Zi) → −β
1
0
1
s+
e−β(t−s)
dS(t)dW(s) + α
1
0
e−β(1−s)
dW(s),
n
i=0
y2
i
σ2
→
1
0
−β
1
s
e−β(t−s)
dS(t) + αe−β(1−s)
2
ds,
in distribution as n → ∞. Combining this with the analogous result (2.13) for
β ≥ 0, completes the proof.
We close this section with some elementary results concerning the relationship
between the limiting Brownian motions S(t) and W(t) that will be used in the
sequel. Since σ = E|Zt|, the process S(t) can be decomposed as
S(t) = W(t) + cV (t) ,
(2.14)
where {W(t)} and {V (t)} are independent standard Bronwnian motions on [0, 1]
and
c =
%
Var(Zt)/σ2 − 1 .](https://image.slidesharecdn.com/505560-250529211005-008cb0ac/85/Time-Series-And-Related-Topics-In-Memory-Of-Chingzong-Wei-Ims-Hwaichung-Ho-28-320.jpg)
![10 F. J. Breidt et al.
In addition, we have the following identities
1
0
V (s)ds = V (1) −
1
0
sdV (s),
1
0
V (s)dW(s) = V (1)W(1) −
1
0
W(s)dV (s),
1
0
dW(s)dW(s) =
1
0
ds = 1,
1
0
dV (s)dW(s) = 0,
where the first two equations can be obtained easily by integration by parts. It
follows that
(2.15)
1
0
dS(s)dW(s) =
1
0
dW(s)dW(s) + c
1
0
dV (s)dW(s) = 1 .
3. Pile-up probabilities
3.1. Joint likelihood
In this section, we will consider the local maximizer of the joint likelihood given
by −n in (2.2). This estimator was also studied by Davis and Dunsmuir [6] in the
Gaussian case. Denote by (θ̂
(J)
n , ẑ
(J)
init,n) the local minimizer of n(θ, zinit) in which
θ̂
(J)
n is closest to 1. Using the (β, α) parameterization given in (2.3) and (2.4), this
is equivalent to finding the local minimizer (β̂
(J)
n , α̂
(J)
n ) of Un(β, α) in which β̂
(J)
n is
closest to zero. Moreover, the respective local minimizers of n and Un are connected
through the following relations:
θ̂(J)
n = 1 +
β̂
(J)
n
n
, ẑ
(J)
init,n = Z0 +
α̂
(J)
n σ
√
n
.
(3.1)
If the convergence of Un to U in Theorem 1 is strengthened to weak convergence
of processes on C(R2
), then the argument given in Davis and Dunsmuir [6] suggests
the convergence in distribution of (β̂
(J)
n , α̂
(J)
n ) to (β(J)
, α(J)
), where (β̂(J)
, α̂(J)
) is
the local minimizer of U(β, α) in which β̂(J)
is closest to 0. It follows that
(n(θ̂(J)
n − 1),
√
n(ẑ
(J)
init,n − Z0)/σ)
d
→ (β̂(J)
, α̂(J)
) .
(3.2)
The proofs of these results are the subject of on-going research and will appear in
a forthcoming manuscript.
Turning to the question of pile-up probabilities, we have that 1 is a local min-
imizer if the derivative of the criterion function from the left is negative and the
derivative from the right is positive; that is,
P(θ̂(J)
n = 1) = P(β̂(J)
n = 0)
= P
lim
β↑0
∂
∂β
Un (β, α̂n(β)) 0 and lim
β↓0
∂
∂β
Un (β, α̂n(β)) 0 ,](https://image.slidesharecdn.com/505560-250529211005-008cb0ac/85/Time-Series-And-Related-Topics-In-Memory-Of-Chingzong-Wei-Ims-Hwaichung-Ho-29-320.jpg)
![12 F. J. Breidt et al.
Similarly, according to (2.7) in Theorem 2.1, we have
lim
β↓0
∂
∂α
U(β, α) = lim
β↓0
1
0
e−β(1−s)
dW(s) + f(0)2α
1
0
e−2β(1−s)
ds
=
1
0
dW(s) + 2αf(0)
1
0
ds
= W(1) + 2αf(0),
and therefore
α̂(0+) = −
W(1)
2f(0)
,
which is same as α̂(0−). The derivative of U(β, α) with respect to β at zero from
righthand side satisfies
∂
∂β
U(β, α) = −
1
0
1
s+
e−β(t−s)
dS(t)dW(s) − β
1
0
1
s
e−β(t−s)
(s − t)dS(t)dW(s)
+ α
1
0
e−β(1−s)
(s − 1)dW(s)
+ f(0)
'
2β
1
0
1
s
e−β(t−s)
dS(t)
2
ds
+ β2
1
0
2
1
s
e−β(t−s)
dS(t)
×
1
s
e−β(t−s)
(s − t)dS(t)
ds
+ α2
1
0
e−2β(1−s)
2(s − 1)ds
− 2α
1
0
e−β(1−s)
1
s
e−β(t−s)
dS(t)
ds
− 2αβ
1
0
1
s
e−β(1+t−2s)
(2s − t − 1)dS(t)ds
.
Taking the limit β ↓ 0 and using the remark in Section 2, we have
lim
β↓0
∂
∂β
U(β, α̂(β))
→ −
1
0
1
s+
dS(t)dW(s) + α̂(0+)
1
0
(s − 1)dW(s)
+ f(0)
α̂2
(0+)
1
0
2(s − 1)ds − 2α̂(0+)
1
0
1
s
dS(t)ds
= −S(1)W(1) +
1
0
S(s)dW(s) + 1 + α̂(0+)
[(s − 1)W(s)]1
0 −
1
0
W(s)ds
+ f(0)
−α̂2
(0+) − 2α̂(0+)
S(1) −
1
0
S(s)ds
=
1
0
S(s)dW(s) − W(1)
1
0
S(s)ds +
W(1)
2f(0)
1
0
W(s)ds −
W(1)
2
+ 1
= Y + 1.](https://image.slidesharecdn.com/505560-250529211005-008cb0ac/85/Time-Series-And-Related-Topics-In-Memory-Of-Chingzong-Wei-Ims-Hwaichung-Ho-31-320.jpg)
![Non-invertible MA(1) 13
Therefore, the pile-up probability in (3.3) can be expressed in terms of Y as
lim
n→∞
P(θ̂(J)
n = 1) = P [Y 0 and Y + 1 0]
= P [−1 Y 0] .
3.2. Exact likelihood estimation
In this section, we consider pile-up probabilities associated with the estimator that
maximizes the exact Laplace likelihood. For θ ≤ 1, the joint density of (xn, zinit)
satisfies
f(xn, zinit) =
n
t=0
f(zt) =
1
2σ
n+1
exp
−
n
t=0 |zt|
σ
=
1
2σ
n+1
exp
−
[n(θ, zinit) − n(1, Z0)] + n(1, Z0)
σ
=
1
2σ
n+1
exp
−
n
t=0 |Zt|
σ
e−Un(β,α)
.
Integrating out the augmented variable zinit, we obtain
∞
−∞
f(xn, zinit)dzinit =
1
2σ
n+1
exp
−
n
t=0 |Zt|
σ
σ
√
n
∞
−∞
e−Un(β,α)
dα,
since under the parameterization (2.4), dzinit = (σ/
√
n)dα. The Laplace log-likeli-
hood of (θ, σ) given xn then satisfies
∗
n(θ, σ) ≡ log
∞
−∞
f(xn, zinit)dzinit
= −(n + 1) log(2σ) −
n
t=0 |Zt|
σ
+ log
σ
√
n
+ log
∞
−∞
e−Un(β,α)
dα,
where the last term does not depend on σ as n → ∞. So maximizing ∗
n with respect
to θ ≤ 1 is approximately the same as maximizing
U∗
n(β) = log
∞
−∞
e−Un(β,α)
dα
(3.5)
with respect to β ≤ 0,
Similarly, for θ 1, the Laplace log-likelihood of (θ, σ) is
∗
n(θ, σ) ≡ log
∞
−∞
f(xn, zinit)dzinit
= −n log |θ| − (n + 1) log(2σ) −
n
t=0 |Zt|
σ|θ|
+ log
σ
√
n
+ log
∞
−∞
e−Un(β,α)|θ|−1
dα,
where again the last term does not depend on σ as n → ∞. As above, maximizing
∗
n with respect to θ 1 is equivalent to maximizing
U∗
n(β) = log
∞
−∞
e−Un(β,α)n/(n+β)
dα
(3.6)](https://image.slidesharecdn.com/505560-250529211005-008cb0ac/85/Time-Series-And-Related-Topics-In-Memory-Of-Chingzong-Wei-Ims-Hwaichung-Ho-32-320.jpg)
![Non-invertible MA(1) 15
Remark 2. The two estimators of θ0 considered in Sections 3.1 and 3.2 were defined
as the local optimizers of objective functions that were closest to 1. One could also
consider the global optimizers of these objective functions. For example, the exact
MLE in the Gaussian case was considered in Davis and Dunsmuir [6] and Davis,
Chen and Dunsmuir [5] and has a different limiting distribution than the local MLE.
In our case, there will be a positive asymptotic pile-up probability for the global
maximum of the joint likelihood and a zero asymptotic pile-up probability for the
global maximum of the exact likelihood.
Remark 3. Suppose Zt has a Laplace distribution with the density function
fZ(z) =
1
2σ
e−|z|/σ
.
Then Y defined in (3.4) satisfies
Y =
1
0
[W(1)s − W(s)] dV (s) −
1
2
,
(3.9)
where W(s) and V (s) are independent standard Brownian motions. To prove (3.9),
note that the constant c in (2.14) is equal to 1 so that
S(t) = W(t) + V (t).
In the following calculations, we use the well-known Itô formula
1
0
W(s)dW(s) =
W2
(1)
2
−
1
2
.
Since f(0) = 1/2, the random variable Y defined in (3.4) can be further simplified
in terms of W(t) and V (t) as
Y =
1
0
S(s)dW(s) − W(1)
1
0
S(s)ds +
W(1)
2f(0)
1
0
W(s)ds −
W(1)
2
=
1
0
V (s)dW(s) +
1
0
W(s)dW(s) − W(1)
1
0
V (s)ds − W(1)
1
0
W(s)ds
+W(1)
1
0
W(s)ds −
W2
(1)
2
= V (1)W(1) −
1
0
W(s)dV (s) +
W2
(1)
2
−
1
2
− W(1)
V (1) −
1
0
sdV (s)
−
W2
(1)
2
=
1
0
[W(1)s − W(s)] dV (s) −
1
2
.](https://image.slidesharecdn.com/505560-250529211005-008cb0ac/85/Time-Series-And-Related-Topics-In-Memory-Of-Chingzong-Wei-Ims-Hwaichung-Ho-34-320.jpg)
![16 F. J. Breidt et al.
Therefore, the pile-up probability for Laplace innovations is
P (−1 Y 0)
= P
−
1
2
1
0
[W(1)s − W(s)] dV (s)
1
2
= E
P
−
1
2
1
0
[W(1)s − W(s)] dV (s)
1
2
$
$
$
$ W(t) on t ∈ [0, 1]
= E
P −
1
2
1
0
[W(1)s − W(s)]2
ds
−1/2
U
1
2
1
0
[W(1)s − W(s)]2
ds
−1/2
= E
Φ
1
2
1
0
[W(1)s − W(s)]2
ds
−1/2
− Φ −
1
2
1
0
[W(1)s − W(s)]2
ds
−1/2
≈ 0.820,
where U has the standard normal distribution and Φ(·) is the corresponding cu-
mulative distribution function. This pile-up probability, which was computed via
simulation based on 100000 replications of W(t) on [0, 1], has a standard error of
0.0010.
Remark 4. From the limiting result (3.2), it follows that the random variable Z0
can be estimated consistently. It may seem odd to have a consistent estimate of a
noise term in a moving average process. On the other hand, an MA(1) process with
a unit root is both invertible and non-invertible. That is, Z0 is an element of the
two Hilbert spaces generated by the linear span of {Xt, t ≤ 0} and {Xt, t ≥ 1},
respectively. It is the latter Hilbert space which allows for consistent estimation
of Z0.
4. Numerical simulation
In this section, we compute the asymptotic pile-up probabilities associated with
the estimator θ̂(J)
which maximizes the joint Laplace likelihood for several dif-
ferent noise distributions. The empirical properties of estimators θ̂
(J)
n (the local
maximizer of the joint Laplace likelihood) and θ̂
(E)
n (the local maximizer of the
exact Laplace likelihood) for finite samples are compared with each other and with
the corresponding asymptotic theory.
For approximating the asymptotic pile-up probabilities and limiting distribu-
tion of β̂
(J)
n , we first simulate 100000 replications of independent standard Wiener
processes W(t) and V (t) on [0, 1] in which W(t) and V (t) are approximated by
the partial sums W(t) =
[10000t]
j=1 Wj/
√
10000 and V (t) =
[10000t]
j=1 Vj/
√
10000,
where {Wj} and {Vj} are independent standard normal random variables. From
the simulation of W(t) and V (t), the distribution of the limit random variable β̂(J)
can be tabulated and the pile-up probability P(−1 Y 0) estimated, where Y
is given in (3.4). The empirical pile-up probabilities and their asymptotic limits are](https://image.slidesharecdn.com/505560-250529211005-008cb0ac/85/Time-Series-And-Related-Topics-In-Memory-Of-Chingzong-Wei-Ims-Hwaichung-Ho-35-320.jpg)
![Non-invertible MA(1) 17
displayed in Table 1 for different noise distributions: Laplace, Gaussian, uniform,
and t with 5 degrees of freedom. Notice that there is good agreement between the
asymptotic and empirical probabilities for sample sizes as small as 50.
For examining the empirical performance of the local maximizers θ̂
(J)
n and θ̂
(E)
n ,
we only consider the process generated with Laplace noise with σ = 1 and sample
sizes n = 20, 50, 100, 200. For each setup, 1000 realizations of the MA(1) process
with θ0 = 1 are generated and the estimates θ̂
(J)
n and θ̂
(E)
n and their corresponding
estimates of the scale parameter are obtained. The estimation results are sum-
marized in Table 2. For comparison, the standard deviation based on the limit
distributions of θ̂
(J)
n and θ̂
(E)
n are also reported (denoted by asymp in the table),
which are obtained numerically based on 100000 replicates of the limit process U.
Generally speaking, the empirical root mean square errors are very close to their
asymptotic values even for very small samples. Moreover, the estimation error of
θ̂
(J)
n is about 1/2 the estimation error of θ̂
(E)
n , which indicates the superiority of
using the joint likelihood over exact likelihood when θ0 = 1.
We also considered performance of the two estimators θ̂
(J)
n and θ̂
(E)
n in the case
when θ0 = 1. A limit theory for these estimators can be derived in this case by
assuming that the true value θ0 is near 1. That is, we can parameterize the MA(1)
parameter by θ0 = 1 + γ/n (e.g., Davis and Dunsmuir [6]). While we have not
pursued the theory in the near unit root case, the relative performance of these
Table 1
Empirical pile-up probabilities of the local maximizer θ̂
(J)
n of the joint Laplace likelihood for an
MA(1) with θ0 = 1 and sample sizes n = 20, 50, 100, 200 (based on 1000 replicates) and their
asymptotic values under various noise distributions.
n Gau Lap Unif t(5)
20 0.827 0.796 0.831 0.796
50 0.859 0.806 0.864 0.823
100 0.873 0.819 0.864 0.817
200 0.844 0.819 0.843 0.831
500 0.855 0.809 0.841 0.846
∞ 0.873 0.820 0.862 0.836
Table 2
Bias, standard deviation and root mean square error of the local maximizers θ̂
(J)
n and θ̂
(E)
n of
the joint and exact Laplace likelihoods, respectively, for an MA(1) process generated by Laplace
noise with θ0 = 1 and σ = 1 ( 1000 replications).
n θ̂
(J)
n θ̂
(E)
n
n = 20 bias -0.003 -0.006
s.d. 0.066 0.144
rmse 0.066 0.144
asymp 0.053 0.121
n = 50 bias -0.000 0.000
s.d. 0.021 0.057
rmse 0.021 0.057
asymp 0.021 0.048
n = 100 bias -0.000 0.001
s.d. 0.011 0.030
rmse 0.011 0.030
asymp 0.011 0.024
n = 200 bias 0.000 0.001
s.d. 0.006 0.014
rmse 0.006 0.014
asymp 0.005 0.012](https://image.slidesharecdn.com/505560-250529211005-008cb0ac/85/Time-Series-And-Related-Topics-In-Memory-Of-Chingzong-Wei-Ims-Hwaichung-Ho-36-320.jpg)
![18 F. J. Breidt et al.
Table 3
Bias, standard deviation and root mean square error of the global maximizers θ̂
(J)
n and θ̂
(E)
n of
the joint and exact Laplace likelihoods, respectively, for an MA(1) process generated by Laplace
noise with θ0 = 0.8, 0.9, 0.95, 1/0.95, 1/0.9, 1/0.8, σ = 1, and n = 50 based on 1000 replications.
First 2 columns record the number of times (out of 1000) that the estimates were less than 1
(invertible) and equal to 1 (unit root).
θ0 1 = 1 bias s.d. rmse
0.8 θ̂
(J)
50 789 95 0.0734 0.1973 0.2105
θ̂
(E)
50 873 19 0.0498 0.1753 0.1822
0.9 θ̂
(J)
50 557 322 0.0578 0.1398 0.1513
θ̂
(E)
50 767 93 0.0327 0.0933 0.0989
0.95 θ̂
(J)
50 404 503 0.0322 0.0708 0.0778
θ̂
(E)
50 632 168 0.0235 0.0821 0.0854
1/0.95 θ̂
(J)
50 90 540 -0.0315 0.0763 0.0825
θ̂
(E)
50 286 114 -0.0207 0.0890 0.0914
1/0.9 θ̂
(J)
50 89 299 -0.0389 0.1227 0.1287
θ̂
(E)
50 207 71 -0.0327 0.1218 0.1261
1/0.8 θ̂
(J)
50 96 109 -0.0338 0.2645 0.2666
θ̂
(E)
50 149 19 -0.0492 0.2280 0.2333
estimators was compared in a limited simulation study. We considered 3 values of
θ0 = 0.8, 0.9, 0.95 and their reciprocals 1/0.8, 1/0.9, 1/0.95. The latter 3 cases cor-
respond to purely non-invertible models. The results reported in Table 3 are based
on the global optimization of the joint and exact likelihoods. The first two columns
contain the number of realizations out of 1000 in which the estimator was invertible
( 1) and on the unit circle (= 1), respectively. For example, in the θ0 = 0.8 and
θ̂
(J)
n case, 78.9% of the realizations produced invertible models, and the empirical
pile-up probability is 0.095. On the other hand, for θ0 = 1/0.8, 79.5% of the realiza-
tions produced a purely non-invertible model with an empirical pile-up probability
of 0.109. Both objective functions do a reasonably good job of discriminating be-
tween invertible and non-invertible models, with a performance edge going to the
exact likelihood. In terms of root mean square error, the performance of θ̂
(E)
n is
superior to θ̂
(J)
n as θ0 moves away from the unit circle.
Remark. The LAD estimate of θ0 is obtained by minimizing the objective function
given in (2.2) with zinit = 0. Although we have not considered the asymptotic pile-
up in this case, the estimator does not perform as well as θ̂
(J)
n and θ̂
(E)
n . For example,
in simulation results, not reported here, the rmse of the LAD estimator tended to
be twice as large as the rmse for the exact MLE.
References
[1] Anderson, T. W. and Takemura, A. (1986). Why do noninvertible esti-
mated moving averages occur? Journal of Time Series Analysis 7 235–254.
[2] Breidt, F. J. and Davis, R. A. (1992). Time-reversibility, identifiably, and
independence of innovations for stationary time series. Journal of Time Series
Analysis 13 377–390.
[3] Brockwell, P. J. and Davis, R. A. (1991). Time Series: Theory and Meth-
ods, 2nd Edition. Springer-Verlag, New York.](https://image.slidesharecdn.com/505560-250529211005-008cb0ac/85/Time-Series-And-Related-Topics-In-Memory-Of-Chingzong-Wei-Ims-Hwaichung-Ho-37-320.jpg)
![Non-invertible MA(1) 19
[4] Chan, N. H. and Wei, C. Z. (1988). Limiting distributions of least squares
estimates of unstable autoregressive processes. Annals of Statistics 16 367–401.
[5] Chen, M., Davis, R. A. and Dunsmuir, W. T. M. (1995). Inference for
MA(1) processes with a root on or near the unit circle. Invited paper in Proba-
bility and Mathematical Statistics, Issue in Honour of Neyman’s 100 Birthday
15 227–242.
[6] Davis, R. A. and Dunsmuir, W. T. M. (1996). Maximum likelihood esti-
mation for MA(1) processes with a root on or near the unit circle. Econometric
Theory 12 1–29.
[7] Tanaka, K. (1996). Time Series Analysis. Nonstationary and Noninvertible
Distribution Theory. Wiley, New York.](https://image.slidesharecdn.com/505560-250529211005-008cb0ac/85/Time-Series-And-Related-Topics-In-Memory-Of-Chingzong-Wei-Ims-Hwaichung-Ho-38-320.jpg)
![IMS Lecture Notes–Monograph Series
Time Series and Related Topics
Vol. 52 (2006) 20–47
c
Institute of Mathematical Statistics, 2006
DOI: 10.1214/074921706000000932
Recursive estimation of possibly
misspecified MA(1) models: Convergence
of a general algorithm
James L. Cantor1
and David F. Findley2
Science Application International Corporation and U.S. Census Bureau
Abstract: We introduce a recursive algorithm of conveniently general form
for estimating the coefficient of a moving average model of order one and
obtain convergence results for both correct and misspecified MA(1) models.
The algorithm encompasses Pseudolinear Regression (PLR—also referred to
as AML and RML1) and Recursive Maximum Likelihood (RML2) without
monitoring. Stimulated by the approach of Hannan (1980), our convergence
results are obtained indirectly by showing that the recursive sequence can be
approximated by a sequence satisfying a recursion of simpler (Robbins-Monro)
form for which convergence results applicable to our situation have recently
been obtained.
1. Introduction and overview
Our focus is on estimating the coefficient θ of an invertible scalar moving average
model of order 1 (MA(1)),
(1.1) yt = θet−1 + et
where et is treated as an unobserved, constant-variance martingale-difference pro-
cess. We do not assume the series yt, −∞ t ∞ from which the observations
come is correctly modeled by (1.1). They can come from any invertible autoregres-
sive moving average (ARMA) model or from more general models; see Section 2.
What we seek is a θ that minimizes the loss function
(1.2) L̄(θ) = E[(yt − yt|t−1(θ))2
] = E[e2
t (θ)]
where et(θ) = yt −yt|t−1(θ) and yt|t−1(θ) is the one-step-ahead-prediction of yt from
ys, −∞ s ≤ t − 1 based on the model defined by θ (see (2.7) below). We define
optimal estimation procedures to be those whose sequence of estimates θt minimizes
(1.2) in the limit. This is a property of (nonrecursive) maximum likelihood-type
estimates of θ, see Pötscher [23].
In this article, we analyze a continuously indexed family of recursive procedures
for estimating θ. Recursive procedures form an estimate θt for time t using the
observation yt at time t, the estimate θt−1 for t−1 and other recursively defined
quantities. Our family encompasses two standard algorithms, Recursive Maximum
1Science Applications International Corporation (SAIC), 4001 North Fairfax Drive, Suite 250,
Arlington, VA 22203, e-mail: james.l.cantor@saic.com
2U.S. Census Bureau, Statistical Research Division, Room 3000-4, Washington, DC 20233-
9100, e-mail: david.f.findley@census.gov
AMS 2000 subject classifications: primary 62M10; secondary 62L20.
Keywords and phrases: time series, Robbins-Monro, PLR, AML, RML1, RML2, misspecified
models.
20](https://image.slidesharecdn.com/505560-250529211005-008cb0ac/85/Time-Series-And-Related-Topics-In-Memory-Of-Chingzong-Wei-Ims-Hwaichung-Ho-39-320.jpg)
![Recursive estimation of possibly misspecified MA(1) models 21
Likelihood (RML) which is referred to throughout as RML2 [12, 21], and the sim-
pler Pseudolinear Regression (PLR) [21]—also known as Approximate Maximum
Likelihood (AML) [24] and RML1 [11, 20]. More specifically, our general recursive
algorithm generating θt depends on an index β, 0 ≤ β ≤ 1. The algorithm reduces
to PLR when β = 0 and to RML2 when β = 1.
Our main convergence result, Theorem 4.1, is obtained by constructing an ap-
proximating sequence θ̂t for which θt −θ̂t
a.s.
−→ 0 holds and which satisfies a Robbins-
Monro recursion,
(1.3) θ̂t = θ̂t−1 − δtf(θ̂t−1, β) + δtγt ,
in which γt
a.s.
−→ 0 and δt 0, δt
a.s.
−→ 0,
∞
k=0 δk = ∞ a.s., and
(1.4) f(θ, β) = −
π
−π
eiω
+ βθ
|(1 + θeiω)(1 + θβeiω)|2
gy(ω)dω.
Here
a.s.
−→ denotes almost sure convergence (convergence with probability one) and
gy(ω) denotes the spectral density of the time series yt. Note that when β = 0, then
(1.5) f(θ, 0) = −
π
−π
eiω
|(1 + θeiω)|2
gy(ω)dω = −E[et−1(θ)et(θ)] ,
and when β = 1, then
(1.6) f(θ, 1) = −
π
−π
eiω
+ θ
|(1 + θeiω)2|2
gy(ω)dω =
1
2
d
dθ
E[e2
t (θ)] =
1
2
L̄
(θ)
where L̄
(θ) denotes the first derivative of L̄(θ). We then apply a result of Fradkov
implicit in [8], as extended and corrected by Findley [9], to show that θ̂t converges
to {θ ∈ Θ : f(θ, β) = 0} where Θ is the open interval (-1,1) of real θ with |θ| 1.
(A similar result is implicit in proofs of Theorems 2.2.2–2.2.3 of Chen [7].) Hence,
for β = 0, θt
a.s.
−→ {θ ∈ Θ : E[et−1(θ)et(θ)] = 0} and for β = 1, θt
a.s.
−→ {θ ∈ Θ :
L̄
(θ) = 0}. Here and below, θt convergence a.s. to a set means that except on a set
of ξ ∈ Ξ with probability zero, every cluster point of θt(ξ) is an element of the set.
In the incorrect model situation, in which gy(ω) is not proportional to |1+θeiω
|2
,
for examples we have analyzed [5], these zero sets will be disjoint, establishing that
PLR converges to different values than RML2. Consequently, under the assumptions
of Theorem 4.1, we recover the results of Cantor [4] that were given in separate
theorems and proofs, establishing that, for certain families of AR(1) and MA(2)
processes, RML2 estimates of θ in the model (1.1) converge to an optimal limit (a
minimizer of (1.2)) whereas PLR estimates converge to a suboptimal limit [4, 5].
When the data come from an invertible MA(1) model, it is known that PLR
and monitored versions of RML2 can provide strongly consistent estimates of θ
[4, 11, 17, 19]. More generally, in the correct model situation for ARMAX models,
i.e., ARMA models with an exogenous input, Lai and Ying [17] provided a rigorous
proof of strong consistency of PLR (under a positive real condition on the MA
polynomial) and also of a monitored version of RML2 whose monitoring scheme
involves non-linear projections and an intermittently used recursive estimator for
which consistency has already been established. In Section 4 of [19], Lai and Ying
consider a simpler modification of RML2 in which, for monitoring, only auxiliary
consistent recursive estimates are used. They present detailed outlines of proofs
of strong consistency and asymptotic normality of the estimates from this new](https://image.slidesharecdn.com/505560-250529211005-008cb0ac/85/Time-Series-And-Related-Topics-In-Memory-Of-Chingzong-Wei-Ims-Hwaichung-Ho-40-320.jpg)
![22 J. L. Cantor and D. F. Findley
monitored RML2 scheme. The construction of Section II of [18] can be used to
obtain auxiliary recursive estimates with the properties required.
There is a rather comprehensive theory of recursive estimation of autoregressive
(AR) models, encompassing certain incorrect model situations for algorithms like
PLR (see e.g., [6]). There are, however, no published convergence results with rig-
orous proofs for MA models in the incorrect model situation. Ljung’s seminal work
on the convergence of recursive algorithms [20, 21] mentions the incorrect model
situation but provides only suggestive results (further discussed in Section 5).
This article has five sections. In Section 2, the assumptions on the data and some
consequences for the MA(1) model are given. In Section 3, the general recursive
algorithm is presented. The Convergence Theorem is stated and proved in Section 4.
Required preliminary technical results are given in Section 4.1 and the proof of the
theorem is provided in Section 4.2. Finally, Section 5 concludes the article with a
brief discussion.
2. Assumptions
The observations yt, t ≥ 1 are assumed to come from a mean zero, covariance
stationary scalar series, yt, −∞ t ∞ defined on the probability space (Ξ, F, P).
We use the following additional assumptions on the process yt:
(D1) y1 is nonzero with probability one; i.e., P{y2
1 0} = 1.
(D2) The series has a linear representation
(2.1) yt =
∞
s=0
κst−s such that κ0 = 1 and
∞
s=0
|κs| ∞
in which κ(z) =
∞
s=0 κszs
is nonzero for |z| ≤ 1 and {t} is a martingale-
difference sequence (m.d.s.) with respect to the sequence of sigma fields Ft =
σ(ys, −∞ s ≤ t). Thus E[t|Ft−1] = 0. By a result of Wiener [25, Theorem
VI 5.2], κ(z)−1
=
∞
s=0 βszs
with
∞
s=0 |βs| ∞, whence
(2.2) t =
∞
s=0
βsyt−s (β0 = 1) .
(D3) The conditional variance E[2
t |Ft−1] is constant almost surely; i.e.,
E[2
t |Ft−1] = σ2
a.s. Equivalently, E[2
t ] = σ2
and 2
t − σ2
is a m.d.s. with
respect to the Ft.
(D4) {t} is bounded a.s.; supt |t| ≤ K a.s. for some K ∞.
From (D2)–(D3), the spectral density gy(ω) can be expressed as
(2.3) gy(ω) =
σ2
2π
κ(eiω
)
2
where κ(eiω
) =
∞
j=0
κjeijω
,
and
(2.4) 0 m ≤ gy(ω) ≤ M ∞ for all −π ≤ ω ≤ π
for positive constants m and M. The series yt is an invertible ARMA process if and
only if κ(z) is a rational function.](https://image.slidesharecdn.com/505560-250529211005-008cb0ac/85/Time-Series-And-Related-Topics-In-Memory-Of-Chingzong-Wei-Ims-Hwaichung-Ho-41-320.jpg)
![Recursive estimation of possibly misspecified MA(1) models 23
Assumption (D4) is used extensively in the proof of the convergence theorem,
Theorem 4.1, in Section 4.
Under (D2)–(D4), we can apply, for example, the First Moment Bound Theorem
of Findley and Wei [10] to show that t−1
t
s=j+1(ysys−j − γy
j )
a.s.
−→ 0. Hence, from
the particular case yt = t in (2.1) and j = 0,
(2.5) t−1
t
s=1
2
s
a.s.
−→ σ2
.
We consider models for yt of the invertible, stationary first-order moving-average
type (MA(1)) given by
(2.6) yt = θet−1 + et, −∞ t ∞ .
For a given coefficient θ such that |θ| 1, the difference equation (2.6) is satisfied
with et = et(θ) given by the mean zero, covariance stationary one-step-ahead-
prediction-error series,
(2.7) et(θ) = (1 + θB)−1
yt =
∞
j=0
(−θ)j
yt−j = yt − yt|t−1(θ) ,
from the MA(1) predictor yt|t−1(θ) = −
∞
j=1(−θ)j
yt−j, see (5.1.21) of [3]. Here
B is the backshift operator; i.e., Byt = yt−1. The coefficient θ is referred to as the
MA coefficient. Thus,
(2.8) yt = et(θ) + θet−1(θ) .
The infinite series in (2.7) converges in mean square and, from (D4) and the rep-
resentation (2.1), also almost surely. Thus, et(θ) represents the optimal one-step-
ahead-prediction-error process from the perspective of the model (2.6). The model
(2.6) is correct if et(θ) coincides (a.s.) with the m.d.s. t in (2.2), in which case
βs = (−θ)s
, k ≥ 0. Whether or not the model is correct for any θ, forecast errors
et(θ) appearing in loss functions such as (1.2) and elsewhere are calculated as in
(2.7). We emphasize that (2.1) allows data processes far more general than MA(1)
processes. In particular, the z-transform,
∞
s=0 κszs
is not required to be rational.
For example, time series conforming to the exponential models of Bloomfield [2]
have non-rational κ(z) without zeroes in |z| ≤ 1.
Let Θ = (−1, 1). From (2.7), the spectral density of et(θ) is ge(θ, ω) = gy(ω) ·
|1 + θeiω
|−2
, so for L̄(θ) defined by (1.2), we have
(2.9) L̄(θ) =
π
−π
gy(ω)
|1 + θeiω|2
dω .
By (2.4) and the continuity of gy(ω), L̄(θ) is positive, infinitely differentiable,
and nonconstant on the interior of [−1, 1], i.e., on Θ, and infinite at the endpoints.
Therefore it has a minimum value over [−1, 1] and
(2.10) Θ∗
≡
θ ∈ [−1, 1]: θ = arg min
θ∈[−1,1]
L̄(θ)
,
is a subset of [−K, K] for some 0 K 1. Also Θ∗
⊆ Θ∗
0 = {θ ∈ Θ: L̄
(θ) = 0}. We
are interested in a.s bounded random recursive sequences θt = θt(ξ) that converge](https://image.slidesharecdn.com/505560-250529211005-008cb0ac/85/Time-Series-And-Related-Topics-In-Memory-Of-Chingzong-Wei-Ims-Hwaichung-Ho-42-320.jpg)
![24 J. L. Cantor and D. F. Findley
a.s. to Θ∗
or at least to Θ∗
0. If Θ∗
0 contains only one point, θ∗
0, then θt converges to
θ∗
0 a.s. Our results will establish convergence of the sequence of estimates θt defined
by the general algorithm presented below to the set of zeroes of f(θ, β) defined by
(1.4).
3. The general recursive algorithm
For 0 ≤ β ≤ 1, we define a general recursion for estimating the MA coefficient θ of
(1.1):
θt = θt−1 + P̄−1
t
1
t
φt−1et; θ1 = 0, t ≥ 2 ,
(3.1a)
P̄t =
1
t
t−1
s=1
φ2
s = P̄t−1 +
1
t
[φ2
t−1 − P̄t−1]; P̄1 = 0; t ≥ 2 ,
(3.1b)
et = yt − θt−1et−1; e1 = y1, t ≥ 2 ,
(3.1c)
φt = xt − θt−1φt−1; φ1 = x1, t ≥ 2 ,
(3.1d)
xt = yt − βθt−1xt−1; x1 = y1, t ≥ 2 .
(3.1e)
From (3.1a), it follows for 0 ≤ s ≤ t − 1, t ≥ 2 that
(3.2) θt−s = θt −
s−1
l=0
(t − l)−1
P̄−1
t−lφt−l−1et−l ,
where
−1
l=0(·) ≡ 0. From (3.1e),
(3.3) xt =
t−1
s=0
(−β)s
s
i=1
θt−i yt−s
where
0
i=1(·) ≡ 1. Next, let z1 = e1 and, for t ≥ 2,
(3.4) zt = et + θt−1φt−1 .
The value of the parameterization with β is that it enables us to simultaneously
obtain results for two important algorithms. When β = 0, then xt = yt from which
it follows that φt = et and zt = yt and therefore (3.1a)–(3.1e) is PLR (AML,
RML1)[11, 20, 21, 24]. When β = 1, then xt = et and φt = et − θt−1φt−1 and thus
(3.1a)–(3.1e) is RML2 [12, 21] without monitoring to ensure that each estimate θt
is in Θ = (−1, 1).
For any β, these θt can be expressed in the form of a regression estimate:
(3.5) θt =
t
s=2
φ2
s−1
−1 t
s=2
zsφs−1, t ≥ 2 .
An induction argument for (3.5) goes as follows. Set Pt = tP̄t =
t
s=2 φ2
s−1. Note
that from (D1), Pt 0 for all t 1 and therefore P−1
t exists a.s. From (3.1a)–(3.1e)
and (3.4), θ2 = 1/2φ2
1
−1
1/2(z2φ1) , which is (3.5) for t = 2. Suppose then it is
true for some t ≥ 2; i.e.,
(3.6) Ptθt =
t
s=2
zsφs−1 .](https://image.slidesharecdn.com/505560-250529211005-008cb0ac/85/Time-Series-And-Related-Topics-In-Memory-Of-Chingzong-Wei-Ims-Hwaichung-Ho-43-320.jpg)
![Recursive estimation of possibly misspecified MA(1) models 25
Then
Pt+1θt+1 = Pt+1(θt + P−1
t+1φtet+1) = (Pt + φ2
t )θt + φtet+1
=
t
s=2
zsφs−1 + φt(φtθt + et+1) (from the induction hypothesis (3.6))
=
t
s=2
zsφs−1 + φtzt+1 =
t+1
s=2
zsφs−1 .
Hence, (3.5) is true for t + 1 and by induction therefore for all t.
For use below, we define the stationary analogues et(θ), xt(θ), φt(θ) and zt(θ) of
et, xt, φt and zt:
et(θ) = (1 + θB)−1
yt ,
(3.7)
xt(θ) = (1 + θβB)−1
yt =
∞
j=0
(−βθ)j
yt−j ,
(3.8)
φt(θ) = (1 + θB)−1
xt(θ) =
∞
j=0
(−θ)j
xt−j(θ)
(3.9)
= (1 + θB)−1
(1 + θβB)−1
yt ,
so φt(θ) = et(θ) when β = 0. From (3.7)–(3.9),
(3.10) zt(θ) = et(θ) + θφt−1(θ) = [(1 + θB)−1
+ θB(1 + θB)−1
(1 + θβB)−1
]yt .
From (3.7)–(3.10),
E[φ2
t (θ)] =
π
−π
1
|(1 + θeiω)(1 + βθeiω)|2
gy(ω)dω ,
(3.11)
E[φt−1(θ)et(θ)] =
π
−π
eiω
(1 + θeiω)(1 + βθeiω)
1
(1 + θe−iω)
gy(ω)dω
(3.12)
=
π
−π
eiω
+ βθ
|(1 + θeiω)(1 + βθeiω)|2
gy(ω)dω ,
and
(3.13) E[zt(θ)φt−1(θ)] =
π
−π
eiω
+ θ(1 + β)
|(1 + θeiω)(1 + βθeiω)|2
gy(ω)dω .
From (1.4) and (3.12), E[φt−1(θ)et(θ)] = −f(θ, β). Let e
t(θ) = det(θ)/dθ. Then,
from (3.7),
(3.14) −e
t(θ) =
B
1 + θB
et(θ) =
B
(1 + θB)2
yt .
Since
1
2
d
dθ
E[e2
t (θ)] = E[e
t(θ)et(θ)] ,
from (2.9) and (3.14), the derivative of L̄(θ), L̄
(θ), is obtained from
−
1
2
L̄
(θ) = E[−e
t(θ)et(θ)] =
π
−π
eiω
(1 + θeiω)2
1
(1 + θe−iω)
gy(ω)dω
(3.15)
=
π
−π
eiω
+ θ
|(1 + θeiω)2|2
gy(ω)dω ,](https://image.slidesharecdn.com/505560-250529211005-008cb0ac/85/Time-Series-And-Related-Topics-In-Memory-Of-Chingzong-Wei-Ims-Hwaichung-Ho-44-320.jpg)
![26 J. L. Cantor and D. F. Findley
which is (3.12) with β = 1, verifying (1.6).
As a consequence of (2.4), we note that since |z| ≤ K∗
1 implies 0 1−K∗
≤
|1 − z| ≤ 1 + K∗
, for (3.11) with |θ| ≤ K∗
1 we have
(3.16)
m
(1 + K∗)
4 ≤
π
−π
1
|(1 + θeiω)(1 + βθeiω)|2
gy(ω)dω ≤
M
(1 − K∗)
4 .
4. The convergence theorem
The following result is a generalization of the PLR and RML2 results proved in [4]
for MA(1) models.
Theorem 4.1 (Convergence theorem). Consider a series yt for which (D1)–
(D4) hold. For each β such that 0 ≤ β ≤ 1, assume that the recursive sequence
defined by (3.1a)–(3.1e) is such that, for some random k∗
= k∗
(ξ) and K∗
=
K∗
(ξ)(ξ ∈ Ξ) satisfying 0 ≤ k∗
∞ and 0 K∗
1 , it holds almost surely that
|θt+k∗ | ≤ K∗
for all t. Then for f(θ, β) as in (1.4):
(a) The sequence θ̂t defined for t ≥ 1 by
θ̂t =
1
t
t
s=1
π
−π
1
|(1 + θs+k∗ eiω)(1 + βθs+k∗ eiω)|2
gy(ω)dω
−1
(4.1)
×
1
t
t
s=1
π
−π
cos ω + (1 + β)θs+k∗
|(1 + θs+k∗ eiω)(1 + βθs+k∗ eiω)|2
gy(ω)dω
has the property that θt − θ̂t
a.s.
−→ 0. Hence, with probability one, there is a
t0(ξ) ≥ 1 such that |θ̂t| ≤ (1 + K∗
)/2 1 holds for all t ≥ t0(ξ).
(b) For all t t0(ξ), θ̂t satisfies a Robbins-Monro recursion,
(4.2) θ̂t = θ̂t−1 − δtf(θ̂t−1, β) + δtγt ,
with γt
a.s.
−→ 0, δt 0 a.s., δt
a.s.
−→ 0, and
∞
s=t0+1 δs = ∞ a.s. where f(θ, β)
has the formula (1.4).
(c) From (a) and (b), it follows that, with Θ = (−1, 1), the sequence θt converges
a.s. to the compact set
(4.3) Θβ
0 = {θ ∈ Θ : f (θ, β) = 0}
in the sense that, on a probability one event Ξ0 that does not depend on β, for
each ξ ∈ Ξ0, the cluster points of θt(ξ) are contained in Θβ
0 . Further, when
yt is an invertible ARMA process, then Θβ
0 is finite, and θ(ξ) = limt→∞ θt(ξ)
exists for every ξ ∈ Ξ0.
Note from (3.5), (3.11) and (3.13) that the assertion θt − θ̂t
a.s.
−→ 0 in part (a) of
Theorem 4.1 can be formulated as the assertion that
1
t
t
s=1
φ2
s−1
−1
1
t
t
s=1
zsφs−1
−
1
t
t
s=1
E[φ2
t (θs+k∗ )]
−1
1
t
t
s=1
E[zt(θs+k∗ )φt−1(θs+k∗ )]](https://image.slidesharecdn.com/505560-250529211005-008cb0ac/85/Time-Series-And-Related-Topics-In-Memory-Of-Chingzong-Wei-Ims-Hwaichung-Ho-45-320.jpg)
![Recursive estimation of possibly misspecified MA(1) models 27
tends to zero a.s. In the expression above, φ0 = 0 and expectation is taken before
evaluation at θs+k∗ .
The proof of Theorem 4.1, given in Section 4.2. In [5], we provide complete
results concerning the existence of k∗
and K∗
with the required properties for
several incorrect model examples as well as for the correct model situation for
β = 0 (PLR) and provide more limited results for the case β = 1 (RML2) with
a particular monitoring scheme. For the latter case, we also report on simulation
results which demonstrate the existence of the variates k∗
, K∗
as in Theorem 4.1
with the consequence that monitoring becomes unnecessary for sufficiently large t.
In the correct model case yt = θt−1 + t with i.i.d. t, Lai and Ying [19] show for
their monitored RML2 that this happens a.s. and the conclusions of Theorem 4.1
concerning our approximating sequence (4.1) apply.
4.1. Preliminary results
Here we present some needed technical results. We first quote, without proof, a
powerful result from martingale theory [17, Lemma 1, part (i)]. Unless specified
otherwise, all limits (liminfs, limsups, etc.) are with respect to t and for simplicity
the t → ∞ will be usually suppressed.
Proposition 4.1. Let {˜
t} be a martingale difference sequence with respect to an
increasing sequence of σ-fields {Ft} such that supt E[|˜
t|2p
|Ft−1] ∞ holds a.s.
for some p 1. Let z̃t be an Ft−1-measurable random variable for every t. Then
t
s=1 z̃s˜
s converges almost surely on {
∞
s=1 z̃2
s ∞}, and for every η 1/2,
t
s=1 z̃s˜
s
t
s=1 z̃2
s
η
a.s.
−→ 0 on
∞
s=1
z̃2
s = ∞ .
Since
1
t
t
s=1
z̃s˜
s =
t
s=1 z̃s˜
s
t
s=1 z̃2
s
1
t
t
s=1
z̃2
s ,
it is clear that a corollary of this Proposition is
Proposition 4.2. Under the assumptions of Proposition 4.1, if lim sup t−1
×
t
s=1 z̃2
s ∞ a.s., then t−1
t
s=1 z̃s˜
s
a.s.
−→ 0.
Recall from (2.1) that yt = t +
∞
s=1 κst−s since κ0 = 1. A second consequence
of Proposition 4.1 is
Proposition 4.3. Suppose that the m.d.s. t in (D2) is such that supt E[|t|2p
|
Ft−1] ∞ holds a.s. for some p 1. Then for any sequence ŷt = yt − ỹt−1 in
which ỹt−1 is Ft−1-measurable, it holds that lim inf t−1
t
s=1 ŷ2
s ≥ σ2
a.s., where
σ2
= E[2
t ].
Proof. From (2.1), ŷt = yt − ỹt−1 = t + z̃t where z̃t = −ỹt−1 +
∞
s=1 κst−s is
Ft−1-measurable since
∞
s=1 κst−s is Ft−1-measurable by (2.2) and ỹt−1 is Ft−1-
measurable by assumption. Then
1
t
t
s=1
ŷ2
s =
1
t
t
s=1
2
s +
2
t
t
s=1
sz̃s +
1
t
t
s=1
z̃2
s
(4.4)
=
1
t
t
s=1
2
s + 2
t
s=1 sz̃s
t
s=1 z̃2
s
+ 1
1
t
t
s=1
z̃2
s .](https://image.slidesharecdn.com/505560-250529211005-008cb0ac/85/Time-Series-And-Related-Topics-In-Memory-Of-Chingzong-Wei-Ims-Hwaichung-Ho-46-320.jpg)
![28 J. L. Cantor and D. F. Findley
Consider first the event that
t
s=1 z̃2
s
a.s.
−→ l ∞. Then t−1
t
s=1 z̃2
s
a.s.
−→ 0 and,
by the preceding Proposition, t−1
t
s=1 sz̃s
a.s.
−→ 0. Hence, from (2.5) and the first
equation in (4.4), lim t−1
t
s=1 ŷ2
s = t−1
t
s=1 2
s = σ2
so the assertion holds in this
event. In the complementary event,
t
s=1 z̃2
s
a.s.
−→ ∞, from (4.4), it follows that
lim inf
1
t
t
s=1
ŷ2
s = lim inf
1
t
t
s=1
2
s + 2
t
s=1 sz̃s
t
s=1 z̃2
s
+ 1
1
t
t
s=1
z̃2
s
(4.5)
= σ2
+ lim inf
2
t
s=1 sz̃s
t
s=1 z̃2
s
+ 1
1
t
t
s=1
z̃2
s a.s.
By Proposition 4.1,
t
s=1 sz̃s/
t
s=1 z̃2
s
a.s.
−→ 0. Hence, the second expression in
(4.5) is nonnegative, and the proof is complete.
Proposition 4.4. Under (2.4), for each β ∈ [0, 1], the function f (θ, β) defined
by (1.4) is infinitely differentiable on Θ = (−1, 1), and Θβ
0 defined by (4.3) is a
nonempty compact subset of Θ. In the case β = 1, Θ1
0 contains the (nonempty) set
of minimizers over Θ of L̄ (θ) defined by (2.9).
Proof. The differentiability assertion follows from (2.4) via the dominated conver-
gence theorem. Except for compactness of Θ1
0, which will be discussed below, the
assertions concerning L̄ (θ) and f (θ, 1) were obtained subsequent to (2.10). The
remaining assertions follow from the continuity of f (θ, β) and the limit properties
(4.6) lim
θ→−1
f (θ, β) = −∞
and
(4.7) lim
θ→1
f (θ, β) = ∞.
Indeed, from (4.6)–(4.7), for any K 0 there exists an 0 (K, β) 1 such that
f(θ, β) ≤ −K for all θ ∈ (−1, −1+) and f(θ, β) ≥ K for all θ ∈ (1−, 1). Therefore
f(θ, β) must change sign over [−1 + ε, 1 − ε]. Hence f(θ, β) is non-constant and has
a zero in this interval and, moreover, Θβ
0 ⊆ [−1 + ε, 1 − ε]. Finally, since f(θ, β) is
continuous on this interval, Θβ
0 is compact. An analogous argument applies to Θ1
0.
To verify (4.6), we note that gy(ω) = gy(−ω), −π ≤ ω ≤ π yields
f (θ, β) = −
π
−π
cos ω + βθ
|(1 + θeiω) (1 + βθeiω)|
2 gy (ω) dω.
Because 0 ≤ β 1, for 0 ε 1 − β there is a δ = δ(ε) ∈ (0, π) such that
cos ω + βθ ≥ ε whenever |ω| ≤ δ and −1 ≤ θ ≤ 0. For such ε, δ, we obtain
lim
θ→−1
π
−π
cos ω + βθ
|(1 + θeiω) (1 + βθeiω)|
2 gy (ω) dω
=
−δ
−π
+
π
δ
cos ω + βθ
|(1 − eiω) (1 − βeiω)|
2 gy (ω) dω
(4.8)
+ lim
θ→−1
δ
−δ
cos ω + βθ
|(1 + θeiω) (1 + βθeiω)|
2 gy (ω) dω
(4.9)
= ∞,](https://image.slidesharecdn.com/505560-250529211005-008cb0ac/85/Time-Series-And-Related-Topics-In-Memory-Of-Chingzong-Wei-Ims-Hwaichung-Ho-47-320.jpg)
![Recursive estimation of possibly misspecified MA(1) models 29
because (4.8) is finite, whereas for (4.9) we have
lim
θ→−1
δ
−δ
cos ω + βθ
|(1 + θeiω) (1 + βθeiω)|
2 gy (ω) dω
≥ ε m lim
θ→−1
δ
−δ
1 + θeiω
1 + βθeiω
−2
dω = ∞.
This yields (4.6), and (4.7) follows by an analogous argument.
Proposition 4.5. Let yt be an invertible ARMA process, then for each β ∈ [0, 1],
the set Θβ
0 = {θ ∈ (−1, 1) : f(θ, β) = 0} is finite.
Proof. κ (z) in (D2) has the form κ (z) = η (z) /φ (z) where η (z) and φ (z) are
polynomials, of degrees dη and dφ, respectively, having no common zeroes and
having all zeros in {|z| 1}. Setting z = eiω
and h (z) = (1 + θz) (1 + βθz), we
obtain from dz = izdω that
−f(θ, β) =
π
−π
eiω
+ βθ
|(1 + θeiω) (1 + βθeiω)|
2 gy (ω) dω
=
σ2
ε
2πi
|z|=1
(z + βθ) η (z) η z−1
zh (z) h (z−1) φ (z) φ (z−1)
dz
=
σ2
ε
2πi
|z|=1
z1+dφ−dη
(z + βθ) η (z)
zdη
η z−1
h (z) {z2h (z−1)} φ (z) {zdφ φ (z−1)}
dz.
The function
w (z) = σ2
ε z1+dφ−dη
(z + βθ) η (z)
zdη
η z−1
h (z) {z2h (z−1)} φ (z) {zdφ φ (z−1)}
is nonzero on {|z| = 1} and has poles interior to the unit circle at −θ, −βθ, at the
zeroes of zdη
φ z−1
, and, if 1 + dφ − dη 0, also at 0. If zj, j = 1, . . . , n are the
distinct poles in {z : |z| 1}, then, by the Residue Theorem of complex analysis,
e.g., (4.7-10) of Henrici [13], it follows that
f (θ, β) = −
n
j=1
Resz=zj w (z) ,
where, if zj is a pole of order J ≥ 1,
Resz=zj w (z) =
1
(J − 1)!
lim
z→zj
dJ−1
dzJ−1
(z − zj)
J
w (z)
.
Thus each Resz=zj w (z) is a rational function of θ, and therefore the same is true
of f (θ, β). Consequently, f (θ, β) = 0 holds for only finitely many θ in (−1, 1).
The final preliminary result addresses convergence of a Robbins-Monro type
recursion that will be applied to demonstrate convergence of the general recursive
algorithm. It is a special case of a correction and extension by Findley [9] of a result
that is implicit in the proof of a theorem of Fradkov presented in Derevitzkiĭ and
Fradkov [8] for the case of monotonically decreasing δt. The result below is also
implicit in the proofs of Theorem 2.2.2 and Corollary 2.2.1 of Chen [7] which cover
the case of vector θ more completely than Findley [9].](https://image.slidesharecdn.com/505560-250529211005-008cb0ac/85/Time-Series-And-Related-Topics-In-Memory-Of-Chingzong-Wei-Ims-Hwaichung-Ho-48-320.jpg)
![of nations ever did, as only one nation in Europe could be excepted
from the general understanding of it. Mr. Pickering, he thought,
seemed not to have given full force to this circumstance, but seemed
to have weakened the evidence. [He referred to what Mr. Pickering
had said upon the subject.] It was Mr. Pickering's idea, that the
stipulation of free ships making free goods, was a mere temporary
provision; that it was not an article in the law of nations, but a new
principle introduced by the contracting parties. In order to prove this
was not the case, Mr. N. referred to the provisions entered into by
the armed neutrality of the north of Europe; to a treaty between
France and Spain; to a note from the Court of Denmark; and to the
declaration of the United States themselves on the subject.
With respect to contraband articles, he had little to say. It was
asserted that the articles stipulated in the British Treaty as
contraband, were made so by the law of nations. Where the doctrine
was found he could not say. It had been quoted from Vattel; this
authority might be correct; but he never found any two writers on
this subject agree as to this article. In a late publication on the law
of nations (Marten's) he found it directly asserted that naval stores
were not contraband. But he said, if the contrary were the law of
nations, they were bound to extend the same privilege to France
which they gave to England: they could not have one rule for the
one nation, and a different one for the other.
The 18th article of the British Treaty, respecting the carrying of
provisions, always struck him as a very important one. It had
heretofore been contended that this article did not go to any
provisions except such as were carrying to besieged or blockaded
places; but he believed the British had constantly made it a pretence
for seizing provisions going to France. Indeed, if he was not
mistaken, the British Minister had publicly declared in the House of
Commons, that the provisions on board the vessels intended for the
Quiberoon expedition had been supplied from what had been
captured in American vessels.](https://image.slidesharecdn.com/505560-250529211005-008cb0ac/85/Time-Series-And-Related-Topics-In-Memory-Of-Chingzong-Wei-Ims-Hwaichung-Ho-58-320.jpg)
![or August, 1793. But two years afterwards, when the British Treaty
was promulgated, the whole country was thrown into a flame by
admitting this very same doctrine. France herself had always acted
under this law of nations, when not restrained by treaty: in Valin's
Ordinances of France this clearly appears. The armed neutrality was
confined to the then existing war; Russia herself, the creator of the
armed neutrality, entered into a compact with England, in 1793,
expressly contravening its principles. The principle was then not
established by our Treaty with England; but such being the
acknowledged law of nations, it was merely stipulated that it should
be exercised in the manner least injurious to us.
2. The next article of complaint was with respect to contraband
goods. If gentlemen will consult the law of nations, they will find
that the articles mentioned in the British Treaty are by the law of
nations contraband articles. They will find that in all the treaties with
Denmark and Sweden, Great Britain had made the same stipulation.
Indeed, the gentleman had acknowledged that it was so stated by
some writers on the law of nations; but he wished to derogate from
the authority of those writers, in the same way as Mr. Genet, in his
correspondence with Mr. Jefferson, had called them worm-eaten
folios and musty aphorisms; to Vattel might be added Valin's
Ordinances, a very respectable work in France. How, then, can the
gentleman with truth say that we have deviated from the law of
nations?
3. The last point which the gentleman took notice of was the
provision article. There was no doubt that this Government would
never allow provisions to be deemed contraband, except when going
to a besieged or blockaded port. Though he made this declaration,
yet it was but candid to acknowledge that this was stated by Vattel
to be the law of nations. [He read an extract from Vattel.]
When this was stated by Lord Grenville to Mr. Pinckney, our then
Minister in London, Mr. Pinckney acknowledged it to be so stated in
Vattel, but very ingeniously argued that France could not be
considered as in the situation mentioned in Vattel, since provisions](https://image.slidesharecdn.com/505560-250529211005-008cb0ac/85/Time-Series-And-Related-Topics-In-Memory-Of-Chingzong-Wei-Ims-Hwaichung-Ho-62-320.jpg)
![Admitting further, that the Treaty had changed the existing state of
things between Great Britain and France, by having granted
commercial favors to Great Britain; by the 2d article of our treaty
with France, the same favors would immediately attach to France, so
that she could have no reason to complain on that ground. Indeed
France had herself new modified the treaty between that country
and this, and had taken to herself what she deemed to be the favors
granted to Great Britain. [Mr. S. read the decree on this subject of 2d
March last.]
Mr. S. said, he believed he had examined all the observations of the
gentleman from Virginia, relative to the Treaty, which were essential
to the subject under consideration. He did not wish to go much
farther on the present occasion, because he agreed with him, that it
was proper they should keep themselves as cool and calm as the
nature of the case would admit; but he thought whilst so much
deference was paid to the feelings of France, some respect ought to
be paid to the feelings of America. He hoped the people of America
would retain a proper respect and consideration for their national
character; and however earnestly he wished that the differences
subsisting between the two countries might be amicably settled, yet,
he trusted that our national dignity would never be at so low an ebb
as to submit to the insults and indignities of any nation whatever. In
saying this, he expressed his hearty wish to keep the door of
negotiation with France unclosed; but at the same time he strongly
recommended to take every necessary step to place us in a situation
to defend ourselves, provided she should still persist in her haughty
demeanor.
Mr. S. said, as he knew indecent and harsh language always recoiled
upon those who used it, he did not wish to adopt it; but, at the
same time, it was due to ourselves to express our feelings with a
proper degree of strength and spirit. He was not in the habit of
quoting any thing from M. Genet, but there was one expression of
his which he thought contained good advice, all this
accommodation and humility, all this condescension attains no end.](https://image.slidesharecdn.com/505560-250529211005-008cb0ac/85/Time-Series-And-Related-Topics-In-Memory-Of-Chingzong-Wei-Ims-Hwaichung-Ho-64-320.jpg)
![After the gentleman from Virginia had dwelt sufficiently upon the
danger of irritating the French, he had emphatically called upon us
to recollect our weakness. It might have been as well if he had left
that to have been discovered from another quarter. He hoped we
had sufficient confidence in the means of defence which we
possessed, if driven to the last resort; and he believed if there was
any one more certain way of provoking war than another, it was that
of proclaiming our own weakness.
He hoped such a language would now be spoken as would make
known to the French Government that the Government and people
of this country were one, and that they would repel any attempt to
gain an influence over our Councils and Government. The gentleman
had said that there did not appear to be any design of this kind, and
had endeavored to do away what was stated as the opinion in
France, in General Pinckney's letter. He did not mean to rest this
altogether upon the reports of an emigrant, whom General Pinckney
mentions as having represented this country divided, and of no
greater consequence than Genoa or Geneva, but he took the whole
information into view. [He read the extract relative to this subject.]
It was evident, Mr. S. said, from this information from France, that
an opinion had been industriously circulated there that the
Government and people of this country were divided; that the
Executive was corrupt and did not pursue the interests of the
people; and that they might, by perseverance, overturn the
Administration, and introduce a new order of things. Was not such
an opinion of things, he asked, calculated to induce France to
believe that she might make her own terms with us? It was well
known what the French wished, and it was time to declare it plainly.
His opinion was that they designed to ruin the commerce of Great
Britain through us. This was evident. They talk of the British Treaty;
but they suffered it to lie dormant for near twelve months, without
complaining about it. Why were they silent till within a few weeks
before the election of our President? Why did they commit
spoliations upon our commerce long before the British Treaty was
ever dreamt of? Their first decree, directing spoliations of our](https://image.slidesharecdn.com/505560-250529211005-008cb0ac/85/Time-Series-And-Related-Topics-In-Memory-Of-Chingzong-Wei-Ims-Hwaichung-Ho-65-320.jpg)
![not imply censure. By introducing it, it forced all those who entertain
even doubts of the propriety of any one Executive measure to vote
against the Address.
The principal causes of the irritation on the part of France, insisted
upon in the Answer, were the rejection of our Minister, and the
sentiments contained in the Speech of the President of the Directory
to our late Minister. If gentlemen would look into the documents laid
before the House by the President, he was confident they would find
the true reason for the refusal to receive our Minister. He came only
as an ordinary Minister, without any power to propose such
modifications as might lead to an accommodation, and when the
Directory discovered this from his credentials they refused him. In
answer to this, it had been urged that M. Delacroix, Minister of
Foreign Affairs, from the first, well knew that Mr. Pinckney was only
the successor to Mr. Monroe, and that his coming in that quality was
not the reason why the French refused to receive him. Mr. F. referred
to the documents which had been laid before the House on this
subject, from which it appeared that the secretary of M. Delacroix
had suggested a reason for the apparent change of opinion on the
subject of receiving Mr. Pinckney. Suppose, the secretary observed,
that M. Delacroix had made a mistake at first in the intentions of the
Directory, was that mistake to be binding on the Directory?
He did not wish to be understood to consider the conduct of the
French as perfectly justifiable; but he could not conceive that it was
such as to justify, on our part, irritating or violent measures. As to
the Speech of the President of the Directory, he could not say much
on it, he did not perfectly understand it. As far as he did, he
considered it a childish gasconade, not to be imitated, and below
resentment. [He read part of it]. It was certainly arrogant in him to
say that we owed our liberty to their exertions. But if the French
could derive any satisfaction from such vain boasting he had no
objection to their enjoying it. There was another part of the Speech
that had been considered as much more obnoxious. It was said to
breathe a design to separate the people here from their
Government. The part alluded to was no more than an expression of](https://image.slidesharecdn.com/505560-250529211005-008cb0ac/85/Time-Series-And-Related-Topics-In-Memory-Of-Chingzong-Wei-Ims-Hwaichung-Ho-68-320.jpg)
Time Series And Related Topics In Memory Of Chingzong Wei Ims Hwaichung Ho
- 1. Time Series And Related Topics In Memory Of Chingzong Wei Ims Hwaichung Ho download https://ebookbell.com/product/time-series-and-related-topics-in- memory-of-chingzong-wei-ims-hwaichung-ho-1011120 Explore and download more ebooks at ebookbell.com
- 2. Here are some recommended products that we believe you will be interested in. You can click the link to download. Research Papers In Statistical Inference For Time Series And Related Models Essays In Honor Of Masanobu Taniguchi 1st Ed 2023 Yan Liu https://ebookbell.com/product/research-papers-in-statistical- inference-for-time-series-and-related-models-essays-in-honor-of- masanobu-taniguchi-1st-ed-2023-yan-liu-50396198 Analysis Of Time Series Structure Ssa And Related Techniques 1st Edition Nina Golyandina https://ebookbell.com/product/analysis-of-time-series-structure-ssa- and-related-techniques-1st-edition-nina-golyandina-1384362 Time Series Analysis In Climatology And Related Sciences 1st Ed Victor Privalsky https://ebookbell.com/product/time-series-analysis-in-climatology-and- related-sciences-1st-ed-victor-privalsky-22496896 Deadline Scheduling For Realtime Systems Edf And Related Algorithms The Springer International Series In Engineering And Computer Science 460 Softcover Reprint Of The Original 1st Ed 1998 Stankovic https://ebookbell.com/product/deadline-scheduling-for-realtime- systems-edf-and-related-algorithms-the-springer-international-series- in-engineering-and-computer-science-460-softcover-reprint-of-the- original-1st-ed-1998-stankovic-7229926
- 3. Time Series And Panel Data Econometrics First Edition M Hashem Pesaran https://ebookbell.com/product/time-series-and-panel-data-econometrics- first-edition-m-hashem-pesaran-11926052 Time Series And Wavelet Analysis Festschrift In Honour Of Pedro A Morettin 2024th Edition Chang Chiann https://ebookbell.com/product/time-series-and-wavelet-analysis- festschrift-in-honour-of-pedro-a-morettin-2024th-edition-chang- chiann-197415618 Time Series And Panel Data Econometrics M Hashem Pesaran https://ebookbell.com/product/time-series-and-panel-data-econometrics- m-hashem-pesaran-7121200 Time Series And Panel Data Econometrics M Hashem Pesaran https://ebookbell.com/product/time-series-and-panel-data-econometrics- m-hashem-pesaran-7125574 Introduction To Time Series And Forecasting Peter J Brockwell Richard A Davis Brockwell https://ebookbell.com/product/introduction-to-time-series-and- forecasting-peter-j-brockwell-richard-a-davis-brockwell-23397854
- 5. Institute of Mathematical Statistics LECTURE NOTES–MONOGRAPH SERIES Time Series and Related Topics In Memory of Ching-Zong Wei Hwai-Chung Ho, Ching-Kang Ing, Tze Leung Lai, Editors Volume 52
- 6. ISBN 978-0-940600-68-3 ISBN 0-940600-68-4 ISSN 0749-2170
- 7. Institute of Mathematical Statistics LECTURE NOTES–MONOGRAPH SERIES Volume 52 Time Series and Related Topics In Memory of Ching-Zong Wei Hwai-Chung Ho, Ching-Kang Ing, Tze Leung Lai, Editors Institute of Mathematical Statistics Beachwood, Ohio, USA
- 8. Institute of Mathematical Statistics Lecture Notes–Monograph Series Series Editor: R. A. Vitale The production of the Institute of Mathematical Statistics Lecture Notes–Monograph Series is managed by the IMS Office: Jiayang Sun, Treasurer and Elyse Gustafson, Executive Director. Library of Congress Control Number: 2006936508 International Standard Book Number (13): 978-0-940600-68-3 International Standard Book Number (10): 0-940600-68-4 International Standard Serial Number: 0749-2170 Copyright c 2006 Institute of Mathematical Statistics All rights reserved Printed in the United States of America
- 9. Contents Contributors to this volume . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v Preface Hwai-Chung Ho, Ching-Kang Ing and Tze Leung Lai . . . . . . . . . . . . . . . . . . . vii CHING-ZONG WEI: BIOGRAPHICAL SKETCH AND BIBLIOGRAPHY Biographical sketch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . x Photographs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiv ESTIMATION AND PREDICTION IN TIME SERIES MODELS Pile-up probabilities for the Laplace likelihood estimator of a non-invertible first order moving average F. Jay Breidt, Richard A. Davis, Nan-Jung Hsu and Murray Rosenblatt . . . . . . . 1 Recursive estimation of possibly misspecified MA(1) models: Convergence of a general algorithm James L. Cantor and David F. Findley . . . . . . . . . . . . . . . . . . . . . . . . . . 20 Estimation of AR and ARMA models by stochastic complexity Ciprian Doru Giurcăneanu and Jorma Rissanen . . . . . . . . . . . . . . . . . . . . . 48 On prediction errors in regression models with nonstationary regressors Ching-Kang Ing and Chor-Yiu Sin . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 Forecasting unstable processes Jin-Lung Lin and Ching-Zong Wei . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 Order determination in general vector autoregressions Bent Nielsen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 The distribution of model averaging estimators and an impossibility result regarding its estimation Benedikt M. Pötscher . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 Conditional-sum-of-squares estimation of models for stationary time series with long memory P. M. Robinson . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130 TIME SERIES MODELING IN FINANCE, MACROECONOMICS AND OTHER APPLICATIONS Modeling macroeconomic time series via heavy tailed distributions J. A. D. Aston . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138 Fractional constant elasticity of variance model Ngai Hang Chan and Chi Tim Ng . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149 Estimation errors of the Sharpe ratio for long-memory stochastic volatility models Hwai-Chung Ho . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165 iii
- 10. iv Contents Cowles commission structural equation approach in light of nonstationary time series analysis Cheng Hsiao . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173 Combining domain knowledge and statistical models in time series analysis Tze Leung Lai and Samuel Po-Shing Wong . . . . . . . . . . . . . . . . . . . . . . . . 193 Multivariate volatility models Ruey S. Tsay . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210 RELATED TOPICS Multi-armed bandit problem with precedence relations Hock Peng Chan, Cheng-Der Fuh and Inchi Hu . . . . . . . . . . . . . . . . . . . . . . 223 Poisson process approximation: From Palm theory to Stein’s method Louis H. Y. Chen and Aihua Xia . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236 Statistical modeling for experiments with sliding levels Shao-Wei Cheng, C. F. J. Wu and Longcheen Huwang . . . . . . . . . . . . . . . . . . 245 Price systems for markets with transaction costs and control problems for some finance problems Tzuu-Shuh Chiang, Shang-Yuan Shiu and Shuenn-Jyi Sheu . . . . . . . . . . . . . . . 257 A note on the estimation of extreme value distributions using maximum product of spacings T. S. T. Wong and W. K. Li . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272 Some results on the Gittins index for a normal reward process Yi-Ching Yao . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 284
- 11. Contributors to this volume Aston, J. A. D. Academia Sinica Breidt, F. J. Colorado State University Cantor, J. L. Science Application International Corporation Chan, H. P. National University of Singapore Chan, N. H. The Chinese University of Hong Kong Chen, L. H. Y. National University of Singapore Chiang, T.-S. Academia Sinica Davis, R. A. Colorado State University Findley, D. F. U.S. Census Bureau Fuh, C.-D. Academia Sinica Giurcăneanu, C. D. Tampere University of Technology Ho, H.-C. Academia Sinica Hsiao, C. University of Southern California Hsu, N.-J. National Tsing-Hua University Hu, I. Hong Kong University of Science and Technology Ing, C.-K. Academia Sinica Lai, T. L. Stanford University Li, W. K. The University of Hong Kong Lin, J.-L. Academia Sinica Ng, C. T. The Chinese University of Hong Kong Nielsen, B. University of Oxford Pötscher, B. M. University of Vienna Rissanen, J. Technical University of Tampere and Helsinki, and Helsinki Institute for Information Technology Robinson, P. M. London School of Economics Rosenblatt, M. University of California at San Diego Sheu, S.-J. Academia Sinica Shiu, S.-Y. University of Utah Sin, C.-Y. Xiamen University Tsay, R. S. University of Chicago Wei, C.-Z. Academia Sinica Wong, S. P.-S. The Chinese Universty of Hong Kong Wong, T. S. T. The University of Hong Kong Xia, A. University of Melbourne Yao, Y.-C. Academia Sinica v
- 12. Preface A major research area of Ching-Zong Wei (1949–2004) was time series models and their applications in econometrics and engineering, to which he made many impor- tant contributions. A conference on time series and related topics in memory of him was held on December 12–14, 2005, at Academia Sinica in Taipei, where he was Director of the Institute of Statistical Science from 1993 to 1999. Of the forty-two speakers at the conference, twenty contributed to this volume. These papers are listed under the following three headings. 1. Estimation and prediction in time series models Breidt, Davis, Hsu and Rosenblatt consider estimation of the unknown moving average parameter θ in an MA(1) model when θ = 1, and derive the limiting pile-up probabilities P(θ̂ = 1) and 1/n-asymptotics for the Laplace likelihood es- timator θ̂. Cantor and Findley introduce a recursive estimator for θ in a possibly misspecified MA(1) model and obtain convergence results by approximating the recursive algorithm for the estimator by a Robbins–Monro-type stochastic approx- imation scheme. Giurcǎneanu and Rissanen consider estimation of the order of AR and ARMA models by stochastic complexity, which is the negative logarithm of a normalized maximum likelihood universal density function. Nielsen investigates estimation of the order in general vector autoregressive models and shows that likelihood-based information criteria, and likelihood ratio tests and residual-based tests can be used, regardless of whether the characteristic roots are inside, or on, or outside the unit disk, and also in the presence of deterministic terms. Instead of model selection, Pötscher considers model averaging in linear regression models, and derives the finite-sample and asymptotic distributions of model averaging esti- mators. Robinson derives the asymptotic properties of conditional-sum-of squares estimates in parametric models of stationary time series with long memory. Ing and Sin consider the final prediction error and the accumulated prediction error of the adaptive least squares predictor in stochastic regression models with non- stationary regressors. The paper by Lin and Wei, which was in preparation when Ching-Zong was still healthy, investigates the adaptive least squares predictor in unit-root nonstationary processes. 2. Time series modeling in finance, macroeconomics and other applications Aston considers criteria for deciding when and where heavy-tailed models should be used for macroeconomic time series, especially those in which outliers are present. Hsiao reviews nonstationary time series analysis from the perspective of the Cowles Commission structural equation approach, and shows that the same rank condi- tion for identification holds for both stationary and nonstationary time series, that certain instrumental variables are needed for consistent parameter estimation, and that classical instrumental-variable estimators have to be modified for valid infer- ence in the presence of unit roots. Chan and Ng investigate option pricing when vii
- 13. viii the volatility of the underlying asset follows a fractional version of the CEV (con- stant elasticity of variance) model. Ho considers linear process models, with a latent long-memory volatility component, for asset returns and provides asymptotically normal estimates, with a slower convergence rate than 1/ √ n, of the Sharpe ratios in these investment models. Tsay reviews some commonly used models for the time- varying multivariate volatility of k (≥ 2) assets and proposes a simple parsimonious approach that satisfies positive definite constraints on the time-varying correlation matrix. Lai and Wong propose a new approach to time series modeling that com- bines subject-matter knowledge of the system dynamics with statistical techniques in time series analysis and regression, and apply this approach to American option pricing and the Canadian lynx data. 3. Related topics Besides time series analysis, Ching-Zong also made important contributions to the multi-armed bandit problem, estimation in branching processes with immigration, stochastic approximation, adaptive control and limit theorems in probability, and had an active interest in the closely related areas of experimental design, stochastic control and estimation in non-regular and non-ergodic models. The paper by Chan, Fu and Hu uses the multi-armed bandit problem with precedence relations to an- alyze a multi-phase management problem and thereby establishes the asymptotic optimality of certain strategies. Yao develops an approximation to Gittins index in the discounted multi-armed bandit problem by using a continuity correction in an associated optional stopping problem. Chen and Xia describe Stein’s method for Poisson approximation and for Poisson process approximation from the points of view of immigration-death processes and Palm distributions. Cheng, Wu and Huwang propose a new approach, which is based on a response surface model, to the analysis of experiments that use the technique of sliding levels to treat related factors, and demonstrate the superiority of this approach over previous methods in the literature. Chiang, Sheu and Shiu formulate the valuation problem of a finan- cial derivative in markets with transaction costs as a stochastic control problem and consider optimization of expected utility by using the price systems for these mar- kets. Wong and Li propose to use the maximum product of spacings (MPS) method for parameter estimation in the GEV (generalized extreme value) family and the generalized Pareto family of distributions, and show that the MPS estimates are asymptotically efficient and can outperform the maximum likelihood estimates. We thank the Institute of Statistical Science of Academia Sinica for providing financial support for the conference. Special thanks also go to the referees who reviewed the manuscripts. A biographical sketch of Ching-Zong and a bibliography of his publications appear after this Preface. Hwai-Chung Ho Ching-Kang Ing Tze Leung Lai
- 14. Biographical sketch Ching-Zong Wei was born in 1949 in south Taiwan. He studied mathematics at National Tsing-Hua University, Taiwan, where he earned a BS degree in 1971 and an MS degree in 1973. He went to the United States in 1976 to pursue advanced studies in statistics at Columbia University, where he earned a PhD degree in 1980. He then joined the Department of Mathematics at the University of Maryland, College Park, as an Assistant Professor in 1980, and was promoted to Associate Professor in 1984 and Full Professor in 1988. In 1990 he returned to Taiwan, his beloved homeland, to join the Institute of Statistical Science at Academia Sinica, where he stayed as Research Fellow for the rest of his life, serving between 1993 and 1999 as Director of the Institute. He also held a joint appointment with the Department of Mathematics at National Taiwan University. In addition to his research and administrative work at Academia Sinica, Ching- Zong also made important contributions to statistical education in Taiwan. To promote statistical thinking among the general public, he published in local news- papers and magazines articles on various topics of general interest such as lottery games and the Bible code. These articles, written in Chinese, introduced basic sta- tistical and probabilistic concepts in a heuristic and reader-friendly manner via entertaining stories, without formal statistical jargon. Ching-Zong made fundamental contributions to stochastic regression, adaptive control, nonstationary time series, model selection and sequential design. In par- ticular, his pioneering works on (i) strong consistency of least squares estimates in stochastic regression models, (ii) asymptotic behavior of least squares estimates in unstable autoregressive models, and (iii) predictive least squares principles in model selection, have been influential in control engineering, econometrics and time series. A more detailed description of his work appears in the Bibliography. He was elected Fellow of the Institute of Mathematical Statistics in 1989, and served as an Associate Editor of the Annals of Statistics (1987–1993) and Statistic Sinica (1991– 1999). In 1999, when Ching-Zong was at the prime of his career, he was diagnosed with brain tumors. He recovered well after the first surgery and remained active in research and education. In 2002, he underwent a second surgery after recurrence of the tumors, which caused deterioration of his vision. He continued his work and courageous fight with brain tumors and passed away on November 18, 2004, after an unsuccessful third surgery. He was survived by his wife of close to 30 years, Mei, and a daughter. In recognition of his path-breaking contributions, Vol. 16 of Statistica Sinica contains a special memorial section dedicated to him. ix
- 15. Bibliography Before listing Ching-Zong’s publications, we give a brief introduction of their back- ground and divide them broadly into five groups, in which the papers are referred to by their numbers in the subsequent list. A. Least squares estimates in stochastic regression models Ching-Zong’s work in this area began with papers [1], [2] and [3], in which the strong consistency of least squares estimates is established in fixed-design linear regression models. In particular, when the errors are square integrable martingale differences, a necessary and sufficient condition for the strong consistency of least squares es- timates is given. However, when the regressors are stochastic, this condition is too weak to ensure consistency. Paper [6] is devoted to resolving this difficulty, and es- tablishes strong consistency and asymptotic normality of least squares estimates in stochastic regression models under mild assumptions on the stochastic regressors and errors. These results can be applied to interval estimation of the regression parameters and to recursive on-line identification and control schemes for linear dynamic systems, as shown in [6]. Papers [7], [12] and [15] extend the results of [6] and establish the asymptotic properties of least squares estimates in more general settings. B. Adaptive control and stochastic approximation Papers [17] and [18] resolve the dilemma between the control objective and the need of information for parameter estimation by occasional use of white-noise prob- ing inputs and by a reparametrization of the model. Asymptotically efficient self- tuning regulators are constructed in [18] by making use of certain basic properties of adaptive predictors involving recursive least squares for the reparametrized model. Paper [16] studies excitation properties of the designs generated by adaptive con- trol schemes. Instead of using least squares, [13] uses stochastic approximation for recursive estimation of the unknown parameters in adaptive control. Paper [20] introduces a multivariate version of adaptive stochastic approximation and demon- strates that it is asymptotically efficient from both the estimation and control points of view, while [28] uses martingale transforms with non-atomic limits to analyze stochastic approximation. Paper [23] introduces irreversibility constraints into the classical multi-armed bandit problem in adaptive control. C. Nonstationary time series For a general autoregressive (AR) process, [9] proves for the first time that the least squares estimate is strongly consistent regardless of whether the roots of the characteristic polynomial lie inside, on, or outside the unit disk. Paper [22] shows that in general unstable AR models, the limiting distribution of the least squares estimate can be characterized as a function of stochastic integrals. The techniques x
- 16. xi developed in [22] and in the earlier paper [19] for deriving the asymptotic distribu- tion soon became standard tools for analyzing unstable time series and led to many important developments in econometric time series, including recent advances in the analysis of cointegration processes. D. Adaptive prediction and model selection Paper [21] considers sequential prediction problems in stochastic regression models with martingale difference errors, and gives an asymptotic expression for the cu- mulative sum of squared prediction errors under mild conditions. Paper [27] shows that Rissanen’s predictive least squares (PLS) criterion can be decomposed as a sum of two terms; one measures the goodness of fit and the other penalizes the complexity of the selected model. Using this decomposition, sufficient conditions for PLS to be strongly consistent in stochastic regression models are given, and the asymptotic equivalence between PLS and the Bayesian information criterion (BIC) is established. Moreover, a new criterion, FIC, is introduced and shown to share most asymptotic properties with PLS while removing some of the difficulties encountered by PLS in finite-sample situations. In [38], the first complete proof of an analogous property for Akaike’s information criterion (AIC) in determining the order of a vector autoregressive model used to fit a weakly stationary time series is given, while in [41], AIC is shown to be asymptotically efficient for same-realization predictions. Closely related papers on model selection and adaptive prediction are [39], [42] and [43]. E. Probability theory, stochastic processes and other topics In [4] and [5], sufficient conditions are given for the law of the iterated logarithm to hold for random subsequences, least squares estimates in linear regression models and partial sums of linear processes. Papers [8] and [14] provide sufficient conditions for a general linear process to be a convergence system, while [10] considers mar- tingale difference sequences that satisfy a local Marcinkiewicz-Zygmund condition. Papers [24], [25] and [26] resolve long-standing estimation problems in branching processes with immigration. Paper [35] studies the asymptotic behavior of the resid- ual empirical process in stochastic regression models. In [36], uniform convergence of sample second moments is established for families of time series arrays, whose modeling by multistep prediction or likelihood methods is considered in [40]. Paper [11], [29], [30] and [33] investigate moment inequalities and their statistical applica- tions. Density estimation, mixtures, weak convergence of recursions and sequential analysis are considered in [31], [32], [34] and [37]. Publications of Ching-Zong Wei [1] Strong consistency of least squares estimates in multiple regression. Proc. Nat. Acad. Sci. USA 75 (1978), 3034–3036. (With T. L. Lai and H. Robbins.) [2] Strong consistency of least squares estimates in multiple regression II. J. Mul- tivariate Anal. 9 (1979), 343–462. (With T. L. Lai and H. Robbins.) [3] Convergence systems and strong consistency of least squares estimates in re- gression models. J. Multivariate Anal. 11 (1981), 319–333. (With G. J. Chen and T. L. Lai.)
- 17. xii [4] Iterated logarithm laws with random subsequences. Z. Warsch. verw. Gebiete 57 (1981), 235–251. (With Y. S. Chow, H. Teicher and K. F. Yu.) [5] A law of the iterated logarithm for double arrays of independent random vari- ables with applications to regression and series models. Ann. Probab. 10 (1982), 320–335. (With T. L. Lai.) [6] Least squares estimates in stochastic regression models with applications to identification and control of dynamic systems. Ann. Statist. 10 (1982), 154– 166. (With T. L. Lai.) [7] Asymptotic properties of projections with applications to stochastic regression problems. J. Multivariate Anal. 12 (1982), 346–370. (With T. L. Lai.) [8] Lacunary systems and generalized linear processes. Stoch. Process. Appl. 14 (1983), 187–199. (With T. L. Lai.) [9] Asymptotic properties of general autoregressive models and strong consistency of least squares estimates of their parameters. J. Multivariate Anal. 13 (1982), 1–23. (With T. L. Lai.) [10] A note on martingale difference sequences satisfying the local Marcinkiewicz- Zygmund condition. Bull. Inst. Math. Acad. Sinica 11 (1983), 1–13. (With T. L. Lai.) [11] Moment inequalities with applications to regression and time series models. In Inequalities in Statistics and Probability (Y. L. Tong, ed.), 165–172. Monograph Series, Institute of Mathematical Statistics, 1984. (With T. L. Lai.) [12] Asymptotic properties of multivariate weighted sums with application to sto- chastic regression in linear dynamic systems. In Multivariate Analysis VI (P. R. Krishnaiah, ed.), 373–393. North-Holland, Amsterdam, 1985. (With T. L. Lai.) [13] Adaptive control with the stochastic approximation algorithm: Geometry and convergence. IEEE Trans. Auto. Contr. 30 (1985), 330–338. (With A. Becker and P. R. Kumar.) [14] Orthonormal Banach systems with applications to linear processes. Z. Warsch. verw. Gebiete 70 (1985), 381–393. (With T. L. Lai.) [15] Asymptotic properties of least squares estimates in stochastic regression mod- els. Ann. Statist. 13 (1985), 1498–1508. [16] On the concept of excitation in least squares identification and adaptive con- trol. Stochastics 16 (1986), 227–254. (With T. L. Lai.) [17] Extended least squares and their application to adaptive control and prediction in linear systems. IEEE Trans. Auto Contr. 31 (1986), 898–906. (With T. L. Lai.) [18] Asymptotically efficient self-tuning regulators. SIAM J. Contr. Optimization 25 (1987), 466–481. (With T. L. Lai.) [19] Asymptotic inference for nearly nonstationary AR(1) process. Ann. Statist., 15 (1987), 1050–1063. (With N. H. Chan.) [20] Multivariate adaptive stochastic approximation. Ann. Statist. 15 (1987), 1115– 1130. [21] Adaptive prediction by least squares predictors in stochastic regression models with applications to time series. Ann. Statist. 15 (1987), 1667–1682. [22] Limiting distributions of least squares estimates of unstable autoregressive processes. Ann. Statist. 16 (1988), 367–401. (With N. H. Chan.)
- 18. xiii [23] Irreversible adaptive allocation rules. Ann. Statist. 17 (1989), 801–823. (With I. Hu.) [24] Some asymptotic results for the branching process with immigration. Stoch. Process. Appl. 31 (1989), 261–282. (With J. Winnicki.) [25] Estimation of the means in the branching process with immigration. Ann. Statist. 18 (1990), 1757–1778. (With J. Winnicki.) [26] Convergence rates for the critical branching process with immigration. Statist. Sinica 1 (1991), 175–184. [27] On predictive least squares principles. Ann. Statist. 20 (1992), 1–42. [28] Martingale transforms with non-atomic limits and stochastic approximation. Probab. Theory Related Fields 95 (1993), 103–114. [29] Moment bounds for deriving time series CLT’s and model selection procedures. Statist. Sinica 3 (1993), 453–480. (With D. F. Findley.) [30] A lower bound for expectation of a convex functional. Statist. Probab. Letters 18 (1993), 191–194. (With M. H. Guo.) [31] A regression point of view toward density estimation. J. Nonparametric Statist. 4 (1994), 191–201. (With C. K. Chu.) [32] How to mix random variables. J. Chinese Statist. Asso. 32 (1994), 295–300. [33] A moment inequality for products. J. Chinese Statist. Asso. 33 (1995), 429– 436. (With Y. S. Chow.) [34] Weak convergence of recursion. Stoch. Process. Appl. 68 (1997), 65–82. (With G. K. Basak and I. Hu.) [35] On residual empirical processes of stochastic regression models with applica- tions to time series. Ann. Statist. 27 (1999), 237–261. (With S. Lee.) [36] Uniform convergence of sample second moments of families of time series arrays. Ann. Statist. 29 (2001), 815–838. (With D. F. Findley and B. M. Pötscher.) [37] Comments on “Sequential Analysis: Some Classical Problems and New Chal- lenges” by T. L. Lai. Statist. Sinica 11 (2001), 378–379. [38] AIC, overfitting principles, and the boundness of moments of inverse matrices for vector autoregressions and related models. J. Multivariate Anal. 83 (2002), 415–450. (With D. F. Findley.) [39] On same-realization prediction in an infinite-order autoregressive process. J. Multivariate Anal. 85 (2003), 130–155. (With C. K. Ing.) [40] Modeling of time series arrays by multistep prediction or likelihood meth- ods. J. Econometrics 118 (2004), 151–187. (With D. F. Findley and B. M. Pötscher.) [41] Order selection for the same-realization prediction in autoregressive processes. Ann. Statist. 33 (2005), 2423–2474. (With C. K. Ing.) [42] A maximal moment inequality for long range dependent time series with appli- cations to estimation and model selection. Statist. Sinica 16 (2006), 721–740. (With C. K. Ing.) [43] Forecasting unstable processes. In Time Series and Related Topics (H. C. Ho, C. K. Ing and T. L. Lai, eds.). Monograph Series, Institute of Mathematical Statistics, 2006. (With J. L. Lin.)
- 19. Ching-Zong Wei, Maryland 1985. In Hualian, Taiwan, with wife and daughter, 2004.
- 20. IMS Lecture Notes–Monograph Series Time Series and Related Topics Vol. 52 (2006) 1–19 c Institute of Mathematical Statistics, 2006 DOI: 10.1214/074921706000000923 Pile-up probabilities for the Laplace likelihood estimator of a non-invertible first order moving average F. Jay Breidt1,∗,† , Richard A. Davis1,†,‡ , Nan-Jung Hsu 2 and Murray Rosenblatt 3 Colorado State University, National Tsing-Hua University and University of California at San Diego Abstract: The first-order moving average model or MA(1) is given by Xt = Zt − θ0Zt−1, with independent and identically distributed {Zt}. This is ar- guably the simplest time series model that one can write down. The MA(1) with unit root (θ0 = 1) arises naturally in a variety of time series applications. For example, if an underlying time series consists of a linear trend plus white noise errors, then the differenced series is an MA(1) with unit root. In such cases, testing for a unit root of the differenced series is equivalent to testing the adequacy of the trend plus noise model. The unit root problem also arises naturally in a signal plus noise model in which the signal is modeled as a ran- dom walk. The differenced series follows a MA(1) model and has a unit root if and only if the random walk signal is in fact a constant. The asymptotic theory of various estimators based on Gaussian likeli- hood has been developed for the unit root case and nearly unit root case (θ = 1+β/n, β ≤ 0). Unlike standard 1/ √ n-asymptotics, these estimation pro- cedures have 1/n-asymptotics and a so-called pile-up effect, in which P(θ̂ = 1) converges to a positive value. One explanation for this pile-up phenomenon is the lack of identifiability of θ in the Gaussian case. That is, the Gaussian likelihood has the same value for the two sets of parameter values (θ, σ2) and (1/θ, θ2σ2). It follows that θ = 1 is always a critical point of the likelihood function. In contrast, for non-Gaussian noise, θ is identifiable for all real values. Hence it is no longer clear whether or not the same pile-up phenomenon will persist in the non-Gaussian case. In this paper, we focus on limiting pile-up probabilities for estimates of θ0 based on a Laplace likelihood. In some cases, these estimates can be viewed as Least Absolute Deviation (LAD) estimates. Simulation results illustrate the limit theory. 1. Introduction The moving average model of order one (MA(1)) given by (1.1) Xt = Zt − θ0Zt−1, 1Department of Statistics, Colorado State University, Ft. Collins, CO 80523, USA, e-mail: jbreidt@stat.colostate.edu; rdavis@stat.colostate.edu 2Institute of Statistics, National Tsing-Hua University, Hsinchu, Taiwan, e-mail: njhsu@stat.nthu.edu.tw 3Department of Mathematics, University of California, San Diego, La Jolla, CA 92093, USA, e-mail: mrosenblatt@ucsd.edu ∗Research supported by NSF grant DMS-9972015. †Research supported by EPA STAR grant CR-829095. ‡Research supported by NSF grant DMS-0308109. AMS 2000 subject classifications: primary 62M10; secondary 60F05. Keywords and phrases: noninvertible moving averages, Laplace likelihood. 1
- 21. 2 F. J. Breidt et al. where {Zt} is a sequence of independent and identically distributed random vari- ables with mean 0 and variance σ2 , is one of the simplest models in time series. The MA(1) model is invertible if and only if |θ0| 1, since in this case Zt can be represented explicitly in terms of past values of the Xt, i.e., Zt = ∞ j=0 θj 0Xt−j. Under this invertibility constraint, standard estimation procedures that produce asymptotically normal estimates are readily available. For example, if θ̂ represents the maximum likelihood estimator, found by maximizing the Gaussian likelihood based on the data X1, . . . , Xn, then it is well known (see Brockwell and Davis [3]), that (1.2) √ n(θ̂ − θ0) d → N(0, 1 − θ2 0) . From the form of the limiting variance in (1.2), the asymptotic behavior of θ̂, let alone the scaling, is not immediately clear in the unit root case corresponding to θ0 = 1. In the Gaussian case, the parameters θ0 and σ2 are not identifiable without the constraint |θ0| ≤ 1. In particular, the profile Gaussian log-likelihood, obtained by concentrating out the variance parameter, satisfies L(θ) = L(1/θ) . It follows that θ = 1 is a critical value of the profile likelihood and hence there is a positive probability that θ = 1 is indeed the maximum likelihood estimator. If θ0 = 1, then it turns out that this probability does not vanish asymptotically (see for example Anderson and Takemura [1], Tanaka [7], and Davis and Dunsmuir [6]). This phenomenon is referred to as the pile-up effect. For the case that θ0 = 1 or is near one in the sense that θ0 = 1 + γ/n, it was shown in Davis and Dunsmuir [6] that n(θ̂ − θ0) d → ξγ, where ξγ is random variable with a discrete component at 0, corresponding to the asymptotic pile-up probability, and a continuous component on (−∞, 0). The MA(1) with unit root (θ0 = 1) arises naturally in a variety of time series applications. For example, if an underlying time series consists of a linear trend plus white noise errors, then the differenced series is an MA(1) with a unit root. In such cases, testing for a unit root of the differenced series is equivalent to testing the adequacy of the trend plus noise model. The unit root problem also arises naturally in a signal plus noise model in which the signal is modeled as a random walk. The differenced series follows a MA(1) model and has a unit root if and only if the random walk signal is in fact a constant. For Gaussian likelihood estimation, the pile-up effect is directly attributable to the non-identifiability of θ0 in the unconstrained parameter space. On the other hand, if the data are non-Gaussian, then θ0 is identifiable (see Breidt and Davis [2]). In this paper, we focus on the pile-up probability for estimates based on a Laplace likelihood. Assuming a Laplace distribution for the noise, we derive an expression for the joint likelihood of θ and zinit, where zinit is an augmented variable that is treated as a parameter and the scale parameter σ is concentrated out of the likelihood. If zinit is set equal to 0, then the resulting joint likelihood corresponds
- 22. Non-invertible MA(1) 3 to the least absolute deviation (LAD) objective function and the estimator of θ is referred to as the LAD estimator of θ0. The exact likelihood can be obtained by integrating out zinit. In this case the resulting estimator is referred to as the quasi-maximum likelihood estimator of θ0. It turns out that the estimator based on maximizing the joint likelihood always has a positive pile-up probability in the limit regardless of the true noise distribution. In contrast, the quasi-maximum likelihood estimator has a limiting pile-up probability of zero. In Section 2, we describe the main asymptotic results. We begin by deriving an expression for computing the joint likelihood function based on the observed data and the augmented variable Zinit, in terms of the density function of the noise. The exact likelihood function can then be computed by integrating out Zinit. After a reparameterizion, we derive the limiting behavior of the joint likelihood for the case when the noise is assumed to follow a Laplace distribution. In Section 3, we focus on the problem of calculating asymptotic pile-up probabilities for estimators which minimize the joint Laplace likelihood (as a function of θ and zinit) and the exact Laplace likelihood. Section 4 contains simulation results which illustrate the asymptotic theory of Section 3. 2. Main result Let {Xt} be the MA(1) model given in (1.1) where θ0 ∈ R, {Zt} is a sequence of iid random variables with EZt = 0 and density function fZ. In order to compute the likelihood based on the observed data Xn = (X1, . . . , Xn) , it is convenient to define an augmented initial variable Zinit defined by Zinit = Z0, if |θ| ≤ 1, Zn − n t=1 Xt, otherwise. A straightforward calculation shows that the joint density of the observed data Xn = (X1, X2, . . . , Xn) and the initial variable Zinit satisfies fX,Zinit (xn, zinit) = n j=0 fZ(zj) 1{|θ|≤1} + |θ|−n 1{|θ|1} , where the residuals {zt} are functions of Xn = xn, θ, and Zinit = zinit which can be solved forward by zt = Xt + θzt−1 for t = 1, 2, . . . , n with the initial z0 = zinit if |θ| ≤ 1 and backward by zt−1 = θ−1 (zt − Xt) for t = n, n − 1, . . . , 1 with the initial zn = zinit + n t=1 Xt, if |θ| 1. The Laplace log-likelihood is obtained by taking the density function for Zt to be fZ(z) = exp{−|z|/σ}/(2σ). If we view zinit as a parameter, then the joint log-likelihood is given by −(n + 1) log 2σ − 1 σ n t=0 |zt| − n(log |θ|)1{|θ|1} . (2.1) Maximizing this function with respect to the scale parameter σ, we obtain σ̂ = n t=0 |zt|/(n + 1).
- 23. 4 F. J. Breidt et al. It follows that maximizing the joint Laplace log-likelihood is equivalent to minimiz- ing the following objective function, n(θ, zinit) = n t=0 |zt|, if |θ| ≤ 1, n t=0 |zt||θ|, otherwise. (2.2) In order to study the asymptotic properties of the minimizer of n when the model θ0 = 1, we follow Davis and Dunsmuir [6] by building the sample size into the parameterization of θ. Specifically, we use θ = 1 + β n , (2.3) where β is any real number. Additionally, since we are also treating zinit as a parameter, this term is reparameterized as zinit = Z0 + ασ √ n . (2.4) Under the (β, α) parameterization, minimizing n with respect to θ and zinit is equivalent to minimizing the function, Un(β, α) ≡ 1 σ [n(θ, zinit) − n(1, Z0)] , with respect to β and α. The following theorem describes the limiting behavior of Un. Theorem 2.1. For the model (1.1) with θ0 = 1, assume the noise sequence {Zt} is IID with EZt = 0, E[ sign(Zt)] = 0 (i.e., median of Zt is zero), EZ4 t ∞ and common probability density function fZ(z) = σ−1 f(z/σ), where σ 0 is the scale parameter. We further assume that the density function fZ has been normalized so that σ = E|Zt|. Then Un(β, α) fidi → U(β, α), (2.5) where fidi → denotes convergence in distribution of finite dimensional distributions and U(β, α) = 1 0 β s 0 eβ(s−t) dS(t) + αeβs dW(s) +f(0) 1 0 β s 0 eβ(s−t) dS(t) + αeβs 2 ds, (2.6) for β ≤ 0, and U(β, α) = 1 0 −β 1 s+ e−β(t−s) dS(t) + αe−β(1−s) dW(s) +f(0) 1 0 −β 1 s e−β(t−s) dS(t) + αe−β(1−s) 2 ds, (2.7) for β 0, in which S(t) and W(t) are the limits of the following partial sums Sn(t) = 1 √ n [nt] i=0 Zi/σ, Wn(t) = 1 √ n [nt] i=0 sign(Zi), respectively.
- 24. Non-invertible MA(1) 5 Remark. The stochastic integrals in (2.6) and (2.7) refer to Itô integrals. The double stochastic stochastic integral in the first term on the right side of (2.7) is computed as 1 0 1 s+ e−β(t−s) dS(t)dW(s) = 1 0 e−βt dS(t) 1 0 eβs dW(s) − 1 0 s 0 e−β(t−s) dS(t)dW(s) − 1 0 dS(t)dW(t), where (see (2.15) below) 1 0 dS(t)dW(t) = E(Zisign(Zi))/σ = E|Zi|/σ = 1 . Proof. We only prove the result (2.5) for a fixed (β, α); the extension to a finite collection of (β, α)’s is relatively straightforward. First consider the case β ≤ 0. For calculating the Laplace likelihood n(θ, zinit) based on model (1.1), the residuals are solved by zt = Xt + θzt−1 for t = 1, 2, . . . , n with the initial value z0 = zinit. Since Xt = Zt −Zt−1, all of the true innovations can be solved forward by Zt = Xt +Zt−1 for t = 1, 2, . . . , n with the initial Z0. Therefore, the centered term n(1, Z0) can be written as n(1, Z0) = |Z0| + n i=1 |Xi + Xi−1 + · · · + X1 + Z0| = n i=0 |Zi|. For β ≤ 0, i.e., θ ≤ 1, zi = Xi + θXi−1 + · · · + θi−1 X1 + θi zinit = (Zi − Zi−1) + θ(Zi−1 − Zi−2) + · · · + θi−1 (Z1 − Z0) + θi zinit = Zi − (1 − θ)Zi−1 − θ(1 − θ)Zi−2 − · · · − θi−1 (1 − θ)Z0 − θi (Z0 − zinit), which, under the true model θ = 1, implies 1 σ [n(θ, zinit) − n(1, Z0)] = 1 σ n i=0 |zi| − n i=0 |Zi| (2.8) = 1 σ n i=0 (|Zi − yi| − |Zi|) , where y0 ≡ Z0 − zinit and yi ≡ (1 − θ) i−1 j=0 θi−1−j Zj + θi (Z0 − zinit), for i = 1, 2, . . . , n. Using the identity |Z − y| − |Z| = −y sign(Z) + 2(y − Z) 1{0Zy} − 1{yZ0} (2.9)
- 25. 6 F. J. Breidt et al. for Z = 0, the equation (2.8) is expressed as two summations, the first of which is − n i=0 yi σ sign(Zi) = (θ − 1) n i=1 i−1 j=0 θi−1−j Zj σ sign(Zi) + zinit − Z0 σ n i=0 θi sign(Zi) = β n n i=1 i−1 j=0 1 + β n i−j−1 Zj σ sign(Zi) + α √ n n i=0 1 + β n i sign(Zi) (2.10) = β 1 0 s− 0 1 + β n −nt dSn(t) 1 + β n ns−1 dWn(s) + α 1 0 1 + β n ns dWn(s) → β 1 0 s 0 eβ(s−t) dS(t)dW(s) + α 1 0 eβs dW(s) , where the limit in (2.10) follows from a simple adaptation of Theorem 2.4 (ii) in Chan and Wei [4]. To handle the second summation in computing Un(β, α), we approximate the sum n i=0 2 yi − Zi σ 1{0Ziyi} − 1{yiZi0} by n i=0 2E yi − Zi σ 1{0Ziyi} − 1{yiZi0} |Fi−1 , where Fi is the σ-field generated by {Zj : j = 0, 1, . . . , i}. First we establish conver- gence of the latter sum and then show that the variance of the difference in sums converges to zero. Since max 1≤i≤n |yi| → 0, yi ∈ Fi−1, we have 2E yi − Zi σ 1{0Ziyi}|Fi−1 = 2 yi 0 yi − Z σ 1 σ f( z σ )dz ≈ f(0) yi 0 2 yi − z σ d z σ = f(0) yi σ 2 ,
- 26. Non-invertible MA(1) 7 for yi 0, and 2E yi − Zi σ 1{yiZi0}|Fi−1 = 2 0 yi yi − z σ 1 σ f( z σ )dz ≈ f(0) 0 yi 2 yi − z σ d z σ = −f(0) yi σ 2 , for yi 0. Combining these two cases, we have 2 n i=0 E yi − Zi σ 1{0Ziyi} − 1{yiZi0} |Fi−1 ≈ f(0) n i=0 yi σ 2 , where n i=0 yi σ 2 = n i=0 (1 − θ) i−1 j=1 θi−1−j Zj σ + θi Z0 − z0 σ 2 = n i=1 −β n i−1 j=1 1 + β n i−1−j Zj σ − α √ n 1 + β n i 2 (2.11) = n i=1 β (i−1)/n 0 1 + β n i−1−sn dSn(s) + α 1 + β n i 2 1 n → 1 0 β s 0 eβ(s−t) dS(t) + αeβs 2 ds in distribution as n → ∞. It is left to show that 2 n i=0 yi − Zi σ 1{0Ziyi} − 1{yiZi0} (2.12) − 2 n i=0 E yi − Zi σ 1{0Ziyi} − 1{yiZi0} |Fi−1 converges to zero in probability. Define y∗ i ≡ 2 yi − Zi σ 1{0Ziyi} − 1{yiZi0} . The expectation of (2.12) is zero and therefore, it is enough to show that the
- 27. 8 F. J. Breidt et al. variance of (2.12) also converges to zero. The variance of (2.12) is equal to n i=0 var (y∗ i − E (y∗ i |Fi−1)) + 2 ij cov y∗ i − E (y∗ i |Fi−1) , y∗ j − E y∗ j |Fj−1 = n i=0 E [y∗ i − E (y∗ i |Fi−1)] 2 = n i=0 EE (y∗ i )2 − (E (y∗ i |Fi−1)) 2 |Fi−1 ! = n i=0 E E (y∗ i )2 |Fi−1 − (E (y∗ i |Fi−1)) 2 ! (2.13) ≈ n i=0 E 4 3 f(0) yi σ 3 − f(0)2 yi σ 4 ≈ 4 3 f(0)E n i=0 yi σ 3 − f(0)2 E n i=0 yi σ 4 → 0, as n → ∞, where cov y∗ i − E (y∗ i |Fi−1) , y∗ j − E y∗ j |Fj−1 = E [y∗ i − E (y∗ i |Fi−1)] y∗ j − E y∗ j |Fj−1 # = EE (y∗ i − E (y∗ i |Fi−1)) y∗ j − E y∗ j |Fj−1 $ $ $ $Fj−1 = E (y∗ i − E (y∗ i |Fi−1)) E y∗ j − E y∗ j |Fj−1 $ $ $ $Fj−1 = 0, for i j, and E (y∗ i |Fi−1) ≈ f(0) yi σ 2 , E (y∗ i ) 2 |Fi−1 ≈ 4 3 f(0) yi σ 3 , √ n n i=0 yi σ 3 → − 1 0 β s 0 eβ(s−t) dS(t) + αeβs 3 ds, n n i=0 yi σ 4 → 1 0 β s 0 eβ(s−t) dS(t) + αeβs 4 ds. Based on (2.10), (2.11), and (2.13), the proof for β ≤ 0 is complete. The proof for β ≥ 0 given in (2.7) is similar to that for β ≤ 0. For β ≥ 0, i.e., θ ≥ 1, the residuals {zt} are solved backward by zt−1 = θ−1 (zt − Xt) for t = n, n − 1, . . . , 1 with the initial zn ≡ zinit + n t=1 Xt. Solving these equations, we have zn−1−i = −θ−1 Xn−i + θ−1 Xn−i−1 + · · · + θ−i Xn − θ−i zn ,
- 28. Non-invertible MA(1) 9 for i = 0, 1, . . . , n − 1. Writing Xt = Zt − Zt−1, we obtain −zn−1−iθ = Xn−i + θ−1 Xn−i−1 + · · · + θ−i Xn − θ−i zn = (Zn−i − Zn−i−1) + θ−1 (Zn−i+1 − Zn−i) + · · · + θ−i (Zn − Zn−1) − θ−i zn = −Zn−i−1 + (1 − θ−1 )Zn−i + · · · + θ−(i−1) (1 − θ−1 )Zn−1 + θ−i (Zn − zn) = −Zn−i−1 + yn−i−1, where yn−1−i ≡ 1 − θ−1 i j=1 (θ−1 )i−j Zn−j + θ−i (Zn − zn) = 1 − θ−1 i j=1 (θ−1 )i−j Zn−j + θ−i n i=1 Xi + Z0 − n i=1 Xi + zinit = 1 − θ−1 i j=1 (θ−1 )i−j Zn−j + θ−i (Z0 − zinit), for i = 0, 1, . . . , n − 1 and yn ≡ Zn − zn = Z0 − zinit. Again, for θ ≥ 1, we have 1 σ [n(θ, zinit) − n(1, Z0)] = 1 σ n i=0 (|Zi − yi| − |Zi|) , which has the same form as that for θ ≤ 1 but with different {yi}. Following a similar derivation for θ ≤ 1, one can show that − n i=1 yi σ sign(Zi) → −β 1 0 1 s+ e−β(t−s) dS(t)dW(s) + α 1 0 e−β(1−s) dW(s), n i=0 y2 i σ2 → 1 0 −β 1 s e−β(t−s) dS(t) + αe−β(1−s) 2 ds, in distribution as n → ∞. Combining this with the analogous result (2.13) for β ≥ 0, completes the proof. We close this section with some elementary results concerning the relationship between the limiting Brownian motions S(t) and W(t) that will be used in the sequel. Since σ = E|Zt|, the process S(t) can be decomposed as S(t) = W(t) + cV (t) , (2.14) where {W(t)} and {V (t)} are independent standard Bronwnian motions on [0, 1] and c = % Var(Zt)/σ2 − 1 .
- 29. 10 F. J. Breidt et al. In addition, we have the following identities 1 0 V (s)ds = V (1) − 1 0 sdV (s), 1 0 V (s)dW(s) = V (1)W(1) − 1 0 W(s)dV (s), 1 0 dW(s)dW(s) = 1 0 ds = 1, 1 0 dV (s)dW(s) = 0, where the first two equations can be obtained easily by integration by parts. It follows that (2.15) 1 0 dS(s)dW(s) = 1 0 dW(s)dW(s) + c 1 0 dV (s)dW(s) = 1 . 3. Pile-up probabilities 3.1. Joint likelihood In this section, we will consider the local maximizer of the joint likelihood given by −n in (2.2). This estimator was also studied by Davis and Dunsmuir [6] in the Gaussian case. Denote by (θ̂ (J) n , ẑ (J) init,n) the local minimizer of n(θ, zinit) in which θ̂ (J) n is closest to 1. Using the (β, α) parameterization given in (2.3) and (2.4), this is equivalent to finding the local minimizer (β̂ (J) n , α̂ (J) n ) of Un(β, α) in which β̂ (J) n is closest to zero. Moreover, the respective local minimizers of n and Un are connected through the following relations: θ̂(J) n = 1 + β̂ (J) n n , ẑ (J) init,n = Z0 + α̂ (J) n σ √ n . (3.1) If the convergence of Un to U in Theorem 1 is strengthened to weak convergence of processes on C(R2 ), then the argument given in Davis and Dunsmuir [6] suggests the convergence in distribution of (β̂ (J) n , α̂ (J) n ) to (β(J) , α(J) ), where (β̂(J) , α̂(J) ) is the local minimizer of U(β, α) in which β̂(J) is closest to 0. It follows that (n(θ̂(J) n − 1), √ n(ẑ (J) init,n − Z0)/σ) d → (β̂(J) , α̂(J) ) . (3.2) The proofs of these results are the subject of on-going research and will appear in a forthcoming manuscript. Turning to the question of pile-up probabilities, we have that 1 is a local min- imizer if the derivative of the criterion function from the left is negative and the derivative from the right is positive; that is, P(θ̂(J) n = 1) = P(β̂(J) n = 0) = P lim β↑0 ∂ ∂β Un (β, α̂n(β)) 0 and lim β↓0 ∂ ∂β Un (β, α̂n(β)) 0 ,
- 30. Non-invertible MA(1) 11 where α̂n(β) = arg minα Un(β, α) for given β. Assuming convergence of the right- and left-hand derivatives of the process Un(β, α̂n(β)), we obtain (3.3) lim n→∞ P(θ̂(J) n = 1) = P lim β↑0 ∂ ∂β U (β, α̂(β)) 0 and lim β↓0 ∂ ∂β U (β, α̂(β)) 0 , where α̂(β) = arg minα U(β, α). We now proceed to simplify the limits of the two derivatives in the brackets of (3.3) in terms of the processes S(t) and W(t). Ac- cording to (2.6) in Theorem 2.1, we have lim β↑0 ∂ ∂α U(β, α) = lim β↑0 1 0 eβs dW(s) + f(0)2α 1 0 e2βs ds = 1 0 dW(s) + 2αf(0) 1 0 ds = W(1) + 2αf(0), and therefore α̂(0−) = − W(1) 2f(0) . The derivative of U(β, α) with respect to β at zero from the left-hand side satisfies ∂ ∂β U(β, α) = 1 0 s 0 eβ(s−t) dS(t)dW(s) + β 1 0 s 0 eβ(s−t) (s − t)dS(t)dW(s) + α 1 0 eβs sdW(s) + f(0) ' 2β 1 0 s 0 eβ(s−t) dS(t) 2 ds + β2 1 0 2 s 0 eβ(s−t) dS(t) s 0 eβ(s−t) (s − t)dS(t) ds + α2 1 0 e2βs 2sds + 2α 1 0 eβs s 0 eβ(s−t) dS(t) ds + 2αβ 1 0 eβs s 0 eβ(s−t) (2s − t)dS(t) ds . Taking the limit as β ↑ 0, we have lim β↑0 ∂ ∂β U(β, α̂(β)) = 1 0 s 0 dS(t)dW(s) + α̂(0−) 1 0 sdW(s) + f(0) α̂2 (0−) 1 0 2sds + 2α̂(0−) 1 0 s 0 dS(t)ds = 1 0 S(s)dW(s) − W(1) 1 0 S(s)ds (3.4) + W(1) 2f(0) 1 0 W(s)ds − W(1) 2 =: Y.
- 31. 12 F. J. Breidt et al. Similarly, according to (2.7) in Theorem 2.1, we have lim β↓0 ∂ ∂α U(β, α) = lim β↓0 1 0 e−β(1−s) dW(s) + f(0)2α 1 0 e−2β(1−s) ds = 1 0 dW(s) + 2αf(0) 1 0 ds = W(1) + 2αf(0), and therefore α̂(0+) = − W(1) 2f(0) , which is same as α̂(0−). The derivative of U(β, α) with respect to β at zero from righthand side satisfies ∂ ∂β U(β, α) = − 1 0 1 s+ e−β(t−s) dS(t)dW(s) − β 1 0 1 s e−β(t−s) (s − t)dS(t)dW(s) + α 1 0 e−β(1−s) (s − 1)dW(s) + f(0) ' 2β 1 0 1 s e−β(t−s) dS(t) 2 ds + β2 1 0 2 1 s e−β(t−s) dS(t) × 1 s e−β(t−s) (s − t)dS(t) ds + α2 1 0 e−2β(1−s) 2(s − 1)ds − 2α 1 0 e−β(1−s) 1 s e−β(t−s) dS(t) ds − 2αβ 1 0 1 s e−β(1+t−2s) (2s − t − 1)dS(t)ds . Taking the limit β ↓ 0 and using the remark in Section 2, we have lim β↓0 ∂ ∂β U(β, α̂(β)) → − 1 0 1 s+ dS(t)dW(s) + α̂(0+) 1 0 (s − 1)dW(s) + f(0) α̂2 (0+) 1 0 2(s − 1)ds − 2α̂(0+) 1 0 1 s dS(t)ds = −S(1)W(1) + 1 0 S(s)dW(s) + 1 + α̂(0+) [(s − 1)W(s)]1 0 − 1 0 W(s)ds + f(0) −α̂2 (0+) − 2α̂(0+) S(1) − 1 0 S(s)ds = 1 0 S(s)dW(s) − W(1) 1 0 S(s)ds + W(1) 2f(0) 1 0 W(s)ds − W(1) 2 + 1 = Y + 1.
- 32. Non-invertible MA(1) 13 Therefore, the pile-up probability in (3.3) can be expressed in terms of Y as lim n→∞ P(θ̂(J) n = 1) = P [Y 0 and Y + 1 0] = P [−1 Y 0] . 3.2. Exact likelihood estimation In this section, we consider pile-up probabilities associated with the estimator that maximizes the exact Laplace likelihood. For θ ≤ 1, the joint density of (xn, zinit) satisfies f(xn, zinit) = n t=0 f(zt) = 1 2σ n+1 exp − n t=0 |zt| σ = 1 2σ n+1 exp − [n(θ, zinit) − n(1, Z0)] + n(1, Z0) σ = 1 2σ n+1 exp − n t=0 |Zt| σ e−Un(β,α) . Integrating out the augmented variable zinit, we obtain ∞ −∞ f(xn, zinit)dzinit = 1 2σ n+1 exp − n t=0 |Zt| σ σ √ n ∞ −∞ e−Un(β,α) dα, since under the parameterization (2.4), dzinit = (σ/ √ n)dα. The Laplace log-likeli- hood of (θ, σ) given xn then satisfies ∗ n(θ, σ) ≡ log ∞ −∞ f(xn, zinit)dzinit = −(n + 1) log(2σ) − n t=0 |Zt| σ + log σ √ n + log ∞ −∞ e−Un(β,α) dα, where the last term does not depend on σ as n → ∞. So maximizing ∗ n with respect to θ ≤ 1 is approximately the same as maximizing U∗ n(β) = log ∞ −∞ e−Un(β,α) dα (3.5) with respect to β ≤ 0, Similarly, for θ 1, the Laplace log-likelihood of (θ, σ) is ∗ n(θ, σ) ≡ log ∞ −∞ f(xn, zinit)dzinit = −n log |θ| − (n + 1) log(2σ) − n t=0 |Zt| σ|θ| + log σ √ n + log ∞ −∞ e−Un(β,α)|θ|−1 dα, where again the last term does not depend on σ as n → ∞. As above, maximizing ∗ n with respect to θ 1 is equivalent to maximizing U∗ n(β) = log ∞ −∞ e−Un(β,α)n/(n+β) dα (3.6)
- 33. 14 F. J. Breidt et al. for β 0. A heuristic argument based on the process convergence of Un to U suggests that U∗ n(β) → U∗ (β) = log ∞ −∞ e−U(β,α) dα , (3.7) where U∗ n is specified by (3.5) for β ≤ 0 and by (3.6) for β 0. Now if β̂ (E) n denotes the local maximum of the exact likelihood, or alternatively the maximizer of U∗ n(β) that is closest to 0, then the convergence in (3.7) suggests convergence in distribution for the local maximizer of the exact likelihood, i.e., n(θ̂(E) n − 1) = β̂(E) n d → β̂(E) , (3.8) where β̂(E) is the local maximizer of U∗ (β) that is closest to 0. The limiting pile-up probabilities for θ̂ (E) n are calculated from lim n→∞ P(θ̂(E) n = 1) = lim n→∞ P(β̂(E) n = 0) = P(β̂(E) = 0) = P lim β↑0 ∂ ∂β U∗ (β) 0 and lim β↓0 ∂ ∂β U∗ (β) 0 . Fortunately, the right- and left-hand derivatives of U∗ can be computed explicitly. These are found to be lim β↑0 ∂ ∂β U∗ (β) = − W2 (1) 4f(0) + W(1) 2f(0) 1 0 W(s)ds − W(1) 1 0 S(s)ds + 1 0 S(s)dW(s) + 1 2 = Y + 1 2 , lim β↓0 ∂ ∂β U∗ (β) = − W2 (1) 4f(0) + W(1) 2f(0) 1 0 W(s)ds − W(1) 1 0 S(s)ds + 1 0 S(s)dW(s) + 1 2 = Y + 1 2 , where Y is defined in (3.4). The limiting pile-up probability for θ̂ (E) n is then lim n→∞ P(θ̂(E) n = 1) = P − 1 2 Y − 1 2 = 0. 3.3. Remarks Here we collect several remarks concerning the results of Sections 3.1 and 3.2. Remark 1. Under the assumptions of Theorem 2.1, the asymptotic pile-up prob- ability for estimator θ̂ (J) n based on the joint likelihood is always positive. On the other hand, the asymptotic pile-up probability for estimator θ̂ (E) n based on the exact likelihood is zero.
- 34. Non-invertible MA(1) 15 Remark 2. The two estimators of θ0 considered in Sections 3.1 and 3.2 were defined as the local optimizers of objective functions that were closest to 1. One could also consider the global optimizers of these objective functions. For example, the exact MLE in the Gaussian case was considered in Davis and Dunsmuir [6] and Davis, Chen and Dunsmuir [5] and has a different limiting distribution than the local MLE. In our case, there will be a positive asymptotic pile-up probability for the global maximum of the joint likelihood and a zero asymptotic pile-up probability for the global maximum of the exact likelihood. Remark 3. Suppose Zt has a Laplace distribution with the density function fZ(z) = 1 2σ e−|z|/σ . Then Y defined in (3.4) satisfies Y = 1 0 [W(1)s − W(s)] dV (s) − 1 2 , (3.9) where W(s) and V (s) are independent standard Brownian motions. To prove (3.9), note that the constant c in (2.14) is equal to 1 so that S(t) = W(t) + V (t). In the following calculations, we use the well-known Itô formula 1 0 W(s)dW(s) = W2 (1) 2 − 1 2 . Since f(0) = 1/2, the random variable Y defined in (3.4) can be further simplified in terms of W(t) and V (t) as Y = 1 0 S(s)dW(s) − W(1) 1 0 S(s)ds + W(1) 2f(0) 1 0 W(s)ds − W(1) 2 = 1 0 V (s)dW(s) + 1 0 W(s)dW(s) − W(1) 1 0 V (s)ds − W(1) 1 0 W(s)ds +W(1) 1 0 W(s)ds − W2 (1) 2 = V (1)W(1) − 1 0 W(s)dV (s) + W2 (1) 2 − 1 2 − W(1) V (1) − 1 0 sdV (s) − W2 (1) 2 = 1 0 [W(1)s − W(s)] dV (s) − 1 2 .
- 35. 16 F. J. Breidt et al. Therefore, the pile-up probability for Laplace innovations is P (−1 Y 0) = P − 1 2 1 0 [W(1)s − W(s)] dV (s) 1 2 = E P − 1 2 1 0 [W(1)s − W(s)] dV (s) 1 2 $ $ $ $ W(t) on t ∈ [0, 1] = E P − 1 2 1 0 [W(1)s − W(s)]2 ds −1/2 U 1 2 1 0 [W(1)s − W(s)]2 ds −1/2 = E Φ 1 2 1 0 [W(1)s − W(s)]2 ds −1/2 − Φ − 1 2 1 0 [W(1)s − W(s)]2 ds −1/2 ≈ 0.820, where U has the standard normal distribution and Φ(·) is the corresponding cu- mulative distribution function. This pile-up probability, which was computed via simulation based on 100000 replications of W(t) on [0, 1], has a standard error of 0.0010. Remark 4. From the limiting result (3.2), it follows that the random variable Z0 can be estimated consistently. It may seem odd to have a consistent estimate of a noise term in a moving average process. On the other hand, an MA(1) process with a unit root is both invertible and non-invertible. That is, Z0 is an element of the two Hilbert spaces generated by the linear span of {Xt, t ≤ 0} and {Xt, t ≥ 1}, respectively. It is the latter Hilbert space which allows for consistent estimation of Z0. 4. Numerical simulation In this section, we compute the asymptotic pile-up probabilities associated with the estimator θ̂(J) which maximizes the joint Laplace likelihood for several dif- ferent noise distributions. The empirical properties of estimators θ̂ (J) n (the local maximizer of the joint Laplace likelihood) and θ̂ (E) n (the local maximizer of the exact Laplace likelihood) for finite samples are compared with each other and with the corresponding asymptotic theory. For approximating the asymptotic pile-up probabilities and limiting distribu- tion of β̂ (J) n , we first simulate 100000 replications of independent standard Wiener processes W(t) and V (t) on [0, 1] in which W(t) and V (t) are approximated by the partial sums W(t) = [10000t] j=1 Wj/ √ 10000 and V (t) = [10000t] j=1 Vj/ √ 10000, where {Wj} and {Vj} are independent standard normal random variables. From the simulation of W(t) and V (t), the distribution of the limit random variable β̂(J) can be tabulated and the pile-up probability P(−1 Y 0) estimated, where Y is given in (3.4). The empirical pile-up probabilities and their asymptotic limits are
- 36. Non-invertible MA(1) 17 displayed in Table 1 for different noise distributions: Laplace, Gaussian, uniform, and t with 5 degrees of freedom. Notice that there is good agreement between the asymptotic and empirical probabilities for sample sizes as small as 50. For examining the empirical performance of the local maximizers θ̂ (J) n and θ̂ (E) n , we only consider the process generated with Laplace noise with σ = 1 and sample sizes n = 20, 50, 100, 200. For each setup, 1000 realizations of the MA(1) process with θ0 = 1 are generated and the estimates θ̂ (J) n and θ̂ (E) n and their corresponding estimates of the scale parameter are obtained. The estimation results are sum- marized in Table 2. For comparison, the standard deviation based on the limit distributions of θ̂ (J) n and θ̂ (E) n are also reported (denoted by asymp in the table), which are obtained numerically based on 100000 replicates of the limit process U. Generally speaking, the empirical root mean square errors are very close to their asymptotic values even for very small samples. Moreover, the estimation error of θ̂ (J) n is about 1/2 the estimation error of θ̂ (E) n , which indicates the superiority of using the joint likelihood over exact likelihood when θ0 = 1. We also considered performance of the two estimators θ̂ (J) n and θ̂ (E) n in the case when θ0 = 1. A limit theory for these estimators can be derived in this case by assuming that the true value θ0 is near 1. That is, we can parameterize the MA(1) parameter by θ0 = 1 + γ/n (e.g., Davis and Dunsmuir [6]). While we have not pursued the theory in the near unit root case, the relative performance of these Table 1 Empirical pile-up probabilities of the local maximizer θ̂ (J) n of the joint Laplace likelihood for an MA(1) with θ0 = 1 and sample sizes n = 20, 50, 100, 200 (based on 1000 replicates) and their asymptotic values under various noise distributions. n Gau Lap Unif t(5) 20 0.827 0.796 0.831 0.796 50 0.859 0.806 0.864 0.823 100 0.873 0.819 0.864 0.817 200 0.844 0.819 0.843 0.831 500 0.855 0.809 0.841 0.846 ∞ 0.873 0.820 0.862 0.836 Table 2 Bias, standard deviation and root mean square error of the local maximizers θ̂ (J) n and θ̂ (E) n of the joint and exact Laplace likelihoods, respectively, for an MA(1) process generated by Laplace noise with θ0 = 1 and σ = 1 ( 1000 replications). n θ̂ (J) n θ̂ (E) n n = 20 bias -0.003 -0.006 s.d. 0.066 0.144 rmse 0.066 0.144 asymp 0.053 0.121 n = 50 bias -0.000 0.000 s.d. 0.021 0.057 rmse 0.021 0.057 asymp 0.021 0.048 n = 100 bias -0.000 0.001 s.d. 0.011 0.030 rmse 0.011 0.030 asymp 0.011 0.024 n = 200 bias 0.000 0.001 s.d. 0.006 0.014 rmse 0.006 0.014 asymp 0.005 0.012
- 37. 18 F. J. Breidt et al. Table 3 Bias, standard deviation and root mean square error of the global maximizers θ̂ (J) n and θ̂ (E) n of the joint and exact Laplace likelihoods, respectively, for an MA(1) process generated by Laplace noise with θ0 = 0.8, 0.9, 0.95, 1/0.95, 1/0.9, 1/0.8, σ = 1, and n = 50 based on 1000 replications. First 2 columns record the number of times (out of 1000) that the estimates were less than 1 (invertible) and equal to 1 (unit root). θ0 1 = 1 bias s.d. rmse 0.8 θ̂ (J) 50 789 95 0.0734 0.1973 0.2105 θ̂ (E) 50 873 19 0.0498 0.1753 0.1822 0.9 θ̂ (J) 50 557 322 0.0578 0.1398 0.1513 θ̂ (E) 50 767 93 0.0327 0.0933 0.0989 0.95 θ̂ (J) 50 404 503 0.0322 0.0708 0.0778 θ̂ (E) 50 632 168 0.0235 0.0821 0.0854 1/0.95 θ̂ (J) 50 90 540 -0.0315 0.0763 0.0825 θ̂ (E) 50 286 114 -0.0207 0.0890 0.0914 1/0.9 θ̂ (J) 50 89 299 -0.0389 0.1227 0.1287 θ̂ (E) 50 207 71 -0.0327 0.1218 0.1261 1/0.8 θ̂ (J) 50 96 109 -0.0338 0.2645 0.2666 θ̂ (E) 50 149 19 -0.0492 0.2280 0.2333 estimators was compared in a limited simulation study. We considered 3 values of θ0 = 0.8, 0.9, 0.95 and their reciprocals 1/0.8, 1/0.9, 1/0.95. The latter 3 cases cor- respond to purely non-invertible models. The results reported in Table 3 are based on the global optimization of the joint and exact likelihoods. The first two columns contain the number of realizations out of 1000 in which the estimator was invertible ( 1) and on the unit circle (= 1), respectively. For example, in the θ0 = 0.8 and θ̂ (J) n case, 78.9% of the realizations produced invertible models, and the empirical pile-up probability is 0.095. On the other hand, for θ0 = 1/0.8, 79.5% of the realiza- tions produced a purely non-invertible model with an empirical pile-up probability of 0.109. Both objective functions do a reasonably good job of discriminating be- tween invertible and non-invertible models, with a performance edge going to the exact likelihood. In terms of root mean square error, the performance of θ̂ (E) n is superior to θ̂ (J) n as θ0 moves away from the unit circle. Remark. The LAD estimate of θ0 is obtained by minimizing the objective function given in (2.2) with zinit = 0. Although we have not considered the asymptotic pile- up in this case, the estimator does not perform as well as θ̂ (J) n and θ̂ (E) n . For example, in simulation results, not reported here, the rmse of the LAD estimator tended to be twice as large as the rmse for the exact MLE. References [1] Anderson, T. W. and Takemura, A. (1986). Why do noninvertible esti- mated moving averages occur? Journal of Time Series Analysis 7 235–254. [2] Breidt, F. J. and Davis, R. A. (1992). Time-reversibility, identifiably, and independence of innovations for stationary time series. Journal of Time Series Analysis 13 377–390. [3] Brockwell, P. J. and Davis, R. A. (1991). Time Series: Theory and Meth- ods, 2nd Edition. Springer-Verlag, New York.
- 38. Non-invertible MA(1) 19 [4] Chan, N. H. and Wei, C. Z. (1988). Limiting distributions of least squares estimates of unstable autoregressive processes. Annals of Statistics 16 367–401. [5] Chen, M., Davis, R. A. and Dunsmuir, W. T. M. (1995). Inference for MA(1) processes with a root on or near the unit circle. Invited paper in Proba- bility and Mathematical Statistics, Issue in Honour of Neyman’s 100 Birthday 15 227–242. [6] Davis, R. A. and Dunsmuir, W. T. M. (1996). Maximum likelihood esti- mation for MA(1) processes with a root on or near the unit circle. Econometric Theory 12 1–29. [7] Tanaka, K. (1996). Time Series Analysis. Nonstationary and Noninvertible Distribution Theory. Wiley, New York.
- 39. IMS Lecture Notes–Monograph Series Time Series and Related Topics Vol. 52 (2006) 20–47 c Institute of Mathematical Statistics, 2006 DOI: 10.1214/074921706000000932 Recursive estimation of possibly misspecified MA(1) models: Convergence of a general algorithm James L. Cantor1 and David F. Findley2 Science Application International Corporation and U.S. Census Bureau Abstract: We introduce a recursive algorithm of conveniently general form for estimating the coefficient of a moving average model of order one and obtain convergence results for both correct and misspecified MA(1) models. The algorithm encompasses Pseudolinear Regression (PLR—also referred to as AML and RML1) and Recursive Maximum Likelihood (RML2) without monitoring. Stimulated by the approach of Hannan (1980), our convergence results are obtained indirectly by showing that the recursive sequence can be approximated by a sequence satisfying a recursion of simpler (Robbins-Monro) form for which convergence results applicable to our situation have recently been obtained. 1. Introduction and overview Our focus is on estimating the coefficient θ of an invertible scalar moving average model of order 1 (MA(1)), (1.1) yt = θet−1 + et where et is treated as an unobserved, constant-variance martingale-difference pro- cess. We do not assume the series yt, −∞ t ∞ from which the observations come is correctly modeled by (1.1). They can come from any invertible autoregres- sive moving average (ARMA) model or from more general models; see Section 2. What we seek is a θ that minimizes the loss function (1.2) L̄(θ) = E[(yt − yt|t−1(θ))2 ] = E[e2 t (θ)] where et(θ) = yt −yt|t−1(θ) and yt|t−1(θ) is the one-step-ahead-prediction of yt from ys, −∞ s ≤ t − 1 based on the model defined by θ (see (2.7) below). We define optimal estimation procedures to be those whose sequence of estimates θt minimizes (1.2) in the limit. This is a property of (nonrecursive) maximum likelihood-type estimates of θ, see Pötscher [23]. In this article, we analyze a continuously indexed family of recursive procedures for estimating θ. Recursive procedures form an estimate θt for time t using the observation yt at time t, the estimate θt−1 for t−1 and other recursively defined quantities. Our family encompasses two standard algorithms, Recursive Maximum 1Science Applications International Corporation (SAIC), 4001 North Fairfax Drive, Suite 250, Arlington, VA 22203, e-mail: james.l.cantor@saic.com 2U.S. Census Bureau, Statistical Research Division, Room 3000-4, Washington, DC 20233- 9100, e-mail: david.f.findley@census.gov AMS 2000 subject classifications: primary 62M10; secondary 62L20. Keywords and phrases: time series, Robbins-Monro, PLR, AML, RML1, RML2, misspecified models. 20
- 40. Recursive estimation of possibly misspecified MA(1) models 21 Likelihood (RML) which is referred to throughout as RML2 [12, 21], and the sim- pler Pseudolinear Regression (PLR) [21]—also known as Approximate Maximum Likelihood (AML) [24] and RML1 [11, 20]. More specifically, our general recursive algorithm generating θt depends on an index β, 0 ≤ β ≤ 1. The algorithm reduces to PLR when β = 0 and to RML2 when β = 1. Our main convergence result, Theorem 4.1, is obtained by constructing an ap- proximating sequence θ̂t for which θt −θ̂t a.s. −→ 0 holds and which satisfies a Robbins- Monro recursion, (1.3) θ̂t = θ̂t−1 − δtf(θ̂t−1, β) + δtγt , in which γt a.s. −→ 0 and δt 0, δt a.s. −→ 0, ∞ k=0 δk = ∞ a.s., and (1.4) f(θ, β) = − π −π eiω + βθ |(1 + θeiω)(1 + θβeiω)|2 gy(ω)dω. Here a.s. −→ denotes almost sure convergence (convergence with probability one) and gy(ω) denotes the spectral density of the time series yt. Note that when β = 0, then (1.5) f(θ, 0) = − π −π eiω |(1 + θeiω)|2 gy(ω)dω = −E[et−1(θ)et(θ)] , and when β = 1, then (1.6) f(θ, 1) = − π −π eiω + θ |(1 + θeiω)2|2 gy(ω)dω = 1 2 d dθ E[e2 t (θ)] = 1 2 L̄ (θ) where L̄ (θ) denotes the first derivative of L̄(θ). We then apply a result of Fradkov implicit in [8], as extended and corrected by Findley [9], to show that θ̂t converges to {θ ∈ Θ : f(θ, β) = 0} where Θ is the open interval (-1,1) of real θ with |θ| 1. (A similar result is implicit in proofs of Theorems 2.2.2–2.2.3 of Chen [7].) Hence, for β = 0, θt a.s. −→ {θ ∈ Θ : E[et−1(θ)et(θ)] = 0} and for β = 1, θt a.s. −→ {θ ∈ Θ : L̄ (θ) = 0}. Here and below, θt convergence a.s. to a set means that except on a set of ξ ∈ Ξ with probability zero, every cluster point of θt(ξ) is an element of the set. In the incorrect model situation, in which gy(ω) is not proportional to |1+θeiω |2 , for examples we have analyzed [5], these zero sets will be disjoint, establishing that PLR converges to different values than RML2. Consequently, under the assumptions of Theorem 4.1, we recover the results of Cantor [4] that were given in separate theorems and proofs, establishing that, for certain families of AR(1) and MA(2) processes, RML2 estimates of θ in the model (1.1) converge to an optimal limit (a minimizer of (1.2)) whereas PLR estimates converge to a suboptimal limit [4, 5]. When the data come from an invertible MA(1) model, it is known that PLR and monitored versions of RML2 can provide strongly consistent estimates of θ [4, 11, 17, 19]. More generally, in the correct model situation for ARMAX models, i.e., ARMA models with an exogenous input, Lai and Ying [17] provided a rigorous proof of strong consistency of PLR (under a positive real condition on the MA polynomial) and also of a monitored version of RML2 whose monitoring scheme involves non-linear projections and an intermittently used recursive estimator for which consistency has already been established. In Section 4 of [19], Lai and Ying consider a simpler modification of RML2 in which, for monitoring, only auxiliary consistent recursive estimates are used. They present detailed outlines of proofs of strong consistency and asymptotic normality of the estimates from this new
- 41. 22 J. L. Cantor and D. F. Findley monitored RML2 scheme. The construction of Section II of [18] can be used to obtain auxiliary recursive estimates with the properties required. There is a rather comprehensive theory of recursive estimation of autoregressive (AR) models, encompassing certain incorrect model situations for algorithms like PLR (see e.g., [6]). There are, however, no published convergence results with rig- orous proofs for MA models in the incorrect model situation. Ljung’s seminal work on the convergence of recursive algorithms [20, 21] mentions the incorrect model situation but provides only suggestive results (further discussed in Section 5). This article has five sections. In Section 2, the assumptions on the data and some consequences for the MA(1) model are given. In Section 3, the general recursive algorithm is presented. The Convergence Theorem is stated and proved in Section 4. Required preliminary technical results are given in Section 4.1 and the proof of the theorem is provided in Section 4.2. Finally, Section 5 concludes the article with a brief discussion. 2. Assumptions The observations yt, t ≥ 1 are assumed to come from a mean zero, covariance stationary scalar series, yt, −∞ t ∞ defined on the probability space (Ξ, F, P). We use the following additional assumptions on the process yt: (D1) y1 is nonzero with probability one; i.e., P{y2 1 0} = 1. (D2) The series has a linear representation (2.1) yt = ∞ s=0 κst−s such that κ0 = 1 and ∞ s=0 |κs| ∞ in which κ(z) = ∞ s=0 κszs is nonzero for |z| ≤ 1 and {t} is a martingale- difference sequence (m.d.s.) with respect to the sequence of sigma fields Ft = σ(ys, −∞ s ≤ t). Thus E[t|Ft−1] = 0. By a result of Wiener [25, Theorem VI 5.2], κ(z)−1 = ∞ s=0 βszs with ∞ s=0 |βs| ∞, whence (2.2) t = ∞ s=0 βsyt−s (β0 = 1) . (D3) The conditional variance E[2 t |Ft−1] is constant almost surely; i.e., E[2 t |Ft−1] = σ2 a.s. Equivalently, E[2 t ] = σ2 and 2 t − σ2 is a m.d.s. with respect to the Ft. (D4) {t} is bounded a.s.; supt |t| ≤ K a.s. for some K ∞. From (D2)–(D3), the spectral density gy(ω) can be expressed as (2.3) gy(ω) = σ2 2π κ(eiω ) 2 where κ(eiω ) = ∞ j=0 κjeijω , and (2.4) 0 m ≤ gy(ω) ≤ M ∞ for all −π ≤ ω ≤ π for positive constants m and M. The series yt is an invertible ARMA process if and only if κ(z) is a rational function.
- 42. Recursive estimation of possibly misspecified MA(1) models 23 Assumption (D4) is used extensively in the proof of the convergence theorem, Theorem 4.1, in Section 4. Under (D2)–(D4), we can apply, for example, the First Moment Bound Theorem of Findley and Wei [10] to show that t−1 t s=j+1(ysys−j − γy j ) a.s. −→ 0. Hence, from the particular case yt = t in (2.1) and j = 0, (2.5) t−1 t s=1 2 s a.s. −→ σ2 . We consider models for yt of the invertible, stationary first-order moving-average type (MA(1)) given by (2.6) yt = θet−1 + et, −∞ t ∞ . For a given coefficient θ such that |θ| 1, the difference equation (2.6) is satisfied with et = et(θ) given by the mean zero, covariance stationary one-step-ahead- prediction-error series, (2.7) et(θ) = (1 + θB)−1 yt = ∞ j=0 (−θ)j yt−j = yt − yt|t−1(θ) , from the MA(1) predictor yt|t−1(θ) = − ∞ j=1(−θ)j yt−j, see (5.1.21) of [3]. Here B is the backshift operator; i.e., Byt = yt−1. The coefficient θ is referred to as the MA coefficient. Thus, (2.8) yt = et(θ) + θet−1(θ) . The infinite series in (2.7) converges in mean square and, from (D4) and the rep- resentation (2.1), also almost surely. Thus, et(θ) represents the optimal one-step- ahead-prediction-error process from the perspective of the model (2.6). The model (2.6) is correct if et(θ) coincides (a.s.) with the m.d.s. t in (2.2), in which case βs = (−θ)s , k ≥ 0. Whether or not the model is correct for any θ, forecast errors et(θ) appearing in loss functions such as (1.2) and elsewhere are calculated as in (2.7). We emphasize that (2.1) allows data processes far more general than MA(1) processes. In particular, the z-transform, ∞ s=0 κszs is not required to be rational. For example, time series conforming to the exponential models of Bloomfield [2] have non-rational κ(z) without zeroes in |z| ≤ 1. Let Θ = (−1, 1). From (2.7), the spectral density of et(θ) is ge(θ, ω) = gy(ω) · |1 + θeiω |−2 , so for L̄(θ) defined by (1.2), we have (2.9) L̄(θ) = π −π gy(ω) |1 + θeiω|2 dω . By (2.4) and the continuity of gy(ω), L̄(θ) is positive, infinitely differentiable, and nonconstant on the interior of [−1, 1], i.e., on Θ, and infinite at the endpoints. Therefore it has a minimum value over [−1, 1] and (2.10) Θ∗ ≡ θ ∈ [−1, 1]: θ = arg min θ∈[−1,1] L̄(θ) , is a subset of [−K, K] for some 0 K 1. Also Θ∗ ⊆ Θ∗ 0 = {θ ∈ Θ: L̄ (θ) = 0}. We are interested in a.s bounded random recursive sequences θt = θt(ξ) that converge
- 43. 24 J. L. Cantor and D. F. Findley a.s. to Θ∗ or at least to Θ∗ 0. If Θ∗ 0 contains only one point, θ∗ 0, then θt converges to θ∗ 0 a.s. Our results will establish convergence of the sequence of estimates θt defined by the general algorithm presented below to the set of zeroes of f(θ, β) defined by (1.4). 3. The general recursive algorithm For 0 ≤ β ≤ 1, we define a general recursion for estimating the MA coefficient θ of (1.1): θt = θt−1 + P̄−1 t 1 t φt−1et; θ1 = 0, t ≥ 2 , (3.1a) P̄t = 1 t t−1 s=1 φ2 s = P̄t−1 + 1 t [φ2 t−1 − P̄t−1]; P̄1 = 0; t ≥ 2 , (3.1b) et = yt − θt−1et−1; e1 = y1, t ≥ 2 , (3.1c) φt = xt − θt−1φt−1; φ1 = x1, t ≥ 2 , (3.1d) xt = yt − βθt−1xt−1; x1 = y1, t ≥ 2 . (3.1e) From (3.1a), it follows for 0 ≤ s ≤ t − 1, t ≥ 2 that (3.2) θt−s = θt − s−1 l=0 (t − l)−1 P̄−1 t−lφt−l−1et−l , where −1 l=0(·) ≡ 0. From (3.1e), (3.3) xt = t−1 s=0 (−β)s s i=1 θt−i yt−s where 0 i=1(·) ≡ 1. Next, let z1 = e1 and, for t ≥ 2, (3.4) zt = et + θt−1φt−1 . The value of the parameterization with β is that it enables us to simultaneously obtain results for two important algorithms. When β = 0, then xt = yt from which it follows that φt = et and zt = yt and therefore (3.1a)–(3.1e) is PLR (AML, RML1)[11, 20, 21, 24]. When β = 1, then xt = et and φt = et − θt−1φt−1 and thus (3.1a)–(3.1e) is RML2 [12, 21] without monitoring to ensure that each estimate θt is in Θ = (−1, 1). For any β, these θt can be expressed in the form of a regression estimate: (3.5) θt = t s=2 φ2 s−1 −1 t s=2 zsφs−1, t ≥ 2 . An induction argument for (3.5) goes as follows. Set Pt = tP̄t = t s=2 φ2 s−1. Note that from (D1), Pt 0 for all t 1 and therefore P−1 t exists a.s. From (3.1a)–(3.1e) and (3.4), θ2 = 1/2φ2 1 −1 1/2(z2φ1) , which is (3.5) for t = 2. Suppose then it is true for some t ≥ 2; i.e., (3.6) Ptθt = t s=2 zsφs−1 .
- 44. Recursive estimation of possibly misspecified MA(1) models 25 Then Pt+1θt+1 = Pt+1(θt + P−1 t+1φtet+1) = (Pt + φ2 t )θt + φtet+1 = t s=2 zsφs−1 + φt(φtθt + et+1) (from the induction hypothesis (3.6)) = t s=2 zsφs−1 + φtzt+1 = t+1 s=2 zsφs−1 . Hence, (3.5) is true for t + 1 and by induction therefore for all t. For use below, we define the stationary analogues et(θ), xt(θ), φt(θ) and zt(θ) of et, xt, φt and zt: et(θ) = (1 + θB)−1 yt , (3.7) xt(θ) = (1 + θβB)−1 yt = ∞ j=0 (−βθ)j yt−j , (3.8) φt(θ) = (1 + θB)−1 xt(θ) = ∞ j=0 (−θ)j xt−j(θ) (3.9) = (1 + θB)−1 (1 + θβB)−1 yt , so φt(θ) = et(θ) when β = 0. From (3.7)–(3.9), (3.10) zt(θ) = et(θ) + θφt−1(θ) = [(1 + θB)−1 + θB(1 + θB)−1 (1 + θβB)−1 ]yt . From (3.7)–(3.10), E[φ2 t (θ)] = π −π 1 |(1 + θeiω)(1 + βθeiω)|2 gy(ω)dω , (3.11) E[φt−1(θ)et(θ)] = π −π eiω (1 + θeiω)(1 + βθeiω) 1 (1 + θe−iω) gy(ω)dω (3.12) = π −π eiω + βθ |(1 + θeiω)(1 + βθeiω)|2 gy(ω)dω , and (3.13) E[zt(θ)φt−1(θ)] = π −π eiω + θ(1 + β) |(1 + θeiω)(1 + βθeiω)|2 gy(ω)dω . From (1.4) and (3.12), E[φt−1(θ)et(θ)] = −f(θ, β). Let e t(θ) = det(θ)/dθ. Then, from (3.7), (3.14) −e t(θ) = B 1 + θB et(θ) = B (1 + θB)2 yt . Since 1 2 d dθ E[e2 t (θ)] = E[e t(θ)et(θ)] , from (2.9) and (3.14), the derivative of L̄(θ), L̄ (θ), is obtained from − 1 2 L̄ (θ) = E[−e t(θ)et(θ)] = π −π eiω (1 + θeiω)2 1 (1 + θe−iω) gy(ω)dω (3.15) = π −π eiω + θ |(1 + θeiω)2|2 gy(ω)dω ,
- 45. 26 J. L. Cantor and D. F. Findley which is (3.12) with β = 1, verifying (1.6). As a consequence of (2.4), we note that since |z| ≤ K∗ 1 implies 0 1−K∗ ≤ |1 − z| ≤ 1 + K∗ , for (3.11) with |θ| ≤ K∗ 1 we have (3.16) m (1 + K∗) 4 ≤ π −π 1 |(1 + θeiω)(1 + βθeiω)|2 gy(ω)dω ≤ M (1 − K∗) 4 . 4. The convergence theorem The following result is a generalization of the PLR and RML2 results proved in [4] for MA(1) models. Theorem 4.1 (Convergence theorem). Consider a series yt for which (D1)– (D4) hold. For each β such that 0 ≤ β ≤ 1, assume that the recursive sequence defined by (3.1a)–(3.1e) is such that, for some random k∗ = k∗ (ξ) and K∗ = K∗ (ξ)(ξ ∈ Ξ) satisfying 0 ≤ k∗ ∞ and 0 K∗ 1 , it holds almost surely that |θt+k∗ | ≤ K∗ for all t. Then for f(θ, β) as in (1.4): (a) The sequence θ̂t defined for t ≥ 1 by θ̂t = 1 t t s=1 π −π 1 |(1 + θs+k∗ eiω)(1 + βθs+k∗ eiω)|2 gy(ω)dω −1 (4.1) × 1 t t s=1 π −π cos ω + (1 + β)θs+k∗ |(1 + θs+k∗ eiω)(1 + βθs+k∗ eiω)|2 gy(ω)dω has the property that θt − θ̂t a.s. −→ 0. Hence, with probability one, there is a t0(ξ) ≥ 1 such that |θ̂t| ≤ (1 + K∗ )/2 1 holds for all t ≥ t0(ξ). (b) For all t t0(ξ), θ̂t satisfies a Robbins-Monro recursion, (4.2) θ̂t = θ̂t−1 − δtf(θ̂t−1, β) + δtγt , with γt a.s. −→ 0, δt 0 a.s., δt a.s. −→ 0, and ∞ s=t0+1 δs = ∞ a.s. where f(θ, β) has the formula (1.4). (c) From (a) and (b), it follows that, with Θ = (−1, 1), the sequence θt converges a.s. to the compact set (4.3) Θβ 0 = {θ ∈ Θ : f (θ, β) = 0} in the sense that, on a probability one event Ξ0 that does not depend on β, for each ξ ∈ Ξ0, the cluster points of θt(ξ) are contained in Θβ 0 . Further, when yt is an invertible ARMA process, then Θβ 0 is finite, and θ(ξ) = limt→∞ θt(ξ) exists for every ξ ∈ Ξ0. Note from (3.5), (3.11) and (3.13) that the assertion θt − θ̂t a.s. −→ 0 in part (a) of Theorem 4.1 can be formulated as the assertion that 1 t t s=1 φ2 s−1 −1 1 t t s=1 zsφs−1 − 1 t t s=1 E[φ2 t (θs+k∗ )] −1 1 t t s=1 E[zt(θs+k∗ )φt−1(θs+k∗ )]
- 46. Recursive estimation of possibly misspecified MA(1) models 27 tends to zero a.s. In the expression above, φ0 = 0 and expectation is taken before evaluation at θs+k∗ . The proof of Theorem 4.1, given in Section 4.2. In [5], we provide complete results concerning the existence of k∗ and K∗ with the required properties for several incorrect model examples as well as for the correct model situation for β = 0 (PLR) and provide more limited results for the case β = 1 (RML2) with a particular monitoring scheme. For the latter case, we also report on simulation results which demonstrate the existence of the variates k∗ , K∗ as in Theorem 4.1 with the consequence that monitoring becomes unnecessary for sufficiently large t. In the correct model case yt = θt−1 + t with i.i.d. t, Lai and Ying [19] show for their monitored RML2 that this happens a.s. and the conclusions of Theorem 4.1 concerning our approximating sequence (4.1) apply. 4.1. Preliminary results Here we present some needed technical results. We first quote, without proof, a powerful result from martingale theory [17, Lemma 1, part (i)]. Unless specified otherwise, all limits (liminfs, limsups, etc.) are with respect to t and for simplicity the t → ∞ will be usually suppressed. Proposition 4.1. Let {˜ t} be a martingale difference sequence with respect to an increasing sequence of σ-fields {Ft} such that supt E[|˜ t|2p |Ft−1] ∞ holds a.s. for some p 1. Let z̃t be an Ft−1-measurable random variable for every t. Then t s=1 z̃s˜ s converges almost surely on { ∞ s=1 z̃2 s ∞}, and for every η 1/2, t s=1 z̃s˜ s t s=1 z̃2 s η a.s. −→ 0 on ∞ s=1 z̃2 s = ∞ . Since 1 t t s=1 z̃s˜ s = t s=1 z̃s˜ s t s=1 z̃2 s 1 t t s=1 z̃2 s , it is clear that a corollary of this Proposition is Proposition 4.2. Under the assumptions of Proposition 4.1, if lim sup t−1 × t s=1 z̃2 s ∞ a.s., then t−1 t s=1 z̃s˜ s a.s. −→ 0. Recall from (2.1) that yt = t + ∞ s=1 κst−s since κ0 = 1. A second consequence of Proposition 4.1 is Proposition 4.3. Suppose that the m.d.s. t in (D2) is such that supt E[|t|2p | Ft−1] ∞ holds a.s. for some p 1. Then for any sequence ŷt = yt − ỹt−1 in which ỹt−1 is Ft−1-measurable, it holds that lim inf t−1 t s=1 ŷ2 s ≥ σ2 a.s., where σ2 = E[2 t ]. Proof. From (2.1), ŷt = yt − ỹt−1 = t + z̃t where z̃t = −ỹt−1 + ∞ s=1 κst−s is Ft−1-measurable since ∞ s=1 κst−s is Ft−1-measurable by (2.2) and ỹt−1 is Ft−1- measurable by assumption. Then 1 t t s=1 ŷ2 s = 1 t t s=1 2 s + 2 t t s=1 sz̃s + 1 t t s=1 z̃2 s (4.4) = 1 t t s=1 2 s + 2 t s=1 sz̃s t s=1 z̃2 s + 1 1 t t s=1 z̃2 s .
- 47. 28 J. L. Cantor and D. F. Findley Consider first the event that t s=1 z̃2 s a.s. −→ l ∞. Then t−1 t s=1 z̃2 s a.s. −→ 0 and, by the preceding Proposition, t−1 t s=1 sz̃s a.s. −→ 0. Hence, from (2.5) and the first equation in (4.4), lim t−1 t s=1 ŷ2 s = t−1 t s=1 2 s = σ2 so the assertion holds in this event. In the complementary event, t s=1 z̃2 s a.s. −→ ∞, from (4.4), it follows that lim inf 1 t t s=1 ŷ2 s = lim inf 1 t t s=1 2 s + 2 t s=1 sz̃s t s=1 z̃2 s + 1 1 t t s=1 z̃2 s (4.5) = σ2 + lim inf 2 t s=1 sz̃s t s=1 z̃2 s + 1 1 t t s=1 z̃2 s a.s. By Proposition 4.1, t s=1 sz̃s/ t s=1 z̃2 s a.s. −→ 0. Hence, the second expression in (4.5) is nonnegative, and the proof is complete. Proposition 4.4. Under (2.4), for each β ∈ [0, 1], the function f (θ, β) defined by (1.4) is infinitely differentiable on Θ = (−1, 1), and Θβ 0 defined by (4.3) is a nonempty compact subset of Θ. In the case β = 1, Θ1 0 contains the (nonempty) set of minimizers over Θ of L̄ (θ) defined by (2.9). Proof. The differentiability assertion follows from (2.4) via the dominated conver- gence theorem. Except for compactness of Θ1 0, which will be discussed below, the assertions concerning L̄ (θ) and f (θ, 1) were obtained subsequent to (2.10). The remaining assertions follow from the continuity of f (θ, β) and the limit properties (4.6) lim θ→−1 f (θ, β) = −∞ and (4.7) lim θ→1 f (θ, β) = ∞. Indeed, from (4.6)–(4.7), for any K 0 there exists an 0 (K, β) 1 such that f(θ, β) ≤ −K for all θ ∈ (−1, −1+) and f(θ, β) ≥ K for all θ ∈ (1−, 1). Therefore f(θ, β) must change sign over [−1 + ε, 1 − ε]. Hence f(θ, β) is non-constant and has a zero in this interval and, moreover, Θβ 0 ⊆ [−1 + ε, 1 − ε]. Finally, since f(θ, β) is continuous on this interval, Θβ 0 is compact. An analogous argument applies to Θ1 0. To verify (4.6), we note that gy(ω) = gy(−ω), −π ≤ ω ≤ π yields f (θ, β) = − π −π cos ω + βθ |(1 + θeiω) (1 + βθeiω)| 2 gy (ω) dω. Because 0 ≤ β 1, for 0 ε 1 − β there is a δ = δ(ε) ∈ (0, π) such that cos ω + βθ ≥ ε whenever |ω| ≤ δ and −1 ≤ θ ≤ 0. For such ε, δ, we obtain lim θ→−1 π −π cos ω + βθ |(1 + θeiω) (1 + βθeiω)| 2 gy (ω) dω = −δ −π + π δ cos ω + βθ |(1 − eiω) (1 − βeiω)| 2 gy (ω) dω (4.8) + lim θ→−1 δ −δ cos ω + βθ |(1 + θeiω) (1 + βθeiω)| 2 gy (ω) dω (4.9) = ∞,
- 48. Recursive estimation of possibly misspecified MA(1) models 29 because (4.8) is finite, whereas for (4.9) we have lim θ→−1 δ −δ cos ω + βθ |(1 + θeiω) (1 + βθeiω)| 2 gy (ω) dω ≥ ε m lim θ→−1 δ −δ 1 + θeiω 1 + βθeiω −2 dω = ∞. This yields (4.6), and (4.7) follows by an analogous argument. Proposition 4.5. Let yt be an invertible ARMA process, then for each β ∈ [0, 1], the set Θβ 0 = {θ ∈ (−1, 1) : f(θ, β) = 0} is finite. Proof. κ (z) in (D2) has the form κ (z) = η (z) /φ (z) where η (z) and φ (z) are polynomials, of degrees dη and dφ, respectively, having no common zeroes and having all zeros in {|z| 1}. Setting z = eiω and h (z) = (1 + θz) (1 + βθz), we obtain from dz = izdω that −f(θ, β) = π −π eiω + βθ |(1 + θeiω) (1 + βθeiω)| 2 gy (ω) dω = σ2 ε 2πi |z|=1 (z + βθ) η (z) η z−1 zh (z) h (z−1) φ (z) φ (z−1) dz = σ2 ε 2πi |z|=1 z1+dφ−dη (z + βθ) η (z) zdη η z−1 h (z) {z2h (z−1)} φ (z) {zdφ φ (z−1)} dz. The function w (z) = σ2 ε z1+dφ−dη (z + βθ) η (z) zdη η z−1 h (z) {z2h (z−1)} φ (z) {zdφ φ (z−1)} is nonzero on {|z| = 1} and has poles interior to the unit circle at −θ, −βθ, at the zeroes of zdη φ z−1 , and, if 1 + dφ − dη 0, also at 0. If zj, j = 1, . . . , n are the distinct poles in {z : |z| 1}, then, by the Residue Theorem of complex analysis, e.g., (4.7-10) of Henrici [13], it follows that f (θ, β) = − n j=1 Resz=zj w (z) , where, if zj is a pole of order J ≥ 1, Resz=zj w (z) = 1 (J − 1)! lim z→zj dJ−1 dzJ−1 (z − zj) J w (z) . Thus each Resz=zj w (z) is a rational function of θ, and therefore the same is true of f (θ, β). Consequently, f (θ, β) = 0 holds for only finitely many θ in (−1, 1). The final preliminary result addresses convergence of a Robbins-Monro type recursion that will be applied to demonstrate convergence of the general recursive algorithm. It is a special case of a correction and extension by Findley [9] of a result that is implicit in the proof of a theorem of Fradkov presented in Derevitzkiĭ and Fradkov [8] for the case of monotonically decreasing δt. The result below is also implicit in the proofs of Theorem 2.2.2 and Corollary 2.2.1 of Chen [7] which cover the case of vector θ more completely than Findley [9].
- 49. 30 J. L. Cantor and D. F. Findley Proposition 4.6. Let θ̂t, t ≥ t0 be a non-stochastic, real-valued sequence satisfying θ̂t = θ̂t−1 − δtf(θ̂t−1) + δtγt, t t0 for some real-valued function f(θ), with γt, t t0 satisfying γt → 0 and with δt, t ≥ t0 satisfying δt ≥ 0, δt → 0, and ∞ t=t0+1 δt = ∞ . Suppose there is a bounded open set Θ̃ on which f (θ) is continuously differentiable and which is such that the sequence θ̂t enters Θ̃ infinitely often and has no cluster point on the boundary of Θ̃. Then θ̂t is bounded, and its cluster points belong to Θ̃0 = {θ ∈ Θ̃: f(θ) = 0}, i.e., θ̂t → Θ̃0. The set of cluster points is compact. If Θ̃0 is finite, then θ̂t converges to some θ ∈ Θ̃0. 4.2. Proof of the convergence theorem The proof of Theorem 4.1 follows from a set of technical lemmas and propositions given below. Proposition 4.7 provides a set of technical results needed to prove the Theorem’s two main assertions: (i) the asymptotic equivalence of θt and the sequence θ̂t (Proposition 4.8) and (ii) (Proposition 4.9) the fact that θ̂t satisfies a.s. a Robbins-Monro recursion of the form considered in Proposition 4.6. Hereafter, K or sometimes k (or these letters with decorations) will denote a generic upper bound (not always the same one) that is finite, or when it is random, finite a.s. A random K will be shown as K(ξ) with ξ ∈ Ξ on first appearance whenever the randomness is not immediately clear from context. Again, unless specified otherwise, all limits (liminfs, limsups, etc.) are with respect to t and usually the t → ∞ will be omitted. The notation oa.s.(1) denotes convergence to zero with probability one. Proposition 4.7. Under the assumptions of Theorem 4.1, for the general recursive algorithm, the assertions (a)–(c) below follow: (a) lim inf t−1 t s=1 φ2 s ≥ σ2 a.s. and (t−1 t s=1 φ2 s)−1 ≤ K(ξ) ∞ , and thus, from (3.1b), P̄−1 t is bounded a.s. (b) For t ≥ 1, et = ∞ j=0 κe j(t)t−j; φt = ∞ j=0 κφ j (t)t−j; xt = ∞ j=0 κx j (t)t−j; and zt = ∞ j=0 κz j (t)t−j where for every j, κe j(t), κφ j (t), κx j (t) and κz j (t) are Ft−1- measurable. Moreover, there exist κ̃j such that max j {|κe j(t)|, |κφ j (t)|, |κx j (t)|, |κz j (t)|} ≤ κ̃j and ∞ j κ̃j ∞ a.s. Hence, the sequences et, φt, xt and zt are uniformly bounded a.s. (c) θt − θt−1 = oa.s.(1). Proof of (a). From (3.1d), φt = xt − θt−1et−1 = yt − θt−1(βxt−1 + et−1) . Since θt−1(βxt−1 + et−1) is Ft−1-measurable, by Proposition 4.3, (4.10) lim inf t−1 t s=1 φ2 s ≥ σ2 a.s. Continuing, from (4.10), for any 0 L1 σ2 , there exists t0 = t0(L1, ξ) such that t−1 t s=1 φ2 s L1 a.s. for all t ≥ t0. Let L2(ξ) ≡ min1≤tt0 t−1 t s=1 φ2 s. Then
- 50. Recursive estimation of possibly misspecified MA(1) models 31 0 L2 ∞ a.s. This follows since t0 is finite and φt is a finite valued sequence with probability one, hence L2 ∞. Moreover, since φ1 = y1, under (D1) it follows that L2 0 a.s. Hence, (t−1 t s=1 φ2 s)−1 ≤ max{L−1 1 , L−1 2 } ∞ a.s. and the proof of part (a) is complete. Proof of (b). Set θ0 = 0. From e1 = y1 and et = yt − θt−1et−1, t ≥ 2, it follows that κe j(1) = κj for all j, that κe 0(t) = κ0 for all t ≥ 1, and that κe j(t) = κj(t) − θt−1κe j(t − 1) for all t ≥ 2, j ≥ 1. It follows by induction that (4.11) κe j(t) = min(j,t−1) l=0 (−1)l κj−l l i=1 θt−i where 0 i=1(·) ≡ 1 . Since for some k∗ finite, |θt+k∗ | 1 for all t ≥ 1, we have that |θt| ≤ K(ξ) ∞. First suppose that K 1. Then from (4.11), |κe j(t)| ≤ min(j,t−1) l=0 |κj−l| l i=1 |θt−i| ≤ j l=0 Kl |κj−l| and since K 1, ∞ j=0 |κe j(t)| ≤ ∞ j=0 j l=0 Kl |κj−l| = ∞ l=0 Kl ∞ p=0 |κp| ∞ where p = j − l. So the result holds for the case of 0 K 1. Otherwise, suppose 1 ≤ K ∞. For all t ≥ k∗ , we have that |θt| ≤ K∗ (ξ) 1, so K(ξ) = λ(ξ)K∗ (ξ) for λ 1. For simplicity of notation, replace K∗ by ρ. We next show that l i=1 |θt−i| ≤ λk∗ ρl for l ≤ t. First suppose t ≤ k∗ . Then l i=1 |θt−i| ≤ λl ρl ≤ λk∗ ρl . Next suppose t k∗ and l ≤ t − k∗ . Then, l i=1 |θt−i| ≤ ρl ρl λk∗ since |θt−i| ≤ ρ for 1 ≤ i ≤ t − s∗ . Finally, suppose t k∗ and l t − s∗ . Then since l ≤ t, l i=1 |θt−i| = t−s∗ i=1 |θt−i| l i=t−s∗+1 |θt−i| ≤ ρt−s∗ λl−(t−s∗ ) ρl−(t−s∗ ) = ρl λl−(t−s∗ ) = λk∗ λl−t ρl ≤ λk∗ ρl . Hence, generally l i=1 |θt−i| ≤ λk∗ ρl . Setting κe j(ξ) = λk∗ j l=0 ρl |κj−l|, we have |κe j(t)| ≤ j l=0 |κj−l| l i=1 |θt−i| ≤ λk∗ j l=0 ρl |κj−l| = κe j , and since |ρ| 1, ∞ j=0 κe j ∞ a.s. Next, from (3.3) (4.12) κx j (t) = min(j,t−1) l=0 (−β)l κj−l l i=1 θt−i , and since 0 ≤ β ≤ 1, an argument like that for et can be applied and to obtain the existence of a κx j such that (4.13) |κx j (t)| ≤ κx j and ∞ j=0 κx j ∞ a.s.
- 51. 32 J. L. Cantor and D. F. Findley Continuing, since φ1 = x1 and φt = xt − θt−1φt−1 for t ≥ 2, it follows similarly that (4.14) κφ j (t) = min(j,t−1) l=0 (−1)l κx j−l(t) l i=1 θt−i . From (4.12) and (4.13), substituting κx j (t) for κj, the same kind of argument can be applied to (4.14) to yield (4.15) |κφ j (t)| ≤ κφ j with ∞ j=0 κφ j ∞ a.s. Finally, for t ≥ 2, we have, from zt = et + θt−1φt−1 , ∞ j=0 κz j (t)t−j = ∞ j=0 κe j(t)t−j + θt−1 ∞ j=0 κφ j (t − 1)t−1−j , for t ≥ 2 from which it follows that (4.16) κz j (t) = κe j(t) + θt−1κφ j−1(t − 1) , where κφ −1(t) ≡ 0. Since supt |θt| ∞ a.s., |κz j (t)| ≤ κe j + sup t |θt|κφ j−1 a.s., where κφ −1 ≡ 0, so there is a κz j such that |κz j (t)| ≤ κz j and ∞ j=0 |κz j | ∞ a.s. for t ≥ 2. Since z1 = e1, it thus follows that κ̃j = maxj{|κe j|, |κφ j |, |κx j |, |κz j |} satisfies ∞ j κ̃j ∞ a.s. From this, we see that et, φt, xt and zt are bounded a.s. For example, |φt| = ∞ j=0 κφ j (t)t−j ≤ sup −∞t∞ |t| ∞ j=0 κ̃j ∞ a.s. From (4.11)–(4.12), (4.14) and (4.16), κe j(t), κφ j (t), κx j (t) and κz j (t) are each Ft−1- measurable for every j. Hence, part (b) of the Proposition is proved. Proof of (c). By parts (a) and (b), |θt −θt−1| ≤ t−1 P̄−1 t |et||φt−1| ≤ t−1 K(ξ) where K(ξ) ∞ and thus part (c) follows and the proof of Proposition 4.7 is complete. Lemma 4.1. Under the assumptions of Theorem 4.1, we have: (a) If κ̃j(t) are Ft−1-measurable such that |κ̃j(t)| ≤ κ̃j for j ≥ 0, with ∞ j=0 κ̃j ∞ a.s., then for all p ≥ 1 and each 0 ≤ j ∞, (4.17) 1 t t s=2 min(j,s−1) l=1 κ̃j−l(s) l i=1 (s − i)−1 P̄−1 s−iφs−i−1es−i p a.s. −→ 0 , and (4.18) 1 t t s=2 min(j,s−1) l=1 κ̃j−l(s) l i=0 (s − i)−1 P̄−1 s−iφs−i−1es−i p a.s. −→ 0 .
- 52. Recursive estimation of possibly misspecified MA(1) models 33 In particular, (4.19) 1 t t s=2 min(j,s−1) l=1 κ̃j−l(s) l i=1 (s − i)−1 P̄−1 s−iφs−i−1es−i p 2 s−j a.s. −→ 0 . (b) For any 0 ≤ j ∞ and i ≤ j, (4.20) 1 t t s=1 (κφ j (s))2 2 s−j = 1 t t s=i+1 (κφ j (s − i))2 2 s−j + oa.s.(1) . (c) For 0 ≤ j, l ∞ and j = l, then (4.21) 1 t t s=max(j+2,l+2) κφ j (s)s−jκφ l (s)s−l a.s. −→ 0 . Proof of (a). By the boundedness of P̄−1 t , φt, et (Proposition 4.7) and since |κ̃m(t)| ≤ κ̃m for all m ≥ 0 and t ≥ 1, 1 t t s=2 min(j,s−1) l=1 κ̃j−l(s) l i=1 (s − i)−1 P̄−1 s−iφs−i−1es−i p ≤ 1 t t s=2 j m=0 κ̃j min(j,s−1) l=1 l i=1 (s − i)−1 |P̄−1 s−i||φs−i−1||es−i| p ≤ K(ξ) 1 t t s=2 min(j,s−1) l=1 l i=1 (s − i)−1 p . And since for all j ≥ 0, p ≥ 1, 1 t t s=2 min(j,s−1) l=1 l i=1 (s − i)−1 p ≤ K t t s=2 (s − min(j, s − 1)) −p −→ 0 , (4.17) follows, as does (4.19), by the boundedness of t. Similarly, 1 t t s=2 min(j,s−1) l=1 κ̃j−l(s) l i=0 (s − i)−1 P̄−1 s−iφs−i−1es−i p ≤ K(ξ) 1 t t s=2 min(j,s−1) l=1 l i=0 (s − i)−1 p (4.22) ≤ K(ξ) K t t s=2 min(j,s−1) i=0 (s − i)−1 p −→ 0 , and (4.18) follows.
- 53. 34 J. L. Cantor and D. F. Findley Proof of (b). From (4.12) and the recursion (3.1a) for θt, we have, for s ≥ j + 2, κx j (s) = j l=0 (−β)l κj−l l i=1 θs−i = j l=0 (−β)l κj−l l i=1 θs−i−1 + (s − i)−1 P̄−1 s−iφs−i−1es−i (4.23) = j l=0 (−β)l κj−l l i=1 θs−i−1 + j l=0 (−β)l κj−l l i=1 (s − i)−1 P̄−1 s−iφs−i−1es−i = κx j (s − 1) + wx j (s) . where (4.24) wx j (s) = j l=0 (−β)l κj−l l i=1 (s − i)−1 P̄−1 s−iφs−i−1es−i, Continuing, from (4.14) and (4.23)–(4.24), for s ≥ j + 2, κφ j (s) = j l=0 (−1)l κx j−l(s) l i=1 θs−i = j l=0 (−1)l κx j−l(s) l i=1 θs−i−1 + (s − i)−1 P̄−1 s−iφs−i−1es−i = j l=0 (−1)l κx j−l(s) l i=1 θs−i−1 + j l=0 (−1)l κx j−l(s) l i=1 (s − i)−1 P̄−1 s−iφs−i−1es−i (4.25) = j l=0 (−1)l (κx j−l(s − 1) + wx j−l(s)) l i=1 θs−i−1 + j l=0 (−1)l κx j−l(s) l i=1 (s − i)−1 P̄−1 s−iφs−i−1es−i = κφ j (s − 1) + wφ j (s), where from (4.24), wφ j (s) = j l=0 (−1)l wx j−l(s) l i=1 θs−i−1 + j l=0 (−1)l κx j−l(s) l i=1 (s − i)−1 P̄−1 s−iφs−i−1es−i (4.26) = j l=0 (−1)l j−l m=0 (−β)m κj−l−m m i=1 (s − i)−1 P̄−1 s−iφs−i−1es−i l n=1 θs−n−1 + j l=0 (−1)l κx j−l(s) l i=1 (s − i)−1 P̄−1 s−iφs−i−1es−i.
- 54. Recursive estimation of possibly misspecified MA(1) models 35 By (4.19) and (4.25)–(4.26), 1 t t s=j+2 (κφ j (s))2 2 s−j = 1 t t s=j+2 (κφ j (s − 1))2 + 2κφ j (s − 1)wφ j (s) + (wφ j (s))2 2 s−j . Applying an argument similar to that used for part (a), it follows by the bound- edness of β and θt and the Cauchy-Schwarz inequality that t−1 t s=j+2(2κφ j (s − 1)wφ j (s) + (wφ j (s))2 )2 s−j = oa.s.(1). Hence, 1 t t s=j+2 (κφ j (s))2 2 s−j = 1 t t s=j+2 (κφ j (s − 1))2 2 s−j + oa.s.(1) . Finally, since j is finite, then for i ≤ j, it follows by applying the recursion (4.25) in κφ j (t) i − 1 additional times that (4.20) holds, because a finite sum of oa.s.(1) terms is oa.s.(1). Proof of (c). By parts (a) and (b), for j = l, 1 t t s=max(j+2,l+2) κφ j (s)s−jκφ l (s)s−l = 1 t t s=max(j+2,l+2) κφ j (s − 1) + min(j,s−1) p=0 (−1)p κx j−p(s) × l q=1 (s − q)−1 P̄−1 s−qφs−q−1es−q (4.27) × κφ l (s − 1) + min(j,s−1) r=0 (−1)r κx j−r(s) × r m=1 (s − m)−1 P̄−1 s−mφs−m−1es−m s−js−l = 1 t t s=max(j+2,l+2) κφ j (s − 1)κφ l (s − 1)s−js−l + oa.s.(1). Without loss of generality, suppose j l ∞. From parts (a)–(b) and applying the argument that led to (4.27) j − 1 additional times, we have that 1 t t s=1 κφ j (s)κφ l (s)s−js−l = 1 t t s=j+1 κφ j (s − j)κφ l (s − j)s−js−l + oa.s.(1) = 1 t t−j s=1 κφ j (s)κφ l (s)s−(l−j)s + oa.s.(1) , = 1 t t s=1 κφ j (s)κφ l (s)s−(l−j)s + oa.s.(1) , since by (D4) and the fact that |κφ m(t)| ≤ K(ξ) ∞ for all m ≥ 0, t−1 t s=t−j+1 κφ j (s)κφ l (s)s−(l−j)s = oa.s.(1) .
- 55. Exploring the Variety of Random Documents with Different Content
- 56. insult had been offered to this country, which could not fail to produce irritation, yet that irritation should stop short of the point where it would produce action, as he was certain any steps taken which might hazard the peace of the country, would not conduce to the welfare of its citizens. There was a subject, he said, which seemed to have involved itself with this, and of which he should take some notice, viz: a charge against certain persons with being attached to the French cause. It might, perhaps, be the opinion of some members of that House, more particularly of strangers, that he was improperly influenced by party zeal in favor of the French, a zeal which it had been blazoned forth existed to an immoderate degree in this country. He had frequently heard insinuations of this sort, which he considered so groundless as to be worthy only of contempt; but when charges of this kind were made in the serious manner in which they were now brought forward, it was necessary to call for proof. Who, said he, is the man who has this proof? He knew of none. For his own part, he had no intercourse with the French but of the commonest kind. He wished those who possessed proofs of improper conduct of this kind, would come forward and show them—show who are the traitors of whom so much is said. He was not afraid of the impressions any such charges brought against him, might make upon his constituents, or where he was known; indeed, he had not the arrogance to believe the charge was levelled against him, though he believed he was frequently charged with a too great attachment to the French cause. When he first came into that House, he found the French embroiled with all their neighbors, who were endeavoring to tear them to pieces. He knew what had been the situation of this country when engaged in a similar cause, and was anxious for their success. Was there not cause for anxiety, when a nation, contending for the right of self-government, was thus attacked? Especially when it was well known, that if the powers engaged against France had proved successful, this country would have been their next object. Had they not, he asked, the strongest proofs (even the declarations of one of
- 57. their Governors) that it was the intention of England to declare war against America, in case of the successful termination of the war against France? It redounded to the honor of the citizens of this country, he said, that they had never shown a disposition to embark in the present European war. The difference, Mr. N. said, between the Address reported, and the proposition he had brought forward was this: the former approved all the measures of the Executive, and the latter recommended an inquiry relative to the operation of the British Treaty. It was this question upon which the committee would decide, and it was of importance, he said, that they should weigh the causes of difference between us and the French Republic, and not decide that we are right, without examination, because, if, after being brought to hostility, we are obliged to retract, it would show our former folly and wantonness. Mr. N. said he would inquire into the rights of France as they respected three principal subjects, which were more particularly causes of complaint between the two countries. These were, the right of our vessels carrying English goods, the article respecting contraband goods, and that respecting the carrying of provisions. He knew no better way to determine how far we could support those articles of the British Treaty, than by extracting the arguments of our own ministerial characters in support of these measures. With respect to the question of free ships making free goods, his impressions were very different from those of the Secretary of State. He says, with respect to the regulation of free ships making free goods, it is not changing a right under the law of nations; that it had never been pretended to be a right, and that our having agreed to it in one instance, and not in another, was no just cause of complaint by the French Government. He advocates this transaction in his letter to Mr. Adet last winter. Mr. N. said, he knew not what was the origin of the law of nations upon the subject; he knew not how it came into existence; it had never been settled by any convention of nations. Perhaps, however, the point now under consideration came as near to a fixed principle, as any other of what are called the laws
- 58. of nations ever did, as only one nation in Europe could be excepted from the general understanding of it. Mr. Pickering, he thought, seemed not to have given full force to this circumstance, but seemed to have weakened the evidence. [He referred to what Mr. Pickering had said upon the subject.] It was Mr. Pickering's idea, that the stipulation of free ships making free goods, was a mere temporary provision; that it was not an article in the law of nations, but a new principle introduced by the contracting parties. In order to prove this was not the case, Mr. N. referred to the provisions entered into by the armed neutrality of the north of Europe; to a treaty between France and Spain; to a note from the Court of Denmark; and to the declaration of the United States themselves on the subject. With respect to contraband articles, he had little to say. It was asserted that the articles stipulated in the British Treaty as contraband, were made so by the law of nations. Where the doctrine was found he could not say. It had been quoted from Vattel; this authority might be correct; but he never found any two writers on this subject agree as to this article. In a late publication on the law of nations (Marten's) he found it directly asserted that naval stores were not contraband. But he said, if the contrary were the law of nations, they were bound to extend the same privilege to France which they gave to England: they could not have one rule for the one nation, and a different one for the other. The 18th article of the British Treaty, respecting the carrying of provisions, always struck him as a very important one. It had heretofore been contended that this article did not go to any provisions except such as were carrying to besieged or blockaded places; but he believed the British had constantly made it a pretence for seizing provisions going to France. Indeed, if he was not mistaken, the British Minister had publicly declared in the House of Commons, that the provisions on board the vessels intended for the Quiberoon expedition had been supplied from what had been captured in American vessels.
- 59. Mr. N. contended that this was the opinion of the Executive of this country, as published in all the public papers, and of course known to the Government of France. In the letter of Mr. Jefferson to Mr. Pinckney in 1793, he declares that there is only one case in which provisions are contraband, and shows the necessity of a neutral nation observing the same rules towards all the powers at war. But, in the present case, the right was ceded during the present war. It was an unfortunate circumstance against the neutrality of this country, to find a doctrine so differently applied at different times. It was a strong proof of the progress of the passions. It might be considered as a fraudulent thing, in one instance, to give up a right for a compensation to ourselves. Mr. N. concluded with observing that he had gone over the subject, he feared, not without being considered tedious by the committee; but he felt himself greatly interested in the present decision. He believed any additional irritation in their measures would place peace out of our reach. He believed, therefore, it was their business to avoid it. He believed it would be for the honor and happiness of the country to do so. Mr. W. Smith said, as the gentleman last up had taken a wide range of argument, he must excuse him if he confined himself, in his reply, to those parts of his observations only which appeared to him essentially to relate to the subject under consideration. He believed the question was, whether they should alter the report in the manner proposed; that is, whether they should strike out words which expressed the sensibility of this House at the unprovoked insults offered by the French Republic to our Government and country, or adopt the gentleman's amendment, which he read. If they agreed to this amendment, they must necessarily expect from the French Republic fresh insult and aggression; for it seemed to admit that hitherto no insult had been intended.
- 60. The amendment might be divided, Mr. S. said, into two parts. The first went to vindicate the French from any intentional insults towards this country: it even held out an idea that the Executive ought to offer some concessions to France, and even designated the kind of concession. He should, therefore, without taking notice of what the gentleman had said about the political parties of this country, or what he had said respecting himself personally, confine his observations to the points in question. The first point was, whether the conduct of France was justifiable in rejecting our Minister, and sending him from the Republic in the manner they had done? He thought the committee had abundant materials before them completely to refute the first proposition; and he was surprised, knowing that these documents were in the hands of every member, that the gentleman from Virginia could expect to impress their minds with the idea that no indignity whatever had been offered by the French Government to this country in that transaction. Mr. S. said, that it appeared most clearly that the French Directory intended to treat this Government with marked indignity; for though the gentleman from Virginia suggested an opinion that their refusal to receive Mr. Pinckney was owing altogether to his not being invested with extraordinary powers, this was evidently not the case, as the Directory had been well informed as to the character in which Mr. Pinckney came, before they received his letters of credence, as appears by the letter of M. Delacroix to Mr. Monroe, styling Mr. Pinckney his successor, and by other documents communicated by the President, (which he read.) There was no doubt, then, with respect to the Directory being well acquainted with the character in which Mr. Pinckney went to France, viz: as Minister Plenipotentiary or ordinary Minister; but, after keeping him in suspense near two months, on the day after the news arrived of Bonaparte's successes in Italy, he was ordered, by a peremptory mandate, in writing, to leave the French Republic. This mandate was accompanied by a circumstance which was certainly intended to convey an insult; it
- 61. was addressed to him as an Anglo-American, a term, it is true, they sometimes used to distinguish the inhabitants of the United States from those of the West India Islands, but, in his opinion, here evidently designed as a term of reproach, as he believed no other similar instance could be mentioned. Upon this circumstance, however, he laid no stress; the other indignities which our Minister had received were too great to require any weight to be given to this circumstance. The gentleman from Virginia had confined the complaints of the French Government to three articles of the British Treaty; though, if the committee referred to the letter of Mr. Delacroix, it would be found that they did not confine them within so narrow a compass. They complain, first, of the inexecution of treaties; there are several points of complaint relative to that head. 2d. Complaints against the decrees of our Federal Courts. 3d. Against the law of June, 1794; and, 4th. Against the Treaty with Great Britain. Yet the gentleman confines himself altogether to the latter. And really he did not expect at this time of day, after the subject had been fully discussed, and determined, and the objections refuted over and over again, that any gentleman would have endeavored to revive and prove their complaints on this head well founded. The three articles were: 1st, that free ships did not make free goods; 2d, the contraband article; and 3d, the provision article. 1. The stipulation with respect to neutral vessels not making neutral goods in the British Treaty, was not contrary to the law of nations; it only provided that the law of nations was to be carried into effect in the manner most convenient for the United States. But this doctrine, he said, was no new thing. It had been acknowledged most explicitly by Mr. Jefferson, Secretary of State, in July, 1793, and was so declared to the Minister of France; yet no objection was made to it until the British Treaty was ratified, though long previous thereto French property was captured on board our vessels. Mr. Jefferson, writing on this subject to the French Minister, said: You have no shadow of complaint; the thing was so perfectly clear and well understood by the law of nations. This happened as long ago as July
- 62. or August, 1793. But two years afterwards, when the British Treaty was promulgated, the whole country was thrown into a flame by admitting this very same doctrine. France herself had always acted under this law of nations, when not restrained by treaty: in Valin's Ordinances of France this clearly appears. The armed neutrality was confined to the then existing war; Russia herself, the creator of the armed neutrality, entered into a compact with England, in 1793, expressly contravening its principles. The principle was then not established by our Treaty with England; but such being the acknowledged law of nations, it was merely stipulated that it should be exercised in the manner least injurious to us. 2. The next article of complaint was with respect to contraband goods. If gentlemen will consult the law of nations, they will find that the articles mentioned in the British Treaty are by the law of nations contraband articles. They will find that in all the treaties with Denmark and Sweden, Great Britain had made the same stipulation. Indeed, the gentleman had acknowledged that it was so stated by some writers on the law of nations; but he wished to derogate from the authority of those writers, in the same way as Mr. Genet, in his correspondence with Mr. Jefferson, had called them worm-eaten folios and musty aphorisms; to Vattel might be added Valin's Ordinances, a very respectable work in France. How, then, can the gentleman with truth say that we have deviated from the law of nations? 3. The last point which the gentleman took notice of was the provision article. There was no doubt that this Government would never allow provisions to be deemed contraband, except when going to a besieged or blockaded port. Though he made this declaration, yet it was but candid to acknowledge that this was stated by Vattel to be the law of nations. [He read an extract from Vattel.] When this was stated by Lord Grenville to Mr. Pinckney, our then Minister in London, Mr. Pinckney acknowledged it to be so stated in Vattel, but very ingeniously argued that France could not be considered as in the situation mentioned in Vattel, since provisions
- 63. were cheaper there than they were in England, and therefore the case did not apply. When our Envoy was sent to London, both parties were tenacious on this ground. Our Minister was unwilling to agree to this construction of the law of nations; but the British Minister insisted upon it, and if there had not been some compromise, the negotiation must have been broken off, and a war probably ensued. The result was, therefore, that, without admitting it to be the law of nations, it was agreed that where provisions were contraband by the law of nations, they should be paid for, but not confiscated, as the law of nations (admitting that construction) would have authorized. Therefore some advantage was secured to France, for if Great Britain had confiscated our vessels going to France with provisions, it would certainly have damped the ardor of our citizens employed in that commerce; but under this regulation our merchants were certain of being paid for their cargoes, whether they arrived in France or were carried into England. These were the three grounds of objection which the gentleman from Virginia had stated as grounds of complaint by the French against the British Treaty. Before he went further, he would observe that, admitting (which he did not admit) that there had been solid grounds of objection against the British Treaty, before it was ratified, yet they ought now to be closed. It had received a full discussion at the time; it had been carried into effect, was become the law of the land, and was generally approved of by the country. Why, then, endeavor to stir up the feelings of the public against it by alleging it to be just cause of complaint? If the committee wanted any proof of the approbation which that instrument had received, he thought it might be gathered from the general approbation which had been given of the administration of the late President on his retirement from office, in doing which the people had doubtless taken into view the whole of his conduct. Nor did he think the people had shown any hostility to the Treaty in their late election of members to that House. Indeed, he believed that the approbation which the Treaty received increased in proportion as the subject came to be understood.
- 64. Admitting further, that the Treaty had changed the existing state of things between Great Britain and France, by having granted commercial favors to Great Britain; by the 2d article of our treaty with France, the same favors would immediately attach to France, so that she could have no reason to complain on that ground. Indeed France had herself new modified the treaty between that country and this, and had taken to herself what she deemed to be the favors granted to Great Britain. [Mr. S. read the decree on this subject of 2d March last.] Mr. S. said, he believed he had examined all the observations of the gentleman from Virginia, relative to the Treaty, which were essential to the subject under consideration. He did not wish to go much farther on the present occasion, because he agreed with him, that it was proper they should keep themselves as cool and calm as the nature of the case would admit; but he thought whilst so much deference was paid to the feelings of France, some respect ought to be paid to the feelings of America. He hoped the people of America would retain a proper respect and consideration for their national character; and however earnestly he wished that the differences subsisting between the two countries might be amicably settled, yet, he trusted that our national dignity would never be at so low an ebb as to submit to the insults and indignities of any nation whatever. In saying this, he expressed his hearty wish to keep the door of negotiation with France unclosed; but at the same time he strongly recommended to take every necessary step to place us in a situation to defend ourselves, provided she should still persist in her haughty demeanor. Mr. S. said, as he knew indecent and harsh language always recoiled upon those who used it, he did not wish to adopt it; but, at the same time, it was due to ourselves to express our feelings with a proper degree of strength and spirit. He was not in the habit of quoting any thing from M. Genet, but there was one expression of his which he thought contained good advice, all this accommodation and humility, all this condescension attains no end.
- 65. After the gentleman from Virginia had dwelt sufficiently upon the danger of irritating the French, he had emphatically called upon us to recollect our weakness. It might have been as well if he had left that to have been discovered from another quarter. He hoped we had sufficient confidence in the means of defence which we possessed, if driven to the last resort; and he believed if there was any one more certain way of provoking war than another, it was that of proclaiming our own weakness. He hoped such a language would now be spoken as would make known to the French Government that the Government and people of this country were one, and that they would repel any attempt to gain an influence over our Councils and Government. The gentleman had said that there did not appear to be any design of this kind, and had endeavored to do away what was stated as the opinion in France, in General Pinckney's letter. He did not mean to rest this altogether upon the reports of an emigrant, whom General Pinckney mentions as having represented this country divided, and of no greater consequence than Genoa or Geneva, but he took the whole information into view. [He read the extract relative to this subject.] It was evident, Mr. S. said, from this information from France, that an opinion had been industriously circulated there that the Government and people of this country were divided; that the Executive was corrupt and did not pursue the interests of the people; and that they might, by perseverance, overturn the Administration, and introduce a new order of things. Was not such an opinion of things, he asked, calculated to induce France to believe that she might make her own terms with us? It was well known what the French wished, and it was time to declare it plainly. His opinion was that they designed to ruin the commerce of Great Britain through us. This was evident. They talk of the British Treaty; but they suffered it to lie dormant for near twelve months, without complaining about it. Why were they silent till within a few weeks before the election of our President? Why did they commit spoliations upon our commerce long before the British Treaty was ever dreamt of? Their first decree, directing spoliations of our
- 66. property, and the capture of our provision ships, was on the 9th of May, 1793, a month before the provision order of Great Britain, which was dated June 8, 1793; and why have they, from that time to this, been committing spoliations on our commerce? The British Treaty was published in Paris in August, 1795; a year after, in July, 1796, they determine to treat us in the same way that we suffer other nations to treat us, and this decree was not made known to our Government till the October following, a few weeks before the election of President. But this was not all; the French had pursued similar measures towards all the other neutral powers. Sweden, in consequence, had no Minister in their country, and was on the eve of a rupture. The intention of the French evidently was, to compel all the neutral powers to destroy the commerce of Great Britain; but he trusted this country had more spirit than to suffer herself to be thus forced to give up her commerce with Great Britain; he trusted they would spurn any such idea. Mr. S. hoped the observations which he had made would not be construed into a wish to see the United States and France involved in a war. He had no objection to such measures being taken for preserving peace between the two countries as should be consistent with national honor. It was a delicate thing for them to suggest what the Executive ought to do. It was out of their province to direct him. The Executive had various considerations to take into view. We had injuries to complain of against France, for the spoliations committed upon our commerce. If the Executive conceive we have a right to redress, that subject will of course constitute a part of our Envoy's instructions. Would it then be proper, said he, for this House to interfere with the Executive, to obtrude its opinion and say, You must give up this point; we take upon us (without any authority by the constitution) to give carte blanche to France, without any indemnification or redress. The gentleman says it is the object of the amendment on the table to recommend to the Executive to remove any inequalities in the
- 67. treaties; that was alone sufficient to vote it out. There had been no period since the Revolution which had so powerfully called on Americans for that fortitude and wisdom which they knew so well how to display in great and solemn emergencies. It was not his intention to offend any one by stating the question in such strong terms; but he was persuaded that when the present situation of our affairs with respect to France was well understood, it would be found that to acquiesce in her present demands was virtually and essentially to surrender our self-government and independence. Tuesday, May 23. Two other members, to wit: from North Carolina, Joseph McDowell, and from Virginia, Josiah Parker, appeared, produced their credentials, were qualified, and took their seats. Answer to the Presidents Speech. The House then went into a Committee of the Whole, Mr. Dent in the chair, on the amendment of Mr. Nicholas to the report of the select committee, in answer to the President's Speech. Mr. Freeman first rose. He observed, that in his observations on the subject before the committee, amid the conflicting opinions of gentlemen whom he respected, he did not mean to express his own either with confidence or with zeal. Though one of the committee that had reported the Address, he could not approve it in toto. He had two principal objections to it. First, to that part which went to an unequivocal approbation of all the measures of the Executive respecting our foreign relations; and, secondly, to that part which contained expressions of resentment and indignation towards France. In framing an answer to the President, he conceived the committee should have refrained from expressing an unqualified approbation of all the measures of the Executive. To omit it would
- 68. not imply censure. By introducing it, it forced all those who entertain even doubts of the propriety of any one Executive measure to vote against the Address. The principal causes of the irritation on the part of France, insisted upon in the Answer, were the rejection of our Minister, and the sentiments contained in the Speech of the President of the Directory to our late Minister. If gentlemen would look into the documents laid before the House by the President, he was confident they would find the true reason for the refusal to receive our Minister. He came only as an ordinary Minister, without any power to propose such modifications as might lead to an accommodation, and when the Directory discovered this from his credentials they refused him. In answer to this, it had been urged that M. Delacroix, Minister of Foreign Affairs, from the first, well knew that Mr. Pinckney was only the successor to Mr. Monroe, and that his coming in that quality was not the reason why the French refused to receive him. Mr. F. referred to the documents which had been laid before the House on this subject, from which it appeared that the secretary of M. Delacroix had suggested a reason for the apparent change of opinion on the subject of receiving Mr. Pinckney. Suppose, the secretary observed, that M. Delacroix had made a mistake at first in the intentions of the Directory, was that mistake to be binding on the Directory? He did not wish to be understood to consider the conduct of the French as perfectly justifiable; but he could not conceive that it was such as to justify, on our part, irritating or violent measures. As to the Speech of the President of the Directory, he could not say much on it, he did not perfectly understand it. As far as he did, he considered it a childish gasconade, not to be imitated, and below resentment. [He read part of it]. It was certainly arrogant in him to say that we owed our liberty to their exertions. But if the French could derive any satisfaction from such vain boasting he had no objection to their enjoying it. There was another part of the Speech that had been considered as much more obnoxious. It was said to breathe a design to separate the people here from their Government. The part alluded to was no more than an expression of
- 69. affection for the people; he could see nothing in this irritating or insulting; it was a mode of expression which they used as to themselves, and by which they wished to convey their affection for the whole nation. The term people, certainly included the Government, and could not with propriety, therefore, be said to separate the people from it. An idea had been thrown out by the gentleman from South Carolina, that the people generally approved of the British Treaty; he inferred it from the fate of the late elections. For his part he could see no great alteration to have been produced by the late elections; and if there had been it would not have been an evidence to his mind that the people approved of the British Treaty. He believed, for his part, that the opinions of a great majority of the people had been uniformly averse to it; and those who advocated it were by this time nearly sick of it. It was true a spirit was aroused by the cry of war at the time the subject of appropriation was pending, that produced petitions, not approving however of the stipulations of the treaty, but asking that it might be carried into effect since it had reached so late a stage. Another engine, he observed, had been wielded with singular dexterity. Much had been effected by the use, or rather abuse, of the terms federalist and anti-federalist, federalism and anti- federalism. When the Federal Constitution was submitted to the people, to approve it, and endeavor to procure its ratification, it was federalism. Afterwards, when the Government was organized and in operation, to approve every measure of the Executive and support every proposition from the Secretary of the Treasury, was federalism; and those who entertained even doubts of their propriety, though they had been instrumental in procuring the adoption of the constitution, were called anti-federalists. In 1794 to be opposed to Madison's propositions, the resolution for the sequestration of the British debts, and the resolution prohibiting all intercourse with Great Britain, was federalism. In 1796 it was federalism to advocate the British Treaty; and now he presumed that it would be federalism to support the report of the committee and
- 70. hightoned measures with respect to France. In 1793 he acknowledged that federalism assumed a very different attitude from what it had on the present occasion; it was then the attitude of meekness, of humanity, and supplication. The men who exclusively styled themselves federalists, could only deplore with unavailing sighs the impotence of their country, and throw it upon the benevolence and magnanimity of the British Monarch. Their perturbed imaginations could even then see our cities sacked and burnt, and our citizens slaughtered. On the frontier they heard the war-hoop, and the groans of helpless women and children, the tortured victims of savage vengeance. Now we are at once risen from youth to manhood, and are ready to meet the haughty Republic of France animated with enthusiasm and flushed with victory. Mr. F. observed, that he rejoiced however that gentlemen adopted a bolder language on this than had been used on the former occasion. He felt his full shame in the national degradation of that moment. He was in favor of firm language; but he would distinguish between the language of manly firmness and that of childish petulance or ridiculous bombast. Mr. Griswold said, if he understood the state of the business, the question was, whether the committee would agree to the amendment proposed by the gentleman from Virginia? If it contained sentiments accordant to the feelings of the committee, it would of course be adopted; if not, it would doubtless be rejected. He supposed it would form an objection to this amendment, if it were found to be inconsistent with the other parts of the report. He believed this to be the case; but he would not make objections to it on this ground. He would examine the paragraph itself, and see whether it contained sentiments in unison with those of the committee. He believed this would not be found to be the case, and that when the committee had taken a view of it, it would be rejected. If he understood the proposition, it contained three distinct principles, viz:
- 71. 1. To make a new apology for the conduct of the French Government towards this country. 2. That the House of Representatives shall interfere with and dictate to the Executive in respect to what concessions ought to be made to the French Republic. 3. It depends upon the spirit of conciliation on the part of France for an adjustment of the differences existing between the two Governments. The apology, he said, was a new one, and one which the French had not thought of making for themselves; for they tell us, as it appears from Mr. Pinckney's letter to the Secretary of State, they will not acknowledge or receive another Minister Plenipotentiary from the United States, until after the redress of the grievances demanded of the American Government, and which the French Republic has a right to expect from it. We say (or rather the gentleman from Virginia says in his amendment) they rejected our Minister because he had not power enough; therefore, for the apology now made for the French Government they were indebted to the ingenuity of the mover. Now, said Mr. G., I do not wish that the House of Representatives should undertake to make apologies for the conduct of the French Government towards this. It was true they needed apology; but he did not think it was proper for us to make it for them. Further, as this apology was not made by themselves, but wholly different from their own assertions, it was not likely that they would fall into it. They say, Permit us to sell our privateers in your ports; annul treaties and repeal laws, and then we will tell you on what terms we will receive Mr. Pinckney, and peace from you. After this declaration, he did not think it would be proper to attempt any new apology for them. He therefore supposed, that so far as this proposition offered a new apology for the French Republic, it could not meet with the approbation of the committee.
- 72. The next proposition contained in the amendment was, that the House of Representatives should interfere with the Executive power of this country, and dictate to it what sort of steps should be taken towards reconciling the French Government. He asked whether this was consonant to the principles of the constitution? Whether the constitution had not delegated the power of making treaties to other branches of the Government? He believed it had, and that therefore we had no right to dictate to the Executive what should or what should not be done with respect to present disputes with the French Government. On this ground, therefore, he considered it as improper. In the next place, the amendment contained another proposition, viz: that we rely upon a spirit of conciliation on the part of France for an accommodation of differences. And, said Mr. G., do we really rely upon this? Have we such evidence as should incline us to rely upon it? Have the French Government expressed any inclination to settle the differences subsisting between them and us? The communications which were received from the Supreme Executive, do not bear this complexion. The communication from the French Minister to this Executive does not wear it. Our proclamations are called insidious; our Minister is insulted and rejected; and attempts are made to divide the people of this country from their Government. Is this conciliation? Does it not rather appear as if they intended to alienate the affections of the people from their Government, in order to effect their own views? He was convinced it did, and that they could not rely upon a spirit of conciliation in them. For his own part, he did not rely upon it; he relied upon this country being able to convince the world that we are not a divided people; that we will not willingly abandon our Government. When the French shall be convinced of this, they will not treat us with indignity. Therefore, he trusted, as the proposed amendment did not contain such sentiments as were likely to accord with the feelings of the committee, that it would be rejected. Mr. Giles said the subject under discussion was a very important one. It appeared to him, from various documents, that all the steps taken
- 73. by the Executive had a view to an eventual appeal to arms, which it was his wish (as it was the wish of many in that House) to avoid. It was proper, therefore, that the clashing opinions should be discussed. If the proposition brought forward for this purpose was not sufficiently simple and explicit, he wished it might be made more so. For he believed the question to be, whether the committee be prepared to pass a vote, approving of the whole course of the conduct of the Executive, or whether France should be put upon the same ground with the other belligerent powers. That she is at present upon the same footing, no gentleman had attempted to show. Gentlemen who wish to get rid of this ground, say this is a thing which should be left to the Executive. He thought it was, however, a proper subject for their discussion; for whatever power the Executive had with respect to making of treaties, that House had the means of checking that power. Suppose, said Mr. G., I were on this occasion called upon to tax my land, was it not necessary I should inquire into the subject, and endeavor to avoid a measure which would probably prove a serious drain upon the blood and treasure of the country? He was unwilling to have his land taxed for the purpose of supporting a war on this principle. It was evident that the French took one ground in this dispute, and the United States another, and whilst this continued to be the case, no negotiation would have any effect. Indeed, said he, it is war; and if the measure proposed was taken, we make war if we do not declare it. Mr. Baldwin said, he had taken the liberty to express his concern several years ago, that this custom of answering the President's Speech, which was but a mere piece of public ceremony, should call up and demand expressions of opinion on all the important business of the session, while the members were yet standing with their hats in their hands, in the attitude of receiving the communications, and had not yet read or opened the papers which were the ground of their being called together. It applied very strongly in this instance, as this was a new Congress, and a greater proportion than common of new members; he thought it an unfavorable attitude in which to be hurried into the very midst of things, and to anticipate business
- 74. of such vast importance to the country, before they had time to attend to the information which had been submitted to them. He trusted some fit occasion would before long be found to disencumber themselves of a ceremony, new in this country, which tended only to evil and to increasing embarrassments. He observed that it was under the influence of these impressions, he had made it a rule to himself, for many sessions, to vote for those amendments and those propositions in the Address which were most delphic and ambiguous, and while they were respectful to the President, left the House unpledged and open to take up the business of the session as it presented itself in its ordinary course. It was on this ground he should vote for the amendment now under consideration. Mr. Rutledge said, when the report of the committee should be before them, he should have some remarks to make upon it; but at present he should offer only a few observations upon the proposed amendment. He said he had strong objections to the amendment; but one so strong that he need not urge any other: it was, that in agreeing to it they should dictate to the Executive, which he believed would be infringing upon the Executive power. As it was his peculiar duty to give instructions to Ministers, it would be improper in them to say what should be the instructions given to a Minister; but if it were not so, he should not vote for those of the gentleman from Virginia. In the instructions of a Minister, it was usual to comprise a variety of propositions. Certain things were first to be proposed; if these could not be obtained, he was instructed to come forward with something else, and if this could not be got, he went on to his ultimatum. But, if the proposition of the gentleman from Virginia were to obtain, his instructions would be publicly known. In vain would it be for him to offer this or that, they will say the House of Representatives has directed you what to do, and we will not agree to any thing else. This would be contrary to all diplomatic proceedings; for that reason he should be opposed to the House saying what should be his instructions. Indeed, if it were usual, he should be against it in this
- 75. instance, as he believed it would encourage an extravagant demand. What, said he, have they said to our Minister—or rather to the person who was formerly our Minister, but who then had no power? They told him to go away; they had nothing to say to him: they would receive no more Ministers from the United States until their grievances were redressed. This country is charged with countenancing an inequality of treaties. The French have said, redress our grievances in a certain way. But, said Mr. R., if we do this, we shall put ourselves under the dominion of a foreign power, and shall have to ask a foreign country what we shall do. This was a situation into which we must not fall without a struggle. Mr. Sitgreaves said, though he had wished to have taken a little more time before he had troubled the committee with his observations; yet, as there now appeared an interval, he should take the opportunity of occupying it for a few minutes. He should not answer the observations of the gentleman from Georgia, with respect to the style of the Answer reported; but he believed that those gentlemen who would look at it without a perverted vision, would not discover the faults in it which that gentleman had discovered. He thought it rather remarkable for the simplicity of its style than for a redundancy of epithet. He discovered more of the latter in the amendment than in the original report. It was true that the superlative was used in different places, but he thought it was used where it ought to be. He would not, however, detain the committee with matter so immaterial, but would proceed to what appeared to him of some consequence. A stranger who had come into the House during this debate, and heard what had fallen from the mover of the proposed amendment, and from members who had followed him, would have supposed, that instead of an act of ordinary course being under discussion, they had been debating the question of a declaration of war against France. He would declare, for himself at least, on the subject of war, that he agreed in certain of the sentiments of gentlemen on the other side
- 76. Welcome to our website – the perfect destination for book lovers and knowledge seekers. We believe that every book holds a new world, offering opportunities for learning, discovery, and personal growth. That’s why we are dedicated to bringing you a diverse collection of books, ranging from classic literature and specialized publications to self-development guides and children's books. More than just a book-buying platform, we strive to be a bridge connecting you with timeless cultural and intellectual values. With an elegant, user-friendly interface and a smart search system, you can quickly find the books that best suit your interests. Additionally, our special promotions and home delivery services help you save time and fully enjoy the joy of reading. Join us on a journey of knowledge exploration, passion nurturing, and personal growth every day! ebookbell.com