![492 B. G. Q u i n n a n d J. M. Fe r nan de s
Let
gr(or) = X {£(<*)+ 6_2(a)}6-i(«)/X €/-1(a )
and put h,(a) =gT(a )-a . Then hr{a) = 2 X,£,_,(a!)/2 £;_,(<*). The following theorem
shows that there is a zero of hT(a) close to a0. We leave all proofs until later, and assume
conditions (1*2), (1*3) and (1*5) throughout. Where order notation is used, the orders
should be understood as being almost surely as T-»oo.
T h e o r e m 1. Let A T={or: |a - a0| < cT '}, where e and c arefixed, 1< e <§, c>0 and
a0=2 cos (o0. Then almost surely as T ->oo, there exists a unique a , e A , with the property
that hT(ciT) = 0.
As a trivial consequence of the theorem we have the following.
Corollary. We have TF(a>T- w0)-*0, almost surely, for all e < §, where a, = 2 cos <aT.
The above results relate only to the ‘fixed point’ aT of the procedure. The following
theorem shows that the successive iterates converge to the fixed point aT. Its corollary
demonstrates the speed of the convergence of the accelerated procedure.
Theorem 2. With A, defined in Theorem 1, gT(a) = 5(0-- a T){l + 0 (T -4)} uniformly
in a e At if e> 1, while, if e ^ 1,
*t(o ) - « r = - <*•,■)[i + o{ T>-'(log r ) !}]+ o{ r i- 2'(iog r ) !}.
Furthermore, aT- a0= 0{T~3,2{log T)’}.
Corollary. Let a, c AT, and let a, = 2gr (a,_,)-a ,., (j =2, 3 ,...). Then if i< £ < l,
ak- aT= o(T 3/2) for k 2sinteger part of {3-log (2e - 1)/log (2)}.
The final theorem provides a central limit theorem for a>T.
T h e o r e m 3. We have that T3/2(<ot - u)0) has a distribution converging to the normal
with mean zero and variance 487r/r(w0)/p*2.
We note here that as the standard deviation of <or is 0( T 3/2), the corollary to Theorem
2 provides the number of iterations until efficiency is attained.
Proofof Theorem 1 and Corollary. We use the fixed point theorem of Dieudonne (1969,
p. 266). We must show that, almost surely as T -»oo:
(i) there exists k with 0 ^ k < 1such that if a, Ar, then |gT(a) —gT(a')|$ ka - a ' |,
and
(ii) hr(oto)< (l-k)T".
Put
dr (a») = sin w £ X ,£ ,(a>), eT{u)) = sin1oj £
Then hr = sin a>dT(a>)eTl(a>). Now
eT((o)= sin (jw)[u,.}+p cos {(/-j)<t>0+ <f>}]
- sin (jo)) cos {(/-j)a)0+(f>}
+2p X X sin (yw)u,_, X sin (ka)) cos {{t-k)(o0+(f>}. (31)](https://image.slidesharecdn.com/afastefficienttechniquefortheestimationoffrequency-180120183053/85/A-fast-efficient-technique-for-the-estimation-of-frequency-4-320.jpg)
![Fast efficient estimation offrequency 493
The first term of (3-1), being differentiable with respect to a>, may be written as
£ ( L sin 0'<w0)w,-;)
r-0 l;=0 J
2 T - l / t
+ 2(o)-o>0) X X sin (>>*)«, i X k cos {ku>*)u,-k,
i-o i-o k - 0
where o>* denotes, gcnerically, a number between w and oj0. Now, |I sin is
0{(t log f)*}» from (1-6), while
i i 1 1~~J
X k cos {ho*)u, k = t Z cos X X cos
fc-0 k—0 Z^-l *-0
is 0{/V2(log f)*}, all orders being uniformly in w*. The first term of (3-1) is thus
Z ( Z sin (y®o)Wi->| + 2(w ~(Oo)0(T- log T).
/-() ly-o J
We note here that, also because of (1-6), 2{E sin (_/<w)m,_^}2 is 0(T 2log T), uniformly
in o).
The second term of (3-1), when divided by p2, may be shown to be
h{Ty+ 0{T 2)}+(a) - <o0)O(T2),
uniformly in a), while the third term is
0{T 5/2(log T)*} +(oj- o>0)0{TV2Vo&DH,
by (1-6). This completes the analysis of the term eT{(o). The term dT{(o) may be treated
in a similar way:
T - l l T - l t
dT(to)= Z ui Z sin (ja>)u,_, + p2 X cos (ro>0+ </>) Z sin{jo>)cos{(t-j)<a0+4)
f - 0 ) - 0 1 - 0 . i - 0
T - l t T - l
+ P X w, Z s in (» c o s { ( /-i/)wo+^)+ X cos (ta)0+ 4>) X sin(>a>)M,_, .
i —O j -0 1 -0 1=0 J
(3-2)
The first term in (3-2) may be written as
X w, X sin {j<o0)ul-J+((o-(o0) X u>X j cos (ju)*)u,-t
r-0 j- 0 i-0 f-0
and is
0{73/2(log T)l} +{ » -w 0)O{T*/2(log T A
uniformly in oo. The second term in (3-2), when divided by p2, may be shown to be
0(T) + (w -w 0){7'3/ 24+0(T 2)}, while the third term is
O {r3/J(log T)!}+ (o>-a>0)O {r5/2(Iog T)>},
uniformly in co. We may now prove (i). Let a =2 cos u>and a '=2 cos w'. Then
hT(a )- hT(a') = sin wdT((o)/eT((o) - sin (o'dT((o')/eT(a)').
Thus
er (a>)e7(a>'){/?r (a) - hT(a')} = (sin a>-sin io')dT((o)eT((o')
+ sin (o'{dT((o) - dT((o')}eT((o)
+ sin a)'dT(u)){eT(a))- er((o')}.
Careful analysis shows that
gT(a ) - gT{a') =i(a -<*')[ 1+ 0{ T ‘(log 7")1}]](https://image.slidesharecdn.com/afastefficienttechniquefortheestimationoffrequency-180120183053/85/A-fast-efficient-technique-for-the-estimation-of-frequency-5-320.jpg)
![494
uniformly in a and a'. Thus (i) follows for any k such that < k< 1. Part (ii) may be
proved with little extra work. The dominant term in r/r (w) is 0{TV2(log T)4}, and so
hr(a0) = O{T V2(log 7)*}, which is o(T e) as e < f.
To prove the corollary, we simply note that the theorem holds for any e such that
1< e < 2, and that A T is contracting as e increases. Thus, almost surely, d r e A r and
Ty(ar -o tu) is 0(1) for any e<2. Hence Tf(d r - a 0) and consequently T'’(wr - w0)
converge to zero almost surely for any e<$. □
Proof of Theorem 2 and Corollary. If e > 1, the proof of the above theorem shows that
g r(« ) - £r(<*r) - J(« “ <*r)[*+ 0 { T ~J(log T)4}].
The result of the theorem thus follows since gr (a r ) = <*r« Also,
B. G. Q ijinn a n d J. M. Ff.rnandf.s
{2gr(a) - a) - aT= 2{gr(a) - d ,} - (a - d , )
= ( a - d T)0 { T ~ log T)4}.
Since a - a T is 0 (T *) for some e> 1, it follows that (2gT(a) - a} - aT is o{T y2).
If 5<e «£l, however, the proof is more difficult. It may be shown that
eT(a)) _ p iL _ r
4(w —w0)2L
sin {T(w —w0)}
dT((o) - d -
4(w - w„) L
T(w —w0)
sin {7~(w- w0)}
l[i + o{ri-f(iog T)‘],
T(w - w0)
U+ o (r-')} + o { rV2(iogT)i}
uniformly in w. Consequently,
hT(a ) = -J(« - ir ) [ l + 0{T*-'(log D>}]+ 0 {T ‘-’'(log T)1}
uniformly in a. But gT(o t)-d 7 = hT(a) + a - d T which is therefore
- <*j-)[l + 0{T-"'(log 7 )!}]+0{T!-2r(log T)1}.
Also
{2gr{oc) - a} - ST=(o ,- aT) 0 { p - ( log 7)*}+ Of P 2’<log T)1}.
This completes the proof of the theorem. To prove the corollary, we first note that if
e > 1, the integer part of {3 - log (2e - l)/log 2} is 2, and the corollary follows from above.
If ! < c ^ l , the above argument show's that {2gT(a) - a} - a T is 0 {T 3~2t(log T)4}, is
therefore o(T4-2r+c), where 8 is generically positive and arbitrarily close to zero, and is
thus of smaller order than (a - aT). By induction, if a, - dr is not already 0( T " ), where
v > , then it must be 0 (T K), where k = - +2j 2-2'~' e + 8. We thus need to continue
iterating until k < —1, that is, until j> 1-log (2e - 1)/log 2, after which the next iterate
is accurate to the desired order. Hence ak- aT - o(T "V2) when fc^the integer part of
{3—log (2e - l)/log 2}. In particular, a3—aT= o(T~y2) if e = 1, which is the case when
the procedure is initialized by the maximizer of the periodogram over the Fourier
frequencies. □
Proof of Theorem 3. From the proof of Theorem 2, it is seen that
<5r - <*„= - ^ [ 1 + 0{ r-!(iog T)1}]
eT(w„)](https://image.slidesharecdn.com/afastefficienttechniquefortheestimationoffrequency-180120183053/85/A-fast-efficient-technique-for-the-estimation-of-frequency-6-320.jpg)
![Fast efficient estimation o ffrequency 495
as
aT- a 0=2hT(a0)l + O{T-'(og T)»}].
Thus, since T~3eT{a>0)->p2/24, almost surely, {T*f2p7f 24){u>r —<d0) has the same
asymptotic distribution as -T~i/2dT(<o0), which may be shown to be equivalent to
-T ~ V2p/2 X u,{(2t - T) sin (/o>0+ 4>)+0( 1)}.
Using the results of Hannan (1979), and the first set of conditions there, it follows that
T~3/2dT(<o0) is asymptotically normal with mean zero and variance 2'n-f,((o0)p2/24, where
f, is the spectral density of {u,}. Consequently, r V2(a>r - a>0) is asymptotically distributed
normally with mean zero and variance 487rft(<o0)/p2-The theorem follows as /j,((a0)/p2=
fA<o0)/p*2- □
We note in passing that if a> is estimated by maximizing the periodogram by the
Gauss-Newton technique, if (a, -co0= 0 (T r), for some e > 1, then <oj is within o(T~*'2)
of the maximizer for 7'^the integer part of {2- log (2s -2)/log 3}. If e ^1, there is no
guarantee that the procedure will yield the maximizer of the periodogram. This should
be considered when the results of the simulations are examined.
4. S im u l a t io n s
Equation (1-1) was used to generate time series of lengths T = 50, 100, 200 and 500,
where {£,} was simulated by pseudo-random Gaussian numbers with standard deviations
cr = 0*l, 0-3, and 0*5. In each case we put p* = 1 and the frequency co0 was tt/S +tt/T,
so that the true frequency would be midway between two successive Fourier frequencies,
a ‘worst case’. For each combination of sample size and standard deviation, 100 replica
tions were generated, and a>was estimated by the following four techniques:
(i) maximization of the periodogram by Gauss-Newton initialized by <a,„
(ii) and (iii) the simple and accelerated techniques of § 2 initialized by a>p,
(iv) the accelerated technique initialized by wPis,
where Sp is the maximizer of the periodogram over the Fourier frequencies 27rj/T
0 = 1 ,..., /t), with n the integer part of J(T - 1), and a>f»is is the Pisarenko estimator.
For each simulation, ‘convergence’was deemed to have occurred once successive iterates
were within 10 s of each other. The results produced by (iv) are surprising as <a,.is-a>0
is theoretically not of small enough order to guarantee convergence.
The results of the simulations are recorded in Table 1. Clearly the best results are
produced by techniques (iii) and (iv). Surprisingly, (iv) does not suffer from the deficiency
evidenced by the others for the combination T = 50 and (7= 0-5, which occurred because
five of the replications produced estimates of ioQnear 2. The poorer performance of (i)
was expected and the theoretical reason for this is indicated at the end of §3.
Table 2 contains the result of a comparison with the technique of Mackisack & Poskitt
(1989), which is labelled (v). The true frequency in each case is 1-24, p = 20, <f>=0-01,
cr2= 1 and 100 replications were carried out. Our technique is clearly superior. It should
be noted that the signal to noise ratio for this experiment was much larger than those of
the other simulations.](https://image.slidesharecdn.com/afastefficienttechniquefortheestimationoffrequency-180120183053/85/A-fast-efficient-technique-for-the-estimation-of-frequency-7-320.jpg)
![497
Ac k n o w led g em en ts
B. G. Quinn was partly funded by grants from the Australian Research Council and
the University of Newcastle. J. M. Fernandes was on Doctoral study leave in the
Department of Electrical and Computer Engineering, University of Newcastle. His work
was supported by CAPES, Brazil.
Fast efficient estimation offrequency
R e f e r e n c e s
An, H.-Z., Chen, Z.-G. & Hannan, B. J. (1983). The maximum of the periodogram. J. Mult. Anal. 13,
383-400.
Brii.LINGER, D. R. (1987). Fitting cosines: some procedures and some physical examples. In Applied
Probability Stochastic Processes and Sampling Theory Ed. I. B. MacNeil) and G. J. Umphrey, pp. 75-100.
Dordrecht: Reidel.
C h a n , Y. T., La v o ie , J. M. M. & Pl a n t , J. B. (1981). A parameter estimation approach to estimation of
frequencies of sinusoids. IEEE Trans. Acoust., Speech, Signal Processing ASSP-29, 214-29.
DtEUDONNE, J. (1969). Foundations of Modern Analysis. New York: Academic Press.
Fe r n a n d e s , J. M., Goodwin, G. C. & DE Souza, C. E. (1987). Estimation of models for systems having
deterministic and random disturbances. In Proceedings of 10//i World Congress on Automatic Control, 1ft,
Ed. R. Isermann, pp. 370-5. Oxford: Pcrgamon Press.
H a n n a n , E. J. (1973). The estimation of frequency. J. Appl. Probab. 10, 510 9.
Hannan, E. J. (1979). The central limit theorem for time series regression. Stoch. Proc. Appl. 9, 281-9.
Hannan, E. J. & Quinn, B. G. (1989). The resolution of closely adjacent spectral lines. J. Time Series
Anal. 1ft, 13-31.
Kay, S. M. & Ma rple, S. L. (1981). Spectrum analysis—a modern perspective. Proc. IEEE 69, 1380-419.
Mackisac k, M. S. & Poskitt, D. S. (1989). Autoregressive frequency estimation. Diometrika 76, 565-75.
Pisarenko, V. F. (1973). The retrieval of harmonics from a covariance function. Geophys. J.R. Astr. Soc.
33, 347-66.
T r u o n g -Va n , B. (1990). A new approach to frequency analysis with amplified harmonics. J. R. Statist.
Soc. B 52, 203 22.
Walker, A. M. (1973). On the estimation of a harmonic component in a time series with stationary1
dependent residuals. Adv. Appl. Prob. 5. 217-41.
Whittle, P. (1952). The simultaneous estimation of a time series harmonic components and covariance
structure. Trab. Estadist. Invesfigacion Operative 3, 43-57.
[Received September 1988. Revised February 1990]](https://image.slidesharecdn.com/afastefficienttechniquefortheestimationoffrequency-180120183053/85/A-fast-efficient-technique-for-the-estimation-of-frequency-9-320.jpg)
A fast efficient technique for the estimation of frequency
- 1. Riometrika (1991), 78, 3, pp. 489-97 Printed in Great Britain A fast efficient technique for the estimation of frequency By B. G. QUINN Defence Science and Technology Organisation, Weapons Systems Research Laboratory, Maritime Systems Division, Department of Defence, P.O. Box 1700, Salisbury, S.A. 5108, Australia a n d J. M. FERNANDES Departamento de Engenharia Eletrica, Universidade Federal de Uberlandia, MG, 38400, Brazil Su m m a r y A technique is presented for the estimation of the frequency of a sinusoid in the presence of noise. The technique is based on fitting a r m a (2,2) models iteratively, in a special way. The estimator is shown to be strongly consistent and as efficient as the least squares estimator of the frequency, or the periodogram maximizer. A simple accelerated version of the technique is shown to converge in a small number of iterations, the number depending on the accuracy of the initiator of the procedure. The results of a number of simulations are reported. Some key words: Frequency estimation; Periodogram; Sinusoidal regression. 1. I n t r o d u c t io n Estimation of the frequency of a sinusoidal component in a time series has long been of interest in many scientific disciplines and, in fact, dates from the eighteenth century. The review article of Brillinger (1987) discusses some ofthe history and practice associated with this and related problems. The statistical theory so far essentially is concerned with the estimation of frequency by nonlinear least squares and shows that frequency may be estimated with extraordinary accuracy (Whittle, 1952; Walker, 1973; Hannan, 1973; Hannan & Quinn, 1989). Several techniques have also been studied in the engineering literature (Pisarenko, 1973; Chan, Lavoie & Plant, 1981; Kay & Marple, 1981). However, if T is the length of the data, these procedures produce estimators accurate to order T *, while the least squares estimator is accurate to order T 3/' and even the maximizer of the periodogram over the Fourier frequencies is accurate to order T The advantage of the less efficient procedures, however, is that they involve fewer computations, based as they are on the estimation of autoregressive moving average, a r m a , representations of the data. Truong-Van (1990) has considered a new approach, based on ‘amplifying’ the sinusoidal component s and has analyzed his technique for the case where the noise is a stationary a r m a process. The technique is similar to the one presented in this paper. Indeed, the estimators should be the same. The algorithms which yield the estimators are, however, different, as are the conditions, justification and theoretical results. Another technique, based on maximizing the spectral density function calculated by fitting long- order autoregressions, has recently been proposed by Mackisack & Poskitt (1989). This technique is conjectured to be accurate to order T 5 but is computationally intensive. The problem addressed in this paper is the estimation of the frequency o>0in the model Y, =p* cos (too+ 0*) + «, (f = 0........T - 1), (11)
- 2. 490 where 0<coo<?r and {e,} is a strictly stationary ergodic process with zero mean and finite variance. In addition, letting be the <r-field generated by {e,; s *£/}, we assume, in order to facilitate the theory, that oc L E ,) —0, o E(v*&t-i)= E(v2,) - o-2, £ |a,|<<», a0= , (1*2) j-o where {u,} is strictly stationary and ergodic. For discussion of these conditions see An, Chen & Hannan (1983) and Hannan (1979). In particular, some of these conditions ensure that {/?,} has an absolutely continuous spectrum with continuous spectral density and that B. G. Q u i n n a n d J. M. F e r n a n d e s lim sup max -----——Of 1), r-cc - log T almost surely, where le(w) = 2T '|1 e, exp (//<*>)|2, the periodogram of {e,}. We shall also assume that /r(tt>o) = (27r)"V |X a} exp {iju>o)2>0. (1*3) With these conditions, the maximizer (oT of the periodogram IYM of {T,} has the property that Ti/2(ioT- <o0) has a distribution converging to the normal with mean zero and variance 48Trfr(co0)/p*2t a property shared with the least squares estimator of a> (Hannan, 1973, 1979). If {Y,} satisfies (1*1) then Y, - 2 cos (o0Y, , 4- Y,_2- e,- 2 cos a)0e, ., + e, .2. Moreover, if X, =1 c,Y,_; and u, = i then X, - 2 cos w{)X, , + X,_2= u, - 2 cos + w,_3. (1-4) Provided that C0= l, L|C;|<°0, X c>exP((/^o)+0, (.1*5) the spectral density Ju(io) of {m,} is positive at o>0, Iu(io) lim sup max -----—= G(l), 7_oo « log / ( 1-6) almost surely, where Iu((o) is the periodogram of {u,}, and (1.2) holds when {«,} is substituted for {e,} and b, for a). The technique introduced by Fernandes, Goodwin & De Souza (1987) consists of the construction of Butterworth filters of successively narrowing bandwidth whose passbands are likely to contain the true frequency. The limiting form of this sequence of filters, and the method of frequency estimation therefrom, suggest the procedure described in § 2, after putting cs = —1 and c, = 0 if ; +0 or 2, which has the effect of eliminating an added constant term in (Tl). An alternative motivation is given in §2 and we accordingly discuss the technique of Fernandes et al. no further. In § 3 it will be shown that the procedure produces an estimator which has the same efficiency as the maximizer of the periodogram. Moreover, owing to the lack of ‘sidelobes’,
- 3. 491 the procedure may be initialized by a relatively inefficient estimator. The accelerated version of the procedure will, for example if the initial estimator is accurate to order 0(7*"'), almost surely, produce an efficient estimator after two iterations. In §4, the results of simulations are reported, which compare the small sample behaviour of the three estimation techniques: periodogram maximization and the simple and accelerated procedures of § 2, the latter initialized in two ways. Another set of simulations compares the accelerated procedure with that of Mackisack & Poskitt (1989). Fast efficient estimation o ffrequency 2. T he estim ation tec h n iq u es As {X,} satisfies (1-4), we may consider the estimation of the parameters a and (3 in the model X ,-p X ,-}+X,_2= uf + w,_2 subject to the constraint a = p as follows. Step 1. Set a = or, = 2 cos w,, where <5, is some estimator of a>(, Step 2. For 1, let f ij- -£ -2 J 0 - 0 , . . . , t - i), where £,., = 0 (/ ——1,—2). Let /3, be the regression coefficient of (£., + £-2j) on That is, let & « I / /= 0 If |o, - Pj is suitably small, put <5= cos ‘(2ft)- Otherwise, put <*,+, = /?, and repeat Step 2. In other words, p is estimated by least squares for a fixed value of a, a is changed to this p value and the procedure iterated until a and p are as close as desired. An ‘accelerated’ version of the algorithm, suggested by the analysis of § 3, is obtained by putting aj+l = 2/3, —a, instead of p, at the end of Step 2. As will be seen, there is a fixed point of the algorithm, that is, a number a T such that p }- a, if a, - a T, and an a, such that ctj —aT is of smaller order than a0—a T|, where a 0= 2cos (a»0). Thus a, has the same relative efficiency as a T. Moreover, the problem of choosing a tolerance is unimportant for this procedure, as the number of iterations to ‘convergence’ may be determined from the accuracy of the initial estimator. 3. Asym ptotic behaviour o f th e procedures We first notice that, with {X,} and {«,} defined by (1-4), X, =p cos (tu)0+ <£)+ u„ where p2= P*2II cj exp (i/a>0)f, p exp (/<£)- p* £ c, exp {-/(>>„-</>*)}. Let £,(<*) = X, + <*£_,(ar)-6 _ 2(a), where £,(0;) = 0 (t = -1, -2; 0 < |a | <2). Then for any f 2*0, if a =2 cos w, £,(«) = (sin o>) 1X sin {(_/+ )<o}X,.
- 4. 492 B. G. Q u i n n a n d J. M. Fe r nan de s Let gr(or) = X {£(<*)+ 6_2(a)}6-i(«)/X €/-1(a ) and put h,(a) =gT(a )-a . Then hr{a) = 2 X,£,_,(a!)/2 £;_,(<*). The following theorem shows that there is a zero of hT(a) close to a0. We leave all proofs until later, and assume conditions (1*2), (1*3) and (1*5) throughout. Where order notation is used, the orders should be understood as being almost surely as T-»oo. T h e o r e m 1. Let A T={or: |a - a0| < cT '}, where e and c arefixed, 1< e <§, c>0 and a0=2 cos (o0. Then almost surely as T ->oo, there exists a unique a , e A , with the property that hT(ciT) = 0. As a trivial consequence of the theorem we have the following. Corollary. We have TF(a>T- w0)-*0, almost surely, for all e < §, where a, = 2 cos <aT. The above results relate only to the ‘fixed point’ aT of the procedure. The following theorem shows that the successive iterates converge to the fixed point aT. Its corollary demonstrates the speed of the convergence of the accelerated procedure. Theorem 2. With A, defined in Theorem 1, gT(a) = 5(0-- a T){l + 0 (T -4)} uniformly in a e At if e> 1, while, if e ^ 1, *t(o ) - « r = - <*•,■)[i + o{ T>-'(log r ) !}]+ o{ r i- 2'(iog r ) !}. Furthermore, aT- a0= 0{T~3,2{log T)’}. Corollary. Let a, c AT, and let a, = 2gr (a,_,)-a ,., (j =2, 3 ,...). Then if i< £ < l, ak- aT= o(T 3/2) for k 2sinteger part of {3-log (2e - 1)/log (2)}. The final theorem provides a central limit theorem for a>T. T h e o r e m 3. We have that T3/2(<ot - u)0) has a distribution converging to the normal with mean zero and variance 487r/r(w0)/p*2. We note here that as the standard deviation of <or is 0( T 3/2), the corollary to Theorem 2 provides the number of iterations until efficiency is attained. Proofof Theorem 1 and Corollary. We use the fixed point theorem of Dieudonne (1969, p. 266). We must show that, almost surely as T -»oo: (i) there exists k with 0 ^ k < 1such that if a, Ar, then |gT(a) —gT(a')|$ ka - a ' |, and (ii) hr(oto)< (l-k)T". Put dr (a») = sin w £ X ,£ ,(a>), eT{u)) = sin1oj £ Then hr = sin a>dT(a>)eTl(a>). Now eT((o)= sin (jw)[u,.}+p cos {(/-j)<t>0+ <f>}] - sin (jo)) cos {(/-j)a)0+(f>} +2p X X sin (yw)u,_, X sin (ka)) cos {{t-k)(o0+(f>}. (31)
- 5. Fast efficient estimation offrequency 493 The first term of (3-1), being differentiable with respect to a>, may be written as £ ( L sin 0'<w0)w,-;) r-0 l;=0 J 2 T - l / t + 2(o)-o>0) X X sin (>>*)«, i X k cos {ku>*)u,-k, i-o i-o k - 0 where o>* denotes, gcnerically, a number between w and oj0. Now, |I sin is 0{(t log f)*}» from (1-6), while i i 1 1~~J X k cos {ho*)u, k = t Z cos X X cos fc-0 k—0 Z^-l *-0 is 0{/V2(log f)*}, all orders being uniformly in w*. The first term of (3-1) is thus Z ( Z sin (y®o)Wi->| + 2(w ~(Oo)0(T- log T). /-() ly-o J We note here that, also because of (1-6), 2{E sin (_/<w)m,_^}2 is 0(T 2log T), uniformly in o). The second term of (3-1), when divided by p2, may be shown to be h{Ty+ 0{T 2)}+(a) - <o0)O(T2), uniformly in a), while the third term is 0{T 5/2(log T)*} +(oj- o>0)0{TV2Vo&DH, by (1-6). This completes the analysis of the term eT{(o). The term dT{(o) may be treated in a similar way: T - l l T - l t dT(to)= Z ui Z sin (ja>)u,_, + p2 X cos (ro>0+ </>) Z sin{jo>)cos{(t-j)<a0+4) f - 0 ) - 0 1 - 0 . i - 0 T - l t T - l + P X w, Z s in (» c o s { ( /-i/)wo+^)+ X cos (ta)0+ 4>) X sin(>a>)M,_, . i —O j -0 1 -0 1=0 J (3-2) The first term in (3-2) may be written as X w, X sin {j<o0)ul-J+((o-(o0) X u>X j cos (ju)*)u,-t r-0 j- 0 i-0 f-0 and is 0{73/2(log T)l} +{ » -w 0)O{T*/2(log T A uniformly in oo. The second term in (3-2), when divided by p2, may be shown to be 0(T) + (w -w 0){7'3/ 24+0(T 2)}, while the third term is O {r3/J(log T)!}+ (o>-a>0)O {r5/2(Iog T)>}, uniformly in co. We may now prove (i). Let a =2 cos u>and a '=2 cos w'. Then hT(a )- hT(a') = sin wdT((o)/eT((o) - sin (o'dT((o')/eT(a)'). Thus er (a>)e7(a>'){/?r (a) - hT(a')} = (sin a>-sin io')dT((o)eT((o') + sin (o'{dT((o) - dT((o')}eT((o) + sin a)'dT(u)){eT(a))- er((o')}. Careful analysis shows that gT(a ) - gT{a') =i(a -<*')[ 1+ 0{ T ‘(log 7")1}]
- 6. 494 uniformly in a and a'. Thus (i) follows for any k such that < k< 1. Part (ii) may be proved with little extra work. The dominant term in r/r (w) is 0{TV2(log T)4}, and so hr(a0) = O{T V2(log 7)*}, which is o(T e) as e < f. To prove the corollary, we simply note that the theorem holds for any e such that 1< e < 2, and that A T is contracting as e increases. Thus, almost surely, d r e A r and Ty(ar -o tu) is 0(1) for any e<2. Hence Tf(d r - a 0) and consequently T'’(wr - w0) converge to zero almost surely for any e<$. □ Proof of Theorem 2 and Corollary. If e > 1, the proof of the above theorem shows that g r(« ) - £r(<*r) - J(« “ <*r)[*+ 0 { T ~J(log T)4}]. The result of the theorem thus follows since gr (a r ) = <*r« Also, B. G. Q ijinn a n d J. M. Ff.rnandf.s {2gr(a) - a) - aT= 2{gr(a) - d ,} - (a - d , ) = ( a - d T)0 { T ~ log T)4}. Since a - a T is 0 (T *) for some e> 1, it follows that (2gT(a) - a} - aT is o{T y2). If 5<e «£l, however, the proof is more difficult. It may be shown that eT(a)) _ p iL _ r 4(w —w0)2L sin {T(w —w0)} dT((o) - d - 4(w - w„) L T(w —w0) sin {7~(w- w0)} l[i + o{ri-f(iog T)‘], T(w - w0) U+ o (r-')} + o { rV2(iogT)i} uniformly in w. Consequently, hT(a ) = -J(« - ir ) [ l + 0{T*-'(log D>}]+ 0 {T ‘-’'(log T)1} uniformly in a. But gT(o t)-d 7 = hT(a) + a - d T which is therefore - <*j-)[l + 0{T-"'(log 7 )!}]+0{T!-2r(log T)1}. Also {2gr{oc) - a} - ST=(o ,- aT) 0 { p - ( log 7)*}+ Of P 2’<log T)1}. This completes the proof of the theorem. To prove the corollary, we first note that if e > 1, the integer part of {3 - log (2e - l)/log 2} is 2, and the corollary follows from above. If ! < c ^ l , the above argument show's that {2gT(a) - a} - a T is 0 {T 3~2t(log T)4}, is therefore o(T4-2r+c), where 8 is generically positive and arbitrarily close to zero, and is thus of smaller order than (a - aT). By induction, if a, - dr is not already 0( T " ), where v > , then it must be 0 (T K), where k = - +2j 2-2'~' e + 8. We thus need to continue iterating until k < —1, that is, until j> 1-log (2e - 1)/log 2, after which the next iterate is accurate to the desired order. Hence ak- aT - o(T "V2) when fc^the integer part of {3—log (2e - l)/log 2}. In particular, a3—aT= o(T~y2) if e = 1, which is the case when the procedure is initialized by the maximizer of the periodogram over the Fourier frequencies. □ Proof of Theorem 3. From the proof of Theorem 2, it is seen that <5r - <*„= - ^ [ 1 + 0{ r-!(iog T)1}] eT(w„)
- 7. Fast efficient estimation o ffrequency 495 as aT- a 0=2hT(a0)l + O{T-'(og T)»}]. Thus, since T~3eT{a>0)->p2/24, almost surely, {T*f2p7f 24){u>r —<d0) has the same asymptotic distribution as -T~i/2dT(<o0), which may be shown to be equivalent to -T ~ V2p/2 X u,{(2t - T) sin (/o>0+ 4>)+0( 1)}. Using the results of Hannan (1979), and the first set of conditions there, it follows that T~3/2dT(<o0) is asymptotically normal with mean zero and variance 2'n-f,((o0)p2/24, where f, is the spectral density of {u,}. Consequently, r V2(a>r - a>0) is asymptotically distributed normally with mean zero and variance 487rft(<o0)/p2-The theorem follows as /j,((a0)/p2= fA<o0)/p*2- □ We note in passing that if a> is estimated by maximizing the periodogram by the Gauss-Newton technique, if (a, -co0= 0 (T r), for some e > 1, then <oj is within o(T~*'2) of the maximizer for 7'^the integer part of {2- log (2s -2)/log 3}. If e ^1, there is no guarantee that the procedure will yield the maximizer of the periodogram. This should be considered when the results of the simulations are examined. 4. S im u l a t io n s Equation (1-1) was used to generate time series of lengths T = 50, 100, 200 and 500, where {£,} was simulated by pseudo-random Gaussian numbers with standard deviations cr = 0*l, 0-3, and 0*5. In each case we put p* = 1 and the frequency co0 was tt/S +tt/T, so that the true frequency would be midway between two successive Fourier frequencies, a ‘worst case’. For each combination of sample size and standard deviation, 100 replica tions were generated, and a>was estimated by the following four techniques: (i) maximization of the periodogram by Gauss-Newton initialized by <a,„ (ii) and (iii) the simple and accelerated techniques of § 2 initialized by a>p, (iv) the accelerated technique initialized by wPis, where Sp is the maximizer of the periodogram over the Fourier frequencies 27rj/T 0 = 1 ,..., /t), with n the integer part of J(T - 1), and a>f»is is the Pisarenko estimator. For each simulation, ‘convergence’was deemed to have occurred once successive iterates were within 10 s of each other. The results produced by (iv) are surprising as <a,.is-a>0 is theoretically not of small enough order to guarantee convergence. The results of the simulations are recorded in Table 1. Clearly the best results are produced by techniques (iii) and (iv). Surprisingly, (iv) does not suffer from the deficiency evidenced by the others for the combination T = 50 and (7= 0-5, which occurred because five of the replications produced estimates of ioQnear 2. The poorer performance of (i) was expected and the theoretical reason for this is indicated at the end of §3. Table 2 contains the result of a comparison with the technique of Mackisack & Poskitt (1989), which is labelled (v). The true frequency in each case is 1-24, p = 20, <f>=0-01, cr2= 1 and 100 replications were carried out. Our technique is clearly superior. It should be noted that the signal to noise ratio for this experiment was much larger than those of the other simulations.
- 8. Tabic 1. Simulation results for techniques (i)-(iv). True frequencies 7t/5+ it/ T 496 B. G. Q u i n n a n d J. M. F l r n a n d b s Technique (i) <r o</-» li k. 7 = 100 T = 200 7 = 500 01 0-69399 0-66015 0-64425 0-63464 4-84 4-60 4 12 4-00 9-97e -6 8-68c-7 7-85e-8 2-84e -9 l-92e-6 2-40e -7 3-00e-8 l-92c-9 0-3 0-69283 0-66058 0-64417 0-63467 5-33 4-92 4-71 4-07 2-IOe-5 3-50e-6 3*30e-7 l-63e -8 1-73C-5 2-16e -6 2-70e-7 l-73e-8 0-5 0-76241 0-66116 0-64434 0-63466 617 5-01 4-78 4-20 9-55e-2 8-31e-6 8-95e-7 4-87e -8 4-80c-5 6-00c- 6 7-50e-7 4-80e-8 Technique (iii) 0-1 0-69256 0-66015 0-64416 0-63462 4-72 3-92 3-04 2-93 3-98e -6 4-49c-7 4-75e-8 2-00e-9 0-3 0-69208 0-66033 0-64411 0-63466 4-74 3-99 3-60 3-02 2-05e-5 3-33e-6 3-37c-7 1-52c- 8 0-5 0-75754 0-66112 0-64438 0-63466 5-67 4-43 3-95 3-27 9-28e-2 8-81e-6 9-75e-7 3-74c -8 II Wi o Technique (ii) 7 = 100 7 = 200 7 = 500 0-69256 12-95 3-99e-6 0-66015 12 09 4-54e-7 0-64417 11-09 4-95e-8 0-63463 10-00 2-27e -( 0-69209 13-34 2-05c-5 0-66033 12 33 3-34e-6 0-64411 11-30 3-38e-7 0-63466 10-00 1-58c—i 0-75754 14-08 9-28c-2 0-66113 12-46 8-82e-6 0-64439 11-31 9-79e-7 0-63466 10-02 3-80e -1 Technique (iv) 0-69255 0-66015 0-64416 0-63462 3-78 314 2-85 2-46 3-98e-6 4-49e -7 4-77e -8 2-01e-< 0-69208 0-66033 0-64411 0-63466 4-05 3-50 3-28 2-88 2-05e- 5 3-33e-6 3-37e-7 1-51c —1 0-69430 0-66112 0-64438 0-63466 5-06 4-43 3-96 3-56 6-92e- 5 8-81e-6 9-79e-7 3-73e —1 Top rows, averages of estimated frequencies; second third rows, sample mean squared errors; fourth rows, theoretical asymptotic variances (which apply to all entries with the same values of a and T). Table 2. Comparison of techniques (iv) and (v) 7 (iv) (v) 64 1-23970 1-23845 5-7224x lO"1 1-628 x lO"-' 128 1-24022 1-24184 2-7771 X10'4 1-214X 10"' 256 1-24000 1-24187 5-7297 xlO-5 3-89x 10 4 512 1-24002 1-24068 2-0835x10 '5 2-06 x10~4 1024 1-24000 1-24008 7-0652 xlO"6 9-7 xlO"5 1900 1-24000 1-24039 3-3342x10"A 4-9 x10“* Top rows, means of estimated frequencies; bottom rows, sample standard deviations.
- 9. 497 Ac k n o w led g em en ts B. G. Quinn was partly funded by grants from the Australian Research Council and the University of Newcastle. J. M. Fernandes was on Doctoral study leave in the Department of Electrical and Computer Engineering, University of Newcastle. His work was supported by CAPES, Brazil. Fast efficient estimation offrequency R e f e r e n c e s An, H.-Z., Chen, Z.-G. & Hannan, B. J. (1983). The maximum of the periodogram. J. Mult. Anal. 13, 383-400. Brii.LINGER, D. R. (1987). Fitting cosines: some procedures and some physical examples. In Applied Probability Stochastic Processes and Sampling Theory Ed. I. B. MacNeil) and G. J. Umphrey, pp. 75-100. Dordrecht: Reidel. C h a n , Y. T., La v o ie , J. M. M. & Pl a n t , J. B. (1981). A parameter estimation approach to estimation of frequencies of sinusoids. IEEE Trans. Acoust., Speech, Signal Processing ASSP-29, 214-29. DtEUDONNE, J. (1969). Foundations of Modern Analysis. New York: Academic Press. Fe r n a n d e s , J. M., Goodwin, G. C. & DE Souza, C. E. (1987). Estimation of models for systems having deterministic and random disturbances. In Proceedings of 10//i World Congress on Automatic Control, 1ft, Ed. R. Isermann, pp. 370-5. Oxford: Pcrgamon Press. H a n n a n , E. J. (1973). The estimation of frequency. J. Appl. Probab. 10, 510 9. Hannan, E. J. (1979). The central limit theorem for time series regression. Stoch. Proc. Appl. 9, 281-9. Hannan, E. J. & Quinn, B. G. (1989). The resolution of closely adjacent spectral lines. J. Time Series Anal. 1ft, 13-31. Kay, S. M. & Ma rple, S. L. (1981). Spectrum analysis—a modern perspective. Proc. IEEE 69, 1380-419. Mackisac k, M. S. & Poskitt, D. S. (1989). Autoregressive frequency estimation. Diometrika 76, 565-75. Pisarenko, V. F. (1973). The retrieval of harmonics from a covariance function. Geophys. J.R. Astr. Soc. 33, 347-66. T r u o n g -Va n , B. (1990). A new approach to frequency analysis with amplified harmonics. J. R. Statist. Soc. B 52, 203 22. Walker, A. M. (1973). On the estimation of a harmonic component in a time series with stationary1 dependent residuals. Adv. Appl. Prob. 5. 217-41. Whittle, P. (1952). The simultaneous estimation of a time series harmonic components and covariance structure. Trab. Estadist. Invesfigacion Operative 3, 43-57. [Received September 1988. Revised February 1990]