SlideShare a Scribd company logo
1
19. Series Representation of Stochastic Processes
Given information about a stochastic process X(t) in
can this continuous information be represented in terms of a countable
set of random variables whose relative importance decrease under
some arrangement?
To appreciate this question it is best to start with the notion of
a Mean-Square periodic process. A stochastic process X(t) is said to
be mean square (M.S) periodic, if for some T > 0
i.e
Suppose X(t) is a W.S.S process. Then
Proof: suppose X(t) is M.S. periodic. Then
,
0 T
t 

(19-1)
.
all
for
0
]
)
(
)
(
[
2
t
t
X
T
t
X
E 


( ) ( ) with 1 for all .
X t X t T probability t
 
)
(
PILLAI
( ) is mean-square perodic ( ) is periodic in the
ordinary sense, where
X t R 

*
( ) [ ( ) ( )]
R E X t X t T
  
2
But from Schwarz’ inequality
Thus the left side equals
or
i.e.,
i.e., X(t) is mean square periodic.
.
period
with
periodic
is
)
( T
R 
2 2 2
*
1 2 2 1 2 2
0
[ ( ){ ( ) ( )} ] [ ( ) ] [ ( ) ( ) ]
E X t X t T X t E X t E X t T X t
    
* *
1 2 1 2 2 1 2 1
[ ( ) ( )] [ ( ) ( )] ( ) ( )
E X t X t T E X t X t R t t T R t t
      
Then
periodic.
is
)
(
Suppose
)
( 
R

0
)
(
)
(
)
0
(
2
]
|
)
(
)
(
[| *
2





 

 R
R
R
t
X
t
X
E
( ) ( ) for any
R T R
  
  
.
0
]
)
(
)
(
[
2


 t
X
T
t
X
E (19-2)
PILLAI
(19-3)
*
1 2 2
[ ( ){ ( ) ( )} ] 0
E X t X t T X t
  
3
Thus if X(t) is mean square periodic, then is periodic and let
represent its Fourier series expansion. Here
In a similar manner define
Notice that are random variables, and
0
0
1
( ) .
T jn
n R e d
T
 
  

 
0
0
1
( )
T jk t
k
c X t e dt
T

 




k
ck,
(19-5)
PILLAI
(19-6)
)
(
R
0
0
2
( ) ,
jn
n
R e
T
  
  


 
 (19-4)
0 1 0 2
0 1 0 2
0 2 1 0 1
* *
1 1 2 2
2 0 0
2 1 1 2
2 0 0
( ) ( )
2 1 2 1 1
0 0
1
[ ] [ ( ) ( ) ]
1
( )
1 1
[ ( ) ( )]
m
T T
jk t jm t
k m
T T jk t jm t
T T jm t t j m k t
E c c E X t e dt X t e dt
T
R t t e e dt dt
T
R t t e d t t e dt
T T

 
 
 
 



   

 
  
 
 
 
4
i.e., form a sequence of uncorrelated random variables,
and, further, consider the partial sum
We shall show that in the mean square sense as
i.e.,
Proof:
But
0 1
,
1 ( )
*
1
0
0,
[ ] { }
0 .
m k
T m
j m k t
k m m T
k m
E c c e dt
k m



    

  


 (19-7)




n
n
n
c }
{
0
( ) .
N
jk t
N k
K N
X t c e 


 
)
(
)
(
~
t
X
t
XN 
2 2 *
2
[ ( ) ( ) ] [ ( ) ] 2Re[ ( ( ) ( )]
[ ( ) ].
N N
N
E X t X t E X t E X t X t
E X t
  

(19-8)
.


N
.
as
0
]
)
(
~
)
(
[
2



 N
t
X
t
X
E N
(19-9)
(19-10)
PILLAI
5
0
0
0
2
* *
( ) *
0
( )
0
[ ( ) ] (0) ,
[ ( ) ( )] [ ( )]
1
[ ( ) ( ) ]
1
[ ( ) ( )] .
k
k
k
N
jk t
N k
k N
N T jk t
k N
N N
T jk t
k
k N k N
E X t R
E X t X t E c e X t
E X e X t d
T
R t e d t
T

 
 


 
  




 

 
 
 


   


 
 

PILLAI
(19-12)
Similarly
i.e.,
0 0
2 ( ) ( )
* *
2
[ ( ) ] [ [ ] .
[ ( ) ( ) ] 2( ) 0 as
N
j k m t j k m t
N k m k m k
k m k m k N
N
N k k
k k N
E X t E c c e E c c e
E X t X t N
 

 
 


 
  
      
  
  (19-13)
0
( ) , .
jk t
k
k
X t c e t




    
 (19-14)
and
6
Thus mean square periodic processes can be represented in the form
of a series as in (19-14). The stochastic information is contained in the
random variables Further these random variables
are uncorrelated and their variances
This follows by noticing that from (19-14)
Thus if the power P of the stochastic process is finite, then the positive
sequence converges, and hence This
implies that the random variables in (19-14) are of relatively less
importance as and a finite approximation of the series in
(19-14) is indeed meaningful.
The following natural question then arises: What about a general
stochastic process, that is not mean square periodic? Can it be
represented in a similar series fashion as in (19-14), if not in the whole
interval say in a finite support
Suppose that it is indeed possible to do so for any arbitrary process
X(t) in terms of a certain sequence of orthonormal functions.
*
,
( { } )
k m k k m
E c c  
 0 as .
k k
   
.
, 



k
ck
2
(0) [ ( ) ] .
k
k
R E X t P



    

k
k



 0 as .
k k
   
,
k  
,




 t 0 ?
t T
 
PILLAI
7
i.e.,
where
and in the mean square sense
Further, as before, we would like the ck s to be uncorrelated random
variables. If that should be the case, then we must have
Now




1
)
(
)
(
~
n
k
k t
c
t
X  (19-15)
(19-16)
(19-17)
( ) ( ) in 0 .
X t X t t T
 
*
,
[ ] .
k m m k m
E c c  
 (19-18)
* * *
1 1 1 2 2 2
0 0
* *
1 1 2 2 2 1
0 0
*
1 1 2 2 2 1
0 0
[ ] [ ( ) ( ) ( ) ( ) ]
( ) { ( ) ( )} ( )
( ){ ( , ) ( ) }
T T
k m k m
T T
k m
T T
k XX m
E c c E X t t dt X t t dt
t E X t X t t dt dt
t R t t t dt dt
 
 
 



 
 
  (19-19)
PILLAI
*
0
*
,
0
( ) ( )
( ) ( ) ,
T
k k
T
k n k n
c X t t dt
t t dt

  





8
and
Substituting (19-19) and (19-20) into (19-18), we get
Since (19-21) should be true for every we must have
or
i.e., the desired uncorrelated condition in (19-18) gets translated into the
integral equation in (19-22) and it is known as the Karhunen-Loeve or
K-L. integral equation.The functions are not arbitrary
and they must be obtained by solving the integral equation in (19-22).
They are known as the eigenvectors of the autocorrelation
*
1 1 2 2 2 1 1
0 0
( ){ ( , ) ( ) ( )} 0.
XX
T T
k m m m
t R t t t dt t dt
   
 
 
1 2 2 2 1
0
( , ) ( ) ( ) 0,
XX
T
m m m
R t t t dt t
  
 

( ), 1 ,
k t k
   
*
, 1 1 1
0
( ) ( ) .
T
m k m m k m
t t dt
    
  (19-20)
(19-21)
1 2 2 2 1 1
0
( , ) ( ) ( ), 0 , 1 .
XX
T
m m m
R t t t dt t t T m
  
     


1
)}
(
{ k
k t

(19-22)
PILLAI
9
function of Similarly the set represent the eigenvalues
of the autocorrelation function. From (19-18), the eigenvalues
represent the variances of the uncorrelated random variables
This also follows from Mercer’s theorem which allows the
representation
where
Here and are known as the eigenfunctions
and eigenvalues of A direct substitution and
simplification of (19-23) into (19-22) shows that
Returning back to (19-15), once again the partial sum
1 2
( , ).
XX
R t t
*
1 2 1 2 1 2
1
( , ) ( ) ( ), 0 , ,
XX k k k
k
R t t t t t t T
  


  
 (19-23)
*
,
0
( ) ( ) .
T
k m k m
t t dt
  


1 2
( , ) respectively.
XX
R t t
)
(t
k
 ,
k

k
( ) ( ), , 1 .
k k k
t t k
   
    
1
( ) ( ) ( ), 0
N
N
k k N
k
X t c t X t t T
 

 
  

PILLAI
(19-24)
(19-25)
1
{ }
k k
 

k

,
k
c
1 .
k   
1
k   
10
in the mean square sense. To see this, consider
We have
Also
Similarly
* * *
1
* *
0
1
*
0
1
*
1
[ ( ) ( )] ( ) ( )
[ ( ) ( )] ( ) ( )
( ( , ) ( ) ) ( )
( ) ( )= | (
N
N k k
k
N T
k k
k
N T
k k
k
N
k k k k k
k
E X t X t X t c t
E X t X t d
R t d t
t t

    
    
    










 
 2
1
) | .
N
k
t


(19-26)



N
k
k
k
N t
t
X
t
X
E
1
2
*
|
)
(
|
)]
(
~
)
(
[ 

PILLAI
2 2 *
* 2
[| ( ) ( ) | ] [| ( ) | ] [ ( ) ( )]
[ ( ) ( )] [| ( ) | ].
N N
N N
E X t X t E X t E X t X t
E X t X t E X t
  
 
2
[| ( ) | ] ( , ).
E X t R t t
 (19-27)
(19-28)
(19-29)
11
and
Hence (19-26) simplifies into
i.e.,
where the random variables are uncorrelated and faithfully
represent the random process X(t) in provided
satisfy the K-L. integral equation.
Example 19.1: If X(t) is a w.s.s white noise process, determine the
sets in (19-22).
Solution: Here
2 * * 2
1
[| ( ) | ] [ ] ( ) ( ) | ( ) | .
N
N k m k m k k
k m k
E X t E c c t t t
   

 
 
.
as
0
|
)
(
|
)
,
(
]
|
)
(
~
)
(
[|
1
2
2





 

N
k
k
k
N t
t
t
R
t
X
t
X
E 

1
( ) ( ), 0 ,
k k
k
X t c t t T



 


1
}
{ k
k
c
0 ,
t T
  ( ),
k t

(19-30)
)
(
)
,
( 2
1
2
1 t
t
q
t
t
RXX 
 
PILLAI
(19-31)
(19-32)
1
{ , }
k k k
  

(19-33)
1 ,
k   
12
and
can be arbitrary so long as they are orthonormal as in (19-17)
and Then the power of the process
and in that sense white noise processes are unrealizable. However, if
the received waveform is given by
and n(t) is a w.s.s white noise process, then since any set of
orthonormal functions is sufficient for the white noise process
representation, they can be chosen solely by considering the other
signal s(t). Thus, in (19-35)
2
1 1
[| ( ) | ] (0) k
k k
P E X t R q

 
 
     
 
)
(
)
(
)
( 2
1
2
1
2
1 t
t
q
t
t
R
t
t
R ss
rr 



 
.
1
, 


 k
q
k

( ) ( ) ( ), 0
r t s t n t t T
   
)
(t
k


PILLAI
(19-34)
(19-35)
(19-36)
1 2 2 1 1 2 2 1
0 0
1 1
( , ) ( ) ( ) ( )
( ) ( )
XX
T T
k k
k k k
R t t t dt q t t t dt
q t t
  
  
 
 
 

13
and if
Then it follows that
Notice that the eigenvalues of get incremented by q.
Example19.2: X(t) is a Wiener process with
In that case Eq. (19-22) simplifies to
and using (19-39) this simplifies to
)
( 2
1 t
t
Rss 
*
1 2 1 2
1
( ) ( ) ( ) ( ).
rr k k k
k
R t t q t t
  


  

2 1 2
1 2 1 2
1 1 2
( , ) min( , ) , 0
XX
t t t
R t t t t
t t t

 



  



(19-39)
1
1
1 2 2 2 1 2 2 2
0 0
1 2 2 2 1
( , ) ( ) ( , ) ( )
( , ) ( ) ( ),
T t
XX k XX k
T
XX k k k
t
R t t t dt R t t t dt
R t t t dt t
 
  

 
 

PILLAI





1
2
*
1
2
1 )
(
)
(
)
(
k
k
k
k
ss t
t
t
t
R 

 (19-37)
(19-38)
0 1
t T
2
dt


1
1
2 2 2 1 2 2 1
0
( ) ( ) ( ).
t T
k k k k
t
t t dt t t dt t
     
 
  (19-40)
14
Derivative with respect to t1 gives [see Eqs. (8-5)-(8-6), Lecture 8]
or
Once again, taking derivative with respect to t1, we obtain
or
and its solution is given by
But from (19-40)
1
1 1 1 1 2 2 1
( ) ( 1) ( ) ( ) ( )
T
k k k k k
t
t t t t t dt t
       
   

( ) cos sin .
k k
k k k
t A t B t
 
 
  
.
)
(
)
( 1
2
2
1
 
T
t k
k
k t
dt
t 


 
,
0
)
0
( 
k

PILLAI
(19-41)
(19-42)
1 1
( 1) ( ) ( )
k k k
t t
   
 
2
1
1
2
1
( )
( ) 0,
k
k
k
d t
t
dt
 


 
(19-43)
15
(19-45)
(19-47)
(0) 0, 1 ,
( ) cos ,
k k
k k
k k
A k
t B t
 
 


    

PILLAI
and from (19-41)
This gives
and using (19-44) we obtain
Also
( ) 0.
k T
  (19-44)
 
2
2 2
1
2
( ) cos 0
2 1
2
, 1 .
( )
k k
k
k k
k
T B T
T k
T
k
k
 
 







 
  
    

(19-46)
16
PILLAI
Further, orthonormalization gives
Hence
with as in (19-47) and as in (19-16),
( ) sin , 0 .
k
k k
t B t t T


    (19-48)
 
2 1 cos2
2 2 2
2
0 0 0
sin2 sin(2 1) 0
2 2 2
2 4
0
2
( ) sin
1
1
2 2 2
2/ .
k
k
k
k k
t
T T T
k k k
T
t k
k k k
k
T
t dt B t dt B dt
T T
B B B
B T




 
 





 
 
 
   
 
 
 
   
     
 
 
 
 
 
  
(19-49)
   
2 2
,
1
2
( ) sin sin
k
k T T
t
T
t t k



   
k
 k
c
17
is the desired series representation.
Example 19.3: Given
find the orthonormal functions for the series representation of the
underlying stochastic process X(t) in 0 < t < T.
Solution: We need to solve the equation
Notice that (19-51) can be rewritten as,
Differentiating (19-52) once with respect to t1, we obtain
PILLAI




1
)
(
)
(
k
k
k t
c
t
X 
| |
( ) , 0,
XX
R e  
 

  (19-50)
0 1
t T
2
dt


2
dt


1 2
| |
2 2 1
0
( ) ( ).
T t t
n n n
e t dt t

  
 

 (19-51)
0 0
1
1 2 2 1
1
t ( ) ( )
2 2 2 2 1
0 t
( ) ( ) ( )
T
t t t t
n n n n
e t dt e t dt t
 
   
 
   
 
  (19-52)
18
Differentiating (19-53) again with respect to t1, we get
or
1
1 2 2 1
1
1
1 2 2 1
1
( ) ( )
1 2 2 1 2 2
0
1
1
( ) ( ) 1
2 2 2 2
0
1
( ) ( ) ( ) ( ) ( )
( )
( )
( ) ( )
t T
t t t t
n n n n
t
n
n
t T
t t t t n n
n n
t
t e t dt t e t dt
d t
dt
d t
e t dt e t dt
dt
 
 
     


 
 

   
   
   

   
 
  (19-53)
1
1 2
2 1
1
( )
1 2 2
0
2
( ) 1
1 2 2 2
1
( ) ( ) ( )
( )
( ) ( )
t t t
n n
T t t n n
n n
t
t e t dt
d t
t e t dt
dt


  
 
  

 
 
  
  


1
1 2 2 1
1
1
( ) ( )
1 2 2 2 2
0
( ) {use (19-52)}
2
1
2
1
2 ( ) ( ) ( )
( )
n n
t T
t t t t
n n n
t
t
n n
t e t dt e t dt
d t
dt
 
 
   
 

   
 
  
 

 
PILLAI
19
or
or
Eq.(19-54) represents a second order differential equation. The solution
for depends on the value of the constant on the
right side. We shall show that solutions exist in this case only if
or
In that case
Let
and (19-54) simplifies to
( )
n t

2
1
1 2
1
( )
( 2) ( ) n n
n n
d t
t
dt
 
 

 
2
1
1
2
1
( ) ( 2)
( ).
n n
n
n
d t
t
dt
  



 
  
 
(19-54)
(19-55)
PILLAI
( 2)/
n n
  

2,
n
 
( 2)/ 0.
n n
  
 
2
0 .
n


 
(19-56)
2
2
1
1
2
1
( )
( ).
n
n n
d t
t
dt

 
  (19-57)
2 (2 )
0,
n
n
n
 



 

20
PILLAI
General solution of (19-57) is given by
From (19-52)
and
Similarly from (19-53)
and
Using (19-58) in (19-61) gives
1 1 1
( ) cos sin .
n n n n n
t A t B t
  
  (19-58)
2
2 2
0
1
(0) ( )
T t
n n
n
e t dt


 

  (19-59)
2
2 2
0
1
( ) ( ) .
T t
n n
n
T e T t dt


 

 
 (19-60)
2
1
1
2 2
0
1 0
( )
(0) ( ) (0)
T t
n
n n n
n
t
d t
e t dt
dt

 
  



  
 (19-61)
2
2 2
0
( ) ( ) ( ).
T t
n n n
n
T e T t dt T


  


    
 (19-62)
21
PILLAI
or
and using (19-58) in (19-62), we have
or
Thus are obtained as the solution of the transcendental equation
,
n n
n
A
B



n n n
B A
 

(19-63)
(19-64)
sin cos ( cos sin ),
( )cos ( )sin
n n n n n n n n n n
n n n n n n n n
A T B T A T B T
A B T A B T
      
     
    
   
2
2 2 / 2 / 2( / )
tan .
( ) 1
1 1
n n n n n n
n
n
n n n n
n n n
n n
A A
B B
A A B A B
T
A B


 


    

 
   
 
 
2
2( / )
tan ,
( / ) 1
n
n
n
T
 

 


s
n

22
which simplifies to
In terms of from (19-56) we get
Thus the eigenvalues are obtained as the solution of the transcendental
equation (19-65). (see Fig 19.1). For each such the
corresponding eigenvector is given by (19-58). Thus
since from (19-65)
and cn is a suitable normalization constant.
2
(or ),
n n
 
2
( ) cos sin
sin( ) sin ( ), 0
n n n n n
n n n n n
T
t A t B t
c t c t t T
  
  
 
     
(19-66)
2 2
2
0.
n
n


 
 

1 1
tan tan /2,
n n
n n
n
A
T
B

 

 
   
    
 
 
 
 
(19-68)
(19-67)
s
n

PILLAI
tan( / 2) .
n
nT



  (19-65)
23
PILLAI
Fig 19.1 Solution for Eq.(19-65).
2

1

T


0
tan( / 2)
T

/
 



24
PILLAI
Karhunen – Loeve Expansion for Rational Spectra
[The following exposition is based on Youla’s classic paper “The
solution of a Homogeneous Wiener-Hopf Integral Equation occurring
in the expansion of Second-order Stationary Random Functions,” IRE
Trans. on Information Theory, vol. 9, 1957, pp 187-193. Youla is
tough. Here is a friendlier version. Even this may be skipped on a first
reading. (Youla can be made only so much friendly.)]
Let X(t) represent a w.s.s zero mean real stochastic process with
autocorrelation function so that its power spectrum
is nonnegative and an even function. If is rational, then the
process X(t) is said to be rational as well. rational and even
implies
( ) ( )
XX XX
R R
 
 
( )
XX
S 
( )
XX
S 
0
( ) ( ) 2 ( )cos
XX XX XX
j t
S R e dt R d

    
 


 
  (19-69)
(19-70)
2
2
( )
( ) 0.
( )
XX
N
S
D



 
25
PILLAI
The total power of the process is given by
and for P to be finite, we must have
(i) The degree of the denominator polynomial
must exceed the degree of the numerator polynomial
by at least two,
and
(ii) must not have any zeros on the real-frequency
axis.
The s-plane extension of is given by
Thus
and the Laplace inverse transform of is given by
2
2
( )
( )
1 1
2 2
( )
XX
N
D
P S d d


 
  
 
 
 
  (19-71)
( ) 2
D n
  2
( )
D 
2
( )
N 
( ) 2
N m
 
2
( )
D  ( )
s j

( )
XX
S 
(19-72)
2 2
( ) k
s  

2 2 2
( ) ( ) i
k
k
k
D s s 
  
 (19-73)
( )
s j
 
 
2
2
2
( )
( ) | ( ) .
( )
XX s j
N s
S S s
D s

 

 


26
PILLAI
Let represent the roots of D(– s2) . Then
Let D+(s) and D–(s) represent the left half plane (LHP) and the right half
plane (RHP) products of these roots respectively. Thus
where
This gives
Notice that has poles only on the LHP and its inverse (for all t > 0)
converges only if the strip of convergence is to the right
1 2
, , , n
  
  
1 2
0 Re Re Re n
  
   
(19-74)
(19-75)
| |
2 2 1
1
1 ( 1) ( 2)! | |
( 1)! ( 1)!( )!
( ) (2 )
k k j
k
k k j
j
k j
e
k j k j
s
  
 


 

  

  


2
( ) ( ) ( ),
D s D s D s
 
  (19-76)
1 ( )
( )
C s
D s

*
0
( ) ( )( ) ( ).
n
k
k k k
k k
D s s s d s D s
 
 

     
  (19-77)
2
2 1 2
2
( ) ( )
( )
( )
( ) ( ) ( )
C s C s
N s
S s
D s D s D s
 

  

(19-78)
27
PILLAI
of all its poles. Similarly
C2(s) /D–(s) has poles only on
the RHP and its inverse will
converge only if the strip is
to the left of all those poles. In
that case, the inverse exists for
t < 0. In the case of from
(19-78) its transform N(s2) /D(–s2)
is defined only for (see Fig 19.2). In particular,
for from the above discussion it follows that is given
by the inverse transform of C1(s) /D+(s). We need the solution to the
integral equation
that is valid only for 0 < t < T. (Notice that in (19-79) is the
reciprocal of the eigenvalues in (19-22)). On the other hand, the right
side (19-79) can be defined for every t. Thus, let
( ),
XX
R 
1 1
Re Re Re
s
 
  
0,
 

0
( ) ( ) ( ) , 0
XX
T
t R t d t T
     
   
 (19-79)
(19-80)
( )
XX
R 
Fig 19.2
 
Re n

 1
Re 
 Re n

1
Re 
strip of convergence
for ( )
XX
R 
s j

 
0
( ) ( ) ( ) ,
XX
T
g t R t d t
   
      


28
PILLAI
and to confirm with the same limits, define
This gives
and let
Clearly
and for t > T
since RXX(t) is a sum of exponentials Hence it
follows that for t > T, the function f (t) must be a sum of exponentials
Similarly for t < 0
, for 0.
k t
k
k
a e t




.
kt
k
k
a e 


( ) 0
( ) .
0 otherwise
t t T
t


 

 

(19-81)
(19-82)
+
( ) ( ) ( )
XX
g t R t d
   


 

+
( ) ( ) ( ) ( ) ( ) ( ) .
XX
f t t g t t R t d
       


    
 (19-83)
( ) 0, 0
f t t T
   (19-84)
   
( ) 0
+
( ) { ( )} ( ) 0,
k
XX
D
d d
dt dt
D f t D R t d

    

 

 

   
 (19-85)
29
PILLAI
and hence f (t) must be a sum of exponentials
Thus the overall Laplace transform of f (t) has the form
where P(s) and Q(s) are polynomials of degree n – 1 at most. Also from
(19-83), the bilateral Laplace transform of f (t) is given by
Equating (19-86) and (19-87) and simplifying, Youla obtains the key
identity
Youla argues as follows: The function is an entire
function of s, and hence it is free of poles on the entire
   
( ) 0
+
( ) { ( )} ( ) 0,
k
XX
D
d d
dt dt
D f t D R t d

    



 

   

, for 0.
k t
k
k
b e t



contributes to 0
contributions
in < 0
contributions in
( ) ( )
( )
( ) ( )
sT
t
t
t T
P s Q s
F s e
D s D s

 


 
(19-86)
2
2 1 1
( )
( )
( ) ( ) 1 , Re Re Re
N s
D s
F s s s
  


 
     
  (19-87)
(19-88)
2 2
( ) ( ) ( ) ( )
( ) .
( ) ( )
sT
P s D s e Q s D s
s
D s N s

  

 
  
0
( ) ( )
T st
s t e dt
 
  
30
PILLAI
finite s-plane However, the denominator on the right
side of (19-88) is a polynomial and its roots contribute to poles of
Hence all such poles must be cancelled by the numerator. As a result
the numerator of in (19-88) must possess exactly the same set of
zeros as its denominator to the respective order at least.
Let be the (distinct) zeros of the
denominator polynomial Here we assume that
is an eigenvalue for which all are distinct. We have
These also represent the zeros of the numerator polynomial
Hence
and
which simplifies into
From (19-90) and (19-92) we get
).
Re
( 


 s
)
(s

1 2
( ), ( ), , ( )
n
     
  
2 2
( ) ( ).
D s N s

   
'
k s

'
k s

( ) ( ) ( ) ( ).
sT
P s D s e Q s D s
  

1 2
0 Re ( ) Re ( ) Re ( ) .
n
     
      (19-89)
( ) ( ) ( ) ( )
kT
k k k k
D P e D Q

   

 
 (19-90)
(19-91)
( ) ( ) ( ) ( )
kT
k k k k
D P e D Q

   
 
    
( ) ( ) ( ) ( ).
kT
k k k k
D P e D Q

   
 
   (19-92)
( ).
s

31
PILLAI
i.e., the polynomial
which is at most of degree n – 1 in s2 vanishes at
(for n distinct values of s2). Hence
or
Using the linear relationship among the coefficients of P(s) and Q(s)
in (19-90)-(19-91) it follows that
are the only solutions that are consistent with each of those equations,
and together we obtain
( ) ( ) ( ) ( ), 1, 2, ,
k k k k
P P Q Q k n
   
    (19-93)
( ) ( ) ( ) ( ) ( )
L s P s P s Q s Q s
    (19-94)
2 2 2
1 2
, , , n
  
2
( ) 0
L s  (19-95)
( ) ( ) ( ) ( ).
P s P s Q s Q s
   (19-96)
( ) ( ) or ( ) ( )
P s Q s P s Q s
     (19-97)
32
PILLAI
as the only solution satisfying both (19-90) and (19-91). Let
In that case (19-90)-(19-91) simplify to (use (19-98))
where
For a nontrivial solution to exist for in (19-100), we
must have
( ) ( )
P s Q s
   (19-98)
1
0
( ) .
n
i
i
i
P s p s


  (19-99)
0 1 1
, , , n
p p p 
1
0
( ) ( ) ( ) ( )
{1 ( 1) } 0, 1,2, ,
kT
k k k k
n
i i
k k i
i
P D e D P
a p k n

   


 



   
 (19-100)
( ) ( )
.
( ) ( )
k k
T T
k k
k
k k
D D
a e e
D D
 
 
 
 
 
 

  (19-101)
33
PILLAI
The two determinant conditions in (19-102) must be solved together to
obtain the eigenvalues that are implicitly contained in the
and (Easily said than done!).
To further simplify (19-102), one may express ak in (19-101) as
so that
'
i s
 '
i
a s
'
i s

2
, 1, 2, ,
k
k
a e k n


 
1 1
1 1 1 1 1
1 1
2 2 2 2 2
1,2
1 1
(1 ) (1 ) (1 ( 1) )
(1 ) (1 ) (1 ( 1) )
0.
(1 ) (1 ) (1 ( 1) )
n n
n n
n n
n n n n n
a a a
a a a
a a a
 
 
 
 
 
 
 
 
  
 
(19-102)
(19-104)
(19-103)
/ 2 / 2
/ 2 / 2
1 ( ) ( )
tan
1 ( ) ( )
( ) ( )
( ) ( )
k
k k
k k k
k k
k k
T
k k k
k T
k k k
T T
k k
T T
k k
a D e D
e e
a
e e D e D
e D e D
e D e D

 
  
 
 
 

 
 
 

 

 
 

 

 
  

  

  
 

 
h
34
PILLAI
Let
and substituting these known coefficients into (19-104) and simplifying
we get
and in terms of in (19-102) simplifies to
if n is even (if n is odd the last column in (19-107) is simply
Similarly in (19-102) can be obtained by
replacing with in (19-107).
0 1
( ) n
n
D s d d s d s

    (19-105)
2
tan ,
k
h  
2 3
0 2 1 3
2 3
0 2 1 3
( )tan ( / 2) ( )
tan
( ) ( )tan ( / 2)
k k k k
k
k k k k
d d T d d
d d d d T
   

   
    

    
(19-106)
1 1 1
1 2
[ , , , ] ).
n n n
n
T
  
  
1

cot k

h
tan k

h
2 3 1
1 1 1 1 1 1 1
2 3 1
2 2 2 2 2 2 2
1 tan tan tan
1 tan tan tan
1 tan
n
n
n
      
      



2 3 1
0
tan tan
n
n n n n n n
     

 (19-107)
h
h
h
h
h
h
h
h
h
h
h
h
35
PILLAI
To summarize determine the roots with that satisfy
in terms of and for every such determine using (19-106).
Finally using these and in (19-107) and its companion
equation , the eigenvalues are determined. Once are
obtained, can be solved using (19-100), and using that can
be obtained from (19-88).
Thus
and
Since is an entire function in (19-110), the inverse Laplace
transform in (19-109) can be performed through any strip of
convergence in the s-plane, and in particular if we use the strip
,
 ,
k
 k

'
k s

2 2
( ) ( ) 0, 1, 2, ,
k k
D N k n
  
     (19-108)
k s
 tanh k s

k s
 k s

k
p s ( )
i s

( )
i s

2 2
( ) ( , ) ( ) ( , )
( )
( ) ( )
sT
i i
i
i
D s P s e D s Q s
s
D s N s
 

  

 
  
(19-109)
1
( ) { ( )}.
i i
t L s
 
  (19-110)
1

Re( ) 0
i
 
36
PILLAI
then the two inverses
obtained from (19-109) will be causal. As a result
will be nonzero only for t > T and using this in (19-109)-(19-110) we
conclude that for 0 < t < T has contributions only from the first
term in (19-111). Together with (19-81), finally we obtain the desired
eigenfunctions to be
that are orthogonal by design. Notice that in general (19-112)
corresponds to a sum of modulated exponentials.
Re Re( ) (to the right of all Re( )),
n i
s  

1 1
2 2 2 2
( ) ( ) ( ) ( )
,
( ) ( ) ( ) ( )
D s P s D s Q s
L L
D s N s D s N s
 
 
 
   
   
     
   
(19-111)
(19-112)
 
2 2
1 ( ) ( )
( ) ( )
sT D s Q s
e
D s N s
L




  
( )
i t

1
2 2
( ) ( , )
( ) , 0 ,
( ) ( )
Re Re 0, 1,2, ,
k
k
k
n
D s P s
t L t T
D s N s
s k n





  
  
 
  
 
  
37
PILLAI
Next, we shall illustrate this procedure through some examples. First,
we shall re-do Example 19.3 using the method described above.
Example 19.4: Given we have
This gives and P(s), Q(s) are constants
here. Moreover since n = 1, (19-102) reduces to
and from (19-101), satisfies
or is the solution of the s-plane equation
But |esT| >1 on the RHP, whereas on the RHP. Similarly
|esT| <1 on the LHP, whereas on the LHP.
| |
( ) ,
XX
R e  
 

( ) , ( )
D s s D s s
 
 
   
1 1
1 0, or 1
a a
   
1

1

sT s
e
s





2
2 2 2
2 ( )
( ) .
( )
XX
N
S
D
 

  
 

(19-113)
(19-114)
1 1 1
1
1
( )
( )
T D
e
D
   
 




 

1
s
s



 
1
s
s



 
38
PILLAI
Thus in (19-114) the solution s must be purely imaginary, and hence
in (19-113) is purely imaginary. Thus with in (19-114)
we get
or
which agrees with the transcendental equation (19-65). Further from
(19-108), the satisfy
or
Notice that the in (19-66) is the inverse of (19-116) because as
noted earlier in (19-79) is the inverse of that in (19-22).
1
 1
s j


n

2 2
0.
2
n
n
 



  (19-116)
(19-115)
2 2 2 2
( ) ( ) 2 0
n
n n n
s j
D s N s 
   

      
1
1
tan( / 2)
T



 
1 1
1
j T j
e
j
  
 



s

39
PILLAI
Finally from (19-112)
which agrees with the solution obtained in (19-67). We conclude this
section with a less trivial example.
Example 19.5
In this case
This gives With n = 2,
(19-107) and its companion determinant reduce to
1
2 2
( ) cos sin , 0
n n n n n
n
s
t L A t B t t T
s

  

  

    
 

 
(19-117)
| | | |
( ) .
XX
R e e
   
  
  (19-118)
2
2 2 2 2 2 2 2 2
2 2 2( )( )
( ) .
( )( )
XX
S
     

       
 
  
   
(19-119)
2
( ) ( )( ) ( ) .
D s s s s s
    

      
2 2 1 1
2 2 1 1
tan tan
cot cot
   
   


h h
h h
40
PILLAI
or
From (19-106)
Finally can be parametrically expressed in terms of
using (19-108) and it simplifies to
This gives
and
(19-120)
(19-121)
2 2
1 2
and
 
2
2
1
( ) ( ) 4 ( )
2
b b c
  

 

1 2
tan tan .
 
 
h h
2
2
( )tan ( / 2) ( )
tan , 1,2
( ) ( ) tan ( / 2)
i i i
i
i i i
T
i
T
     

     
  
 
  
h
h
h

2 2 4 2 2 2
2 2
4 2
( ) ( ) ( 2 ( ))
2 ( )
0.
D s N s s s
s bs c
     
     
       
  
   

41
PILLAI
and
and substituting these into (19-120)-(19-121) the corresponding
transcendental equation for can be obtained. Similarly the
eigenfunctions can be obtained from (19-112).
2
2 2 2
2 1
( ) ( ) 4 ( )
( ) 4 ( )
2
b b c
b c
  
   
 
   
i s


More Related Content

PPT
Explaination of power spectrum ectr18.ppt
PPT
Poisson process and explaination lectr15.ppt
PPTX
Laplace periodic function with graph
PPT
Linear transformation.ppt
PDF
Remark on variance swaps pricing
PDF
Remark on variance swaps pricing new
PPT
lecture about different methods and explainanation.ppt
PPT
Scholastic process and explaination lectr14.ppt
Explaination of power spectrum ectr18.ppt
Poisson process and explaination lectr15.ppt
Laplace periodic function with graph
Linear transformation.ppt
Remark on variance swaps pricing
Remark on variance swaps pricing new
lecture about different methods and explainanation.ppt
Scholastic process and explaination lectr14.ppt

Similar to Series representation of solistics lectr19.ppt (20)

PPTX
Linear transformations-thestuffpoint.com
PDF
Asymptotic Behavior of Solutions of Nonlinear Neutral Delay Forced Impulsive ...
PPT
chapter-2.ppt control system slide for students
PDF
ENGI5671Matcont
PDF
Beamer 4.pdf
PDF
Stability Result of Iterative procedure in normed space
PDF
Ck4201578592
PDF
17 16512 32451-1-sm (edit ari)
PDF
17 16512 32451-1-sm (edit ari)
PDF
A0212010109
PPT
The fourier series signals and systems by R ismail
PDF
D021018022
PPT
lectr20 extinction probability of different.ppt
PPT
nabil-201-chap-06.ppt
PPTX
EC8352-Signals and Systems - Laplace transform
PDF
8 Continuous-Time Fourier Transform Solutions To Recommended Problems
PDF
Research Inventy : International Journal of Engineering and Science
PDF
Signal and Systems part i
PDF
8fbf4451c622e6efbcf7452222d21ea5_MITRES_6_007S11_hw02.pdf
DOCX
Continous time fourier series and Examples
Linear transformations-thestuffpoint.com
Asymptotic Behavior of Solutions of Nonlinear Neutral Delay Forced Impulsive ...
chapter-2.ppt control system slide for students
ENGI5671Matcont
Beamer 4.pdf
Stability Result of Iterative procedure in normed space
Ck4201578592
17 16512 32451-1-sm (edit ari)
17 16512 32451-1-sm (edit ari)
A0212010109
The fourier series signals and systems by R ismail
D021018022
lectr20 extinction probability of different.ppt
nabil-201-chap-06.ppt
EC8352-Signals and Systems - Laplace transform
8 Continuous-Time Fourier Transform Solutions To Recommended Problems
Research Inventy : International Journal of Engineering and Science
Signal and Systems part i
8fbf4451c622e6efbcf7452222d21ea5_MITRES_6_007S11_hw02.pdf
Continous time fourier series and Examples
Ad

More from sadafshahbaz7777 (20)

PDF
PastandpresentofPunjabi 24567642245(1).pdf
PPTX
resham_localgovts and powers and functions.pptx
PPT
ITPTeaching201026899393876892827799866.ppt
PPTX
IOP-Limit-Less-careers-lesson-for-teachers.pptx
PPT
Ch5_slides Qwertr12234543234433444344.ppt
PPT
Lecture 4-Structure and function of carbohydrates .ppt
PPTX
social_media_for_research 234664445.pptx
PDF
647883385-Role-of-Sufi-in-the-spread-of-Islam-in-subcontinent.pdf
PPT
Kee_Pookong_01.ppt 2579975435676667788888
PPTX
160572975823-intro-to-social-studies.pptx
PPT
C3LC_Waring_ap_Run Through_4-27-15_compressed.ppt
PDF
320936716-c-Constitutional-Development-of-Pakistan-Since-1947-to-Date (2).pdf
PDF
373745833-235433Constitutional-Issues.pdf
PDF
Choudhury-ConstitutionMakingDilemmasPakistan-1955.pdf
PPTX
Adverse Childhood Experiences Supplemental PowerPoint Slides (PPTX).pptx
PDF
DevelopmentofLocalGovernanceandDecentralizationtoempowerCitizensinPakistan-AH...
PPT
Lecture 1-disp2456542234566.2456655555ppt
PPT
First financial management 23566432245556
PPT
lecture_223⁵4323564334555543343333334.ppt
PPTX
CECS ejn working 11-8-246787654455517.pptx
PastandpresentofPunjabi 24567642245(1).pdf
resham_localgovts and powers and functions.pptx
ITPTeaching201026899393876892827799866.ppt
IOP-Limit-Less-careers-lesson-for-teachers.pptx
Ch5_slides Qwertr12234543234433444344.ppt
Lecture 4-Structure and function of carbohydrates .ppt
social_media_for_research 234664445.pptx
647883385-Role-of-Sufi-in-the-spread-of-Islam-in-subcontinent.pdf
Kee_Pookong_01.ppt 2579975435676667788888
160572975823-intro-to-social-studies.pptx
C3LC_Waring_ap_Run Through_4-27-15_compressed.ppt
320936716-c-Constitutional-Development-of-Pakistan-Since-1947-to-Date (2).pdf
373745833-235433Constitutional-Issues.pdf
Choudhury-ConstitutionMakingDilemmasPakistan-1955.pdf
Adverse Childhood Experiences Supplemental PowerPoint Slides (PPTX).pptx
DevelopmentofLocalGovernanceandDecentralizationtoempowerCitizensinPakistan-AH...
Lecture 1-disp2456542234566.2456655555ppt
First financial management 23566432245556
lecture_223⁵4323564334555543343333334.ppt
CECS ejn working 11-8-246787654455517.pptx
Ad

Recently uploaded (20)

PDF
Systems Analysis and Design, 12th Edition by Scott Tilley Test Bank.pdf
PPTX
Topic 5 Presentation 5 Lesson 5 Corporate Fin
PDF
Introduction to the R Programming Language
DOCX
Factor Analysis Word Document Presentation
PPTX
sac 451hinhgsgshssjsjsjheegdggeegegdggddgeg.pptx
PPTX
A Complete Guide to Streamlining Business Processes
PDF
Data Engineering Interview Questions & Answers Cloud Data Stacks (AWS, Azure,...
PPTX
FMIS 108 and AISlaudon_mis17_ppt_ch11.pptx
PDF
Optimise Shopper Experiences with a Strong Data Estate.pdf
PDF
Data Engineering Interview Questions & Answers Batch Processing (Spark, Hadoo...
PPTX
chrmotography.pptx food anaylysis techni
PDF
Introduction to Data Science and Data Analysis
PPTX
Managing Community Partner Relationships
PPTX
Business_Capability_Map_Collection__pptx
PDF
Tetra Pak Index 2023 - The future of health and nutrition - Full report.pdf
PDF
Capcut Pro Crack For PC Latest Version {Fully Unlocked 2025}
PDF
Votre score augmente si vous choisissez une catégorie et que vous rédigez une...
PDF
Microsoft 365 products and services descrption
PPTX
Copy of 16 Timeline & Flowchart Templates – HubSpot.pptx
PPT
Predictive modeling basics in data cleaning process
Systems Analysis and Design, 12th Edition by Scott Tilley Test Bank.pdf
Topic 5 Presentation 5 Lesson 5 Corporate Fin
Introduction to the R Programming Language
Factor Analysis Word Document Presentation
sac 451hinhgsgshssjsjsjheegdggeegegdggddgeg.pptx
A Complete Guide to Streamlining Business Processes
Data Engineering Interview Questions & Answers Cloud Data Stacks (AWS, Azure,...
FMIS 108 and AISlaudon_mis17_ppt_ch11.pptx
Optimise Shopper Experiences with a Strong Data Estate.pdf
Data Engineering Interview Questions & Answers Batch Processing (Spark, Hadoo...
chrmotography.pptx food anaylysis techni
Introduction to Data Science and Data Analysis
Managing Community Partner Relationships
Business_Capability_Map_Collection__pptx
Tetra Pak Index 2023 - The future of health and nutrition - Full report.pdf
Capcut Pro Crack For PC Latest Version {Fully Unlocked 2025}
Votre score augmente si vous choisissez une catégorie et que vous rédigez une...
Microsoft 365 products and services descrption
Copy of 16 Timeline & Flowchart Templates – HubSpot.pptx
Predictive modeling basics in data cleaning process

Series representation of solistics lectr19.ppt

  • 1. 1 19. Series Representation of Stochastic Processes Given information about a stochastic process X(t) in can this continuous information be represented in terms of a countable set of random variables whose relative importance decrease under some arrangement? To appreciate this question it is best to start with the notion of a Mean-Square periodic process. A stochastic process X(t) is said to be mean square (M.S) periodic, if for some T > 0 i.e Suppose X(t) is a W.S.S process. Then Proof: suppose X(t) is M.S. periodic. Then , 0 T t   (19-1) . all for 0 ] ) ( ) ( [ 2 t t X T t X E    ( ) ( ) with 1 for all . X t X t T probability t   ) ( PILLAI ( ) is mean-square perodic ( ) is periodic in the ordinary sense, where X t R   * ( ) [ ( ) ( )] R E X t X t T   
  • 2. 2 But from Schwarz’ inequality Thus the left side equals or i.e., i.e., X(t) is mean square periodic. . period with periodic is ) ( T R  2 2 2 * 1 2 2 1 2 2 0 [ ( ){ ( ) ( )} ] [ ( ) ] [ ( ) ( ) ] E X t X t T X t E X t E X t T X t      * * 1 2 1 2 2 1 2 1 [ ( ) ( )] [ ( ) ( )] ( ) ( ) E X t X t T E X t X t R t t T R t t        Then periodic. is ) ( Suppose ) (  R  0 ) ( ) ( ) 0 ( 2 ] | ) ( ) ( [| * 2          R R R t X t X E ( ) ( ) for any R T R       . 0 ] ) ( ) ( [ 2    t X T t X E (19-2) PILLAI (19-3) * 1 2 2 [ ( ){ ( ) ( )} ] 0 E X t X t T X t   
  • 3. 3 Thus if X(t) is mean square periodic, then is periodic and let represent its Fourier series expansion. Here In a similar manner define Notice that are random variables, and 0 0 1 ( ) . T jn n R e d T         0 0 1 ( ) T jk t k c X t e dt T        k ck, (19-5) PILLAI (19-6) ) ( R 0 0 2 ( ) , jn n R e T            (19-4) 0 1 0 2 0 1 0 2 0 2 1 0 1 * * 1 1 2 2 2 0 0 2 1 1 2 2 0 0 ( ) ( ) 2 1 2 1 1 0 0 1 [ ] [ ( ) ( ) ] 1 ( ) 1 1 [ ( ) ( )] m T T jk t jm t k m T T jk t jm t T T jm t t j m k t E c c E X t e dt X t e dt T R t t e e dt dt T R t t e d t t e dt T T                            
  • 4. 4 i.e., form a sequence of uncorrelated random variables, and, further, consider the partial sum We shall show that in the mean square sense as i.e., Proof: But 0 1 , 1 ( ) * 1 0 0, [ ] { } 0 . m k T m j m k t k m m T k m E c c e dt k m                (19-7)     n n n c } { 0 ( ) . N jk t N k K N X t c e      ) ( ) ( ~ t X t XN  2 2 * 2 [ ( ) ( ) ] [ ( ) ] 2Re[ ( ( ) ( )] [ ( ) ]. N N N E X t X t E X t E X t X t E X t     (19-8) .   N . as 0 ] ) ( ~ ) ( [ 2     N t X t X E N (19-9) (19-10) PILLAI
  • 5. 5 0 0 0 2 * * ( ) * 0 ( ) 0 [ ( ) ] (0) , [ ( ) ( )] [ ( )] 1 [ ( ) ( ) ] 1 [ ( ) ( )] . k k k N jk t N k k N N T jk t k N N N T jk t k k N k N E X t R E X t X t E c e X t E X e X t d T R t e d t T                                       PILLAI (19-12) Similarly i.e., 0 0 2 ( ) ( ) * * 2 [ ( ) ] [ [ ] . [ ( ) ( ) ] 2( ) 0 as N j k m t j k m t N k m k m k k m k m k N N N k k k k N E X t E c c e E c c e E X t X t N                           (19-13) 0 ( ) , . jk t k k X t c e t           (19-14) and
  • 6. 6 Thus mean square periodic processes can be represented in the form of a series as in (19-14). The stochastic information is contained in the random variables Further these random variables are uncorrelated and their variances This follows by noticing that from (19-14) Thus if the power P of the stochastic process is finite, then the positive sequence converges, and hence This implies that the random variables in (19-14) are of relatively less importance as and a finite approximation of the series in (19-14) is indeed meaningful. The following natural question then arises: What about a general stochastic process, that is not mean square periodic? Can it be represented in a similar series fashion as in (19-14), if not in the whole interval say in a finite support Suppose that it is indeed possible to do so for any arbitrary process X(t) in terms of a certain sequence of orthonormal functions. * , ( { } ) k m k k m E c c    0 as . k k     . ,     k ck 2 (0) [ ( ) ] . k k R E X t P          k k     0 as . k k     , k   ,      t 0 ? t T   PILLAI
  • 7. 7 i.e., where and in the mean square sense Further, as before, we would like the ck s to be uncorrelated random variables. If that should be the case, then we must have Now     1 ) ( ) ( ~ n k k t c t X  (19-15) (19-16) (19-17) ( ) ( ) in 0 . X t X t t T   * , [ ] . k m m k m E c c    (19-18) * * * 1 1 1 2 2 2 0 0 * * 1 1 2 2 2 1 0 0 * 1 1 2 2 2 1 0 0 [ ] [ ( ) ( ) ( ) ( ) ] ( ) { ( ) ( )} ( ) ( ){ ( , ) ( ) } T T k m k m T T k m T T k XX m E c c E X t t dt X t t dt t E X t X t t dt dt t R t t t dt dt                (19-19) PILLAI * 0 * , 0 ( ) ( ) ( ) ( ) , T k k T k n k n c X t t dt t t dt         
  • 8. 8 and Substituting (19-19) and (19-20) into (19-18), we get Since (19-21) should be true for every we must have or i.e., the desired uncorrelated condition in (19-18) gets translated into the integral equation in (19-22) and it is known as the Karhunen-Loeve or K-L. integral equation.The functions are not arbitrary and they must be obtained by solving the integral equation in (19-22). They are known as the eigenvectors of the autocorrelation * 1 1 2 2 2 1 1 0 0 ( ){ ( , ) ( ) ( )} 0. XX T T k m m m t R t t t dt t dt         1 2 2 2 1 0 ( , ) ( ) ( ) 0, XX T m m m R t t t dt t       ( ), 1 , k t k     * , 1 1 1 0 ( ) ( ) . T m k m m k m t t dt        (19-20) (19-21) 1 2 2 2 1 1 0 ( , ) ( ) ( ), 0 , 1 . XX T m m m R t t t dt t t T m            1 )} ( { k k t  (19-22) PILLAI
  • 9. 9 function of Similarly the set represent the eigenvalues of the autocorrelation function. From (19-18), the eigenvalues represent the variances of the uncorrelated random variables This also follows from Mercer’s theorem which allows the representation where Here and are known as the eigenfunctions and eigenvalues of A direct substitution and simplification of (19-23) into (19-22) shows that Returning back to (19-15), once again the partial sum 1 2 ( , ). XX R t t * 1 2 1 2 1 2 1 ( , ) ( ) ( ), 0 , , XX k k k k R t t t t t t T          (19-23) * , 0 ( ) ( ) . T k m k m t t dt      1 2 ( , ) respectively. XX R t t ) (t k  , k  k ( ) ( ), , 1 . k k k t t k          1 ( ) ( ) ( ), 0 N N k k N k X t c t X t t T          PILLAI (19-24) (19-25) 1 { } k k    k  , k c 1 . k    1 k   
  • 10. 10 in the mean square sense. To see this, consider We have Also Similarly * * * 1 * * 0 1 * 0 1 * 1 [ ( ) ( )] ( ) ( ) [ ( ) ( )] ( ) ( ) ( ( , ) ( ) ) ( ) ( ) ( )= | ( N N k k k N T k k k N T k k k N k k k k k k E X t X t X t c t E X t X t d R t d t t t                              2 1 ) | . N k t   (19-26)    N k k k N t t X t X E 1 2 * | ) ( | )] ( ~ ) ( [   PILLAI 2 2 * * 2 [| ( ) ( ) | ] [| ( ) | ] [ ( ) ( )] [ ( ) ( )] [| ( ) | ]. N N N N E X t X t E X t E X t X t E X t X t E X t      2 [| ( ) | ] ( , ). E X t R t t  (19-27) (19-28) (19-29)
  • 11. 11 and Hence (19-26) simplifies into i.e., where the random variables are uncorrelated and faithfully represent the random process X(t) in provided satisfy the K-L. integral equation. Example 19.1: If X(t) is a w.s.s white noise process, determine the sets in (19-22). Solution: Here 2 * * 2 1 [| ( ) | ] [ ] ( ) ( ) | ( ) | . N N k m k m k k k m k E X t E c c t t t          . as 0 | ) ( | ) , ( ] | ) ( ~ ) ( [| 1 2 2         N k k k N t t t R t X t X E   1 ( ) ( ), 0 , k k k X t c t t T        1 } { k k c 0 , t T   ( ), k t  (19-30) ) ( ) , ( 2 1 2 1 t t q t t RXX    PILLAI (19-31) (19-32) 1 { , } k k k     (19-33) 1 , k   
  • 12. 12 and can be arbitrary so long as they are orthonormal as in (19-17) and Then the power of the process and in that sense white noise processes are unrealizable. However, if the received waveform is given by and n(t) is a w.s.s white noise process, then since any set of orthonormal functions is sufficient for the white noise process representation, they can be chosen solely by considering the other signal s(t). Thus, in (19-35) 2 1 1 [| ( ) | ] (0) k k k P E X t R q              ) ( ) ( ) ( 2 1 2 1 2 1 t t q t t R t t R ss rr       . 1 ,     k q k  ( ) ( ) ( ), 0 r t s t n t t T     ) (t k   PILLAI (19-34) (19-35) (19-36) 1 2 2 1 1 2 2 1 0 0 1 1 ( , ) ( ) ( ) ( ) ( ) ( ) XX T T k k k k k R t t t dt q t t t dt q t t             
  • 13. 13 and if Then it follows that Notice that the eigenvalues of get incremented by q. Example19.2: X(t) is a Wiener process with In that case Eq. (19-22) simplifies to and using (19-39) this simplifies to ) ( 2 1 t t Rss  * 1 2 1 2 1 ( ) ( ) ( ) ( ). rr k k k k R t t q t t          2 1 2 1 2 1 2 1 1 2 ( , ) min( , ) , 0 XX t t t R t t t t t t t             (19-39) 1 1 1 2 2 2 1 2 2 2 0 0 1 2 2 2 1 ( , ) ( ) ( , ) ( ) ( , ) ( ) ( ), T t XX k XX k T XX k k k t R t t t dt R t t t dt R t t t dt t            PILLAI      1 2 * 1 2 1 ) ( ) ( ) ( k k k k ss t t t t R    (19-37) (19-38) 0 1 t T 2 dt   1 1 2 2 2 1 2 2 1 0 ( ) ( ) ( ). t T k k k k t t t dt t t dt t           (19-40)
  • 14. 14 Derivative with respect to t1 gives [see Eqs. (8-5)-(8-6), Lecture 8] or Once again, taking derivative with respect to t1, we obtain or and its solution is given by But from (19-40) 1 1 1 1 1 2 2 1 ( ) ( 1) ( ) ( ) ( ) T k k k k k t t t t t t dt t              ( ) cos sin . k k k k k t A t B t        . ) ( ) ( 1 2 2 1   T t k k k t dt t      , 0 ) 0 (  k  PILLAI (19-41) (19-42) 1 1 ( 1) ( ) ( ) k k k t t       2 1 1 2 1 ( ) ( ) 0, k k k d t t dt       (19-43)
  • 15. 15 (19-45) (19-47) (0) 0, 1 , ( ) cos , k k k k k k A k t B t             PILLAI and from (19-41) This gives and using (19-44) we obtain Also ( ) 0. k T   (19-44)   2 2 2 1 2 ( ) cos 0 2 1 2 , 1 . ( ) k k k k k k T B T T k T k k                       (19-46)
  • 16. 16 PILLAI Further, orthonormalization gives Hence with as in (19-47) and as in (19-16), ( ) sin , 0 . k k k t B t t T       (19-48)   2 1 cos2 2 2 2 2 0 0 0 sin2 sin(2 1) 0 2 2 2 2 4 0 2 ( ) sin 1 1 2 2 2 2/ . k k k k k t T T T k k k T t k k k k k T t dt B t dt B dt T T B B B B T                                                     (19-49)     2 2 , 1 2 ( ) sin sin k k T T t T t t k        k  k c
  • 17. 17 is the desired series representation. Example 19.3: Given find the orthonormal functions for the series representation of the underlying stochastic process X(t) in 0 < t < T. Solution: We need to solve the equation Notice that (19-51) can be rewritten as, Differentiating (19-52) once with respect to t1, we obtain PILLAI     1 ) ( ) ( k k k t c t X  | | ( ) , 0, XX R e        (19-50) 0 1 t T 2 dt   2 dt   1 2 | | 2 2 1 0 ( ) ( ). T t t n n n e t dt t         (19-51) 0 0 1 1 2 2 1 1 t ( ) ( ) 2 2 2 2 1 0 t ( ) ( ) ( ) T t t t t n n n n e t dt e t dt t                 (19-52)
  • 18. 18 Differentiating (19-53) again with respect to t1, we get or 1 1 2 2 1 1 1 1 2 2 1 1 ( ) ( ) 1 2 2 1 2 2 0 1 1 ( ) ( ) 1 2 2 2 2 0 1 ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) t T t t t t n n n n t n n t T t t t t n n n n t t e t dt t e t dt d t dt d t e t dt e t dt dt                                       (19-53) 1 1 2 2 1 1 ( ) 1 2 2 0 2 ( ) 1 1 2 2 2 1 ( ) ( ) ( ) ( ) ( ) ( ) t t t n n T t t n n n n t t e t dt d t t e t dt dt                        1 1 2 2 1 1 1 ( ) ( ) 1 2 2 2 2 0 ( ) {use (19-52)} 2 1 2 1 2 ( ) ( ) ( ) ( ) n n t T t t t t n n n t t n n t e t dt e t dt d t dt                          PILLAI
  • 19. 19 or or Eq.(19-54) represents a second order differential equation. The solution for depends on the value of the constant on the right side. We shall show that solutions exist in this case only if or In that case Let and (19-54) simplifies to ( ) n t  2 1 1 2 1 ( ) ( 2) ( ) n n n n d t t dt        2 1 1 2 1 ( ) ( 2) ( ). n n n n d t t dt              (19-54) (19-55) PILLAI ( 2)/ n n     2, n   ( 2)/ 0. n n      2 0 . n     (19-56) 2 2 1 1 2 1 ( ) ( ). n n n d t t dt      (19-57) 2 (2 ) 0, n n n        
  • 20. 20 PILLAI General solution of (19-57) is given by From (19-52) and Similarly from (19-53) and Using (19-58) in (19-61) gives 1 1 1 ( ) cos sin . n n n n n t A t B t      (19-58) 2 2 2 0 1 (0) ( ) T t n n n e t dt        (19-59) 2 2 2 0 1 ( ) ( ) . T t n n n T e T t dt         (19-60) 2 1 1 2 2 0 1 0 ( ) (0) ( ) (0) T t n n n n n t d t e t dt dt              (19-61) 2 2 2 0 ( ) ( ) ( ). T t n n n n T e T t dt T              (19-62)
  • 21. 21 PILLAI or and using (19-58) in (19-62), we have or Thus are obtained as the solution of the transcendental equation , n n n A B    n n n B A    (19-63) (19-64) sin cos ( cos sin ), ( )cos ( )sin n n n n n n n n n n n n n n n n n n A T B T A T B T A B T A B T                       2 2 2 / 2 / 2( / ) tan . ( ) 1 1 1 n n n n n n n n n n n n n n n n n A A B B A A B A B T A B                       2 2( / ) tan , ( / ) 1 n n n T        s n 
  • 22. 22 which simplifies to In terms of from (19-56) we get Thus the eigenvalues are obtained as the solution of the transcendental equation (19-65). (see Fig 19.1). For each such the corresponding eigenvector is given by (19-58). Thus since from (19-65) and cn is a suitable normalization constant. 2 (or ), n n   2 ( ) cos sin sin( ) sin ( ), 0 n n n n n n n n n n T t A t B t c t c t t T               (19-66) 2 2 2 0. n n        1 1 tan tan /2, n n n n n A T B                        (19-68) (19-67) s n  PILLAI tan( / 2) . n nT      (19-65)
  • 23. 23 PILLAI Fig 19.1 Solution for Eq.(19-65). 2  1  T   0 tan( / 2) T  /     
  • 24. 24 PILLAI Karhunen – Loeve Expansion for Rational Spectra [The following exposition is based on Youla’s classic paper “The solution of a Homogeneous Wiener-Hopf Integral Equation occurring in the expansion of Second-order Stationary Random Functions,” IRE Trans. on Information Theory, vol. 9, 1957, pp 187-193. Youla is tough. Here is a friendlier version. Even this may be skipped on a first reading. (Youla can be made only so much friendly.)] Let X(t) represent a w.s.s zero mean real stochastic process with autocorrelation function so that its power spectrum is nonnegative and an even function. If is rational, then the process X(t) is said to be rational as well. rational and even implies ( ) ( ) XX XX R R     ( ) XX S  ( ) XX S  0 ( ) ( ) 2 ( )cos XX XX XX j t S R e dt R d               (19-69) (19-70) 2 2 ( ) ( ) 0. ( ) XX N S D     
  • 25. 25 PILLAI The total power of the process is given by and for P to be finite, we must have (i) The degree of the denominator polynomial must exceed the degree of the numerator polynomial by at least two, and (ii) must not have any zeros on the real-frequency axis. The s-plane extension of is given by Thus and the Laplace inverse transform of is given by 2 2 ( ) ( ) 1 1 2 2 ( ) XX N D P S d d                (19-71) ( ) 2 D n   2 ( ) D  2 ( ) N  ( ) 2 N m   2 ( ) D  ( ) s j  ( ) XX S  (19-72) 2 2 ( ) k s    2 2 2 ( ) ( ) i k k k D s s      (19-73) ( ) s j     2 2 2 ( ) ( ) | ( ) . ( ) XX s j N s S S s D s        
  • 26. 26 PILLAI Let represent the roots of D(– s2) . Then Let D+(s) and D–(s) represent the left half plane (LHP) and the right half plane (RHP) products of these roots respectively. Thus where This gives Notice that has poles only on the LHP and its inverse (for all t > 0) converges only if the strip of convergence is to the right 1 2 , , , n       1 2 0 Re Re Re n        (19-74) (19-75) | | 2 2 1 1 1 ( 1) ( 2)! | | ( 1)! ( 1)!( )! ( ) (2 ) k k j k k k j j k j e k j k j s                    2 ( ) ( ) ( ), D s D s D s     (19-76) 1 ( ) ( ) C s D s  * 0 ( ) ( )( ) ( ). n k k k k k k D s s s d s D s              (19-77) 2 2 1 2 2 ( ) ( ) ( ) ( ) ( ) ( ) ( ) C s C s N s S s D s D s D s        (19-78)
  • 27. 27 PILLAI of all its poles. Similarly C2(s) /D–(s) has poles only on the RHP and its inverse will converge only if the strip is to the left of all those poles. In that case, the inverse exists for t < 0. In the case of from (19-78) its transform N(s2) /D(–s2) is defined only for (see Fig 19.2). In particular, for from the above discussion it follows that is given by the inverse transform of C1(s) /D+(s). We need the solution to the integral equation that is valid only for 0 < t < T. (Notice that in (19-79) is the reciprocal of the eigenvalues in (19-22)). On the other hand, the right side (19-79) can be defined for every t. Thus, let ( ), XX R  1 1 Re Re Re s      0,    0 ( ) ( ) ( ) , 0 XX T t R t d t T            (19-79) (19-80) ( ) XX R  Fig 19.2   Re n   1 Re   Re n  1 Re  strip of convergence for ( ) XX R  s j    0 ( ) ( ) ( ) , XX T g t R t d t             
  • 28. 28 PILLAI and to confirm with the same limits, define This gives and let Clearly and for t > T since RXX(t) is a sum of exponentials Hence it follows that for t > T, the function f (t) must be a sum of exponentials Similarly for t < 0 , for 0. k t k k a e t     . kt k k a e    ( ) 0 ( ) . 0 otherwise t t T t         (19-81) (19-82) + ( ) ( ) ( ) XX g t R t d          + ( ) ( ) ( ) ( ) ( ) ( ) . XX f t t g t t R t d                 (19-83) ( ) 0, 0 f t t T    (19-84)     ( ) 0 + ( ) { ( )} ( ) 0, k XX D d d dt dt D f t D R t d                   (19-85)
  • 29. 29 PILLAI and hence f (t) must be a sum of exponentials Thus the overall Laplace transform of f (t) has the form where P(s) and Q(s) are polynomials of degree n – 1 at most. Also from (19-83), the bilateral Laplace transform of f (t) is given by Equating (19-86) and (19-87) and simplifying, Youla obtains the key identity Youla argues as follows: The function is an entire function of s, and hence it is free of poles on the entire     ( ) 0 + ( ) { ( )} ( ) 0, k XX D d d dt dt D f t D R t d                  , for 0. k t k k b e t    contributes to 0 contributions in < 0 contributions in ( ) ( ) ( ) ( ) ( ) sT t t t T P s Q s F s e D s D s        (19-86) 2 2 1 1 ( ) ( ) ( ) ( ) 1 , Re Re Re N s D s F s s s                (19-87) (19-88) 2 2 ( ) ( ) ( ) ( ) ( ) . ( ) ( ) sT P s D s e Q s D s s D s N s           0 ( ) ( ) T st s t e dt     
  • 30. 30 PILLAI finite s-plane However, the denominator on the right side of (19-88) is a polynomial and its roots contribute to poles of Hence all such poles must be cancelled by the numerator. As a result the numerator of in (19-88) must possess exactly the same set of zeros as its denominator to the respective order at least. Let be the (distinct) zeros of the denominator polynomial Here we assume that is an eigenvalue for which all are distinct. We have These also represent the zeros of the numerator polynomial Hence and which simplifies into From (19-90) and (19-92) we get ). Re (     s ) (s  1 2 ( ), ( ), , ( ) n          2 2 ( ) ( ). D s N s      ' k s  ' k s  ( ) ( ) ( ) ( ). sT P s D s e Q s D s     1 2 0 Re ( ) Re ( ) Re ( ) . n             (19-89) ( ) ( ) ( ) ( ) kT k k k k D P e D Q          (19-90) (19-91) ( ) ( ) ( ) ( ) kT k k k k D P e D Q             ( ) ( ) ( ) ( ). kT k k k k D P e D Q           (19-92) ( ). s 
  • 31. 31 PILLAI i.e., the polynomial which is at most of degree n – 1 in s2 vanishes at (for n distinct values of s2). Hence or Using the linear relationship among the coefficients of P(s) and Q(s) in (19-90)-(19-91) it follows that are the only solutions that are consistent with each of those equations, and together we obtain ( ) ( ) ( ) ( ), 1, 2, , k k k k P P Q Q k n         (19-93) ( ) ( ) ( ) ( ) ( ) L s P s P s Q s Q s     (19-94) 2 2 2 1 2 , , , n    2 ( ) 0 L s  (19-95) ( ) ( ) ( ) ( ). P s P s Q s Q s    (19-96) ( ) ( ) or ( ) ( ) P s Q s P s Q s      (19-97)
  • 32. 32 PILLAI as the only solution satisfying both (19-90) and (19-91). Let In that case (19-90)-(19-91) simplify to (use (19-98)) where For a nontrivial solution to exist for in (19-100), we must have ( ) ( ) P s Q s    (19-98) 1 0 ( ) . n i i i P s p s     (19-99) 0 1 1 , , , n p p p  1 0 ( ) ( ) ( ) ( ) {1 ( 1) } 0, 1,2, , kT k k k k n i i k k i i P D e D P a p k n                  (19-100) ( ) ( ) . ( ) ( ) k k T T k k k k k D D a e e D D                (19-101)
  • 33. 33 PILLAI The two determinant conditions in (19-102) must be solved together to obtain the eigenvalues that are implicitly contained in the and (Easily said than done!). To further simplify (19-102), one may express ak in (19-101) as so that ' i s  ' i a s ' i s  2 , 1, 2, , k k a e k n     1 1 1 1 1 1 1 1 1 2 2 2 2 2 1,2 1 1 (1 ) (1 ) (1 ( 1) ) (1 ) (1 ) (1 ( 1) ) 0. (1 ) (1 ) (1 ( 1) ) n n n n n n n n n n n a a a a a a a a a                      (19-102) (19-104) (19-103) / 2 / 2 / 2 / 2 1 ( ) ( ) tan 1 ( ) ( ) ( ) ( ) ( ) ( ) k k k k k k k k k k T k k k k T k k k T T k k T T k k a D e D e e a e e D e D e D e D e D e D                                                  h
  • 34. 34 PILLAI Let and substituting these known coefficients into (19-104) and simplifying we get and in terms of in (19-102) simplifies to if n is even (if n is odd the last column in (19-107) is simply Similarly in (19-102) can be obtained by replacing with in (19-107). 0 1 ( ) n n D s d d s d s      (19-105) 2 tan , k h   2 3 0 2 1 3 2 3 0 2 1 3 ( )tan ( / 2) ( ) tan ( ) ( )tan ( / 2) k k k k k k k k k d d T d d d d d d T                     (19-106) 1 1 1 1 2 [ , , , ] ). n n n n T       1  cot k  h tan k  h 2 3 1 1 1 1 1 1 1 1 2 3 1 2 2 2 2 2 2 2 1 tan tan tan 1 tan tan tan 1 tan n n n                  2 3 1 0 tan tan n n n n n n n         (19-107) h h h h h h h h h h h h
  • 35. 35 PILLAI To summarize determine the roots with that satisfy in terms of and for every such determine using (19-106). Finally using these and in (19-107) and its companion equation , the eigenvalues are determined. Once are obtained, can be solved using (19-100), and using that can be obtained from (19-88). Thus and Since is an entire function in (19-110), the inverse Laplace transform in (19-109) can be performed through any strip of convergence in the s-plane, and in particular if we use the strip ,  , k  k  ' k s  2 2 ( ) ( ) 0, 1, 2, , k k D N k n         (19-108) k s  tanh k s  k s  k s  k p s ( ) i s  ( ) i s  2 2 ( ) ( , ) ( ) ( , ) ( ) ( ) ( ) sT i i i i D s P s e D s Q s s D s N s             (19-109) 1 ( ) { ( )}. i i t L s     (19-110) 1  Re( ) 0 i  
  • 36. 36 PILLAI then the two inverses obtained from (19-109) will be causal. As a result will be nonzero only for t > T and using this in (19-109)-(19-110) we conclude that for 0 < t < T has contributions only from the first term in (19-111). Together with (19-81), finally we obtain the desired eigenfunctions to be that are orthogonal by design. Notice that in general (19-112) corresponds to a sum of modulated exponentials. Re Re( ) (to the right of all Re( )), n i s    1 1 2 2 2 2 ( ) ( ) ( ) ( ) , ( ) ( ) ( ) ( ) D s P s D s Q s L L D s N s D s N s                         (19-111) (19-112)   2 2 1 ( ) ( ) ( ) ( ) sT D s Q s e D s N s L        ( ) i t  1 2 2 ( ) ( , ) ( ) , 0 , ( ) ( ) Re Re 0, 1,2, , k k k n D s P s t L t T D s N s s k n                     
  • 37. 37 PILLAI Next, we shall illustrate this procedure through some examples. First, we shall re-do Example 19.3 using the method described above. Example 19.4: Given we have This gives and P(s), Q(s) are constants here. Moreover since n = 1, (19-102) reduces to and from (19-101), satisfies or is the solution of the s-plane equation But |esT| >1 on the RHP, whereas on the RHP. Similarly |esT| <1 on the LHP, whereas on the LHP. | | ( ) , XX R e      ( ) , ( ) D s s D s s         1 1 1 0, or 1 a a     1  1  sT s e s      2 2 2 2 2 ( ) ( ) . ( ) XX N S D          (19-113) (19-114) 1 1 1 1 1 ( ) ( ) T D e D              1 s s      1 s s     
  • 38. 38 PILLAI Thus in (19-114) the solution s must be purely imaginary, and hence in (19-113) is purely imaginary. Thus with in (19-114) we get or which agrees with the transcendental equation (19-65). Further from (19-108), the satisfy or Notice that the in (19-66) is the inverse of (19-116) because as noted earlier in (19-79) is the inverse of that in (19-22). 1  1 s j   n  2 2 0. 2 n n        (19-116) (19-115) 2 2 2 2 ( ) ( ) 2 0 n n n n s j D s N s              1 1 tan( / 2) T      1 1 1 j T j e j         s 
  • 39. 39 PILLAI Finally from (19-112) which agrees with the solution obtained in (19-67). We conclude this section with a less trivial example. Example 19.5 In this case This gives With n = 2, (19-107) and its companion determinant reduce to 1 2 2 ( ) cos sin , 0 n n n n n n s t L A t B t t T s                    (19-117) | | | | ( ) . XX R e e          (19-118) 2 2 2 2 2 2 2 2 2 2 2 2( )( ) ( ) . ( )( ) XX S                         (19-119) 2 ( ) ( )( ) ( ) . D s s s s s              2 2 1 1 2 2 1 1 tan tan cot cot           h h h h
  • 40. 40 PILLAI or From (19-106) Finally can be parametrically expressed in terms of using (19-108) and it simplifies to This gives and (19-120) (19-121) 2 2 1 2 and   2 2 1 ( ) ( ) 4 ( ) 2 b b c        1 2 tan tan .     h h 2 2 ( )tan ( / 2) ( ) tan , 1,2 ( ) ( ) tan ( / 2) i i i i i i i T i T                      h h h  2 2 4 2 2 2 2 2 4 2 ( ) ( ) ( 2 ( )) 2 ( ) 0. D s N s s s s bs c                            
  • 41. 41 PILLAI and and substituting these into (19-120)-(19-121) the corresponding transcendental equation for can be obtained. Similarly the eigenfunctions can be obtained from (19-112). 2 2 2 2 2 1 ( ) ( ) 4 ( ) ( ) 4 ( ) 2 b b c b c              i s 