probability Chapter Four-Random_Processes.ppt

Chapter 4: Random Processes
Prepared by Getahun Sh.(MSc)
Ambo University
Hachalu Hundessa Campus
School of Electrical Engineering and Computing
Department of Electrical & Computer Engineering
Probability and Random Process (ECEg-2114)

Outline
 Introduction
 Definition of a Random Process
 Characterization of Random Processes
 Mean, Correlation, and Covariance Functions
 Classification of Random Processes
 Power Spectral Densities of Random Processes
 Response of Linear Systems to Random Inputs
2
11/04/24

Introduction
 A random process is the mathematical model of an empirical
process whose development is governed by probability laws.
 Random processes provides useful models for the studies of
diverse fields such as statistical physics, communication and
control, time series analysis, population growth, and etc.
3
11/04/24

Definition of a Random Process
 A random process is a family of random variables {X(t), tϵT} defined on a given
probability space, indexed by the parameter t, where t varies over an index set T.
 In a random process {X(t), tϵT}, the index set T is called the parameter set of the
random process.
 The values assumed by X(t) are called states, and the set of all possible values
forms the state space E of the random process.
 If the index set T of a random process is discrete, then the process is called a
discrete-time random process.
4
11/04/24

Definition of a Random Process Cont’d…..
 A discrete-time random process is also called a random sequence and
is denoted by {Xn , n = 1, 2, 3, . . .).
 If T is continuous, then we have a continuous-time random process.
 In fact, a random process {X(t), tϵT} is a function of two arguments
{X(t, ω), tϵT, ω Ω
ϵ }.
 For a fixed time t=tk, X(tk, ω) = Xk(ω) is a random variable denoted by
X(tk), as ω varies over the sample space Ω.
5
11/04/24

Definition of a Random Process Cont’d…..
 On the other hand, for a fixed sample point ωi Ω
ϵ , X(t, ωi) =
Xi(t) is a single function of time t, called a sample function or a
realization of the process.
 The totality of all sample functions is called an ensemble.
 Of course if both ω and t are fixed, X(tk , ωi) is simply a real
number.
6
11/04/24

Characterization of Random Processes
 If X(t) is a random process, then for fixed t=t1, X1=X(t1) represents a
random variable.
 Its distribution function is given by:
 Notice that FX(x, t) depends on t, since for a different t, we obtain a
different random variable.
 The first-order probability density function of the process X(t) is defined
as:
7
}
)
(
{
)
,
( 1
1
1 x
t
X
P
t
x
FX


1
1
1
1
1
)
,
(
)
,
(
dx
t
x
dF
t
x
f X
X

11/04/24

Characterization of Random Processes Cont’d…..
 For t = t1 and t = t2, X(t) represents two different random variables X1 = X(t1) and X2 = X(t2)
respectively.
 Their joint distribution is given by:
 The second-order probability density function of the random process X(t) is:
 Similarly represents the nth
order density function of the process
X(t).
8
}
)
(
,
)
(
{
)
,
,
,
( 2
2
1
1
2
1
2
1 x
t
X
x
t
X
P
t
t
x
x
FX



2
1 2 1 2
1 2 1 2
1 2
( , , , )
( , , , ) X
X
F x x t t
f x x t t
x x


 
)
,
,
,
,
,
( 2
1
2
1 n
n t
t
t
x
x
x
fX


11/04/24

Mean, Correlation, and Covariance Functions
 The mean of X(t) is defined by:
where X(t) is a random variable for a fixed value of t.
 In general, μX(t) is a function of time, and it is often called the
ensemble average of X(t).
9
 
)
(
)
( t
X
E
t
X 

11/04/24

Mean, Correlation, and Covariance Functions ….
 A measure of dependence among the random variables of X(t) is
provided by its autocorrelation function, defined by:
 Note that:
 The autocovariance function of X(t) is defined by:
10
 
)
(
)
(
)
,
( 2
1
2
1 t
X
t
X
E
t
t
RXX 
 
)
(
)
,
(
and
)
,
(
)
,
( 2
1
2
2
1 t
X
E
t
t
R
t
t
R
t
t
R XX
XX
XX 

    
 
)
(
)
(
)
,
(
)
(
)
(
)
(
)
(
)
(
,
)
(
)
,
(
2
1
2
1
2
2
1
1
2
1
2
1
t
t
t
t
R
t
t
X
t
t
X
E
t
X
t
X
Cov
t
t
C
X
X
XX
X
X
XX










11/04/24

Mean, Correlation, and Covariance Functions …...
 It is clear that if the mean of X(t) is zero, then:
 Note that the variance of X(t) is given by:
 If X(t) is a complex random process, then its autocorrelation function RXX(t1,
t2) and autocovariance function CXX(t1, t2) are defined, respectively, by:
11
   
 
2
2
)
(
)
(
)
(
)
( t
t
X
E
t
X
Var
t X
X 
 


)
,
(
)
,
( 2
1
2
1 t
t
R
t
t
C XX
XX 
 
  
 
*
2
2
1
1
2
1
2
*
1
2
1
)
(
)
(
)
(
)
(
)
,
(
and
)
(
)
(
)
,
(
t
t
X
t
t
X
E
t
t
C
t
X
t
X
E
t
t
R
X
X
XX
XX

 



11/04/24

Classification of Random Processes
I. Stationary Processes
A random process {X(t), tϵT} is said to be stationary or strict-sense stationary
(SSS) if, for all n and for every set of time instants (ti T, i = 1,2, . . . , n
ϵ ),
Hence, the distribution of a stationary process will be unaffected by a shift in
the time origin, and X(t) and X(t+τ) will have the same distributions for any τ.
Nonstationary processes are characterized by distributions depending on the
points t1, t2, . . . , tn.
12
)
......,
,
,
........,
,
(
)
.....,
,
,
,........,
( 1
1
1
1 
 

 n
n
X
n
n
X t
t
x
x
F
t
t
x
x
F
11/04/24

Classification of Random Processes Cont’d……
II. Wide-Sense Stationary Processes
A random process X(t) is wide-sense stationary (WSS) if:
Note that a strict-sense stationary process is also a WSS process, but,
in general, the converse is not true.
13
 
   
1
2
2
1
2
1 )
(
)
(
)
,
(
.
2
)
constant
(
)
(
.
1
t
t
R
t
X
t
X
E
t
t
R
t
X
E
XX
XX
X




11/04/24

III. Ergodic process
• A random process is said to be ergodic if the time
averages of sample functions of the process are equal to
the corresponding statistical or ensemble averages.
• Consider a random process {X(t), - ∞ < t < ∞) with a
typical sample function x(t). The time average of x(t) is
defined as:
• The time autocorrelation function Rx(τ) of x(t) is defined
as
14
11/04/24

Chapter 6
Power Spectral Estimation and
Stochastic Filter Design
16
11/04/24

Power Spectral Densities of Random Processes
 The autocorrelation function of a continuous-time random
process X(t) is defined as:
 Properties of RXX(τ):
17
 
)
(
)
(
)
( 
 
 t
X
t
X
E
RXX
  0
)
(
)
0
(
.
3
)
0
(
)
(
.
2
)
(
)
(
.
1
2





t
X
E
R
R
R
R
R
XX
XX
XX
XX
XX



11/04/24

Power Spectral Densities of Random Processes……
 In case of a discrete-time random process X(n), the
autocorrelation function of X(n) is defined by:
 Properties of RX(k):
18
 
)
(
)
(
)
( k
n
X
n
X
E
k
RXX 

  0
)
(
)
0
(
.
3
)
0
(
)
(
.
2
)
(
)
(
.
1
2





n
X
E
R
R
k
R
k
R
k
R
XX
XX
XX
XX
XX
11/04/24

 Two processes X(t) and Y(t) are called (mutually) orthogonal if:
 Similarly, the cross-correlation function of two discrete-time
jointly WSS random processes X(n) and Y(n) is defined by:
19

 ,
0
)
( all
for
RXY 
 
)
(
)
(
)
( k
n
Y
n
X
E
k
RXY 

11/04/24

 The power spectral density (or power spectrum) SXX(ω) of a continuous-
time random process X(t) is defined as the Fourier transform of RXX(τ), i.e. ,
 Thus, taking the inverse Fourier transform of SX(ω), we obtain:
 The above equations are known as the Wiener-Khinchin relations.
20
  

 
d
e
R
S j
XX
XX





 )
(
  


 
d
e
S
R j
XX
XX 



 )
(
2
1
11/04/24

 Properties of SXX(ω):
 Similarly, the power spectral density SXX(Ω) of a discrete-time
random process X(n) is defined as the Fourier transform of RXX(k):
21
  















d
S
R
t
X
E
S
S
S
S
XX
XX
XX
XX
XX
XX
)
(
2
1
)
0
(
)
(
.
3
)
(
)
(
.
2
0
)
(
and
real
is
)
(
.
1
2
  







k
k
j
XX
XX e
k
R
S )
(
11/04/24

 Thus, taking the inverse Fourier transform of SXX(Ω), we obtain:
 Properties of SXX(Ω):
22









d
e
S
k
R k
j
XX
XX )
(
2
1
)
(
  




















d
S
R
n
X
E
S
S
S
S
S
S
XX
XX
XX
XX
XX
XX
XX
XX
)
(
2
1
)
0
(
)
(
.
3
)
(
)
(
.
3
0
)
(
and
real
is
)
(
.
2
)
(
)
2
(
.
1
2
11/04/24

 The cross power spectral density (or cross power spectrum) SXY(ω) of
two continuous-time random processes X(t) and Y(t) is defined as the
Fourier transform of RXY(τ):
 Thus, taking the inverse Fourier transform of SXY(ω), we get:
23
  

 
d
e
R
S j
XY
XY





 )
(
  


 
d
e
S
R j
XY
XY 



 )
(
2
1
11/04/24

 Properties of SXY(ω):
 Unlike SXX(ω), which is a real-valued function of ω, SXY(ω), in general, is a complex-
valued function.
 Similarly, the cross power spectral density SXY(Ω) of two discrete-time random processes
X(n) and Y(n) is defined:
24
)
(
)
(
.
2
)
(
)
(
.
1
*




XY
XY
YX
XY
S
S
S
S












k
k
j
XY
XY e
k
R
S )
(
)
(
11/04/24

 Taking the inverse Fourier transform of SXY(Ω), we get:
 Properties of SXY(ω):
 Unlike SXX(Ω), which is a real-valued function of Ω, SXY(Ω), in general, is a complex-valued
function.
25









d
e
S
k
R k
j
XY
XY )
(
2
1
)
(
)
(
)
(
.
3
)
(
)
(
.
2
)
(
)
2
(
.
1
*












XY
XY
YX
XY
XY
XY
S
S
S
S
S
S 
11/04/24

Example on Random Processes
Example:
Consider a random process X(t) defined by
).
(
of
density
spectral
power
the
Find
.
not.
or
process
random
WSS
is
)
(
whether
Determine
.
).
,
(
function
ance
autocovari
the
Find
.
).
,
(
function
ation
autocorrel
the
Find
.
).
(
mean
the
Find
.
)
2
,
0
(
interval
over the
variable
random
uniform
a
is
and
constants
are
and
where
)
cos(
)
(
2
1
2
1
X
0
0
t
X
e
t
X
d
t
t
C
c
t
t
R
b
t
a
A
t
A
t
X
XX
XX





 

26
11/04/24

Example on Random Processes Cont’d……
Solution:
     
   
   
 
 
  0
)
(
)
(
0
sin
2
1
sin
,
0
cos
2
1
cos
sin
t)
sin(
cos
t)
cos(
t)sin
sin(
-
t)cos
cos(
)
(
)
(
t)sin
sin(
-
t)cos
cos(
)
cos(
,
)
cos(
)
cos(
)
(
)
(
.
X
2
0
2
0
0
0
0
0
X
0
0
0
0
0
X





















t
X
E
t
d
E
Similarly
d
E
E
A
E
A
AE
t
X
E
t
t
But
t
AE
t
A
E
t
X
E
t
a































27
11/04/24

Solution:
 
 
 
 
 
 
)
(
cos
2
)
,
(
0
)
2
)
(
cos(
and
)
(
cos
)
(
cos
,
)
2
)
(
cos(
)
(
cos
2
)
cos(
)
cos(
)
cos(
)
cos(
)
(
)
(
)
,
(
.
1
2
0
2
2
1
2
1
0
1
2
0
1
2
0
2
1
0
1
2
0
2
2
0
1
0
2
2
0
1
0
2
1
2
1
t
t
A
t
t
R
t
t
E
t
t
t
t
E
But
t
t
t
t
E
A
t
t
E
A
t
A
t
A
E
t
X
t
X
E
t
t
R
b
XX
XX





































28
11/04/24

Solution:
process.
random
a
is
)
(
only,
difference
on time
depends
function
ation
autocorrel
the
and
constant
is
mean
the
Since
.
)
(
cos
2
)
,
(
0
)
(
cos
2
)
(
)
(
)
,
(
)
,
(
.
1
2
0
2
2
1
1
2
0
2
2
1
2
1
2
1
WSS
t
X
d
t
t
A
t
t
C
t
t
A
t
t
t
t
R
t
t
C
c
XX
X
X
XX
XX












29
11/04/24

Solution:
 
)
(
2
)
(
2
)
(
)
(
)
(
)
cos(
:
have
we
,
pair table
ansform
Fourier tr
from
But
)
(
)
(
:
by
given
is
)
(
of
density
spectral
power
The
)
cos(
2
)
(
:
as
tten
simply wri
be
can
function
lation
autocorre
the
process,
random
a
is
)
(
Since
.
0
2
0
2
0
0
0
0
2







































A
A
S
t
FT
d
e
R
S
t
X
A
R
WSS
t
X
e
XX
j
XX
XX
XX
30
11/04/24

Response of Linear Systems to Random Inputs
 If a WSS random process X(t) with autocorrelation function RXX(τ) is applied to a linear system
with impulse response h(t), then the cross correlation function RXY(τ) and the output
autocorrelation function RYY(τ) are given as follows.
31
h(t)
X(t
)
Y(t)
)
(
*
)
(
*
)
(
)
(
*
)
(
)
(
,
)
(
*
)
(
)
(
*
*









h
h
R
h
R
R
And
h
R
R
XX
XY
YY
XX
XY





11/04/24

Response of Linear Systems to Random Inputs…..
 Using properties of Fourier transform, we get:
 Then using the above property, the cross and output power spectral densities can be evaluated
as:
32
)
(
)
(
)
(
*
)
(
)
(
)
(
and
)
(
)
(




G
F
t
g
t
f
G
t
g
F
t
f
FT
FT
FT







 
   
2
*
*
)
(
)
(
)
(
)
(
)
(
*
)
(
)
(
)
(
,
)
(
)
(
)
(
*
)
(
)
(













H
S
H
S
h
R
FT
R
FT
S
And
H
S
h
R
FT
S
XX
XY
XY
YY
YY
XX
XX
XY







11/04/24

Example on Response of Linear Systems
Example:
Consider a WSS random process X(t) with autocorrelation
function given by:
Let the random process X(t) be applied to the input of an LTI
system with impulse response given by:
Find the autocorrelation function of the output Y(t) of the
system.
33
constant
positive
real
a
is
where
,
)
( a
e
R
a
XX

 

constant
positive
real
a
is
where
,
)
(
)
( b
t
u
e
t
h bt


11/04/24

Example on Response of Linear Systems Cont’d…
Solution:
 
 






















2
2
2
2
2
2
2
2
1
)
(
)
(
)
(
:
by
given
is
)
(
of
density
spectral
power
the
Then,
2
)
(
)
(
:
is
)
(
of
density
spectral
power
The
1
)
(
)
(
:
is
system
the
of
)
(
response
frequency
The
a
a
b
H
S
S
t
Y
a
a
R
FT
S
t
X
b
j
t
h
FT
H
H
XX
YY
XX
XX











34
11/04/24

Example on Response of Linear Systems Cont’d…
Solution:
   
 
 






a
b
YY
YY
be
ae
b
b
a
R
b
a
b
b
a
b
b
b
b
b
a
a
S
























2
2
2
2
2
2
2
2
2
2
1
)
(
:
obtain
we
equation,
above
the
of
sides
both
of
ansform
Fourier tr
inverse
the
Taking
2
2
)
(
35
11/04/24

Exercise on Random Processes
1. Consider a random process X(t) defined by
).
(
of
density
spectral
power
the
Find
.
not.
or
process
random
WSS
is
)
(
whether
Determine
.
).
,
(
function
ance
autocovari
the
Find
.
).
,
(
function
ation
autocorrel
the
Find
.
).
(
mean
the
Find
.
2)
,
0
(
interval
over the
variable
random
uniform
a
is
and
constants
are
and
where
)
cos(
)
(
2
1
2
1
X
0
0
t
X
e
t
X
d
t
t
C
c
t
t
R
b
t
a
A
t
A
t
X
XX
XX




 

36
11/04/24

Exercise on Random Processes
2. Consider a random process X(t) defined by:
).
(
of
density
spectral
power
the
Find
.
not.
or
process
random
WSS
is
)
(
whether
Determine
.
).
,
(
function
ance
autocovari
the
Find
.
).
,
(
function
ation
autocorrel
the
Find
.
).
(
mean
the
Find
.
constant.
a
is
and
ly
respective
2
,
2
and
1]
[0,
intervals
over the
d
distribute
uniformly
are
which
variables
random
t
independen
are
and
where
)
sin(
)
(
2
1
2
1
X
0
0
t
X
e
t
X
d
t
t
C
c
t
t
R
b
t
a
A
t
A
t
X
XX
XX
















37
11/04/24

Exercise on Random Processes Cont’d……
3. Two random processes X(t) and Y(t) are given by:
38
)
(
)
(-
t
Verify tha
.
).
(
and
)
(
of
function
n
correlatio
cross
the
Find
.
).
2
(0,
interval
over the
variable
random
uniform
a
is
and
constants
are
and
where
)
sin(
)
(
and
)
cos(
)
(
0
0
0









XY
XY R
R
b
t
Y
t
X
a
A
t
A
t
Y
t
A
t
X





11/04/24

Exercise on Random Processes Cont’d……
4. Consider a discrete-time WSS random process X(n) with
autocorrelation function given by:
Find the power spectral density of X(n).
39
|
|
2
)
( k
XX e
k
R 

11/04/24

probability Chapter Four-Random_Processes.ppt

More Related Content

Similar to probability Chapter Four-Random_Processes.ppt (20)

Recently uploaded (20)

probability Chapter Four-Random_Processes.ppt