Section2 stochastic
0 likes17 views
1. The document discusses concepts related to expectation and variance of random variables including expected value, variance, moments, and examples of calculating these for different probability distributions like uniform, normal, exponential, and Rayleigh. 2. Problems at the end provide examples of computing expected value, variance, and cumulative distribution function for random variables following different distributions. Solutions show the calculations and formulas used. 3. Key formulas introduced include definitions of expected value and variance, the relationship between them, and formulas for calculating moments, expected value, and variance for specific distributions. Examples demonstrate applying the concepts and formulas to problems.
![Stochastic Section # 2
Expectation and Variance
Eslam Adel
Notes are inspired by sections made by TA Eng.Tamim Rushdi
March 15, 2018
1 Expected Value
Expected Value of a random variable in probability theory is the average value. For random variable X expected
value of X is
E[X] =
x
Xf(x)dx (1)
where f(x) is the probability density function of x
Expected value is also called mean, average, and first moment.
Notes
1. E[X] is a linear operator
E[X + Y ] = E[X] + E[Y ]
E[aX] = aE[X]
Example:
E[X2
+ 3X + 2] = E[X2
] + 3E[X] + 2
2. For independent random variables X and Y E[XY ] = E[X]E[Y ].
3. Random variables X and Y are orthogonal when E[XY ] = 0
2 Variance
Variance is the average square deviation of X about the mean.
V ar[X] = E[(X − m)2
] =
x
(X − m)2
f(x)dx (2)
Another formula
V ar[X] = E[X2
] − (E[X])2
= E[X2
] − m2
(3)
and V ar[X] = σ2
where σ is the standard deviation and m is the mean (expected value).
E[X2
] is called the second moment.
nth moment for n > 1 of random variable X is defined as E[Xn
] where :
E[Xn
] =
x
Xn
f(x)dx (4)
1](https://image.slidesharecdn.com/section2-200826012314/85/Section2-stochastic-1-320.jpg)
![3 Problems
3.1 Problem 1
Suppose that a uniformly distributed random variable X can have each of the seven values
−1, 0, 1, 2, 3, 4, 5
1. Determine the probability mass function of the random variable: Y = X2
− 2X
2. What is the expected value of Y ?
3. What is the standard deviation of Y ?
Solution
1. Y = X2
− 2X
X -1 0 1 2 3 4 5
Y 3 0 -1 0 3 8 15
So
f(y) =
2
7 , for y = 0, 3
1
7 , for y = −1, 8, 15
2. E[Y ]
E[Y ] = y yf(y) = (0 + 3)(2
7 ) + (−1 + 8 + 15)(1
7 )
E[Y ] = 4
3. σy
V ar[Y ] = E[Y 2
] − (E[Y ])2
)
E[Y 2
] = y y2
f(y) = (0 + 9)(2
7 ) + (1 + 64 + 225)(1
7 ) = 44
V ar[Y ] = 44 − 16 = 28 and σy =
√
28
3.2 Problem 2
Consider the probability distribution
x 1 2 3 4
f(x) a b 1
3
1
4
1. By considering the probabilities, find an equation involving a and b.
2. Given that E[X] = 2.75 , find another equation involving a and b.
3. find a and b.
4. Calculate V ar[X].
2](https://image.slidesharecdn.com/section2-200826012314/85/Section2-stochastic-2-320.jpg)
![Solution
1. Given E[X] = 2.75
From probability density function
x
f(x) = 1 (5)
a + b + 1
3 + 1
4 = 1
From Expectation Equation
E[X] = x xf(x)
a + 2b + 1 + 1 = 2.75
So
a = 1
12 , b = 1
3
‘
2. V ar[X]
V ar[X] = E[X2
] − (E[X])2
E[X2
] = x x2
f(x) = 101
12
V ar[X] = 101
12 − 2.752
= 41
48
3.3 Problem 3
Let Y = X2
, where X ∼ N(0, 1) . Calculate the mean and variance of the random variable Y .
Solution
1. E[Y ]
E[Y ] = E[X2
]
And
V ar[X] = E[X2
] − (E[X])2
= 1, E[X] = 0
E[X2
] = 1
E[Y ] = 1
2. V ar[Y ]
V ar[Y ] = E[Y 2
] − (E[Y ])2
E[Y 2
] = E[X4
] =
∞
−∞
x4
f(x)dx
X is normal distribution ∼ N(0, 1)
f(x) =
1
√
2π
e( −x2
2 )
(6)
3](https://image.slidesharecdn.com/section2-200826012314/85/Section2-stochastic-3-320.jpg)
![E[Y 2
] =
∞
−∞
x4 1√
2π
e( −x2
2 )
dx
Integration by parts
let v = e( −x2
2 )
and dv = −xe( −x2
2 )
u = x3
E[Y 2
] = −1√
2π
[x3
e( −x2
2 )
∞
−∞
−
∞
−∞
3x2
e( −x2
2 )
dx]
x3
e( −x2
2 )
∞
−∞
= 0
E[Y 2
] = 3
∞
−∞
x2
( 1√
2π
e( −x2
2 )
)dx] = 3
∞
−∞
x2
f(x)dx
E[Y 2
] = 3E[X2
] = 3
So
Y ∼ N(1, 2)
3.4 Problem 4
Let X be a random variable with the exponential probability density
f(x) =
1
µ
e
−x
µ , x ≥ 0 (7)
See figure 1.
Figure 1: The PDF of exponential distribution for different values of λ where λ = 1
µ
1. Show that X has mean µ and variance µ2
.
2. Determine the cumulative distribution function F(x).
Solution
1. E[X]
E[X] =
∞
x=0
xf(x) E[X] =
∞
x=0
x 1
µ e
−x
µ dx
Let v = e
−x
µ so dv = −1
µ e
−x
µ and u = x
4](https://image.slidesharecdn.com/section2-200826012314/85/Section2-stochastic-4-320.jpg)
![E[X] = −[xe
−x
µ
∞
0
−
∞
0
e
−x
µ dx
E[X] =
∞
0
e
−x
µ dx = −µe
−x
µ
∞
0
= µ
2. V ar[X]
E[X2
] =
∞
x=0
x2
f(x)
E[X2
] =
∞
x=0
x2 1
µ e
−x
µ dx
Let v = e
−x
µ so dv = −1
µ e
−x
µ and u = x2
E[X2
] = −[x2
e
−x
µ
∞
0
−
∞
0
2xe
−x
µ dx]
E[X2
] = 2µ
∞
0
x( 1
µ e
−x
µ )dx = 2µ
∞
0
xf(x)dx = 2µ2
V ar[X] = E[X2
] − (E[X])2
= 2µ2
− µ2
= µ2
3. CDF
F(x) =
x
0
f(x)dx
F(x) =
x
0
1
µ e( −x
µ )
dx = −e( −x
µ )
x
0
F(x) = 1 − e( −x
µ )
, x ≥ 0
F(x) with different values of µ is shown in figure 2. Remember CDF is increasing function with max value
of one.
Figure 2: The CDF of exponential probability distribution for different values of λ where λ = 1
µ
3.5 Problem 5
The Rayleigh probability density finds application in fading communication channels
f(x) =
x
σ2
e
−x2
2σ2
, x ≥ 0 (8)
PDF of Rayleigh PDF with different values of σ shown in figure 3
1. Find mean and variance.
5](https://image.slidesharecdn.com/section2-200826012314/85/Section2-stochastic-5-320.jpg)
![Figure 3: The PDF of Rayleigh distribution for different values of σ
2. Determine the cumulative distribution function F(x).
Solution
1. E[X]
E[X] =
∞
0
x2
σ2 e( −x2
2σ2 )
dx
Integration by parts v = e
−x2
σ2
, u = x
E[X] = −[xe
−x2
σ2
∞
0
−
∞
0
e( −x2
2σ2 )
dx]
xe
−x2
σ2
∞
0
= 0
E[X] =
∞
0
e( −x2
2σ2 )
dx = (
√
2πσ)
∞
0
1√
2πσ
e( −x2
2σ2 )
dx =
√
2πσ
2
Where
∞
0
1√
2πσ
e( −x2
2σ2 )
dx is half area of normal distribution.
2. V ar[X]
E[X2
] =
∞
0
x3
σ2 e( −x2
2σ2 )
dx E[X2
] = −[x2
e( −x2
2σ )
∞
0
−
∞
0
2xe( −x2
2σ2 )
dx]
E[X2
] = 2σ2 ∞
0
x
σ2 e( −x2
σ2 )
dx = 2σ2 ∞
0
f(x)dx = 2σ2
(1)
E[X2
] = 2σ2
V ar[X] = 4−π
2 σ2
3. CDF
F(x) =
x
0
f(x)dx =
x
0
x
σ2 e( −x2
2σ2 )
6](https://image.slidesharecdn.com/section2-200826012314/85/Section2-stochastic-6-320.jpg)
Section2 stochastic
- 1. Stochastic Section # 2 Expectation and Variance Eslam Adel Notes are inspired by sections made by TA Eng.Tamim Rushdi March 15, 2018 1 Expected Value Expected Value of a random variable in probability theory is the average value. For random variable X expected value of X is E[X] = x Xf(x)dx (1) where f(x) is the probability density function of x Expected value is also called mean, average, and first moment. Notes 1. E[X] is a linear operator E[X + Y ] = E[X] + E[Y ] E[aX] = aE[X] Example: E[X2 + 3X + 2] = E[X2 ] + 3E[X] + 2 2. For independent random variables X and Y E[XY ] = E[X]E[Y ]. 3. Random variables X and Y are orthogonal when E[XY ] = 0 2 Variance Variance is the average square deviation of X about the mean. V ar[X] = E[(X − m)2 ] = x (X − m)2 f(x)dx (2) Another formula V ar[X] = E[X2 ] − (E[X])2 = E[X2 ] − m2 (3) and V ar[X] = σ2 where σ is the standard deviation and m is the mean (expected value). E[X2 ] is called the second moment. nth moment for n > 1 of random variable X is defined as E[Xn ] where : E[Xn ] = x Xn f(x)dx (4) 1
- 2. 3 Problems 3.1 Problem 1 Suppose that a uniformly distributed random variable X can have each of the seven values −1, 0, 1, 2, 3, 4, 5 1. Determine the probability mass function of the random variable: Y = X2 − 2X 2. What is the expected value of Y ? 3. What is the standard deviation of Y ? Solution 1. Y = X2 − 2X X -1 0 1 2 3 4 5 Y 3 0 -1 0 3 8 15 So f(y) = 2 7 , for y = 0, 3 1 7 , for y = −1, 8, 15 2. E[Y ] E[Y ] = y yf(y) = (0 + 3)(2 7 ) + (−1 + 8 + 15)(1 7 ) E[Y ] = 4 3. σy V ar[Y ] = E[Y 2 ] − (E[Y ])2 ) E[Y 2 ] = y y2 f(y) = (0 + 9)(2 7 ) + (1 + 64 + 225)(1 7 ) = 44 V ar[Y ] = 44 − 16 = 28 and σy = √ 28 3.2 Problem 2 Consider the probability distribution x 1 2 3 4 f(x) a b 1 3 1 4 1. By considering the probabilities, find an equation involving a and b. 2. Given that E[X] = 2.75 , find another equation involving a and b. 3. find a and b. 4. Calculate V ar[X]. 2
- 3. Solution 1. Given E[X] = 2.75 From probability density function x f(x) = 1 (5) a + b + 1 3 + 1 4 = 1 From Expectation Equation E[X] = x xf(x) a + 2b + 1 + 1 = 2.75 So a = 1 12 , b = 1 3 ‘ 2. V ar[X] V ar[X] = E[X2 ] − (E[X])2 E[X2 ] = x x2 f(x) = 101 12 V ar[X] = 101 12 − 2.752 = 41 48 3.3 Problem 3 Let Y = X2 , where X ∼ N(0, 1) . Calculate the mean and variance of the random variable Y . Solution 1. E[Y ] E[Y ] = E[X2 ] And V ar[X] = E[X2 ] − (E[X])2 = 1, E[X] = 0 E[X2 ] = 1 E[Y ] = 1 2. V ar[Y ] V ar[Y ] = E[Y 2 ] − (E[Y ])2 E[Y 2 ] = E[X4 ] = ∞ −∞ x4 f(x)dx X is normal distribution ∼ N(0, 1) f(x) = 1 √ 2π e( −x2 2 ) (6) 3
- 4. E[Y 2 ] = ∞ −∞ x4 1√ 2π e( −x2 2 ) dx Integration by parts let v = e( −x2 2 ) and dv = −xe( −x2 2 ) u = x3 E[Y 2 ] = −1√ 2π [x3 e( −x2 2 ) ∞ −∞ − ∞ −∞ 3x2 e( −x2 2 ) dx] x3 e( −x2 2 ) ∞ −∞ = 0 E[Y 2 ] = 3 ∞ −∞ x2 ( 1√ 2π e( −x2 2 ) )dx] = 3 ∞ −∞ x2 f(x)dx E[Y 2 ] = 3E[X2 ] = 3 So Y ∼ N(1, 2) 3.4 Problem 4 Let X be a random variable with the exponential probability density f(x) = 1 µ e −x µ , x ≥ 0 (7) See figure 1. Figure 1: The PDF of exponential distribution for different values of λ where λ = 1 µ 1. Show that X has mean µ and variance µ2 . 2. Determine the cumulative distribution function F(x). Solution 1. E[X] E[X] = ∞ x=0 xf(x) E[X] = ∞ x=0 x 1 µ e −x µ dx Let v = e −x µ so dv = −1 µ e −x µ and u = x 4
- 5. E[X] = −[xe −x µ ∞ 0 − ∞ 0 e −x µ dx E[X] = ∞ 0 e −x µ dx = −µe −x µ ∞ 0 = µ 2. V ar[X] E[X2 ] = ∞ x=0 x2 f(x) E[X2 ] = ∞ x=0 x2 1 µ e −x µ dx Let v = e −x µ so dv = −1 µ e −x µ and u = x2 E[X2 ] = −[x2 e −x µ ∞ 0 − ∞ 0 2xe −x µ dx] E[X2 ] = 2µ ∞ 0 x( 1 µ e −x µ )dx = 2µ ∞ 0 xf(x)dx = 2µ2 V ar[X] = E[X2 ] − (E[X])2 = 2µ2 − µ2 = µ2 3. CDF F(x) = x 0 f(x)dx F(x) = x 0 1 µ e( −x µ ) dx = −e( −x µ ) x 0 F(x) = 1 − e( −x µ ) , x ≥ 0 F(x) with different values of µ is shown in figure 2. Remember CDF is increasing function with max value of one. Figure 2: The CDF of exponential probability distribution for different values of λ where λ = 1 µ 3.5 Problem 5 The Rayleigh probability density finds application in fading communication channels f(x) = x σ2 e −x2 2σ2 , x ≥ 0 (8) PDF of Rayleigh PDF with different values of σ shown in figure 3 1. Find mean and variance. 5
- 6. Figure 3: The PDF of Rayleigh distribution for different values of σ 2. Determine the cumulative distribution function F(x). Solution 1. E[X] E[X] = ∞ 0 x2 σ2 e( −x2 2σ2 ) dx Integration by parts v = e −x2 σ2 , u = x E[X] = −[xe −x2 σ2 ∞ 0 − ∞ 0 e( −x2 2σ2 ) dx] xe −x2 σ2 ∞ 0 = 0 E[X] = ∞ 0 e( −x2 2σ2 ) dx = ( √ 2πσ) ∞ 0 1√ 2πσ e( −x2 2σ2 ) dx = √ 2πσ 2 Where ∞ 0 1√ 2πσ e( −x2 2σ2 ) dx is half area of normal distribution. 2. V ar[X] E[X2 ] = ∞ 0 x3 σ2 e( −x2 2σ2 ) dx E[X2 ] = −[x2 e( −x2 2σ ) ∞ 0 − ∞ 0 2xe( −x2 2σ2 ) dx] E[X2 ] = 2σ2 ∞ 0 x σ2 e( −x2 σ2 ) dx = 2σ2 ∞ 0 f(x)dx = 2σ2 (1) E[X2 ] = 2σ2 V ar[X] = 4−π 2 σ2 3. CDF F(x) = x 0 f(x)dx = x 0 x σ2 e( −x2 2σ2 ) 6
- 7. The Result is : F(x) = 1 − e( −x2 2σ2 ) , x ≥ 0 The Cumulative probability density function (CDF) of Rayleigh distribution for different values of σ is shown in figure 4. As we can see it is an increasing function and tends to 1. Figure 4: The CDF of Rayleigh distribution for different values of σ 7