Mean Squared Error (MSE) of an Estimator
- 1. MEAN SQUARED ERROR (MSE) OF AN ESTIMATOR By Suruchi Somwanshi M.Sc. (Mathematics) M.Sc. (Statistics)
- 2. CONTENT Introduction Mean Squared Error Mean Squared Error Consistency Consistent Estimator Unbiased Estimator Tips
- 3. INTRODUCTION Let 𝜃 be the estimator of the unknown parameter θ from the random sample X1, X2, · · · , Xn. Then clearly the deviation from 𝜃 to the true value of θ, | 𝜃 − θ|, measures the quality of the estimator, or equivalently, we can use { 𝜃 - }2 for the ease of computation. Since 𝜃 is a random variable, we should take average to evaluation the quality of the estimator.
- 4. Thus, we introduce the following Definition: The mean square error (MSE) of an estimator 𝜃 of a parameter θ is the function of θ defined by E{ 𝜃 −}2, and this is denoted as 𝑀𝑆𝐸 𝜃 . This is also called the risk function of an estimator, with { 𝜃 −}2 called the quadratic loss function. The expectation is with respect to the random variables X1, · · · , Xn since they are the only random components in the expression.
- 5. Notice that the MSE measures the average squared difference between the estimator 𝜃 and the parameter θ, a somewhat reasonable measure of performance for an estimator. In general, any increasing function of the absolute distance | 𝜃 − θ| would serve to measure the goodness of an estimator (mean absolute error, E{ 𝜃 −} 2 ), is a reasonable alternative. But MSE has at least two advantages over other distance measures: First, it is analytically tractable and, secondly, it has the interpretation
- 6. MEAN SQUARED ERROR (MSE) The Mean Square Error (MSE) of an estimator for estimating is 𝑀𝑆𝐸𝜃 መ 𝜃 = 𝐸( መ 𝜃 − 𝜃)2 = 𝑉𝑎𝑟 መ 𝜃 + (𝐵𝑖𝑎𝑠𝜃 መ 𝜃 )2 If 𝑀𝑆𝐸𝜃 መ 𝜃 is smaller, መ 𝜃 is a better estimator of 𝜃. For two estimators, መ 𝜃1 and መ 𝜃2 𝑜𝑓𝜃, If 𝑀𝑆𝐸𝜃 መ 𝜃1 < 𝑀𝑆𝐸𝜃 መ 𝜃2 , 𝜃 ∈ Ω መ 𝜃1 is better estimator of 𝜃 than መ 𝜃2.
- 7. 𝑀𝑆𝐸 𝜃 = E{ 𝜃 −}2 = Var( 𝜃 ) + {𝐸( 𝜃 )−}2 = Var( 𝜃 ) + {𝐵𝑎𝑖𝑠 𝑜𝑓 𝜃 }2 This is so because E{ 𝜃 −}2 = E( 𝜃 2 ) + E(2) − 2θE( 𝜃 ) = Var( 𝜃 ) + {𝐸( 𝜃 )}2 + −2 − 2θE( 𝜃 ) = Var( 𝜃 ) + {𝐸( 𝜃 )−}2
- 8. MEAN SQUARED ERROR CONSISTENCY 𝜃 is called mean squared error consistent (or consistent in quadratic mean) if, E{ 𝜃 - }2→ 0 as n → . Theorem: 𝜃 is consistent in MSE iff i) Var( 𝜃 )→ 0 as n → . = → ) ˆ ( lim ) E ii n If E{ 𝜃 −}2→ 0 as n → , 𝜃 is also a CE of .
- 9. CONSISTENT ESTIMATOR (CE): An estimator which converges in probability to an unknown parameter for all is called a CE of . ˆ . p ⎯⎯ → For large n, a CE tends to be closer to the unknown population parameter. MLEs are generally CEs.
- 10. UNBIASED ESTIMATOR (UE): We know that, the bias of an estimator 𝜃 of a parameter θ is the difference between the expected value of 𝜃 and θ; that is, Bias ( 𝜃 ) = E( 𝜃 )−θ. And an estimator whose bias is identically equal to 0 is called unbiased estimator and satisfies E( 𝜃 ) = θ for all θ. For an unbiased estimator 𝜃 , we have 𝑀𝑆𝐸 𝜃 = E{ 𝜃 −}2 = Var( 𝜃 ) + {𝐸( 𝜃 )−}2 = Var( 𝜃 ) and so, if an estimator is unbiased, its MSE is equal to its variance.
- 11. TIPS MSE has two components, 1. One measures the variability of the estimator (precision) and 2. Other measures its bias (accuracy). An estimator that has good MSE properties has small combined variance and bias. To find an estimator with good MSE properties, we need to find estimators that control both variance and bias.
- 12. THANK YOU