SlideShare a Scribd company logo
3
Most read
4
Most read
7
Most read
Estimation
Properties of Estimators
- Bias
- Mean square error
By
SURUCHI SOMWANSHI
M.Sc. (Mathematics)
M.Sc. (Statistics)
Estimate and Estimator
 The estimate is a single value
Ex.− ො
𝜇 = 5 in particular ො
𝜇 = ത
𝑋
 The estimator is the distribution of the estimates
Ex. - ො
𝜇 = ത
𝑋
BIAS
 Definition:
The bias of an estimator መ
𝜃 of a parameter 𝜃 is the difference between
the expected value of መ
𝜃 and 𝜃; that is,
𝐵𝑎𝑖𝑠 መ
𝜃 = 𝐸( መ
𝜃) - 𝜃
 An estimator whose bias is identically equal to 0 is called unbiased
estimator and satisfies 𝐸( መ
𝜃) = 𝜃 for all 𝜃.
Simply Bias is
 The difference between the average value of the estimator and the true value.
i.e., 𝐵𝑎𝑖𝑠 መ
𝜃 = 𝐸( መ
𝜃) - 𝜃
 If 𝐵𝑎𝑖𝑠 መ
𝜃 = 0 then the estimator is said to be unbiased.
 Example1
Determine the Bias of the estimator ො
𝜇 = ത
𝑋 of the parameter 𝜇 from a poi(𝜇) distribution.
Solution :
𝐵𝑎𝑖𝑠 ො
𝜇 = 𝐸(ො
𝜇) − 𝜇
= E( ത
𝑋) − 𝜇 -------- Since ො
𝜇 = ത
𝑋 is given
= 𝐸(
1
𝑛
σ𝑖=1
𝑛
𝑥𝑖) − 𝜇
=
1
𝑛
σ𝑖=1
𝑛
𝐸(𝑥𝑖) − 𝜇
=
1
𝑛
σ𝑖=1
𝑛
𝜇 − 𝜇 -------- Since as 𝑥𝑖~𝑃𝑜𝑖(𝜇) => 𝐸 𝑥𝑖 = 𝜇
=
1
𝑛
𝑛𝜇 − 𝜇 --------- As 𝜇 𝑖𝑠 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡 𝑜𝑣𝑒𝑟 𝑠𝑢𝑚𝑚𝑎𝑡𝑖𝑜𝑛
= 𝜇 − 𝜇
=0
Which means the estimator ത
𝑋 is unbiased . On average the value of the sample mean will give the
true mean of the distribution.
 Example2
Determine the Bias of the estimator Ƹ
𝑝 =
𝑋
𝑛
of the parameter 𝑝 from a Bin(n,p) distribution.
Solution :
𝐵𝑎𝑖𝑠 Ƹ
𝑝 = 𝐸 Ƹ
𝑝 − 𝑝
= 𝐸
𝑋
𝑛
− 𝑝 -------- Since Ƹ
𝑝 =
𝑋
𝑛
is given
=
1
𝑛
E(X) −𝑝
=
1
𝑛
𝑛𝑝 − 𝑝 -------- Since as 𝑥𝑖~Bin(n,p) ⇒ 𝐸(𝑥𝑖) = 𝑛𝑝
= 𝑝 − 𝑝
= 0
Which means the estimator
𝑋
𝑛
is unbiased for P . On an average will give the true value for p.
Mean Square Error (MSE)
 Mean square error is The average of the error squared.
 Where error means the difference between the true value of the
parameter (𝜃) and the estimated value ( መ
𝜃)of parameter.
 𝑀𝑆𝐸 መ
𝜃 = 𝐸 መ
𝜃 − 𝜃
2
= 𝑉 መ
𝜃 + 𝐵𝑖𝑎𝑠 ( መ
𝜃)
2
This formula gives us the measure of spread of the result around the
true value of parameter.
 Thus, MSE has two components, one measures the variability of the
estimator (precision) and the other measures its bias (accuracy).
 An estimator that has good MSE properties has small combined
variance and bias.
 To find an estimator with good MSE properties, we need to find
estimators that control both variance and bias.
 For an unbiased estimator መ
𝜃, we have
𝑀𝑆𝐸 መ
𝜃 = 𝐸 መ
𝜃 − 𝜃
2
= 𝑉 መ
𝜃 and so, if an estimator is unbiased, its MSE
is equal to its variance.
 An estimator with smaller MSE is said to be more efficient. As if it has
a smaller spread around the true value.
 If the MSE of the estimator tends to zero as 𝑛 → ∞ then it is said to be
consistent.
 Example1
Determine the MSE of the estimator Ƹ
𝜇 = ത
𝑋 of the parameter 𝜇 from a poi(𝜇) distribution.
Solution :
𝑀𝑆𝐸 Ƹ
𝜇 = 𝑉 Ƹ
𝜇 + 𝐵𝑖𝑎𝑠 ( Ƹ
𝜇 )2
= V( ത
𝑋) + 02
-------- From example1 of Bias the 𝐵𝑖𝑎𝑠 ( Ƹ
𝜇 )=0 (Slide 4)
-------- Since Ƹ
𝜇 = ത
𝑋 is given
= V (
1
𝑛
σ𝑖=1
𝑛
𝑥𝑖)
=
1
𝑛2
σ𝑖=1
𝑛
𝑉(𝑥𝑖)
=
1
𝑛2
σ𝑖=1
𝑛
𝜇 -------- Since as 𝑥𝑖~𝑃𝑜𝑖(𝜇) ⇒ 𝑉 𝑥𝑖 = 𝜇
=
1
𝑛2 𝑛𝜇 -------- As 𝜇 𝑖𝑠 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡 𝑜𝑣𝑒𝑟 𝑠𝑢𝑚𝑚𝑎𝑡𝑖𝑜𝑛
=
𝜇
𝑛
For larger values of n our MSE will be more efficient, as you will have smaller MSE and we can see as n gets larger and
larger the MSE tends to zero and so this is also a consistent estimator. It becomes more accurate as sample size
increases.
 Example2
Determine the MSE of the estimator Ƹ
𝑝 =
𝑋
𝑛
of the parameter 𝑝 from a Bin(n,p) distribution.
Solution :
𝑀𝑆𝐸 Ƹ
𝑝 = 𝑉 Ƹ
𝑝 + 𝐵𝑖𝑎𝑠( Ƹ
𝑝 )2
= V
𝑋
𝑛
+ 02
-------- From example2 of Bias the 𝐵𝑖𝑎𝑠( Ƹ
𝑝 )=0 (Slide 5)
-------- Since Ƹ
𝑝 =
𝑋
𝑛
is given
=
1
𝑛2 V (X)
=
1
𝑛2 𝑛𝑝q -------- Since as 𝑥𝑖~Bin(n,p) ⇒ V(𝑥𝑖) = 𝑛𝑝𝑞
=
1
𝑛
𝑝 q
For larger values of n our MSE will be more efficient, as you will have smaller MSE and we can
see as n gets larger and larger the MSE tends to zero and so this is a consistent estimator.
THANK YOU

More Related Content

PPT
Chi square mahmoud
PDF
Introduction to Generalized Linear Models
PDF
Mean Squared Error (MSE) of an Estimator
PDF
Chapter 4 part2- Random Variables
PDF
Simple linear regression
PPTX
Basic Concepts of Probability
PPTX
Logistic regression
PPTX
Multinomial Logistic Regression Analysis
Chi square mahmoud
Introduction to Generalized Linear Models
Mean Squared Error (MSE) of an Estimator
Chapter 4 part2- Random Variables
Simple linear regression
Basic Concepts of Probability
Logistic regression
Multinomial Logistic Regression Analysis

What's hot (20)

PPTX
Binomial probability distributions ppt
PDF
Multiple linear regression
ODP
Correlation
PPTX
Testing of hypotheses
PPTX
Stat 3203 -pps sampling
PPTX
Mathematical Expectation And Variance
PPTX
Two Means, Independent Samples
PDF
Assumptions of Linear Regression - Machine Learning
PPTX
Statistical inference: Estimation
PDF
Statistical Distributions
PPT
sampling distribution
PPTX
Inferential statistics powerpoint
PPT
08 test of hypothesis large sample.ppt
PPTX
Logistic regression with SPSS
PPTX
Psych stats Probability and Probability Distribution
PPTX
Lesson 2 stationary_time_series
DOCX
Estimation in statistics
PPTX
Statistical inference
PPTX
Point estimation
PPT
In Anova
Binomial probability distributions ppt
Multiple linear regression
Correlation
Testing of hypotheses
Stat 3203 -pps sampling
Mathematical Expectation And Variance
Two Means, Independent Samples
Assumptions of Linear Regression - Machine Learning
Statistical inference: Estimation
Statistical Distributions
sampling distribution
Inferential statistics powerpoint
08 test of hypothesis large sample.ppt
Logistic regression with SPSS
Psych stats Probability and Probability Distribution
Lesson 2 stationary_time_series
Estimation in statistics
Statistical inference
Point estimation
In Anova
Ad

Similar to Bias and Mean square error (20)

PDF
Statistical parameters
PDF
Estimation Theory Class (Summary and Revision)
PDF
Bio-L8- Correlation and Regression Analysis.pdf
PDF
IUT Probability and Statistics - Chapter 02_Part-1.pdf
PDF
Methods of point estimation
PDF
Class of Estimators of Population Median Using New Parametric Relationship fo...
PDF
InnerSoft STATS - Methods and formulas help
DOCX
Makalah ukuran penyebaran
PPT
Correlation and Regression analysis .ppt
PDF
PDF
Simple Linear Regression
PDF
Econometrics 1 Slide from the masters degree 1
DOCX
Course pack unit 5
PDF
A Mathematical Model for the Enhancement of Stress Induced Hypoglycaemia by A...
PDF
A Mathematical Model for the Enhancement of Stress Induced Hypoglycaemia by A...
PPTX
Simple Regression.pptx
PPTX
UNIT 4 PTRP final Convergence in probability.pptx
PPTX
L1 updated introduction.pptx
PPTX
#3Measures of central tendency
PDF
ch02.pdf
Statistical parameters
Estimation Theory Class (Summary and Revision)
Bio-L8- Correlation and Regression Analysis.pdf
IUT Probability and Statistics - Chapter 02_Part-1.pdf
Methods of point estimation
Class of Estimators of Population Median Using New Parametric Relationship fo...
InnerSoft STATS - Methods and formulas help
Makalah ukuran penyebaran
Correlation and Regression analysis .ppt
Simple Linear Regression
Econometrics 1 Slide from the masters degree 1
Course pack unit 5
A Mathematical Model for the Enhancement of Stress Induced Hypoglycaemia by A...
A Mathematical Model for the Enhancement of Stress Induced Hypoglycaemia by A...
Simple Regression.pptx
UNIT 4 PTRP final Convergence in probability.pptx
L1 updated introduction.pptx
#3Measures of central tendency
ch02.pdf
Ad

Recently uploaded (20)

PPTX
Microbiology with diagram medical studies .pptx
PPTX
famous lake in india and its disturibution and importance
PPTX
GEN. BIO 1 - CELL TYPES & CELL MODIFICATIONS
PPTX
Introduction to Cardiovascular system_structure and functions-1
PPTX
DRUG THERAPY FOR SHOCK gjjjgfhhhhh.pptx.
PDF
. Radiology Case Scenariosssssssssssssss
PPTX
BIOMOLECULES PPT........................
PPTX
TOTAL hIP ARTHROPLASTY Presentation.pptx
PPTX
Protein & Amino Acid Structures Levels of protein structure (primary, seconda...
PPTX
Comparative Structure of Integument in Vertebrates.pptx
PDF
Phytochemical Investigation of Miliusa longipes.pdf
PPTX
cpcsea ppt.pptxssssssssssssssjjdjdndndddd
DOCX
Viruses (History, structure and composition, classification, Bacteriophage Re...
PPTX
7. General Toxicologyfor clinical phrmacy.pptx
PDF
SEHH2274 Organic Chemistry Notes 1 Structure and Bonding.pdf
PDF
An interstellar mission to test astrophysical black holes
PPTX
microscope-Lecturecjchchchchcuvuvhc.pptx
PDF
Mastering Bioreactors and Media Sterilization: A Complete Guide to Sterile Fe...
PDF
bbec55_b34400a7914c42429908233dbd381773.pdf
PDF
AlphaEarth Foundations and the Satellite Embedding dataset
Microbiology with diagram medical studies .pptx
famous lake in india and its disturibution and importance
GEN. BIO 1 - CELL TYPES & CELL MODIFICATIONS
Introduction to Cardiovascular system_structure and functions-1
DRUG THERAPY FOR SHOCK gjjjgfhhhhh.pptx.
. Radiology Case Scenariosssssssssssssss
BIOMOLECULES PPT........................
TOTAL hIP ARTHROPLASTY Presentation.pptx
Protein & Amino Acid Structures Levels of protein structure (primary, seconda...
Comparative Structure of Integument in Vertebrates.pptx
Phytochemical Investigation of Miliusa longipes.pdf
cpcsea ppt.pptxssssssssssssssjjdjdndndddd
Viruses (History, structure and composition, classification, Bacteriophage Re...
7. General Toxicologyfor clinical phrmacy.pptx
SEHH2274 Organic Chemistry Notes 1 Structure and Bonding.pdf
An interstellar mission to test astrophysical black holes
microscope-Lecturecjchchchchcuvuvhc.pptx
Mastering Bioreactors and Media Sterilization: A Complete Guide to Sterile Fe...
bbec55_b34400a7914c42429908233dbd381773.pdf
AlphaEarth Foundations and the Satellite Embedding dataset

Bias and Mean square error

  • 1. Estimation Properties of Estimators - Bias - Mean square error By SURUCHI SOMWANSHI M.Sc. (Mathematics) M.Sc. (Statistics)
  • 2. Estimate and Estimator  The estimate is a single value Ex.− ො 𝜇 = 5 in particular ො 𝜇 = ത 𝑋  The estimator is the distribution of the estimates Ex. - ො 𝜇 = ത 𝑋
  • 3. BIAS  Definition: The bias of an estimator መ 𝜃 of a parameter 𝜃 is the difference between the expected value of መ 𝜃 and 𝜃; that is, 𝐵𝑎𝑖𝑠 መ 𝜃 = 𝐸( መ 𝜃) - 𝜃  An estimator whose bias is identically equal to 0 is called unbiased estimator and satisfies 𝐸( መ 𝜃) = 𝜃 for all 𝜃.
  • 4. Simply Bias is  The difference between the average value of the estimator and the true value. i.e., 𝐵𝑎𝑖𝑠 መ 𝜃 = 𝐸( መ 𝜃) - 𝜃  If 𝐵𝑎𝑖𝑠 መ 𝜃 = 0 then the estimator is said to be unbiased.
  • 5.  Example1 Determine the Bias of the estimator ො 𝜇 = ത 𝑋 of the parameter 𝜇 from a poi(𝜇) distribution. Solution : 𝐵𝑎𝑖𝑠 ො 𝜇 = 𝐸(ො 𝜇) − 𝜇 = E( ത 𝑋) − 𝜇 -------- Since ො 𝜇 = ത 𝑋 is given = 𝐸( 1 𝑛 σ𝑖=1 𝑛 𝑥𝑖) − 𝜇 = 1 𝑛 σ𝑖=1 𝑛 𝐸(𝑥𝑖) − 𝜇 = 1 𝑛 σ𝑖=1 𝑛 𝜇 − 𝜇 -------- Since as 𝑥𝑖~𝑃𝑜𝑖(𝜇) => 𝐸 𝑥𝑖 = 𝜇 = 1 𝑛 𝑛𝜇 − 𝜇 --------- As 𝜇 𝑖𝑠 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡 𝑜𝑣𝑒𝑟 𝑠𝑢𝑚𝑚𝑎𝑡𝑖𝑜𝑛 = 𝜇 − 𝜇 =0 Which means the estimator ത 𝑋 is unbiased . On average the value of the sample mean will give the true mean of the distribution.
  • 6.  Example2 Determine the Bias of the estimator Ƹ 𝑝 = 𝑋 𝑛 of the parameter 𝑝 from a Bin(n,p) distribution. Solution : 𝐵𝑎𝑖𝑠 Ƹ 𝑝 = 𝐸 Ƹ 𝑝 − 𝑝 = 𝐸 𝑋 𝑛 − 𝑝 -------- Since Ƹ 𝑝 = 𝑋 𝑛 is given = 1 𝑛 E(X) −𝑝 = 1 𝑛 𝑛𝑝 − 𝑝 -------- Since as 𝑥𝑖~Bin(n,p) ⇒ 𝐸(𝑥𝑖) = 𝑛𝑝 = 𝑝 − 𝑝 = 0 Which means the estimator 𝑋 𝑛 is unbiased for P . On an average will give the true value for p.
  • 7. Mean Square Error (MSE)  Mean square error is The average of the error squared.  Where error means the difference between the true value of the parameter (𝜃) and the estimated value ( መ 𝜃)of parameter.  𝑀𝑆𝐸 መ 𝜃 = 𝐸 መ 𝜃 − 𝜃 2 = 𝑉 መ 𝜃 + 𝐵𝑖𝑎𝑠 ( መ 𝜃) 2 This formula gives us the measure of spread of the result around the true value of parameter.
  • 8.  Thus, MSE has two components, one measures the variability of the estimator (precision) and the other measures its bias (accuracy).  An estimator that has good MSE properties has small combined variance and bias.  To find an estimator with good MSE properties, we need to find estimators that control both variance and bias.
  • 9.  For an unbiased estimator መ 𝜃, we have 𝑀𝑆𝐸 መ 𝜃 = 𝐸 መ 𝜃 − 𝜃 2 = 𝑉 መ 𝜃 and so, if an estimator is unbiased, its MSE is equal to its variance.  An estimator with smaller MSE is said to be more efficient. As if it has a smaller spread around the true value.  If the MSE of the estimator tends to zero as 𝑛 → ∞ then it is said to be consistent.
  • 10.  Example1 Determine the MSE of the estimator Ƹ 𝜇 = ത 𝑋 of the parameter 𝜇 from a poi(𝜇) distribution. Solution : 𝑀𝑆𝐸 Ƹ 𝜇 = 𝑉 Ƹ 𝜇 + 𝐵𝑖𝑎𝑠 ( Ƹ 𝜇 )2 = V( ത 𝑋) + 02 -------- From example1 of Bias the 𝐵𝑖𝑎𝑠 ( Ƹ 𝜇 )=0 (Slide 4) -------- Since Ƹ 𝜇 = ത 𝑋 is given = V ( 1 𝑛 σ𝑖=1 𝑛 𝑥𝑖) = 1 𝑛2 σ𝑖=1 𝑛 𝑉(𝑥𝑖) = 1 𝑛2 σ𝑖=1 𝑛 𝜇 -------- Since as 𝑥𝑖~𝑃𝑜𝑖(𝜇) ⇒ 𝑉 𝑥𝑖 = 𝜇 = 1 𝑛2 𝑛𝜇 -------- As 𝜇 𝑖𝑠 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡 𝑜𝑣𝑒𝑟 𝑠𝑢𝑚𝑚𝑎𝑡𝑖𝑜𝑛 = 𝜇 𝑛 For larger values of n our MSE will be more efficient, as you will have smaller MSE and we can see as n gets larger and larger the MSE tends to zero and so this is also a consistent estimator. It becomes more accurate as sample size increases.
  • 11.  Example2 Determine the MSE of the estimator Ƹ 𝑝 = 𝑋 𝑛 of the parameter 𝑝 from a Bin(n,p) distribution. Solution : 𝑀𝑆𝐸 Ƹ 𝑝 = 𝑉 Ƹ 𝑝 + 𝐵𝑖𝑎𝑠( Ƹ 𝑝 )2 = V 𝑋 𝑛 + 02 -------- From example2 of Bias the 𝐵𝑖𝑎𝑠( Ƹ 𝑝 )=0 (Slide 5) -------- Since Ƹ 𝑝 = 𝑋 𝑛 is given = 1 𝑛2 V (X) = 1 𝑛2 𝑛𝑝q -------- Since as 𝑥𝑖~Bin(n,p) ⇒ V(𝑥𝑖) = 𝑛𝑝𝑞 = 1 𝑛 𝑝 q For larger values of n our MSE will be more efficient, as you will have smaller MSE and we can see as n gets larger and larger the MSE tends to zero and so this is a consistent estimator.