SlideShare a Scribd company logo

Math stat 3

T
thiduongnguyen

This document discusses statistical estimation theory and parameters. It covers estimating nonparametric population features like means, correlations, and proportions. It also covers estimating parameters by fitting distributions like the mean and variance of a lognormal distribution, and estimating relationships in parametric models. The document indicates it will cover estimation theory as it relates to statistical inference and regression analysis.

EducationTechnology
1 of 86
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
Statistical Inference and Regression Analysis: GB.3302.30 Professor William Greene Stern School of Business IOMS Department Department of Economics
Statistics and Data Analysis Part 3 – Estimation Theory
Estimation  Nonparametric population features  Mean - income  Correlation – disease incidence and smoking  Ratio – income per household member  Proportion – proportion of ASCAP music played that is produced by Dave Matthews  Parameters  Fitting distributions – mean and variance of lognormal distribution of income  Parametric models of populations – relationship of loan rates to attributes of minorities and others in Bank of America settlement on mortgage bias Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 3 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
Measurements as Observations Population Measurement Theory The theory argues that there are Characteristics meaningful quantities to be statistically Behavior Patterns analyzed. Choices Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
Application – Health and Income German Health Care Usage Data, 7,293 Households, Observed 1984-1995 Data downloaded from Journal of Applied Econometrics Archive. Some variables in the file are DOCVIS = number of visits to the doctor in the observation period HOSPVIS = number of visits to a hospital in the observation period HHNINC = household nominal monthly net income in German marks / 10000. (4 observations with income=0 were dropped) HHKIDS = children under age 16 in the household = 1; otherwise = 0 EDUC = years of schooling AGE = age in years PUBLIC = decision to buy public health insurance HSAT = self assessed health status (0,1,…,10) Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
Observed Data Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 6 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
Inference about Population Population Measurement Characteristics Behavior Patterns Choices Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
Classical Inference The population is all 40 million German households (or all households in the entire world). The sample is the 7,293 German households in 1984-1995. Population Sample Measurement Imprecise inference about Characteristics the entire population – Behavior Patterns sampling theory and asymptotics Choices Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
Bayesian Inference Population Sample Measurement Sharp, ‘exact’ inference about Characteristics only the sample – the ‘posterior’ density is posterior to the data. Behavior Patterns Choices Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
Estimation of Population Features  Estimators and Estimates  Estimator = strategy for use of the data  Estimate = outcome of that strategy  Sampling Distribution and Uncertainty Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 10 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
Estimation  Point Estimator: Provides a single estimate of the feature in question based on prior and sample information.  Interval Estimator: Provides a range of values that incorporates both the point estimator and the uncertainty about the ability of the point estimator to find the population feature exactly. Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 11 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
„Repeated Sampling‟ - A Sampling Distribution  The true mean is 500. Sample means vary around 500, some quite far off.  The sample mean has a sampling mean and a sampling variance.  The sample mean also has a probability distribution. Looks like a normal distribution. This is a histogram for 1,000 means of samples of 20 observations from Normal[500,1002]. Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
Application: Credit Modeling  1992 American Express analysis of  Application process: Acceptance or rejection; X = 0 (reject) or 1 (accept).  Cardholder behavior • Loan default (D = 0 or 1). • Average monthly expenditure (E = $/month) • General credit usage/behavior (Y = number of charges)  13,444 applications in November, 1992 Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
X in 100 samples with N = 144 in each sample 0.7809 is the true proportion in the population of 13,444 we are sampling from. Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
Estimation Concepts  Random Sampling  Finite populations  i.i.d. sample from an infinite population  Information  Prior  Sample X (X1 , X 2 ,..., X N ) a random sample = a set of 'outside', nonsample information about the population or the feature of interest = a feature of the population of interest ˆ (X , X ,..., X | ) = an estimator of N 1 2 N Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 15 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
Properties of Estimators ˆ is a function of a random sample, so it is a random variable N Unbiasedness: E ˆ N Asymptotic Unbiasedness : lim N E ˆN (Usually not useful) Consistency: Plim ˆ N . (Convergence in mean square is usually sufficient.) Efficiency: The 'best' use of the data when there is more than one alternative estimator available. Sampling Distribution: Properties of the estimator used for constructing statistical inference. Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 16 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
Unbiasedness The sample mean of the 100 sample estimates is 0.7844. The population mean (true proportion) is 0.7809. Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
Consistency N=144 .7 to .88 N=1024 .7 to .88 N=4900 .7 to .88 Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
Competing Estimators of a Parameter Di s tri b u ti o n o f Co s t o f 5 0 0 Ba n k s (i n 5 y e a rs ) 232 174 F req u en cy 116 58 0 8. 538 9. 681 10. 824 11. 967 13. 110 14. 253 15. 395 16. 538 C Bank costs are normally distributed with mean . Which is a better estimator of , the mean (11.46) or the median (11.27)? Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 19 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
Interval Estimates of the Acceptance Rate Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
Methods of Estimation  Information about the source population  Approaches  Method of Moments  Maximum Likelihood  Bayesian Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 21 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
The Method of Moments Estimating Parameters of Distributions Using Moment Equations Population Moment k E[xk ] fk ( 1, 2 ,..., K ) Sample Moment 1 N k 1 N mk N i 1 i x --- mk may also be N i 1 k h (xi ), need not be powers Law of Large Numbers plim mk k fk ( 1, 2 ,..., K ) 'Moment Equation' (k = 1,...,K) = sample analog to population. Equate the sample moment to the function of population parameters. 1 N k mk N i 1 i x fk ( 1, 2 ,..., K ) Method of Moments Estimator. Invert the moment equations. ˆ g (m ,...,m ), k = 1,...,K k k 1 K Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
Estimating a Parameter  Mean of Poisson  p(y)=exp(-λ) λy / y!, y = 0,1,…; λ > 0  E[y]= λ.  E[(1/N)Σiyi]= λ. This is the estimator  Mean of Exponential  f(y) = exp(- y), y > 0; > 0  E[y] = 1/ .  E(1/N)Σiyi = 1/ .  1/{(1/N)Σiyi } is the estimator of Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
Mean and Variance of a Normal Distribution 2 1 1 y f(y) exp 2 2 Population Moments E[y] , E[y 2 ] 2 2 Moment Equations 1 N N i 1 i y , 1 N 2 2 2 N i 1 i y Method of Moments Estimators ˆ =y, ˆ2 1 N N 2 i 1 i y (y 2 ) 1 N N i 1 (y i y)2 Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
Gamma Distribution P exp( y)yP 1 f(y) (P) P E[y] P(P 1) E[y 2 ] 2 E[1/ y] P 1 E[log y] (P) log , (P)=dln (P)/dP Any pair of moments can be used to estimate and P. Each pair gives a different answer. Is there a 'best' pair? Yes, the ones that are 'sufficient' statistics. E[y] and E[logy]. Later. Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
Proportion for Bernoulli  In the AmEx data, the true population acceptance rate is 0.7809 =  Y = 1 if application accepted, 0 if not.  E[y] =  E[(1/N)Σiyi] = paccept = .  This is the estimator Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 26 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
Method of Moments Gamma Distribution Parameters 2 Plot of Psi(P) Function 0 P yi )yP i exp( 1 p(yi ) -2 (P) -4 PSI Population Moments -6 -8 P -10 E[yi ] , E[logy i ] (P) log -12 0 1 2 3 4 5 6 P Moment Equations: N m1( ,P) = E[{(1/N) i=1 i y }] P/ N (P) = (P) / (P) mlog ( ,P) = E[{(1/N) i=1 log yi }] (P) log = dlog (P)/dP Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 28 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
Estimate One Parameter  Assume known to be 0.1.  Estimate P  E[y] = P/ = P/.1 = 10P  m1 = mean of y = 31.278  Estimate of P is 31.278/10 = 3.1278.  One equation in one unknown Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 29 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
Application Moment equations N m1( ,P) = E[{(1/N) i=1 i y }] P/ N 2 2 m2 ( ,P) = E[{(1/N) i=1 i y }] P(P 1) / N mlog ( ,P) = E[{(1/N) i=1 log y i }] (P) log Solving the moment equations using 'least squares:' (This is a convenient approach.) Minimize {m1 E[m1]} 2 {mlog E[m1]} 2 (m1 (P / ))2 (mlog ( (P) log ))2 m1 31.278 mlog 3.221387 Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
Method of Moments Solutions create ; y1=y ; y2=log(y) ; ysq=y*y$ calc ; m1=xbr(y1) ; mlog=xbr(y2); m2=xbr(ysq) $ Minimize; start = 2.0, .06 ; labels = p,l ; fcn= (m1 - p/l)^2 + (mlog – (psi(p)-log(l)))^2 $ ------------------------------------------------------------------------------ P| 2.41074 L| .07707 --------+-------------------------------------------------------------------- Minimize; start = 2.0, .06 ; labels = p,l ; fcn= (m1 - p/l)^2 + (m2 – p*(p+1)/l^2 )^2 $ --------+-------------------------------------------------------------------- P| 2.06182 L| .06589 --------+-------------------------------------------------------------------- Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
Properties of MoM estimator  Unbiased? Sometimes, e.g., normal, Bernoulli and Poisson means  Consistent? Yes by virtue of Slutsky Theorem  Assumes parameters can vary continuously  Assumes moment functions are continuous and smooth  Efficient? Maybe – remains to be seen. (Which pair of moments should be used for the gamma distribution?)  Sampling distribution? Generally normal by virtue of Lindeberg-Levy central limit theorem and the Slutsky theorem. Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 32 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
Estimating Sampling Variance  Exact sampling results – Poisson Mean, Normal Mean and Variance  Approximation based on linearization  Bootstrapping – discussed later with maximum likelihood estimator. Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 33 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
Exact Variance of MoM  Estimate normal or Poisson mean  Estimator is sample mean = (1/N) i Yi.  Exact variance of sample mean is 1/N * population variance. Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 34 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
Linearization Approach – 1 Parameter THEORY: Variance of the Method of Moments Estimator APPLICATION: Exponential Distribution Distribution is a function of a parameter f(x| ) = exp(- x), > 0, x 0. 1 N E[x]=1/ , Var[x]=1/ 2 Moment equation: E g(x i ) m f ( ) N i1 ˆ = 1/x. Write the equation in the form m( ) m f ( ) m( ) = x - 1/ Linearization: m( ˆ ) m( ) + m'( )( ˆ ) m'( ) = 1/ 2 . 1/m'( ) 2 ˆ m( ˆ ) m( ) m( ) + but m( ˆ ) 0 so ˆ - Var[x] 1 / 2 m'( ) m'( ) Var[m(θ)] Var[x] 2 N N 1 Var[ ˆ ] Var m( ) . 1/ 2 2 m'( ) ˆ Var[θ] 2 2 N N Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 35 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
Linearization Approach - General Estimator is derived from moment equations m a - f a ( 1 , 2 ,..., K ) 0 mb - fb ( 1 , 2 ,..., K ) 0 ... 1 N Moments are sample means, m j g j ( yi ), j 1,..., K N i1 There are the same number of moment equations as there are parameters Functions may be powers of x, log of x, 1/x, or other functions. Solutions are ˆ Q(m ,m ,...,m ) etc. j a b K Two items needed to compute the sampling variance Vjk Cov[m j ,m k ] for all pairs arranged in a matrix f a ( 1 , 2 ,..., K ) J = K K matrix of derivatives, 1 , 2 ,..., K The variance is then J -1 VJ -1 . (Requires matrix algebra. Later in the course.) Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 36 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
Exercise: Gamma Parameters  m1 = 1/N yi = P/  m2 = 1/N yi2 = P(P+1)/ 2 1. What is the Jacobian? 2. How to compute the variance of m1, the variance of m2 and the covariance of m1 and m2? (The variance of m1 is 1/N times the variance of y; the variance of m2 is 1/N times the variance of y2. The covariance is 1/N times the covariance of y and y2.) Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 37 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
Sufficient Statistics Moment equations N m1( ,P) = E[{(1/N) i=1 i y }] P/ N 2 2 m2 ( ,P) = E[{(1/N) i=1 i y }] P(P 1) / N mlog ( ,P) = E[{(1/N) i=1 log y i }] (P) log N m 1( ,P) = E[{(1/N) i=1 (1/ y i )}] / (P 1) Any pair can be used to estimate P and . Is there a best choice? If 'sufficient statistics' exist, the estimator that is a function of them will have a smaller variance than one that is not. Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 38 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
Sufficient Statistic x has density f(x| ) where is a parameter. The joint density of a random sample is f(x1 ,x 2 ,...,x N | ) T(x1 ,x 2 ,...,x N ) is a statistic formed from the N observations. If the conditional distribution, f(x1 ,x 2 ,...,x N | T(...), ) is not a function of , then T(...) is a sufficient statistic for . Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 39 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
Sufficient Statistic N Bernoulli Trials, X = 0 or 1 with probability i X i = t successes t Prob(X1 =x1 ,...,X N =x N ) = (1 )N t t Prob(X1 =x1 ,...,X N =x N and i X i = t) = (1 )N t N t Prob( i X i = t) = (1 )N t t t (1 )N t 1 Prob(X1 =x1 ,...,X N =x N | i X i = t) = N t N (1 )N t t t Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 40 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
Sufficient Statistics Sufficient Statistics only exist for 'Exponential Families' of distributions. f(x i | ) is an exponential family if and only if log f(x i | ) = c( )T(x i ) + d( ) + S(x i ). N Then, i 1 T ( xi ) is the sufficient statistic. When there are K parameters, K log f(x i | 1 ,..., K ) = k 1 ck ( 1 ,..., K )Tk (x i ) + d( 1 ,..., K ) + S(x i ) x x 1 x Bernoulli: f(x| ) (1 (1 ).) 1 c( ) log[ / (1 )], T(x i ) = x i , d( ) log(1 ), S ( x) 0 N N The sufficient statistic is i 1 T(x i ) = i 1 xi Poisson: log f(x| ) = x log - - log x! N N The sufficient statistic is i 1 T(x i ) = i 1 xi Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 41 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
Gamma Density P x P 1e x f(x| ,P)= ( P) log f(x i | ,P) P log + (P-1)log x i - x i - log (P) N N The sufficient statistics are m1 = i x and mlog = 1 i i 1 log x i Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 42 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
Rao Blackwell Theorem  The mean squared error of an estimator based on sufficient statistics is smaller than one not based on sufficient statistics  We deal in consistent estimators, so a large sample (approximate) version of the theorem is that estimators based on sufficient statistics are more efficient than those that are not. Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 43 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
Maximum Likelihood  Estimation Criterion  Comparable to method of moments  Several virtues: Broadly, uses all the sample and nonsample information available  efficient (better than MoM in many cases) Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 44 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
Setting Up the MLE The distribution of the observed random variable is written as a function of the parameter(s) to be estimated P(yi| ) = Probability density of data | parameters. L( |yi) = likelihood of parameter | data The likelihood function is constructed from the density Construction: Joint probability density function of the observed sample of data – generally the product when the data are a random sample. The estimator is chosen to maximize the likelihood of the data (essentially the probability of observing the sample in hand). Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
Regularity Conditions  What they are  1. logf(.) has three continuous derivatives wrt parameters  2. Conditions needed to obtain expectations of derivatives are met. (E.g., range of the variable is not a function of the parameters.)  3. Third derivative has finite expectation.  What they mean  Moment conditions and convergence. We need to obtain expectations of derivatives.  We need to be able to truncate Taylor series.  We will use central limit theorems  MLE exists for nonregular densities (see text). Questionable statistical properties. Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
Regular Exponential Density Exponential density f(yi| )=(1/ )exp(-yi/ ) Average time until failure, , of light bulbs. yi = observed life until failure. Regularity (1) Range of y is 0 to free of (2) logf(yi| ) = -log – y/ ∂logf(yi| )/∂ = -1/ + yi/ 2 E[yi]= , E[∂logf( )/∂ ]=0 (3) ∂2logf(yi| )/∂ 2 = 1/ 2 - 2yi/ 3 finite expectation = -1/ 2 (4) ∂3logf(yi| )/∂ 3 = -2/ 3 + 6yi/ 4 has finite expectation = 4/ 3 (5) All derivatives are continuous functions of Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
Likelihood Function  L( )=Πi f(yi| )  MLE = the value of that maximizes the likelihood function.  Generally easier to maximize the log of L. The same maximizes log L  In random sampling, logL= i log f(yi| ) Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 48 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
Poisson Likelihood y = 5, 0, 1, 1, 0, 3, 2, 3, 4, 1 y exp(- ) f(y| ) = =Poisson y! exp( ) 5 exp( ) 0 exp( ) 1 Likelihood = ... 5! 0! 1! exp( 10 ) 20 = 207,360 Log likelihood = -10 + 20log - 12.242 Maximum occurs at = 2 log and ln both mean natural log throughout the course Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 49 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
The MLE The log-likelihood function: log-L( |data)= Σi logf(yi| ) The likelihood equation(s) = first derivative: First derivatives of log-L equals zero at the MLE. ∂[Σi logf(yi| )]/∂ MLE = 0. (Interchange sum and differentiation) Σi [∂logf(yi| )/∂ MLE]= 0. Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
Applications  Bernoulli  Exponential  Poisson  Normal  Gamma Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 51 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
Bernoulli f(y|θ)=(1-θ)1-y θ y log f=(1-y)log(1-θ)+ylogθ N log likelihood = logL= i 1 (1 yi )log(1-θ)+yi logθ log L N -(1-yi ) yi likelihood equation = 0 + i=1 1-θ θ N N solution: θ i=1 (1-yi ) = (1-θ) i=1 yi N N N Nθ - θ y = i=1 i y -θ i=1 i i=1 yi N yi ˆ = i=1 y MLE N Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 52 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
Exponential Estimating the average time until failure, , of light bulbs. yi = observed life until failure. f(yi| )=(1/ )exp(-yi/ ) L( )=Πi f(yi| )= -N exp(-Σyi/ ) logL ( )=-Nlog ( ) - Σyi/ Likelihood equation: ∂logL( )/∂ =-N/ + Σyi/ 2 =0 Solution: (Multiply both sides of equation by 2) = Σyi /N (sample average estimates population average) Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
Poisson Distribution y exp(- ) exp(- ) y P(y)= = y! Γ(y+1) N N logL=-N + i=1 yi log - i=1 logΓ(y+1)] log L N yi likelihood equation: -N+ i=1 =0 y = 5, 0, 1, 1, 0, 3, 2, 3, 4, 1 Solution: Multiply equation by ; y=2 N yi ˆ= i=1 =y N Sample mean estimates population mean. Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 54 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
Normal Distribution 1 1 (yi )2 f(yi |μ,σ ) = f(yi |μ, ) = 2 exp . is 2 2 2 1 1 11 N log L log 2 log i 1 (yi )2 2 2 2 log L 11 N 1 N i 1 2(yi ) i 1 (yi ) 0 2 Multiply both sides by and solve. ˆ = y. log L 1 1 N (using the solution for ) = (yi y) 2 0 2 2 2 i 1 N (yi y) 2 Multiply both sides by 2 2 and solve ˆ = i 1 N N N (yi y) 2 (yi y) 2 Reclaim the name:  2 = i 1
Math stat 3
Math stat 3
Math stat 3
Math stat 3
Math stat 3
Math stat 3
Math stat 3
Math stat 3
Math stat 3
Math stat 3
Math stat 3
Math stat 3
Math stat 3
Math stat 3
Math stat 3
Math stat 3
Math stat 3
Math stat 3
Math stat 3
Math stat 3
Math stat 3
Math stat 3
Math stat 3
Math stat 3
Math stat 3
Math stat 3
Math stat 3
Math stat 3
Math stat 3
Math stat 3
Math stat 3

More Related Content

PDF
Ddo infoloop
Khellil Khellil
 
PPTX
Payment Instrument Utilization for Specific Transaction Types
NMSU - Family Resource Management
 
PDF
11.15.12 CBIG Event - Kalvin & Vantiv Presentation
Subrata Debnath
 
PDF
Engg.eco sec b_ch01-2
Sachin Paurush
 
PPTX
Music magazine evaluation
JamesBolton
 
PPTX
Music magazine evaluation
JamesBolton
 
PDF
Presentazione comunità finanziaria 10nov11
MondadoriGroup
 
PPTX
Aed303 minilesson
enp5037
 
Ddo infoloop
Khellil Khellil
 
Payment Instrument Utilization for Specific Transaction Types
NMSU - Family Resource Management
 
11.15.12 CBIG Event - Kalvin & Vantiv Presentation
Subrata Debnath
 
Engg.eco sec b_ch01-2
Sachin Paurush
 
Music magazine evaluation
JamesBolton
 
Music magazine evaluation
JamesBolton
 
Presentazione comunità finanziaria 10nov11
MondadoriGroup
 
Aed303 minilesson
enp5037
 

Recently uploaded (20)

PPTX
Cell Types and Its function , kingdom of life
ClaireMangundayao1
 
PDF
Computing-Curriculum for Schools in Ghana
abigailanane203
 
PDF
O5-L3 Freight Transport Ops (International) V1.pdf
labyankof
 
PPTX
human mycosis Human fungal infections are called human mycosis..pptx
shilpa939953
 
PPTX
Final Presentation General Medicine 03-08-2024.pptx
ZaheerAhmad228692
 
PPTX
GDM (1) (1).pptx small presentation for students
shrawanbpkihs
 
PDF
Saundersa Comprehensive Review for the NCLEX-RN Examination.pdf
sreejithcareers
 
PDF
Black Hat USA 2025 - Micro ICS Summit - ICS/OT Threat Landscape
Chris Sistrunk
 
PDF
FourierSeries-QuestionsWithAnswers(Part-A).pdf
Raji Murugan
 
PDF
ANTIBIOTICS.pptx.pdf………………… xxxxxxxxxxxxx
HakHe
 
PDF
Supply Chain Operations Speaking Notes -ICLT Program
labyankof
 
PPTX
Pharmacology of Heart Failure /Pharmacotherapy of CHF
Rajshri Ghogare
 
PDF
grade 11-chemistry_fetena_net_5883.pdf teacher guide for all student
banteamlakmebratu738
 
PPTX
Institutional Correction lecture only . . .
limvtheghins
 
PDF
Anesthesia in Laparoscopic Surgery in India
mohitsuren827
 
PDF
BÀI TẬP BỔ TRỢ 4 KỸ NĂNG TIẾNG ANH 9 GLOBAL SUCCESS - CẢ NĂM - BÁM SÁT FORM Đ...
Nguyen Thanh Tu Collection
 
PDF
Classroom Observation Tools for Teachers
Nicaramirez
 
PDF
O7-L3 Supply Chain Operations - ICLT Program
labyankof
 
PDF
Pre independence Education in Inndia.pdf
goofy5603
 
PPTX
master seminar digital applications in india
ananyaray35
 
Cell Types and Its function , kingdom of life
ClaireMangundayao1
 
Computing-Curriculum for Schools in Ghana
abigailanane203
 
O5-L3 Freight Transport Ops (International) V1.pdf
labyankof
 
human mycosis Human fungal infections are called human mycosis..pptx
shilpa939953
 
Final Presentation General Medicine 03-08-2024.pptx
ZaheerAhmad228692
 
GDM (1) (1).pptx small presentation for students
shrawanbpkihs
 
Saundersa Comprehensive Review for the NCLEX-RN Examination.pdf
sreejithcareers
 
Black Hat USA 2025 - Micro ICS Summit - ICS/OT Threat Landscape
Chris Sistrunk
 
FourierSeries-QuestionsWithAnswers(Part-A).pdf
Raji Murugan
 
ANTIBIOTICS.pptx.pdf………………… xxxxxxxxxxxxx
HakHe
 
Supply Chain Operations Speaking Notes -ICLT Program
labyankof
 
Pharmacology of Heart Failure /Pharmacotherapy of CHF
Rajshri Ghogare
 
grade 11-chemistry_fetena_net_5883.pdf teacher guide for all student
banteamlakmebratu738
 
Institutional Correction lecture only . . .
limvtheghins
 
Anesthesia in Laparoscopic Surgery in India
mohitsuren827
 
BÀI TẬP BỔ TRỢ 4 KỸ NĂNG TIẾNG ANH 9 GLOBAL SUCCESS - CẢ NĂM - BÁM SÁT FORM Đ...
Nguyen Thanh Tu Collection
 
Classroom Observation Tools for Teachers
Nicaramirez
 
O7-L3 Supply Chain Operations - ICLT Program
labyankof
 
Pre independence Education in Inndia.pdf
goofy5603
 
master seminar digital applications in india
ananyaray35
 
Ad

Featured (20)

PDF
2024 Trend Updates: What Really Works In SEO & Content Marketing
Search Engine Journal
 
PDF
Storytelling For The Web: Integrate Storytelling in your Design Process
Chiara Aliotta
 
PDF
Artificial Intelligence, Data and Competition – SCHREPEL – June 2024 OECD dis...
OECD Directorate for Financial and Enterprise Affairs
 
PDF
How to Leverage AI to Boost Employee Wellness - Lydia Di Francesco - SocialHR...
SocialHRCamp
 
PDF
2024 State of Marketing Report – by Hubspot
Marius Sescu
 
PDF
Everything You Need To Know About ChatGPT
Expeed Software
 
PDF
Product Design Trends in 2024 | Teenage Engineerings
Pixeldarts
 
PDF
How Race, Age and Gender Shape Attitudes Towards Mental Health
ThinkNow
 
PDF
AI Trends in Creative Operations 2024 by Artwork Flow.pdf
marketingartwork
 
PDF
Skeleton Culture Code
Skeleton Technologies
 
PDF
PEPSICO Presentation to CAGNY Conference Feb 2024
Neil Kimberley
 
PDF
Content Methodology: A Best Practices Report (Webinar)
contently
 
PPTX
How to Prepare For a Successful Job Search for 2024
Albert Qian
 
PDF
Social Media Marketing Trends 2024 // The Global Indie Insights
Kurio // The Social Media Age(ncy)
 
PDF
Trends In Paid Search: Navigating The Digital Landscape In 2024
Search Engine Journal
 
PDF
5 Public speaking tips from TED - Visualized summary
SpeakerHub
 
PDF
ChatGPT and the Future of Work - Clark Boyd
Clark Boyd
 
PDF
Getting into the tech field. what next
Tessa Mero
 
PDF
Google's Just Not That Into You: Understanding Core Updates & Search Intent
Lily Ray
 
PDF
How to have difficult conversations
Rajiv Jayarajah, MAppComm, ACC
 
2024 Trend Updates: What Really Works In SEO & Content Marketing
Search Engine Journal
 
Storytelling For The Web: Integrate Storytelling in your Design Process
Chiara Aliotta
 
Artificial Intelligence, Data and Competition – SCHREPEL – June 2024 OECD dis...
OECD Directorate for Financial and Enterprise Affairs
 
How to Leverage AI to Boost Employee Wellness - Lydia Di Francesco - SocialHR...
SocialHRCamp
 
2024 State of Marketing Report – by Hubspot
Marius Sescu
 
Everything You Need To Know About ChatGPT
Expeed Software
 
Product Design Trends in 2024 | Teenage Engineerings
Pixeldarts
 
How Race, Age and Gender Shape Attitudes Towards Mental Health
ThinkNow
 
AI Trends in Creative Operations 2024 by Artwork Flow.pdf
marketingartwork
 
Skeleton Culture Code
Skeleton Technologies
 
PEPSICO Presentation to CAGNY Conference Feb 2024
Neil Kimberley
 
Content Methodology: A Best Practices Report (Webinar)
contently
 
How to Prepare For a Successful Job Search for 2024
Albert Qian
 
Social Media Marketing Trends 2024 // The Global Indie Insights
Kurio // The Social Media Age(ncy)
 
Trends In Paid Search: Navigating The Digital Landscape In 2024
Search Engine Journal
 
5 Public speaking tips from TED - Visualized summary
SpeakerHub
 
ChatGPT and the Future of Work - Clark Boyd
Clark Boyd
 
Getting into the tech field. what next
Tessa Mero
 
Google's Just Not That Into You: Understanding Core Updates & Search Intent
Lily Ray
 
How to have difficult conversations
Rajiv Jayarajah, MAppComm, ACC
 
Ad

Math stat 3

  • 1. Statistical Inference and Regression Analysis: GB.3302.30 Professor William Greene Stern School of Business IOMS Department Department of Economics
  • 2. Statistics and Data Analysis Part 3 – Estimation Theory
  • 3. Estimation  Nonparametric population features  Mean - income  Correlation – disease incidence and smoking  Ratio – income per household member  Proportion – proportion of ASCAP music played that is produced by Dave Matthews  Parameters  Fitting distributions – mean and variance of lognormal distribution of income  Parametric models of populations – relationship of loan rates to attributes of minorities and others in Bank of America settlement on mortgage bias Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 3 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
  • 4. Measurements as Observations Population Measurement Theory The theory argues that there are Characteristics meaningful quantities to be statistically Behavior Patterns analyzed. Choices Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
  • 5. Application – Health and Income German Health Care Usage Data, 7,293 Households, Observed 1984-1995 Data downloaded from Journal of Applied Econometrics Archive. Some variables in the file are DOCVIS = number of visits to the doctor in the observation period HOSPVIS = number of visits to a hospital in the observation period HHNINC = household nominal monthly net income in German marks / 10000. (4 observations with income=0 were dropped) HHKIDS = children under age 16 in the household = 1; otherwise = 0 EDUC = years of schooling AGE = age in years PUBLIC = decision to buy public health insurance HSAT = self assessed health status (0,1,…,10) Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
  • 6. Observed Data Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 6 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
  • 7. Inference about Population Population Measurement Characteristics Behavior Patterns Choices Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
  • 8. Classical Inference The population is all 40 million German households (or all households in the entire world). The sample is the 7,293 German households in 1984-1995. Population Sample Measurement Imprecise inference about Characteristics the entire population – Behavior Patterns sampling theory and asymptotics Choices Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
  • 9. Bayesian Inference Population Sample Measurement Sharp, ‘exact’ inference about Characteristics only the sample – the ‘posterior’ density is posterior to the data. Behavior Patterns Choices Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
  • 10. Estimation of Population Features  Estimators and Estimates  Estimator = strategy for use of the data  Estimate = outcome of that strategy  Sampling Distribution and Uncertainty Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 10 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
  • 11. Estimation  Point Estimator: Provides a single estimate of the feature in question based on prior and sample information.  Interval Estimator: Provides a range of values that incorporates both the point estimator and the uncertainty about the ability of the point estimator to find the population feature exactly. Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 11 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
  • 12. „Repeated Sampling‟ - A Sampling Distribution  The true mean is 500. Sample means vary around 500, some quite far off.  The sample mean has a sampling mean and a sampling variance.  The sample mean also has a probability distribution. Looks like a normal distribution. This is a histogram for 1,000 means of samples of 20 observations from Normal[500,1002]. Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
  • 13. Application: Credit Modeling  1992 American Express analysis of  Application process: Acceptance or rejection; X = 0 (reject) or 1 (accept).  Cardholder behavior • Loan default (D = 0 or 1). • Average monthly expenditure (E = $/month) • General credit usage/behavior (Y = number of charges)  13,444 applications in November, 1992 Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
  • 14. X in 100 samples with N = 144 in each sample 0.7809 is the true proportion in the population of 13,444 we are sampling from. Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
  • 15. Estimation Concepts  Random Sampling  Finite populations  i.i.d. sample from an infinite population  Information  Prior  Sample X (X1 , X 2 ,..., X N ) a random sample = a set of 'outside', nonsample information about the population or the feature of interest = a feature of the population of interest ˆ (X , X ,..., X | ) = an estimator of N 1 2 N Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 15 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
  • 16. Properties of Estimators ˆ is a function of a random sample, so it is a random variable N Unbiasedness: E ˆ N Asymptotic Unbiasedness : lim N E ˆN (Usually not useful) Consistency: Plim ˆ N . (Convergence in mean square is usually sufficient.) Efficiency: The 'best' use of the data when there is more than one alternative estimator available. Sampling Distribution: Properties of the estimator used for constructing statistical inference. Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 16 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
  • 17. Unbiasedness The sample mean of the 100 sample estimates is 0.7844. The population mean (true proportion) is 0.7809. Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
  • 18. Consistency N=144 .7 to .88 N=1024 .7 to .88 N=4900 .7 to .88 Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
  • 19. Competing Estimators of a Parameter Di s tri b u ti o n o f Co s t o f 5 0 0 Ba n k s (i n 5 y e a rs ) 232 174 F req u en cy 116 58 0 8. 538 9. 681 10. 824 11. 967 13. 110 14. 253 15. 395 16. 538 C Bank costs are normally distributed with mean . Which is a better estimator of , the mean (11.46) or the median (11.27)? Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 19 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
  • 20. Interval Estimates of the Acceptance Rate Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
  • 21. Methods of Estimation  Information about the source population  Approaches  Method of Moments  Maximum Likelihood  Bayesian Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 21 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
  • 22. The Method of Moments Estimating Parameters of Distributions Using Moment Equations Population Moment k E[xk ] fk ( 1, 2 ,..., K ) Sample Moment 1 N k 1 N mk N i 1 i x --- mk may also be N i 1 k h (xi ), need not be powers Law of Large Numbers plim mk k fk ( 1, 2 ,..., K ) 'Moment Equation' (k = 1,...,K) = sample analog to population. Equate the sample moment to the function of population parameters. 1 N k mk N i 1 i x fk ( 1, 2 ,..., K ) Method of Moments Estimator. Invert the moment equations. ˆ g (m ,...,m ), k = 1,...,K k k 1 K Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
  • 23. Estimating a Parameter  Mean of Poisson  p(y)=exp(-λ) λy / y!, y = 0,1,…; λ > 0  E[y]= λ.  E[(1/N)Σiyi]= λ. This is the estimator  Mean of Exponential  f(y) = exp(- y), y > 0; > 0  E[y] = 1/ .  E(1/N)Σiyi = 1/ .  1/{(1/N)Σiyi } is the estimator of Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
  • 24. Mean and Variance of a Normal Distribution 2 1 1 y f(y) exp 2 2 Population Moments E[y] , E[y 2 ] 2 2 Moment Equations 1 N N i 1 i y , 1 N 2 2 2 N i 1 i y Method of Moments Estimators ˆ =y, ˆ2 1 N N 2 i 1 i y (y 2 ) 1 N N i 1 (y i y)2 Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
  • 25. Gamma Distribution P exp( y)yP 1 f(y) (P) P E[y] P(P 1) E[y 2 ] 2 E[1/ y] P 1 E[log y] (P) log , (P)=dln (P)/dP Any pair of moments can be used to estimate and P. Each pair gives a different answer. Is there a 'best' pair? Yes, the ones that are 'sufficient' statistics. E[y] and E[logy]. Later. Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
  • 26. Proportion for Bernoulli  In the AmEx data, the true population acceptance rate is 0.7809 =  Y = 1 if application accepted, 0 if not.  E[y] =  E[(1/N)Σiyi] = paccept = .  This is the estimator Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 26 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
  • 27. Method of Moments Gamma Distribution Parameters 2 Plot of Psi(P) Function 0 P yi )yP i exp( 1 p(yi ) -2 (P) -4 PSI Population Moments -6 -8 P -10 E[yi ] , E[logy i ] (P) log -12 0 1 2 3 4 5 6 P Moment Equations: N m1( ,P) = E[{(1/N) i=1 i y }] P/ N (P) = (P) / (P) mlog ( ,P) = E[{(1/N) i=1 log yi }] (P) log = dlog (P)/dP Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
  • 28. Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 28 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
  • 29. Estimate One Parameter  Assume known to be 0.1.  Estimate P  E[y] = P/ = P/.1 = 10P  m1 = mean of y = 31.278  Estimate of P is 31.278/10 = 3.1278.  One equation in one unknown Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 29 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
  • 30. Application Moment equations N m1( ,P) = E[{(1/N) i=1 i y }] P/ N 2 2 m2 ( ,P) = E[{(1/N) i=1 i y }] P(P 1) / N mlog ( ,P) = E[{(1/N) i=1 log y i }] (P) log Solving the moment equations using 'least squares:' (This is a convenient approach.) Minimize {m1 E[m1]} 2 {mlog E[m1]} 2 (m1 (P / ))2 (mlog ( (P) log ))2 m1 31.278 mlog 3.221387 Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
  • 31. Method of Moments Solutions create ; y1=y ; y2=log(y) ; ysq=y*y$ calc ; m1=xbr(y1) ; mlog=xbr(y2); m2=xbr(ysq) $ Minimize; start = 2.0, .06 ; labels = p,l ; fcn= (m1 - p/l)^2 + (mlog – (psi(p)-log(l)))^2 $ ------------------------------------------------------------------------------ P| 2.41074 L| .07707 --------+-------------------------------------------------------------------- Minimize; start = 2.0, .06 ; labels = p,l ; fcn= (m1 - p/l)^2 + (m2 – p*(p+1)/l^2 )^2 $ --------+-------------------------------------------------------------------- P| 2.06182 L| .06589 --------+-------------------------------------------------------------------- Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
  • 32. Properties of MoM estimator  Unbiased? Sometimes, e.g., normal, Bernoulli and Poisson means  Consistent? Yes by virtue of Slutsky Theorem  Assumes parameters can vary continuously  Assumes moment functions are continuous and smooth  Efficient? Maybe – remains to be seen. (Which pair of moments should be used for the gamma distribution?)  Sampling distribution? Generally normal by virtue of Lindeberg-Levy central limit theorem and the Slutsky theorem. Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 32 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
  • 33. Estimating Sampling Variance  Exact sampling results – Poisson Mean, Normal Mean and Variance  Approximation based on linearization  Bootstrapping – discussed later with maximum likelihood estimator. Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 33 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
  • 34. Exact Variance of MoM  Estimate normal or Poisson mean  Estimator is sample mean = (1/N) i Yi.  Exact variance of sample mean is 1/N * population variance. Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 34 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
  • 35. Linearization Approach – 1 Parameter THEORY: Variance of the Method of Moments Estimator APPLICATION: Exponential Distribution Distribution is a function of a parameter f(x| ) = exp(- x), > 0, x 0. 1 N E[x]=1/ , Var[x]=1/ 2 Moment equation: E g(x i ) m f ( ) N i1 ˆ = 1/x. Write the equation in the form m( ) m f ( ) m( ) = x - 1/ Linearization: m( ˆ ) m( ) + m'( )( ˆ ) m'( ) = 1/ 2 . 1/m'( ) 2 ˆ m( ˆ ) m( ) m( ) + but m( ˆ ) 0 so ˆ - Var[x] 1 / 2 m'( ) m'( ) Var[m(θ)] Var[x] 2 N N 1 Var[ ˆ ] Var m( ) . 1/ 2 2 m'( ) ˆ Var[θ] 2 2 N N Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 35 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
  • 36. Linearization Approach - General Estimator is derived from moment equations m a - f a ( 1 , 2 ,..., K ) 0 mb - fb ( 1 , 2 ,..., K ) 0 ... 1 N Moments are sample means, m j g j ( yi ), j 1,..., K N i1 There are the same number of moment equations as there are parameters Functions may be powers of x, log of x, 1/x, or other functions. Solutions are ˆ Q(m ,m ,...,m ) etc. j a b K Two items needed to compute the sampling variance Vjk Cov[m j ,m k ] for all pairs arranged in a matrix f a ( 1 , 2 ,..., K ) J = K K matrix of derivatives, 1 , 2 ,..., K The variance is then J -1 VJ -1 . (Requires matrix algebra. Later in the course.) Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 36 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
  • 37. Exercise: Gamma Parameters  m1 = 1/N yi = P/  m2 = 1/N yi2 = P(P+1)/ 2 1. What is the Jacobian? 2. How to compute the variance of m1, the variance of m2 and the covariance of m1 and m2? (The variance of m1 is 1/N times the variance of y; the variance of m2 is 1/N times the variance of y2. The covariance is 1/N times the covariance of y and y2.) Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 37 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
  • 38. Sufficient Statistics Moment equations N m1( ,P) = E[{(1/N) i=1 i y }] P/ N 2 2 m2 ( ,P) = E[{(1/N) i=1 i y }] P(P 1) / N mlog ( ,P) = E[{(1/N) i=1 log y i }] (P) log N m 1( ,P) = E[{(1/N) i=1 (1/ y i )}] / (P 1) Any pair can be used to estimate P and . Is there a best choice? If 'sufficient statistics' exist, the estimator that is a function of them will have a smaller variance than one that is not. Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 38 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
  • 39. Sufficient Statistic x has density f(x| ) where is a parameter. The joint density of a random sample is f(x1 ,x 2 ,...,x N | ) T(x1 ,x 2 ,...,x N ) is a statistic formed from the N observations. If the conditional distribution, f(x1 ,x 2 ,...,x N | T(...), ) is not a function of , then T(...) is a sufficient statistic for . Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 39 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
  • 40. Sufficient Statistic N Bernoulli Trials, X = 0 or 1 with probability i X i = t successes t Prob(X1 =x1 ,...,X N =x N ) = (1 )N t t Prob(X1 =x1 ,...,X N =x N and i X i = t) = (1 )N t N t Prob( i X i = t) = (1 )N t t t (1 )N t 1 Prob(X1 =x1 ,...,X N =x N | i X i = t) = N t N (1 )N t t t Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 40 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
  • 41. Sufficient Statistics Sufficient Statistics only exist for 'Exponential Families' of distributions. f(x i | ) is an exponential family if and only if log f(x i | ) = c( )T(x i ) + d( ) + S(x i ). N Then, i 1 T ( xi ) is the sufficient statistic. When there are K parameters, K log f(x i | 1 ,..., K ) = k 1 ck ( 1 ,..., K )Tk (x i ) + d( 1 ,..., K ) + S(x i ) x x 1 x Bernoulli: f(x| ) (1 (1 ).) 1 c( ) log[ / (1 )], T(x i ) = x i , d( ) log(1 ), S ( x) 0 N N The sufficient statistic is i 1 T(x i ) = i 1 xi Poisson: log f(x| ) = x log - - log x! N N The sufficient statistic is i 1 T(x i ) = i 1 xi Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 41 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
  • 42. Gamma Density P x P 1e x f(x| ,P)= ( P) log f(x i | ,P) P log + (P-1)log x i - x i - log (P) N N The sufficient statistics are m1 = i x and mlog = 1 i i 1 log x i Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 42 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
  • 43. Rao Blackwell Theorem  The mean squared error of an estimator based on sufficient statistics is smaller than one not based on sufficient statistics  We deal in consistent estimators, so a large sample (approximate) version of the theorem is that estimators based on sufficient statistics are more efficient than those that are not. Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 43 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
  • 44. Maximum Likelihood  Estimation Criterion  Comparable to method of moments  Several virtues: Broadly, uses all the sample and nonsample information available  efficient (better than MoM in many cases) Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 44 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
  • 45. Setting Up the MLE The distribution of the observed random variable is written as a function of the parameter(s) to be estimated P(yi| ) = Probability density of data | parameters. L( |yi) = likelihood of parameter | data The likelihood function is constructed from the density Construction: Joint probability density function of the observed sample of data – generally the product when the data are a random sample. The estimator is chosen to maximize the likelihood of the data (essentially the probability of observing the sample in hand). Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
  • 46. Regularity Conditions  What they are  1. logf(.) has three continuous derivatives wrt parameters  2. Conditions needed to obtain expectations of derivatives are met. (E.g., range of the variable is not a function of the parameters.)  3. Third derivative has finite expectation.  What they mean  Moment conditions and convergence. We need to obtain expectations of derivatives.  We need to be able to truncate Taylor series.  We will use central limit theorems  MLE exists for nonregular densities (see text). Questionable statistical properties. Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
  • 47. Regular Exponential Density Exponential density f(yi| )=(1/ )exp(-yi/ ) Average time until failure, , of light bulbs. yi = observed life until failure. Regularity (1) Range of y is 0 to free of (2) logf(yi| ) = -log – y/ ∂logf(yi| )/∂ = -1/ + yi/ 2 E[yi]= , E[∂logf( )/∂ ]=0 (3) ∂2logf(yi| )/∂ 2 = 1/ 2 - 2yi/ 3 finite expectation = -1/ 2 (4) ∂3logf(yi| )/∂ 3 = -2/ 3 + 6yi/ 4 has finite expectation = 4/ 3 (5) All derivatives are continuous functions of Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
  • 48. Likelihood Function  L( )=Πi f(yi| )  MLE = the value of that maximizes the likelihood function.  Generally easier to maximize the log of L. The same maximizes log L  In random sampling, logL= i log f(yi| ) Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 48 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
  • 49. Poisson Likelihood y = 5, 0, 1, 1, 0, 3, 2, 3, 4, 1 y exp(- ) f(y| ) = =Poisson y! exp( ) 5 exp( ) 0 exp( ) 1 Likelihood = ... 5! 0! 1! exp( 10 ) 20 = 207,360 Log likelihood = -10 + 20log - 12.242 Maximum occurs at = 2 log and ln both mean natural log throughout the course Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 49 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
  • 50. The MLE The log-likelihood function: log-L( |data)= Σi logf(yi| ) The likelihood equation(s) = first derivative: First derivatives of log-L equals zero at the MLE. ∂[Σi logf(yi| )]/∂ MLE = 0. (Interchange sum and differentiation) Σi [∂logf(yi| )/∂ MLE]= 0. Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
  • 51. Applications  Bernoulli  Exponential  Poisson  Normal  Gamma Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 51 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
  • 52. Bernoulli f(y|θ)=(1-θ)1-y θ y log f=(1-y)log(1-θ)+ylogθ N log likelihood = logL= i 1 (1 yi )log(1-θ)+yi logθ log L N -(1-yi ) yi likelihood equation = 0 + i=1 1-θ θ N N solution: θ i=1 (1-yi ) = (1-θ) i=1 yi N N N Nθ - θ y = i=1 i y -θ i=1 i i=1 yi N yi ˆ = i=1 y MLE N Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 52 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
  • 53. Exponential Estimating the average time until failure, , of light bulbs. yi = observed life until failure. f(yi| )=(1/ )exp(-yi/ ) L( )=Πi f(yi| )= -N exp(-Σyi/ ) logL ( )=-Nlog ( ) - Σyi/ Likelihood equation: ∂logL( )/∂ =-N/ + Σyi/ 2 =0 Solution: (Multiply both sides of equation by 2) = Σyi /N (sample average estimates population average) Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
  • 54. Poisson Distribution y exp(- ) exp(- ) y P(y)= = y! Γ(y+1) N N logL=-N + i=1 yi log - i=1 logΓ(y+1)] log L N yi likelihood equation: -N+ i=1 =0 y = 5, 0, 1, 1, 0, 3, 2, 3, 4, 1 Solution: Multiply equation by ; y=2 N yi ˆ= i=1 =y N Sample mean estimates population mean. Marginal Plot of Listing vs IncomePC Pie Chart of Percent vs Type Boxplot of Listing Scatterplot of Listing vs IncomePC Probability Plot of Listing Scatterplot of Listing vs IncomePC Histogram of Listing Empirical CDF of Listing Normal - 95% CI Normal C ategory Meatball 900000 900000 900000 14 Pepperoni 99 Mean 369687 Garlic 5.0% Mean 369687 100 2.3% Plain StDev 156865 Mushroom and Onion Pepperoni Mushroom 800000 800000 StDev 156865 800000 9.2% 21.8% 95 N 51 12 N 51 Sausage AD 0.994 80 Pepper and Onion 700000 90 700000 54 700000 P-Value 0.012 Mushroom and Onion 10 Garlic 80 1000000 600000 600000 60 Frequency Pepper and Onion Percent Meatball 70 600000 Listing Listing 7.3% 8 Percent 60 800000 500000 500000 Listing 50 40 Sausage 500000 40 6 Listing 5.8% 400000 30 400000 600000 400000 20 20 300000 300000 4 10 400000 300000 200000 5 200000 2 0 Mushroom Plain 16.2% 200000 32.5% 200000 0 00 00 00 00 00 00 00 00 00 100000 100000 1 0 00 00 00 00 00 00 00 00 00 15000 17500 20000 22500 25000 27500 30000 32500 0 200000 400000 600000 800000 1000000 15000 17500 20000 22500 25000 27500 30000 32500 200000 300000 400000 500000 600000 700000 800000 900000 10 20 30 40 50 60 70 80 90 15000 20000 25000 30000 IncomePC Listing IncomePC Listing Listing IncomePC 100000
  • 55. Normal Distribution 1 1 (yi )2 f(yi |μ,σ ) = f(yi |μ, ) = 2 exp . is 2 2 2 1 1 11 N log L log 2 log i 1 (yi )2 2 2 2 log L 11 N 1 N i 1 2(yi ) i 1 (yi ) 0 2 Multiply both sides by and solve. ˆ = y. log L 1 1 N (using the solution for ) = (yi y) 2 0 2 2 2 i 1 N (yi y) 2 Multiply both sides by 2 2 and solve ˆ = i 1 N N N (yi y) 2 (yi y) 2 Reclaim the name:  2 = i 1