Econometrics ch5
Download as PPT, PDF1 like818 views
This document discusses the classical normal linear regression model (CNLRM) and its assumptions. It states that under the CNLRM, the error terms ui are assumed to be independently and normally distributed with mean 0 and variance σ2. This normality assumption allows us to derive the probability distributions of the OLS estimators βˆ1, βˆ2, and σˆ2. Specifically, it is stated that βˆ1 and βˆ2 are normally distributed with means equal to the true parameter values β1 and β2, and variance inversely proportional to sample size. Ratios of βˆ1 - β1 and βˆ2 - β2 to their standard errors follow standard normal distributions. σˆ2 is chi-
![THE NORMALITY ASSUMPTION FOR ui
• The classicalThe classical normal linear regression model assumes that eachnormal linear regression model assumes that each uuii isis
distributed normally withdistributed normally with
• Mean:Mean: E(uE(uii) = 0) = 0 (4.2.1)(4.2.1)
• Variance:Variance: E[uE[uii − E(u− E(uii)])]22
= Eu= Eu22
ii == σσ22
(4.2.2)(4.2.2)
• cov (cov (uuii, u, ujj):): E{[(uE{[(uii − E(u− E(uii)][u)][ujj − E(u− E(ujj )]} = E(u)]} = E(uii uujj ) = 0 i ≠ j) = 0 i ≠ j (4.2.3)(4.2.3)
• The assumptions given above can be more compactly stated asThe assumptions given above can be more compactly stated as
• uuii N(0,∼ N(0,∼ σσ22
)) (4.2.4)(4.2.4)
• The terms in the parentheses are the mean and the variance.The terms in the parentheses are the mean and the variance. ui and ujui and uj areare
not only uncorrelated but are also independently distributed. Therefore, wenot only uncorrelated but are also independently distributed. Therefore, we
can write (4.2.4) ascan write (4.2.4) as
• uuii NID(0,∼ NID(0,∼ σσ22
)) (4.2.5)(4.2.5)
• where NID stands forwhere NID stands for normally and independently distributed.normally and independently distributed.](https://image.slidesharecdn.com/econometricsch5-160908053042/85/Econometrics-ch5-4-320.jpg)
Econometrics ch5
- 1. 405 ECONOMETRICS Chapter # 4: CLASSICAL NORMAL LINEAR REGRESSION MODEL (CNLRM) Domodar N. Gujarati Prof. M. El-SakkaProf. M. El-Sakka Dept of Economics: Kuwait UniversityDept of Economics: Kuwait University
- 2. • the classical theory of statistical inference consists of two branches, namely,the classical theory of statistical inference consists of two branches, namely, estimationestimation andand hypothesis testinghypothesis testing. We have thus far covered the topic of. We have thus far covered the topic of estimation.estimation. • Under the assumptions of theUnder the assumptions of the CLRM, we were able to show that theCLRM, we were able to show that the estimators of these parametersestimators of these parameters,, βˆβˆ11, βˆ, βˆ22,, andand σˆσˆ22 , satisfy several desirable, satisfy several desirable statisticalstatistical properties, such as unbiasedness, minimum variance, etc. Noteproperties, such as unbiasedness, minimum variance, etc. Note that, since these are estimators, their values will change from sample tothat, since these are estimators, their values will change from sample to sample, they aresample, they are random variablesrandom variables.. • In regression analysis our objective is not only to estimate the SRF, but alsoIn regression analysis our objective is not only to estimate the SRF, but also to use it toto use it to draw inferences about the PRFdraw inferences about the PRF. Thus, we would like to find out. Thus, we would like to find out how closehow close βˆβˆ11 is to the trueis to the true ββ11 or how closeor how close σσˆˆ22 is to the trueis to the true σσ22 .. sincesince βˆβˆ11,, βˆβˆ22, and, and σˆσˆ22 are random variables,are random variables, we need to find out their probability distributionswe need to find out their probability distributions,, otherwise, we willotherwise, we will not be able to relate them to their true values.not be able to relate them to their true values.
- 3. THE PROBABILITY DISTRIBUTION OF DISTURBANCES ui • considerconsider βˆβˆ22. As showed in Appendix 3A.2,. As showed in Appendix 3A.2, • βˆβˆ22 = ∑= ∑ kkiiYYii (4.1.1)(4.1.1) • WhereWhere kkii = x= xii// ∑∑xxii 22 . But since the. But since the X’sX’s are assumed fixed, Eq. (4.1.1) shows thatare assumed fixed, Eq. (4.1.1) shows that βˆβˆ22 is a linear function ofis a linear function of YYii,, which is randomwhich is random by assumption. But sinceby assumption. But since YYii = β= β11 + β+ β22XXii + u+ uii , we can write (4.1.1) as, we can write (4.1.1) as • βˆβˆ22 = ∑= ∑ kkii((ββ11 + β+ β22XXii + u+ uii)) (4.1.2)(4.1.2) • BecauseBecause kkii, the betas, and X, the betas, and Xii are all fixed,are all fixed, βˆβˆ22 is ultimately a linear function ofis ultimately a linear function of uuii,, which is random by assumptionwhich is random by assumption. Therefore, the. Therefore, the probability distribution ofprobability distribution of βˆβˆ22 (and also of(and also of βˆβˆ11) will depend on the assumption) will depend on the assumption made about themade about the probability distribution ofprobability distribution of uuii .. • OLS does not make any assumption about the probabilistic nature ofOLS does not make any assumption about the probabilistic nature of uuii.. This void can be filled if we are willing to assume that theThis void can be filled if we are willing to assume that the u’su’s follow somefollow some probability distribution.probability distribution.
- 4. THE NORMALITY ASSUMPTION FOR ui • The classicalThe classical normal linear regression model assumes that eachnormal linear regression model assumes that each uuii isis distributed normally withdistributed normally with • Mean:Mean: E(uE(uii) = 0) = 0 (4.2.1)(4.2.1) • Variance:Variance: E[uE[uii − E(u− E(uii)])]22 = Eu= Eu22 ii == σσ22 (4.2.2)(4.2.2) • cov (cov (uuii, u, ujj):): E{[(uE{[(uii − E(u− E(uii)][u)][ujj − E(u− E(ujj )]} = E(u)]} = E(uii uujj ) = 0 i ≠ j) = 0 i ≠ j (4.2.3)(4.2.3) • The assumptions given above can be more compactly stated asThe assumptions given above can be more compactly stated as • uuii N(0,∼ N(0,∼ σσ22 )) (4.2.4)(4.2.4) • The terms in the parentheses are the mean and the variance.The terms in the parentheses are the mean and the variance. ui and ujui and uj areare not only uncorrelated but are also independently distributed. Therefore, wenot only uncorrelated but are also independently distributed. Therefore, we can write (4.2.4) ascan write (4.2.4) as • uuii NID(0,∼ NID(0,∼ σσ22 )) (4.2.5)(4.2.5) • where NID stands forwhere NID stands for normally and independently distributed.normally and independently distributed.
- 5. • Why the Normality Assumption? There are several reasons:Why the Normality Assumption? There are several reasons: • 1.1. uuii represent the influencerepresent the influence omitted variables, we hope that the influence ofomitted variables, we hope that the influence of these omitted variablesthese omitted variables is smallis small and atand at best randombest random. By the central limit. By the central limit theorem (CLT) of statistics, it can be shown that if there are a large numbertheorem (CLT) of statistics, it can be shown that if there are a large number of independent and identically distributed random variables, then, theof independent and identically distributed random variables, then, the distribution of their sum tends to a normal distributiondistribution of their sum tends to a normal distribution as the number of suchas the number of such variables increase indefinitely.variables increase indefinitely. • 2. A variant of the CLT states that, even if the number of variables is not2. A variant of the CLT states that, even if the number of variables is not very large or if these variables arevery large or if these variables are not strictly independentnot strictly independent, their, their sum maysum may still be normally distributedstill be normally distributed.. • 3. With the normality assumption, the probability distributions of OLS3. With the normality assumption, the probability distributions of OLS estimators can beestimators can be easily derivedeasily derived because one property of the normalbecause one property of the normal distribution is thatdistribution is that any linear function of normally distributed variablesany linear function of normally distributed variables isis itselfitself normally distributednormally distributed. OLS estimators. OLS estimators βˆβˆ11 andand βˆβˆ22 areare linear functionslinear functions ofof uuii . Therefore, if. Therefore, if uuii are normallyare normally distributed, so aredistributed, so are βˆβˆ11 andand βˆβˆ22, which makes, which makes our task of hypothesisour task of hypothesis testing very straightforward.testing very straightforward.
- 6. • 4. The normal distribution is a comparatively4. The normal distribution is a comparatively simple distributionsimple distribution involvinginvolving only two parameters (mean and variance).only two parameters (mean and variance). • 5. Finally, if we are dealing with a small, or finite, sample size, say data of5. Finally, if we are dealing with a small, or finite, sample size, say data of less than 100 observations,less than 100 observations, the normalitythe normality not only helps us to derive thenot only helps us to derive the exact probability distributions of OLS estimators but also enables us to useexact probability distributions of OLS estimators but also enables us to use thethe t, F, and χ2t, F, and χ2 statistical tests for regression modelsstatistical tests for regression models..
- 7. PROPERTIES OF OLS ESTIMATORS UNDER THE NORMALITY ASSUMPTION • WithWith uuii follow the normal distribution,follow the normal distribution, OLS estimators have the followingOLS estimators have the following properties;.properties;. • 1. They are1. They are unbiasedunbiased.. • 2. They have2. They have minimum varianceminimum variance. Combined with 1., this means that they are. Combined with 1., this means that they are minimum-variance unbiased, orminimum-variance unbiased, or efficient estimatorsefficient estimators.. • 3. They have3. They have consistencyconsistency; that is, as the sample size increases indefinitely,; that is, as the sample size increases indefinitely, the estimatorsthe estimators converge to their true populationconverge to their true population values.values. • 4.4. βˆβˆ11 (being a linear function of(being a linear function of uuii) is normally distributed with) is normally distributed with • Mean:Mean: E(βˆ1) = β1E(βˆ1) = β1 (4.3.1)(4.3.1) • var (var (βˆ1):βˆ1): σσ22 ββˆˆ11 == ((∑∑ XX22 ii/n/n ∑∑ xx22 ii))σσ22 = (3.3.3)= (3.3.3) (4.3.2)(4.3.2) • Or more compactly,Or more compactly, • βˆβˆ11 ∼∼ N (N (ββ11, σ, σ22 ββ ˆˆ11)) • then by the properties of the normal distribution the variablethen by the properties of the normal distribution the variable ZZ, which is, which is • defined as:defined as: • Z = (Z = (βˆβˆ11 − β− β11 )/)/ σβˆσβˆ11 (4.3.3)(4.3.3)
- 8. • follows the standard normal distribution, that is, a normal distribution withfollows the standard normal distribution, that is, a normal distribution with zero mean and unit ( = 1) variance, orzero mean and unit ( = 1) variance, or • Z N(0, 1)∼Z N(0, 1)∼ • 5.5. βˆβˆ22 (being a linear function of(being a linear function of uuii) is normally distributed with) is normally distributed with • Mean:Mean: E(βˆE(βˆ22) = β) = β22 (4.3.4)(4.3.4) • var (var (βˆβˆ22):): σσ22 ββˆˆ22 =σ=σ22 // ∑∑ xx22 ii = (3.3.1)= (3.3.1) (4.3.5)(4.3.5) • Or, more compactly,Or, more compactly, • βˆβˆ22 ∼∼ N(N(ββ22, σ, σ22 ββˆˆ22)) • Then, as in (4.3.3),Then, as in (4.3.3), • Z = (Z = (βˆβˆ22 − β− β22 )/)/σσβˆ2βˆ2 (4.3.6)(4.3.6) • also follows the standard normal distribution.also follows the standard normal distribution. • Geometrically, the probability distributions ofGeometrically, the probability distributions of βˆβˆ11 andand βˆβˆ22 are shown inare shown in Figure 4.1.Figure 4.1.
- 10. • 6.6. ((n− 2)( ˆσn− 2)( ˆσ22 /σ/σ 22 )) is distributed as theis distributed as the χχ22 (chi-square) distribution with(chi-square) distribution with ((n −n − 2)df2)df.. • 7.7. ((βˆβˆ11, βˆ, βˆ22)) are distributed independently ofare distributed independently of σˆσˆ22 .. • 8.8. βˆβˆ11 and βˆand βˆ22 have minimum variance in the entire class of unbiasedhave minimum variance in the entire class of unbiased estimators, whether linear or not. This result, due to Rao, is very powerfulestimators, whether linear or not. This result, due to Rao, is very powerful because,because, unlike the Gauss–Markov theorem, it is not restricted to the classunlike the Gauss–Markov theorem, it is not restricted to the class of linear estimators only. Therefore, we can say that the least-squaresof linear estimators only. Therefore, we can say that the least-squares estimators areestimators are best unbiased estimators (BUEbest unbiased estimators (BUE); that is, they have minimum); that is, they have minimum variancevariance in the entire class of unbiased estimatorsin the entire class of unbiased estimators..
- 11. • To sum up: The important point to note is that the normality assumptionTo sum up: The important point to note is that the normality assumption enables us to derive the probability, or sampling, distributions ofenables us to derive the probability, or sampling, distributions of βˆβˆ11 and βˆand βˆ22 (both normal) and ˆ(both normal) and ˆσσ22 (related to the chi square).(related to the chi square). This simplifies the task ofThis simplifies the task of establishing confidence intervals and testing (statistical) hypotheses.establishing confidence intervals and testing (statistical) hypotheses. • In passing, note that, with the assumption thatIn passing, note that, with the assumption that uuii N(0, σ∼ N(0, σ∼ 22 ), Y), Yii , being a, being a linear function oflinear function of uuii, is itself normally distributed with the mean and variance, is itself normally distributed with the mean and variance given bygiven by • E(YE(Yii)) == ββ11 + β+ β22XXii (4.3.7)(4.3.7) • var (var (YYii)) == σσ22 (4.3.8)(4.3.8) • More neatly, we can writeMore neatly, we can write • YYii N(β∼ N(β∼ 11 + β+ β22XXii , σ, σ22 )) (4.3.9)(4.3.9)