![Statistics for Business and Economics, 6e © 2007 Pearson Education, Inc. Chap 12-9
Least Squares Estimators
§ b0 and b1 are obtained by finding the values
of b0 and b1 that minimize the sum of the
squared differences between y and :
2
i
1
0
i
2
i
i
2
i
)]
x
b
(b
[y
min
)
y
(y
min
e
min
SSE
min
+
-
=
-
=
=
å
å
å
ˆ
ŷ
Differential calculus is used to obtain the
coefficient estimators b0 and b1 that minimize SSE](https://image.slidesharecdn.com/simpleregression-1-230513173309-175a524e/85/simple-regression-1-pdf-9-320.jpg)
![Statistics for Business and Economics, 6e © 2007 Pearson Education, Inc. Chap 12-12
Linear Regression Model
Assumptions
§ The true relationship form is linear (Y is a linear function
of X, plus random error)
§ The error terms, εi are independent of the x values
§ The error terms are random variables with mean 0 and
constant variance, σ2
(the constant variance property is called homoscedasticity)
§ The random error terms, εi, are not correlated with one
another, so that
n)
,
1,
(i
for
σ
]
E[ε
and
0
]
E[ε 2
2
i
i !
=
=
=
j
i
all
for
0
]
ε
E[ε j
i ¹
=](https://image.slidesharecdn.com/simpleregression-1-230513173309-175a524e/85/simple-regression-1-pdf-12-320.jpg)
simple regression-1.pdf
- 1. Chap 12-1 Statistics for Business and Economics, 6e © 2007 Pearson Education, Inc. Chapter 12 Simple Regression Statistics for Business and Economics 6th Edition
- 2. Statistics for Business and Economics, 6e © 2007 Pearson Education, Inc. Chap 12-2 Correlation Analysis § Correlation analysis is used to measure strength of the association (linear relationship) between two variables § Correlation is only concerned with strength of the relationship § No causal effect is implied with correlation § Correlation was first presented in Chapter 3
- 3. Statistics for Business and Economics, 6e © 2007 Pearson Education, Inc. Chap 12-3 Correlation Analysis § The population correlation coefficient is denoted ρ (the Greek letter rho) § The sample correlation coefficient is y x xy s s s r = 1 n ) y )(y x (x s i i xy - - - = å where
- 4. Statistics for Business and Economics, 6e © 2007 Pearson Education, Inc. Chap 12-4 Introduction to Regression Analysis § Regression analysis is used to: § Predict the value of a dependent variable based on the value of at least one independent variable § Explain the impact of changes in an independent variable on the dependent variable Dependent variable: the variable we wish to explain (also called the endogenous variable) Independent variable: the variable used to explain the dependent variable (also called the exogenous variable)
- 5. Statistics for Business and Economics, 6e © 2007 Pearson Education, Inc. Chap 12-5 Linear Regression Model § The relationship between X and Y is described by a linear function § Changes in Y are assumed to be caused by changes in X § Linear regression population equation model § Where b0 and b1 are the population model coefficients and e is a random error term. i i 1 0 i ε x β β Y + + =
- 6. Statistics for Business and Economics, 6e © 2007 Pearson Education, Inc. Chap 12-6 i i 1 0 i ε X β β Y + + = Linear component Simple Linear Regression Model The population regression model: Population Y intercept Population Slope Coefficient Random Error term Dependent Variable Independent Variable Random Error component
- 7. Statistics for Business and Economics, 6e © 2007 Pearson Education, Inc. Chap 12-7 (continued) Random Error for this Xi value Y X Observed Value of Y for Xi Predicted Value of Y for Xi i i 1 0 i ε X β β Y + + = Xi Slope = β1 Intercept = β0 εi Simple Linear Regression Model
- 8. Statistics for Business and Economics, 6e © 2007 Pearson Education, Inc. Chap 12-8 i 1 0 i x b b y + = ˆ The simple linear regression equation provides an estimate of the population regression line Simple Linear Regression Equation Estimate of the regression intercept Estimate of the regression slope Estimated (or predicted) y value for observation i Value of x for observation i The individual random error terms ei have a mean of zero ) ) ˆ ( i 1 0 i i i i x b (b - y y - y e + = =
- 9. Statistics for Business and Economics, 6e © 2007 Pearson Education, Inc. Chap 12-9 Least Squares Estimators § b0 and b1 are obtained by finding the values of b0 and b1 that minimize the sum of the squared differences between y and : 2 i 1 0 i 2 i i 2 i )] x b (b [y min ) y (y min e min SSE min + - = - = = å å å ˆ ŷ Differential calculus is used to obtain the coefficient estimators b0 and b1 that minimize SSE
- 10. Statistics for Business and Economics, 6e © 2007 Pearson Education, Inc. Chap 12-10 § The slope coefficient estimator is § And the constant or y-intercept is § The regression line always goes through the mean x, y X Y xy n 1 i 2 i n 1 i i i 1 s s r ) x (x ) y )(y x (x b = - - - = å å = = x b y b 1 0 - = x x Least Squares Estimators (continued)
- 11. Statistics for Business and Economics, 6e © 2007 Pearson Education, Inc. Chap 12-11 Finding the Least Squares Equation § The coefficients b0 and b1 , and other regression results in this chapter, will be found using a computer § Hand calculations are tedious § Statistical routines are built into Excel § Other statistical analysis software can be used
- 12. Statistics for Business and Economics, 6e © 2007 Pearson Education, Inc. Chap 12-12 Linear Regression Model Assumptions § The true relationship form is linear (Y is a linear function of X, plus random error) § The error terms, εi are independent of the x values § The error terms are random variables with mean 0 and constant variance, σ2 (the constant variance property is called homoscedasticity) § The random error terms, εi, are not correlated with one another, so that n) , 1, (i for σ ] E[ε and 0 ] E[ε 2 2 i i ! = = = j i all for 0 ] ε E[ε j i ¹ =
- 13. Statistics for Business and Economics, 6e © 2007 Pearson Education, Inc. Chap 12-13 § b0 is the estimated average value of y when the value of x is zero (if x = 0 is in the range of observed x values) § b1 is the estimated change in the average value of y as a result of a one-unit change in x Interpretation of the Slope and the Intercept
- 14. Statistics for Business and Economics, 6e © 2007 Pearson Education, Inc. Chap 12-14 Simple Linear Regression Example § A real estate agent wishes to examine the relationship between the selling price of a home and its size (measured in square feet) § A random sample of 10 houses is selected § Dependent variable (Y) = house price in $1000s § Independent variable (X) = square feet
- 15. Statistics for Business and Economics, 6e © 2007 Pearson Education, Inc. Chap 12-15 Sample Data for House Price Model House Price in $1000s (Y) Square Feet (X) 245 1400 312 1600 279 1700 308 1875 199 1100 219 1550 405 2350 324 2450 319 1425 255 1700
- 16. Statistics for Business and Economics, 6e © 2007 Pearson Education, Inc. Chap 12-16 0 50 100 150 200 250 300 350 400 450 0 500 1000 1500 2000 2500 3000 Square Feet House Price ($1000s) Graphical Presentation § House price model: scatter plot
- 17. Statistics for Business and Economics, 6e © 2007 Pearson Education, Inc. Chap 12-17 Regression Using Excel § Tools / Data Analysis / Regression
- 18. Statistics for Business and Economics, 6e © 2007 Pearson Education, Inc. Chap 12-18 Excel Output Regression Statistics Multiple R 0.76211 R Square 0.58082 Adjusted R Square 0.52842 Standard Error 41.33032 Observations 10 ANOVA df SS MS F Significance F Regression 1 18934.9348 18934.9348 11.0848 0.01039 Residual 8 13665.5652 1708.1957 Total 9 32600.5000 Coefficients Standard Error t Stat P-value Lower 95% Upper 95% Intercept 98.24833 58.03348 1.69296 0.12892 -35.57720 232.07386 Square Feet 0.10977 0.03297 3.32938 0.01039 0.03374 0.18580 The regression equation is: feet) (square 0.10977 98.24833 price house + =
- 19. Statistics for Business and Economics, 6e © 2007 Pearson Education, Inc. Chap 12-19 0 50 100 150 200 250 300 350 400 450 0 500 1000 1500 2000 2500 3000 Square Feet House Price ($1000s) Graphical Presentation § House price model: scatter plot and regression line feet) (square 0.10977 98.24833 price house + = Slope = 0.10977 Intercept = 98.248
- 20. Statistics for Business and Economics, 6e © 2007 Pearson Education, Inc. Chap 12-20 Interpretation of the Intercept, b0 § b0 is the estimated average value of Y when the value of X is zero (if X = 0 is in the range of observed X values) § Here, no houses had 0 square feet, so b0 = 98.24833 just indicates that, for houses within the range of sizes observed, $98,248.33 is the portion of the house price not explained by square feet feet) (square 0.10977 98.24833 price house + =
- 21. Statistics for Business and Economics, 6e © 2007 Pearson Education, Inc. Chap 12-21 Interpretation of the Slope Coefficient, b1 § b1 measures the estimated change in the average value of Y as a result of a one- unit change in X § Here, b1 = .10977 tells us that the average value of a house increases by .10977($1000) = $109.77, on average, for each additional one square foot of size feet) (square 0.10977 98.24833 price house + =
- 22. Statistics for Business and Economics, 6e © 2007 Pearson Education, Inc. Chap 12-22 317.85 0) 0.1098(200 98.25 (sq.ft.) 0.1098 98.25 price house = + = + = Predict the price for a house with 2000 square feet: The predicted price for a house with 2000 square feet is 317.85($1,000s) = $317,850 Predictions Using Regression Analysis