IBM401 Lecture 5
- 1. Quantitative Analysis for BusinessLecture 5August 9th, 2010
- 2. Multiple Regression AnalysisMultiple regression models are extensions to the simple linear model and allow the creation of models with several independent variablesY = 0 + 1X1 + 2X2 + … + kXk + whereY = dependent variable (response variable)Xi = ith independent variable (predictor or explanatory variable)0 = intercept (value of Y when all Xi= 0)I = coefficient of the ith independent variablek = number of independent variables = random error
- 3. where = predicted value of Yb0 = sample intercept (and is an estimate of 0)bi= sample coefficient of the ith variable (and is an estimate of i)Multiple Regression AnalysisTo estimate these values, a sample is taken the following equation developedInterpreting Multiple Regression ResultsIntercept – the value of the dependent variable when the independent variables are all equal to zeroEach slope coefficient – the estimated change in the dependent variable for a one-unit change in the independent variable, holding the other independent variables contantSometimes it’s called “partial slope coefficients”
- 4. Interpreting example10-year real earnings growth of S&P500 (EG10)Intercept termIf dividend payout ratio (PR) is zero and the slope of the yield curve (YC) is zero, we would expect the subsequent 10-year real earnings growth rate to be -11.6% interceptSlope coefficient of PRIf they payout ratio increases by 1%, we would expect the subsequent 10-year earnings growth rate to increase by 0.25%, holding YC constantSlope coefficient of YCIf the yield curve slope increases by 1%, we would expect the subsequent 10-year earnings growth rate to increase by 0.14%, holding PR constant
- 5. Hypothesis testing of regression coefficientst-statistic – used to test the significance of the individual coefficient in a multiple regressiont-statistic has n-k-1 degrees of freedomEstimated regression coefficient – hypothesized valueCoefficient standard error of bj
- 6. Ex: testing the statistical significance of a regression coefficientTest the statistical significance of the independent variable PR in the real earnings growth example at the 10% significance level. Data based on 46 observations
- 7. Ex: testing the statistical significance of a regression coefficientWe are testing the following hypothesis:The 10% two-tailed critical t-value with 43 degree of freedom (46-2-1) is approximately 1.68We should reject the hypothesis if the t-statistic is greater than 1.68 or less than -1.68Greater than 1.68, we can reject the null hypothesis and conclude that PR regression coefficient is statistically significant a the 10% significant level
- 8. where = predicted value of dependent variable (selling price)b0 = Y interceptX1 and X2 = value of the two independent variables (square footage and age) respectivelyb1 andb2 = slopes for X1 and X2 respectivelyExample: Jenny Wilson RealtyJenny Wilson wants to develop a model to determine the suggested listing price for houses based on the size and age of the house
- 9. She selects a sample of houses that have sold recently and records the data shown in Table 4.5Jenny Wilson RealtyTable 4.5
- 11. Evaluating Multiple Regression ModelsEvaluation is similar to simple linear regression models
- 12. The p-value for the F-test and r2 are interpreted the same
- 13. The hypothesis is different because there is more than one independent variable
- 14. The F-test is investigating whether all the coefficients are equal to 0
- 15. p-value – the smallest level of significance for which the null hypothesis can be rejected
- 16. p-value < significance level
- 18. p-value > significance level
- 19. Null hypothesis cannot be rejectedEvaluating Multiple Regression ModelsTo determine which independent variables are significant, tests are performed for each variable
- 20. The test statistic is calculated and if the p-value is lower than the level of significance (), the null hypothesis is rejectedEx: interpreting p-valuesGiven the following regression results, determine which regression parameters for the independent variables are statistically significantly different from zero at the 1% significant level, assuming the sample size is 60The independent variable is statistically significant if the p-value is less than 1%, or 0.01X1 and X3 are statistically significantly different than zero
- 21. F-statisticF-test assesses how well the set of independent variables, as a group, explains the variation of the dependent variableF-statistic is used to test whether at least one of the independent variables explains a significant portion of the variation of the dependent variable
- 22. F-statisticF-statistic is calculated asWhere:SSR = Sum of Square of RegressionSSE = Sum of Square of ErrorsMSR = Mean Regression Sum of SquaresMSE = Mean Squared ErrorReject H0 if F-statistic > Fc (critical value)
- 23. EX: calculating and interpreting f-statisticAn analyst runs a regression of monthly value-stock returns on five independent variables over 60 months. The total sum of squares is 460, and the sum of squared errors is 170. Test the null hypothesis at the 5% significance level that all five of the independent variables are equal to zeroThe critical F-value for 5 and 54 degrees of freedom at 5% significance level is approximately 2.40
- 24. EX: calculating and interpreting f-statisticThe null and alternative hypothesis areCalculationsF-statistic > F-criticalWe reject null hypothesis!At least one independent variable is significantly different than zero
- 25. EX: Jenny Wilson RealtyThe model is statistically significant
- 26. The p-value for the F-test is 0.002
- 27. r2 = 0.6719 so the model explains about 67% of the variation in selling price (Y)
- 28. But the F-test is for the entire model and we can’t tell if one or both of the independent variables are significant
- 29. By calculating the p-value of each variable, we can assess the significance of the individual variables
- 30. Since the p-value for X1 (square footage) and X2 (age) are both less than the significance level of 0.05, both null hypotheses can be rejectedCoefficient of determination (R2)Multiple coefficient of determination, R2, can be used to test the overall effectiveness of the entire set of independent variables in explaining the dependent variable.
- 31. Adjusted R2Unfortunately, R2 by itself may not be a reliable measure of the multiple regression modelR2almost always increases as variables are added to the modelWe need to take new variables into accountWheren = number of observationsk = number of independent variablesRa2 = adjusted R2
- 32. Adjusted R2Whenever there is more than 1 independent variableRa2 is less than or equal to R2So adding new variables to the model will increase R2 but may increase or decrease the Ra2Ra2 maybe less than 0 if R2 is low enough
- 33. EX: adjusted R2An analyst runs a regression model of monthly value-stock returns on five independent variables over 60 months. The total sum of squares for the regression is 460, and the sum of squared errors is 170. Calculate Ra2 = adjusted R2
- 34. The R2 of 63% suggests that the five independent variables together explain 63% of the variation in monthly value-stock returnsEX: adjusted R2Suppose the analyst now adds four more independent variables to the regression and the R2 increases to 65%, which model the analyst would most likely prefer?
- 35. The analyst would prefer the first model because the adjusted Ra2 is higher and the model has five independent variables as opposed to nineBinary or Dummy VariablesBinary (or dummy or indicator) variables are special variables created for qualitative data
- 36. A dummy variable is assigned a value of 1 if a particular condition is met and a value of 0 otherwise
- 37. The number of dummy variables must equal one less than the number of categories of the qualitative variableJenny Wilson RealtyJenny’s Qualitative Category ConditionExcellent
- 38. Mint
- 39. GoodTable 4.5
- 40. Jenny Wilson RealtyJenny believes a better model can be developed if she includes information about the condition of the propertyX3 = 1 if house is in excellent condition = 0 otherwiseX4 = 1 if house is in mint condition = 0 otherwiseTwo dummy variables are used to describe the three categories of condition
- 41. No variable is needed for “good” condition since if both X3 and X4 = 0, the house must be in good conditionJenny Wilson Realty
- 42. Model explains about 90% of the variation in selling priceF-value indicates significanceLow p-values indicate each variable is significantJenny Wilson RealtyProgram 4.3
- 43. Model BuildingThe best model is a statistically significant model with a high r2and few variables
- 44. As more variables are added to the model, the r2-value usually increases
- 45. For this reason, the adjusted r2 value is often used to determine the usefulness of an additional variable
- 46. The adjusted r2 takes into account the number of independent variables in the modelThe formula for r2
- 47. The formula for adjusted r2Model BuildingAs the number of variables increases, the adjusted r2 gets smaller unless the increase due to the new variable is large enough to offset the change in kModel BuildingIn general, if a new variable increases the adjusted r2, it should probably be included in the model
- 48. In some cases, variables contain duplicate information
- 49. When two independent variables are correlated, they are said to be collinear
- 50. When more than two independent variables are correlated, multicollinearity exists
- 51. When multicollinearity is present, hypothesis tests for the individual coefficients are not valid but the model may still be useful********************Linear relationshipNonlinear relationshipNonlinear RegressionIn some situations, variables are not linearTransformations may be used to turn a nonlinear model into a linear model
- 52. Ex: Colonel MotorsThe engineers want to use regression analysis to improve fuel efficiency
- 53. They have been asked to study the impact of weight on miles per gallon (MPG) Table 4.6
- 54. 45 – 40 – 35 – 30 – 25 – 20 – 15 – 10 – 5 – 0 – MPG| | | | | 1.00 2.00 3.00 4.00 5.00Weight (1,000 lb.)Colonel MotorsLinear modelFigure 4.6A
- 55. Colonel MotorsA useful model with a small F-test for significance and a good r2 value45 – 40 – 35 – 30 – 25 – 20 – 15 – 10 – 5 – 0 – MPG| | | | | 1.00 2.00 3.00 4.00 5.00Weight (1,000 lb.)Colonel MotorsNonlinear modelFigure 4.6B
- 56. The nonlinear model is a quadratic model
- 57. The easiest way to work with this model is to develop a new variable
- 58. This gives us a model that can be solved with linear regression softwareColonel Motors
- 59. Colonel MotorsA better model with a smaller F-test for significance and a larger adjusted r2 valueCautions and PitfallsIf the assumptions are not met, the statistical test may not be validCorrelation does not necessarily mean causationMulticollinearity makes interpreting coefficients problematic, but the model may still be goodUsing a regression model beyond the range of X is questionable, the relationship may not hold outside the sample data
- 60. Cautions and Pitfallst-tests for the intercept (b0) may be ignored as this point is often outside the range of the modelA linear relationship may not be the best relationship, even if the F-test returns an acceptable valueA nonlinear relationship can exist even if a linear relationship does notJust because a relationship is statistically significant doesn't mean it has any practical value