Tbs910 regression models

Regression ModelsRegression ModelsRegression ModelsRegression Models
TBS910 BUSINESS ANALYTICSTBS910 BUSINESS ANALYTICS
by
Prof. Stephen Ong
Visiting Professor, Shenzhen
University
Visiting Fellow, Sydney Business
School, University of Wollongong

Today’s OverviewToday’s Overview

Learning ObjectivesLearning Objectives
1.1. Identify variables and use them in a regression model.Identify variables and use them in a regression model.
2.2. Develop simple linear regression equations. from sample data andDevelop simple linear regression equations. from sample data and
interpret the slope and intercept.interpret the slope and intercept.
3.3. Compute the coefficient of determination and the coefficient ofCompute the coefficient of determination and the coefficient of
correlation and interpret their meanings.correlation and interpret their meanings.
4.4. Interpret theInterpret the FF-test in a linear regression model.-test in a linear regression model.
5.5. List the assumptions used in regression and use residual plots toList the assumptions used in regression and use residual plots to
identify problems.identify problems.
6.6. Develop a multiple regression model and use it for predictionDevelop a multiple regression model and use it for prediction
purposes.purposes.
7.7. Use dummy variables to model categorical data.Use dummy variables to model categorical data.
8.8. Determine which variables should be included in a multipleDetermine which variables should be included in a multiple
regression model.regression model.
9.9. Transform a nonlinear function into a linear one for use in regression.Transform a nonlinear function into a linear one for use in regression.
10.10. Understand and avoid common mistakes made in the use ofUnderstand and avoid common mistakes made in the use of
regression analysis.regression analysis.
After completing this lecture, students will be able to:After completing this lecture, students will be able to:

Regression Models : OutlineRegression Models : Outline
4.14.1 IntroductionIntroduction
4.24.2 Scatter DiagramsScatter Diagrams
4.34.3 Simple Linear RegressionSimple Linear Regression
4.44.4 Measuring the Fit of the Regression ModelMeasuring the Fit of the Regression Model
4.54.5 Using Computer Software for RegressionUsing Computer Software for Regression
4.64.6 Assumptions of the Regression ModelAssumptions of the Regression Model
4.74.7 Testing the Model for SignificanceTesting the Model for Significance
4.84.8 Multiple Regression AnalysisMultiple Regression Analysis
4.94.9 Binary or Dummy VariablesBinary or Dummy Variables
4.104.10 Model BuildingModel Building
4.114.11 Nonlinear RegressionNonlinear Regression
4.124.12 Cautions and Pitfalls in Regression AnalysisCautions and Pitfalls in Regression Analysis

5-5
RegressionRegression
AnalysisAnalysis
MultipleMultiple
Moving
Average
Exponential
Smoothing
Trend
Projections
Decomposition
Delphi
Methods
Jury of Executive
Opinion
Sales Force
Composite
Consumer
Market Survey
Time-SeriesTime-Series
MethodsMethods
QualitativeQualitative
ModelsModels
CausalCausal
MethodsMethods
Forecasting ModelsForecasting Models
ForecastingForecasting
TechniquesTechniques
Figure 5.1

IntroductionIntroduction
 Regression analysisRegression analysis is a very valuable toolis a very valuable tool
for a manager.for a manager.
 Regression can be used to:Regression can be used to:
 Understand the relationship between variables.Understand the relationship between variables.
 Predict the value of one variable based onPredict the value of one variable based on
another variable.another variable.
 Simple linear regression models have onlySimple linear regression models have only
two variables.two variables.
 Multiple regression models have moreMultiple regression models have more
variables.variables.

IntroductionIntroduction
 The variable to be predicted is calledThe variable to be predicted is called
thethe dependent variabledependent variable..
 This is sometimes called theThis is sometimes called the responseresponse
variable.variable.
 The value of this variable depends onThe value of this variable depends on
the value of thethe value of the independent variable.independent variable.
 This is sometimes called theThis is sometimes called the explanatoryexplanatory
oror predictor variable.predictor variable.
IndependentIndependent
variablevariable
DependentDependent
variablevariable
IndependentIndependent
variablevariable
= +

4-8
Scatter DiagramScatter Diagram
 AA scatter diagramscatter diagram oror scatter plotscatter plot
is often used to investigate theis often used to investigate the
relationship between variables.relationship between variables.
 The independent variable isThe independent variable is
normally plotted on thenormally plotted on the XX axis.axis.
 The dependent variable isThe dependent variable is
normally plotted on thenormally plotted on the YY axis.axis.

Triple A ConstructionTriple A Construction
 Triple A Construction renovates old homes.Triple A Construction renovates old homes.
 Managers have found that the dollar volumeManagers have found that the dollar volume
of renovation work is dependent on the areaof renovation work is dependent on the area
payroll.payroll.
TRIPLE A’STRIPLE A’S
SALESSALES
($100,000s)($100,000s)
LOCAL PAYROLLLOCAL PAYROLL
($100,000,000s)($100,000,000s)
66 33
88 44
99 66
55 44
4.54.5 22
9.59.5 55
Table 4.1

4-10
Figure 4.1
Scatter Diagram of Triple A Construction Company DataScatter Diagram of Triple A Construction Company Data

Simple Linear RegressionSimple Linear Regression
wherewhere
YY = dependent variable (response)= dependent variable (response)
XX = independent variable (predictor or explanatory)= independent variable (predictor or explanatory)
ββ00 = intercept (value of= intercept (value of YY whenwhen XX = 0)= 0)
ββ11 = slope of the regression line= slope of the regression line
εε = random error= random error
 Regression modelsRegression models are used to test if there isare used to test if there is
a relationship between variables.a relationship between variables.
 There is someThere is some random errorrandom error that cannot bethat cannot be
predicted.predicted.
εββ ++= XY 10

Simple Linear RegressionSimple Linear Regression
 True values for the slope and interceptTrue values for the slope and intercept
are not known so they are estimatedare not known so they are estimated
using sample data.using sample data.
XbbY 10 +=ˆ
wherewhere
YY = predicted value of= predicted value of YY
bb00 = estimate of= estimate of ββ00, based on sample results, based on sample results
bb11 = estimate of= estimate of ββ11, based on sample results, based on sample results
^

Triple A Construction is trying toTriple A Construction is trying to
predict sales based on area payroll.predict sales based on area payroll.
YY = Sales= Sales
XX = Area payroll= Area payroll
The line chosen in Figure 4.1 is the oneThe line chosen in Figure 4.1 is the one
that minimizes the errors.that minimizes the errors.
Error = (Actual value) – (Predicted value)Error = (Actual value) – (Predicted value)
YYe ˆ−=

For the simple linear regression model, theFor the simple linear regression model, the
values of the intercept and slope can bevalues of the intercept and slope can be
calculated using the formulas below.calculated using the formulas below.
XbbY 10 +=ˆ
valuesof(mean)average X
n
X
X ==
∑
valuesof(mean)average Y
n
Y
Y ==
∑
∑
∑
−
−−
= 21
)(
))((
XX
YYXX
b
XbYb 10 −=

YY XX ((XX –– XX))22
((XX –– XX)()(YY –– YY))
66 33 (3 – 4)(3 – 4)22
= 1= 1 (3 – 4)(6 – 7) = 1(3 – 4)(6 – 7) = 1
88 44 (4 – 4)(4 – 4)22
= 0= 0 (4 – 4)(8 – 7) = 0(4 – 4)(8 – 7) = 0
99 66 (6 – 4)(6 – 4)22
= 4= 4 (6 – 4)(9 – 7) = 4(6 – 4)(9 – 7) = 4
55 44 (4 – 4)(4 – 4)22
= 0= 0 (4 – 4)(5 – 7) = 0(4 – 4)(5 – 7) = 0
4.54.5 22 (2 – 4)(2 – 4)22
= 4= 4 (2 – 4)(4.5 – 7) = 5(2 – 4)(4.5 – 7) = 5
9.59.5 55 (5 – 4)(5 – 4)22
= 1= 1 (5 – 4)(9.5 – 7) = 2.5(5 – 4)(9.5 – 7) = 2.5
ΣΣYY = 42= 42
YY = 42/6 = 7= 42/6 = 7
ΣΣXX = 24= 24
XX = 24/6 = 4= 24/6 = 4
ΣΣ((XX –– XX))22
= 10= 10 ΣΣ((XX –– XX)()(YY –– YY)) = 12.5= 12.5
Regression calculations for Triple ARegression calculations for Triple A
ConstructionConstruction

4
6
24
6
===
∑ X
X
7
6
42
6
===
∑Y
Y
251
10
512
21 .
.
)(
))((
==
−
−−
=
∑
∑
XX
YYXX
b
24251710 =−=−= ))(.(XbYb
Regression calculationsRegression calculations
XY 2512 .ˆ +=ThereforeTherefore

4
6
24
6
===
∑ X
X
7
6
42
6
===
∑Y
Y
251
10
512
21 .
.
)(
))((
==
−
−−
=
∑
∑
XX
YYXX
b
24251710 =−=−= ))(.(XbYb
Regression calculationsRegression calculations
XY 2512 .ˆ +=Therefore
sales = 2 + 1.25(payroll)sales = 2 + 1.25(payroll)
If the payroll next year isIf the payroll next year is
$600 million$600 million
000950$or5962512 ,.)(.ˆ =+=Y

Measuring the FitMeasuring the Fit
of the Regression Modelof the Regression Model
 Regression models can be developed forRegression models can be developed for
any variablesany variables XX andand Y.Y.
 How do we know the model is actuallyHow do we know the model is actually
helpful in predictinghelpful in predicting YY based onbased on XX??
 We could just take the average error, but the positive andWe could just take the average error, but the positive and
negative errors would cancel each other out.negative errors would cancel each other out.
 Three measures of variability are:Three measures of variability are:
 SSTSST – Total variability about the mean.– Total variability about the mean.
 SSESSE – Variability about the regression line.– Variability about the regression line.
 SSRSSR – Total variability that is explained by the model.– Total variability that is explained by the model.

 Sum of the squares totalSum of the squares total ::
2
)(∑ −= YYSST
 Sum of the squared errorSum of the squared error::
∑ ∑ −== 22
)ˆ( YYeSSE
 Sum of squares due to regressionSum of squares due to regression::
∑ −= 2
)ˆ( YYSSR
SSESSRSST +=

YY XX ((YY –– YY))22
YY ((YY –– YY))22
((YY –– YY))22
66 33 (6 – 7)(6 – 7)22
= 1= 1 2 + 1.25(3) = 5.752 + 1.25(3) = 5.75 0.06250.0625 1.5631.563
88 44 (8 – 7)(8 – 7)22
= 1= 1 2 + 1.25(4) = 7.002 + 1.25(4) = 7.00 11 00
99 66 (9 – 7)(9 – 7)22
= 4= 4 2 + 1.25(6) = 9.502 + 1.25(6) = 9.50 0.250.25 6.256.25
55 44 (5 – 7)(5 – 7)22
= 4= 4 2 + 1.25(4) = 7.002 + 1.25(4) = 7.00 44 00
4.54.5 22 (4.5 – 7)(4.5 – 7)22
= 6.25= 6.25 2 + 1.25(2) = 4.502 + 1.25(2) = 4.50 00 6.256.25
9.59.5 55 (9.5 – 7)(9.5 – 7)22
= 6.25= 6.25 2 + 1.25(5) = 8.252 + 1.25(5) = 8.25 1.56251.5625 1.5631.563
∑∑((YY –– YY))22
= 22.5= 22.5 ∑∑((YY –– YY))22
= 6.875= 6.875 ∑∑((YY –– YY))22
==
15.62515.625
YY = 7= 7 SSTSST = 22.5= 22.5 SSESSE = 6.875= 6.875 SSRSSR = 15.625= 15.625
^
^^
^^
Table 4.3
Sum of Squares for Triple A ConstructionSum of Squares for Triple A Construction

 Sum of the squares total
2
)(∑ −= YYSST
 Sum of the squared error
∑ ∑ −== 22
)ˆ( YYeSSE
 Sum of squares due to regression
∑ −= 2
)ˆ( YYSSR
 An important relationship
SSESSRSST +=
For Triple A ConstructionFor Triple A Construction
SSTSST = 22.5= 22.5
SSESSE = 6.875= 6.875
SSRSSR = 15.625= 15.625

Figure 4.2
Deviations from the Regression Line and from the MeanDeviations from the Regression Line and from the Mean

Coefficient of DeterminationCoefficient of Determination
 The proportion of the variability inThe proportion of the variability in YY explained byexplained by
the regression equation is called thethe regression equation is called the coefficientcoefficient
of determination.of determination.
 The coefficient of determination isThe coefficient of determination is rr22
..
SST
SSE
SST
SSR
r −== 12
69440
522
625152
.
.
.
==r
 About 69% of the variability inAbout 69% of the variability in YY is explained byis explained by
the equation based on payroll (the equation based on payroll (XX).).

4-24
Correlation CoefficientCorrelation Coefficient
 The correlation coefficient is an expression of the
strength of the linear relationship.
 It will always be between +1 and –1.
 The correlation coefficient is r.
2
rr =
 For Triple A Construction:For Triple A Construction:
8333069440 .. ==r

Four Values of theFour Values of the
Correlation CoefficientCorrelation Coefficient
*
*
*
*
(a)(a) Perfect PositivePerfect Positive
Correlation:Correlation:
rr = +1= +1
X
Y
*
* *
*
(c)(c) NoNo
rr = 0= 0
X
Y
* *
*
*
* *
* **
*
(d)(d)PerfectPerfect
NegativeNegative
rr == ––11
X
Y
*
*
*
*
* *
*
*
*
(b)(b)PositivePositive
0 <0 < rr < 1< 1
X
Y
*
*
*
*
*
*
*
Figure 4.3

4-26
Using Computer Software forUsing Computer Software for
Program 4.1A
Accessing the Regression Option in Excel 2010Accessing the Regression Option in Excel 2010

Program 4.1B
Data Input for Regression in ExcelData Input for Regression in Excel

4-28
Program 4.1C
Excel Output for the Triple A Construction ExampleExcel Output for the Triple A Construction Example

4-29
Assumptions of theAssumptions of the
Regression ModelRegression Model
1.1. Errors are independent.Errors are independent.
2.2. Errors are normally distributed.Errors are normally distributed.
3.3. Errors have a mean of zero.Errors have a mean of zero.
4.4. Errors have a constant variance.Errors have a constant variance.
 If we make certain assumptions about the errors in aIf we make certain assumptions about the errors in a
regression model, we can perform statistical tests toregression model, we can perform statistical tests to
determine if the model is useful.determine if the model is useful.
 A plot of the residuals (errors) willA plot of the residuals (errors) will
often highlight any glaring violationsoften highlight any glaring violations
of the assumption.of the assumption.

Residual PlotsResidual Plots
Pattern of Errors Indicating RandomnessPattern of Errors Indicating Randomness
Figure 4.4A
Error
X

4-31
Nonconstant error varianceNonconstant error variance
Figure 4.4B
Error
X

4-32
Errors Indicate Relationship is not LinearErrors Indicate Relationship is not Linear
Figure 4.4C
Error
X

Estimating the VarianceEstimating the Variance
 Errors are assumed to have a constantErrors are assumed to have a constant
variance (variance (σσ 22
), but we usually don’t know), but we usually don’t know
this.this.
 It can be estimated using theIt can be estimated using the meanmean
squared errorsquared error ((MSEMSE),), ss2.2.
1
2
−−
==
kn
SSE
MSEs
wherewhere
nn = number of observations in the sample= number of observations in the sample
kk = number of independent variables= number of independent variables

4-34
Estimating the VarianceEstimating the Variance
 For Triple A Construction:For Triple A Construction:
71881
4
87506
116
87506
1
2
.
..
==
−−
=
−−
==
kn
SSE
MSEs
 We can estimate the standard deviation,We can estimate the standard deviation, s.s.
 This is also called theThis is also called the standard error of thestandard error of the
estimateestimate or theor the standard deviation of thestandard deviation of the
regression.regression.
31171881 .. === MSEs

4-35
Testing the Model forTesting the Model for
SignificanceSignificance
 When the sample size is too small, youWhen the sample size is too small, you
can get good values forcan get good values for MSEMSE andand rr22
even if there is no relationship betweeneven if there is no relationship between
the variables.the variables.
 Testing the model for significanceTesting the model for significance
helps determine if the values arehelps determine if the values are
meaningful.meaningful.
 We do this by performing a statisticalWe do this by performing a statistical
hypothesis test.hypothesis test.

Testing the Model for SignificanceTesting the Model for Significance
 We start with the general linearWe start with the general linear
modelmodel
εββ ++= XY 10
 IfIf ββ11 = 0, the null hypothesis is that there is= 0, the null hypothesis is that there is
nono relationship betweenrelationship between XX andand Y.Y.
 The alternate hypothesis is that thereThe alternate hypothesis is that there isis aa
linear relationship (linear relationship (ββ11 ≠ 0).≠ 0).
 If the null hypothesis can be rejected, weIf the null hypothesis can be rejected, we
have proven there is a relationship.have proven there is a relationship.
 We use theWe use the FF statistic for this test.statistic for this test.

4-37
Testing the Model forTesting the Model for
SignificanceSignificance
 TheThe FF statistic is based on thestatistic is based on the MSEMSE andand
MSR:MSR:
k
SSR
MSR =
wherewhere
kk == number of independent variables in the modelnumber of independent variables in the model
 TheThe FF statistic is:statistic is:
MSE
MSR
F =
 This describes anThis describes an FF distribution with:distribution with:
degrees of freedom for the numerator =degrees of freedom for the numerator = dfdf11 == kk
degrees of freedom for the denominator =degrees of freedom for the denominator = dfdf22 == nn –– kk – 1– 1

4-38
Testing the Model for SignificanceTesting the Model for Significance
 If there is very little error, theIf there is very little error, the MSEMSE would bewould be
small and thesmall and the FF--statistic would be largestatistic would be large
indicating the model is useful.indicating the model is useful.
 If theIf the FF-statistic is large-statistic is large, the significance, the significance
level (level (pp-value) will be low, indicating it is-value) will be low, indicating it is
unlikely this would have occurred byunlikely this would have occurred by
chance.chance.
 So when theSo when the FF--value is large,value is large, we can rejectwe can reject
the null hypothesisthe null hypothesis and accept that there is aand accept that there is a
linear relationship betweenlinear relationship between XX andand YY and theand the
values of thevalues of the MSEMSE andand rr22
are meaningful.are meaningful.

Steps in a Hypothesis TestSteps in a Hypothesis Test
1.1. Specify null and alternativeSpecify null and alternative
hypotheses:hypotheses: 010 =β:H
011 ≠β:H
2.2. Select the level of significance (Select the level of significance (αα).).
Common values are 0.01 and 0.05.Common values are 0.01 and 0.05.
3.3. Calculate the value of the test statisticCalculate the value of the test statistic
using the formula:using the formula:
MSE
MSR
F =

Steps in a Hypothesis TestSteps in a Hypothesis Test
4.4. Make a decision using one of theMake a decision using one of the
following methods:following methods:
a)a) Reject the null hypothesis if the test statistic is greater thanReject the null hypothesis if the test statistic is greater than
thethe FF-value from the table in Appendix D. Otherwise, do not-value from the table in Appendix D. Otherwise, do not
reject the null hypothesis:reject the null hypothesis:
21
ifReject dfdfcalculated FF ,,α>
kdf =1
12 −−= kndf
b)b) Reject the null hypothesis if the observed significanceReject the null hypothesis if the observed significance
level, orlevel, or pp-value, is less than the level of significance-value, is less than the level of significance
((αα). Otherwise, do not reject the null hypothesis:). Otherwise, do not reject the null hypothesis:
)( statistictestcalculatedvalue- >= FPp
α<value-ifReject p

Step 1.Step 1.
HH00:: ββ11 = 0= 0 (no linear relationship(no linear relationship
betweenbetween XX andand YY))
HH11:: ββ11 ≠ 0≠ 0 (linear relationship exists(linear relationship exists
betweenbetween XX andand YY))
Step 2.Step 2.
SelectSelect αα = 0.05= 0.05
625015
1
625015
.
.
===
k
SSR
MSR
099
71881
625015
.
.
.
===
MSE
MSR
F
Step 3.Step 3.
Calculate the value of the
test statistic.

Step 4.Step 4.
Reject the null hypothesis if the test statistic
is greater than the F-value in Appendix D.
dfdf11 == kk = 1= 1
dfdf22 == nn –– kk – 1 = 6 – 1 – 1 = 4– 1 = 6 – 1 – 1 = 4
The value ofThe value of FF associated with a 5% level ofassociated with a 5% level of
significance and with degrees of freedom 1 and 4 issignificance and with degrees of freedom 1 and 4 is
found in Appendix D.found in Appendix D.
FF0.05,1,40.05,1,4 = 7.71= 7.71
FFcalculatedcalculated = 9.09= 9.09
RejectReject HH00 because 9.09 > 7.71because 9.09 > 7.71

F = 7.71
0.05
9.09
Figure 4.5
 We can conclude there is aWe can conclude there is a
statistically significantstatistically significant
relationshiprelationship betweenbetween XX andand Y.Y.
 TheThe rr22
value of 0.69 means aboutvalue of 0.69 means about
69% of the variability in sales (69% of the variability in sales (YY))
is explained by local payroll (is explained by local payroll (XX).).

4-44
Analysis of VarianceAnalysis of Variance
(ANOVA) Table(ANOVA) Table
 When software is used to develop a regressionWhen software is used to develop a regression
model, anmodel, an ANOVA tableANOVA table is typically created thatis typically created that
shows the observed significance level (shows the observed significance level (pp-value) for-value) for
the calculatedthe calculated FF value.value.
 This can be compared to the level of significanceThis can be compared to the level of significance
((αα) to make a decision.) to make a decision.
DFDF SSSS MSMS FF SIGNIFICANCESIGNIFICANCE
RegressionRegression kk SSRSSR MSRMSR == SSRSSR//kk MSRMSR//MSEMSE PP((FF >>
MSRMSR//MSEMSE))
ResidualResidual nn -- kk - 1- 1 SSESSE MSEMSE ==
SSESSE//((nn -- kk - 1)- 1)
TotalTotal nn - 1- 1 SSTSST
Table 4.4

4-45
ANOVA for Triple A ConstructionANOVA for Triple A Construction
Because this probability is less than 0.05, we rejectBecause this probability is less than 0.05, we reject
the null hypothesis of no linear relationship andthe null hypothesis of no linear relationship and
conclude there is a linear relationship betweenconclude there is a linear relationship between XX
andand Y.Y.
Program 4.1C
(partial)
PP((FF > 9.0909) = 0.0394> 9.0909) = 0.0394

4-46
Multiple Regression AnalysisMultiple Regression Analysis
 Multiple regression models are extensions
to the simple linear model and allow the
creation of models with more than one
independent variable.
YY == ββ00 ++ ββ11XX11 ++ ββ22XX22 + … ++ … + ββkkXXkk ++ εε
wherewhere
YY == dependent variable (response variable)dependent variable (response variable)
XXii == iithth
independent variable (predictor or explanatoryindependent variable (predictor or explanatory
variable)variable)
ββ00 == intercept (value ofintercept (value of YY when allwhen all XXii = 0)= 0)
ββii == coefficient of thecoefficient of the iithth
independent variableindependent variable
kk == number of independent variablesnumber of independent variables

Multiple Regression AnalysisMultiple Regression Analysis
To estimate these values, a sample isTo estimate these values, a sample is
taken the following equation developedtaken the following equation developed
kk XbXbXbbY ++++= ...ˆ 22110
wherewhere
== predicted value ofpredicted value of YY
bb00 == sample intercept (and is an estimate ofsample intercept (and is an estimate of
ββ00))
bbii == sample coefficient of thesample coefficient of the iithth
variable (andvariable (and
is an estimate ofis an estimate of ββii))
Yˆ

4-48
Jenny Wilson RealtyJenny Wilson Realty
Jenny Wilson wants to develop a model toJenny Wilson wants to develop a model to
determine the suggested listing price fordetermine the suggested listing price for
houses based on the size and age of thehouses based on the size and age of the
house.house.
22110
ˆ XbXbbY ++=
wherewhere
== predicted value of dependent variablepredicted value of dependent variable
(selling price)(selling price)
bb00 == YY interceptintercept
XX11 andand XX22 == value of the two independentvalue of the two independent
variables (square footage and age) respectivelyvariables (square footage and age) respectively
bb11 andand bb22 ==slopes forslopes for XX11 andand XX22 respectivelyrespectively
Yˆ
She selects a sample of houses that have soldShe selects a sample of houses that have sold
recently and records the data shown in Table 4.5recently and records the data shown in Table 4.5

4-49
Jenny Wilson Real Estate DataJenny Wilson Real Estate Data
SELLINGSELLING
PRICE ($)PRICE ($)
SQUARESQUARE
FOOTAGEFOOTAGE AGEAGE CONDITIONCONDITION
95,00095,000 1,9261,926 3030 GoodGood
119,000119,000 2,0692,069 4040 ExcellentExcellent
135,000135,000 1,3961,396 1515 GoodGood
142,000142,000 1,7061,706 3232 MintMint
145,000145,000 1,8471,847 3838 MintMint
159,000159,000 1,9501,950 2727 MintMint
182,000182,000 2,2852,285 2626 MintMint
183,000183,000 3,7523,752 3535 GoodGood
200,000200,000 2,3002,300 1818 GoodGood
211,000211,000 2,5252,525 1717 GoodGood
219,000219,000 1,7401,740 1212 MintMint
Table 4.5

4-50
Program 4.2A
Input Screen for the Jenny WilsonInput Screen for the Jenny Wilson
Realty Multiple Regression ExampleRealty Multiple Regression Example

4-51
Program 4.2B
Output for the Jenny Wilson Realty MultipleOutput for the Jenny Wilson Realty Multiple
Regression ExampleRegression Example

Evaluating MultipleEvaluating Multiple
Regression ModelsRegression Models
 Evaluation is similar to simple linearEvaluation is similar to simple linear
regression models.regression models.
 TheThe pp-value for the-value for the FF-test and-test and rr22
areare
interpreted the same.interpreted the same.
 The hypothesis is different because there isThe hypothesis is different because there is
more than one independent variable.more than one independent variable.
 TheThe FF-test is investigating whether all-test is investigating whether all
the coefficients are equal to 0 at the samethe coefficients are equal to 0 at the same
time.time.

Evaluating MultipleEvaluating Multiple
Regression ModelsRegression Models
 To determine which independentTo determine which independent
variables are significant, tests arevariables are significant, tests are
performed for each variable.performed for each variable.
010 =β:H
011 ≠β:H
 The test statistic is calculated and if theThe test statistic is calculated and if the
pp-value is lower than the level of-value is lower than the level of
significance (significance (αα), the null hypothesis is), the null hypothesis is
rejected.rejected.

4-54
 The model is statistically significantThe model is statistically significant
 TheThe pp-value for the-value for the FF-test is 0.002.-test is 0.002.
 rr22
= 0.6719 so the model explains about 67% of the= 0.6719 so the model explains about 67% of the
variation in selling price (variation in selling price (YY).).
 But theBut the FF-test is for the entire model and we can’t tell if-test is for the entire model and we can’t tell if
one or both of the independent variables are significant.one or both of the independent variables are significant.
 By calculating theBy calculating the pp-value of each variable, we can-value of each variable, we can
assess the significance of the individual variables.assess the significance of the individual variables.
 Since the p-value forSince the p-value for XX11 (square footage) and(square footage) and XX22 (age)(age)
are both less than the significance level of 0.05, bothare both less than the significance level of 0.05, both
null hypotheses can be rejected.null hypotheses can be rejected.

Binary or Dummy VariablesBinary or Dummy Variables
 BinaryBinary (or(or dummydummy oror indicatorindicator))
variables are special variablesvariables are special variables
created for qualitative data.created for qualitative data.
 A dummy variable is assigned aA dummy variable is assigned a
value of 1 if a particular conditionvalue of 1 if a particular condition
is met and a value of 0 otherwise.is met and a value of 0 otherwise.
 The number of dummy variablesThe number of dummy variables
must equal one less than themust equal one less than the
number of categories of thenumber of categories of the
qualitative variable.qualitative variable.

4-56
 Jenny believes a better model can beJenny believes a better model can be
developed if she includes informationdeveloped if she includes information
about the condition of the property.about the condition of the property.
XX33 = 1 if house is in excellent condition= 1 if house is in excellent condition
= 0 otherwise= 0 otherwise
XX44 = 1 if house is in mint condition= 1 if house is in mint condition
= 0 otherwise= 0 otherwise
 Two dummy variables are used to describe theTwo dummy variables are used to describe the
three categories of condition.three categories of condition.
 No variable is needed for “good” conditionNo variable is needed for “good” condition
since if bothsince if both XX33 andand XX44 = 0, the house must be in= 0, the house must be in
good condition.good condition.

4-57
Program 4.3A
Input Screen for the Jenny Wilson RealtyInput Screen for the Jenny Wilson Realty
Example with Dummy VariablesExample with Dummy Variables

4-58
Program 4.3B
Output for the Jenny Wilson Realty ExampleOutput for the Jenny Wilson Realty Example
with Dummy Variableswith Dummy Variables

Model BuildingModel Building
 The best model is a statisticallyThe best model is a statistically
significant model with a highsignificant model with a high rr22
andand
few variables.few variables.
 As more variables are added to theAs more variables are added to the
model, themodel, the rr22
-value usually increases.-value usually increases.
 For this reason, theFor this reason, the adjustedadjusted rr22
valuevalue
is often used to determine theis often used to determine the
usefulness of an additional variable.usefulness of an additional variable.
 The adjustedThe adjusted rr22
takes into account thetakes into account the
number of independent variables innumber of independent variables in
the model.the model.

SST
SSE
SST
SSR
−== 12
r
The formula forThe formula for rr22
 The formula for adjustedThe formula for adjusted rr22
)/(SST
)/(SSE
1
1
1Adjusted 2
−
−−
−=
n
kn
r
 As the number of variables increases, theAs the number of variables increases, the
adjustedadjusted rr22
gets smaller unless the increasegets smaller unless the increase
due to the new variable is large enough todue to the new variable is large enough to
offset the change inoffset the change in k.k.

 In general, if a new variable increases theIn general, if a new variable increases the
adjustedadjusted rr22
, it should probably be included in the, it should probably be included in the
model.model.
 In some cases, variables contain duplicateIn some cases, variables contain duplicate
information.information.
 When two independent variables are correlated,When two independent variables are correlated,
they are said to bethey are said to be collinear.collinear.
 When more than two independent variables areWhen more than two independent variables are
correlated,correlated, multicollinearitymulticollinearity exists.exists.
 When multicollinearity is present,When multicollinearity is present, hypothesishypothesis
tests for the individual coefficients are not validtests for the individual coefficients are not valid
but the model may still be useful.but the model may still be useful.

4-62
Nonlinear RegressionNonlinear Regression
 In some situations, variables are notIn some situations, variables are not
linear.linear.
 Transformations may be used to turnTransformations may be used to turn
a nonlinear model into a linear model.a nonlinear model into a linear model.
*
* **
** *
* *
Linear relationshipLinear relationship Nonlinear relationshipNonlinear relationship
**
** **
*
*
**
*

4-63
Colonel MotorsColonel Motors
 Engineers at Colonel Motors want to use
regression analysis to improve fuel efficiency.
 They have been asked to study the impact of
weight on miles per gallon (MPG).
MPGMPG
WEIGHTWEIGHT
(1,000(1,000
LBS.)LBS.) MPGMPG
WEIGHTWEIGHT
(1,000(1,000
LBS.)LBS.)
1212 4.584.58 2020 3.183.18
1313 4.664.66 2323 2.682.68
1515 4.024.02 2424 2.652.65
1818 2.532.53 3333 1.701.70
1919 3.093.09 3636 1.951.95
1919 3.113.11 4242 1.921.92
Table 4.6

4-64
Figure 4.6A
Linear Model for MPG DataLinear Model for MPG Data

4-65
Program 4.4
This is a useful model with a smallThis is a useful model with a small FF-test-test
for significance and a goodfor significance and a good rr22
value.value.
Excel Output for Linear RegressionExcel Output for Linear Regression
Model with MPG DataModel with MPG Data

4-66
Figure 4.6B
Nonlinear Model for MPG DataNonlinear Model for MPG Data

4-67
 The nonlinear model is a quadratic model.The nonlinear model is a quadratic model.
 The easiest way to work with this model isThe easiest way to work with this model is
to develop a new variable.to develop a new variable.
2
2 weight)(=X
 This gives us a model that can beThis gives us a model that can be
solved with linear regression software:solved with linear regression software:
22110 XbXbbY ++=ˆ

4-68
Program 4.5
A better model with a smallerA better model with a smaller FF-test for-test for
significance and a larger adjustedsignificance and a larger adjusted rr22
valuevalue
21 43230879 XXY ...ˆ +−=

Cautions and PitfallsCautions and Pitfalls
 If the assumptions are not met, theIf the assumptions are not met, the
statistical test may not be valid.statistical test may not be valid.
 Correlation does not necessarily meanCorrelation does not necessarily mean
causation.causation.
 Multicollinearity makes interpretingMulticollinearity makes interpreting
coefficients problematic, but the model maycoefficients problematic, but the model may
still be good.still be good.
 Using a regression model beyond the rangeUsing a regression model beyond the range
ofof XX is questionable, as the relationship mayis questionable, as the relationship may
not hold outside the sample data.not hold outside the sample data.

Cautions and PitfallsCautions and Pitfalls
 AA tt-test for the intercept (-test for the intercept (bb00) may be ignored) may be ignored
as this point is often outside the range ofas this point is often outside the range of
the model.the model.
 A linear relationship may not be the bestA linear relationship may not be the best
relationship, even if therelationship, even if the FF-test returns an-test returns an
acceptable value.acceptable value.
 A nonlinear relationship can exist even if aA nonlinear relationship can exist even if a
linear relationship does not.linear relationship does not.
 Even though a relationship is statisticallyEven though a relationship is statistically
significant it may not have any practicalsignificant it may not have any practical
value.value.

TutorialTutorial
Lab Practical : SpreadsheetLab Practical : Spreadsheet
1 - 71

Further ReadingFurther Reading
 Render, B., Stair Jr.,R.M. & Hanna, M.E.
(2013) Quantitative Analysis for
Management, Pearson, 11th
Edition
 Waters, Donald (2007) Quantitative
Methods for Business, Prentice Hall, 4th
Edition.
 Anderson D, Sweeney D, & Williams T.
(2006) Quantitative Methods For
Business Thompson Higher Education,
10th Ed.

Tbs910 regression models

More Related Content

What's hot (20)

Viewers also liked (11)

Similar to Tbs910 regression models (20)

More from Stephen Ong (20)

Recently uploaded (20)

Tbs910 regression models