Lecture slides stats1.13.l20.air
- 1. Statistics One Lecture 20 Binary Logistic Regression 1
- 2. Two segments • Overview • Example 2
- 3. Lecture 20 ~ Segment 1 Binary Logistic Regression Overview 3
- 4. Binary logistic regression • Appropriate when predicting a binary categorical outcome variable from a set of predictor variables that may be continuous and/or categorical – Same logic as multiple regression but outcome variable is categorical and binary
- 5. Binary logistic regression • When outcome has two levels – Binary logistic regression • When outcome has multiple levels – Multinomial regression
- 6. Multiple regression • Ŷ= B0 + Σ(BkXk) Ŷ = predicted value on the outcome variable Y B0 = predicted value on Y when all X = 0 Xk = predictor variables Bk = unstandardized regression coefficients (Y – Ŷ) = residual (prediction error) k = the number of predictor variables 6
- 7. Binary logistic regression • ln(Ŷ / (1 - Ŷ)) = B0 + Σ(BkXk) Ŷ = predicted value on the outcome variable Y B0 = predicted value on Y when all X = 0 Xk = predictor variables Bk = unstandardized regression coefficients (Y – Ŷ) = residual (prediction error) k = the number of predictor variables 7
- 8. Binary logistic regression • Why ln(Ŷ / (1 – Ŷ))? • Predicted score must fall between 0 and 1 8
- 10. Binary logistic regression • Why not P(outcome) = B0 + Σ(BkXk) ??? • There is no guarantee that the linear combination of predictors will produce a score between 0 and 1 • A transformation is therefore applied 10
- 11. Binary logistic regression • Odds = P(outcome) / (1 – P(outcome)) • For example, what are the odds a flipped coin will land heads? Odds = .5 / .5 = 1 • Then take the natural log of the odds, which is called the log-odds or logit • Logit = ln(P(outcome) / (1 – P(outcome)) • Logit = ln(Ŷ / (1 – Ŷ)) 11
- 12. Binary logistic regression • Logit = ln(Ŷ / (1 – Ŷ)) • Ŷ = P(outcome)
- 13. Binary logistic regression • P(outcome) = odds / (1 + odds) • Odds = P(outcome) / P(~outcome) • For example, • If P = .50 then Odds = 1 and Logit = 0
- 14. Binary logistic regression • Example • Outcome variable = Faculty Promotion to tenure • Predictor variable = Publications (Pubs) • Logit(Promotion) = B0 + B1(Pubs) • Logit(Promotion) = 0.00 + .39(Pubs) • For every one unit increase in Pubs, the Logit increases .39
- 15. Binary logistic regression • Logit = ln(P(outcome) / (1 – P(outcome)) • Odds = P(outcome) / (1 – P(outcome)) • Logit = .39 translates to an odds ratio of 1.48 – This means that the odds of promotion are multiplied by 1.48 for each increment in Pubs
- 16. Binary logistic regression • Thus, if the odds of Promotion with 16 publications is 1.27 then the Odds of Promotion with 17 publications is 1.27*1.48 = 1.88 • This can also be presented in terms of probability • Pubs = 17 means P(Promotion) = .65 because P(Promotion) = Odds / (1 + Odds) = 1.88/2.88 = .65
- 17. Binary logistic regression • Hypothesis tests • Is an individual predictor variable significant? • Is the overall model significant? • Is Model A significantly better than Model B?
- 18. Binary logistic regression • To test each predictor variable • Regression coefficient • Odds ratio • Wald test • Tests the model vs. the model without the predictor
- 19. Binary logistic regression • To test the overall model • Compare the chi-square for the model to the chi-square of a model with no predictors (the null model) • And/or compare multiple models • Also, does the model classify cases correctly?
- 20. Segment summary • Binary logistic regression is appropriate when predicting a binary categorical outcome variable from a set of predictor variables that may be continuous and/or categorical
- 21. Segment summary • Main components of the output are – Regression coefficients – Odds ratios – Wald tests – Model chi-square – Classification success
- 22. END SEGMENT 22
- 23. Lecture 20 ~ Segment 2 Binary Logistic Regression Example 23
- 24. Binary logistic regression • This example is based on “mock jury” research by Diamond & Casper (1992) – People (mock jurors) watched a video of the sentencing phase of a murder trial in which the defendant had already been found guilty – The issue for the jurors to decide was whether the defendant deserved the death penalty
- 25. Binary logistic regression • This example is based on “mock jury” research by Diamond & Casper (1992) – Assume the data were collected “pre-deliberation”, which means that each juror was asked to provide his or her vote on the death penalty verdict before the jurors met as a group to decide the overall jury verdict
- 26. Binary logistic regression • Outcome variable (Y) • Verdict • 1 = Voted for the death penalty • 0 = Voted against the death penalty • Predictors (Xs) • • • • • • Danger Rehab Punish Gendet Specdet Incap • All measured on a scale of 0 – 10
- 27. Binary logistic regression • Danger (Dangerousness) • Individual’s beliefs as to the future dangerousness of the defendant • Rehab (Rehabilitation) • Individual’s beliefs as to the importance of rehabilitation as a goal of criminal sentencing • Punish (Punishment) • Individual’s beliefs as to the importance of punishment as a goal of criminal sentencing
- 28. Binary logistic regression • Gendet (General deterrence) • Individual’s beliefs as to the importance of general deterrence as a goal of criminal sentencing (sentencing should deter the general public) • Specdet (Specific deterrence) • Individual’s beliefs as to the importance of specific deterrence as a goal of criminal sentencing (sentencing should deter the specific defendant) • Incap (Incapacitation) • Individual’s beliefs as to the importance of punishment as a goal of criminal sentencing
- 29. Binary logistic regression • The General Linear Model will not guarantee a predicted outcome score between 0 and 1 • The Logit transformation is a feature of an even more “general” mathematical framework in regression • The Generalized Linear Model • Allows for non-linear relationships between predictors and the outcome variable (see Lecture 23)
- 37. Binary logistic regression • Evaluation of individual predictors – Odds ratios • For a one unit increase in X, the predicted change in odds • Can also report confidence intervals for odds – Wald test • A function of the regression coefficient. A Wald tests is calculated for each predictor variable and compares the fit of the model to the fit of the model without the predictor.
- 38. Binary logistic regression • Evaluation of the model – Model chi-square – Compares the fit of the model to the fit of the null model – Classification success • Percentage of cases classified correctly
- 39. Binary logistic regression • More than 2 categories on the outcome – Multinomial logistic regression • A-1 logistic regression equations are formed – Where A = # of groups – One group serves as reference group
- 40. END SEGMENT 40