Logistic regression with low event rate (rare events)
18 likes23,670 views
The document discusses issues with using logistic regression for datasets with rare events and low event rates, providing alternatives like exact logistic regression and penalized likelihood estimation. It provides examples of scenarios with rare events, explains why traditional logistic regression is biased for these cases, and gives steps to implement exact logistic regression in SAS and penalized likelihood regression in R to address the issues.
Logistic regression with low event rate (rare events)
- 1. 31 August 2021 ©TejamoyGhosh – Data Science ATG - New Delhi, India
- 2. Problem with logistic regression with low event rate Way out How to do them in SAS? How to do them in R? 31 August 2021 ©TejamoyGhosh – Data Science ATG - New Delhi, India
- 3. Analyst 1: I’m in some trouble, my manager wants me to build a logistic regression model but I have only a 2% event rate in my data. The logistic regression won’t be a good choice here – the ML estimate will be biased. Analyst 2: Not necessarily. It’s the total count rather than the percentage of events that matters. How many cases do you have for the rarer event and how big is your dataset? Analyst1: We’ve got about 1800 odd events in a dataset of about 100,000 cases a less than 2% scenario Analyst2: Hmm. With these many cases for the rarer event, you can very well use logistic regression. There are methods to address such skewed, or sparse data situations. Analyst1: Wow. Really! Please tell me more!! Analyst2: There are couple of alternatives. For one you can use ‘exact’ logistic regression – this is to be used when sample size is too small for your usual logistic regression using the regular maximum-likelihood-based estimation. Another option in your scenario is to use the penalized- likelihood estimation method. This second one has the advantage of being computationally less demanding than the exact logistic method. 31 August 2021 ©TejamoyGhosh – Data Science ATG - New Delhi, India
- 4. Low event rate/Rare Event: In the current context, this refers to the scenario where under a binary outcome space (response/no-response, good/bad, default/no-default, purchase/no-purchase, etc.) one of the two events are far fewer than the other ▪ Suppose in a sample of 1000 applicants for a position only 20 are selected – here the event of being selected is the rare event with a low event rate of 2% ▪ Suppose, in a sample of 100,000 purchases from an online retailer, about 1800 are returned by the customer – here the event of goods being returned is the rare event with a low event rate of 1.8% Some real life examples: ▪ Charge backs in credit card transactions ▪ Goods returned in online retailing Why is this a problem for logistic regression – it’s still binary anyway? The problem here is with the estimation method – the usual maximum-likelihood method is susceptible to ‘small sample bias’ and this bias is strongly dependent on the count (as opposed to percentage) of the rarer of the events 31 August 2021 ©TejamoyGhosh – Data Science ATG - New Delhi, India
- 5. In case of small sample and/or very unbalanced binary data (When you have just 20 cases in a sample of 1000) – ‘exact logistic’ regression is to be used Exact logistic regression approach provides an alternative to the maximum likelihood method for making inferences about the parameters of the logistic regression model The method is based on appropriate exact distributions of sufficient statistics for parameters of interest and the estimates given by exact logistic regression do not depend on asymptotic results It is useful for analyzing small or unbalanced binary data with covariates This method is usually very computationally intensive If, however, you have a larger count of the rarer of the two events, say, 1000, (even better if it’s 2000) in a sample of 100,000 with the same low event rate (1% to 2%) you can use logistic regression – the estimation will have to be done using ‘penalized likelihood’ method (also called Firth’s penalized likelihood approach, after its inventor While we mentioned this method in the context of only small sample size/rare event scenario, this is a method of addressing issues of separability, small sample sizes, and bias of the parameter estimates 31 August 2021 ©TejamoyGhosh – Data Science ATG - New Delhi, India
- 6. 31 August 2021 ©TejamoyGhosh – Data Science ATG - New Delhi, India
- 7. Proc Logistic Data = YourRareEventData descending; Freq CellCount; /* the CellCount variable is weight vbl here */ model RareEvent = X1 X2; Exact X1 / estimate = both; Run; You can add other options for what you want to have in your output The option Exact after the model statement and the Freq statements are the key differences here An alternative Event/Trial Syntax: Proc Logistic Data = YourRareEventData; model RareEvent / CellCount = X1 X2; Exact X1 / estimate = both; Run; 31 August 2021 ©TejamoyGhosh – Data Science ATG - New Delhi, India
- 8. Proc Logistic Data = YourRareEventData; class CategoricalVbl1 CategoricalVbl2/ param=ref; Model Y = CategoricalVbl1 CategoricalVbl2 X1 X1 / firth ; run; You can add other options for what you want to have in your output The option ‘FIRTH’ in the model statement is the key here 31 August 2021 ©TejamoyGhosh – Data Science ATG - New Delhi, India
- 9. 31 August 2021 ©TejamoyGhosh – Data Science ATG - New Delhi, India
- 10. Package Required: ‘elrm’ This package implements (approximate) exact inference for binomial logistic regression models in R 31 August 2021 ©TejamoyGhosh – Data Science ATG - New Delhi, India
- 11. Package: ‘logistf’ This package runs Firth’s bias reduced logistic regression approach with penalized profile likelihood based confidence intervals for parameter estimates Another package ‘penalized’ runs penalized generalized linear models, penalized regression models 31 August 2021 ©Arup Guha - Indian Institute of ForeignTrade - New Delhi, India
- 12. Data Sciences ATG EDUCATION Econometrics, Statistics, Economics Vanderbilt, Cincinnati, Indian Statistical Institute, Jawaharlal Nehru University Research Scholars Journal Articles Free Solutions to Challenging Data Problems EXPERIENCE 18 years combined, Marketing analytics, Risk analytics, Financial analytics, Analytic Solution & Tools development, Analytics CoE set-up, Advanced Analytics Training EXISTING/SERVED CLIENTS A large Global Beverage company, A small insurance company, A renowned business school, A large Global HR & Compensation Consulting Group, A large Global IT Research group, A third party analytics vendor, A mid sized analytics consulting EXPERTISE Predictive modelling, Segmentation, Market research, Clickstream data analysis, Forecasting, Financial Time Series, Simulation, Bayesian econometrics, Machine Learning Techniques, Decision Trees, SAS, SPSS, R, Octave, Stata, Eviews, Matlab, Maxima, Netlogo What we don’t do… Quick and dirty back of the envelope calculation Use jargon presentations with little impact on your problem Hide that we are stumped What we do… FREE analytics help to stuck analysts and consultants Customized analytics solutions to institutes and companies FREE snapshot to companies considering entering analytics Apply analytics in non-traditional areas including films & education FREE data analysis help to students researchers and faculty