NG BB 36 Simple Linear Regression
4 likes3,371 views
This document provides an overview of simple linear regression modeling. It defines key regression terminology like independent and dependent variables. It explains how to visualize relationships between variables using scatter plots and how to calculate correlation coefficients. While correlation does not necessarily imply causation, regression can be used to generate prediction models. The document reviews best practices like planning data collection and model validation. It provides examples of applying regression in administrative, market research, and hospitality contexts.
NG BB 36 Simple Linear Regression
- 1. UNCLASSIFIED / FOUO UNCLASSIFIED / FOUO National Guard Black Belt Training Module 36 Simple Linear Regression UNCLASSIFIED / FOUO This material is not for general distribution, and its contents should not be quoted, extracted for publication, or otherwise copied or distributed without prior coordination with the Department of the Army, ATTN: ETF. UNCLASSIFIED / FOUO
- 2. UNCLASSIFIED / FOUO CPI Roadmap – Analyze 8-STEP PROCESS 6. See 1.Validate 2. Identify 3. Set 4. Determine 5. Develop 7. Confirm 8. Standardize Counter- the Performance Improvement Root Counter- Results Successful Measures Problem Gaps Targets Cause Measures & Process Processes Through Define Measure Analyze Improve Control ACTIVITIES TOOLS • Value Stream Analysis • Identify Potential Root Causes • Process Constraint ID • Reduce List of Potential Root • Takt Time Analysis Causes • Cause and Effect Analysis • Brainstorming • Confirm Root Cause to Output • 5 Whys Relationship • Affinity Diagram • Estimate Impact of Root Causes • Pareto on Key Outputs • Cause and Effect Matrix • FMEA • Prioritize Root Causes • Hypothesis Tests • Complete Analyze Tollgate • ANOVA • Chi Square • Simple and Multiple Regression Note: Activities and tools vary by project. Lists provided here are not necessarily all-inclusive. UNCLASSIFIED / FOUO
- 3. UNCLASSIFIED / FOUO Learning Objectives Terminology and data requirements for conducting a regression analysis Interpretation and use of scatter plots Interpretation and use of correlation coefficients The difference between correlation and causation How to generate, interpret, and use regression equations Simple Linear Regression UNCLASSIFIED / FOUO 3
- 4. UNCLASSIFIED / FOUO Application Examples Administrative – A financial analyst wants to predict the cash needed to support growth and increases in training Market/Customer Research – The main exchange wants to determine how to predict a customer’s buying decision from demographics and product characteristics Hospitality – The MWR Guest House wants to see if there is a relationship between room service delays and order size Simple Linear Regression UNCLASSIFIED / FOUO 4
- 5. UNCLASSIFIED / FOUO When Should I Use Regression? Independent Variable (X) Continuous Attribute Continuous Dependent Variable (Y) Regression ANOVA Attribute Logistic Chi-Square (2) Regression Test The tool depends on the data type. Regression is typically used with a continuous input and a continuous response but can also be used with count or categorical inputs and outputs. Simple Linear Regression UNCLASSIFIED / FOUO 5
- 6. UNCLASSIFIED / FOUO General Strategy for Regression Modeling Planning and • What variables? Data Collection • How will I get the data? • How much data do I need? Initial Analysis and • What input variables have the biggest Reduction of Variables effect on the response variable? • What are some candidate prediction models? Select and Refine • What is the best model? Models Validate • How well does the model predict new Model observations? Simple Linear Regression UNCLASSIFIED / FOUO 6
- 7. UNCLASSIFIED / FOUO Regression Terminology Types of Variables Input Variable (Xs) These are also called predictor variables or independent variables Best if the variables are continuous, Error but can be count or categorical X1 Output Variable (Ys) Process or X2 Y These are also called response Product X3 variables or dependent variables (what we’re trying to predict) Best if the variables are continuous, but can be count or categorical Simple Linear Regression UNCLASSIFIED / FOUO 7
- 8. UNCLASSIFIED / FOUO Visualize the Data – A Good Start! Scatter Plot: A graph showing a relationship (or correlation) between two factors or variables Lets you “see” patterns in data Supports or refutes theories about the data Helps create or refine hypotheses Predicts effects under other circumstances (be careful extending predictions beyond the range of data used) Be Careful Correlation does not guarantee causation! Simple Linear Regression UNCLASSIFIED / FOUO 8
- 9. UNCLASSIFIED / FOUO Correlation vs. Causation Correlation by itself does not imply a cause and effect relationship! Other examples? Average life expectancy Gas mileage # divorces/10,000 Price of automobiles Lurking variables! When is it correct to infer causation? Simple Linear Regression UNCLASSIFIED / FOUO 9
- 10. UNCLASSIFIED / FOUO Example: Mortgage Estimates A Belt is trying to reduce the call length for military clients calling for a good faith estimate on a VA loan The Belt thinks that there is a relationship between broker experience and call length, and creates a scatter plot to visualize the relationship Simple Linear Regression UNCLASSIFIED / FOUO 10
- 11. UNCLASSIFIED / FOUO Example: Mortgage Estimate Scatter Plot Hypothesis: Brokers with more experience can provide estimates in a shorter time. 60 50 Call Length 40 30 20 10 20 30 Broker Experience Does it look like a relationship exists between Broker Experience and Call Length? Simple Linear Regression UNCLASSIFIED / FOUO 11
- 12. UNCLASSIFIED / FOUO Scatter Plot - Structure Y Axis 60 Paired (Result?) Data 50 Call Length 40 X Axis 30 ( Suspected Influence ) 20 10 20 30 Broker Experience Paired Data? To use a scatter plot, you must have measured two factors for a single observation or item (ex: for a given measurement, you need to know both the call length and the broker’s experience). You have to make sure that the data “pair-up” properly in Minitab, or the diagram will be meaningless. Simple Linear Regression UNCLASSIFIED / FOUO 12
- 13. UNCLASSIFIED / FOUO Input, Process, Output Context PREDICTOR MEASURES RESULTS MEASURES Y (X) (X) (Y) Input Process Output • Arrival • Customer Time Satisfaction • Accuracy • Total • Cost Defects • Key Specs • Cycle Time • Cost • Time Per Task • In-Process Errors • Labor Hours • Exceptions X Axis – Y Axis – Independent Variable Dependent Variable X Simple Linear Regression UNCLASSIFIED / FOUO 13
- 14. UNCLASSIFIED / FOUO Scatter Plots No Correlation Negative Curvilinear Positive See how one factor relates to changes in another Develop and/or verify hypotheses Judge strength of relationship by width or tightness of scatter Don’t assume a causal relationship! Simple Linear Regression UNCLASSIFIED / FOUO 14
- 15. UNCLASSIFIED / FOUO Exercise: Interpreting Scatter Plots 1. As a team, review assigned Scatter Plots – see next pages 2. What kind of correlation do you see? (Name) 3. What does it mean? 4. What can you conclude? 5. What data might this represent? (Example) Simple Linear Regression UNCLASSIFIED / FOUO 15
- 16. UNCLASSIFIED / FOUO Example One Simple Linear Regression UNCLASSIFIED / FOUO 16
- 17. UNCLASSIFIED / FOUO Example Two Simple Linear Regression UNCLASSIFIED / FOUO 17
- 18. UNCLASSIFIED / FOUO Example Three Simple Linear Regression UNCLASSIFIED / FOUO 18
- 19. UNCLASSIFIED / FOUO Minitab Example: Scatter Plot Next, we will work through a Minitab example using data collected at the Anthony’s Pizza company The Belt suspects that the customers have to wait too long on days when there are many deliveries to make at Anthony’s Pizza Simple Linear Regression UNCLASSIFIED / FOUO 19
- 20. UNCLASSIFIED / FOUO Minitab Example: Pizza Scatter Plot A month of data was collected, and stored in the Minitab file Regression-Pizza.mtw Simple Linear Regression UNCLASSIFIED / FOUO 20
- 21. UNCLASSIFIED / FOUO Pizza Scatter Plot (Cont.) 1. Open worksheet Regression-Pizza.mtw 2. Choose Graph>Scatterplot Simple Linear Regression UNCLASSIFIED / FOUO 21
- 22. UNCLASSIFIED / FOUO Pizza Scatter Plot (Cont.) When you click on Scatterplots, this is the first dialog box that comes up 3. Select the Simple Scatterplot 4. Click on OK to move to the next dialog box Simple Linear Regression UNCLASSIFIED / FOUO 22
- 23. UNCLASSIFIED / FOUO Pizza Scatter Plot (Cont.) 5. Double click on C5 Wait Time to enter it as the Y variable, then double click on C6 Deliveries to enter it as the X variable 6. Edit dialog box options (Optional) 7. Click OK Simple Linear Regression UNCLASSIFIED / FOUO 23
- 24. UNCLASSIFIED / FOUO Pizza Scatter Plot (Cont.) Does it look like the number of Deliveries influences the customer’s Wait Time? Scatterplot of Wait Time vs Deliveries 55 50 Wait Time 45 40 35 10 15 20 25 30 35 Deliveries Simple Linear Regression UNCLASSIFIED / FOUO 24
- 25. UNCLASSIFIED / FOUO Pizza Scatter Plot (Cont.) Note: Hold your cursor over any point on the Scatterplot and Minitab will identify the Row, X-Value and Y-Value for that point Simple Linear Regression UNCLASSIFIED / FOUO 25
- 26. UNCLASSIFIED / FOUO Correlation Coefficients (r & r2) Numbers that indicate the strength of the correlation between two factors r - strength and the direction of the relationship Also called Pearson’s Correlation Coefficient r2 - percentage of variation in Y attributable to the independent variable X. Adds precision to a person’s visual judgment about correlation Test the power of your hypothesis How much influence does this factor have? Are there other, more important, “vital few” causes? Simple Linear Regression UNCLASSIFIED / FOUO 26
- 27. UNCLASSIFIED / FOUO Interpreting Correlation Coefficients r falls on or between -1 and 1 Calculate in Minitab Figures below -0.65 and above 0.65 indicate a meaningful correlation 1 = “Perfect” positive correlation r=0 -1 = “Perfect” negative correlation Use to calculate r2 r=-.8 Simple Linear Regression UNCLASSIFIED / FOUO 27
- 28. UNCLASSIFIED / FOUO Pearson Correlation Coefficient (r) – Mortgage Betty Black Belt used the scatter plot to get a visual picture of the relationship between broker experience and call length Now she uses the Pearson Correlation Coefficient, r, to quantify the strength of the relationship 60 50 Call Length 40 r = - 0.896 30 (a strong negative correlation) 20 10 20 30 Broker Experience Simple Linear Regression UNCLASSIFIED / FOUO 28
- 29. UNCLASSIFIED / FOUO Exercise: Correlation The scatter plot shows that the customers are waiting longer when Anthony’s Pizza has to make more deliveries Next, the Belt wants to quantify the strength of that relationship To do that, we will calculate the Pearson Correlation Coefficient, r Simple Linear Regression UNCLASSIFIED / FOUO 29
- 30. UNCLASSIFIED / FOUO Pizza Correlation 1. Choose Stat > Basic Statistics > Correlation Simple Linear Regression UNCLASSIFIED / FOUO 30
- 31. UNCLASSIFIED / FOUO Correlation Input Window 2. Double click on C5 Wait Time and C6 Deliveries to add them to the Variables box 3. Uncheck the box, Display p-values 4. Click OK Simple Linear Regression UNCLASSIFIED / FOUO 31
- 32. UNCLASSIFIED / FOUO Correlation Coefficient Since r, the Pearson correlation, is 0.970, there is a meaningful correlation between the wait time and number of deliveries Simple Linear Regression UNCLASSIFIED / FOUO 32
- 33. UNCLASSIFIED / FOUO Interpreting Coefficients – r2 First, we obtained r from the Correlation analysis Next, in Regression, we will look at r2 to see how good our model (regression equation) is r2: Compute by multiplying r x r (Pearson correlation squared) Example: With an r value of .970, in the Pizza example, the team computed r2 : .970 x .970 = .941 or 94.1% So, 94% of the variation in wait time is explained by the variability in deliveries Simple Linear Regression UNCLASSIFIED / FOUO 33
- 34. UNCLASSIFIED / FOUO Regression Analysis Regression Analysis is used in conjunction with Correlation and Scatter Plots to predict future performance using past results While Correlation shows how much linear relationship exists between two variables, Regression defines the relationship more precisely Use this tool when there is existing data over a defined range Regression analysis is a tool that uses data on relevant variables to develop a prediction equation, or model Simple Linear Regression UNCLASSIFIED / FOUO 34
- 35. UNCLASSIFIED / FOUO Linear Regression In Simple Linear Regression, a single variable “X” is used to define/predict “Y” e.g.; Wait Time = B1 + (B2) x (Deliveries) + (error) Simple Regression Equation: Y = B1 + (B2) x (X) + Y B2 = Slope y x X Simple Linear Regression UNCLASSIFIED / FOUO 35
- 36. UNCLASSIFIED / FOUO Exercise: Regression Since the Pearson Correlation (r) was .970, we know that there is a strong positive correlation between the number of deliveries and the wait time Next, the Belt would like to get an equation to predict how long the customers will be waiting Simple Linear Regression UNCLASSIFIED / FOUO 36
- 37. UNCLASSIFIED / FOUO Regression (Cont.) 1. Choose Stat>Regression>Fitted Line Plot Simple Linear Regression UNCLASSIFIED / FOUO 37
- 38. UNCLASSIFIED / FOUO Fitted Line Input Window 2. Double click on C5 Wait Time to enter it as the Response (Y) variable 3. Double click on C6 Deliveries to enter it as the Predictor (X) variable 4. Make sure Linear is checked for the type of Regression 5.Edit dialog box options (Optional) 6. Click OK Simple Linear Regression UNCLASSIFIED / FOUO 38
- 39. UNCLASSIFIED / FOUO Pizza Regression Plot Fitted Line Plot Wait Time = 32.05 + 0.5825 Deliveries 55 S 1.11885 R-Sq 94.1% R-Sq(adj) 93.9% 50 Wait Time 45 40 35 10 15 20 25 30 35 Deliveries Simple Linear Regression UNCLASSIFIED / FOUO 39
- 40. UNCLASSIFIED / FOUO Regression Analysis Results – Session Window Prediction Equation (Regression Model) R-Sq is the amount of variation in the data explained by the model. Notice that 94.1 = .970 * .970. R-Sq is the square of the Pearson correlation from the previous analysis. Simple Linear Regression UNCLASSIFIED / FOUO 40
- 41. UNCLASSIFIED / FOUO Using the Prediction Equation If we have 20 deliveries to make, how long will the customer have to wait for their order? Based on our 30 minute guarantee, how acceptable is our performance? Simple Linear Regression UNCLASSIFIED / FOUO 41
- 42. UNCLASSIFIED / FOUO Method of “Least Squares” Regression – Technical Note Fitted Line Plot Wait Time = 32.05 + 0.5825 Deliveries 55 ˆ Y 50 “fitted” observation (the line) Wait Time 45 Y 40 true observation (the data point) 35 10 15 20 25 30 35 Deliveries Minitab will find the “best fitting” line for us. How does it do that? •We want to have as little difference as possible between the true observations and the fitted line •Minitab minimizes the sums of squares of the distance between the fitted and true observations Simple Linear Regression UNCLASSIFIED / FOUO 42
- 43. UNCLASSIFIED / FOUO Multiple Regression Use this when you want to consider more than one predictor variable The benefit is that you might need more predictors to create an accurate model In the case of our Anthony’s Pizza example, we may want to look at the impact that incorrect orders, damaged pizzas, and cold pizzas have on wait time Simple Linear Regression UNCLASSIFIED / FOUO 43
- 44. UNCLASSIFIED / FOUO Individual Exercise: Pizza As a Anthony’s Pizza Belt, you suspect that the number of pizza defects increases when more pizzas are ordered. You want to visualize the data and quantify the relationship Use the Minitab file Pizza Exercise.mtw data to investigate the relationship between “Total Pizzas” and “Defects” Create a scatter plot Determine correlation Create a fitted line plot Determine the prediction equation How many defects do we usually have when 50 pizzas are on order? What do you think of this model? Simple Linear Regression UNCLASSIFIED / FOUO 44
- 45. UNCLASSIFIED / FOUO Another Exercise: Absentee Rate The human resources director of a chain of fast-food restaurants studied the absentee rate of employees. Whenever employees called in sick, or simply did not show up, the restaurant manager had to find replacements in a hurry, or else work short-handed The director had data on the number of absences per 100 employees per week (Y) and the average number of months’ experience at the restaurant (X) for 10 restaurants in the chain. The director expected that long-term employees would be more reliable and absent less often Simple Linear Regression UNCLASSIFIED / FOUO 45
- 46. UNCLASSIFIED / FOUO Absentee Rate 1. Open an blank Minitab worksheet Experience Absences and input the data 18.1 31.5 2. Create a scatter plot and decide 20.0 33.1 whether a straight line is a 20.8 27.4 reasonable model 21.5 24.5 3. Conduct a regression analysis and 22.0 27.0 get the linear prediction equation 22.4 27.8 4. Predict the number of absences for 22.9 23.3 employees with 19.5 months of 24.0 24.7 experience 25.4 16.9 27.3 18.1 Simple Linear Regression UNCLASSIFIED / FOUO 46
- 47. UNCLASSIFIED / FOUO Takeaways Start with a visual tool – create a scatter plot Determine the Pearson correlation coefficient, r, to determine the strength of the relationship Remember that correlation does not guarantee causation! Create and interpret the Regression Plot Use the prediction equation Validate the prediction model’s r-squared using new data (not part of the data set used in creating the prediction equation) Simple Linear Regression UNCLASSIFIED / FOUO 47
- 48. UNCLASSIFIED / FOUO What other comments or questions do you have? UNCLASSIFIED / FOUO