Machine Learning with Multiple Regression - APAC

© 2020 Minitab, LLC.
Machine Learning with Multiple Regression

• 25+ years of
experience
• Minitab Trainer
• Statistical Consultant
• Minitab Software
Designer
• Master's in Statistics
Meet the Presenter:
Cheryl Pammer
Senior Advisory Statistician

Learning Objectives
 The Language of Machine
Learning
 Multiple Regression in a
Machine Learning
Environment
 Validation and Cross
Validation
4

So What is Machine Learning?
Machine learning helps to
analyze the patterns in the
collected data.
5

WAIT… WHAT?
6
Introduction to Data Science
UC Irvine
Data Mining and Applications
Stanford

Machine Learning
7
MACHINE
LEARNING
UNSUPERVISED LEARNING
SUPERVISED LEARNING
Group and interpret
data based on
input data only
(X’s).
Develop predictive
models using input
and output data
(X’s and Y’s).
CLUSTERING
CLASSIFICATION
REGRESSION

Basic Supervised Machine Learning Algorithms
►Continuous Y: Regression, CART Regression Trees
►Categorical Y: Logistic Regression, CART Classification Trees
8

What Has a Machine Learned?
9
Training Data Test Data
?Data is split into a training set and a test set:
►Training (or learn) = creates model
►Test = assesses model performance

Why is Model Testing (Validation) Important?
►Assessing model with the same data used to fit model leads to
overfitting.
►Overfit models do not predict well.
10

Bias-Variance Trade-Off
11
Model with High Bias Model with High Variance

Validation
12
Validation helps find the best balance between too simple (high bias)
and too complex (high variance)

Methods of Validation
►Leave-one-out validation
►Validation with a test set
►K-fold cross validation
13

Example: Smart Water System
►Manufacturer of smart water
meters investigates what causes
bias in usage readings.
►Data from various test
sequences provides insight into
contributors of biased readings.
14

Multiple Regression
 Examines relationship between continuous response
(Y) and multiple predictors (X).
 Relationship can be expressed as an equation:
Y = β0 + β1X1 + β2X2 + … βnXn + ε
15

Leave-One-Out Validation
1. Fit model using every row except one.
2. Compute error for held-out observation.
3. Repeat for every row of data.
16

Leave-One-Out Validation
4. Calculate sum of squares of errors from held-out rows. (Predicted
Residual Sum of Squares)
5. Select model using summary statistics based on PRESS.
17
𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃 = � 𝑦𝑦𝑖𝑖 − �𝑦𝑦−𝑖𝑖
2
𝑅𝑅2
(𝑝𝑝𝑝𝑝𝑝𝑝𝑝𝑝) = 1 −
𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃
𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆

Validation With a Test Set
1. Hold out random
X% of data when
fitting model.
18

2. Compute error for held-out data.
19
𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇 𝑆𝑆 =
∑ 𝑦𝑦𝑖𝑖 − �𝑦𝑦𝑖𝑖
2
𝑛𝑛𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇
Held-out observation
Prediction from model
from training data

3. Select model using summary statistics based on test error.
20
𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇 𝑅𝑅2 = 1 −
𝑆𝑆𝑆𝑆𝑆𝑆(𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇)
𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆(𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇)
𝑆𝑆𝑆𝑆𝑆𝑆 𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇 = � 𝑦𝑦𝑖𝑖 − �𝑦𝑦𝑖𝑖
2

Important Effects
21

Model Comparison
22
“Optimal” Model Simple Model

Regression Assumptions
►No distinct patterns
►Assumptions are met
 Normality
 Equal Variance
 Independence
23

Model Visualizations
24

Key Result
Use regression equation to predict bias in readings and calibrate
sensors.
25

Example: Amusement Park Wait Times
►An amusement park
needs a model to predict
ride wait times for each
attraction.
►Predictors: park
attendance, ticket sales,
ride intensity rating, time
of day, whether it is a
weekend, percentage of
sun and temperature.
26

K-Fold Cross Validation
1. Split data into k random folds.
27
Fold 1
Fold 2
Fold 3
Fold 4
Fold 5

2. Fit model using every fold except one.
3. Compute test error using held-out fold.
4. Repeat k times, holding out each fold once.
28
K-fold 𝑆𝑆 =
∑ ∑ 𝑦𝑦𝑖𝑖− �𝑦𝑦𝑖𝑖
2
𝑛𝑛
Held-out observation Prediction from model
from remaining folds

5. Select model using summary statistics based on k-fold test error.
29

When Are the Highest Wait Times?
30

Key Results
31

Take-aways
►The language of machine learning
►Multiple regression in a machine learning environment
►Proven validation techniques for model selection
Cheryl Pammer
cpammer@minitab.com
32

Upcoming Webinars and Virtual Events
• Machine Learning with Binary Logistic Regression
Time: Tuesday 30 June, 12PM AEST (10AM HKT / 2PM NZST)
• Machine Learning with Classification & Regression Trees
(CART® )
Time: Wednesday 15 July, 12PM AEST (10AM HKT / 2PM NZST)
See all the details and sign up at:
https://info.minitab.com/resources/webinars/webinar-wednesdays

Upcoming Webinars and Virtual Events
• Online/Virtual Training
Minitab is now offering virtual training taught by
Minitab experts – perfect for remote/home workers.
Visit www.minitab.com/training/training for more info.
• Talk to Minitab
Complimentary resources to help you deal quickly with today's challenges and changing environment.
Visit www.minitab.com and click on the Talk to Minitab button and a Minitab representative will be in touch!

Our Approach: More Than Business Analytics… Solutions Analytics
Software
Services
Training
Learn first-hand by attending public or
customized trainings in your facilities
according to your requirements.
Statistical
Consulting
Personalized help with statistical
challenges from collecting the right data
to interpreting analysis more.
Support
Assistance with installation,
implementation, version updates
and license management.
Master statistics and
Minitab anywhere
with online training
Machine learning and
predictive analytics
software
Start, track, manage
and execute
improvement projects
with real-time
dashboards
Powerful statistical
software everyone
can use.
Data Analysis Predictive Modeling Visual Business Tools Project Oversight
Visual tools to
process and product
excellence.
Online Training
Solutions analytics is our integrated approach to providing software and services that enable organizations to make better decisions that drive business excellence.

Machine Learning with Multiple Regression - APAC

More Related Content

What's hot (16)

Similar to Machine Learning with Multiple Regression - APAC (20)

More from Minitab, LLC (17)

Recently uploaded (20)

Machine Learning with Multiple Regression - APAC