Regression

Regression and Classification
3/31/2020 Shivani Saluja(G D Goenka University ) 1

Regression
• It is a form of predictive modelling technique which investigates the
relationship between a dependent and independent variable
• It involves graphing a line over a set of data points which closely fits
the overall shape of data
• Change of dependent variable on y and independent on x

Applications
• Determing strength of indicators(impact of independent and
dependent relationship between sales and marketing )
• Forecasting the effect
• Trend forecasting

Classification
• A classification problem is when the output variable is a category, such as “red” or “blue”
or “disease” and “no disease”.
• A classification model attempts to draw some conclusion from observed values.
• Given one or more inputs a classification model will try to predict the value of one or
more outcomes.
For example, when filtering emails “spam” or “not spam”, when looking at transaction
data, “fraudulent”, or “authorized”.
• Classification either predicts categorical class labels or classifies data (construct a
model) based on the training set and the values (class labels) in classifying attributes and
uses it in classifying new data.
• There are a number of classification models. Classification models include logistic
regression, decision tree, random forest, gradient-boosted tree, multilayer perceptron,
one-vs-rest, and Naive Bayes

Examples
• Predicting the gender of a person by his/her handwriting style(C)
• Predicting house price based on area(R)
• Predicting whether monsoon will be normal next year(C)
• Predict the number of copies a music album will be sold next
month(R).

Linear vs Logistic Regression
Linear Regression
1. Data is modelling using straight line
2. Used with continuous variable
3. Output /Prediction is value of variable
4. Accuracy is measured by method of
least squares
Logistic Regression
1. Probability of some event is
represented as a linear function of
combination of predictor variables
2. Used with categorical variable
3. Output /Prediction is Probability of
occurrence of an event
4. Accuracy is measured in terms of
precision and recall

Linear(Univariate)Regression
• Linear regression may be defined as the statistical model that analyzes the linear relationship
between a dependent variable with given set of independent variables.
• Linear relationship between variables means that when the value of one or more independent
variables will change (increase or decrease), the value of dependent variable will also change
accordingly (increase or decrease).
• Mathematically the relationship can be represented with the help of following equation
𝑌 = 𝑚𝑥 + 𝑏
Y is the dependent variable we are trying to predict.
X is the dependent variable we are using to make predictions.
m is the slop of the regression line which represents the effect X has on Y
b is a constant, known as the Y-intercept. If X = 0,Y would be equal to b.
• It relates one predictor and one response

Positive vs Negative Linear Relationship
A linear relationship will be called
positive if both independent and
dependent variable increases.
A linear relationship will be called
negative if independent increases and
dependent variable decreases.

Assumptions
The following are some assumptions about dataset that is made by
Linear Regression model −
• Multi-collinearity − Linear regression model assumes that there is
very little or no multi-collinearity in the data. Basically, multi-
collinearity occurs when the independent variables or features have
dependency in them.
• Auto-correlation − Another assumption Linear regression model
assumes is that there is very little or no auto-correlation in the data.
Basically, auto-correlation occurs when there is dependency between
residual errors.

Types of Linear Regression
• Simple Linear Regression
• Multiple Linear Regression

Principle of Least Squares
• The Least Squares Regression Line is the line that makes the vertical
distance from the data points to the regression line as small as
possible. It's called a “least squares” because the best line of fit is one
that minimizes the variance (the sum of squares of the errors).
• Y must be the best possible estimate of the real data.

Numerical
• The values of dependent (y) and independent (x) variable are given in
table below .Find the least square regression line y=ax+b .Estimate
the value of y when x is 6
X Y
1 3
2 4
3 2
4 4
5 5

x y (x-x’) (y-y’) Square(x-x’) (x-x’)(y-y’)
1 3
-2 -.6 4 1.2
2 4
-1 0.4 1 -0.4
3 2
0 -1.6 0 0
4 4
1 0.4 1 0.4
5 5
2 1.4 4 2.8
Mean =3 Mean =3.6
ɛ =10 ɛ =4
Y=mx+c
Substitute mean value of y and x in equation
3.6= .4*3 +c
C=3.6-1.2 =2.4
When x is 6
Y= .4*6 +2.4= 4.8

Multiple Regression
It r elates more than one predictor and one
response

Multivariate Regression
It is a method used to measure the degree at which more than one
independent variable (predictors) and more than one dependent
variable (responses), are linearly related.

Logistic Regression

Definition
• Logistic Regression is used when the dependent variable(target) is
categorical.
For example,
• To predict whether an email is spam (1) or (0)
• Whether the tumor is malignant (1) or not (0)

Types of logistic Regression
1) Binary Logistic Regression
The categorical response has only two 2 possible outcomes. Example:
Spam or Not
2. Multinomial Logistic Regression
Three or more categories without ordering. Example: Predicting which
food is preferred more (Veg, Non-Veg, Vegan)
3. Ordinal Logistic Regression
• Three or more categories with ordering. Example: Movie rating from 1
to 5

Regression

More Related Content

What's hot (17)

Similar to Regression (20)

More from shivani saluja (6)

Recently uploaded (20)

Regression