Introduction to Artificial Intelligence_ Lec 7
- 2. • What Is Supervised learning ? • Linear Regression. • Regularized Regression • Support Vector Machine
- 3. • Supervised learning is an area of machine learning where the chosen algorithm tries to fit a target using the given input. • A set of training data that contains labels is supplied to the algorithm. Based on a massive set of data, the algorithm will learn a rule that it uses to predict the labels for new observations. • In other words, supervised learning algorithms are provided with historical data and asked to find the relationship that has the best predictive power. • There are two varieties of supervised learning algorithms: • Regression algorithms. • Classification algorithms.
- 5. • Linear regression is a linear model, e.g., a model that assumes a linear relationship between the input variables (x) and the single output variable (y). • The goal of linear regression is to train a linear model to predict a new y given a previously unseen x with as little error as possible. • Our model will be a function that predicts y given x1, x2...xi: • y = β0 + β1x1 + ... + βixi • where, β0 is called intercept and β1...βi are the coefficient of the regression.
- 6. • The cost function for linear regression is the sum of squared residuals (RSS): • In this equation, β0 is the intercept; βj represents the coefficient; β1, .., βj are the coefficients of the regression; and xij represents the ith observation and jth variable.
- 7. Advantages and Disadvantages • In terms of advantages, linear regression is easy to understand and interpret. • It may not work well when there is a nonlinear relationship between predicted and predictor variables. • Linear regression is prone to overfitting and when a large number of features are present, it may not handle irrelevant features well.
- 8. • Regularized Regression modifies Linear Regression by adding a penalty to the coefficients to prevent overfitting. The two main types are: ❖ Ridge Regression (L2 Regularization): Adds a penalty proportional to the sum of squared coefficients (shrinks but does not remove features). ❖ Lasso Regression (L1 Regularization): Adds a penalty proportional to the absolute value of coefficients (can shrink some coefficients to zero, performing feature selection).
- 9. • Logistic Regression is used for classification tasks, not regression. It predicts the probability of a class using a logistic (sigmoid) function. • Key Differences from Linear Regression: • Output is a probability (between 0 and 1), not a continuous value. • Uses the sigmoid function to map predictions to probabilities. • Instead of minimizing squared errors, it minimizes log loss (cross-entropy loss).
- 10. • The objective of the support vector machine (SVM) algorithm is to maximize the margin (shown as shaded area in the Figure), which is defined as the distance between the separating hyperplane (or decision boundary) and the training samples that are closest to this hyperplane, the so-called support vectors. • The margin is calculated as the perpendicular distance from the line to only the closest points, as shown in Figure. Hence, SVM calculates a maximum-margin boundary that leads to a homogeneous partition of all data points.
- 11. • SVM is used for both classification and regression.
- 12. In terms of advantages, • SVM is fairly robust against overfitting, especially in higher dimensional space. • It handles the nonlinear relationships quite well. In terms of disadvantages, • SVM can be inefficient to train and memory-intensive to run and tune. • It doesn’t perform well with large datasets. It requires the feature scaling of the data.
- 13. • What is the Supervised learning? • What are the SVM Advantages and disadvantages?