Ml ch17
- 1. CH 17 GOING A STEP BEYOND USING SUPPORT VECTOR MACHINES 10766012陳遠任 Jason
- 2. Revisiting the Separation Problem • Nonseparability of classes • There is no straight line that traces a precise border between different examples. • Other options • K-Nearest Neighbors : Ch14 • Logistic regression : Ch15 • Transforming the features : Solves the problem by employing both feature creation • Decision trees、Neural networks
- 3. Characteristics of Support Vector Machines • Binary and multiclass classification, regression, and detection of anomalous or novelty data • Robust handling of overfitting, noisy data, and outliers • A capability to handle situations with many variables • Easy and timely handling of up to about 10,000 training examples • Automatic detection of nonlinearity in data
- 5. SVM - Linear Negative hyperplane Positive hyperplane Support Vector
- 6. Applying Nonlinearity • Nonlinearly separable points requiring feature transformation (left) to be fit by a line (right). • Make the existing features onto a feature space of higher dimensionality
- 7. Applying Nonlinearity • Problems and limits : • The number of features increases exponentially, making computations cumbersome 計算繁複 • The expansion creates many redundant features, causing overfitting. 創造冗餘特徵 • Difficult to determine becoming linearly or not, requiring many iterations of expansion and test
- 8. Kernel functions • kernel functions project the original features into a higher dimensional space by combining them in a nonlinear way • rely on algebra calculations
- 9. Discovering the different kernels • Linear: Suitable for linear • No extra parameters • Radial Basis Function: Suitable for non-linear • parameters: gamma • Polynomial: suitable for non-linear • parameters: gamma, degree, and coef0 • Sigmoid: Binary classification like Logistic Regression • parameters: gamma and coef0 • Custom-made kernels: Depends upon the kernel
- 10. Radial Basis Function • An RBF kernel that uses diverse hyper- parameters to create unique SVM solutions. • The RBF kernel can adapt itself to different learning strategies • the error cost is high -> bended hyperplane • the error cost is low -> smoother curve line
- 11. Kernels • The polynomial and sigmoid kernels aren’t as adaptable as RBF, thus showing more bias • Most data problems are easily solved using the RBF sigmoid polynomial
- 12. Classifying and Estimating with SVM • handwritten recognition task • the digits dataset (from Scikit-learn) • nonlinear kernel, using the RBF • a series of 8-x-8 grayscale pixel images of handwritten numbers ranging from 0 to 9.