Supervised Learning Algorithms - Analysis of different approaches
2 likes406 views
The document analyzes supervised learning algorithms, defining machine learning and its evolution, and discussing methods for data collection and model evaluation. It covers concepts like the bias-variance trade-off, types of response data, and dimensionality reduction techniques such as SVD and PCA. Additionally, the document outlines the use of generalized linear models and provides references for further reading.
![RestricCons of Linear models
Although the General linear model is a useful
framework, it is not appropriate in the following cases:
• The range of Y is restricted (e.g. binary, count,
posiCve/negaCve)
• Var[Y] depends on the mean E[Y] (for the Gaussian
they are independent)
Name Mean Variance
Bernoulli(p) p p(1 - p)
Binomial(p, n) np np(1 - p)
Poisson(p) p p](https://image.slidesharecdn.com/supervised-learning-algorithms-analysis-of-different-approaches-170314105518/85/Supervised-Learning-Algorithms-Analysis-of-different-approaches-34-320.jpg)
Supervised Learning Algorithms - Analysis of different approaches
- 2. ML DefiniCon • There are plenty of definiCons... • Informal: The field of study that gives computers the ability to learn without being explicitly programmed (Arthur Samuel, 1959) • Formal: A computer program is said to learn from experience E, with respect to some task T, and some performance measure P, if its performance on T as measured by P improves with experience E (Tom Mitchell, 1998).
- 4. Why ML?
- 5. Why ML?
- 6. Key factors enabling ML growth today • Cloud Compu)ng • Internet of Things • Big Data (+ Unstructured Data)
- 12. Methods for collecCng data • Download – Spreadsheet – Text • API • Crawling / scraping
- 14. Task Description
- 15. Pipeline
- 16. IniCal example
- 17. NotaCon
- 19. • Asdasd • Asdasd • Asdasd • Asdasd The regression funcCon f(x)
- 20. • as • as • as
- 22. Pipeline
- 23. Assessing the Model Accuracy
- 27. Cross-validaCon
- 30. Pipeline
- 31. Data types and Generalized Linear model • Simple and General linear models • RestricCons of the linear model • Data type of the response Y 1) (General) Linear model R, Y ~ Gaussian(µ, σ^2) -- conCnuous 2) LogisCc regression {0, 1}, Y ~ Bernoulli(p) -- binary data 3) Poisson regression {0, 1,...}, Y ~ Poisson(µ) -- counCng data
- 34. RestricCons of Linear models Although the General linear model is a useful framework, it is not appropriate in the following cases: • The range of Y is restricted (e.g. binary, count, posiCve/negaCve) • Var[Y] depends on the mean E[Y] (for the Gaussian they are independent) Name Mean Variance Bernoulli(p) p p(1 - p) Binomial(p, n) np np(1 - p) Poisson(p) p p
- 35. Binary response Y – {0, 1} • The Bernoulli(p) is discrete r.v. with two possible outcomes: • p and q = 1 – p • The parameter p does not change over Cme • Bernoulli is building block for other more complicated distribuCons • Examples: • Coin flips {Heads, Tails} – if unbiased • then p = 0.5 • Click on Ad, Fail/Success on Exam
- 39. Binary Data
- 42. Preprocessing
- 43. Pipeline
- 45. • Instead of using two coordinates ( 𝒙, 𝒚) to describe point locaCons, let’s use only one coordinate (𝒛) • Point’s posiCon is its locaCon along vector 𝒗↓ 𝟏 • How to choose 𝒗↓ 𝟏 ? Minimize reconstruc)on error SVD – Dimensionality ReducCon v1 first right singular vector Movie 1 rating Movie2rating
- 46. SVD - Dimensionality ReducCon More details • Q: How exactly is dim. reduc)on done? • A: Set smallest singular values to zero 46 0.56 0.59 0.56 0.09 0.09 0.12 -0.02 0.12 -0.69 -0.69 0.40 -0.80 0.40 0.09 0.09 x x 1 1 1 0 0 3 3 3 0 0 4 4 4 0 0 5 5 5 0 0 0 2 0 4 4 0 0 0 5 5 0 1 0 2 2 0.13 0.02 -0.01 0.41 0.07 -0.03 0.55 0.09 -0.04 0.68 0.11 -0.05 0.15 -0.59 0.65 0.07 -0.73 -0.67 0.07 -0.29 0.32 12.4 0 0 0 9.5 0 0 0 1.3 ≈
- 47. SVD - Dimensionality ReducCon More details • Q: How exactly is dim. reduc)on done? • A: Set smallest singular values to zero 47 x x 1 1 1 0 0 3 3 3 0 0 4 4 4 0 0 5 5 5 0 0 0 2 0 4 4 0 0 0 5 5 0 1 0 2 2 0.13 0.02 -0.01 0.41 0.07 -0.03 0.55 0.09 -0.04 0.68 0.11 -0.05 0.15 -0.59 0.65 0.07 -0.73 -0.67 0.07 -0.29 0.32 12.4 0 0 0 9.5 0 0 0 1.3 0.56 0.59 0.56 0.09 0.09 0.12 -0.02 0.12 -0.69 -0.69 0.40 -0.80 0.40 0.09 0.09 ≈
- 48. SVD - Dimensionality ReducCon More details • Q: How exactly is dim. reduc)on done? • A: Set smallest singular values to zero ≈ x x 1 1 1 0 0 3 3 3 0 0 4 4 4 0 0 5 5 5 0 0 0 2 0 4 4 0 0 0 5 5 0 1 0 2 2 0.13 0.02 0.41 0.07 0.55 0.09 0.68 0.11 0.15 -0.59 0.07 -0.73 0.07 -0.29 12.4 0 0 9.5 0.56 0.59 0.56 0.09 0.09 0.12 -0.02 0.12 -0.69 -0.69
- 49. ǁA-BǁF = √ Σij (Aij-Bij)2 is “small” SVD – Dimensionality ReducCon (PCA generalizaCon) More details • Q: How exactly is dim. reduc)on done? • A: Set smallest singular values to zero ≈ 1 1 1 0 0 3 3 3 0 0 4 4 4 0 0 5 5 5 0 0 0 2 0 4 4 0 0 0 5 5 0 1 0 2 2 0.92 0.95 0.92 0.01 0.01 2.91 3.01 2.91 -0.01 -0.01 3.90 4.04 3.90 0.01 0.01 4.82 5.00 4.82 0.03 0.03 0.70 0.53 0.70 4.11 4.11 -0.69 1.34 -0.69 4.78 4.78 0.32 0.23 0.32 2.01 2.01 Frobenius norm: ǁMǁF = √Σij Mij 2
- 50. Feature selection - example
- 51. Dummy Encoding
- 52. (De)MoCvaCon
- 54. Pipeline
- 56. General Applications of FMs
- 57. Summary Pipeline
- 58. Pipeline
- 59. From prototype to producCon • Prototype vs ProducCon Cme? – model (pipeline) should stay the same
- 60. Libraries
- 61. QuesCons?
- 62. Thank you!!!
- 63. References • hRps://www.coursera.org/learn/machine- learning • hRp://www.cs.cmu.edu/~tom/ • hRp://scikit-learn.org/stable/ • hRp://www.scalanlp.org/ • hRp://www.algo.uni-konstanz.de/members/ rendle/pdf/Rendle2010FM.pdf • hRps://securityintelligence.com/factorizaCon- machines-a-new-way-of-looking-at-machine- learning/
- 64. References • An IntroducCon to Generalized Linear Models – AnneRe Dobson, Adrian BarneR • Applying Generalized Linear Models – James Lindsey • hRps://www.codementor.io/jadianes/ building-a-recommender-with-apache-spark- python-example-app-part1-du1083qbw • hRps://www.chrisstucchio.com/blog/ index.html