Kernel Method
Download as PPTX, PDF3 likes1,751 views
This document provides an overview of kernel methods and Gaussian processes. It discusses dual representations, constructing kernels such as polynomial and radial basis function kernels. It also covers Gaussian processes for regression and classification, including learning hyperparameters, automatic relevance determination, and using the Laplace approximation. The document contains section headings and mathematical equations but no complete paragraphs of text.
![6.4.1 Linear regression revisited
Var[w]
p(y) = N(y|0, K)
y(x1), y(x2), … , y(xN)](https://image.slidesharecdn.com/gaussianprocess-180523074252/85/Kernel-Method-21-320.jpg)
Kernel Method
- 2. 목차 • 6.1 Dual Representations • 6.2 Constructing Kernels • 6.3 Radial Basis Function Networks o 6.3.1 Nadaraya-Watson model • 6.4 Gaussian Process o 6.4.1 Linear regression revisited o 6.4.2 Gaussian processes for regression o 6.4.3 Learning the hyperparameters o 6.4.4 Automatic relevance determination o 6.4.5 Gaussian processes for classification o 6.4.6 Laplace approximation o 6.4.7 Connection to neural networks
- 3. 6.1. Dual Representation Sum-of-squares error function in a linear regression model
- 4. 6.1. Dual Representation k(x) = k(x1, x) k(x2, x) : k(xN, x) Regularized least squares
- 6. 6.2. Constructing Kernels Polynomial kernel https://en.wikipedia.org/wiki/Polynomial_kernel
- 11. 6.3 Radial Basis Function Networks
- 12. 6.3 Radial Basis Function Networks y(x) x m xm
- 14. 6.3.1 Nadaraya-Watson model 1. Kernel regression by definition of conditional by (6.42) t’=t+tn , dt’/dt = 1 2. Density estimation
- 16. 6.4 Gaussian Processes 1 0 0 1 1 0.9 0.9 1 1 -0.9 -0.9 1 1 0.4 0.4 1
- 17. 6.4 Gaussian Processes p(y) ~ N(y|0, K)
- 18. 6.4 Gaussian Processes K = IN K = ? data를 잘 sampling 해낼 수 있는 covariance matrix K를 찾는 것이 목표!
- 19. 6.4 Gaussian Processes covariance matrix = positive-semidefinite matrix = gram matrix
- 20. 6.4.1 Linear regression revisited
- 21. 6.4.1 Linear regression revisited Var[w] p(y) = N(y|0, K) y(x1), y(x2), … , y(xN)
- 22. 6.4.1 Linear regression revisited Gaussian kernel exponential kernel Gram matrix K
- 23. 6.4.1 Linear regression revisited linear kernel k(xn, xm) = (xn)T xm constant kernel k(xn, xm) = C
- 24. 6.4.2 Gaussian processes for regression length scale of the correlation constant linear radius basis Gaussian Gram matrix constant Gram matrix linear Gram matrix
- 25. 6.4.1 Linear regression revisited “In cases where the input vector x is two dimensional, this may also be known as a Gaussian random field.” p.305
- 26. 6.4.2 Gaussian processes for regression Find p(y)
- 27. 6.4.2 Gaussian processes for regression Find p(t) C = β-1IN+K p(t, y)
- 28. 6.4.2 Gaussian processes for regression Find p(tn+1)
- 29. 6.4.2 Gaussian processes for regression Find p(tn+1|t)
- 30. 6.4.3 Learning the hyperparameters (기계학습, Machine Learning) Week 3 Gaussian Process | Lecture 15 https://www.youtube.com/watch?v=A_ER5Dgg4ck non-convex function, multiple maxima
- 31. 6.4.4 Automatic relevance determination x2 x1 k(x, x’)
- 32. 6.4.4 Automatic relevance determination η1 η2 η3
- 33. 6.4.5 Gaussian processes for classification y(x) in Gaussian process regression
- 34. 6.4.5 Gaussian processes for classification Find p(aN+1) = p(tN+1) in Gaussian process regression
- 35. 6.4.5 Gaussian processes for classification Find p(tN+1=1|tN) 1. Sampling methods (Neal, 1997) 2. Analytical approximation - variational inference (Gibbs and Mackay, 2000) - expectation propagation (Opper and Winther, 2000b; Minka, 2001b; Seeger, 2003) - Laplace approximation
- 36. 6.4.6 Laplace approximation Find p(aN+1|tN) ∵ Bayes’ theorem Bayes’ theorem ∵ ∵ Laplace approx.
- 37. 6.4.6 Laplace approximation Find ∇p(aN|tN), ∇∇p(aN|tN) ln p(aN|tN) = ln p(aN) + ln p(tN|aN) + ln(tN) const. ∵
- 38. 6.4.6 Laplace approximation Find a★ N http://people.seas.harvard.edu/~yaron/AM221/lecture_notes/AM221_lecture10.pdf http://nptel.ac.in/courses/106108056/module5/ConvexFunctions.pdf Proof. Function is convex iff. Hessian is positive semi-definte
- 39. 6.4.6 Laplace approximation Find q(aN) that approximate p(aN|tN)
- 40. 6.4.6 Laplace approximation Find p(aN+1|tN)
- 42. 끝