Training machine learning knn 2017
0 likes309 views
1. K-nearest neighbors (k-NN) is a simple machine learning algorithm that stores all training data and classifies new data based on the majority class of its k nearest neighbors. 2. It is a lazy, non-parametric algorithm that makes no assumptions about the distribution of the data. Learning involves storing training examples, while classification assigns a class based on similarity to stored examples. 3. k-NN has applications in areas like credit ratings, political science, handwriting recognition, and image recognition. It works by finding the k closest training examples in feature space and assigning the new example the majority class of those neighbors.
Training machine learning knn 2017
- 1. 1 KOTA BARU PARAHYANGAN BANDUNG Machine Learning k-Nearest Neighbour
- 2. 2 k-NN
- 3. 3 References • KTH Royal Institute – Lecture Notes • K-Nearest Neighbor Learning - Dipanjan Chakraborty • T. Mitchell, Machine Learning, McGraw-Hill (Recommended) • Papers, Website, tutorial
- 4. • The k-NN algorithm is among the simplest of all machine learning algorithms • Supervised learning • Data are represented in a vector space • Most basic instance-based method – Disadvantage of instance-based methods is that the costs of classifying new instances can be high – Nearly all computation takes place at classification time rather than learning time k-NN Overview
- 5. • KNN is a non-parametric, lazy learning algorithm – Non-parametric: it does not make any assumptions on the underlying data distribution. in the “real world”, most of the data does not obey the typical theoretical assumptions made (as in linear regression models, for example) – Lazy: it does not use the training data points to do any generalization. In other words, there is no explicit training phase or it is very minimal.In contrast to so called “eager learning” algorithms (which carries out learning without knowing the test example, and after learning training examples can be discarded) • Its purpose is to use a database in which the data points are separated into several classes to predict the classification of a new sample point • KNN can be used for classification k-NN Overview
- 6. • Learning = storing all training instances • Classification = assigning target function to a new instance • Referred to as “Lazy” learning K-NN Overview
- 7. • Credit ratings — collecting financial characteristics vs. comparing people with similar financial features to a database • In political science — classing a potential voter to a “will vote” or “will not vote”, or to “vote Democrat” or “vote Republican”. • Handwriting detection (like OCR), image recognition and even video recognition. k-NN Application
- 9. k-NN Algorithm • Given a new set of measurements, perform the following test: – Find (using Euclidean distance, for example), the k nearest entities from the training set. These entities have known labels. The choice of k is left to us. – Among these k entities, which label is most common? That is the label for the unknown entity.
- 10. k-NN Algorithm • Distance Metric • k-Nearest Neighbor Predictions • Distance Weighting
- 14. k-NN Algorithm – Example • For each training example <x,f(x)>, add the example to the list of training_example • Given a query instance xq to be classified, – Let x1, x2, ... xk denote the k instances from training_examples that are nearest to xq – Return the class that represents the maximum of the k instances
- 15. • If K = 5, then in this case query instance xq will be classified as negative since three of its nearest neighbors are classified as negative
- 16. k-NN Summary • Advantages – Learning is extremely simple and intuitive – Very flexible decision boundaries – Variable-sized hypothesis space • Disadvantages – distance function must be carefully chosen or tuned – irrelevant or correlated features have high impact and must be eliminated – typically cannot handle high dimensionality computational – computational costs: memory and classification-time computation
- 17. When to Consider Nearest Neighbor ? • Lots of training data and no training stage, all the work is done during the test stage • Advantages: – Can be applied to the data from any distribution • for example, data does not have to be separable with a linear boundary – Training is very fast – Learn complex target functions – Don’t lose information • Disadvantages: – Choosing k may be tricky – Slow at query time – Easily fooled by irrelevant attributes – Need large number of samples for accuracy