K Means Clustering Algorithm | K Means Example in Python | Machine Learning Algorithms | Edureka
11 likes2,551 views
The document discusses the K-Means clustering algorithm. It begins by defining clustering as grouping similar data points together. It then describes K-Means clustering, which groups data into K number of clusters by minimizing distances between points and cluster centers. The K-Means algorithm works by randomly selecting K initial cluster centers, assigning each point to the closest center, and recalculating centers as points are assigned until clusters stabilize. The best number of K clusters is found through trial and error to minimize variation between points and clusters.
K Means Clustering Algorithm | K Means Example in Python | Machine Learning Algorithms | Edureka
- 1. Copyright © 2017, edureka and/or its affiliates. All rights reserved.
- 2. Copyright © 2017, edureka and/or its affiliates. All rights reserved. ▪ What is Clustering? ▪ Types of Clustering ▪ What is K- Means Clustering? ▪ How does a K-Means Algorithm works? ▪ K-Means with Python Copyright © 2018, edureka and/or its affiliates. All rights reserved. Agenda of Today’s Session
- 3. Copyright © 2018, edureka and/or its affiliates. All rights reserved. What is Clustering?
- 4. “Clustering is the process of dividing the datasets into groups, consisting of similar data-points” What is Clustering? ▪ Points in the same group are as similar as possible ▪ Points in different group are as dissimilar as possible
- 5. What is Clustering? Group of diners in a restaurant Items arranged in a mall
- 6. Where is it Used? Recommendation System Recommended Movies Fickr’s Photos
- 8. Types of Clustering Exclusive Clustering Overlapping Clustering Hierarchical Clustering Exclusive Clustering ▪ Hard Clustering ▪ Data Point / Item belongs exclusively to one cluster ▪ For Example: K-Means Clustering
- 9. Types of Clustering Exclusive Clustering Overlapping Clustering Hierarchical Clustering Overlapping Clustering ▪ Soft Cluster ▪ Data Point/ Item belongs to multiple cluster ▪ For Example: Fuzzy/ C-Means Clustering
- 10. Types of Clustering Exclusive Clustering Overlapping Clustering Hierarchical Clustering Hierarchical Clustering 1 2 3 4
- 11. What is K-Means Clustering? “K-Means is a clustering algorithm whose mail goal is to group similar elements or data points into a cluster.” NOTE: ‘K’ in K-Means represent the number of clusters
- 12. What is K-Means Clustering? Pile of dirty clothes
- 15. K-Means Algorithm Number of Clusters, K = 3
- 16. K-Means Algorithm Distance from point 1 to the red cluster Distance from point 1 to the blue cluster Distance from point 1 to the green cluster ▪ Step 1: Select the number of clusters to be identified, i.e select a value for K =3 in this case ▪ Step 2: Randomly select 3 distinct data point ▪ Step 3: Measure the distance between the 1st point and selected 3 clusters
- 17. K-Means Algorithm Step 4: Assign the 1st point to nearest cluster (red in this case).
- 18. K-Means Algorithm Step 5: Calculate the mean value including the new point for the red cluster
- 19. K-Means Algorithm Distance from point 2 to the red cluster Distance from point 2 to the blue cluster Distance from point 2 to the green cluster Find to which cluster does point 2 belongs to, how? ▪ Repeat the same procedure but measure the distance to the red mean Add the point to the nearest cluster
- 20. K-Means Algorithm Calculate the cluster mean including the new point
- 21. K-Means Algorithm Find to which cluster does point 3 belongs to, how? ▪ Repeat the same procedure but measure the distance to the red mean
- 22. K-Means Algorithm Measure the distance and add the 3rd point to the nearest cluster, (red)
- 23. K-Means Algorithm Calculate the new cluster mean using the new point
- 24. K-Means Algorithm To which cluster does this point belongs to? ▪ Measure the distance ▪ Assign the point to the nearest cluster ▪ Calculate the cluster mean using the new point
- 25. K-Means Algorithm ▪ Measure the distance ▪ Assign the point to the nearest cluster ▪ Calculate the cluster mean using the new point
- 26. K-Means Algorithm ▪ Measure the distance ▪ Assign the point to the nearest cluster ▪ Calculate the cluster mean using the new point
- 27. K-Means Algorithm To which cluster does this point belongs to? ▪ Measure the distance from the cluster mean (centroids) ▪ Assign the point to the nearest cluster ▪ Calculate the cluster mean using the new point
- 28. K-Means Algorithm ▪ Measure the distance from the cluster mean (centroids) ▪ Assign the point to the nearest cluster ▪ Calculate the cluster mean using the new point Since the point is located closet to green cluster
- 29. K-Means Algorithm ▪ Measure the distance from the cluster mean (centroids) ▪ Assign the point to the nearest cluster ▪ Calculate the cluster mean using the new point
- 30. K-Means Algorithm Since all of these points are located closet to green cluster so all of them will be assigned to green cluster
- 31. K-Means Algorithm Original/Expected Result Result from 1st iteration
- 32. K-Means Algorithm Total variation within the cluster According to the K-Means Algorithm it iterates over again and again unless and until the data points within each cluster stops changing
- 33. K-Means Algorithm Iteration 2: Again we will start from the beginning. But this time we will be selecting different initial random point (as compared to what we chose in the 1st iteration) ▪ Step 1: Select the number of clusters to be identified, i.e. K =3 in this case ▪ Step 2: Randomly select 3 distinct data point ▪ Step 3: Measure the distance between the 1st point and selected 3 clusters
- 34. K-Means Algorithm Algorithm picks 3 initial clusters and adds the remaining points to the cluster with the nearest mean, and again recalculating the mean each time a new point is added to the cluster
- 35. K-Means Algorithm Algorithm picks 3 initial clusters and adds the remaining points to the cluster with the nearest mean, and again recalculating the mean each time a new point is added to the cluster
- 36. K-Means Algorithm Algorithm picks 3 initial clusters and adds the remaining points to the cluster with the nearest mean, and again recalculating the mean each time a new point is added to the cluster
- 37. K-Means Algorithm Algorithm picks 3 initial clusters and adds the remaining points to the cluster with the nearest mean, and again recalculating the mean each time a new point is added to the cluster Total variation within the cluster
- 38. K-Means Algorithm Iteration 3: Again we will start from the beginning and select different initial random point (as compared to what we chose in the 1st and 2nd iteration) Pick 3 initial clusters
- 39. K-Means Algorithm Cluster the remaining points
- 40. K-Means Algorithm Total variation within the cluster Finally sum the variation within each cluster
- 41. K-Means Algorithm The algorithm can now compare the result and select the best variance out of it 1st Iteration 2nd Iteration 3rd Iteration
- 42. K-Means Algorithm Now what if we have our data plotted on the X and Y axis X-Axis Y-Axis
- 43. K-Means Algorithm Similarly, pick initial 3 random points.. X-Axis Y-Axis
- 44. K-Means Algorithm We will be using the Euclidean distance (in 2D its same as that of a Pythagorean Theorem) X-Axis Y-Axis ℎ2 = 𝑝2 + 𝑏2 p b
- 45. K-Means Algorithm Again assign the point to the nearest cluster X-Axis Y-Axis
- 46. K-Means Algorithm Finally calculate the centroid (mean of cluster) including the new point X-Axis Y-Axis
- 47. K-Means Algorithm Finally in first iteration you get something like this…again you have to iterate this process to get the final cluster X-Axis Y-Axis
- 48. How will you find K value In the previous scenario k value was known to be 3, but this is not always true
- 49. How will you find K value For deciding the value of k, you have to use hit and trail method, starting from K = 1 K=1 is the worst case scenario, even you cross- verify it with total variation
- 50. How will you find K value Now try with K = 2 K=2 is still better then K = 1 (Total Variation) K = 1 K = 2
- 51. How will you find K value Now try with K = 3 K=3 is even better than K =2 (Total Variation) K = 1 K = 2 K = 3
- 52. How will you find K value Now try with K = 4 Total variation in K=4 is less than K =3 K = 1 K = 2 K = 3 K = 4
- 53. How will you find K value Now try with K = 4 Total variation in K=4 is less than K =3 K = 1 K = 2 K = 3 K = 4 Each time you increase the cluster the variation decreases, no. of clusters = no. of data points then in that case the variation = 0
- 54. How will you find K value Reductioninvariance Number of Cluster This point is the elbow point and it is used to determine the value of K(clusters)
- 55. Copyright © 2017, edureka and/or its affiliates. All rights reserved. DEMO Let’s learn to code © MadeByShape Ltd. Web Design Manchester 2018. All Rights Reserved.
- 56. K-Means Algorithm Summarizing the K-Means Algorithm
- 57. Copyright © 2017, edureka and/or its affiliates. All rights reserved.