Deep Learning A-Z™: Self Organizing Maps (SOM) - Module 4
- 3. © SuperDataScienceDeep Learning A-Z What we will learn in this section: • How do Self-Organizing Maps work? • K-Means Clustering • How do Self-Organizing Maps Learn? (Part 1) • How do Self-Organizing Maps Learn? (Part 2) • Live SOM example • Reading an Advanced SOM
- 5. © SuperDataScienceDeep Learning A-Z Used for Regression & ClassificationArtificial Neural Networks Used for Computer VisionConvolutional Neural Networks Used for Time Series AnalysisRecurrent Neural Networks Used for Feature DetectionSelf-Organizing Maps Used for Recommendation SystemsDeep Boltzmann Machines Used for Recommendation SystemsAutoEncoders SupervisedUnsupervised
- 6. © SuperDataScienceDeep Learning A-Z Image Source: aka.fi
- 7. © SuperDataScienceDeep Learning A-Z Image Adapted From: arxiv.org/pdf/1312.5753.pdf
- 8. © SuperDataScienceDeep Learning A-Z Image Source: cis.hut.fi
- 9. © SuperDataScienceDeep Learning A-Z Image Source: cis.hut.fi
- 10. © SuperDataScienceDeep Learning A-Z Image Source: cis.hut.fi
- 11. © SuperDataScienceDeep Learning A-Z The Self-Organizing Map By Tuevo Kohonen (1990) Link: http://sci2s.ugr.es/keel/pdf/algorithm/articulo/1990-Kohonen-PIEEE.pdf Additional Reading:
- 12. © SuperDataScienceDeep Learning A-Z
- 13. © SuperDataScienceDeep Learning A-Z
- 14. © SuperDataScienceDeep Learning A-Z
- 15. © SuperDataScienceDeep Learning A-Z Before K-Means
- 16. © SuperDataScienceDeep Learning A-Z Before K-Means After K-Means K-Means
- 17. © SuperDataScienceDeep Learning A-Z Before K-Means After K-Means K-Means
- 18. © SuperDataScienceDeep Learning A-Z STEP 1: Choose the number K of clusters STEP 2: Select at random K points, the centroids (not necessarily from your dataset) STEP 3: Assign each data point to the closest centroid That forms K clusters STEP 4: Compute and place the new centroid of each cluster STEP 5: Reassign each data point to the new closest centroid. If any reassignment took place, go to STEP 4, otherwise go to FIN. Your Model is Ready
- 19. © SuperDataScienceDeep Learning A-Z STEP 1: Choose the number K of clusters: K = 2
- 20. © SuperDataScienceDeep Learning A-Z STEP 2: Select at random K points, the centroids (not necessarily from your dataset)
- 21. © SuperDataScienceDeep Learning A-Z STEP 2: Select at random K points, the centroids (not necessarily from your dataset)
- 22. © SuperDataScienceDeep Learning A-Z STEP 3: Assign each data point to the closest centroid That forms K clusters
- 23. © SuperDataScienceDeep Learning A-Z STEP 3: Assign each data point to the closest centroid That forms K clusters
- 24. © SuperDataScienceDeep Learning A-Z STEP 4: Compute and place the new centroid of each cluster
- 25. © SuperDataScienceDeep Learning A-Z STEP 4: Compute and place the new centroid of each cluster
- 26. © SuperDataScienceDeep Learning A-Z STEP 5: Reassign each data point to the new closest centroid. If any reassignment took place, go to STEP 4, otherwise go to FIN.
- 27. © SuperDataScienceDeep Learning A-Z STEP 5: Reassign each data point to the new closest centroid. If any reassignment took place, go to STEP 4, otherwise go to FIN.
- 28. © SuperDataScienceDeep Learning A-Z STEP 4: Compute and place the new centroid of each cluster
- 29. © SuperDataScienceDeep Learning A-Z STEP 4: Compute and place the new centroid of each cluster
- 30. © SuperDataScienceDeep Learning A-Z STEP 5: Reassign each data point to the new closest centroid. If any reassignment took place, go to STEP 4, otherwise go to FIN.
- 31. © SuperDataScienceDeep Learning A-Z STEP 5: Reassign each data point to the new closest centroid. If any reassignment took place, go to STEP 4, otherwise go to FIN.
- 32. © SuperDataScienceDeep Learning A-Z STEP 4: Compute and place the new centroid of each cluster
- 33. © SuperDataScienceDeep Learning A-Z STEP 4: Compute and place the new centroid of each cluster
- 34. © SuperDataScienceDeep Learning A-Z STEP 5: Reassign each data point to the new closest centroid. If any reassignment took place, go to STEP 4, otherwise go to FIN.
- 35. © SuperDataScienceDeep Learning A-Z STEP 5: Reassign each data point to the new closest centroid. If any reassignment took place, go to STEP 4, otherwise go to FIN.
- 36. © SuperDataScienceDeep Learning A-Z STEP 4: Compute and place the new centroid of each cluster
- 37. © SuperDataScienceDeep Learning A-Z STEP 4: Compute and place the new centroid of each cluster
- 38. © SuperDataScienceDeep Learning A-Z STEP 5: Reassign each data point to the new closest centroid. If any reassignment took place, go to STEP 4, otherwise go to FIN.
- 39. © SuperDataScienceDeep Learning A-Z FIN: Your Model Is Ready
- 40. © SuperDataScienceDeep Learning A-Z FIN: Your Model Is Ready
- 41. © SuperDataScienceDeep Learning A-Z STEP 2: Select at random K points, the centroids (not necessarily from your dataset)
- 42. © SuperDataScienceDeep Learning A-Z FIN: Your Model Is Ready
- 43. © SuperDataScienceDeep Learning A-Z
- 44. © SuperDataScienceDeep Learning A-Z Visible Input Nodes Visible Output Nodes (Map) X1 X3 X2
- 45. © SuperDataScienceDeep Learning A-Z Visible Input Nodes Visible Output Nodes X1 X3 X2
- 46. © SuperDataScienceDeep Learning A-Z Visible Input Nodes Visible Output Nodes X1 X3 X2 W1,1 W1,2 W1,3
- 47. © SuperDataScienceDeep Learning A-Z ( ; ; ) Visible Input Nodes Visible Output Nodes X1 X3 X2 W1,1 W1,2 W1,3Node1:
- 48. © SuperDataScienceDeep Learning A-Z ( ; ; ) Visible Input Nodes Visible Output Nodes X1 X3 X2 W1,1 W1,2 W1,3Node1: ( ; ; )W2,1 W2,2 W2,3Node2:
- 49. © SuperDataScienceDeep Learning A-Z ( ; ; ) Visible Input Nodes Visible Output Nodes X1 X3 X2 W1,1 W1,2 W1,3Node1: ( ; ; )W2,1 W2,2 W2,3Node2: ( ; ; )W3,1 W3,2 W3,3Node3:
- 50. © SuperDataScienceDeep Learning A-Z ( ; ; ) Visible Input Nodes X3 W1,1 W1,2 W1,3Node1: ( ; ; )W2,1 W2,2 W2,3Node2: ( ; ; )W3,1 W3,2 W3,3Node3: ( ; ; )W4,1 W4,2 W4,3Node4: X2 X1
- 51. © SuperDataScienceDeep Learning A-Z ( ; ; )W1,1 W1,2 W1,3Node1: ( ; ; )W2,1 W2,2 W2,3Node2: ( ; ; )W3,1 W3,2 W3,3Node3: ( ; ; )W4,1 W4,2 W4,3Node4: ( ; ; )W5,1 W5,2 W5,3Node5: ( ; ; )W6,1 W6,2 W6,3Node6: ( ; ; )W7,1 W7,2 W7,3Node7: ( ; ; )W8,1 W8,2 W8,3Node8: ( ; ; )W9,1 W9,2 W9,3Node9: Distance = (𝑥𝑖 − 𝑤1,𝑖)2 = 1.2 X3 X2 X1
- 52. © SuperDataScienceDeep Learning A-Z ( ; ; )W1,1 W1,2 W1,3Node1: ( ; ; )W2,1 W2,2 W2,3Node2: ( ; ; )W3,1 W3,2 W3,3Node3: ( ; ; )W4,1 W4,2 W4,3Node4: ( ; ; )W5,1 W5,2 W5,3Node5: ( ; ; )W6,1 W6,2 W6,3Node6: ( ; ; )W7,1 W7,2 W7,3Node7: ( ; ; )W8,1 W8,2 W8,3Node8: ( ; ; )W9,1 W9,2 W9,3Node9: Distance = (𝑥𝑖 − 𝑤1,𝑖)2 = 1.2 Distance = (𝑥𝑖 − 𝑤2,𝑖)2 = 0.8 X3 X2 X1
- 53. © SuperDataScienceDeep Learning A-Z ( ; ; )W1,1 W1,2 W1,3Node1: ( ; ; )W2,1 W2,2 W2,3Node2: ( ; ; )W3,1 W3,2 W3,3Node3: ( ; ; )W4,1 W4,2 W4,3Node4: ( ; ; )W5,1 W5,2 W5,3Node5: ( ; ; )W6,1 W6,2 W6,3Node6: ( ; ; )W7,1 W7,2 W7,3Node7: ( ; ; )W8,1 W8,2 W8,3Node8: ( ; ; )W9,1 W9,2 W9,3Node9: Distance = (𝑥𝑖 − 𝑤1,𝑖)2 = 1.2 Distance = (𝑥𝑖 − 𝑤2,𝑖)2 = 0.8 Distance = (𝑥𝑖 − 𝑤3,𝑖)2 = 0.4 X3 X2 X1
- 54. © SuperDataScienceDeep Learning A-Z ( ; ; )W1,1 W1,2 W1,3Node1: ( ; ; )W2,1 W2,2 W2,3Node2: ( ; ; )W3,1 W3,2 W3,3Node3: ( ; ; )W4,1 W4,2 W4,3Node4: ( ; ; )W5,1 W5,2 W5,3Node5: ( ; ; )W6,1 W6,2 W6,3Node6: ( ; ; )W7,1 W7,2 W7,3Node7: ( ; ; )W8,1 W8,2 W8,3Node8: ( ; ; )W9,1 W9,2 W9,3Node9: Distance = (𝑥𝑖 − 𝑤1,𝑖)2 = 1.2 Distance = (𝑥𝑖 − 𝑤2,𝑖)2 = 0.8 Distance = (𝑥𝑖 − 𝑤3,𝑖)2 = 0.4 Distance = (𝑥𝑖 − 𝑤4,𝑖)2 = 1.1 X3 X2 X1
- 55. © SuperDataScienceDeep Learning A-Z ( ; ; )W1,1 W1,2 W1,3Node1: ( ; ; )W2,1 W2,2 W2,3Node2: ( ; ; )W3,1 W3,2 W3,3Node3: ( ; ; )W4,1 W4,2 W4,3Node4: ( ; ; )W5,1 W5,2 W5,3Node5: ( ; ; )W6,1 W6,2 W6,3Node6: ( ; ; )W7,1 W7,2 W7,3Node7: ( ; ; )W8,1 W8,2 W8,3Node8: ( ; ; )W9,1 W9,2 W9,3Node9: Distance = (𝑥𝑖 − 𝑤1,𝑖)2 = 1.2 Distance = (𝑥𝑖 − 𝑤2,𝑖)2 = 0.8 Distance = (𝑥𝑖 − 𝑤3,𝑖)2 = 0.4 Distance = (𝑥𝑖 − 𝑤4,𝑖)2 = 1.1 Distance = (𝑥𝑖 − 𝑤5,𝑖)2 = 1.3 Distance = (𝑥𝑖 − 𝑤6,𝑖)2 = 1.0 Distance = (𝑥𝑖 − 𝑤7,𝑖)2 = 0.6 Distance = (𝑥𝑖 − 𝑤8,𝑖)2 = 1.2 Distance = (𝑥𝑖 − 𝑤9,𝑖)2 = 0.9 X3 X2 X1
- 56. © SuperDataScienceDeep Learning A-Z
- 57. © SuperDataScienceDeep Learning A-Z
- 58. © SuperDataScienceDeep Learning A-Z Image Source: Wikipedia
- 59. © SuperDataScienceDeep Learning A-Z
- 60. © SuperDataScienceDeep Learning A-Z
- 61. © SuperDataScienceDeep Learning A-Z
- 62. © SuperDataScienceDeep Learning A-Z
- 63. © SuperDataScienceDeep Learning A-Z
- 64. © SuperDataScienceDeep Learning A-Z
- 65. © SuperDataScienceDeep Learning A-Z
- 66. © SuperDataScienceDeep Learning A-Z
- 67. © SuperDataScienceDeep Learning A-Z
- 68. © SuperDataScienceDeep Learning A-Z
- 69. © SuperDataScienceDeep Learning A-Z
- 70. © SuperDataScienceDeep Learning A-Z
- 71. © SuperDataScienceDeep Learning A-Z
- 72. © SuperDataScienceDeep Learning A-Z
- 73. © SuperDataScienceDeep Learning A-Z
- 74. © SuperDataScienceDeep Learning A-Z
- 75. © SuperDataScienceDeep Learning A-Z
- 76. © SuperDataScienceDeep Learning A-Z
- 77. © SuperDataScienceDeep Learning A-Z
- 78. © SuperDataScienceDeep Learning A-Z Important to know: • SOMs retain topology of the input set • SOMs reveal correlations that are not easily identified • SOMs classify data without supervision • No target vector -> no backpropagation • No lateral connections between output nodes
- 79. © SuperDataScienceDeep Learning A-Z Kohonen's Self Organizing Feature Maps By Mat Buckland (2004?) Link: http://www.ai-junkie.com/ann/som/som1.html Additional Reading:
- 80. © SuperDataScienceDeep Learning A-Z
- 81. © SuperDataScienceDeep Learning A-Z
- 82. © SuperDataScienceDeep Learning A-Z Image Source: Wikipedia
- 83. © SuperDataScienceDeep Learning A-Z Image Sources in order of appearance: Wikipedia, boylelab.org, R-Bloggers, This Course, stackoverflow.com, Viscovery.com, visualcinnamon.com
- 84. © SuperDataScienceDeep Learning A-Z SOM - Creating hexagonal heatmaps with D3.js By Nadieh Bremer (2003) Link: https://www.visualcinnamon.com/2013/07/self-organizing-maps-creating- hexagonal.html Additional Reading:
- 85. © SuperDataScienceDeep Learning A-Z
- 86. © SuperDataScienceDeep Learning A-Z If we choose K = 3 clusters...
- 87. © SuperDataScienceDeep Learning A-Z ...this correct random initialisation would lead us to...
- 88. © SuperDataScienceDeep Learning A-Z …the following three clusters
- 89. © SuperDataScienceDeep Learning A-Z …the following three clusters
- 90. © SuperDataScienceDeep Learning A-Z But what would happen if we had a bad random initialisation ?
- 91. © SuperDataScienceDeep Learning A-Z STEP 1: Choose the number K of clusters STEP 2: Select at random K points, the centroids (not necessarily from your dataset) STEP 3: Assign each data point to the closest centroid That forms K clusters STEP 4: Compute and place the new centroid of each cluster STEP 5: Reassign each data point to the new closest centroid. If any reassignment took place, go to STEP 4, otherwise go to FIN. Your Model is Ready
- 92. © SuperDataScienceDeep Learning A-Z STEP 1: Choose the number K of clusters: K = 3
- 93. © SuperDataScienceDeep Learning A-Z STEP 2: Select at random K points, the centroids (not necessarily from your dataset)
- 94. © SuperDataScienceDeep Learning A-Z STEP 2: Select at random K points, the centroids (not necessarily from your dataset)
- 95. © SuperDataScienceDeep Learning A-Z STEP 3: Assign each data point to the closest centroid That forms K clusters
- 96. © SuperDataScienceDeep Learning A-Z STEP 3: Assign each data point to the closest centroid That forms K clusters
- 97. © SuperDataScienceDeep Learning A-Z STEP 4: Compute and place the new centroid of each cluster
- 98. © SuperDataScienceDeep Learning A-Z STEP 5: Reassign each data point to the new closest centroid. If any reassignment took place, go to STEP 4, otherwise go to FIN.
- 99. © SuperDataScienceDeep Learning A-Z STEP 5: Reassign each data point to the new closest centroid. If any reassignment took place, go to STEP 4, otherwise go to FIN. Your Model is Ready
- 100. © SuperDataScienceDeep Learning A-Z Your Model is Ready STEP 5: Reassign each data point to the new closest centroid. If any reassignment took place, go to STEP 4, otherwise go to FIN.
- 101. © SuperDataScienceDeep Learning A-Z Your Model is Ready STEP 5: Reassign each data point to the new closest centroid. If any reassignment took place, go to STEP 4, otherwise go to FIN.
- 102. © SuperDataScienceDeep Learning A-Z
- 103. © SuperDataScienceDeep Learning A-Z
- 104. © SuperDataScienceDeep Learning A-Z Solution K-Means++
- 105. © SuperDataScienceDeep Learning A-Z
- 106. © SuperDataScienceDeep Learning A-Z
- 107. © SuperDataScienceDeep Learning A-Z Cluster 1 Cluster 2 Cluster 3 C1 C3 C2Pi Pi Pi
- 108. © SuperDataScienceDeep Learning A-Z
- 109. © SuperDataScienceDeep Learning A-Z Cluster 1 Cluster 2 Cluster 3 C1 C3 C2Pi Pi Pi
- 110. © SuperDataScienceDeep Learning A-Z
- 111. © SuperDataScienceDeep Learning A-Z Cluster 1 C1 Pi
- 112. © SuperDataScienceDeep Learning A-Z Cluster 1 Cluster 2 C2 Pi Pi C1
- 113. © SuperDataScienceDeep Learning A-Z Cluster 1 Cluster 2 Cluster 3 C1 C3 C2Pi Pi Pi
- 114. © SuperDataScienceDeep Learning A-Z
- 115. © SuperDataScienceDeep Learning A-Z The Elbow Method
- 116. © SuperDataScienceDeep Learning A-Z The Elbow Method Optimal number of clusters