Semi supervised learning machine learning made simple
- 2. Context Supervised learning is amazing. It has been used to solve a whole bunch of problems. It is also very intuitive when it comes to the idea. With lots of data and a directed learning mechanism, it makes sense that our agents should be able to learn specific things. Labeling/annotating data can get extremely expensive. For eg. medical images are typically very expensive to annotate (check out the video on SinGAN-Seg for more details). There is a lot of unlabeled data that is lying around, which can be used for cheap.
- 3. Enter Semi-Supervised Learning (Prequel) Don’t confuse this with Self-Supervised Learning (I make that mistake a lot.) How can we use the simpler concept and higher performance of Supervised Learning with the cheap, large scale data availability of using Unsupervised Learning? That’s a good question. How would you try to use both processes to leverage the strengths of both types (pause the video and think for a second)
- 4. An Image to help you think
- 5. SSL: Overview We take a (relatively) small amount of labelled data. Use that to learn. Then take the unlabeled data use your model to figure out the (pseudo) labels. Use that to train our models further. (Optional) Rinse and Repeat. Do some other things like add noise if you have the resources/want to get creative
- 6. Transduction vs Induction Often you will see learning presented as either transductive or inductive learning. Transductive learning: I give you specific examples and ask you to predict specific examples. Inductive learning: I give you examples, and you figure out a pattern. Use that pattern to predict samples. Sound similar? Think back to the video on discriminative vs generative classification.
- 7. Assumptions used in Semi Supervised Learning Take a second to pause. Think about the kinds of assumptions you make when implementing SSL. This is a good habit when dealing with ML/Data. Understanding your implicit assumptions gives you a way to figure out how to improve. SSL algorithms have 3 assumptions. To use SSL we need to make atleast one: ● Continuity assumption ● Cluster assumption ● Manifold assumption
- 8. Assumptions- Continuity Wikipedia: Points that are close to each other are more likely to share a label. This is also generally assumed in supervised learning and yields a preference for geometrically simple decision boundaries. In the case of semi-supervised learning, the smoothness assumption additionally yields a preference for decision boundaries in low-density regions, so few points are close to each other but in different classes.
- 9. Assumptions: Cluster Wikipedia: The data tend to form discrete clusters, and points in the same cluster are more likely to share a label (although data that shares a label may spread across multiple clusters). Gives rise to feature learning with clustering algorithms.
- 10. Assumptions: Manifold The data lie approximately on a manifold of much lower dimension than the input space. Learning the manifold using both the labeled and unlabeled data can avoid the curse of dimensionality. Then learning can proceed using distances and densities defined on the manifold.
- 11. SSL-> Most human way to learn? Semi Supervised Learning is often compared to how we humans learn. Think back to your favorite skills. How did you learn to do them? The SSL formula: 1) Have the learner train through specific guided examples with lots of detail. 2) As the learner becomes more proficient, we start using mixing in samples that aren’t fully worked out. Learner must work through them on their own. 3) If we really want to check proficiency of the learner, we throw in incorrect/broken examples, and have the student correct the mistakes. Let’s see how this squares with our learning process. Let’s take 3 skills I learnt: Math, French, and Brazilian JiuJitsu.
- 12. Reach out to me Check out my other articles on Medium. : https://machine-learning-made- simple.medium.com/ My YouTube: https://rb.gy/88iwdd Reach out to me on LinkedIn: https://www.linkedin.com/in/devansh-devansh-516004168/ My Instagram: https://rb.gy/gmvuy9 My Twitter: https://twitter.com/Machine01776819 My Substack: https://devanshacc.substack.com/ Live conversations at twitch here: https://rb.gy/zlhk9y Get a free stock on Robinhood: https://join.robinhood.com/fnud75