Visual concept learning
1 like613 views
The document presents an overview of visual concept learning, particularly focusing on few-shot learning where machine learning models aim to learn concepts from very few instances. It outlines common challenges, popular datasets, different approaches such as siamese and memory-augmented networks, and various related works in the field. The author, Vaibhav Singh, shares insights into methods and references significant studies in this domain.
Visual concept learning
- 1. Visual Concept Learning Vaibhav Singh Machine Learning Engineer https://www.linkedin.com/in/techytux/
- 2. Hello! I am Vaibhav You can contact me on linkedin https://www.linkedin.com/in/techytux/ ● Payment Fraud @ Klarna ● Previously worked in Moderation Team @ OLX Group ● Software Engineer turned Machine Learning Engineer
- 3. AGENDAPopular datasets Paper summaries What is few shot learning Approaches and related work
- 4. What the heck is this talk about Zero / few shot learning 1
- 5. ““Learning how to represent objects (e.g. images) given only a few instances of each object! ”
- 9. Lamprey
- 10. Star nosed mole
- 12. ???
- 13. Classify a reference image into one of N candidate classes, given K representatives of each class. For example, 2-shot 4-way classification: Concept : K Shot N Way
- 14. Humans can learn new concepts fast without much training data But for Deep Learning Networks we need hundreds or even thousands of examples What did we learn
- 15. Motivation for few shot learning ● Machine learning fails to generalize when little supervised information is available. ● Getting labeled data can be expensive. ● Adding domain information / features for each task is cumbersome and does not scale well. ● Gradient based methods like Adagrad, ADAM, Adadelta, Nesterov momentum weren’t designed to perform well under the constraint of a set number of examples ● Random initialization of weights hurts the networks ability to converge with few updates
- 16. What are commonly used datasets for few shot learning Popular Datasets 2
- 17. Created by Lake in 2015 105 x 105 Grayscale Images 1623 chars from 50 alphabets each with 20 handwritten examples Omniglot
- 18. Mini Imagenet 100 real world classes, each with 600 examples
- 20. Approaches to do low shot learning Data Based ● Siamese networks (Koch et al. 2015) ● Memory Augmented Networks (Santoro et al. 2016) ● Matching networks (Vinyals et al. 2016) ● Meta Optimization (Ravi & Larochelle 2017) Metric or Parameter Based ● Tap into external data sources ● Generate new labels using semi supervised learning. Propagate labels using a distance metric ● Generate new examples using GAN’s. For example: Create new images of birds from different angles if there are examples present in the training set
- 21. Related Work ● Fei Fei li -- Bayesian framework -- previously learned classes can be leveraged to help forecast future ones. Fei Fei et al, 2003; Fei Fei et al, 2006 ● Lake -- HBPL -- Concept learning through probabilistic programming ○ Penstrokes as domain information Lake et al. 2015
- 23. Siamese Neural Networks for One Shot Image Recognition Koch et al. 2015
- 24. Verification Networks Koch, Zemel, Salak., 2015
- 25. Koch, Zemel, Salak., 2015 Network Architecture
- 26. Evaluation Koch, Zemel, Salak., 2015
- 27. Concept : Episodes 1 2 3 1 4
- 28. Concept : Meta Learning
- 29. Concept : Neural Turing Machine Graves et al. 2014
- 30. One shot learning with Memory Augmented Networks Santoro et al. 2016
- 31. Memory Augmented Networks Santoro et al. 2016
- 32. Evaluation
- 33. Matching Networks for One Shot Learning Vinyals et al. 2017
- 34. Matching Networks ● Matching nets is built on top of ○ metric learning on Deep Neural features and ○ memory networks, basically external memory with attention mechanism used to access the memory ● Proposes that training and test conditions must match
- 35. Matching Networks Vinyals et al. 2016 ● Network is shown sample set S and asked to give probability that x(hat) is an instance of class in set S.
- 36. Evaluation Vinyals et al. 2016
- 37. Evaluation Vinyals et al. 2016
- 38. Optimization as a model for few shot learning Ravi & Larochelle 2017
- 39. ● Instead of using gradient based optimization they propose ○ LSTM based meta learner to learn exact optimization algorithm used to train another neural network classifier ● General initialization of network so that it converges faster Meta Optimization
- 40. Network structure Rave & Larochelle 2017
- 41. Evaluation Rave & Larochelle 2017
- 42. References ● One shot Imitation Learning -- Duan et al 2017 ● Low-shot Visual Recognition by Shrinking and Hallucinating Features -- Hariharan, Girshick 2017 ● NTM -- https://github.com/tristandeleu/ntm-one-shot/ ● Siamese Networks -- https://sorenbouma.github.io/blog/oneshot/