Meta-Learning Presentation
- 2. Goal of AI is to outperform the abilities of human beings. Necessary to have large, balanced and labeled datasets. Unrealistic to expect the perfect datasets for all the tasks. It can also involve human efforts and time to create one.
- 3. Learning to Learn. Meta-Learning intends to train the model with a large dataset, containing varied classes. While in the testing phase, the model is tested with a novel set where the number of classes will not be more than 5. The main objective of Meta-Learning is to find the best hyperparameters and model weights for the model so that it can generalize more to the novel set. Few-shot or low-shot and n-shot learning are meta- learning techniques with a minimalistic dataset and n can be between 0 to 5.
- 4. Data Augmentation Based Embedding Based Optimization Based Semantic Based
- 5. Data Augmentation in supervised learning involves techniques like scaling, cropping, rotating. LaSO: Label-Set Operations networks. Recognition by Shrinking and Hallucinating Features. Learning via Saliency-guided Hallucination. Low-Shot Learning from Imaginary Data. A Maximum-Entropy Patch Sampler. Image Deformation Meta-Networks.
- 6. Metric-based technique, uses data transformed to lower dimension representation, clustering, and comparison based on distance (metric). Relation Network. Prototypical Network. Learning in localization of realistic settings. Cross-Validation using Leave-one-Out Approach. Learning for Semi-Supervised Classification. Transferable Prototypical Networks. Task dependent adaptive metric learning. Representative-based metric learning. Task-Aware Feature Embedding.
- 7. Use meta-optimizers during the training process. A memory network, LSTM, RNN, a holistic gradient descent optimizer are some of the meta- optimizers. Memory Augmented Networks based Learning. Model Agnostic based Meta-Learning. Task-Agnostic Meta-Learning. Meta-SGD.
- 8. Popular in zero-shot learning. Learning with Multiple Semantics. Learning via Aligned Variational Autoencoders (VAE).
- 9. The paper provides a comparison of different model performances on the two prominent datasets Omniglot and MiniImageNet. Based on the information given in the paper, on using both Data Augmentation and embedding learning techniques the models performed well for the Omniglot dataset. But the same techniques produce lower accuracies comparatively for the MiniImageNet dataset. Interestingly, the performance of the current state-of-the- art models is very low compared to a toddler’s performance. There is a scope for improvement in the future. A hybrid model using the mentioned techniques like data- augmentation, embedding techniques, using semantic information, etc., all at once, can be used in the future.