Machine Learning with Small Data
Download as PPTX, PDF0 likes340 views
The presentation discusses the challenges and methodologies of machine learning with small data, emphasizing the limitations of big data approaches and the need for innovative techniques such as data augmentation and adversarial data generation. Key topics include the importance of semantic representations, inductive transfer learning, and effective experimental design to enhance model performance with limited annotated data. Additionally, the author highlights the relevance of understanding the unique characteristics of small datasets in various applications.
Machine Learning with Small Data
- 1. Machine Learning with Small Data John C. Liu, Ph.D. CFA June 18, 2019 Twitter: @drjohncliu
- 2. Disclaimer THE INFORMATION SET FORTH HEREIN HAS BEEN OBTAINED OR DERIVED FROM SOURCES GENERALLY AVAILABLE TO THE PUBLIC AND BELIEVED BY THE AUTHOR TO BE RELIABLE, BUT THE AUTHOR DOES NOT MAKE ANY REPRESENTATION OR WARRANTY, EXPRESS OR IMPLIED, AS TO ITS ACCURACY OR COMPLETENESS. THE INFORMATION IS FOR EDUCATIONAL PURPOSES ONLY AND IS NOT INTENDED TO BE USED AS THE BASIS OF ANY BUSINESS OR INVESTMENT DECISIONS BY ANY PERSON OR ENTITY. ALL OF THE INFORMATION CONTAINED IN THE PRESENTATION IS SUBJECT TO FURTHER MODIFICATION AND ANY AND ALL FORECASTS, PROJECTIONS OR FORWARD-LOOKING STATEMENTS CONTAINED HEREIN SHALL NOT BE RELIED UPON AS FACTS NOR RELIED UPON AS ANY REPRESENTATION OF FUTURE RESULTS WHICH MAY MATERIALLY VARY FROM SUCH PROJECTIONS AND FORECASTS.
- 3. Roadmap • Introduction • Big Data Revolution • What about Small Data? • Dealing with Reality – Semantic/Contextualized Representations – Experimental Design – Adversarial Data Generation • Conclusion
- 4. Big Data Source: Bernard Marr & Co.
- 6. Data is the New Oil Source: James Corbett
- 7. More Data = Better Models Source: Andrew Ng
- 8. What’s Wrong With this Picture Train Set? Source: The Simpsons
- 9. Data Annotation is Expensive Source: Jia, Yangqing. 2014). Learning Semantic Image Representations at a Large Scale.
- 10. Annotator (Dis)Agreement? Source: Stephen Yip & Chintan Parmar
- 12. • 14 million images • 20,000 categories • 25 Human Years to annotate! Source: Li Fei-Fei. (2010). ImageNet: Crowdsourcing, benchmarking & other cool things
- 13. Reality = Small Annotated Data Source: NASA/JPL/UCSD/JSC
- 14. Ways to Deal with Small Data • AWS Mechanical Turk (e.g., ImageNet) • CrowdFlower/Figure8/Appen • Hire SMEs • Data Augmentation/Synthetic Generation (SMOTE)
- 15. Synthetic Minority Oversampling Nearest Neighbor Algorithm Source: Bart Baesens
- 17. Not All Data is Created Equal https://pypi.org/project/imbalanced-learn/ Source: Rishabh Misra
- 18. Training a Cat/Dog Classifier • Which training samples are more useful? Photograph:American Kennel Club Photograph:Atchoumfan Photograph:Sujoy Roychowdhury
- 19. Oncology Text Classifier Which training samples are more useful? 1. Left medial foot and ankle pain and swelling. Plantar metatarsal pain for 5 weeks. No known trauma. 2. Dorsal right medial upper back pain for 10 weeks. Right parotid mass. 3. History pancreatic cancer. Status post aortic chemotherapy and Whipple procedure
- 21. Machine Learning with Small Data What Data Scientists Should Care Most About
- 22. Kid Saw This in a Toy Store Tiger Photograph:Nat & Jules Brown
- 23. At the Zoo a Few Weeks Later Tiger Photograph:Skip O’Rourke
- 24. Inductive Transfer Learning • Learning new tasks using knowledge learned from other tasks Source: Dipanjan Sarkar
- 25. Semantic Image Representations Source: Jia, Yangqing. 2014). Learning Semantic Image Representations at a Large Scale.
- 26. Word Embeddings Corpus Docs Sentences Words Vectors Word embeddings encode semantic relationships learned from corpus.
- 27. Word2Vec Context too Narrow Source: Kamath, Uday, Liu, John & Whitaker, James. (2019). Deep Learning for NLP and Speech Recognition.
- 28. Neural Language Model Source: Kamath, Uday, Liu, John & Whitaker, James. (2019). Deep Learning for NLP and Speech Recognition.
- 29. ELMO Embeddings from Language Models – Bidirectional Language Models (forward & backward) – Using LSTMs – Concatenate hidden layers Source: Karan Purohit
- 30. Concept Embeddings • RDF2Vec Source: Kamath, Uday, Liu, John & Whitaker, James. (2019). Deep Learning for NLP and Speech Recognition.
- 31. Did We Solve the Tiger Problem? • Generalize with only a single label? (One-Shot Learning) • If I described a lion, would you recognize one if you never ever saw one? (Zero-Shot Learning) • Did the chicken come before the egg, or vice versa? (Causality)
- 32. THE WORLD IS NOT RANDOM INHERENT STRUCTURE EXISTS Source: NASA/JPL/UCSD/JSC
- 33. Not Random • Each CIFAR-10 image = 32x32 pixels by 3x256 colors • Number of possible permutations = 786432! Source: Krizhevsky, Alex. (2009). Learning Multiple Layers of Features from Tiny Images.
- 34. Not a Possible Permutation Source: Goodfellow, Ian. (2016). Generative Adversarial Nettworks.
- 35. How many Laws of Physics are sufficient to describe motion? Photograph: Richard Jognston
- 36. Bayesian Networks Factorizing Joint PDF Source: Sato, Renato and Sato, Graziela. (2015). Probabilistic graphic models applied to identification of diseases.
- 37. Adversarial Data Generation Source: Mino, Ajkel & Spanakis, Gerasimos. (2018). LoGAN: Generating Logos with a Generative Adversarial Neural Network Conditioned on color.
- 38. Last Word Photograph: Gregor Schmidt
- 39. My New Book A comprehensive resource that builds up from elementary deep learning, text, and speech principles to advanced state-of- the-art neural architectures. On Amazon, BN, Springer https://www.amazon.com/Deep-Learning- NLP-Speech-Recognition/dp/3030145956
- 40. Thank you. AI/ML Solutions to Solve Business Problems