Self driving computers active learning workflows with human interpretable vector spaces (2)
Download as ODP, PDF1 like597 views
Founded in 2014, Skymind has raised $6.5 million and serves over 25 large enterprises and governments, focusing on deep learning and AI solutions. The document discusses various technical concepts, including human-in-the-loop approaches, the use of word embeddings, transfer learning, and autoencoders, as well as their applications in data analysis and visualization. It emphasizes the importance of addressing issues like data imbalance and the learning process of autoencoders to improve AI models.
Self driving computers active learning workflows with human interpretable vector spaces (2)
- 2. Founded 2014 Funding 6.5 Million USD Investors Y Combinator, Tencent Customers Over 25 large enterprises and governments, DL4J gets over 160,000 donloads a month Employees Around 40 (mostly engineers, includes Ph. Ds) About Skymind
- 3. Production is part of your Training Set • Edge cases exist in your data • Imbalanced classes are a problem • Data/Trends can change over time • Expanded scope of problem due to unforeseen difficulties or new business problem
- 4. Why is this Important?
- 6. What can we do?
- 7. Human in The Loop • Allow humans to have input • Use Deep Learning to create friendly vector spaces to inspect • Use probabilities from models and decision boundaries to control behavior • More thorough data analysis to understand outliers • Human helps update models
- 8. Friendly Vector Spaces • Word Embeddings • Transfer Learning Feature Extractors • Autoencoder Bottlenecks as an embedding space
- 9. Word Embeddings Fast Text Word2vec
- 10. Word Embeddings: A 2 minute primer • Do an SGD variant on co located pairs of words minimizing a distance function between the 2 words • Run sparse SGD updates on various rows (each word is a row) • Various ways of computing accuracy
- 12. Transfer Learning: A 2 minute primer • Download a pre-trained neural net architecture (usually cnn) • Tune final Layer if doing classification • Otherwise just use feature extractor as a compression algorithm for high dimensional images • Intuition is similar to layerwise pretraining of old
- 13. Join raw data Transform Feed groups into autoencoder and save reconstruction error of center Input Data Reconstruction Autoencoders 13
- 14. Learns to cover more of vector space over time as reconstruction error goes down Auto-Encoder learning process 14
- 15. Auto-Encoders: A 2 minute primer • Minimize KL Divergence (see previous slide) between reconstruction and input • Learn a bottleneck low dimensional vector for use in other algos or visualizations
- 16. Different kinds of auto-encoders Variational Autoencoders GANs (there are 1000s I am not covering them all here) https://github.com/kozi str/Awesome-GANs
- 17. Commonalities • Latent vector spaces automatically learned via SGD • Low dimension vectors meant to be consumed externally
- 18. Various ways of consuming
- 19. Consuming • Kmeans • KNN Algos • Visualization (UMap,Tsne, LargeVis)
- 20. KMeans
- 21. Using Kmeans • Tune with target number of classes • Use as a way of seeing how the neural net groups your data in to classes • Pseudo labeling mechanism • Key: Run on latent vector space
- 22. KNN
- 23. Various kinds of KNN • RPTrees (neighbor of my neighbors is also likely my neighbor) • VPTrees (Segment the space in to quadrants, repeated updates using trees to index vector space • KDTrees
- 24. RPTrees
- 25. VPTrees
- 26. KDTrees
- 27. Visualization
- 28. Visualization • UMap • Barnes Hut Tsne • LargeVis • All dimensionality reduction algorithms focused on building a coordinate space via similarities in the vector space
- 30. Dataset troubleshooting • Examine class imbalance • Weighted loss functions • Resampling • Decision thresholds (mainly for binary classification)
- 32. We achieve this workflow with Discover
Editor's Notes
- #3: スカイマインド株式会社カントリーマネージャー 堀と申します。親会社のスカイマインドインクは2014年に設立、ディープラーニングに特化したソフトウェア会社で、サンフランシスコが本社です。日本を中心としたアジア・アメリカ・欧州のファーストクラスエンジニアを多数抱えています。 <number>
- #4: <number>
- #5: <number>
- #6: <number>
- #7: <number>
- #8: <number>
- #9: <number>
- #10: <number>
- #11: <number>
- #12: <number>
- #13: <number>
- #14: <number>
- #15: <number>
- #16: <number>
- #17: <number>
- #18: <number>
- #19: <number>
- #20: <number>
- #21: <number>
- #22: <number>
- #23: <number>
- #24: <number>
- #25: <number>
- #26: <number>
- #27: <number>
- #28: <number>
- #29: <number>
- #30: <number>
- #31: <number>
- #32: <number>
- #33: 「ファーストＡｐｐ」 は、ディープラーニングデータを可視化して「診断する」ツールです。これには2つの特徴があります。 ①まず、ＵＩがシンプルでわかりやすい データサイエンティストでなくても、データをインポートするだけで、データの品質を瞬時に読み取れます。 ②次に、安価に拡張できる 24時間365日サポート付きの統合プラットフォーム上のアップですから、安心してスケールアップできます。 <number>