AutoML - The Future of AI
- 1. AutoML The Future of AI What is AutoML? What OneClick.ai brings to AutoML?
- 2. Ning Jiang Co-founder and CTO of OneClick.ai. Previously Dev Manager at Microsoft Bing, Ning has over 17 years of R&D experience in AI for ads relevance, local search, and cyber security.
- 3. OneClick.ai, World’s first AutoDL platform
- 5. Never enough experienced data scientists
- 6. Unpredictability Data acquisition & cleansing 1-2 WEEKS Feature engineering 1-3 MONTHS Feature selection 1-2 WEEKS Model evaluation 1-2 WEEKS Model training and tuning 1-2 MONTHS 05 01 02 03 04 Iterations
- 7. 60% AI projects failed in 2017
- 8. The Rise of AutoML
- 9. What is AutoML? Progressive Automation of Machine Learning
- 10. Academic research Long Beach NIPS 2017 Stockholm ICML 2018 Skopje ECMLPKDD 2017 Paris COSEAL’18 Seattle AutoML 2018 Nanjing PRICAI 2018
- 11. $57.6B* $11.5B 20% *International Data Corporation (IDC) Industry projection by 2021
- 15. Data Cleansing Feature Extraction Feature Selection Model Selection & Tuning Missing values Data types Anomalies Text Encoding Data partition Numeric Discrete Textual/Images Time-series Linear proj. N/L proj. Reduction Selection Hyper-params Training Cross-features Machine Learning Framework
- 16. Infer Data Types
- 17. Cross Features Textual cross features ● Text similarity ● N-gram set relations ● Word embedding diff. ● Substrings ● Fuzzy match ● ... Numeric cross features ● a - b ● |a - b| ● a > b ● a * b ● a / b ● (a - b)**2 ● ...
- 18. Feature Selection Feature Selection Stepwise Regression Feature Importance Random Projection Locality-Sensitive Hashing Random Projection Linear Projection PCA LDA Non-linear Projection Auto-Encoder GDA
- 19. Model Selection & Tuning Model Selection ● Brute force Hyperparameter Tuning ● Grid search ● Random search ● Bayes Optimization
- 20. AutoML Players Structured data only Structured + Unstructured data Model-based search Grid/random search
- 21. Other Perspectives of AutoML Modeling which we just covered :-) Data Imports File formats, data bases, Hadoo, clouds, NFS Deployment API serving, live updates, A/B testing, batch serving, scalability and failure recovery.
- 23. Data Cleansing Encoding Model Architecture & Training Missing values Data types Anomalies Text encoding Partitioning Scalar Sequence Tensors Loss Deep Learning Framework
- 25. Non-linear architectures (e.g. DenseNet)
- 26. Model Architecture Validation setTraining set Average loss on the validation set Neural Architecture Search (NAS) Controller Updating model architectures in response to the validation feedback Loss Training On the training set Validation On the validation set
- 27. Controller Prune Stop developing less promising branches Generate Enumerate model architectures on a predefined search space Reinforcement Use Policy Gradient to update RL models and stochastic sampling for model instantiation
- 28. AutoDL Players Less restrictive on input data Microsoft Custom Vision Adaptive to different applications Application-specific
- 29. Challenges 1. Solutions are application-specific 2. New solutions for new applications 3. Heavily depends on human knowledge on the application 4. Assumes a linear architecture with skip connections 5. Cold start 6. Slow to converge
- 30. Elias - OneClick.ai’s AutoML Engine More details in the coming tech talks
- 31. Data Cleansing Feature Extraction Feature Selection Model Selection & Tuning Missing values Data types Anomalies Encoding Data partition Numeric Discrete Textual/Image Time-series Linear proj. N/L proj. Reduction Selection Hyperparameters Training Cross-features Machine Learning Framework
- 32. Data Cleansing Feature Extraction Feature Selection Model Selection & tuning ML Model Converted to DAG
- 33. Data Cleansing Encoding Model Architecture & Training Missing values Data types Anomalies Text encoding Partitioning Scalar Sequence Tensors Loss Deep Learning Framework
- 34. Loss Converted to DAG Encoding Model ArchitectureData Cleansing
- 35. Individual losses on the validation set The Elias Engine Validation set Model Architecture Validation setTraining set Controller Updatig model architectures in response to the validation feedback Loss Training On the training set Validation On the validation set
- 36. Where it stands out? 1. Works with both Deep Learning and traditional Machine Learning 2. Learning arbitrary DAGs 3. feature extraction coordinating with model architecture/selection 4. Shared Controller RL models to avoid cold start 5. Fewer models to train (20-30 models vs. thousands)
- 37. Automated Feature Engineering Automated Model Selection & Tuning Automated ML/DL Engine Elias #US patent pending# Time-series Forecasting Deep Learning helps find more complex patterns in and between time series than any data scientists Unstructured Data Supports numeric and categorical data, text, images, time-series, and any mix of them Performance Custom DL models often lead to better performance Functions Algorithms Versatility & Performance Automated Neural Architecture Search OneClick.ai Platform
- 38. Use AI to Build AI Developed world’s first automated DL engine OneClick.ai Incorporated in Belelvue, WA Early 2017 Seeds Round led by Sinovation 2017/4 2018/7 OneClick.ai on AWS Public beta launched 2017/11 OneClick.ai Enterprise On-premise deployment for enhanced privacy and data security Roadmap Open to Public One step closer to our goal of making AI accessible to everyone 2018/8
- 39. Zero Coding and User Friendly
- 40. AI Leaderboard and One-Click Deployment
- 41. Thank You Free Sign-up Scan to get double tokens for a limited time, or sign up at http://www.oneclick.ai using promo code AUTOML4K Follow us on Twitter http://twitter.com/oneclickai