AI/ML/DL: Introduction to Deep Learning with Cognitive ToolKit
- 1. Introduction to Deep Learning with Cognitive Toolkit (CNTK) Marvin Heng Blog : http://hmheng.pinsland.com Twitter : @hmheng YouTube: http://bit.ly/hmheng_yt SlideShare: http://bit.ly/hmheng_ss Github: https://github.com/hmheng HMHENG. PINSLAND.COM Microsoft Cognitive Service Microsoft Cognitive ToolKit
- 2. Who is Marvin? Software + Web Developer AI Enthusiast Blog : https://hmheng.pinsland.com Twitter : @hmheng YouTube: http://bit.ly/hmheng_yt SlideShare: http://bit.ly/hmheng_ss Github: https://github.com/hmheng Scan & Subscribe! hmheng.pinsland.com HMHENG. PINSLAND.COM
- 4. Started from AI, ML then Deep Learning… HMHENG. PINSLAND.COM
- 5. What is Deep Learning? A Subfield of Machine Learning Self Learning More complex than Machine Learning Neural Networks mimics the neural systems in our brains. HMHENG. PINSLAND.COM
- 6. What is Deep Learning? HMHENG. PINSLAND.COM Credit: Nvidia
- 7. What is Microsoft Cognitive ToolKit (CNTK)? HMHENG. PINSLAND.COM
- 9. Microsoft Cognitive Toolkit (CNTK)? • Open-Source Development on GitHub (released 2.5 on 15th Mar 18) • Fast • Optimized for GPUs & Libraries • Best-in-Class multi-GPU/Multi-server algorithms • Flexible • BrainScript, C#/.NET, C++, Python, Network Definition Language, Model Editing Language(MEL) • Linux, Windows and Docker Container • Ease of Use & Wide-range of Support • FF-NN, CNN, RNN, LSTM, DSSM,… HMHENG. PINSLAND.COM
- 10. Microsoft Cognitive Toolkit (CNTK) HMHENG. PINSLAND.COM Arbitrary Neural Networks Expression Composing simple building blocks into complex computational networks. Production Ready State-of-the-art accuracy Efficient Scalable to Multi-GPU/Multi- Server
- 11. Microsoft Cognitive Toolkit HMHENG. PINSLAND.COM Design Number of layers, number of units Dependent on concept complexity Experimentation often required Activation Function Enables non- linear models Examples include sigmoid, hyperbolic tangent, ReLU, softmax, … Loss Function Measure the performance during training Examples include cross entropy Training Mechanism for getting optimal weights Overfitting Stop models from over-optimizing on training data set.
- 12. Microsoft Cognitive Toolkit (CNTK) HMHENG. PINSLAND.COM Windows CPU|GPU NuGet – CPU | NuGet – GPU | NuGet – UWP CPU Linux CPU|GPU|Docker – CPU| Docker – GPU Requires OpenMPI MacOS Docker – CPU Cloud Data Science Virtual Machine | Azure Notebooks Azure Batch AI Training (CPU/GPU) | Project BrainWave (FPGA)
- 13. Sharing Marvin Heng Blog : http://hmheng.pinsland.com Twitter : @hmheng YouTube: http://bit.ly/hmheng_yt SlideShare: http://bit.ly/hmheng_ss Github: https://github.com/hmheng Microsoft Cognitive Service HMHENG. PINSLAND.COM Microsoft Cognitive ToolKit
- 14. What is Deep Learning HMHENG. PINSLAND.COM
- 15. Open Neural Network Exchange HMHENG. PINSLAND.COM • An open format to represent deep learning models. • Collaboration between Microsoft, AWS and Facebook. • Supported by community of partners like Intel, ARM, AMD, nvidia, etc. • A share library with Caffe2, PyTorch, MXNet, Cognitive Toolkit. • Permissive MIT license and no patents.
- 16. ONNX Aims HMHENG. PINSLAND.COM • Allow interoperability between frameworks • CNTK, Caffe2 and PyTorch • Allow hardware vendor to focus on one IR in their backend optimization • Allow training in one toolkit and deploy in another.
- 17. ONNX Aims HMHENG. PINSLAND.COM • Allow interoperability between frameworks • CNTK, Caffe2 and PyTorch • Allow hardware vendor to focus on one IR in their backend optimization • Allow training in one toolkit and deploy in another.
- 18. More… CTNK.ai Tutorial, Source Code, Example @ https://github.com/Microsoft/CNTK ONNX.ai HMHENG. PINSLAND.COM
- 19. Marvin Heng Blog : http://hmheng.pinsland.com Twitter : @hmheng YouTube: http://bit.ly/hmheng_yt SlideShare: http://bit.ly/hmheng_ss Github: https://github.com/hmheng Thank You HMHENG. PINSLAND.COM
- 20. Introduction to Deep Learning with Cognitive Toolkit (CNTK) Marvin Heng Blog : http://hmheng.pinsland.com Twitter : @hmheng YouTube: http://bit.ly/hmheng_yt SlideShare: http://bit.ly/hmheng_ss Github: https://github.com/hmheng HMHENG. PINSLAND.COM Microsoft Cognitive Service Microsoft Cognitive ToolKit