Scalable Deep Learning on AWS using Apache MXNet (May 2017)
5 likes790 views
1) The document discusses deep learning and Apache MXNet. It provides an overview of deep learning applications like image recognition and natural language processing. 2) It introduces Apache MXNet as an open-source deep learning framework that provides simple syntax, high performance across GPUs, and is optimized for AWS. 3) It demonstrates MXNet through examples like training a neural network on MNIST data and using object detection with MXNet on a Raspberry Pi. Additional resources for learning MXNet and deep learning on AWS are also provided.
Scalable Deep Learning on AWS using Apache MXNet (May 2017)
- 1. © 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Julien Simon, Principal Technical Evangelist, AWS @julsimon DEVit, Thessaloniki, 20/05/2017 Scalable Deep Learning on AWS using Apache MXNet
- 2. Agenda • AI: The Story So Far • Applications of Deep Learning • Apache MXNet Overview • Apache MXNet API • Code and Demos • Tools and Resources
- 4. • Machine Learning is now a commodity, but still no HAL in sight • Traditional Machine Learning doesn’t work well with problems where features can’t be explicitly defined • So what about solving tasks that are easy for people to perform, but hard to describe formally? • Is there a way to get informal knowledge into a computer? Where is HAL?
- 5. • Universal approximation machine • Through training, a neural network discovers features automatically • Not new technology! • Perceptron - Rosenblatt, 1958 image recognition, 20x20 pixels • Backpropagation - Werbos, 1975 • They failed back then because: • Data sets were too small • Solving large problems with fully connected networks required too much memory and computing power, aka the Curse of Dimensionality Neural Networks, Revisited
- 6. Everything is digital: large data sets are available • Imagenet: 14M+ labeled images - http://www.image-net.org/ • YouTube-8M: 7M+ labeled videos - https://research.google.com/youtube8m/ • AWS public data sets - https://aws.amazon.com/public-datasets/ The parallel computing power of GPUs make training possible • Simard (2005), Ciresan (2011) • State of the art networks have hundreds of layers • Baidu’s Chinese speech recognition: 4TB of training data, +/- 10 Exaflops Cloud scalability and elasticity make training affordable • Grab a lot of resources for fast training, then release them • Using a DL model is lightweight: you can do it on a Raspberry Pi Why It’s Different This Time
- 7. Applications of Deep Learning
- 8. ImageNet Large Scale Visual Recognition Challenge (ILSVRC) Same breed? Humans: 5,1%
- 9. Amazon Echo
- 12. Apache MXNet Programmable Portable High Performance Near linear scaling across hundreds of GPUs Highly efficient models for mobile and IoT Simple syntax, multiple languages Most Open Best On AWS Optimized for deep learning on AWS Accepted into the Apache Incubator
- 13. 0! 4! 8! 12! 16! 1! 2! 4! 8! 16! Ideal Inception v3 Resnet Alexnet 91% Efficiency Multi-GPU Scaling With MXNet
- 14. Ideal Inception v3 Resnet Alexnet 88% Efficiency 0! 64! 128! 192! 256! 1! 2! 4! 8! 16! 32! 64! 128! 256! Multi-Machine Scaling With MXNet
- 15. Apache MXNet API
- 16. Demo #1 – Training MXNet on MNIST https://medium.com/@julsimon/training-mxnet-part-1-mnist-6f0dc4210c62 https://github.com/juliensimon/aws/tree/master/mxnet/mnist
- 17. Demo #2 – Object Detection on a Raspberry Pi https://medium.com/@julsimon/an-introduction-to-the-mxnet-api-part-6-fcdd7521ae87 GoPiGo Arduino Yùn AWS IoT MQTT@CallMeJohnnyPi
- 18. One-Click GPU or CPU Deep Learning AWS Deep Learning AMI Up to~40k CUDA cores Apache MXNet TensorFlow Theano Caffe Torch Keras Pre-configured CUDA drivers, MKL Anaconda, Python3 Ubuntu and Amazon Linux + CloudFormation template + Container Image
- 19. Additional Resources MXNet Resources • MXNet Blog Post | AWS Endorsement • Read up on MXNet and Learn More: mxnet.io • MXNet Github Repo • MXNet Recommender Systems Talk | Leo Dirac AWS Resources • Deep Learning AMI |Amazon Linux • Deep Learning AMI | Ubuntu • CloudFormation Template Instructions • Deep Learning Benchmark • MXNet on Lambda • MXNet on ECS/Docker
- 20. Thank You! Julien Simon, Principal Technical Evangelist, AWS @julsimon