Using MXNet to Train and Deploy your Deep Learning Model
- 1. Using Apache MXNet to Train and Deploy your Deep Learning model Qing Lan PPMC Member of Apache MXNet T r a c k : M a c h i n e L e a r n i n g
- 2. Agenda • Introduction to Deep Learning • Introduction to Apache MXNet • Train your model with MXNet • Use MXNet for predictions • Start Learning Apache MXNet • Apache MXNet: Now and Future
- 4. Neural network Output layer Input layer Hidden layers ManyMore… • Non-linear • Hierarchical feature learning • Scalable architecture • Computationally intensive
- 5. Forward pass Backwards pass Input Data Neural Network Output Loss Back Propagate Update Weights Forward-backward repeats across multiple epochs, each epoch goes through the entire training dataset. Training neural networks
- 7. Apache MXNet - Background ● Framework for building, training, and deploying Deep Neural Nets ● Apache (incubating) open source project ● Created by academia (CMU and UW) ● Adopted by AWS as DNN framework of choice, Nov 2016 http://mxnet.apache.org
- 9. Apache MXNet for Training • Simple and Powerful API: Gluon • Data Science compatibility: Numpy support • Distributed Training: Horovod, PSLite, BytePS • Training Speed improvement: • GPU: CUDNN and Float16 support (NVIDIA AMP) • CPU: Intel MKLDNN
- 10. Current Deep Learning scenario • Prototype code is hard to maintain • Setting up baseline for different workloads is hard • Pre-trained models are hard to obtain • Models trained in Python cannot be easily deployed to production system
- 11. MXNet Community: Gluon Toolkits • Carefully designed API for versatile needs • Implementation for state-of-the-art models • One-command download of hundreds of pre-trained models • Easy model export and deployment in C++, Java, Scala with support for Control flows and model quantization • Gluon Toolkits • GluonCV: Computer Vision • GluonNLP: Natural Language Processing • GluonTS: Probabilistic Time Series Modeling • Deep Graph Library
- 12. GluonCV: A Vision Toolkit • Scripts for reproducing SOTA results • State-of-the-Art pretrained Models • Easy Deployment Detection Pose Est. Classification Semantic Segmentation Instance Segmentation
- 13. GluonNLP: A Natural Language Toolkit • Data Processing APIs • Data API, support Multiprocessing, Batching, Vocabulary loading, Tokenizing… • Embedding Methods (~500 Pretrained) • Word2Vec, GloVe, FastText, ELMo, BERT, RoBERTa… • Sequence Sampler • Beam Search, Random Sampling • Models • Encoder/Decoder, AWD-LSTM, Transformer, Transformer-XL
- 15. Apache MXNet for Inference • Train in one Language, deploy in many: • JVM: Java, Scala, Clojure • Other languages: C++, R • Model support • Gluon Model Zoo • Open Neural Network Exchange (ONNX) models • Keras Model* • Model Deployment: MXNet Model Server
- 16. Gluon Model Zoo • CV: 194 models • NLP: 450 models
- 17. Multi-Model Server • Low latency, high throughput • Language agnostic: Python/Java • Model loading at runtime • Serving multiple models • Highly customizable (use plugins)
- 19. How can I make a start? • Book: Dive into Deep Learning (CHN: 动手学深度学习) • MXNet Community project • Comprehensive knowledge to learn Deep Learning • Include code to practice in MXNet • Course: STAT 157 Introduction to Deep Learning • UC Berkerly Spring 2019 • Instructed by Mu Li and Alexander Smola
- 20. Apache MXNet: Future plan (2.0) • Full numpy operator support • Gluon usability improvement • Accelerator support • TVM: operator integration • TVM: Relay IR integration (Experimental)
- 21. Contribute to Apache MXNet ● GitHub: https://github.com/apache/incubator-mxnet ● Subscribe to our developer mailing list: dev@mxnet.incubator.apache.org ● Slack Channel: https://the-asf.slack.com and go to #mxnet
- 22. Thank you! Qing Lan PPMC Member of Apache MXNet lanking@apache.org
Editor's Notes
- #5: Neural networks are composed of neurons. The neurons are organized into layers. There’s the input layer, the output layer, and in between is what we refer to as the hidden layers. If there are ”many” hidden layers, we call the network deep. Hence the term Deep Learning. Since the neurons in the network have a non-linear activation function, the whole network is non-linear and able to approximate complex functions. Able to learn hierarchical features about the data Scalable architecture. You can increase the number of layers. Increase the number of neurons in a layer… But this gets computationally expensive.
- #6: The difficult part is training the network, so we can find the right weights that will approximate the function modeling the problem we are trying to solve. We start with the “Forward Pass”, in which we take a sample from our labeled input data, feed it through the network to get the inference, or prediction result. We then do the “Backwards Pass”, also called “Backprop”, where we calculate the loss, i.e. how bad did the network did compared to the “Ground Truth” – the label of the sample input data – and then we back propagate the loss across the network, finding the gradients of each weight to identify the direction of the error. We then update the weights across the network, in a direction opposite to the gradient, and in a value that is typically a fraction of the gradient – this fraction is called the “Learning Rate”. The Backwards Pass is where learning happens. Through repeated iterations, we are leveraging the gradient to take down the loss, until we converge into a low error rate. After we’ve finished training the model, we no longer do the backwards pass. Instead we do only the forward pass to make predictions. We refer to this process as inference.
- #8: Just a bit of background on MXNet: MXNet is a deep learning framework for building, training, and deploying Deep Neural Nets. This last part, deploying DNNs, is probably the most interesting to you and what we’ll be focusing on today. MXNet is an Apache project. This means that no one group or company controls the project. Decisions about the direction of MXNet are made by the MXNet community. It originated in the academia, CMU and UW Aws adopted MXNet late 2016 as “DL FW of choice), there’s a nice blog post by AWS CTO (Vogels) explaining more in details. A lot of it is about scalability and MXNet being good for production use.
- #13: Reproducing state of art result are hard. We give you the sauce needed to do that 100+ SOTA pretrained models. Export model run on python/C++ We maintain the brazil package.
- #21: TVM operators