DeepLearning001&ApacheMXNetWithSparkForInference-ACNA2018
1 like129 views
The document discusses distributed deep learning inference using Apache MXNet and Apache Spark, highlighting the framework's capabilities and advantages in training deep neural networks. It covers types of learning, deep learning limitations, and the integration of MXNet with Spark for efficient inference. It also mentions recent updates to MXNet and encourages community involvement in democratizing AI.
DeepLearning001&ApacheMXNetWithSparkForInference-ACNA2018
- 1. Naveen Swamy Distributed Deep Learning Inference using Apache MXNet* and Apache Spark Senior Software Engineer Amazon AI *
- 2. Outline • Review of Deep Learning • Apache MXNet Framework • Distributed Inference using MXNet and Spark
- 3. Input layer (Raw pixels) Output (object identity) 3rd hidden layer (object parts) 2nd hidden layer (corners & contours) 1st hidden layer (edges) • Originally inspired by our biological neural systems. • A System that learns important features from experience. • Layers of Neurons learning concepts. • Deep learning != deep understanding Deep Learning Credit: Ian Goodfellow etal., Deep Learning Book CAR PERSON DOG
- 4. Algorithmic Advances (Faster Learning) Abundance of Data (Deeper Networks) High Performance Compute GPUs (Faster Experiments) Bigger and Better Models = Better AI Products
- 5. Why does Deep Learning matter? Autonomous Vehicles Personal Assistants Solve Intelligence ??? Health care
- 6. Deep Learning & AI, Limitations Artificial Intelligence DL Limitations: • Requires lots of data and compute power. • Cannot detect Inherent bias in data - Transparency. • Uninterpretable Results. Machine Learning Deep Learning
- 7. Deep Learning Training forward dog dog ? error labels data backward • Pass data through the network – forward pass • Define an objective – Loss function • Send the error back – backward pass Model: Output of Training a neural network X2 h2 w6 = 0.5 y = 1.0 y` = 0.9 loss = y l = 0.1 y X1 h1 w2 = 0.5 w1 = 0.5 w3 = 0.5 w5 = 0.4 w 4 = 0.5 0.1 0.1 backward pass forward pass
- 8. Deep Learning Inference • Real time Inference: Tasks that require immediate result. • Batch Inference: Tasks where you need to run on a large data sets. o Pre-computations are necessary - Recommender Systems. o Backfilling with state-of-the art models. o Testing new models on historic data. model forward dog
- 9. Types of Learning • Supervised Learning – Uses labeled training data learning to associate input data to output. Example: Image classification, Speech Recognition, Machine translation • Unsupervised Learning - Learns patterns from Unlabeled data. Example: Clustering, Association discovery. • Active Learning – Semi-supervised, human in the middle.. • Reinforcement Learning – learn from environment, using rewards and feedback.
- 10. Outline • Apache MXNet Framework • Distributed Inference using MXNet and Spark
- 11. Apache MXNet - Background • Apache (incubating) open source project • Framework for building and training DNNs • Created by academia (CMU and UW) • Adopted by AWS as DNN framework of choice, Nov 2016 http://mxnet.apache.org
- 12. Why MXNet
- 13. chik-fil-A waffle-fry freshness with Apache MXNet Computer Vision
- 14. Apache MXNet Customer Momentum
- 15. MXNet – NDArray & Symbol • NDArray– Imperative Tensor Operations that work on both CPU and GPUs. • Symbol APIs – similar to NDArray but adopts declarative programming for optimization. Computation GraphSymbolic Program
- 16. MXNet - Module High level APIs to work with Symbol 1) Create Graph 2) Bind 3) Pass data
- 17. Demo
- 18. Simple, Easy-to- Understand Code Flexible, Imperative Structure Dynamic Graphs High Performance § Neural networks can be defined using simple, clear, concise code § Plug-and-play neural network building blocks – including predefined layers, optimizers, and initializers § Eliminates rigidity of neural network model definition and brings together the model with the training algorithm § Intuitive, easy-to-debug, familiar code § Neural networks can change in shape or size during the training process to address advanced use cases where the size of data fed is variable § Important area of innovation in Natural Language Processing (NLP) § There is no sacrifice with respect to training speed § When it is time to move from prototyping to production, easily cache neural networks for high performance and a reduced memory footprint MXNet Gluon – Imperative & Fast Credits: Cyrus Vahid
- 19. Outline • Distributed Inference using MXNet and Spark
- 20. Distributed Inference Challenges • Similar to large scale data processing systems High Performance DL framework Distributed Cluster Resource Management Job Management Efficient Partition of Data Deep Learning Setup Apache Spark: • Multiple Cluster Managers • Works well with MXNet. • Integrates with Hadoop & big data tools.
- 21. MXNet + Spark for Inference. • ImageNet trained ResNet-18 classifier. • For demo, CIFAR-10 test dataset with 10K Images. • PySpark on Amazon EMR, MXNet is also available in Scala. • Inference on CPUs, can be extended to use GPUs.
- 22. Distributed Inference Pipeline download S3 keys on driver create RDD and partition fetch batch of images on executor decode to numpy array run prediction collect predictions initialize model only once mapPartitions
- 23. MXNet + Spark for Inference. Onthedriver
- 24. Ontheexecutor
- 26. Summary • Overview of Deep Learning o How Deep Learning works and Why Deep Learning is a big deal. o Phases of Deep Learning o Types of Learning • Apache MXNet – Efficient deep learning library o NDArray/Symbol/Module • Apache MXNet and Spark for distributed Inference.
- 27. What’s Next ? • Released v1.3.0 with many new features and improvements – Addition of Clojure support, User friendly APIs in Scala, Improved RNN support in Gluon, … • Working on Java APIs for Inference. • MXNet community is fast evolving, join hands to democratize AI.
- 28. Resources/References • https://github.com/apache/incubator-mxnet • Blog- Distributed Inference using MXNet and Spark • Distributed Inference code sample on GitHub • Apache MXNet Gluon Tutorials • Apache MXNet – Flexible and efficient deep learning. • The Deep Learning Book • MXNet – Using pre-trained models • Amazon Elastic MapReduce