Apache MXNet EcoSystem - ACNA2018
2 likes412 views
The document discusses Apache MXNet, an open-source deep learning framework. It provides an overview of MXNet's history and key features, including support for multiple programming languages, an ecosystem of tools like GluonCV and GluonNLP, and model serving capabilities. It also describes MXNet's use of ONNX for model interchange, integration with Keras, and performance optimization using technologies like CUDA, MKL, and TVM. The document highlights MXNet's large community and adoption by customers.
![© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Build and train your model with PyTorch
Load your ONNX model with MXNet
Run inference, fine tune or save as MXNet model.
Back
Import ONNX model in MXNet– Usage Example
# Synthetic input for tracing
x = Variable(torch.randn(batch_size, 1, 224, 224), requires_grad=True)
# Export the model
torch_out = torch.onnx.export(model, x, "model.onnx")
# Import into MXNet (from MXNet 1.2)
sym, arg_params, aux_params = onnx_mxnet.import_model('model.onnx’)
# create module
mod = mx.mod.Module(symbol=sym, data_names=['input_0'], label_names=None)
mod.bind(for_training=False, data_shapes=[('input_0', input_img.shape)])
mod.set_params(arg_params=arg_params, aux_params=aux_params)
Credits: Hagay Lupesko](https://image.slidesharecdn.com/apachemxnetecosystem-acna2018-180929153931/85/Apache-MXNet-EcoSystem-ACNA2018-13-320.jpg)
![© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Build and train your model in MXNet
Export trained MXNet model to ONNX format
Import in other framework like cntk, caffe2 for inference
Back
Export MXNet model to ONNX – Usage
# Export MXNet model to ONNX format(from MXNet 1.3)
onnx_file_path = onnx_mxnet.export_model(sym, params, [input_shape],
input_data_type, onnx_file_path)
Credits: Hagay Lupesko](https://image.slidesharecdn.com/apachemxnetecosystem-acna2018-180929153931/85/Apache-MXNet-EcoSystem-ACNA2018-14-320.jpg)
![© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Keras – Apache MXNet
• Deep Learning for Humans
• 2nd most popular Deep Learning framework
• Keras users leverage MXNet’s great performance
from keras.models import Sequential
model = Sequential()
from keras.layers import Dense
model.add(Dense(units=64, activation='relu', input_dim=100))
model.add(Dense(units=10, activation='softmax'))
model.compile(loss='categorical_crossentropy',
optimizer='sgd',
metrics=['accuracy'])
model.fit(x_train, y_train, epochs=5, batch_size=32)
model.train_on_batch(x_batch, y_batch)
loss_and_metrics = model.evaluate(x_test, y_test, batch_size=128)
classes = model.predict(x_test, batch_size=128)
pip install mxnet-(mkl|cu92)
pip install keras-mxnet
---
~/.keras/keras.json
backend: mxnet
image_data_format: channels_first
---](https://image.slidesharecdn.com/apachemxnetecosystem-acna2018-180929153931/85/Apache-MXNet-EcoSystem-ACNA2018-18-320.jpg)
![© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Keras Benchmarks
Setup: https://github.com/awslabs/keras-apache-mxnet/tree/master/benchmark
Training Inference
Instance P3.8x Large, P3.16x Large C5.xLarge, C4.8xLarge
Network ResNet50v1 ResNet50v1
Batch size 32 * Num of GPUs 32
Image size 3*256*256 3*256*256
GPUs
Keras-MXNet
[ Image/sec ]
Keras-TensorFlow
[ Image/sec ] Speed Up
1 194 184 1.05
4 764 393 1.94
8 1068 261 4.09
0
500
1000
1500
1 GPU 4 GPU 8 GPU
ResNet50-Synthetic data
Keras-MXNet Keras-TensorFlow](https://image.slidesharecdn.com/apachemxnetecosystem-acna2018-180929153931/85/Apache-MXNet-EcoSystem-ACNA2018-19-320.jpg)
Apache MXNet EcoSystem - ACNA2018
- 1. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Naveen Swamy & Thomas Delteil * ecosystem 27th September 2018 * *
- 2. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Apache MXNet - History Amazon’s deep-learning framework of choice since November 2016. Accepted into Apache Incubator in January 2017. Created by academia (CMU and UW)
- 3. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Multi-language Support C++ C++ ClojureJuliaPerlR ScalaPython Frontend Backend While keeping high performance from efficient backend
- 4. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Apache MXNet Ecosystem MXBoard Model Server GluonCV GluonNLP ONNX Model Zoo Keras TensorRT TVM
- 5. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Model Server Credits: Hagay Lupesko
- 6. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Model Model Server Mobile Desktop IoT Internet So what does a deployed model looks like? Credits: Hagay Lupesko
- 7. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Performance Availability Networking Monitoring Model Decoupling Cross Framework Cross Platform The Undifferentiated Heavy Lifting of Model Serving Credits: Hagay Lupesko
- 8. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Trained Network Model Signature Custom Code Auxiliary Assets Model Archive Model Export CLI Model Archive Back Credits: Hagay Lupesko
- 9. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
- 10. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark MMS Dockerfile Pull or Build Push Launch Containerization Container Cluster MMS Container MMS ContainerMMS Container MXNet NGINX MXNet Model Server Lightweight virtualization, isolation, runs anywhere Back
- 11. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark ONNX Credits: Hagay Lupesko
- 12. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark O(n2) Pairs MXNet Caffe2 PyTorch TF CNTKCoreML TensorRT NGraph SNPEMany Frameworks Open Neural Network eXchange - Overview Many Platforms ONNX: Common IR • Open source • Community driven • Simple Credits: Hagay Lupesko
- 13. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Build and train your model with PyTorch Load your ONNX model with MXNet Run inference, fine tune or save as MXNet model. Back Import ONNX model in MXNet– Usage Example # Synthetic input for tracing x = Variable(torch.randn(batch_size, 1, 224, 224), requires_grad=True) # Export the model torch_out = torch.onnx.export(model, x, "model.onnx") # Import into MXNet (from MXNet 1.2) sym, arg_params, aux_params = onnx_mxnet.import_model('model.onnx’) # create module mod = mx.mod.Module(symbol=sym, data_names=['input_0'], label_names=None) mod.bind(for_training=False, data_shapes=[('input_0', input_img.shape)]) mod.set_params(arg_params=arg_params, aux_params=aux_params) Credits: Hagay Lupesko
- 14. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Build and train your model in MXNet Export trained MXNet model to ONNX format Import in other framework like cntk, caffe2 for inference Back Export MXNet model to ONNX – Usage # Export MXNet model to ONNX format(from MXNet 1.3) onnx_file_path = onnx_mxnet.export_model(sym, params, [input_shape], input_data_type, onnx_file_path) Credits: Hagay Lupesko
- 15. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark ONNX – Internals Protocol Buffers Graph Operator Tensor, … Operator Definitions ONNX Python APIProtocol Buffers: • Binary compact format • Statically defined • APIs for de/serialization • Cross platform Credits: Hagay Lupesko
- 16. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Back ONNX – Coverage Framework Export Import MXNet Supported Supported Caffe2 Supported Supported PyTorch Supported Coming… CNTK Supported Supported Chainer Supported (external) N/A TensorFlow Supported (external) Supported (external) CoreML Supported (external) Supported (external) SciKit-Learn Supported (external) N/A Credits: Hagay Lupesko
- 17. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Keras-MXNet https://github.com/awslabs/keras-apache-mxnet )
- 18. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Keras – Apache MXNet • Deep Learning for Humans • 2nd most popular Deep Learning framework • Keras users leverage MXNet’s great performance from keras.models import Sequential model = Sequential() from keras.layers import Dense model.add(Dense(units=64, activation='relu', input_dim=100)) model.add(Dense(units=10, activation='softmax')) model.compile(loss='categorical_crossentropy', optimizer='sgd', metrics=['accuracy']) model.fit(x_train, y_train, epochs=5, batch_size=32) model.train_on_batch(x_batch, y_batch) loss_and_metrics = model.evaluate(x_test, y_test, batch_size=128) classes = model.predict(x_test, batch_size=128) pip install mxnet-(mkl|cu92) pip install keras-mxnet --- ~/.keras/keras.json backend: mxnet image_data_format: channels_first ---
- 19. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Keras Benchmarks Setup: https://github.com/awslabs/keras-apache-mxnet/tree/master/benchmark Training Inference Instance P3.8x Large, P3.16x Large C5.xLarge, C4.8xLarge Network ResNet50v1 ResNet50v1 Batch size 32 * Num of GPUs 32 Image size 3*256*256 3*256*256 GPUs Keras-MXNet [ Image/sec ] Keras-TensorFlow [ Image/sec ] Speed Up 1 194 184 1.05 4 764 393 1.94 8 1068 261 4.09 0 500 1000 1500 1 GPU 4 GPU 8 GPU ResNet50-Synthetic data Keras-MXNet Keras-TensorFlow
- 20. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Keras Benchmarks GPUs Keras-MXNet Keras-TensorFlow Speed Up 1 135 52 2.59 4 536 162 3.30 8 722 211 3.420 500 1000 1500 1 GPU 4 GPU 8 GPU ResNet50-ImageNet data Keras-MXNet Keras-TensorFlow Instance Keras-MXNet Keras-TensorFlow Speed Up C5.X Large 5.79 3.27 1.782 C5.8X Large 27.9 18.2 1.53 0 10 20 30 1 2 ResNet50 - Batch Inference Keras-MXNet Keras-TensorFlow
- 21. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark ScalableDebuggable Flexible Imperative API
- 22. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark e.g. Inception StageSymbolic is “define, compile, run” Imperative is “define-by-run” Symbolic vs Imperative
- 23. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark GluonCV: a Deep Learning Toolkit for Computer Vision https://gluon-cv.mxnet.io 50+ Pre-trained models, with training scripts, datasets, tutorials
- 24. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark GluonCV: pre-trained models, help to choose
- 25. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark GluonCV: example code
- 26. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark GluonNLP: a Deep Learning Toolkit for Natural Language Processing https://gluon-nlp.mxnet.io Features (as of 0.3.2) - Pre-trained models: over 300 word-embedding - 5 language models - Neural Machine Translation (Google NMT, Transformer) - Flexible data pipeline tools and many public datasets. - NLP examples such as sentiment analysis.
- 27. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Word embedding: car dog 0 apple 0 eat 0 car 1 van 0 leaf 0 the 0 at 0 car 0.1 0.5 -1.1 0.4 van 0.2 0.3 -1.0 0.5
- 28. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Language modeling Trained to predict the next word (! "# "$ … "#&' ): - The winner of the 2018 FIFA world cup is ………. ?
- 29. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark first most <unk> only same best greatest main second last name largest highest final worst sixth third way primary This movie is considered the <?>
- 30. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark MXBoard: MXNet plugin to TensorBoard
- 31. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark MXBoard
- 32. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Deep Learning acceleration CUDA & CuDNN MKL, MKLML & MKLDNN pip install mxnet-cu92 e.g. pip install mxnet-mkl TensorRT pip install mxnet-tensorrt-cu92 TVM
- 33. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Apache MXNet community
- 34. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Keeping Up to Date Medium: https://medium.com/apache-mxnet
- 35. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Keeping Up to Date: Social YouTube: /apachemxnet Twitter: @apachemxnet Reddit: r/mxnet
- 36. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Community GitHub: https://github.com/apache/incubator-mxnet
- 37. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Community Discuss Forum: https://discuss.mxnet.io/
- 38. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Community Mailing list: dev@mxnet.apache.org user@mxnet.apache.org
- 39. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark MXNet Customer Momentum
- 40. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Thank you!