Yangqing Jia at AI Frontiers: Towards Better DL Frameworks
4 likes1,834 views
This document discusses desirable features for deep learning frameworks. It outlines several key aspects ("MAPS"): scalability, portability, augmented computation patterns, and modularity. For scalability, frameworks should support distributed training across multiple GPUs and machines. For portability, frameworks need to run on various platforms like mobile, IoT, and embedded devices. They should also support quantized and sparse math to improve efficiency. Finally, frameworks benefit from a modular design that reuses components from optimized math libraries and communication primitives. The goal is flexible yet efficient frameworks that can satisfy the needs of both research and industry applications.
![Source: XKCD, [Girshick et al. CVPR 2014]](https://image.slidesharecdn.com/yangqingjia-towardsbetterdlframeworks-170114081616/85/Yangqing-Jia-at-AI-Frontiers-Towards-Better-DL-Frameworks-2-320.jpg)
Yangqing Jia at AI Frontiers: Towards Better DL Frameworks
- 1. Towards Better DL Frameworks Yangqing Jia Research Lead on AI Platforms, Facebook
- 2. Source: XKCD, [Girshick et al. CVPR 2014]
- 3. • Researchers: "I will need to reproduce the ResNet paper." • Companies: "I need to apply DL to drive cars." The Needs Two sides of the same coin
- 4. • A grad student driven project • Started by doing one job really well: image classification • Adopted by industry participants • Popular deep learning framework run by a non- profit. Democratizing Deep Learning w/ CaffeGetting AlexNet running in 10 mins
- 6. What makes a better DL library? ???
- 7. "MAPS" !!!
- 9. Scalability Run fast, run far “How do I train on multiple GPUs and machines?” - Probably the most question we got from Caffe users
- 10. Scalability Run fast, run far L1 L2 L3 L3b L2b L1b U3 U2 U1
- 11. Scalability Run fast, run far L1 L2 L3 L3b L2b L1b U3 U2 U1R3 R2 R1
- 12. Scalability Run fast, run far L1 L2 L3 L3b L2b L1b U3 U2 U1R3 R2 R1 L1 L2 L3 L3b L2b L1b U3 U2 U1R3 R2 R1
- 13. Scalability Run fast, run far L1 L2 L3 L3b L2b L1b U3 U2 U1R3 R2 R1 L1 L2 L3 L3b L2b L1b U3 U2 U1R3 R2 R1
- 14. The Return of MPI "I'm your father", said Allreduce. Allreduce Tree based - O(MlogN) Ring based - O(M) etc.
- 15. Scalability Sitting on top of giants ... and many more
- 17. Portable System Cloud, Mobile, IoT, Cars, Drones, Coffee makers AI Math and Algorithms Deployment Platforms
- 19. Portable System Cloud, Mobile, IoT, Cars, Drones, Coffee makers Model auto predictor = caffe2::Predictor(model_file) public class Predictor implements Caffe2ModelInterface;
- 20. Still, a lot of thoughts needed • Limited computation • Battery life is a thing • Our models may be luxurious • Ecosystem less developed Portable System Challenges
- 23. Augmented Comp Patterns Forget about float dense math, the world is bigger • Quantized Computation • Sparse Math Libraries • Model Compression • Rethinking Existing Operations
- 24. Quantized Computation Forget about float, the world is bigger 8 23 5 10 16 8 float fp16 fixed16 fixed8
- 25. Quantized Computation Forget about float, the world is bigger float add fp16 add fixed16 add fixed8 add 0.9 0.4 0.05 0.03 float mul fp16 mul fixed8 mul 4.0 1.0 0.2
- 27. Rethinking Existing Operations ResNEXT is coming to town gconv gconv g g g g g g g g g .. . g g g g g g g g g g .. . g AlexNet Group Conv ResNext
- 28. Augmented Math Challenges Forget about float, the world is bigger • Solutions • Eigen fp16 • CuDNN • NNPack • gemmlowp • Challenges • Seamless conversion? • Model training? • Performance tuning? • ...
- 30. A Repeated Pattern Many key components in deep learning are reusable across frameworks.
- 31. In 2013 it used to be... Caffe Torch Theano ...
- 32. Unix Philosophy? Applications Caffe,Torch,TF, MXNet, etc... Core Math Eigen CuDNN NNPack THNN MKL Comms NCCL MPI ZeroMQ Redis ... Low Level CUDA OpenGL OpenCL Vulkan ... Compilers DataBases LevelDB RocksDB Hadoop Amazon S3 your old disk or, "UnFramework"
- 33. Modular Designs MAPS for a good framework Augmented Mathematics Portable System Scalability Interface to Existing Toolkits Efficient Mobile Runtimes Tuned Collective Primitives Optimized Math Libraries + Flexible Framework Design
- 35. There is no silver bullet Industry: Stability Scale & speed Data Integration Relatively Fixed Research: Flexible Fast Iteration Debuggable Relatively bare- Caffe Torch TheanoTensorFlowD4J etc.
- 36. There is no silver bullet Industry: Stability Scale & speed Data Integration Relatively Fixed Research: Flexible Fast Iteration Debuggable Relatively bare- Caffe Torch
- 37. “In open source, we feel strongly that to really do something well, you have to get a lot of people involved.” — Linus Torvalds
- 38. Thank you! Towards Better Deep Learning Frameworks Yangqing Jia, Research Lead on AI Platforms, Facebook