Making AI Ubiquitous
3 likes793 views
Qualcomm Technologies is advancing AI by integrating on-device intelligence with cloud capabilities, fostering a future with ubiquitous and efficient AI applications across various industries by 2025. The document discusses the evolution from cloud-centric AI to distributed learning, highlighting the need for power-efficient hardware and software solutions. It emphasizes Qualcomm's leadership in AI research, innovations in deep learning, and the development of new models for enhanced performance and efficiency.
Making AI Ubiquitous
- 1. Making AI ubiquitous Qualcomm Technologies, Inc. @qualcomm_techJuly 2019
- 2. 2 Devices,machines, and things are becoming more intelligent
- 3. 3 Reasoning Learn, infer context, and anticipate Perception Hear, see, monitor, and observe Action Act intuitively, interact naturally, and protect privacy Offering new capabilities to enrich our lives
- 4. 4 A world where virtually everyone and everything is intelligently connected
- 5. 5 Edge cloud On-device On-device AI, processing, sensing, vision,… augmented by edge cloud New experiences Distributed autonomy Processing over 5G Customized/local value Privacy/security Private/public networks Immediacy Personalization Reliability Efficiency Process data closest to the source to scale for massive amount of data and connected things The intelligent wireless edge
- 6. 1. Such as distributed/virtualized core, distributed packet gateway functionality for low latency, mobile edge compute, related to MEC Multi Access Edge Computing as defined by ETSI Privacy/security Reliability Immediacy Efficiency Personalization In the past: Cloud-centric AI AI training and AI inference in the central cloud On-device inference converts input data to higher value, passed on to cloud for aggregated value Future: Fully-distributed AI with lifelong on-device learning To scale, process massive amount of data close to source—on the device Today: Partially-distributed AI Power efficient on-device AI interface
- 7. Privacy Reliability Low latency Efficient use of network bandwidth Process data closest to the source, complement the cloud On-device intelligence is paramount
- 8. On-device intelligence is quickly gaining momentum Key segments are expected to see full AI attach rates by 2025 2018 2025 10% 100%AI attach rate AI attach rate Mobile Automotive XR PCs / Tablets Smart speakers Source: Tractica, 2019
- 9. 9 Mobile is becoming the pervasive AI platform Source: IDC Aug. ‘18 ~7.8 Billion Cumulative smartphone unit shipments forecast between 2018–2022
- 10. 10 Mobile scale changes everything Superior scaleRapid replacement cycles Integrated/optimized technologies Healthcare Extended reality Smart cities Networking Automotive Industrial IoT Smart homes Smartphones Mobile computing Wearables Bringing AI to the masses
- 11. 11 AI offers enhanced experiences and new capabilities for smartphones Superior photographyTrue personal assistance Extended battery life Enhanced security Natural user interfaces Enhanced connectivity A new development paradigm where things repeatedly improve
- 14. 14 Personalized driver settings Driver awareness monitoring Greater autonomous capabilities Shaping the future of transportation
- 15. 15 Powering the factory of the future
- 16. 1616 AI for IoT across the home, industrial/enterprise, and Smart Cities More efficient use of energy and utilities Digitized logistics and retail Home hubs and smart appliances Sustainable cities and infrastructure Smarter agriculture Smart displays and speakers Smart security for home and enterprise Autonomous manufacturing and robotics IoT
- 17. 17 Power and thermal efficiency are essential for on-device AI The challenge of AI workloads Very compute intensive Large, complicated neural network models Complex concurrencies Real-time Always-on Constrained mobile environment Must be thermally efficient for sleek, ultra-light designs Requires long battery life for all-day use Storage/Memory bandwidth limitations
- 18. 18 Making power efficient AI pervasive Focusing on high performance HW/SW and optimized network design Algorithmic advancements Algorithmic research that benefits from state-of-the-art deep neural networks Optimization for space and runtime efficiency Efficient hardware Developing heterogeneous compute to run demanding neural networks at low power and within thermal limits Selecting the right compute block for the right task Software tools Software accelerated run-time for deep learning SDK/development frameworks
- 19. 19 Consistent AI R&D investment is the foundation for product leadership Our AI leadership Qualcomm Artificial Intelligence Research is an initiative of Qualcomm Technologies, Inc. Qualcomm Snapdragon, Qualcomm Neural Processing SDK, Qualcomm Vision Intelligence Platform, Qualcomm AI Engine, Qualcomm Cloud A, and Qualcomm QCS400I are products of Qualcomm Technologies, Inc. and/or its subsidiaries. Over a decade of cutting-edge AI R&D, speeding up commercialization and enabling scale Qualcomm® Vision Intelligence Platform Qualcomm® Neural Processing SDK 1st Gen Qualcomm® AI Engine (Qualcomm® Snapdragon™ 820 Mobile Platform) 2nd Gen AI Engine (Snapdragon 835) 3rd Gen AI Engine (Snapdragon 845) Snapdragon 660 Snapdragon 630 Brain Corp raises $114M Announced Facebook Caffe2 support Collaboration with Google on TensorFlow MWC demo showcasing photo sorting and hand writing recognition Acquired EuVision Opened Qualcomm Research Netherlands Research face detection with deep learning Completed Brain Corp joint research Research artificial neural processing architectures Investment and collaboration with Brain Corp Research in spiking neural networks Qualcomm Research initiates first AI project 2007 Deep-learning based AlexNet wins ImageNet competition Qualcomm Technologies ships ONNX supported by Microsoft, Facebook, Amazon 201820162014 20152009 2013 2017 2019 Acquired Scyfer Opened joint research lab with University of Amsterdam Qualcomm Technologies researchers win best paper at ICLR 4th Gen AI Engine (Snapdragon 855) Qualcomm® Artificial Intelligence Research initiated Snapdragon 710 3rd Gen Snapdragon Automotive Cockpit Qualcomm® Cloud AI 100 Qualcomm® QCS400 (First audio SoC) Snapdragon ® 665, 730, 730G Mobile AI Enablement Center in Taiwan to open
- 20. 20 Fundamental research Applied research G-CNN Bayesian combinatorial optimization Neural network compression Neural network quantization Deep generative models Deep transfer learning Graph and kernel optimization Machine learning training tools Source compression CV DL for new sensors Voice UICompute in memory Hybrid reinforcement learning Video recognition & prediction Deep learning for graphics Power management Bayesian distributed learning Hardware- aware deep learning Fingerprint Leading research and development across the entire spectrum of AI
- 21. 21 Can we apply foundational mathematics of physics, like quantum field theory, to deep learning?
- 22. 22 G-CNN Video
- 23. 23 Advancing fundamental AI research, such as generalized CNNs Applying foundational mathematics of physics Translation works Rotation doesn’t work (Generalized CNNs (G-CNN): Gauge equivariant CNN, Group, and Steerable CNN pioneered by Qualcomm AI Research do not need to be retrained) (Convolutional neural networks would need to be retrained with new rotated images to determine new set of parameters—like filter weights) Today’s deep learning Traditional CNNs Produce state-of-the art results but… do not generalize input like rotations No matter how you rotate or move the object, the generalized model will still identify the object Tomorrow’s deep learning Gauge Equivariant CNNs Rotated objects and images applicable to drones, robots, cars, fisheye-lens cameras. VR, AR,.. Like quantum field theory, to deep learning
- 24. 24 Unifying framework Gauge equivariant CNN unify special cases like Group CNNs and Steerable CNNs, all pioneered by Qualcomm AI Research Robust performance, faster training, and fewer training examples required Broad societal benefits Use cases like drones, robots, cars, XR, fisheye lenses, 3D gaming, … But also areas like state-of-the-art accuracy on climate pattern segmentation Pioneering deep learning research in generalized CNNs Equivariance No matter how you rotate or move the object, it will still be identified G-CNN can generalize models for different symmetries — traditional CNNs must be retrained Generalized geometry Traditional CNNs work well on narrow field-of-view cameras, but fail on e.g. fish-eye cameras G-CNN can analyze image data on any curved space, from flat to spherical
- 25. 25 Trained neural network model Inference output New input data Hardware awareness AI Acceleration (scalar, vector, tensor) Acceleration research Such as compute-in-memory Advancing AI research to increase power efficiency QuantizationCompression Compilation Learning to reduce bit-precision while keeping desired accuracy Learning to prune model while keeping desired accuracy Learning to compile AI models for efficient hardware execution Applying AI to optimize AI model through automated techniques
- 26. 26 Compression with less than 1% loss in accuracy13x Perf. per watt improvement from savings in memory and compute2 >4x Performance improvement over TensorFlow Lite34x Trained neural network model New input data Recent examples Advancing AI research to increase power efficiency 1: With both Bayesian compression and spatial SVD with ResNet18 as baseline. 2: For a quantized INT8 model vs a FP32 model that is not quantized. 3: On average improvement of tested AI models. QuantizationCompression Compilation Learning to reduce bit-precision while keeping desired accuracy Learning to prune model while keeping desired accuracy Learning to compile AI models for efficient hardware execution Applying AI to optimize AI model through automated techniques Inference output
- 27. 27 Mobile Apps Cores Qualcomm® Adreno™ GPUQualcomm® Kryo™ CPU Qualcomm® Hexagon™ DSP Scalar Vector Tensor NN Frameworks Cognitive Toolkit Libraries Qualcomm® Math Libraries OpenCL Hexagon NN Runtime Software Frameworks TensorFlow Lite Google NN API Qualcomm® Neural Processing SDK 4th Gen AI Engine Qualcomm® Artificial Intelligence Engine The hardware and software components for efficient on-device machine learning Qualcomm Math Libraries, Qualcomm Artificial Intelligence Engine, Qualcomm Kryo, Qualcomm Adreno, Qualcomm Hexagon, and Qualcomm Neural Processing SDK are products of Qualcomm Technologies, Inc. and/or its subsidiaries.
- 28. Qualcomm® Cloud AI 100 • Built on 7nm • >350 TOPS Peak AI Performance • Sampling 2nd half of 2019
- 29. 29 Qualcomm® Neural Processing SDK Software accelerated runtime for the execution of deep neural networks on device Qualcomm Kryo, Qualcomm Adreno and Qualcomm Hexagon are products of Qualcomm Technologies, Inc. Available at: developer.qualcomm.com Efficient execution on Snapdragon • Takes advantage of Snapdragon heterogeneous computing capabilities • Runtime and libraries accelerate deep neural net processing on all engines: CPU, GPU, and DSP with HVX and HTA Model framework/Network support • Convolutional neural networks and LSTMs • Support for Caffe/Caffe2, TensorFlow, and user/developer defined layers Optimization/Debugging tools • Offline network conversion tools • Debug and analyze network performance • API and SDK documentation with sample code • Ease of integration into customer applications Qualcomm® Kryo™ CPU Qualcomm® Adreno™ GPU Qualcomm® Hexagon™ DSP Sample code Offline conversion tools Analyze performance Ease of integration
- 30. 30 Frameworks Cognitive Toolkit OS Ecosystem Features Face Recognition Night Shot Super Resolution Noise Suppression Speech Recognition Object Detection Video Segmentation Devices Bokeh Qualcomm AI Engine
- 31. Foundational R&D 5G + AI technology leadership Ecosystem investment Advanced silicon Systems design expertise Qualcomm Ventures AI Fund Uniquely positioned to power the intelligently connected future X50
- 32. 32 Intelligence is becoming more distributed, with power-efficient on- device AI complementing the cloud Mobile is democratizing AI and bringing it to new frontiers Qualcomm Technologies is well positioned to provide superior AI solutions and make AI ubiquitous
- 34. Follow us on: For more information, visit us at: www.qualcomm.com & www.qualcomm.com/blog Thank you! Nothing in these materials is an offer to sell any of the components or devices referenced herein. ©2018 Qualcomm Technologies, Inc. and/or its affiliated companies. All Rights Reserved. Qualcomm, Snapdragon, Hexagon, Adreno, and Kryo are trademarks of Qualcomm Incorporated, registered in the United States and other countries. Other products and brand names may be trademarks or registered trademarks of their respective owners. References in this presentation to “Qualcomm” may mean Qualcomm Incorporated, Qualcomm Technologies, Inc., and/or other subsidiaries or business units within the Qualcomm corporate structure, as applicable. Qualcomm Incorporated includes Qualcomm’s licensing business, QTL, and the vast majority of its patent portfolio. Qualcomm Technologies, Inc., a wholly-owned subsidiary of Qualcomm Incorporated, operates, along with its subsidiaries, substantially all of Qualcomm’s engineering, research and development functions, and substantially all of its product and services businesses, including its semiconductor business, QCT.