“How to Run Audio and Vision AI Algorithms at Ultra-low Power,” a Presentation from Synaptics
0 likes42 views
The document outlines strategies for implementing audio and vision AI algorithms in IoT devices with a focus on ultra-low power consumption. It describes a multistage hardware approach where AI models are activated at different levels based on the need, enabling critical applications like security and maintenance while conserving battery life. The paper emphasizes the importance of tight software integration and model optimization to ensure effective performance across various use cases.
![• Resolution – [28x28 – 32x32]
• Kernel size – [3x3, 5x5, 7x7]
• Depth – [2, 3, 4]
• Width (channel expansion factor) – [2, 3, 4]
• Mixed-precision quantization parameters –
[4 bit, 6 bit, 8 bit]
Multi-precision NAS search range for classification
8
© 2024 Synaptics Inc](https://image.slidesharecdn.com/e1w09mitalsynaptics2024-240705122029-039f0bd8/85/How-to-Run-Audio-and-Vision-AI-Algorithms-at-Ultra-low-Power-a-Presentation-from-Synaptics-8-320.jpg)
![• Resolution – [320x240 –
640x480]
• Kernel size – [3x3, 5x5, 7x7]
• Depth – [2, 3, 4]
• Width (channel expansion
factor) – [2, 3, 4]
• Mixed-precision quantization
parameters – [4 bit, 6 bit, 8 bit]
Object detection dataset
11
© 2024 Synaptics Inc](https://image.slidesharecdn.com/e1w09mitalsynaptics2024-240705122029-039f0bd8/85/How-to-Run-Audio-and-Vision-AI-Algorithms-at-Ultra-low-Power-a-Presentation-from-Synaptics-11-320.jpg)
“How to Run Audio and Vision AI Algorithms at Ultra-low Power,” a Presentation from Synaptics
- 1. How to Run Audio and Vision AI Algorithms at Ultra-Low Power Presenter: Deepak Mital Sr. Director, Architecture Synaptics Incorporated
- 2. • Many IoT applications do not require “continuous maximum” compute • Continuous monitoring results in battery drain • Examples: • Security camera: Turn on main processing for actual detection only when confirmed necessary • Human presence detection (HPD) and identification to turn device on: Run HPD detection and identification algorithm only when detected “potential” presence • Predictive maintenance: Enable advanced detection only when initial metrics are met • Shoplift prevention: Enable detailed analytics only when “potential” threat detected Problem statement 2 © 2024 Synaptics Inc
- 3. • Multistage hardware: Capable of running Audio and Video AI algorithms • Highly efficient AI models with different KPIs for each stage • Tight orchestration of software to invoke each stage Solution 3 © 2024 Synaptics Inc Always-on domain High performance High efficiency Power management System memories Security USB / serial / MIPI U55 NPU Cortex-M55 μNPU Cortex-M4 Vision AI pipeline JPEG Audio VAD ISP, encoders Sensing logic Deep sleep: GPIO (Wake), internal clock Reset
- 4. • Ultra-low power: Microwatts hardware, always on • Sound detection • Image change detection • Critical model requirements are for very few false negatives • False negatives will render device unresponsive Solution – Stage 1 4 © 2024 Synaptics Inc Always-on domain High performance High efficiency Power management System memories Security USB / serial / MIPI U55 NPU Cortex-M55 μNPU Cortex-M4 Vision AI pipeline JPEG Audio VAD ISP, encoders Sensing logic Deep sleep: GPIO (Wake), internal clock Reset
- 5. • Mid- to low power – 10s of microwatts hardware, activated by stage 1 via software • AI algorithms (example): • Wake-word detection • Human presence detection • Critical model requirements are for very few false negatives and false positives • False negatives will render device unresponsive • False positives will increase power consumption Solution – Stage 2 5 © 2024 Synaptics Inc Always-on domain High performance High efficiency Power management System memories Security USB / serial / MIPI U55 NPU Cortex-M55 μNPU Cortex-M4 Vision AI pipeline JPEG Audio VAD ISP, encoders Sensing logic Deep sleep: GPIO (Wake), internal clock Reset
- 6. • High performance, activated by Stage 2 via software • AI algorithms (example): • Person identification • Object detection • Critical model requirements are for very high performance at low power • Slow run times will increase power consumption Solution – Stage 3 6 © 2024 Synaptics Inc Always-on domain High performance High efficiency Power management System memories Security USB / serial / MIPI U55 NPU Cortex-M55 μNPU Cortex-M4 Vision Ai pipeline JPEG Audio VAD ISP, encoders Sensing logic Deep sleep: GPIO (Wake), internal clock Reset
- 7. • Different requirements for AI models at each stage • Need AI models optimized for different KPIs: accuracy, performance, and size • NAS-based model generation architecture where the models are purpose built for the constrained silicon • Primary factors affecting inference KPI • Model architecture design • Model quantization • Approach: Jointly optimize model architecture and quantization under memory constraints AI models 7 © 2024 Synaptics Inc
- 8. • Resolution – [28x28 – 32x32] • Kernel size – [3x3, 5x5, 7x7] • Depth – [2, 3, 4] • Width (channel expansion factor) – [2, 3, 4] • Mixed-precision quantization parameters – [4 bit, 6 bit, 8 bit] Multi-precision NAS search range for classification 8 © 2024 Synaptics Inc
- 9. CIFAR-10 classification – Mixed vs 8- or 4-bit precision 9 © 2024 Synaptics Inc
- 10. CIFAR-10 classification comparison 10 © 2024 Synaptics Inc
- 11. • Resolution – [320x240 – 640x480] • Kernel size – [3x3, 5x5, 7x7] • Depth – [2, 3, 4] • Width (channel expansion factor) – [2, 3, 4] • Mixed-precision quantization parameters – [4 bit, 6 bit, 8 bit] Object detection dataset 11 © 2024 Synaptics Inc
- 12. COCO person detection – Mixed vs 8- or 4-bit precision 12 © 2024 Synaptics Inc
- 13. COCO person detection comparison 13 © 2024 Synaptics Inc
- 14. • Model development stage KPI: • COCO Instance Mask mAP: 0.636 • Latency: 92.19 ms • Resolution: 480x640 (VGA) • Weights: 1.57 M parameters • Model run on hardware: • Inference time: 96 ms • Total frame time: 120 ms Segmentation run on Stage 3 14 © 2024 Synaptics Inc
- 15. • Building full applications running at ultra-low power requires high levels of integration of hardware and software • Multiple levels of processing is needed to wake up silicon components as needed • Stage 2 and Stage 3 come out of deep sleep based on results from previous stage • The low-power orchestration demands tight software integration • Each stage requires AI models with different KPIs on accuracy, model size, and speed • Need to have NAS-based model generation/training software to enable the complete solution • Solution enables battery-powered devices that are AI capable and can run for many months/years Summary 15 © 2024 Synaptics Inc
- 16. Resources 16 Synaptics Astra embedded processors https://www.synaptics.com/products/embedded-processors Synaptics Astra evaluation Kit https://synacsm.atlassian.net/servicedesk/customer/portal/543/grou p/563/create/6387 Synaptics Astra software https://github.com/synaptics-astra © 2024 Synaptics Inc