“Why It’s Critical to Have an Integrated Development Methodology for Edge AI,” a Presentation from Lattice Semiconductor
0 likes90 views
For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2025/06/why-its-critical-to-have-an-integrated-development-methodology-for-edge-ai-a-presentation-from-lattice-semiconductor/ Sreepada Hegade, Director of ML Systems and Software at Lattice Semiconductor, presents the “Why It’s Critical to Have an Integrated Development Methodology for Edge AI” tutorial at the May 2025 Embedded Vision Summit. The deployment of neural networks near sensors brings well-known advantages such as lower latency, privacy and reduced overall system cost—but also brings significant challenges that complicate development. These challenges can be addressed effectively by choosing the right solution and design methodology. The low-power FPGAs from Lattice are well poised to enable efficient edge implementation of models, while Lattice’s proven development methodology helps to mitigate the challenges and risks associated with edge model deployment. In this presentation, Hegade explains the importance of an integrated framework that tightly consolidates different aspects of edge AI development, including training, quantization of networks for edge deployment, integration with sensors and inferencing. He also illustrates how Lattice’s simplified tool flow helps to achieve the best trade-off between power, performance and efficiency using low-power FPGAs for edge deployment of various AI workloads.
“Why It’s Critical to Have an Integrated Development Methodology for Edge AI,” a Presentation from Lattice Semiconductor
- 1. Why It’s Critical to Have an Integrated Development Methodology for Edge AI Sreepada V Hegade Director, ML Software and Solutions Lattice Semiconductor
- 2. Disclaimer Lattice makes no warranty, representation, or guarantee regarding the accuracy of information contained in this document or the suitability of its products for any particular purpose. All information herein is provided AS IS, with all faults, and all associated risk is the responsibility entirely of the recipient of the information. The information provided herein is for informational purposes only and may contain technical or other inaccuracies or omissions, and may be otherwise rendered inaccurate for many reasons, and Lattice assumes no obligation to update or otherwise correct or revise this information. Products sold by Lattice have been subject to limited testing and it is the responsibility of a buyer of Lattice products to independently determine the suitability of any products and to test and verify the same. Lattice products and services are not designed, manufactured, or tested for use in life or safety critical systems, hazardous environments, or any other environments requiring fail-safe performance, including any application in which the failure of the product or service could lead to death, personal injury, severe property damage or environmental harm (collectively, “high-risk uses”). Further, a buyer must take prudent steps to protect against product and service failures, including providing appropriate redundancies, fail-safe features, and/or shut-down mechanisms. Lattice expressly disclaims any express or implied warranty of fitness of the products or services for high-risk uses. Lattice Semiconductor Corporation, Lattice Semiconductor (& design) and specific product designations are either registered trademarks or trademarks of Lattice Semiconductor Corporation or its subsidiaries in the United States and/or other countries. © 2025 Lattice Semiconductor 2
- 3. • Widespread Adoption • Across multiple market segments (Consumer, Industrial, Automotive, Healthcare) • Technological Advancements • Compact models • Tiny inference devices • Realized Benefits • Lower cost, power, latency • Enhanced security, reliability $18.7B Market by 2025 Endoscopy Camera Retail Camera Machine Vision Drone Augmented Reality Industrial Sensor Automation Night Vision Autonomous Robotics 360 Camera Robotics Smart Home Surround View Smart Traffic Camera Surveillance Camera Range Finder Edge AI Advancements © 2025 Lattice Semiconductor 3
- 4. © 2025 Lattice Semiconductor Challenges Complexity How to find right trade-off between performance, power, cost and other KPIs ? Sustainability How do I update solution with latest innovations ? Adaptability How can I support wide array of sensors ? 4
- 5. FPGA Value Proposition © 2025 Lattice Semiconductor Predictable Provide low and deterministic latency essential for many edge applications Scalable Helps to address scalability issues like integration with multiple sensors of different types Flexible Same solution can be targeted to different devices with trade-off on power, performance and cost 5
- 6. Low and Deterministic Latency • Time Complexity of Convolution • Image is w x h • Filter is k x k • Each output requires O(k²) operations • Total time complexity: • O(whk²) • Number of output entries: • (w+k-1)(h+k-1) © 2025 Lattice Semiconductor = -74 -74 76 * 6
- 7. Flexibility © 2025 Lattice Semiconductor Lattice iCE40 UltraPlusTM Lattice CrossLinkTM- NX 17 Lattice CrossLinkTM- NX 40 Lattice CrossLinkTMNX 33 Lattice CertusProTM-NX Lattice AvantTM Footprint (mm) 2.15 x 2.55 3.7 x 4.1 6 x 6 3.1 x 7.4 9 x 9 10 x 10 – 27 x 27 # of DSPs (18 x 18) 8 24 56 64 156 320 - 1800 # of 8 x 8 multipliers per DSP 1x 2x 2x 2x 2x 4x Distributed Memory (kbits) 120 432 1512 1512 3400 - SPRAM (kbits) 1024 2560 1024 2560 3072 - # of Cores Compact 1 2 2 6 12 - 72 Speed (MHz) 40 150 150 - 150 650 Power (W) 0.02 0.05 0.2 0.150 0.45 1 7
- 8. Scalability © 2025 Lattice Semiconductor Camera Interface Lidar/Radar Interface Audio Interface Other sensor Interfaces ISP Point Cloud Processing Spectrogram Transformation Sensor Hub Sensor Hub Multimodal Neural Network Inference Engine Host Inference Sensors 8
- 9. © 2025 Lattice Semiconductor Key Observations Implementation of efficient edge AI solution requires hardware aware model development and training Configurable and flexible hardware is essential to achieve optimal solution for a given application Integrated design tools and methodology critical to successfully co- design model and hardware for edge AI 9
- 10. Lattice sensAI Solution © 2025 Lattice Semiconductor 10
- 11. © 2025 Lattice Semiconductor 11
- 12. © 2025 Lattice Semiconductor Customizable ML Accelerator • Reduce memory access • Customize engine with absolutely required operators • Configure data width and precision Operator Granularity High Low Memory Bandwidth Low (large data width) High (small data width) How to support new operations at higher granularity level ? How can I reduce width of data in memory without sacrificing accuracy ? Higher Performance, Lower Power 12
- 13. © 2025 Lattice Semiconductor Zero Code System Generator Configure ML System by extracting parameters from trained neural networks ML Model GUI Golden System Template RISC-V Firmware HW description ML Runtime (.bit) HW Config Tool and flow that makes system easy to use for ML engineers System Level FPS Resource constraint – LUT, EBR, DSP budget Peripherals Memory Interface External/Internal Camera pipeline RGB vs Monochrome HW/SW Interface Pre/post processing algorithm (Blur, NMS) specifications IP ML IP Configuration USER IP AXI Interface IPs Data Bus Control Bus Bypass Lattice FPGA Imaging Input Interfaces (MIPI-CSI I2C) Image Processing Shared Memory AI Accelerators Imaging output Interface (to CPU) CPU Subsystem Secure Boot LPDDR4 Controller Control Peripherals System Generator ML Compiler 13
- 14. © 2025 Lattice Semiconductor Integrated Tool and Flow .bin file ML compiler HW Config .bit file .elf file Model Info System Req System Description System compiler Tools for System Dev System Generator Code Generator Network Analyzer Optimizer Scheduler C Compiler RTL Compiler System Simulation Framework Database Customer Model Data Tools for ML Engineers Lattice Training Environment (LatTE) Quantization Fine Tuning C Models IP (RTL) Model Zoo 14
- 15. Case Studies © 2025 Lattice Semiconductor 15
- 16. Face Landmark Detection Variation Quantization Landmark Pixel MAE 1 float32 1.09952 2 Fixed Step Size Quantization (int8) 1.28866 3 Learned Step Size Quantization (int8) 1.11622 Variation of MobileNet V2 Model Model input resolution 96x96 Detects 23 landmarks on the face © 2025 Lattice Semiconductor Learned Step Size Quantization (LSQ) Helps Improve Accuracy 16
- 17. Barcode Detection YOLOv5 Input Resolution 320x240 (QVGA) Detect barcode(s) on packages that are moving on a conveyer belt © 2025 Lattice Semiconductor Network Type YOLOv5 mAP with fixed quantization 77.43% mAP with Learned Step Quantization 96.79% Few layers quantized to 4-bit data 89.73% Learned Step Quantization helped reduce data width, thereby saving memory utilization 17
- 18. Lattice low power FPGAs are perfect for edge AI model inferences Production-proven solutions and tools offer a good trade-off when developing applications Hardware can be configured for optimal inference of a given network topology Reference designs and models are a great a starting point for building your application © 2025 Lattice Semiconductor Summary 18
- 19. Thank You! Visit Lattice and see our technologies at Booth #416 More information can be found at https://www.latticesemi.com/en/Solutions/Solutions/SolutionsDetails02/sensAI © 2025 Lattice Semiconductor 19