“Accelerate Tomorrow’s Models with Lattice FPGAs,” a Presentation from Lattice Semiconductor
0 likes73 views
The document discusses how FPGAs from Lattice Semiconductor can accelerate edge AI applications. It notes that on-device AI inference is growing rapidly at the edge due to latency, security, and bandwidth limitations. FPGAs offer scalable and flexible performance for multiple use cases, secure configuration, hardware programmability to adapt to changing algorithms, and ultra-low power consumption from 1mW to 1W. Lattice's FPGAs allow parallel processing to improve FPS for AI tasks while reducing power compared to other solutions. The document provides examples of edge applications and Lattice's software and development tools to optimize AI models for implementation on their FPGA platforms.
“Accelerate Tomorrow’s Models with Lattice FPGAs,” a Presentation from Lattice Semiconductor
- 1. Accelerate Tomorrow’s Models with Lattice FPGAs Hussein Osman Segment Marketing Director Lattice Semiconductor
- 2. Rapidly Emerging Edge Computing Trend Driven by Latency, Security, Privacy, and Bandwidth Limitations On-device AI inference capability is expected to reach 60% of all devices by 2024 – ABI Research Edge Networking Cloud IoT Communication Gateway Wireless / Wireline Access Core Network Many Enabling Edge Technologies Resource-Efficiency and Increase Compute Sustainability 2 © 2021 Lattice Semiconductor
- 3. Why FPGA for Edge AI Accelerating Innovation in low power applications Scalable Performance To Handle Multiple Use Case Executes multiple use cases in parallel or serial Secure Secure device configuration Hardware Programmable Adapts to fast changing machine learning algorithms Ultra-low Power 1 milliwatt to 1 W power consumption Flexible Computation Resources Pre and post processing of data such as ISP, FFT and filtering 3 © 2021 Lattice Semiconductor
- 4. The Speed of AI Innovation Benefits Programmable Logic to Keep Delivering the Best Experience Hardware Optimization Algorithm Optimization Continuous Improvement and New Use Cases = & AI Models are rapidly evolving 80 70 60 50 2012 2021 AI Classification Model Innovation 4 © 2021 Lattice Semiconductor
- 5. AI Models Rapidly Evolve Requiring an Agile Development that ASIC Solutions Cannot Match 5 © 2021 Lattice Semiconductor ASIC Gen 2 Market leadership FPGA Gen1 Design 1 Design 2 Design 3 Design 4 Fast iterations to improve experience Introduction of new experiences ASIC Gen 1 Design 5 Design 6 Design 1 Design 2
- 6. 6 © 2021 Lattice Semiconductor Parallel and Concurrent Acceleration Postprocess 10-15 FPS Pre- process AI Preprocess AI Inference Postprocess Postprocess Pre- process AI 20-30 FPS Postprocess Pre- process AI 60 FPS Running in CPU Running on Accelerated HW Parallel and Concurrent Acceleration
- 7. Ultra Low Power Small Form Factor Customizable iCE40™ UltraPlus 1-20 mW, 5.5 mm2 ECP5™ 250 - 850 mW, 100 mm2 CrossLink™-NX 50-200 mW, 16 mm2 CertusPro™-NX 400 mW, 81 mm2 CUSTOM DESIGN SERVICES Smart Home Smart Car Smart City Smart Factory Client Compute Experiences REFERENCE DESIGNS / DEMOS Hand Gesture Detection Key Phrase Detection Object Counting Object Classification Human Attention Tracking Human Presence Detection SOFTWARE TOOLS sensAI Studio Neural Network Compiler IP CORES Neural Network Accelerators CNN Compact Accelerator CNN Co-Processor Accelerator CNN Accelerator CNN Plus Accelerator HARDWARE PLATFORMS Upduino ECP5™ EVDK CrossLink™-NX Voice and Vision ML Board CertusPro™-NX Voice and Vision ML Board GENERAL NOTICE: Other entityand product names usedin this publication arefor identificationpurposes only and may be trademarks of their respective holders. 7 © 2021 Lattice Semiconductor
- 8. Advanced General Purpose FPGA CLASS-LEADING POWER EFFICIENCY BEST-IN-CLASS SYSTEM BANDWIDTH OPTIMIZED EDGE PROCESSING INDUSTRY-LEADING RELIABILITY SMALLEST FORM FACTOR 2x Performance at Half the Power 8 © 2021 Lattice Semiconductor
- 9. CertusPro-NX Application Example Smart Camera AI Processing Offload AI processing and ISP from SoC Interface with popular image sensors over MIPI at up to 1.5 Gbps per lane, using High Performance IO (HPIO) Up to 7.3 Mb on-chip memory and flexible DSP resources to efficiently perform AI processing and ISP LPDDR4 external memory support for frame buffering Flexible output over MIPI or PCIe to SoC to enable smart devices with high quality imaging Image Signal Processing MIPI D-PHY MIPI D-PHY / PCIe Gen 3 MIPI / PCIe AI Accelerator MIPI I2C CertusPro-NX SoC LPDDR4 Image Sensor 9 © 2021 Lattice Semiconductor
- 10. Hardware Optimized Design Methodology Trained Model Quantized Weights and Instructions FPGA Bitstream NN Models NN IP System Interface Training Dataset Training Scripts NN Compiler Training FPGA Design Lattice sensAI Components Lattice FPGA Design Tools ML Frameworks 10 © 2021 Lattice Semiconductor
- 11. Software Optimized Design Methodology TFLite/ Converter Quantization TF TFLite C++ Converter Lattice Propel Builder RTL Synthesis / P&R Programmer JTAG Server Reveal Debugger C++ C++ Library IDE Compiler Linker GNU Debugger CPU Description Memory Map Drivers Executable Code Lattice Propel SDK RISC-V CPU Peripheral Bus Protocols Lattice Radiant Create Processor-Based Systems in Minutes with C/C++/Python ML mVision RISC-V AXI4 AHB Security ADC APB I2C SPI Accelerators 11 © 2021 Lattice Semiconductor
- 12. Lattice sensAI Studio Data Capture/ Label Transfer Learning Evaluate Configure Optimize Train Model Zoo Visualize (Dashboard) iCE40 UltraPlus CrossLink-NX Certus-NX ECP5 Model Selection Compile Runtime (Inference) Model Automatic Model Selection Based on the Dataset CertusPro-NX © 2021 Lattice Semiconductor 12 TM TM TM TM TM
- 13. Object Classification – Multiple Objects Tracking at Low Power BARCODE DETECTION ROBOT NAVIGATION FEATURES Sensor CMOS image sensor (IMX258) Speed 33 frames per second Power 400 mW on CertusPro-NX Resolution 224 x 224 x 3 DEFECT DETECTION 13 © 2021 Lattice Semiconductor
- 14. Lattice sensAI Enables Next Generation Smart PC Experiences
- 15. PC Usage Dominates the Daytime Productive Hours Source: https://www.smartinsights.com/mobile-marketing/mobile-marketing-analytics/mobile-marketing-statistics/ % of each platform’s average daily impressions by hour Late Night 12am – 7am Early Morning 7am – 10am Daytime 10am – 5pm Early Evening 5pm – 8pm Prime 8pm – 12am 15 © 2021 Lattice Semiconductor
- 16. Next Generation PC Trends Smart and Aware Great Collaboration Slim and New Form-Factors 16 © 2021 Lattice Semiconductor
- 17. Lattice sensAI - Smart and Aware AI Solutions Face Framing Attention Tracking Onlooker Detection User Presence Detection Instant On and Aware Delivers up to 28% Additional Battery Life Guards the User Private Data Collaborative Conference Experience 17 © 2021 Lattice Semiconductor
- 18. GENERAL NOTICE: Other entity and product names used in this publication are for identification purposes only and may be trademarks of their respective holders. Lattice Semiconductor Corporation, Lattice Semiconductor (& design), and specific product designations are either registered trademarks or trademarks of Lattice Semiconductor Corporation or its subsidiaries in the United States and/or other countries. Accelerate Tomorrow’s Models with Lattice FPGAs Hussein Osman Segment Marketing Director Lattice Semiconductor – Booth# 519 www.latticesemi.com/sensAI