![© 2022 MIPI Alliance, Inc. 2
Agenda
• Introduction to SOM
• SOM market & projections
• Vision applications: Concept to production with SOM
• Accelerated applications
• System level challenges
• Q&A
[Public]](https://image.slidesharecdn.com/2022-mipi-devcon-sensor-system-interop-and-debug-for-vision-appllications-som-240721135858-a46c0792/85/2022-MIPI-DevCon-Sensor-System-Interop-and-Debug-for-Vision-Appllications-SOM-pdf-2-320.jpg)
![© 2022 MIPI Alliance, Inc. 3
• Alternative to single board computers
focused on enabling customized
embedded systems
• A credit card sized module with an
integrated SoC (CPU/GPU/FPGA), power,
security module, & flexible I/O
• Offers more flexibility & contoured
solutions
• Plugs into a carrier/base board for flexible
application specific peripheral carrier
card design
[Public]
What’s a System-on-Module (SOM)?](https://image.slidesharecdn.com/2022-mipi-devcon-sensor-system-interop-and-debug-for-vision-appllications-som-240721135858-a46c0792/85/2022-MIPI-DevCon-Sensor-System-Interop-and-Debug-for-Vision-Appllications-SOM-pdf-3-320.jpg)
![© 2022 MIPI Alliance, Inc. 4
• Reduces effort of ground-up electronics design
• Customers start at a more evolved point vs. chip-down design
• Enables customers to build multiple products based off the same SOM platform
• SW developers to start sooner, and HW designers to finish early in the cycle.
[Public]
Advantages of SOM](https://image.slidesharecdn.com/2022-mipi-devcon-sensor-system-interop-and-debug-for-vision-appllications-som-240721135858-a46c0792/85/2022-MIPI-DevCon-Sensor-System-Interop-and-Debug-for-Vision-Appllications-SOM-pdf-4-320.jpg)
![© 2022 MIPI Alliance, Inc. 5
• Global SOM Sales by Application • Global SOM Sales forecast by Application
Source: Expert Interview, Secondary Sources and
QYR Electronics Research Center, July 2019
[Public]
SOM Market Size and Application](https://image.slidesharecdn.com/2022-mipi-devcon-sensor-system-interop-and-debug-for-vision-appllications-som-240721135858-a46c0792/85/2022-MIPI-DevCon-Sensor-System-Interop-and-Debug-for-Vision-Appllications-SOM-pdf-5-320.jpg)
![© 2022 MIPI Alliance, Inc. 6
SOM: Board Level HW Abstraction
• Chip-down style
• Evaluate & select specific device
• Build hardware prototype & prove all functional interfaces
• SOM based
• Production ready OTS board with SoC, DDR4, and multiple MIPI compliant programmable interfaces
• SW references with FPGA based acceleration prebuilt configurations
[Public]](https://image.slidesharecdn.com/2022-mipi-devcon-sensor-system-interop-and-debug-for-vision-appllications-som-240721135858-a46c0792/85/2022-MIPI-DevCon-Sensor-System-Interop-and-Debug-for-Vision-Appllications-SOM-pdf-6-320.jpg)
![© 2022 MIPI Alliance, Inc. 7
KV260 Vision AI Starter Kit
• Provides a framework for building & customizing video platforms
– Capture pipeline
– Video processing pipeline
– Acceleration pipeline
– Output pipeline
[Public]](https://image.slidesharecdn.com/2022-mipi-devcon-sensor-system-interop-and-debug-for-vision-appllications-som-240721135858-a46c0792/85/2022-MIPI-DevCon-Sensor-System-Interop-and-Debug-for-Vision-Appllications-SOM-pdf-7-320.jpg)
![© 2022 MIPI Alliance, Inc. 8
Kria Starter Kit Accelerated Applications
• Adaptive SOM simplifies application
carrier card HW design & developer to
focus on SW and AI development
• Prebuilt reference platforms , API’s
enable full customization of FPGA
based HW & acceleration capabilities
• AMD-Xilinx tools provide mapping of
ML development environments like
Python, C++, TensorFlow & PyTorch to
FPGA accelerated HW on SOM
[Public]](https://image.slidesharecdn.com/2022-mipi-devcon-sensor-system-interop-and-debug-for-vision-appllications-som-240721135858-a46c0792/85/2022-MIPI-DevCon-Sensor-System-Interop-and-Debug-for-Vision-Appllications-SOM-pdf-8-320.jpg)
![© 2022 MIPI Alliance, Inc. 9
System Level Challenges
Issue: Image freezes
• Status at Controller: No output
• Status at MIPI D-PHYSM: Receiving packets without errors
MIPI
D-PHYSM Controller
PPI
Serial
[Public]](https://image.slidesharecdn.com/2022-mipi-devcon-sensor-system-interop-and-debug-for-vision-appllications-som-240721135858-a46c0792/85/2022-MIPI-DevCon-Sensor-System-Interop-and-Debug-for-Vision-Appllications-SOM-pdf-9-320.jpg)
![© 2022 MIPI Alliance, Inc. 10
System Level Challenge (Sensor Debug)
• Sensor output: HS <-> LP patterns looks good
• Non-continuous clock mode
[Public]](https://image.slidesharecdn.com/2022-mipi-devcon-sensor-system-interop-and-debug-for-vision-appllications-som-240721135858-a46c0792/85/2022-MIPI-DevCon-Sensor-System-Interop-and-Debug-for-Vision-Appllications-SOM-pdf-10-320.jpg)
![© 2022 MIPI Alliance, Inc. 11
System Level Challenge (Controller Debug)
• Debug at Controller FSM:
• PPI data not processed fully
• Is it due to less rxbyteclkhs? (recovered clock)
[Public]](https://image.slidesharecdn.com/2022-mipi-devcon-sensor-system-interop-and-debug-for-vision-appllications-som-240721135858-a46c0792/85/2022-MIPI-DevCon-Sensor-System-Interop-and-Debug-for-Vision-Appllications-SOM-pdf-11-320.jpg)
![© 2022 MIPI Alliance, Inc. 12
System Level Challenges (Analysis & Solution)
• Utilized debug capabilities (ILA) in SOM FPGA PL fabric
• Analyzed sensor Tclk-post vs IP requirements
• Analyzed the system behavior and internal fabric logic for
different Tclk-post settings
• SOLUTION: Increasing Tclk-post setting of sensor resolved the issues
• Batch testing PASSED
• Sensors used in SOM design: AR1335, AR0144, RPi etc.
[Public]](https://image.slidesharecdn.com/2022-mipi-devcon-sensor-system-interop-and-debug-for-vision-appllications-som-240721135858-a46c0792/85/2022-MIPI-DevCon-Sensor-System-Interop-and-Debug-for-Vision-Appllications-SOM-pdf-12-320.jpg)
![© 2022 MIPI Alliance, Inc. 13
System Level Challenges (Other Debug Scenarios)
• Other general challenges
• Source generate user defined data along with Pixel data
• Impact: Image corruption
• Solution: Downstream video pipe must demux Pixel vs user defined data type
• Source slightly out of spec (Ex: Less Tlpx period)
• Impact: Controller don’t detect LP-HS transitions (LP-11 -> LP-01 -> LP-00)
• Solution: Tune Tlpx of source
• Source need more settle time before sending Sync pattern
• Impact: Controller reports synchronization errors
• Solution: Tune THS-SETTLE parameter of receiver
[Public]](https://image.slidesharecdn.com/2022-mipi-devcon-sensor-system-interop-and-debug-for-vision-appllications-som-240721135858-a46c0792/85/2022-MIPI-DevCon-Sensor-System-Interop-and-Debug-for-Vision-Appllications-SOM-pdf-13-320.jpg)
![© 2022 MIPI Alliance, Inc. 14
Summary
• SOM with industry standard interfaces & communication
protocols reduces development time
• SOM facilitates application and HW design with a greater
degree of abstraction from chip-down designs
• SOM FPGA PL provides flexible I/O and I/O debug
capabilities
• Closure of “Global timing parameters” to ease system level
challenges
[Public]](https://image.slidesharecdn.com/2022-mipi-devcon-sensor-system-interop-and-debug-for-vision-appllications-som-240721135858-a46c0792/85/2022-MIPI-DevCon-Sensor-System-Interop-and-Debug-for-Vision-Appllications-SOM-pdf-14-320.jpg)
2022-MIPI-DevCon-Sensor-System-Interop-and-Debug-for-Vision-Appllications-SOM.pdf
- 1. © 2022 MIPI Alliance, Inc. Kondalarao Polisetti, Wesley Skeffington Advanced Micro Devices, Inc./Xilinx Inc. MIPI Sensor System-Interop and Debug for Vision Applications on a SOM
- 2. © 2022 MIPI Alliance, Inc. 2 Agenda • Introduction to SOM • SOM market & projections • Vision applications: Concept to production with SOM • Accelerated applications • System level challenges • Q&A [Public]
- 3. © 2022 MIPI Alliance, Inc. 3 • Alternative to single board computers focused on enabling customized embedded systems • A credit card sized module with an integrated SoC (CPU/GPU/FPGA), power, security module, & flexible I/O • Offers more flexibility & contoured solutions • Plugs into a carrier/base board for flexible application specific peripheral carrier card design [Public] What’s a System-on-Module (SOM)?
- 4. © 2022 MIPI Alliance, Inc. 4 • Reduces effort of ground-up electronics design • Customers start at a more evolved point vs. chip-down design • Enables customers to build multiple products based off the same SOM platform • SW developers to start sooner, and HW designers to finish early in the cycle. [Public] Advantages of SOM
- 5. © 2022 MIPI Alliance, Inc. 5 • Global SOM Sales by Application • Global SOM Sales forecast by Application Source: Expert Interview, Secondary Sources and QYR Electronics Research Center, July 2019 [Public] SOM Market Size and Application
- 6. © 2022 MIPI Alliance, Inc. 6 SOM: Board Level HW Abstraction • Chip-down style • Evaluate & select specific device • Build hardware prototype & prove all functional interfaces • SOM based • Production ready OTS board with SoC, DDR4, and multiple MIPI compliant programmable interfaces • SW references with FPGA based acceleration prebuilt configurations [Public]
- 7. © 2022 MIPI Alliance, Inc. 7 KV260 Vision AI Starter Kit • Provides a framework for building & customizing video platforms – Capture pipeline – Video processing pipeline – Acceleration pipeline – Output pipeline [Public]
- 8. © 2022 MIPI Alliance, Inc. 8 Kria Starter Kit Accelerated Applications • Adaptive SOM simplifies application carrier card HW design & developer to focus on SW and AI development • Prebuilt reference platforms , API’s enable full customization of FPGA based HW & acceleration capabilities • AMD-Xilinx tools provide mapping of ML development environments like Python, C++, TensorFlow & PyTorch to FPGA accelerated HW on SOM [Public]
- 9. © 2022 MIPI Alliance, Inc. 9 System Level Challenges Issue: Image freezes • Status at Controller: No output • Status at MIPI D-PHYSM: Receiving packets without errors MIPI D-PHYSM Controller PPI Serial [Public]
- 10. © 2022 MIPI Alliance, Inc. 10 System Level Challenge (Sensor Debug) • Sensor output: HS <-> LP patterns looks good • Non-continuous clock mode [Public]
- 11. © 2022 MIPI Alliance, Inc. 11 System Level Challenge (Controller Debug) • Debug at Controller FSM: • PPI data not processed fully • Is it due to less rxbyteclkhs? (recovered clock) [Public]
- 12. © 2022 MIPI Alliance, Inc. 12 System Level Challenges (Analysis & Solution) • Utilized debug capabilities (ILA) in SOM FPGA PL fabric • Analyzed sensor Tclk-post vs IP requirements • Analyzed the system behavior and internal fabric logic for different Tclk-post settings • SOLUTION: Increasing Tclk-post setting of sensor resolved the issues • Batch testing PASSED • Sensors used in SOM design: AR1335, AR0144, RPi etc. [Public]
- 13. © 2022 MIPI Alliance, Inc. 13 System Level Challenges (Other Debug Scenarios) • Other general challenges • Source generate user defined data along with Pixel data • Impact: Image corruption • Solution: Downstream video pipe must demux Pixel vs user defined data type • Source slightly out of spec (Ex: Less Tlpx period) • Impact: Controller don’t detect LP-HS transitions (LP-11 -> LP-01 -> LP-00) • Solution: Tune Tlpx of source • Source need more settle time before sending Sync pattern • Impact: Controller reports synchronization errors • Solution: Tune THS-SETTLE parameter of receiver [Public]
- 14. © 2022 MIPI Alliance, Inc. 14 Summary • SOM with industry standard interfaces & communication protocols reduces development time • SOM facilitates application and HW design with a greater degree of abstraction from chip-down designs • SOM FPGA PL provides flexible I/O and I/O debug capabilities • Closure of “Global timing parameters” to ease system level challenges [Public]
- 15. © 2022 MIPI Alliance, Inc.
- 16. © 2022 MIPI Alliance, Inc.