"Memory Innovation for Embedded Vision Systems," a Presentation from Samsung Electronics
1 like983 views
The presentation discusses Samsung's innovations in memory technology for embedded vision systems, emphasizing the growing demand for high-performance and power-efficient memory in mobile devices, automotive ADAS, and VR applications. It highlights the need for collaboration in developing advanced memory solutions that cater to the evolving requirements of smart devices and AI capabilities. Key topics include the adaptation of memory technology to support higher bandwidth and efficiency in the context of rapidly increasing data generation.
![10/20
Automotive, evolving fast
0
20
40
60
'12 13 '14 15 '16 17
Mobile
Auto
0
2
4
6
8
10
'12 13 '14 15 '16 17
Mobile
Auto
DRAM Bandwidth DRAM Capacity
GB/s GB
Vision
Sensor
D/L Data
Processing
100
50
200
[GB/s]
300
‘21‘17 ‘19
OS , Apps
D/L Weight
8x
Number of objects to recognize
40ea(’13) ▶ 1,000ea (’18)
Bandwidth for Compute ADAS
• Autonomous driving adoption, faster than expected
• Compute ADAS drives high B/W (object detection/scene seg./depth extraction)
Source: Samsung](https://image.slidesharecdn.com/10embeddedvisionalliancemembermeetingfebruary2017samsung-170328180005/85/Memory-Innovation-for-Embedded-Vision-Systems-a-Presentation-from-Samsung-Electronics-10-320.jpg)
![11/20
VR, high-performance & low-power
• VR now : FHD 90fps, MTP* latency(~50ms), 2D audio VR sickness
• Immersive VR : 160FoV, 8K120fps, MTP(~20ms), 360 audio
Enthusiastic Game
Professional/Industrial
Military/Health Care
Education/Retail
VR Cinema
360 Video Streaming
Casual Game
8K120 streaming, ~51.2GB/s
4GB+
8K120 commercial, ~150GB/s
8GB+
8K120 gaming, 1TB+/s
16GB+
TetheredAllInOneDropIn
4K65 4K75 4K120 8K604K90
Display
17 13 911
Latency
’16 ’17 ’18 ’19 ’20
ms
TDP
[Watt]
100 300200
All-in-One
Tethered
LP4/5(~34GB/s)
HBM2/GD6(~500GB/s)
LP5/WIO(~60GB/s)
GPU[TFLOPS]
8
3
1
∬
Drop-In
fps
Performance
Low
Power
10
<9
*MTP: Motion To Photon
Type & DRAM Requirements VR Ecosystem Readiness
Source: Samsung](https://image.slidesharecdn.com/10embeddedvisionalliancemembermeetingfebruary2017samsung-170328180005/85/Memory-Innovation-for-Embedded-Vision-Systems-a-Presentation-from-Samsung-Electronics-11-320.jpg)
![13/20
Memory technology trend
Power Efficiency
[mW/GBps]
100%
80%
60%
40%
20%
20202016 2018
Performance
[Gbps/pin]
15
12
6
3
LP5
LP4X
LP4
2016 2018 2020
DDR4
9
DDR5
GDDR5
LP4
• GDDR6 with over 14Gbps, beyond 10Gbps GDDR5
• LP5, 20% more power-efficient than LP4X
LP5
GDDR6
DDR5
LP4X
GDDR5
DDR4
LP3
DDR3
Source: Hotchips2016, Samsung
GDDR6](https://image.slidesharecdn.com/10embeddedvisionalliancemembermeetingfebruary2017samsung-170328180005/85/Memory-Innovation-for-Embedded-Vision-Systems-a-Presentation-from-Samsung-Electronics-13-320.jpg)
![17/20
A.I. mobile memory
[ source : ISSCC]
DDR Main Memory
(DDR4/5)
PCIeGen-Z/CCIX/CAPI
Accelerator
(GPU/FPGA/ASIC)
CPU
NPU
HBMs
LPDDR Main Memory
(LP4/5)
AMBA/AHB
Accelerator
(DSP/VPU/NPU)
AP
A.I . mem
iGPU iGPU
Inference
(DSP/NPU/VPU)Training
(GPU/FPGA/ASIC) Trained
model
Output
Energy efficient solution requirement
Dedicated H/W
(accelerator + memory)
• Vision IoT requires A.I. specific memory
‒ Deep-learning/parallel/inference processing
Deep-learning on IoT device
Source: Samsung](https://image.slidesharecdn.com/10embeddedvisionalliancemembermeetingfebruary2017samsung-170328180005/85/Memory-Innovation-for-Embedded-Vision-Systems-a-Presentation-from-Samsung-Electronics-17-320.jpg)
"Memory Innovation for Embedded Vision Systems," a Presentation from Samsung Electronics
- 1. February 22nd, 2017 l JIN KIM l Samsung Electronics Memory innovation for embedded vision systems
- 2. 2/20 Disclaimer This presentation is intended to provide information concerning memory industry. We do our best to make sure that information presented is accurate and fully up-to-date. However, the presentation may be subject to technical inaccuracies, information that is not up-to-date or typographical errors. As a consequence, Samsung does not in any way guarantee the accuracy or completeness of information provided on this presentation. Samsung reserves the right to make improvements, corrections and/or changes to this presentation at any time. The information in this presentation or accompanying oral statements may include forward-looking statements. These forward-looking statements include all matters that are not historical facts, statements regarding the Samsung Electronics' intentions, beliefs or current expectations concerning, among other things, market prospects, growth, strategies, and the industry in which Samsung operates. By their nature, forward-looking statements involve risks and uncertainties, because they relate to events and depend on circumstances that may or may not occur in the future. Samsung cautions you that forward looking statements are not guarantees of future performance and that the actual developments of Samsung, the market, or industry in which Samsung operates may differ materially from those made or suggested by the forward-looking statements contained in this presentation or in the accompanying oral statements. In addition, even if the information contained herein or the oral statements are shown to be accurate, those developments may not be indicative developments in future periods.
- 3. 3/20 Contents • Vision IoT, divergence • Embedded vision memory ‒ Auto/VR/Mobile • Innovative memory solutions • Summary
- 5. 5/20 To divergence DIVERGENCECONVERGENCE • Better smartphone UX with vision IoTs • Need divergence of vision IoT
- 6. 6/20 Changing role, enriching mobile eco Service Intelligent Hub Data Explosion 6 10 15 22 31 '15 '16 '17 '18 '19 '20 10X Data Generation & Distribution Personalization Cloud Data Analytics, Deep Learning Source: Global Mobile Data Traffic, CISCO’16 Active Interaction
- 7. 7/20 For artificial intelligence, On-device • On-device service expansion ‒ Privacy/security/picture quality On-device A.I.(deep learning) ‒ Energy-efficiency/high-bandwidth/low-latency Accuracy(# of layers) Coverage (10 layers) (8 layers) Object detect Medical Natural Language Audio Translation Self-Drive Face pay Number Recognition Cloud service On-device service Recommend Super resolution 4-layer 5-layer 6-layer
- 8. 8/20 Embedded vision memory : Auto/VR/Mobile
- 9. 9/20 Vision IoT, more memory Influence level on memory Imaging (CIS+memory) Recognition (SoC+memory) A.I. (Cloud+memory) Wearable ★ ★ Automotive ★★ ★★ ★★★★ Camera ADAS ★★ ★★★ IP camera ★★ ★★ Robot/Drone ★ ★★ ★★★ Gateway/Edge ★ ★★ Cellular IoT/M2M ★ Smart Home ★ ★ ★ Retail ★ ★ Automotive ADAS Tele- matics Info/ Cluster IP CAM Smart Gateway Robotics Drone Smart Home Thermo -stat Appliance Wearable IoT (50B unit) 100M C-IoT/ M2M Retail • Vision for Imaging Vision for Sensing and A.I. • Top 10 IoT applications, driving memory consumption Imaging: improved picture quality/high speed shooting/continuous shooting Recognition: 3D mapping/multi-camera/object/motion/biometrics/face recognition A.I.: autonomous/emotion recognition/disease diagnosis/VR
- 10. 10/20 Automotive, evolving fast 0 20 40 60 '12 13 '14 15 '16 17 Mobile Auto 0 2 4 6 8 10 '12 13 '14 15 '16 17 Mobile Auto DRAM Bandwidth DRAM Capacity GB/s GB Vision Sensor D/L Data Processing 100 50 200 [GB/s] 300 ‘21‘17 ‘19 OS , Apps D/L Weight 8x Number of objects to recognize 40ea(’13) ▶ 1,000ea (’18) Bandwidth for Compute ADAS • Autonomous driving adoption, faster than expected • Compute ADAS drives high B/W (object detection/scene seg./depth extraction) Source: Samsung
- 11. 11/20 VR, high-performance & low-power • VR now : FHD 90fps, MTP* latency(~50ms), 2D audio VR sickness • Immersive VR : 160FoV, 8K120fps, MTP(~20ms), 360 audio Enthusiastic Game Professional/Industrial Military/Health Care Education/Retail VR Cinema 360 Video Streaming Casual Game 8K120 streaming, ~51.2GB/s 4GB+ 8K120 commercial, ~150GB/s 8GB+ 8K120 gaming, 1TB+/s 16GB+ TetheredAllInOneDropIn 4K65 4K75 4K120 8K604K90 Display 17 13 911 Latency ’16 ’17 ’18 ’19 ’20 ms TDP [Watt] 100 300200 All-in-One Tethered LP4/5(~34GB/s) HBM2/GD6(~500GB/s) LP5/WIO(~60GB/s) GPU[TFLOPS] 8 3 1 ∬ Drop-In fps Performance Low Power 10 <9 *MTP: Motion To Photon Type & DRAM Requirements VR Ecosystem Readiness Source: Samsung
- 12. 12/20 Mobile, overcoming technology barrier SOC Performance DRAM Performance 0.1 1 10 100 ‘10 ‘12 ‘14 ‘16 (log) ‘18 BenchmarkScore(3Dgraphic) 0 1.6 3.2 4.8 '10 '12 '14 '16 '18 Gbps Time ‘12 ‘14 ‘16 • Need performance, but can’t increase it Source: Samsung
- 13. 13/20 Memory technology trend Power Efficiency [mW/GBps] 100% 80% 60% 40% 20% 20202016 2018 Performance [Gbps/pin] 15 12 6 3 LP5 LP4X LP4 2016 2018 2020 DDR4 9 DDR5 GDDR5 LP4 • GDDR6 with over 14Gbps, beyond 10Gbps GDDR5 • LP5, 20% more power-efficient than LP4X LP5 GDDR6 DDR5 LP4X GDDR5 DDR4 LP3 DDR3 Source: Hotchips2016, Samsung GDDR6
- 15. 15/20 High Bandwidth Memory: HBM PCB DRAM BufferLogic Processor Si Interposer HBM TSV Technology 1,024 I/O Architecture Benefits Microbump 8H stacked 20nm 8GB HBM HBM GDDR5 X 0.8 Power Efficiency High Bandwidth1TB/s X 2.7 Performance HBM GDDR5 Source: Samsung
- 16. 16/20 3D/2.5D SiP memory PKG • Mobile, small F/F, high-speed, low-power requirement 3D/2.5D SiP PKG • Close collaboration with SoC Stacked FBGA FOPLP Dual Flip Chip 2.5D Si Interposer 3D SiP Performance Form Factor time SiP PoP Wire Bump FOPLP TSV Mobile MemorySVR/Gfx Memory time TSV Wire /FPGA Wire/FBGA FC/TSV High-IO Interposer High IO TSV (HBM) Interposer Based Platform DDR TSV FC-CSP Stacked FBGA BOC interposer Performance Source: Samsung
- 17. 17/20 A.I. mobile memory [ source : ISSCC] DDR Main Memory (DDR4/5) PCIeGen-Z/CCIX/CAPI Accelerator (GPU/FPGA/ASIC) CPU NPU HBMs LPDDR Main Memory (LP4/5) AMBA/AHB Accelerator (DSP/VPU/NPU) AP A.I . mem iGPU iGPU Inference (DSP/NPU/VPU)Training (GPU/FPGA/ASIC) Trained model Output Energy efficient solution requirement Dedicated H/W (accelerator + memory) • Vision IoT requires A.I. specific memory ‒ Deep-learning/parallel/inference processing Deep-learning on IoT device Source: Samsung
- 18. 18/20 Memory-stacked Photography • Require DSLR-level performance (dual camera/pixel and high-speed shooting) • Chance to processing in stacked memory ‒ Low power, thermal spread, super resolution, real time HDR and slow motion control Source: ISSCC 2017, Sony e.g. data rate control, high speed read-out, multi stream output, high speed binning
- 19. 19/20 PIM for embedded vision IoT • Added value from reduced power consumption ‒ Reduce the unnecessary data transfer and frame rate control • Possible collaboration with SoC/AP APCISDisplay AMBA AHB Display CIS AP Added ValueMemory B/W Traffic VPU Recognition Distortion FRCCorrection Pre/Post Processing In Memory Source: Samsung
- 20. 20/20 Summary • Smartphone as a Intelligent Hub will continue to enrich mobile ecosystem ‒ Performance never be enough in mobile. The more we have, the better we use ‒ Mobile challenges with power optimization and continued innovation • High performance, more memory are needed in embedded vision IoT ‒ Automotive ADAS drives high bandwidth memory ‒ Immersive VR requires high-performance and low-power ‒ Mobile memory is getting more power-efficient • Close collaboration is essential ‒ Keeps innovating technology to correspond to the requirements ‒ Artificial Intelligent memory, CIS stacked memory, Vision processing in memory