Ei2004 presentation
1 like551 views
The document summarizes two approaches to implementing foveated imaging in CMOS image sensors: (1) A pyramidal architecture with multiple rings of pixels having different integration times, allowing for dynamic range enhancement. (2) A universal multiresolution sensor using a 3T pixel design that allows pixels to be grouped and averaged, enabling adaptive resolution. Both designs aim to mimic the human retina and improve efficiency over traditional sensors. The pyramidal and multiresolution sensors were fabricated in 0.18um CMOS technology and are being tested for applications like video conferencing and industrial inspection.
Ei2004 presentation
- 1. Foveated Architectures for CMOS Image Sensors Fayçal Saffih1, Richard Hornsey2 1 Integrated Camera Group, Electrical & Computer Engineering Department, University of Waterloo, ON, Canada 2 Centre for Vision Research, Department of Computer Science, York University, Toronto, Ontario, Canada Electronic Imaging 2004, San Jose
- 2. Outline Introduction & Motivation Motivation Implementation Pyramidal CMOS Image Sensor Applications Universal Multiresolution CMOS Image Sensor Applications Final Conclusion. Electronic Imaging 2004, San Jose
- 3. Introduction & Motivation CMOS image sensor development levels: Device level: Optical charge devices such as photodiodes. Goal: Dynamic range enhancement, Fill factor increase…etc. Circuit Level: Amplifiers, buffers, ADC’s…etc. Goal: Efficient charge transfer, high speed acquisition, noise filtering…etc System Level: Signal processing blocks such as motion detection, image processing blocks, …etc. Goal: Image information extraction (motion, segmentation…etc), performance enhancement (data reduction, multiresolution, foveated vision…etc). Motivation Imitation of biological visual systems such as Human Visual System. Efficient Data Reduction. Efficient Image Information Transfer. Minimization of Power Consumption. Emphasize on the importance of architecture design in CMOS Imagers. Philosophy: Image acquisition is a Vision needs vision Electronic Imaging 2004, San Jose
- 4. Motivation Implementation System level implementation The pixel structure is the 3T Active Pixel Sensor (APS). The acquisition system is a non-classical (non- orthogonal) architecture. Solution: Pyramidal CMOS image sensor Device level implementation The pixel structure is a non-classical (3T active pixel sensor). The acquisition system is a classical orthogonal architecture. Solution: Universal Multiresolution CMOS Image Sensor Electronic Imaging 2004, San Jose
- 5. System Level approach Pyramidal CMOS Image Sensor Electronic Imaging 2004, San Jose
- 6. Classical Acquisition System The orthogonal acquisition architecture along with raster scanning suffers from the following issues: 1D sampling architecture. Unique integration time Different sampling speed between horizontal and vertical axis of the image. This leads to anisotropic distribution of motion blur, being higher in the vertical axis than in the horizontal. Electronic Imaging 2004, San Jose
- 7. Human Fovea Architecture Dynamic Range is higher in cones than Circular symmetry of the fovea photocells rods. (rods and cones) has lead to symmetrical yet HVS system DR (~106) is much larger co-centric image sampling. than the photoreceptor DR (~102). Electronic Imaging 2004, San Jose
- 8. Pyramidal Acquisition Architecture Electronic Imaging 2004, San Jose
- 9. Pyramidal Acquisition Architecture Floorplan of the imager is composed of square pixels (16µmx16µm) orthogonally compacted. Reset and select signals are shared among each ring The output buses are diagonals to dump the ring output into 8 CDS block at the base of the pyramid Electronic Imaging 2004, San Jose
- 10. Bouncing Scanning Scheme Estimated Integration time of a ring r for inward Scanning: r +1 Trin = 2 ∑ iT s + ( R − r )Tspl + rT s i = R i → i −1 Estimated Integration time of a ring r for outward Scanning: r −1 Trout = 2 ∑ iTs + ( r − 1)Tspl + rTs i =1 i → i +1 Electronic Imaging 2004, San Jose
- 11. Ring Integration Time Estimated of all rings’ integration time for inward (ISc) and Rings’ RMS of inward (ISc) and bounced scanning bounced scanning (BSc) at 40Fps. (BSc) at 8Fps under light intensity of 43.33uW (current testing) Electronic Imaging 2004, San Jose
- 12. Dynamic Range Fovea Each ring in the pyramidal imager The resulting dynamic range will have two different integration enhancement is made with two time. Fusing their two output will acquired scenes. Thus, this type enhance the ring’s dynamic range of dynamic range enhancement is by:20log(Tint1/Tint2), where Tint1 ≥Tint2 called, intrascene dynamic range. Electronic Imaging 2004, San Jose
- 13. Experimental Results Inward and Bounced Images from the Pyramid Imager at 14lux@29Fps Fused Image of the inward Pyramidal CMOS Image Sensor and bounced images Layout Electronic Imaging 2004, San Jose
- 14. Applications Applications that need optimal data transfer such as: Video-Phone Internet cameras Applications that need foveated vision: IndustrialInspection Surveillance Cameras Low Vision Enhancement Consumers Cameras Electronic Imaging 2004, San Jose
- 15. Disadvantages Complex data structure for image re-construction Mismatch between sampling capacitors between the different pyramid clusters may create some undesired artifacts. The central diagonal in the pyramid is sampled by capacitors with twice the capacity of their neighbor pixels. Although the above are real problems in our CMOS imager, they are not limitations as they are all solvable problems. Electronic Imaging 2004, San Jose
- 16. Conclusion New architecture (hardware) and scanning scheme (software) for CMOS imagers have been suggested. The benefits of the new approach in image sampling are due to: The 2D nature of the sampling rings Centricity of the sampling units has lead to foveated dynamic range enhancement topology. The difference in sampling architectures in CMOS imagers greatly impacts their performances. Electronic Imaging 2004, San Jose
- 17. Device Level approach Universal Multiresolution CMOS Image Sensor Electronic Imaging 2004, San Jose
- 18. Multiresolution CMOS Image Sensor Motivation Only small parts of image are regions of interest. Regions of low interest need to be sub-sampled or averaged. With increasing image resolution the ultimate limitation is image transfer especially for video broadcasting. Minimizing of power consumption requires a minimization of power consumption of the imager OR minimization of “data of interest” to be readout Programmability and expandability are the most important feature of such an architecture in order for it to span the largest field of applications: Universality Electronic Imaging 2004, San Jose
- 19. The Kernelling Just two clock cycles are enough for the kernel averaging: Fast Kernelling Electronic Imaging 2004, San Jose
- 20. Building Block: The Pixel Electronic Imaging 2004, San Jose
- 21. Multiresolution Decoder for Row reset and select CDS Block CDS Block CDS Block CDS Block CDS Block CDS Block CDS Block CDS Block CDS Block The Architecture CDS Block CDS Block Multiresolution Decoder for CDS Block CDS Block Decoder for Column Select of CDS Block CDS Block Correlated Double Sampling (CDS) CDS Block CDS Block Row-Average Support & Column-Average CDS Block Multiresolution Decoder for Row-Average and Sampling Electronic Imaging 2004, San Jose Image Output
- 22. Application: Programmability Fundamental Foveated Multiresolution Random Foveated Multiresolution Multi-Foveated Image Sampling Electronic Imaging 2004, San Jose
- 23. Application: Programmability Fundamental Foveated Multiresolution Fundamental Foveated Multiresolution With horizontally-rectangular kernels With vertically-rectangular kernels Effect of kernel topology on spatial filtering Electronic Imaging 2004, San Jose
- 24. Applications Multiple Object Tracking Industrial inspection Image processing prototyping Remote image acquisition requiring minimal image transfer bandwidth. Electronic Imaging 2004, San Jose
- 25. Final Conclusion Two different approaches for implementing foveated imaging have been suggested. The two imagers were fabricated in dual voltage 1P6M standard 0.18µm CMOS technology. The two imagers under test. Parallelism, programmability and expandability were the keys behind the proposed CMOS image sensors’ architectures Electronic Imaging 2004, San Jose
- 26. Acknowledgement The authors are grateful to: Canadian Microelectronics Corporation. NSERC Canada. Betacom Inc. Thank you ! Electronic Imaging 2004, San Jose
Editor's Notes
- #4: Efficient data reduction=transferring just important (central) image information. Enhance Image Information Transfer=creating enough room for transferring important image information (details)
- #7: Viewpoint: Sampling architecture is not necessarily the same as the display architecture. Raster scanning was originally suggested for CRT displays but was adopted since 60’s for MOS imagers. Biological sampling architectures are very promising to implement efficient smart sampling architectures. Human fovea is a clear example.
- #11: 1- Integration time is the time elapsed between successive ring reset and ring signal readout. 2- Ring integration time in the bouncing scanning scheme is deduced including the bouncing at the inner and outer rings. 3- Due to the different of cardinals of the inner and outer rings, bouncing scanning at the inner ring will result lower integration time for outward scanning than bouncing at the outer ring for the inward scanning.