1.pdf
- 1. Introduction to Computer Vision
- 2. What is Computer Vision? Nice sunset! “Making computers see and understand”
- 3. Main Objectives: Theory + Algorithms • Development of the theoretical and algorithmic basis by which useful information about the 3D world can be automatically extracted and analyzed from a single or multiple 2D images of the world.
- 4. Origins of computer vision L. G. Roberts, Machine Perception of Three Dimensional Solids Ph.D. thesis, MIT Department of Electrical Engineering, 1963
- 5. Progress in Computer Vision • First generation: Military/Early Research – Few systems, each custom-built, cost $Ms – “Users” have PhDs – 1 hour per frame • Second generation: Industrial/Medical – Numerous systems, 1000s of each, cost $10Ks – “Users” have college degree – Special hardware • Third generation: Consumer – 100000(00) systems, cost $100s – “Users” have little or no training – Emphasis on software Medical and scientific images
- 6. Computer Vision, Also Known As ... • Computational Vision – Includes modeling of biological vision • Image Understanding – Automated scene analysis (e.g., satellite images, robot navigation) • Machine Vision – Industrial, factory-floor systems for inspection, measurements, part placement, etc.
- 7. Connections to other disciplines Computer Vision Image Processing Pattern Recognition & Machine Learning Artificial Intelligence Robotics Psychology & Neuroscience Computer Graphics
- 10. Computer Graphics Image Output: Geometric Models Synthetic Camera Projection, shading, lighting models
- 11. Computer Vision
- 13. Why is Computer Vision Difficult? (1) It is a many-to-one mapping – A variety of surfaces having different material and geometrical properties, possibly under different lighting conditions, could lead to similar images. – Inverse mapping has non-unique solution; a lot of information is lost in the transformation from the 3D world to the 2D image.
- 14. Why is Computer Vision Difficult? (cont’d) (2) It is computationally intensive - A typical video is 30 frames / sec (3) We do not understand the recognition problem!
- 15. Main Challenges • Viewpoint variations • Illumination changes • Scale changes • Deformation • Occlusions • Background clutter • Motion • Intra/Inter-class variations
- 18. Scale changes
- 19. Deformations
- 20. Occlusions
- 22. Motion blurring
- 24. Local ambiguity
- 25. Three Processing Levels (1) Low Level (2) Mid Level (3) High Level
- 26. Low Level Vision
- 27. Low Level Vision - Examples Corner and blob detection Edge detection
- 28. Low Level Vision - Examples • Region segmentation
- 29. Mid Level Vision
- 30. Mid Level Vision - Examples • 3D Reconstruction
- 31. Mid Level Vision - Examples • Structure (i.e., 3D) from motion 3D teacup model reconstructed from a 240-frame video sequence Optical flow
- 33. slide credit: Fei-Fei, Fergus & Torralba Scene Interpretation
- 34. Object categorization sky building flag wall banner bus cars bus face street lamp slide credit: Fei-Fei, Fergus & Torralba
- 35. Qualitative geometric information slanted rigid moving object horizontal vertical slide credit: Fei-Fei, Fergus & Torralba rigid moving object non-rigid moving object
- 36. Scene and context categorization • outdoor • city • traffic • … slide credit: Fei-Fei, Fergus & Torralba
- 37. Visual Cues • People use information from various visual cues for recognition (e.g., color, shape, texture etc.) (1) How important is each visual cue? (2) How do we combine information from various visual cues?
- 38. Color Cues
- 39. Texture Cues
- 40. Shape Cues
- 41. Grouping Cues Similarity (color, texture, proximity)
- 42. Depth Cues
- 43. Shading Cues Source: J. Koenderink
- 44. Computer Vision Applications • Industrial inspection/quality control • Surveillance and security • Face recognition • Gesture recognition • Space applications • Medical image analysis • Autonomous vehicles • Virtual reality and much more …...
- 45. Industrial Computer Vision (Machine Vision) Industrial computer vision systems work really well. Make strong assumptions about lighting conditions Make strong assumptions about the position of objects Make strong assumptions about the type of objects
- 47. Optical character recognition (OCR) Digit recognition, AT&T labs http://yann.lecun.com/exdb/lenet/ • Technology to convert scanned docs to text License plate readers http://en.wikipedia.org/wiki/Automatic_number_plate_recognition Automatic check processing
- 48. Biometrics
- 49. Login without a password… Fingerprint scanners on many new laptops, other devices Face recognition systems now beginning to appear more widely http://www.sensiblevision.com/
- 51. Hand-based Biometrics at G. Amayeh, G. Bebis, A. Erol, and M. Nicolescu, "Hand-Based Verification and Identification Using Palm-Finger Segmentation and Fusion", Computer Vision and Image Understanding (CVIU) vol 113, pp. 477-501, 2009.
- 52. Fingerprint Biometrics minutiae small overlapping area matching input ID
- 53. Fingerprint Biometrics at Super-Template Synthesis matching ID super-template T. Uz, G. Bebis, A. Erol, and S. Prabhakar, "Minutiae-Based Template Synthesis and Matching for Fingerprint Authentication", Computer Vision and Image Understanding (CVIU), vol 113, pp. 979-992, 2009.
- 54. How the Afghan Girl was Identified by Her Iris Patterns Iris Biometrics
- 56. Object Recognition (cont’d) (1) Object-centered (2) Viewer-centered
- 57. Object Recognition at Synthesize new 2D views of a 3D object using linear combinations of a set of 2D reference views
- 58. Object Recognition at reference view 1 reference view 2 • No 3D models required. • “Predict” novel 2D views from known 2D views W. Li, G. Bebis, and N. Bourbakis, "3D Object Recognition Using 2D Views", IEEE Transactions on Image Processing, vol. 17, no. 11, pp. 2236-2255, 2008. novel view recognized
- 59. Object Recognition at Reference Views Recognition Results
- 60. Segmentation
- 61. Segmentation at L. Loss, G. Bebis, M. Nicolescu, and A. Skurikhin, "An Iterative Multi-Scale Tensor Voting Scheme for Perceptual Grouping of Natural Shapes in Cluttered Backgrounds", Computer Vision and Image Understanding (CVIU), vol. 113, no. 1, pp. 126-149, January 2009. Iterative Tensor Voting
- 62. Object Recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart camera is flush-mounted in the checkout lane, continuously watching for items. When an item is detected and recognized, the cashier verifies the quantity of items that were found under the basket, and continues to close the transaction. The item can remain under the basket, and with LaneHawk,you are assured to get paid for it… “
- 63. Image Retrieval • Color, texture http://corbis.demo.ltutech.com/en/demos/corbis/
- 66. Face Detection • Many new digital cameras now detect faces – Canon, Sony, Fuji, …
- 67. Face Detection at J. Dowdall, I. Pavlidis, and G. Bebis, "Face Detection in the Near-IR Spectrum", Image and Vision Computing, vol 21, no. 7, pp. 565-578, 2003. Human skin exhibits an abrupt change in reflectance around 1.4 mm.
- 69. Face Recognition: Apple iPhoto http://www.apple.com/ilife/iphoto/
- 70. Face Recognition at • Visible spectrum – High resolution, less sensitive to the presence of eyeglasses. – Particularly sensitive to changes in illumination direction and facial expression. • Thermal IR spectrum – Not sensitive to illumination changes. – Low resolution, sensitive to air currents, face heat patterns, aging, and the presence of eyeglasses (i.e., IR is opaque to glass). LWIR visible
- 71. Face Recognition at Fuse visible with thermal infrared imagery G. Bebis, A. Gyaourova, S. Singh, and I. Pavlidis, "Face Recognition by Fusing Thermal Infrared and Visible Imagery", Image and Vision Computing, vol. 24, no. 7, pp. 727-742, 2006. Wavelet Transform Fusion Using Genetic Algorithms Inverse Wavelet Transform Fused Image
- 72. Gender Classification at Discover gender-specific features using Genetic Algorithms (GAs) Z. Sun, G. Bebis, X. Yuan, and S. Louis, "Genetic Feature Subset Selection for Gender Classification: A Comparison Study", IEEE Workshop on Applications of Computer Vision, pp. 165-170, 2002.
- 73. Gender Classification at Original images Reconstructed using traditional features Reconstructed using GA-based features Preserve gender-related information but not identity specific features! Z. Sun, G. Bebis, and R. Miller, "Object Detection Using Feature Subset Selection", Pattern Recognition, vol. 37, pp. 2165-2176, 2004.
- 76. Smile detection? Sony Cyber-shot® T70 Digital Still Camera
- 77. Hand Gesture Recognition • Smart Human-Computer User Interfaces • Sign Language Recognition
- 78. Vision-based Interaction and Games Nintendo Wii has camera-based IR tracking built in. See Lee’s work at CMU on clever tricks on using it to create a multi-touch display! Assistive technologies Kinect
- 79. Visual Surveillance and Human Activity Recognition Surveillance and security
- 80. Human Activity Recognition at • Recognize simple human actions using 3D head trajectories J. Usabiaga, G. Bebis, A. Erol, Mircea Nicolescu, and Monica Nicolescu, "Recognizing Simple Human Actions Using 3D Head Trajectories", Computational Intelligence (special issue on Ambient Intelligence), vol. 23, no. 4, pp. 484-496, 2007.
- 82. Vehicle Detection and Tracking at Ford’s low light camera Ford’s Concept Car
- 83. Vehicle Detection and Tracking • Can process 10 fps on average; 6% errorrs (FP + FN) (a) (b) FN FP Z. Sun, G. Bebis, and R. Miller, "Monocular Pre-crash Vehicle Detection: Features and Classifiers", IEEE Transactions on Image Processing , vol. 15, no. 7, pp. 2019-2034, July 2006.
- 84. Smart cars: – Vision systems currently in high-end BMW, GM, Volvo models. Mobileye
- 86. Automatic Panorama Stitching (cont’d)
- 87. 3D urban modeling: Photosynth http://photosynth.net/ Photosynth allows you to take a bunch of photos of the same scene or object and automatically stitch them all together into one big interactive 3D viewing experience
- 88. Automatic 3D reconstruction from internet photo collections “Statue of Liberty” 3D model Flickr photos “Half Dome, Yosemite” “Colosseum, Rome”
- 89. Robotics http://www.robocup.org/ Semantic Robot Vision Challenge http://www.semantic-robot-vision-challenge.org/ http://www.youtube.com/watch?v=GItjILILB50
- 90. Vision in space • Vision systems used for several tasks – Panorama stitching – 3D terrain modeling – Obstacle detection, position tracking – For more, read “Computer Vision on Mars” by Matthies et al. NASA'S Mars Exploration Rover Spirit
- 91. Movie Special Effects Movie special effects • Insert synthetic objects in real image sequences; • Change artificially the position or the orientation of a camera; • Freeze a moving 3D scene.
- 92. Medical Imaging Skin/Breast Cancer Detection 3D imaging MRI, CT Enable surgeons to visualize internal structures through an automated overlay of 3D reconstructions of internal anatomy on top of live video views of a patient. Image guided surgery Grimson et al., MIT
- 94. Computer Vision Jobs !! • Academia – MIT, UC-Berkeley, CMU, UIUC, USC …… UNR! • National Labs and Government – Los Alamos National Lab – Lawrence Livermore National Lab – Navy, Air-force, Army • Industry – Microsoft, Intel, IBM, Xerox, Compaq, Siemens, HP, TI, Motorola, Phillips, Honeywell, Ford See: http://www.cs.ubc.ca/spider/lowe/vision.html
- 95. What skills do you need to succeed in this field? • Strong programming skills (i.e., C, C++, Matlab) • Good knowledge of Data Structures and Algorithms • Good skills in analyzing algorithm performance (i.e., time and memory requirements). • Good background in mathematics, especially in: – Linear Algebra – Probabilities and Statistics – Numerical Analysis
- 96. Related Courses at UNR • CS474/674 Image Processing and Interpretation • CS485/685 Computer Vision • CS486/686 Advanced Computer Vision • CS480/680 Computer Graphics • CS479/679 Pattern Recognition • CS482/682 Artificial Intelligence • CS773A Machine Intelligence • CS791Q Machine Learning • Special Topics – Biometrics, Object Recognition, Neural Networks, – Mathematical Methods for Computer Vision
- 97. More information on Computer Vision • Computer Vision Home Page http://www.cs.cmu.edu/afs/cs/project/cil/ftp/html/vision .html • UNR Computer Vision Laboratory http://www.cs.unr.edu/CVL