Lec01 introduction
- 1. COMP 776: Computer Vision
- 2. Basic Info • Instructor: Svetlana Lazebnik (lazebnik@cs.unc.edu) • Office hours: By appointment, FB 244 • Textbook (recommended): Forsyth & Ponce, Computer Vision: A Modern Approach • Class webpage: http://www.cs.unc.edu/~lazebnik/spring09
- 3. Today • Introduction to computer vision • Course overview • Course requirements
- 4. The goal of computer vision • To perceive the story behind the picture What we see What a computer sees Source: S. Narasimhan
- 5. The goal of computer vision • To perceive the story behind the picture • What exactly does this mean? – Vision as a source of metric 3D information – Vision as a source of semantic information
- 6. Vision as measurement device Real-time stereo Structure from motion NASA Mars Rover Pollefeys et al. Multi-view stereo for community photo collections Goesele et al.
- 7. Vision as a source of semantic information slide credit: Fei-Fei, Fergus & Torralba
- 8. Object categorization sky building flag wall banner bus cars bus face street lamp slide credit: Fei-Fei, Fergus & Torralba
- 9. Scene and context categorization • outdoor • city • traffic • … slide credit: Fei-Fei, Fergus & Torralba
- 10. Qualitative spatial information slanted rigid moving object horizontal vertical slide credit: Fei-Fei, Fergus & Torralba rigid moving object non-rigid moving object
- 11. Why study computer vision? Personal photo albums Surveillance and security Movies, news, sports Medical and scientific images • Vision is useful: Images and video are everywhere!
- 12. Why study computer vision? • Vision is useful • Vision is interesting • Vision is difficult – Half of primate cerebral cortex is devoted to visual processing – Achieving human-level visual perception is probably “AI-complete”
- 13. Why is computer vision difficult?
- 14. Challenges: viewpoint variation Michelangelo 1475-1564 slide credit: Fei-Fei, Fergus & Torralba
- 15. Challenges: illumination image credit: J. Koenderink
- 16. Challenges: scale slide credit: Fei-Fei, Fergus & Torralba
- 17. Challenges: deformation Xu, Beihong 1943 slide credit: Fei-Fei, Fergus & Torralba
- 18. Challenges: occlusion Magritte, 1957 slide credit: Fei-Fei, Fergus & Torralba
- 21. Challenges: object intra-class variation slide credit: Fei-Fei, Fergus & Torralba
- 22. Challenges: local ambiguity slide credit: Fei-Fei, Fergus & Torralba
- 23. Challenges or opportunities? • Images are confusing, but they also reveal the structure of the world through numerous cues • Our job is to interpret the cues! Image source: J. Koenderink
- 24. Depth cues: Linear perspective
- 25. Depth cues: Aerial perspective
- 26. Depth ordering cues: Occlusion Source: J. Koenderink
- 27. Shape cues: Texture gradient
- 28. Position and lighting cues: Cast shadows Source: J. Koenderink
- 29. Grouping cues: Similarity (color, texture, proximity)
- 30. Grouping cues: “Common fate” Image credit: Arthus-Bertrand (via F. Durand)
- 31. Bottom line • Perception is an inherently ambiguous problem – Many different 3D scenes could have given rise to a particular 2D picture Image source: F. Durand
- 32. Bottom line • Perception is an inherently ambiguous problem – Many different 3D scenes could have given rise to a particular 2D picture • Possible solutions – Bring in more constraints (more images) – Use prior knowledge about the structure of the world • Need a combination of different methods Image source: F. Durand
- 33. Connections to other disciplines Computer Vision Image Processing Machine Learning Artificial Intelligence Robotics Psychology Neuroscience Computer Graphics
- 34. Origins of computer vision L. G. Roberts, Machine Perception of Three Dimensional Solids, Ph.D. thesis, MIT Department of Electrical Engineering, 1963.
- 35. Progress to date The next slides show some examples of what current vision systems can do
- 36. Earth viewers (3D modeling) Image from Microsoft’s Virtual Earth (see also: Google Earth) Source: S. Seitz
- 37. Photosynth http://labs.live.com/photosynth/ NOTE: Noah Snavely talk at GLUNCH tomorrow! Source: S. Seitz
- 38. Optical character recognition (OCR) Digit recognition, AT&T labs http://www.research.att.com/~yann/ Technology to convert scanned docs to text • If you have a scanner, it probably came with OCR software License plate readers http://en.wikipedia.org/wiki/Automatic_number_plate_recognition Source: S. Seitz
- 39. Face detection Many new digital cameras now detect faces • Canon, Sony, Fuji, … Source: S. Seitz
- 40. Smile detection? Sony Cyber-shot® T70 Digital Still Camera Source: S. Seitz
- 41. Object recognition (in supermarkets) LaneHawk by EvolutionRobotics “A smart camera is flush-mounted in the checkout lane, continuously watching for items. When an item is detected and recognized, the cashier verifies the quantity of items that were found under the basket, and continues to close the transaction. The item can remain under the basket, and with LaneHawk,you are assured to get paid for it… “ Source: S. Seitz
- 42. Face recognition Who is she? Source: S. Seitz
- 43. Vision-based biometrics “How the Afghan Girl was Identified by Her Iris Patterns” Read the story Source: S. Seitz
- 44. Login without a password… Fingerprint scanners on many new laptops, other devices Face recognition systems now beginning to appear more widely http://www.sensiblevision.com/ Source: S. Seitz
- 45. Object recognition (in mobile phones) This is becoming real: • Microsoft Research • Point & Find Source: S. Seitz
- 48. The Matrix movies, ESC Entertainment, XYZRGB, NRC Special effects: shape capture Source: S. Seitz
- 49. Pirates of the Carribean, Industrial Light and Magic Special effects: motion capture Source: S. Seitz
- 50. Sports Sportvision first down line Nice explanation on www.howstuffworks.com Source: S. Seitz
- 51. Smart cars Mobileye • Vision systems currently in high-end BMW, GM, Volvo models • By 2010: 70% of car manufacturers. Slide content courtesy of Amnon Shashua Source: S. Seitz
- 52. Vision-based interaction (and games) Nintendo Wii has camera-based IR tracking built in. See Lee’s work at CMU on clever tricks on using it to create a multi-touch display! Source: S. Seitz Assistive technologies Sony EyeToy
- 53. Vision in space Vision systems (JPL) used for several tasks • Panorama stitching • 3D terrain modeling • Obstacle detection, position tracking • For more, read “Computer Vision on Mars” by Matthies et al. NASA'S Mars Exploration Rover Spirit captured this westward view from atop a low plateau where Spirit spent the closing months of 2007. Source: S. Seitz
- 54. Robotics http://www.robocup.org/NASA’s Mars Spirit Rover http://en.wikipedia.org/wiki/Spirit_rover Source: S. Seitz
- 55. The computer vision industry • A list of companies here: http://www.cs.ubc.ca/spider/lowe/vision.html
- 56. Course overview I. Early vision: Image formation and processing II. Mid-level vision: Grouping and fitting III. Multi-view geometry IV. Recognition V. Advanced topics
- 57. I. Early vision Cameras and sensors Light and color Linear filtering Edge detection * = Feature extraction: corner and blob detection • Basic image formation and processing
- 58. II. “Mid-level vision” Fitting: Least squares Hough transform RANSAC Alignment • Fitting and grouping
- 59. III. Multi-view geometry Projective structure from motion Stereo Affine structure from motion Tomasi & Kanade (1993) Epipolar geometry
- 60. IV. Recognition Patch description and matching Clustering and visual vocabularies Bag-of-features models Classification Sources: D. Lowe, L. Fei-Fei
- 61. V. Advanced Topics • Time permitting… Segmentation Articulated models Face detection Motion and tracking
- 62. Course requirements • Philosophy: computer vision is best experienced hands-on • Programming assignments: 50% – Three or four assignments – Expect the first one in the next couple of classes – Brush up on your MATLAB skills (see web page for tutorial) • Final assignment: 30% – Recognition competition – Winner gets a prize! • Participation: 20% – Come to class regularly – Ask questions – Answer questions
- 63. Collaboration policy • Feel free to discuss assignments with each other, but coding must be done individually • Feel free to incorporate code or tips you find on the Web, provided this doesn’t make the assignment trivial and you explicitly acknowledge your sources • Remember: I can Google too!
- 64. For next time • Self-study: MATLAB tutorial • Reading: cameras and image formation (F&P chapter 1)