Lecture08
- 1. Robert Collins CSE486, Penn State Lecture 08: Introduction to Stereo Reading: T&V Section 7.1
- 2. Robert Collins CSE486, Penn State Stereo Vision Inferring depth from images taken at the same time by two or more cameras.
- 3. Robert Collins CSE486, Penn State Basic Perspective Projection Scene Point Perspective Projection Eqns P = (X,Y,Z) X Image Point y x f Y p = (x,y,f) X Y Z x Y Z y y f x X f Z Z O O.Camps, PSU
- 4. Robert Collins CSE486, Penn State Basic Perspective Projection Scene Point Perspective Projection Eqns P = (X,Y,Z) X Image Point y x f Y p = (x,y,f) X Y Z x Y Z y y f x X f Z Z O x X f Z O.Camps, PSU
- 5. Robert Collins CSE486, Penn State Basic Perspective Projection Scene Point Perspective Projection Eqns P = (X,Y,Z) X Image Point y x f Y p = (x,y,f) X Y Z x Y Z y y f x X f Z Z O X Y y x f f Z Z O.Camps, PSU
- 6. Robert Collins CSE486, Penn State Why Stereo Vision? P = (X,Y,Z) X kX y Y x f f Y p = (x,y,f) X Z kZ x y Y kY Z X x y f f f Z kZ Z O Fundamental Ambiguity: Any point on the ray OP has image p O.Camps, PSU
- 7. Robert Collins CSE486, Penn State Why Stereo Vision? P p OL OR A second camera can resolve the ambiguity, enabling measurement of depth via triangulation.
- 8. Robert Collins CSE486, Penn State Why Stereo Vision? ~63mm Your two eyes form a stereo system The right and left eyes see the world from slightly shifted vantage points.
- 9. Robert Collins CSE486, Penn State Key Concepts for Today • Parallax • Anaglyphs • Random Dot Stereograms • Mathematics of Simple Stereo
- 10. Robert Collins CSE486, Penn State Do-it-Yourself Parallax Demo Show: •Points at different depths displace differently •Nearby points displace more than far ones
- 11. Robert Collins A Hitchhiker’s Guide to Parallax CSE486, Penn State Parallax = apparent motion of scene features located at different distances Very distant Very small displacement mountain peak More distant tree Smaller displacement Nearby guardrail Large displacement
- 12. Robert Collins CSE486, Penn State General Idea of Stereo Infer distance to scene points by measuring parallax. INFER Very small displacement Far Midrange Smaller displacement Close Large displacement
- 13. Robert Collins CSE486, Penn State Anaglyphs Anaglyphs are a way of encoding parallax in a single picture. Two slightly different perspectives of the same subject are superimposed on each other in contrasting colors, producing a three-dimensional effect when viewed through two correspondingly colored filters Put red filter over left eye
- 14. Robert Collins CSE486, Penn State
- 15. Robert Collins CSE486, Penn State
- 16. Robert Collins CSE486, Penn State
- 17. Robert Collins CSE486, Penn State
- 18. Robert Collins CSE486, Penn State
- 19. Robert Collins CSE486, Penn State
- 20. Robert Collins CSE486, Penn State
- 21. Robert Collins CSE486, Penn State How Anaglyphs Work Close right eye, then close left. What do you observe? Red filter selectively passes red color, and similarly for cyan filter and cyan color.
- 22. Robert Collins CSE486, Penn State Making an Anaglyph Take a greyscale stereo pair. Copy the left image to the red channel of a new image (the anaglyph image) Copy the right image to the green and blue channels of the anaglyph image (note: green+blue = cyan) Now when you view with red-cyan glasses, the left eye sees only the left image, and the right eye sees only the right image. The brain fuses to form 3D.
- 23. Robert Collins CSE486, Penn State Stereo Pyschophysics How does stereo depth perception work? In particular, at what “level” in the visual system does it occur at? An early debate: do we infer depth from higher-level information like perspective and contours, or does it occur at a much lower level? "The basis of this three-dimensional perception was hotly debated between Wheatstone and fellow physicist Sir David Brewster. (Though it may seem odd for physicists to concern themselves with the physiology of optics, this was felt to be a natural extension of the study of the physics of optics.) Brewster opined that perspective was the source of the apprehension of an object's shape. Wheatstone insisted that the images in the each eye had identifiable landmarks that were combined to assign depth to the landmarks.” -- Ralph M. Siegel Choices: The Science of Bela Julesz
- 24. Robert Collins CSE486, Penn State Higher-level Depth Cues Perspective (vanishing points)
- 25. Robert Collins CSE486, Penn State Higher-level Depth Cues Similar sized objects appear smaller at a distance (this is also related to perspective)
- 26. Robert Collins CSE486, Penn State Higher-level Depth Cues Occluded contours (perceptual completion)
- 27. Robert Collins CSE486, Penn State Stereo Pyschophysics Obviously perspective and contours are important, (particularly for monocular depth perception), but are they necessary for binocular stereo depth perception? Bela Julesz answered this question in 1960 with his experiments with random dot stereograms. “In 1960, Bela's experiment with what eventually became known as Julesz random dot stereograms unambiguously demonstrated that stereoscopic depth could be computed in the absence of any identifiable objects, in the absence of any perspective, in the absence of any cues available to either eye alone.” -- Ralph M. Siegel Choices: The Science of Bela Julesz
- 28. Robert Collins CSE486, Penn State Julesz Random-Dot Experiment Generate a random dot pattern using a computer e.g. im = roicolor(rand(300,300), 0.5, 1); By definition, this is just “noise”, so there are obviously no monocular depth cues here.
- 29. Robert Collins CSE486, Penn State Julesz Random-Dot Experiment Clip out a square region and shift it to the left
- 30. Robert Collins CSE486, Penn State Julesz Random-Dot Experiment Clip out a square region and shift it to the left Fill in the “hole” left behind with more random dots.
- 31. Robert Collins CSE486, Penn State Julesz Random-Dot Experiment Original dot image Dot image with shifted square Now view as a stereo pair. Julesz used a special viewer, but we will display as an anaglyph (get your glasses!)
- 32. Robert Collins CSE486, Penn State
- 33. Robert Collins CSE486, Penn State Make Your Own %make an image with random dots im = roicolor(rand(300,300),.5,1); %second image starts as a copy of that im2 = im; %shift a square of pixels to the right im2(100:200,110:210) = im(100:200,100:200); %fill in the "hole" with more random dots im2(100:200,100:110) = roicolor(rand(101,11),.5,1); %encode image2 in red channel of a color image ana = 255*im2; %encode image1 in blue and green channels ana(:,:,2) = 255*im; ana(:,:,3) = 255*im; %take a look (remember to wear your red/cyan glasses!) image(uint8(ana)) Try this: what happens when you shift the square to the left instead of to the right?
- 34. Robert Collins CSE486, Penn State Stereograms Another method of encoding parallax in a single image. Subtle shifts of repeated texture encode disparity of depths in a scene (a technique made famous under the “Magic Eye” brand name). Unlike anaglyphs, you don’t need special glasses to see these, just some practice focusing your eyes behind the page.
- 35. Robert Collins CSE486, Penn State Stereograms
- 36. Robert Collins Give your eyes a break CSE486, Penn State before we move on... http://www.floa.org/
- 37. Robert Collins CSE486, Penn State A Simple Stereo System Y Z z left y camera located at z (0,0,0) y x right camera Tx located at x (Tx,0,0) X Right camera is simply shifted by Tx units along the X axis. Otherwise, the cameras are identical (same orientation / focal lengths) Camps, PSU
- 38. Robert Collins CSE486, Penn State A Simple Stereo System Top Down View (XZ plane) P=(X,Y,Z) Left Z? Right camera xl xr camera f Tx Translated by a distance Tx along X axis (Tx is also called the stereo “baseline”)
- 39. Robert Collins CSE486, Penn State A Simple Stereo System Y Z (X,Y,Z) z left y camera located at z (0,0,0) y x right camera Tx located at x (Tx,0,0) Image coords of point (X,Y,Z) in Left Camera: X What are image coords of that same point in the Right Camera? Camps, PSU
- 40. Robert Collins CSE486, Penn State A Simple Stereo System (X-Tx, Y, Z) -Tx (X,Y,Z) z left y camera located at z (0,0,0) y x right camera Tx located at x (Tx,0,0) Insight: translating camera to the right by Tx is equivalent to leaving the camera stationary and translating the world to the left by Tx. Camps, PSU
- 41. Robert Collins CSE486, Penn State A Simple Stereo System (X-Tx, Y, Z) -Tx (X,Y,Z) z left y camera located at z (0,0,0) y x right camera Tx located at x (Tx,0,0) Camps, PSU
- 42. Robert Collins CSE486, Penn State Stereo Disparity Left camera Right camera Stereo Disparity baseline depth disparity Important equation!
- 43. Robert Collins CSE486, Penn State Stereo Disparity Left camera Right camera depth Note: Depth and stereo disparity are disparity inversely proportional Important equation!
- 44. Robert Collins CSE486, Penn State Stereo Disparity / Parallax Tie in with Intro: for our purposes Disparity = Parallax Disparity/Parallax inversely proportional to depth this is why near objects appear to move more than far away ones when the camera translates sideways