Presentatie Willem En Kevin

Willem Mestdagh - Kevin Cordemans

Papers MS Research PlayAnywhere: Compact Interactive Tabletop Projection-Vision System [] Andy Wilson Thinsight: Integrated Optical Multi-touch Sensing through Thin Form-factor Displays [] Bill Buxton, Steve Hodges, ...

PlayAnywhere PlayAnywhere: A Compact Interactive Tabletop Projection-Vision System Andy Wilson at Microsoft Research, Redmond 2005

Introduction Looks great... But... Calibration? Installation? Portability? Cost?

Mainstream Consumer Acceptance Multi-touch: new sensing and display technologies Ubiquitious computing  Portability Ease of installation Ease of use: any surface? Calibration Cost  Devise an easy-to-use, low-cost multi-touch system...

PlayAnywhere Front-projected computer vision-based interactive table system which uses a new projecting technology to obtain an exceptionnaly self-contained form factor Requirements: Quick set up: no calibration beyond factory No “installation” Usable on any flat surface Compact but still displaying and sensing over a large surface (form factor) General purpose (Low cost)

Future? Canesta projection keyboard

PlayAnywhere What is it? Known problems in existing systems How does it work? PlayAnywhere configuration Projector Camera and IR Illuminant Image Processing Image Rectification Touch and Hover Analysis of shadow PlayAnywhere Visual Code Flow Move Computer Vision Pros and Cons

Known problems in existing systems

PlayAnywhere Configuration All-in-One IR illuminant Projector (with mirrors) Camera with IR-passing-filter Prototype

Projector NEC WT600 DLP 40’’ diagonale afbeelding Voordelen van de plaats van de projector Gevaarlijke plaatsing projector is vermeden Wanneer camera en projector samen zijn geplaatst  geen nood om de camera te hercalibreren bij verplaatsing van het toestel Nadelen Beeldkwaliteit kan niet gegarandeerd worden, want projectie-oppervlak is niet gekend

Camera en IR Illuminant De projector belicht de scene met zichtbaar straling De illuminant belicht de scene met infrarode straling Camera ontvangt alleen gereflecteerde infrarode stralen dankij de infrared-passing-filter die aan de camera bevestigd is (Camera blind voor reflecties van de projectorstraling)

How does it work? Image Processing Preparation: Image Rectification Computer Vision Techniques: Analysis of Shadow Hover and Touch Flow Move PlayAnywhere Visual Code

Image Rectification (1)  Remove distortion via standard bilinear interpolation techniques Parameters for transformations need only be determined once (factory): Lens specs known Fixed topology device-plane (assumption) Setup Good thing Bad thing Wide lens Maximize usable surface Creates barrel distortion Oblique position of camera Optimizes form factor (compact device) Produces projective distortion

Image Rectification (3) Input image and projected image are in a one-to-one correspondence, meaning: A rectangular object on the surface appears as a rectangular object in the input image at the same (scaled) coordinates. Rectified image suited for computer vision techniques Limitation: objects further away from the unit will appear at a lower resolution (Minimum effective resolution is less than that of the acquired image)

Hover and Touch Detect touch without relying on special instrumentation of the surface, so that the device may operate on any available flat surface (goal) PlayAnywhere: exploite change in appearance of shadows as an object approaches the surface No tracking of fingers, ...

Hover and Touch thresholding operation Hover Candidate finger positions: highest on each of the distinct shadows which enter the scene from the bottom of the image Touch Analysis of shadow shapes: threshold width of finger shadow at a location slightly below the topmost point

Hover and Touch To limit false positives, candidate finger must lie on a shadow that extends to the bottom of the image  Objects on table can corrupt touch detection by mimicing arm shadows, meaning their shadows extends to the bottom of the image Presently, only one finger per hand (more fingers requires more spohisticated analysis) Precision limited by image resolution

Flow Move Support direct manipulation of virtual objects (doc’s, photo’s) Gestures that are natural to a specific activity E.g. resizing and rotating photo’s as if they were lying on a table PlayAnywhere: Optical flow techniques Based on local texture on the movig object No absolute position-based tracker, no widgets Instead simple statistics summarizing the flow field, computing only relative motion information

Optical Flow (1) Optical Flow computed from the most recently acquired rectified image I(t) and the previous image I(t-1) Mask images with grid Image (t-1) Image (t)

Optical Flow (2) For each (x,y) on a regular grid, the integer vector quantity (dx, dy) is determined, such that: image patch P centered on (x, y) at time t-1 most closely matches image patch P’ centered on (x+dx, y+dy) at time t. Image patches are compared by computing pixelwise absolute differences (low values  close match). If image patch P closely matches image patch P’, there is a good chance that the two patches are representing the same content, shifted to a different gridpoint in the interval. (dx, dy) gives the direction of this shift. Image(t) (x+dx, y+dy) Image patch P’ Image(t-1) (x, y) Image patch P Resulting vector

Optical Flow (3) For a given patch in the image, select (dx, dy) that minimizes Vector field can be used as an argument in a function to infer meaning from user’s gesture Resulting vector field

PlayAnywhere Visual Codes Ook mogelijk te werken met infrarood systemen in objecten  mogelijkheid tot werken met barcodes (derde vb met barcode)

Resultaat http://youtube.com/watch?v=YRfWhUnYyY8 Nu : Thingsight

Beschrijving “ Optical sensing system, fully integrated into a thin form factor display” Scherm met multi-touch systeem 2 mogelijkheden om multitouchsystemen uit te werken : - een arrangement van electroden op het oppervlak - werken met cameras : voor of achter het scherm

Vergelijking Beide Touch-algoritme op basis van schaduwinval van handen Beide Handig te gebruiken en te verplaatsen Principe werking ongeveer gelijk  verschil : thinsight werkt met elektroden op het scherm, playanywhere werkt met camera’s boven het scherm Beiden hebben de mogelijkheid tot werken met externe objecten met een infrarode bron Voordeel bij thinsight : geen oppervlakte voor apparaat nodig

Presentatie Willem En Kevin

More Related Content

What's hot (20)

Viewers also liked (7)

Similar to Presentatie Willem En Kevin (20)

Recently uploaded (20)

Presentatie Willem En Kevin