426 lecture3: AR Tracking

COSC 426: Augmented Reality

Mark Billinghurst
mark.billinghurst@hitlabnz.org

July 25th 2012

Lecture 3: AR Tracking

Tracking Requirements

Head Stabilized Body Stabilized World Stabilized
  Augmented Reality Information Display
  World Stabilized
Increasing Tracking
  Body Stabilized
Requirements
  Head Stabilized

Tracking Technologies
•  Mechanical
•  Electromagnetic
•  Optical
•  Acoustic
•  Inertial and dead reckoning
•  GPS
•  Hybrid

AR Tracking Taxonomy
AR
TRACKING

Indoor Outdoor
Environment Environment

Limited Range Extended Range Low Accuracy & High Accuracy
Not Robust & Robust

Low Accuracy High Accuracy Many Fiducials Not Hybridized Hybrid Tracking
at 15-60 Hz & High Speed in space/time GPS or GPS and
Hybrid but Camera or Camera and
Tracking no GPS Compass Compass

e.g. AR Toolkit e.g. IVRD e.g. HiBall e.g. WLVA e.g. BARS

Tracking Types

Magnetic Inertial Ultrasonic Optical Mechanical
Tracker Tracker Tracker Tracker Tracker

Specialized Marker-Based Markerless
Tracking Tracking Tracking

Edge-Based Template-Based Interest Point

Tracking Systems
  Mechanical Tracker
  Magnetic Tracker
  Ultrasonic Tracker
  Inertial Tracker
  Computer Vision (Optical Tracking)
  Specialized (Infrared, Retro-Reflective)
  Monocular (DVCam, Webcam)

Mechanical Tracker
  Idea: mechanical arms with joint sensors

Microscribe

  ++: high accuracy, haptic feedback
  -- : cumbersome, expensive

Magnetic Tracker
  Idea: difference between a magnetic transmitter
and a receiver

Flock of Birds (Ascension)

  ++: 6DOF, robust
  -- : wired, sensible to metal, noisy, expensive

Ultrasonics Tracker
  Idea: Time of Flight or Phase-Coherence Sound Waves

Ultrasonic
Logitech IS600

  ++: Small, Cheap
  -- : 3DOF, Line of Sight, Low resolution, Affected
Environment Conditon (pressure, temperature)

Inertial Tracker
  Idea: measuring linear and angular orientation rates
(accelerometer/gyroscope)

IS300 (Intersense)
Wii Remote

  ++: no transmitter, cheap, small, high frequency, wireless
  -- : drift, hysteris only 3DOF

Mobile Sensors
  Inertial compass
  Earth’s magnetic field
  Measures absolute orientation
  Accelerometers
  Measures acceleration about axis
  Used for tilt, relative rotation
  Can drift over time

Global Positioning System (GPS)
  Created by US in 1978
  Currently 29 satellites
  Satellites send position + time
  GPS Receiver positioning
  4 satellites need to be visible
  Differential time of arrival
  Triangulation
  Accuracy
  5-30m+, blocked by weather, buildings etc

Problems with GPS
  Takes time to get satellite fix
  Satellites moving around
  Earths atmosphere affects signal
  Assumes consistent speed (the speed of light).
  Delay depends where you are on Earth
  Weather effects
  Signal reflection
  Multi-path reflection off buildings
  Signal blocking
  Trees, buildings, mountains
  Satellites send out bad data
  Misreport their own position

Accurate to < 5cm close to base station (22m/100 km)
Expensive - $20-40,000 USD

Assisted-GPS (A-GPS)
  Use external location server to send GPS signal
  GPS receivers on cell towers, etc
  Sends precise satellite position (Ephemeris)
  Speeds up GPS Tracking
  Makes it faster to search for satellites
  Provides navigation data (don’t decode on phone)
  Other benefits
  Provides support for indoor positioning
  Can use cheaper GPS hardware
  Uses less battery power on device

Cell Tower Triangulation
  Calculate phone position
from signal strength
  < 50 m in cities
  > 1 km in rural

WiFi Positioning
  Estimate location by using WiFi access points
  Can use know locations of WiFi access points
  Triangulate through signal strength
  Eg. PlaceEngine (www.placeengine.com)
  Client software for PC and mobiles
  SDK returns position
  Accuracy
  5 – 100m (depends on WiFi density)

Indoor WiFi Location Sensing
  Indoor Location
  Asset, people tracking
  Aeroscout
  http://aeroscout.com/
  WiFi + RFID
  Ekahau
  http://www.ekahau.com/
  WiFi + LED tracking

Integrated Systems
  Combine GPS, Cell tower, WiFi signals
  Skyhook (www.skyhookwireless.com)
  Core Engine
  Database of known locations
  700 million Wi-Fi access points and cellular towers.

Comparative Accuracies
  Study testing iPhone 3GS cf. low cost GPS
  A-GPS
  8 m error
  WiFi
  74 m error
  Cell Tower Positioning
  600 m error

Accuracy of iPhone Locations: A Comparison of Assisted GPS, WiFi, and
Cellular Positioning
In GIScience on July 15, 2009 at 8:11 pm By Paul A Zandbergen
Transactions in GIS, Volume 13 Issue s1, Pages 5 - 25

Optical Tracker
  Idea: Image Processing and Computer Vision
  Specialized
  Infrared, Retro-Reflective, Stereoscopic

ART Hi-Ball

  Monocular Based Vision Tracking

Outside-In vs. Inside-Out Tracking

Optical Tracking Technologies

  Scalable active trackers
  InterSense IS-900, 3rd Tech HiBall
3rd Tech, Inc.
  Passive optical computer vision
  Line of sight, may require landmarks
  Can be brittle.
  Computer vision is computationally-intensive

HiBall Tracking System (3rd Tech)
  Inside-Out Tracker
  $50K USD
  Scalable over large area
  Fast update (2000Hz)
  Latency Less than 1 ms.
  Accurate
  Position 0.4mm RMS
  Orientation 0.02° RMS

Starting simple: Marker tracking
  Has been done for more than 10 years
  Several open source solutions exist
  Fairly simple to implement
  Standard computer vision methods
  A rectangular marker provides 4 corner points
  Enough for pose estimation!

Marker Based Tracking: ARToolKit

http://artoolkit.sourceforge.net/

Tracking Range with Pattern Size

Rule of thumb – range = 10 x pattern width

Tracking challenges in ARToolKit

Occlusion Unfocused camera, Dark/unevenly lit Jittering
(image by M. Fiala) motion blur scene, vignetting (Photoshop illustration)

Image noise
False positives and inter-marker confusion (e.g. poor lens, block coding /
compression, neon tube)
(image by M. Fiala)

Limitations of ARToolKit
  Partial occlusions cause tracking failure
  Affected by lighting and shadows
  Tracking range depends on marker size
  Performance depends on number of markers
  cf artTag, ARToolKitPlus
  Pose accuracy depends on distance to marker
  Pose accuracy depends on angle to marker

Other Marker Tracking Libraries
  arTag
  http://www.artag.net/
  ARToolKitPlus [Discontinued]
  http://studierstube.icg.tu-graz.ac.at/handheld_ar/
artoolkitplus.php
  stbTracker
  http://studierstube.icg.tu-graz.ac.at/handheld_ar/
stbtracker.php
  MXRToolKit
  http://sourceforge.net/projects/mxrtoolkit/

Markerless Tracking
  No more Markers! Markerless Tracking

Magnetic Tracker Inertial Ultrasonic Optical
Tracker Tracker Tracker

Specialized Marker-Based Markerless

Edge-Based Template-Based Interest Point

Natural feature tracking
  Tracking from features of the surrounding
environment
  Corners, edges, blobs, ...
  Generally more difficult than marker tracking
  Markers are designed for their purpose
  The natural environment is not…
  Less well-established methods
  Usually much slower than marker tracking

Natural Feature Tracking
Features Points
  Use Natural Cues of Real Elements
Contours
  Edges
  Surface Texture
  Interest Points
  Model or Model-Free
  ++: no visual pollution

Surfaces

Edge Based Tracking
  RAPiD [Drummond et al. 02]
  Initialization, Control Points, Pose Prediction (Global Method)

Line Based Tracking
  Visual Servoing [Comport et al. 2004]

Model Based Tracking
  Track from 3D model
  Eg OpenTL - www.opentl.org
  General purpose library for model based visual tracking

Marker vs. natural feature tracking
  Marker tracking
  + Can require no image database to be stored
  + Markers can be an eye-catcher
  + Tracking is less demanding
  - The environment must be instrumented with markers
  - Markers usually work only when fully in view
  Natural feature tracking
  - A database of keypoints must be stored/downloaded
  + Natural feature targets might catch the attention less
  + Natural feature targets are potentially everywhere
  + Natural feature targets work also if partially in view

Hybrid Tracking
Combining several tracking modalities together

Sensor tracking
  Used by many “AR browsers”
  GPS, Compass, Accelerometer, (Gyroscope)
  Not sufficient alone (drift, interference)

Outdoor Hybrid Tracking
  Combines
  computer vision
-  natural feature tracking
  inertial gyroscope sensors
  Both correct for each other
  Inertial gyro - provides frame to frame
prediction of camera orientation
  Computer vision - correct for gyro drift

Combining Sensors and Vision
  Sensors
-  Produce noisy output (= jittering augmentations)
-  Are not sufficiently accurate (= wrongly placed augmentations)
-  Gives us first information on where we are in the world,
and what we are looking at
  Vision
-  Is more accurate (= stable and correct augmentations)
-  Requires choosing the correct keypoint database to track from
-  Requires registering our local coordinate frame (online-
generated model) to the global one (world)

Outdoor AR Tracking System

You, Neumann, Azuma outdoor AR system (1999)

Robust Outdoor Tracking

  Hybrid Tracking
  Computer Vision, GPS, inertial
  Going Out
  Reitmayer & Drummond (Univ. Cambridge)

Wrap-up
  Tracking and Registration are key problems
  Registration error
  Measures against static error
  Measures against dynamic error
  AR typically requires multiple tracking technologies
  Research Areas: Hybrid Markerless Techniques,
Deformable Surface, Mobile, Outdoors

More Information
•  Mark Billinghurst

–  mark.billinghurst@hitlabnz.org

•  Websites

–  www.hitlabnz.org

426 lecture3: AR Tracking

More Related Content

What's hot (20)

Similar to 426 lecture3: AR Tracking (20)

More from Mark Billinghurst (20)

Recently uploaded (20)

426 lecture3: AR Tracking