Computational Imaging with
Multi-Camera Time-of-Flight Systems
Felix Heide*1,2Shikhar Shrestha*1 Wolfgang Heidrich2,3 Gordon Wetzstein1
2University of British Columbia 3KAUST1Stanford University
www.computationalimaging.org
* joint first authors
Time-of-Flight Imaging
Kinect for Xbox One Google Tango Phone
Direct Sampling Transient Imaging
[Velten et al. 2013]
Google Self-Driving Car
Facial capture in VR
[Li et al. 2015]
Correlation Transient Imaging
[Heide et al. 2013]
Multi-Camera ToF System
• Multiple Camera/Sources
• Phased Array
• Control over modulation
• Extensible
• Reproducible
light source
camera
Time-of-Flight ImagingTime-of-Flight Imaging
Image formation
Electron
Field
Photon
Charge
storage
bin 1
Charge
storage
bin 2
Image formation
Image formation
1,2,3,4
Image formation
1,2,3,4
Static frame
Captured with TI OPT8241
Slow-Motion Video
Captured with TI OPT8241
Time-of-Flight ImagingTime-of-Flight Imaging – Dynamic Scene
Doppler shift:
Orthogonal
Frequencies:
Doppler Shift
Image formation – Multicam
Image formation – Frequency and Phase Synchronization
Image formation – Frequency and Phase Synchronization
Multi-Camera ToF System
• Exposure Synchronization
• Individual Sensor/Source
Modulation Signals
• High Frequency Accuracy
• Compatible with Recent
ToF Cameras
• Extensibility
Multi-Camera ToF System – High-Level System Architecture
Multi-Camera ToF System – Key System Components
Direct Digital Synthesis (DDS)
Ch 0
Ch 1
Ch 2
Ch 3
- Programmable Frequency/Phase
- Resolution : 0.12 Hz/0.02°
- Range : 0-200 MHz/0-360°
- Control over SPI
• Custom Board for End-to-End
Signal Conditioning
• Combines a High Speed
Comparator with Exposure
Gating IC
• Interfaces Directly with TI ToF
Camera
Signal Conditioning Circuitry
ToF Daughter BoardAD9959
Real-Time Controller
STMF4 Discovery Board
• Controller responsible for
triggering frames, controlling
waveform generation and
synchronization
• Runs low-level firmware based
on an interrupt driven state
machine architecture
Multi-Camera ToF System – System Workflow
Signal Conditioning Circuitry
(Now on ToF Daughter Board)
DDS Block
MCU Host
Real-Time Controller
Ref. Signal
(Now uses SAW Oscillator
@ 5—MHz)
PC
ToF Cameras
Multi-Camera ToF System – System Workflow
Signal Conditioning Circuitry
(Now on ToF Daughter Board)
DDS Block
Ref. Signal
(Now uses SAW Oscillator
@ 500 MHz)
Configure Modulation Parameters
ToF Cameras
Multi-Camera ToF System – System Workflow
Signal Conditioning Circuitry
(Now on ToF Daughter Board)
DDS Block
Ref. Signal
(Now uses SAW Oscillator
@ 5—MHz)
Configure Camera Registers
Multi-Camera ToF System – System Workflow
Signal Conditioning Circuitry
(Now on ToF Daughter Board)
DDS Block
Ref. Signal
(Now uses SAW Oscillator
@ 5—MHz)
Initiate a Frame
Multi-Camera ToF System – System Workflow
DDS Block
Ref. Signal
(Now uses SAW Oscillator
@ 5—MHz)
Start of Micro-Exposure/Quad
Multi-Camera ToF System – System Workflow
Enable Modulation
Multi-Camera ToF System – System Workflow
Modulation Signals to Sensor/Illum
Multi-Camera ToF System – System Workflow
Raw Capture Data
Multi-Camera ToF System – System Workflow
Multi-Camera ToF System – Demo Unit Design
DC-DC Voltage Regulators
Host MCUDDS Board
ToF Daughter Board
ToF Camera
Demo
Applications
Multi-Camera
Interference
Parallel
Phase Capture
Doppler Velocity
Capture
Motion
Non-Line-Of-Sight
Motion
in Scattering Media
Applications
Multi-Camera
Interference
Parallel
Phase Capture
Doppler Velocity
Capture
Motion
Non-Line-Of-Sight
Motion
in Scattering Media
Phased Camera Depth Imaging
Applications
Multi-Camera
Interference
Parallel
Phase Capture
Doppler Velocity
Capture
Motion
Non-Line-Of-Sight
Motion
in Scattering Media
Simultaneous Doppler Velocity Imaging
Simultaneous Doppler Velocity Imaging
Applications
Multi-Camera
Interference
Parallel
Phase Capture
Doppler Velocity
Capture
Motion
Non-Line-Of-Sight
Motion
in Scattering Media
Non-Line-of-Sight Motion Detection
Non-Line-of-Sight Motion Detection
Non-Line-of-Sight Motion Detection
Applications
Multi-Camera
Interference
Parallel
Phase Capture
Doppler Velocity
Capture
Motion
Non-Line-Of-Sight
Motion
in Scattering Media
Motion in Scattering Media
Motion in Scattering Media
Multi-Camera ToF System
Open-Source Hardware Platform:
www.computationalimaging.org

More Related Content

PPTX
Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport
PPTX
SIGGRAPH 2014 Course on Computational Cameras and Displays (part 4)
PPTX
Optical Computing for Fast Light Transport Analysis
PPTX
Primal-Dual Coding to Probe Light Transport
PPTX
3D Shape and Indirect Appearance by Structured Light Transport
PPTX
SIGGRAPH 2014 Course on Computational Cameras and Displays (part 2)
PPTX
SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)
PPT
Raskar Keynote at Stereoscopic Display Jan 2011
Temporal Frequency Probing for 5D Transient Analysis of Global Light Transport
SIGGRAPH 2014 Course on Computational Cameras and Displays (part 4)
Optical Computing for Fast Light Transport Analysis
Primal-Dual Coding to Probe Light Transport
3D Shape and Indirect Appearance by Structured Light Transport
SIGGRAPH 2014 Course on Computational Cameras and Displays (part 2)
SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)
Raskar Keynote at Stereoscopic Display Jan 2011

What's hot (20)

PPTX
Light field
PPTX
Light Field Photography Introduction
PPTX
SIGGRAPH 2012 Computational Plenoptic Imaging Course - 3 Spectral Imaging
PPT
Introduction to Camera Challenges - Ramesh Raskar
PPTX
Optical volume holograms and their applications
PPTX
SIGGRAPH 2014 Course on Computational Cameras and Displays (part 1)
PPTX
mihara_iccp16_presentation
PPTX
Single photon 3D Imaging with Deep Sensor Fusion
PPTX
Non-line-of-sight Imaging with Partial Occluders and Surface Normals | TOG 2019
PDF
Light Field Technology
PPTX
From STC (Stereo Camera onboard on Bepi Colombo ESA Mission) to Blender
PDF
Nadia2013 research
PDF
Lecture 02 yasutaka furukawa - 3 d reconstruction with priors
PPTX
Wave-Based Non-Line-of-Sight Imaging Using Fast f–k Migration | SIGGRAPH 2019
PDF
Keynote at Tracking Workshop during ISMAR 2014
PPTX
Evaluation of geometrical parameters of buildings from SAR images
PDF
Calibration Issues in FRC: Camera, Projector, Kinematics based Hybrid Approac...
PDF
DimEye Corp Presents Revolutionary VLS (Video Laser Scan) at SS IMMR 2013
PDF
Goddard-DR-2010
PDF
Introduction of slam
Light field
Light Field Photography Introduction
SIGGRAPH 2012 Computational Plenoptic Imaging Course - 3 Spectral Imaging
Introduction to Camera Challenges - Ramesh Raskar
Optical volume holograms and their applications
SIGGRAPH 2014 Course on Computational Cameras and Displays (part 1)
mihara_iccp16_presentation
Single photon 3D Imaging with Deep Sensor Fusion
Non-line-of-sight Imaging with Partial Occluders and Surface Normals | TOG 2019
Light Field Technology
From STC (Stereo Camera onboard on Bepi Colombo ESA Mission) to Blender
Nadia2013 research
Lecture 02 yasutaka furukawa - 3 d reconstruction with priors
Wave-Based Non-Line-of-Sight Imaging Using Fast f–k Migration | SIGGRAPH 2019
Keynote at Tracking Workshop during ISMAR 2014
Evaluation of geometrical parameters of buildings from SAR images
Calibration Issues in FRC: Camera, Projector, Kinematics based Hybrid Approac...
DimEye Corp Presents Revolutionary VLS (Video Laser Scan) at SS IMMR 2013
Goddard-DR-2010
Introduction of slam
Ad

Viewers also liked (6)

PDF
"Time of Flight Sensors: How Do I Choose Them and How Do I Integrate Them?," ...
PDF
Facebook F8
PDF
Melexis Time of Flight Imager for Automotive Applications 2017 teardown rever...
PPTX
Kinect sensor
PDF
"The Evolution of Depth Sensing: From Exotic to Ubiquitous," a Presentation f...
PDF
Tobias Rothermel (pmd technologies): pmd ToF – the Swiss Army Knife of 3D Sen...
"Time of Flight Sensors: How Do I Choose Them and How Do I Integrate Them?," ...
Facebook F8
Melexis Time of Flight Imager for Automotive Applications 2017 teardown rever...
Kinect sensor
"The Evolution of Depth Sensing: From Exotic to Ubiquitous," a Presentation f...
Tobias Rothermel (pmd technologies): pmd ToF – the Swiss Army Knife of 3D Sen...
Ad

Similar to Multi-camera Time-of-Flight Systems | SIGGRAPH 2016 (20)

PDF
Survey of tof and imaging systems
PDF
"What is Neuromorphic Event-based Computer Vision? Sensors, Theory and Applic...
PDF
"How to Choose a 3D Vision Sensor," a Presentation from Capable Robot Components
PPT
Raskar Computational Camera Fall 2009 Lecture 01
PPTX
Unit 3 machine vision
PPTX
Unit 3 machine vision
PDF
PDF
TOFD PRO System
PDF
Wave optics analysis of camera image
PDF
Wave Optics Analysis of Camera Image Formation With Respect to Rectangular Ap...
PDF
03 cameras & their geometry
PDF
Time of Flight and Structured Light Depth Cameras Technology and Applications...
PDF
“Introduction to Depth Sensing,” a Presentation from Meta
PPTX
Digital Image Processing
PPTX
Camera Sensor Technologies – An Overview [2023]
PPT
PPTX
Mitchell Reifel (pmdtechnologies ag): pmd Time-of-Flight – the Swiss Army Kni...
PPTX
Digital radiography-Avinesh Shrestha
PPTX
Coded Photography - Ramesh Raskar
Survey of tof and imaging systems
"What is Neuromorphic Event-based Computer Vision? Sensors, Theory and Applic...
"How to Choose a 3D Vision Sensor," a Presentation from Capable Robot Components
Raskar Computational Camera Fall 2009 Lecture 01
Unit 3 machine vision
Unit 3 machine vision
TOFD PRO System
Wave optics analysis of camera image
Wave Optics Analysis of Camera Image Formation With Respect to Rectangular Ap...
03 cameras & their geometry
Time of Flight and Structured Light Depth Cameras Technology and Applications...
“Introduction to Depth Sensing,” a Presentation from Meta
Digital Image Processing
Camera Sensor Technologies – An Overview [2023]
Mitchell Reifel (pmdtechnologies ag): pmd Time-of-Flight – the Swiss Army Kni...
Digital radiography-Avinesh Shrestha
Coded Photography - Ramesh Raskar

More from StanfordComputationalImaging (17)

PDF
Gaze-Contingent Ocular Parallax Rendering for Virtual Reality
PPTX
Autofocals: Evaluating Gaze-Contingent Eyeglasses for Presbyopes - Siggraph 2019
PPTX
End-to-end Optimization of Cameras and Image Processing - SIGGRAPH 2018
PPTX
Computational Near-eye Displays with Focus Cues - SID 2017 Seminar
PPTX
Accommodation-invariant Computational Near-eye Displays - SIGGRAPH 2017
PPTX
Build Your Own VR Display Course - SIGGRAPH 2017: Part 5
PPTX
Build Your Own VR Display Course - SIGGRAPH 2017: Part 4
PPTX
Build Your Own VR Display Course - SIGGRAPH 2017: Part 3
PPTX
Build Your Own VR Display Course - SIGGRAPH 2017: Part 2
PPTX
Build Your Own VR Display Course - SIGGRAPH 2017: Part 1
PPTX
VR2.0: Making Virtual Reality Better Than Reality?
PPTX
ProxImaL | SIGGRAPH 2016
PPTX
Light Field, Focus-tunable, and Monovision Near-eye Displays | SID 2016
PPTX
Adaptive Spectral Projection
PPTX
The Light Field Stereoscope | SIGGRAPH 2015
PPTX
Compressive Light Field Projection @ SIGGRAPH 2014
PPTX
Vision-correcting Displays @ SIGGRAPH 2014
Gaze-Contingent Ocular Parallax Rendering for Virtual Reality
Autofocals: Evaluating Gaze-Contingent Eyeglasses for Presbyopes - Siggraph 2019
End-to-end Optimization of Cameras and Image Processing - SIGGRAPH 2018
Computational Near-eye Displays with Focus Cues - SID 2017 Seminar
Accommodation-invariant Computational Near-eye Displays - SIGGRAPH 2017
Build Your Own VR Display Course - SIGGRAPH 2017: Part 5
Build Your Own VR Display Course - SIGGRAPH 2017: Part 4
Build Your Own VR Display Course - SIGGRAPH 2017: Part 3
Build Your Own VR Display Course - SIGGRAPH 2017: Part 2
Build Your Own VR Display Course - SIGGRAPH 2017: Part 1
VR2.0: Making Virtual Reality Better Than Reality?
ProxImaL | SIGGRAPH 2016
Light Field, Focus-tunable, and Monovision Near-eye Displays | SID 2016
Adaptive Spectral Projection
The Light Field Stereoscope | SIGGRAPH 2015
Compressive Light Field Projection @ SIGGRAPH 2014
Vision-correcting Displays @ SIGGRAPH 2014

Recently uploaded (20)

PDF
Design Guidelines and solutions for Plastics parts
PPTX
Information Storage and Retrieval Techniques Unit III
PDF
Influence of Green Infrastructure on Residents’ Endorsement of the New Ecolog...
PDF
Soil Improvement Techniques Note - Rabbi
PDF
Abrasive, erosive and cavitation wear.pdf
PPTX
Module 8- Technological and Communication Skills.pptx
PPTX
Petroleum Refining & Petrochemicals.pptx
PDF
First part_B-Image Processing - 1 of 2).pdf
PPTX
Management Information system : MIS-e-Business Systems.pptx
PDF
August 2025 - Top 10 Read Articles in Network Security & Its Applications
PPTX
Amdahl’s law is explained in the above power point presentations
PDF
Unit I -OPERATING SYSTEMS_SRM_KATTANKULATHUR.pptx.pdf
PPT
Chapter 1 - Introduction to Manufacturing Technology_2.ppt
PPTX
Chapter 2 -Technology and Enginerring Materials + Composites.pptx
PDF
Computer System Architecture 3rd Edition-M Morris Mano.pdf
PDF
null (2) bgfbg bfgb bfgb fbfg bfbgf b.pdf
PPTX
CN_Unite_1 AI&DS ENGGERING SPPU PUNE UNIVERSITY
PPTX
ai_satellite_crop_management_20250815030350.pptx
PDF
Accra-Kumasi Expressway - Prefeasibility Report Volume 1 of 7.11.2018.pdf
PDF
MLpara ingenieira CIVIL, meca Y AMBIENTAL
Design Guidelines and solutions for Plastics parts
Information Storage and Retrieval Techniques Unit III
Influence of Green Infrastructure on Residents’ Endorsement of the New Ecolog...
Soil Improvement Techniques Note - Rabbi
Abrasive, erosive and cavitation wear.pdf
Module 8- Technological and Communication Skills.pptx
Petroleum Refining & Petrochemicals.pptx
First part_B-Image Processing - 1 of 2).pdf
Management Information system : MIS-e-Business Systems.pptx
August 2025 - Top 10 Read Articles in Network Security & Its Applications
Amdahl’s law is explained in the above power point presentations
Unit I -OPERATING SYSTEMS_SRM_KATTANKULATHUR.pptx.pdf
Chapter 1 - Introduction to Manufacturing Technology_2.ppt
Chapter 2 -Technology and Enginerring Materials + Composites.pptx
Computer System Architecture 3rd Edition-M Morris Mano.pdf
null (2) bgfbg bfgb bfgb fbfg bfbgf b.pdf
CN_Unite_1 AI&DS ENGGERING SPPU PUNE UNIVERSITY
ai_satellite_crop_management_20250815030350.pptx
Accra-Kumasi Expressway - Prefeasibility Report Volume 1 of 7.11.2018.pdf
MLpara ingenieira CIVIL, meca Y AMBIENTAL

Multi-camera Time-of-Flight Systems | SIGGRAPH 2016