SlideShare a Scribd company logo

Next Generation Intelligent Camera technologies - Composec 2014 Keynote

Download as PPTX, PDF
Jacob Jose, profile picture
Jacob Jose

This document summarizes the current and future intelligent camera technologies from Texas Instruments (TI). It outlines TI's 30+ year history in video and imaging innovation, their current leadership in intelligent video technologies like their DMVAx processors. It provides details on TI's DMVA3 and DM8127 solutions that enable customizable video analytics at the edge for mainstream cameras. Finally, it previews TI's next generation DaVinci video processors that will enable megapixel smart analytics.

1 of 29
Downloaded 77 times
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
Next Generation Intelligent Camera technologies Jacob Jose Product Manager Texas Instruments
Agenda • TI’s investment in Video and Vision markets • Current technologies for intelligent camera markets • What’s coming next?
TI Confidential - NDA Restrictions 3 TI has over 30 years of Video & Imaging innovation
TI leads in intelligent video technologies Military Public Innovators Early adopters Early majority DM642 Surveillance Processor with DSP for analytics 2001 Lower Power Higher Performance Decreasing Form Factor• Lower system cost • Affordable software options • Solve specific market needs • Smaller form factors • Reduced power Thermal Efficiency Support Evolving Standard• Cost of ownership reduction • Standardization begins • Productivity maximized • Easy to implement • Standardization begins TI’s DMVAx-based products in market NOW Residential Do It Yourself TI DMVAx based products in market NOW Critical Facilities Gaming Retail Banking DM6467 Surveillance Processor with 1GHz C64x DSP for analytics 2005 VLIB, a library of 50+ optimized kernels available for C64x, C64x+ DSPs 2007 TI announces first video SoC (DMVA1) with Vision Co- Processor 2009 TI adds HD compression & Tamper Detect, Motion Detection, People Counting on DMVA2 2010 First customers in production using DMVA1 & DMVA2 2011 4 TI opens Vision Co- Processor for programming using the Vision Co- Processor Library 2013 Various Analytics technology development 1990 -
TI’s DMVA3: A high performance programmable vision coprocessor • 720MHz ARM® Cortex™-A8 for control heuristics • Up to 2 megapixel real-time H.264/SVC-T • Capable of compressing videos at 20 megapixels or higher resolution while running video analytics algorithms • Customers can port their own algorithms onto the DMVA3 5 Loitering Detection Left Object Detection Object Classification Animal
TI’s DM8127: Leading solution for Traffic Cameras • Up to 750 MHz of C674x floating point DSP • 1GHz ARM Cortex-A8 for control heuristics • 5 Megapixel Fish-eye correction + perspective correction (bird’s eye view • TI’s stereo module library to sense depth and increase accuracy by 5x- 10x • Reference design available 6 Left Image Right Image Leopard Imaging’s LNC-DUALCAM-3D based on TMS320DM8127 Depth Map
DMVA3 and DM8127 enables customizable analytics at the edge for mainstream cameras 7 TI’s DaVinci video processors enable megapixel smart analytics IP People counting Trip zone Streaming metadata Tamper detection Motion detection
8 Intelligent Motion Detection (IMD) Configuration Example
Intelligent Motion Detection Add ROI Delete ROI Clear All Save Cancel Advanced Enable ROI Label Region of Interest Settings IMD Global Settings Settings Info Enable IMD Display Object Bounding Box Display Enabled Zones Sensitivity Moderate Max. Object Size Min. Object Size 100 8000  Opening page of IMD Control Panel GUI Behavior For illustrative purposes, the yellow arrow represents an end- user’s mouse. This is the opening page for IMD’s GUI, which is called the IMD Control Panel. It contains a live preview of the camera’s field of view. ‘IMD Global Settings’, which establish parameters that apply for all regions of interest (ROIs), are set in the sub-window below preview. ROIs are managed by the sub-window to the right of the preview window. The lower-right sub-window displays feedback messages to the user when mouseover events occur or to acknowledge user input. When this page is viewed for the first time, we assume that no ROI has been defined by the user. Note that default settings are installed and buttons to delete or clear ROIs have been disabled. Whenever the ‘Defaults’ button is clicked, all settings are restored to those shown here. Intelligent motion detection is disabled. Click ‘Enable IMD’ to turn this feature on.   Defaults
Intelligent Motion Detection Add ROI Delete ROI Clear All Save Cancel Advanced Enable ROI Label Region of Interest Settings IMD Global Settings Settings Info Enable IMD Display Object Bounding Box Display Enabled Zones Sensitivity Moderate Max. Object Size Min. Object Size 100 8000  Enabling IMD for the 1st time GUI Behavior When a mouseover event occurs over the ‘Enable IMD’ checkbox, its present status is acknowledged in the Settings Info sub-window. If the checkbox is clear and the user left-clicks it, report ‘Intelligent motion detection is enabled.’ in Settings Info. Likewise, if the checkbox has been checked and the user left-clicks it, report ‘Intelligent motion detection is disabled.’ If the mouse returns to the checkbox (after loosing focus), report ‘Click this button to turn intelligent motion detection off.’ in Settings Info. Click this button to turn intelligent motion detection on.   Defaults
Intelligent Motion Detection Add ROI Delete ROI Clear All Save Cancel Advanced Enable ROI Label Region of Interest Settings IMD Global Settings Settings Info Enable IMD Display Object Bounding Box Display Enabled Zones Sensitivity Moderate Max. Object Size Min. Object Size 3 67  Changing default settings GUI Behavior The ‘Sensitivity’ dropdown list can be set to the following states: • Low • Low-to-Moderate • Moderate • Moderate-to-High • High When a mouseover event occurs over the ‘Sensitivity’ dropdown list, instructive information is shared in the Settings Info sub-window: ‘Try Low sensitivity to improve performance in outdoor or challenging environments. High- sensitivity works best for fast moving objects in stable lighting environments.’ The Min. Object Size slider changes the radius of a green circle rendered on the screen. Likewise, Max. Object Size will affect the radius of a red circle. These circles are drawn anytime a mouseover event occurs on the slider or adjacent input window. The user is free to move the circles to any location inside of the preview window. Move the slider or type in a value to set the minimum object size. Objects that are smaller than this value will be ignored by Intelligent motion detection.   Defaults Min Obj Size Max Obj Size
Intelligent Motion Detection Add ROI Delete ROI Clear All Save Cancel Advanced Enable ROI Label Region of Interest Settings IMD Global Settings Settings Info Enable IMD Display Object Bounding Box Display Enabled Zones Sensitivity Moderate Max. Object Size Min. Object Size 100 8000  Enabling with no user-defined ROIs GUI Behavior If the user clicks ‘Save’ when no user-defined regions of interest have been selected, a DMVA Settings Manager dialog pops up and displays a message. This dialog has the focus of attention over any other control owned by the browser window. The dialog has redundant controls (same as those presented within the IMD Control Panel). If the user clicks ‘Save’, a new ROI is automatically defined by a 4-point polygon that shares the same coordinates as the entire frame. This ROI is enabled and labeled as ‘Field_of_View’. The dialog is closed and the page is redirected to DMVA home. If the user clicks ‘Cancel’, the dialog is closed and the focus returns to the IMD Control Panel. Intelligent motion detection is enabled.    Defaults Since no regions of interest have been explicitly defined, an alert event will be signaled when any object motion is detected within the entire field of view. Press Save to accept or Cancel to return to the IMD Control Panel. DMVA Settings Manager Save Cancel
Intelligent Motion Detection Add ROI Delete ROI Clear All Save Cancel Advanced Enable ROI Label ROI_0 Region of Interest Settings IMD Global Settings Settings Info Enable IMD Display Object Bounding Box Display Enabled Zones Sensitivity Moderate Max. Object Size Min. Object Size 100 8000  Adding a region of interest (1) GUI Behavior If a mouseover event occurs over the ‘Add ROI’ button, an informative message is presented in ‘Settings Info’. If the user clicks this button, the top row in the scrolling list is highlighted in a brighter color. In this row, an enabled checkbox appears in the 1st column and a default ROI label, e.g. ROI_0, appears in the 2nd column. Click on this button to define a polygon to represent a new region of interest..    Defaults 
Intelligent Motion Detection Add ROI Delete ROI Clear All Save Cancel Advanced Enable ROI Label ROI_0 Region of Interest Settings IMD Global Settings Settings Info Enable IMD Display Object Bounding Box Display Enabled Zones Sensitivity Moderate Max. Object Size Min. Object Size 100 8000  Adding a region of interest (2) GUI Behavior After the ‘Add ROI’ button is clicked, the mouse pointer changes from arrow to crosshairs when the mouse moves inside preview window. An instructive message is presented in ‘Settings Info’ that explains how to define the ROI. Clicks outside the preview window force a warning message inside Settings Info, e.g. Invalid point selected! Please pick a point inside the preview window. If 16 points have been defined while setting the ROI, the next single click will complete the polygon based on those points. Define the region of interest. Each left mouse button click within the live preview window defines a point in the polygon. Up to 16 points can be used. Double-clicking the left mouse button will automatically complete the polygon.    Defaults  
Intelligent Motion Detection Add ROI Delete ROI Clear All Save Cancel Advanced Enable ROI Label ROI_1 ROI_0 Region of Interest Settings IMD Global Settings Settings Info Enable IMD Display Object Bounding Box Display Enabled Zones Sensitivity Moderate Max. Object Size Min. Object Size 100 8000  Adding a region of interest (3) GUI Behavior After the ROI has been established, the area inside the lines is shaded and the label displayed. Buttons ‘Delete ROI’ and ‘Clear All’ become enabled and the top row of the ROI list loses the focus (no highlighting). To add more ROIs, the user must click ‘Add ROI’ for each region. Once activated, the user must click the left mouse button inside the preview over a location not already defined by an existing ROI to create a new ROI. The elements in the ROI list will grow accordingly. Define the region of interest. Each left mouse button click within the live preview window defines a point in the polygon. Up to 16 points can be used. Double-clicking the left mouse button will automatically complete the polygon.    Defaults  ROI_0  
Intelligent Motion Detection Add ROI Delete ROI Clear All Save Cancel Advanced Enable ROI Label DESK ROI_0 Region of Interest Settings IMD Global Settings Settings Info Enable IMD Display Object Bounding Box Display Enabled Zones Sensitivity Moderate Max. Object Size Min. Object Size 100 8000  Editing a region of interest GUI Behavior The user can change the label name at any point by click on the label in the ROI list. Any name under 64 characters is exceptable as long as the name is unique, case-insensitive, begins with a letter, etc. (same restrictions as filenames). When editing, the cursor is displayed and the entire row is highlighted. If the another ROI was being defined when the user tries to edit a label name, store the settings to a valid state, e.g. close the polygon as if a double-click were registered. Re-label the name of the region of interest. Limit names to 64 characters and avoid leading numbers and the following characters: ` , ~ ! @ # $ % ^ & * ( ) / ? ‘ { } |    Defaults  ROI_0  DESK
Intelligent Motion Detection Add ROI Delete ROI Clear All Save Cancel Advanced Enable ROI Label DOOR DESK MAIN_FLOOR Region of Interest Settings IMD Global Settings Settings Info Enable IMD Display Object Bounding Box Display Enabled Zones Sensitivity Moderate Max. Object Size Min. Object Size 100 8000  Deleting a region of interest GUI Behavior If a mouseover event happens over the ‘Delete ROI’ button, Settings Info reports ‘Click to delete a highlighted ROI.’ Pressing this button without first selected a ROI label name to do nothing; Settings Info will report ‘You must select a ROI label name first to delete.’ Double-clicking a label name gives it the focus of attention, e.g. highlights it. If ‘Delete ROI’ is clicked after the row is highlighted, the corresponding overlay is also brightened. Clicking this button a second time will delete the element in the list as well as the overlay region in the preview window. If the focus of attention is diverted to the keyboard or another location on the page not over the button, the delete operation is canceled. No overlap tolerated. Max. ROI = 16 Click again to delete a highlighted region of interest.    Defaults  ROI_0  DESK  DOOR
18 DMVA Event Monitor Excerpts from DMVA Graphical User Interface (GUI) Specification
DMVA Event Monitor Timestamp Event Type Event Details Tu Oct 7 09 13:29:39.1 IMD Zone = LOBBY Duration = 00:1:39.2 Tu Oct 7 09 13:34:14.3 IMD Zone = LOBBY Duration = 00:0:16.3 Tu Oct 7 09 13:38:24.5 IMD Zone = LOBBY Duration = 00:0:59.5 Tu Oct 7 09 13:45:19.0 COUNT Num = 2 Direction = LATERAL Tu Oct 7 09 13:54:35.8 TRIP ZONE Direction = Zone 12 Tu Oct 7 09 13:55:19.3 IMD Zone = DESK Duration = 00:19:49.2 Event List DMVA Message Window Timestamp Metadata Overlay DMVA Live Event Monitor GUI Behavior The DMVA Event Monitor page is the main DMVA page, e.g. homepage. It features a video window, a scrolling list of event summaries, function indicators/selectors, controls for display and event recording, and a message window. The video window shows both live (default) and archived video, which can be marked up by graphical overlays. The ‘Display’ subwindow allows the user to select the desired overlay. A row of indicators/selectors sits below. Indicator “lamp” for an active analytic function glows “green”; Tamper, which runs at all times, can also glow “red” during active tamper events. Lamps also used to 1 of the 4 analytic functions that can run simultaneously with tamper. After function is selected, user can click ‘Configure’ to bring up the dedicated configuration page. During each event, the compressed audio and/or video, metadata, and detailed event log file can be saved. Tamper alerts are displayed in the DMVA Message window. Overall DMVA health and status information is shown here as well. Tamper alert: Camera movement detected Tues Oct 7 2009 13:56:09.2! LOBBY DESK Tamper     Analytics Display Object Bounding Box Enabled Zones Event Recording Video Stream Audio Stream Metadata Stream Event Log CancelConfigure Advanced IMD CountingTripZone SMOS
DMVA Event Monitor Timestamp Event Type Event Details Tu Oct 7 09 13:29:39.1 IMD Zone = LOBBY Duration = 00:1:39.2 Tu Oct 7 09 13:34:14.3 IMD Zone = LOBBY Duration = 00:0:16.3 Tu Oct 7 09 13:38:24.5 IMD Zone = LOBBY Duration = 00:0:59.5 Tu Oct 7 09 13:45:19.0 COUNT Num = 2 Direction = LATERAL Tu Oct 7 09 13:54:35.8 TRIP ZONE Direction = Zone 12 Tu Oct 7 09 13:55:19.3 IMD Zone = DESK Duration = 00:19:49.2 Event List DMVA Message Window Timestamp Metadata Overlay DMVA Live Event Monitor Cllick ‘Advanced’ to fine-tune camera tamper detection parameters. LOBBY DESK Tamper     Analytics Display Object Bounding Box Enabled Zones Event Recording Video Stream Audio Stream Metadata Stream Event Log CancelConfigure Advanced IMD CountingTripZone SMOS GUI Behavior If the Advanced button is pushed, a dialog box appears. For now, it will show camera tamper options. Press Save to accept or Cancel to return to the DMVA Event Monitor. Save Cancel Advanced DMVA Settings Detect lights on-to-off events Detect lights off-to-on events Camera Tamper Detection Sensitivity Moderate  Reset Sec. 30
DMVA Event Monitor Timestamp Event Type Event Details Tu Oct 7 09 13:29:39.1 IMD Zone = LOBBY Duration = 00:1:39.2 Tu Oct 7 09 13:34:14.3 IMD Zone = LOBBY Duration = 00:0:16.3 Tu Oct 7 09 13:38:24.5 IMD Zone = LOBBY Duration = 00:0:59.5 Tu Oct 7 09 13:45:19.0 COUNT Num = 2 Direction = LATERAL Tu Oct 7 09 13:54:35.8 TRIP ZONE Direction = Zone 12 Tu Oct 7 09 13:55:19.3 IMD Zone = DESK Duration = 00:19:49.2 Event List DMVA Live Event Monitor GUI Behavior DMVA generates an output response for every frame of video. For each event, an summary record is created that establishes the duration of the event and associates the timestamp, type, and relevant zone information (context dependent based on the enabled analytic function). When a user selects an event from the list and ‘Video Stream’ has also been selected (at time of event as well as while user is interacting) under ‘Event Recording’, the compressed video of the event is retrieved and replayed in the window. If ‘Metadata Stream’ has also been select, bounding box information is available for display (depends on whether ‘Metadata Overlay’ has been checked or not). Video and metadata recorded for this event. LOBBY DESK     Tamper Analytics IMD CountingTripZone SMOS CancelConfigure Advanced Timestamp Metadata Overlay Object Bounding Box Enabled Zones Video Stream Audio Stream Metadata Stream Event Log Display Event Recording       DMVA Message Window
Image Signal Processing (ISP) video processing smart plug-ins 22 Video stabilization: Camera shakes due to weather, traffic → degrades video quality TI’s advanced video stabilization reduces jitter, improves visual quality Face detection tracking, recognition: For identification, access control, privacy mask TI’s face detection HW tracks multiple faces (>30) with detection in <20 milliseconds TI’s Video Noise Filters preserve detail, enhance visual quality, enable low light imaging Image noise: Degrades low light image quality, details are lost Problem Solution John Smith 
Problem Solution Image Signal Processing (ISP) video processing smart plug-ins TI Confidential - NDA 23 Lens Distortion, Fish Eye Correction Fish eye is lenses can get to 360 degree view TI’s Lens Distortion Correction technology enables to use cheaper fish eye lenses Auto White Balance, Auto Exposure For exposure correction and compensate for type of illumination TI’s H3A hardware provides for Auto Exposure, Auto White Balance and Auto Focus Security application: Provide clear picture in high dynamic range scenes TI’s WDR brings details in the shadow & compensate for low cost sensors
WHAT’S NEXT? 24
What problem we are trying to solve? • Two eyes help humans to perceive depth. Still, humans are prone to illusions. Images from: http://www.qianqin.de/2007/10/15/clockwise-or-counterclockwise/ • Most security cameras have only one “eye”, so they cannot easily distinguish between – shadows and a person – two people standing behind each other • Depth perception can reduce false detections by 5-10x
50 Years of Intelligent Technology Development at TI • TI started very early – In 1964, TI started developing a guidance system for a laser-guided artillery shell[1], By 1969, the first guidance kits were delivered [2] • TI’s analytics algorithms are internally developed, robust video and non- video based intelligence technologies enhanced over a long period of time. • Today, TI is a leading technology supplier to the automotive market where self-driving cars require far more accuracy than for IPCameras. 26 [2] Note: In 1997 TI exited the defense business, selling it’s defense unit to Raytheon [1] http://www.ti.com/corp/docs/company/history/timeline/defense/1960/docs/65-ti_demos_laser_guidance.htm Front Camera Surround View Smart Rear Camera Long Range Radar Others Fusion Night Vision Mirror Replacement
Time of Flight technology • Time of Flight technology measures depth based on how soon the modulated pulses reflect back to source • Works great at night when stereo is ineffective • More on TOF technology: http://www.ti.com/ww/en/analog/3dtof/in dex.shtml
Digital Light Processing (DLP) technology • DLP technology relies on structured lighting to illuminate a grid on the scene and observe the distortions in the grid to perceive depth • DLP offers higher resolution than TOF technology • More on DLP technology: http://www.ti.com/solution/3d_machine_ vision Left Hand Electric Drill Metal Screw Color maps of measured objects
Comparison of 3D Imaging Technologies Time of Flight (ToF) Stereoscopic Vision Fixed Structured Light Programmable Structured Light (DLP) Operational Principle IR pulse, measure light transit time Two 2D sensors emulate human eyes Single pattern visible or IR illumination, detects distortion Multiple pattern visible or IR illumination, detects distortion Point Cloud Generation Direct out of chipset High SW Processing Medium SW Processing SW Processing scales with # of patterns Latency Low Medium Medium Medium Active Illumination Yes No Yes Yes – Customizable Spectrum Low Light Performance Good Weak Good Good Bright Light Performance Medium Good Medium / Weak Depends on illumination power Medium / Weak Depends on illumination power Power Consumption Medium/High Scales w/ distance Low Medium Medium Scales with distance Range Short to long range Depends on laser power & modulation Mid range Depends on spacing between cameras Very short to mid range Depends on illumination power Very short to mid range Depends on illumination power Resolution QQVGA, QVGA -> Roadmap to VGA Camera Dependent Camera Dependent WVGA to 1080p -> Roadmap to WQXGA Depth Accuracy mm to cm Depends on resolution of sensor mm to cm Difficulty with smooth surface mm to cm µm to cm Scanning Speed Fast Limited by sensor speed Medium Limited by software complexity Fast Limited by camera speed Fast / Medium Limited by camera speed Applications Location X X X Identification X X X X Measurement & Inspection X X X X Biometrics X Augmented Reality X X X

More Related Content

PPSX
MEMS Laser Scanning, the platform for next generation of 3D Depth Sensors
Jari Honkanen
 
PDF
UAV-Borne LiDAR with MEMS Mirror Based Scanning Capability
Ping Hsu
 
PPT
Microvision Eyewear Ismar 2009 3
baverch
 
PPTX
MicroVision Scanning Engines Overview | January 2017
MicroVision
 
PPTX
MEMS Laser Scanning, the platform for next generation of 3D Depth Sensors
MicroVision
 
PPTX
Applications Generated from a MEMS-based Laser Beam Scanning Technology Platform
MicroVision
 
PPSX
Laser Beam Scanning LiDAR: MEMS-Driven 3D Sensing Automotive Applications fro...
Jari Honkanen
 
PPTX
MEMS and Sensors in Automotive Applications on the Road to Autonomous Vehicle...
MicroVision
 
MEMS Laser Scanning, the platform for next generation of 3D Depth Sensors
Jari Honkanen
 
UAV-Borne LiDAR with MEMS Mirror Based Scanning Capability
Ping Hsu
 
Microvision Eyewear Ismar 2009 3
baverch
 
MicroVision Scanning Engines Overview | January 2017
MicroVision
 
MEMS Laser Scanning, the platform for next generation of 3D Depth Sensors
MicroVision
 
Applications Generated from a MEMS-based Laser Beam Scanning Technology Platform
MicroVision
 
Laser Beam Scanning LiDAR: MEMS-Driven 3D Sensing Automotive Applications fro...
Jari Honkanen
 
MEMS and Sensors in Automotive Applications on the Road to Autonomous Vehicle...
MicroVision
 

What's hot (20)

PPTX
Laser Beam Scanning Short Throw Displays & an Exploration of Laser-Based Virt...
MicroVision
 
PPSX
MEMS and Sensors in Automotive Applications on the Road to Autonomous Vehicle...
Jari Honkanen
 
PPSX
MEMS-Driven Laser Beam Scanning LiDAR: The Future of Variable Spatial Resolu...
Jari Honkanen
 
PPSX
MEMS-based Laser Beam Scanning Technology Platform; Basis for Applications fr...
Jari Honkanen
 
PPSX
Laser Beam Scanning LiDAR: MEMS-Driven 3D Sensing Automotive Applications fro...
MicroVision
 
PPTX
Mitchell Reifel (pmdtechnologies ag): pmd Time-of-Flight – the Swiss Army Kni...
AugmentedWorldExpo
 
PDF
Automotive LIDAR with SensL SiPM Sensors
jcjacks
 
PDF
Dave Goldman (Lumus): State of Smartglasses Today and How We Got Here
AugmentedWorldExpo
 
PPTX
Recent Trends And Challenges In Augmented Reality
saurabh kapoor
 
PPT
Augmented reality
Anugya Shukla
 
PPT
Augmented Reality - From Marketing Buzzword To A Better Consumer Experience
Matthew Szymczyk
 
PDF
Drone Market Forecasts: Promises and Reality
Colin Snow
 
PDF
Chris Pickett (DigiLens): XR is Hard: Here’s Why
AugmentedWorldExpo
 
PPTX
AWE USA 2018 Startup Pitch: Isadore Katz with Suometry
AugmentedWorldExpo
 
PDF
The Future Of Augmented Reality - Lynne d Johnson WebVisions Portland 2014 #W...
Lynne d Johnson
 
PPT
Future force presentation_to_ministry_of_defence
Auggy Augmate
 
PDF
NextMed: How to enhance 3D radiological images with Augmented and Virtual Rea...
Technological Ecosystems for Enhancing Multiculturality
 
PPTX
Augmented reality : Possibilities and Challenges - An IEEE talk at DA-IICT
Parth Darji
 
PPTX
9 Powerful Real-World Applications Of Augmented Reality (AR) Today
Bernard Marr
 
PPT
Challenges In Augmented Reality
TELiBrahma Convergent Communications. Pvt. Ltd.
 
Laser Beam Scanning Short Throw Displays & an Exploration of Laser-Based Virt...
MicroVision
 
MEMS and Sensors in Automotive Applications on the Road to Autonomous Vehicle...
Jari Honkanen
 
MEMS-Driven Laser Beam Scanning LiDAR: The Future of Variable Spatial Resolu...
Jari Honkanen
 
MEMS-based Laser Beam Scanning Technology Platform; Basis for Applications fr...
Jari Honkanen
 
Laser Beam Scanning LiDAR: MEMS-Driven 3D Sensing Automotive Applications fro...
MicroVision
 
Mitchell Reifel (pmdtechnologies ag): pmd Time-of-Flight – the Swiss Army Kni...
AugmentedWorldExpo
 
Automotive LIDAR with SensL SiPM Sensors
jcjacks
 
Dave Goldman (Lumus): State of Smartglasses Today and How We Got Here
AugmentedWorldExpo
 
Recent Trends And Challenges In Augmented Reality
saurabh kapoor
 
Augmented reality
Anugya Shukla
 
Augmented Reality - From Marketing Buzzword To A Better Consumer Experience
Matthew Szymczyk
 
Drone Market Forecasts: Promises and Reality
Colin Snow
 
Chris Pickett (DigiLens): XR is Hard: Here’s Why
AugmentedWorldExpo
 
AWE USA 2018 Startup Pitch: Isadore Katz with Suometry
AugmentedWorldExpo
 
The Future Of Augmented Reality - Lynne d Johnson WebVisions Portland 2014 #W...
Lynne d Johnson
 
Future force presentation_to_ministry_of_defence
Auggy Augmate
 
NextMed: How to enhance 3D radiological images with Augmented and Virtual Rea...
Technological Ecosystems for Enhancing Multiculturality
 
Augmented reality : Possibilities and Challenges - An IEEE talk at DA-IICT
Parth Darji
 
9 Powerful Real-World Applications Of Augmented Reality (AR) Today
Bernard Marr
 
Challenges In Augmented Reality
TELiBrahma Convergent Communications. Pvt. Ltd.
 
Ad

Viewers also liked (20)

PPTX
Intelligent wireless video monitoring system using computer111111
venkatesh deekonda
 
PPTX
Best Paper winning PPT
Kamal Lamichhane
 
PDF
Intelligent video monitoring
ivangomezconde
 
PDF
Light Field Technology
Jeffrey Funk
 
PDF
3.survey on wireless intelligent video surveillance system using moving objec...
Alexander Decker
 
PPTX
NANOTECHNOLOGY
Pooja Khetwal
 
PPT
Captchas are becoming security theater
Peter Gordon
 
PPTX
War Field Spying Robot with Night Vision Wireless Camera
Edgefxkits & Solutions
 
PPTX
Li-Fi – Light Fidelity (Animated)
Mohammed Khan
 
PPTX
Advanced vehicle security system 2016 upload
Nishanth C
 
PPTX
optical vehicle to vehicle communication
Jisa T George
 
PPT
Captchas
NIKHIL NAIR
 
PPT
Captcha ppt
Abhimanyu Sood
 
PPT
CAPTCHA- Newly Attractive Presentation for Youth
WebCrazyLabs
 
PPTX
CAPTCHA
Preetam Rout
 
PPTX
Nanotechnology & applications in electronics
appynikki
 
DOC
Seminar report on captcha
kunalkiit
 
PPTX
Advanced vehicle security system using fingerprint & gsm new
Vedang Radio Tech. Pvt. Ltd
 
PPTX
Captcha
Vruti Surani
 
PPTX
HAAPS Technology
smart488
 
Intelligent wireless video monitoring system using computer111111
venkatesh deekonda
 
Best Paper winning PPT
Kamal Lamichhane
 
Intelligent video monitoring
ivangomezconde
 
Light Field Technology
Jeffrey Funk
 
3.survey on wireless intelligent video surveillance system using moving objec...
Alexander Decker
 
NANOTECHNOLOGY
Pooja Khetwal
 
Captchas are becoming security theater
Peter Gordon
 
War Field Spying Robot with Night Vision Wireless Camera
Edgefxkits & Solutions
 
Li-Fi – Light Fidelity (Animated)
Mohammed Khan
 
Advanced vehicle security system 2016 upload
Nishanth C
 
optical vehicle to vehicle communication
Jisa T George
 
Captchas
NIKHIL NAIR
 
Captcha ppt
Abhimanyu Sood
 
CAPTCHA- Newly Attractive Presentation for Youth
WebCrazyLabs
 
CAPTCHA
Preetam Rout
 
Nanotechnology & applications in electronics
appynikki
 
Seminar report on captcha
kunalkiit
 
Advanced vehicle security system using fingerprint & gsm new
Vedang Radio Tech. Pvt. Ltd
 
Captcha
Vruti Surani
 
HAAPS Technology
smart488
 
Ad

Similar to Next Generation Intelligent Camera technologies - Composec 2014 Keynote (20)

PDF
Introduction to Machine Vision by Cognex
RicardoCunha203173
 
PPT
(2011 10) rug - san ramon - autonomics and modernization
evgeni77
 
PDF
Product Quick Guide_Project-Oriented Premium Products_2024.pdf
ChapoGuevarra
 
PDF
Linovision intelligent ipc key features introduction
linovision
 
PDF
Automation Technology for the Machine Tools Industry Catalogue 2014/2015
ifm electronic gmbh
 
PDF
Welcome to the New-Era in Automation]
P.S.Prasad Warrier
 
PDF
Air vision2 user_guide
Helvio Matusse
 
PDF
MVS.pdf
United Institute of Technology
 
PPT
fdocuments.in_introduction-to-machine-vision-systems.ppt
EXAMCELLNPSBCET4114
 
PPTX
Intelligent image processing
Andrew Stewart
 
PPTX
Machine Vision
sanjana
 
PDF
AMB110: IT Asset Management – How to Start When You Don’t Know Where to Start
Ivanti
 
PDF
Digital Watchdog DWBJCUBE2TLX Data Sheet
JMAC Supply
 
PDF
Digital Watchdog DW-BJCUBE12T Data Sheet
JMAC Supply
 
PDF
Digital Watchdog DW-BJE2U12T Data Sheet
JMAC Supply
 
PDF
Digital Watchdog DW-BJCUBE12T Data Sheet
JMAC Supply
 
PDF
Digital Watchdog DW-BJE2U12T Data Sheet
JMAC Supply
 
PDF
Digital Watchdog DW-BJCUBE2T-LX Data Sheet
JMAC Supply
 
PPTX
BACOM Project Safe City New 14082012.pptx
PawachMetharattanara
 
PDF
vision_pdf.pdf
KarrtikBaheti1
 
Introduction to Machine Vision by Cognex
RicardoCunha203173
 
(2011 10) rug - san ramon - autonomics and modernization
evgeni77
 
Product Quick Guide_Project-Oriented Premium Products_2024.pdf
ChapoGuevarra
 
Linovision intelligent ipc key features introduction
linovision
 
Automation Technology for the Machine Tools Industry Catalogue 2014/2015
ifm electronic gmbh
 
Welcome to the New-Era in Automation]
P.S.Prasad Warrier
 
Air vision2 user_guide
Helvio Matusse
 
MVS.pdf
United Institute of Technology
 
fdocuments.in_introduction-to-machine-vision-systems.ppt
EXAMCELLNPSBCET4114
 
Intelligent image processing
Andrew Stewart
 
Machine Vision
sanjana
 
AMB110: IT Asset Management – How to Start When You Don’t Know Where to Start
Ivanti
 
Digital Watchdog DWBJCUBE2TLX Data Sheet
JMAC Supply
 
Digital Watchdog DW-BJCUBE12T Data Sheet
JMAC Supply
 
Digital Watchdog DW-BJE2U12T Data Sheet
JMAC Supply
 
Digital Watchdog DW-BJCUBE12T Data Sheet
JMAC Supply
 
Digital Watchdog DW-BJE2U12T Data Sheet
JMAC Supply
 
Digital Watchdog DW-BJCUBE2T-LX Data Sheet
JMAC Supply
 
BACOM Project Safe City New 14082012.pptx
PawachMetharattanara
 
vision_pdf.pdf
KarrtikBaheti1
 

Next Generation Intelligent Camera technologies - Composec 2014 Keynote

  • 1. Next Generation Intelligent Camera technologies Jacob Jose Product Manager Texas Instruments
  • 2. Agenda • TI’s investment in Video and Vision markets • Current technologies for intelligent camera markets • What’s coming next?
  • 3. TI Confidential - NDA Restrictions 3 TI has over 30 years of Video & Imaging innovation
  • 4. TI leads in intelligent video technologies Military Public Innovators Early adopters Early majority DM642 Surveillance Processor with DSP for analytics 2001 Lower Power Higher Performance Decreasing Form Factor• Lower system cost • Affordable software options • Solve specific market needs • Smaller form factors • Reduced power Thermal Efficiency Support Evolving Standard• Cost of ownership reduction • Standardization begins • Productivity maximized • Easy to implement • Standardization begins TI’s DMVAx-based products in market NOW Residential Do It Yourself TI DMVAx based products in market NOW Critical Facilities Gaming Retail Banking DM6467 Surveillance Processor with 1GHz C64x DSP for analytics 2005 VLIB, a library of 50+ optimized kernels available for C64x, C64x+ DSPs 2007 TI announces first video SoC (DMVA1) with Vision Co- Processor 2009 TI adds HD compression & Tamper Detect, Motion Detection, People Counting on DMVA2 2010 First customers in production using DMVA1 & DMVA2 2011 4 TI opens Vision Co- Processor for programming using the Vision Co- Processor Library 2013 Various Analytics technology development 1990 -
  • 5. TI’s DMVA3: A high performance programmable vision coprocessor • 720MHz ARM® Cortex™-A8 for control heuristics • Up to 2 megapixel real-time H.264/SVC-T • Capable of compressing videos at 20 megapixels or higher resolution while running video analytics algorithms • Customers can port their own algorithms onto the DMVA3 5 Loitering Detection Left Object Detection Object Classification Animal
  • 6. TI’s DM8127: Leading solution for Traffic Cameras • Up to 750 MHz of C674x floating point DSP • 1GHz ARM Cortex-A8 for control heuristics • 5 Megapixel Fish-eye correction + perspective correction (bird’s eye view • TI’s stereo module library to sense depth and increase accuracy by 5x- 10x • Reference design available 6 Left Image Right Image Leopard Imaging’s LNC-DUALCAM-3D based on TMS320DM8127 Depth Map
  • 7. DMVA3 and DM8127 enables customizable analytics at the edge for mainstream cameras 7 TI’s DaVinci video processors enable megapixel smart analytics IP People counting Trip zone Streaming metadata Tamper detection Motion detection
  • 8. 8 Intelligent Motion Detection (IMD) Configuration Example
  • 9. Intelligent Motion Detection Add ROI Delete ROI Clear All Save Cancel Advanced Enable ROI Label Region of Interest Settings IMD Global Settings Settings Info Enable IMD Display Object Bounding Box Display Enabled Zones Sensitivity Moderate Max. Object Size Min. Object Size 100 8000  Opening page of IMD Control Panel GUI Behavior For illustrative purposes, the yellow arrow represents an end- user’s mouse. This is the opening page for IMD’s GUI, which is called the IMD Control Panel. It contains a live preview of the camera’s field of view. ‘IMD Global Settings’, which establish parameters that apply for all regions of interest (ROIs), are set in the sub-window below preview. ROIs are managed by the sub-window to the right of the preview window. The lower-right sub-window displays feedback messages to the user when mouseover events occur or to acknowledge user input. When this page is viewed for the first time, we assume that no ROI has been defined by the user. Note that default settings are installed and buttons to delete or clear ROIs have been disabled. Whenever the ‘Defaults’ button is clicked, all settings are restored to those shown here. Intelligent motion detection is disabled. Click ‘Enable IMD’ to turn this feature on.   Defaults
  • 10. Intelligent Motion Detection Add ROI Delete ROI Clear All Save Cancel Advanced Enable ROI Label Region of Interest Settings IMD Global Settings Settings Info Enable IMD Display Object Bounding Box Display Enabled Zones Sensitivity Moderate Max. Object Size Min. Object Size 100 8000  Enabling IMD for the 1st time GUI Behavior When a mouseover event occurs over the ‘Enable IMD’ checkbox, its present status is acknowledged in the Settings Info sub-window. If the checkbox is clear and the user left-clicks it, report ‘Intelligent motion detection is enabled.’ in Settings Info. Likewise, if the checkbox has been checked and the user left-clicks it, report ‘Intelligent motion detection is disabled.’ If the mouse returns to the checkbox (after loosing focus), report ‘Click this button to turn intelligent motion detection off.’ in Settings Info. Click this button to turn intelligent motion detection on.   Defaults
  • 11. Intelligent Motion Detection Add ROI Delete ROI Clear All Save Cancel Advanced Enable ROI Label Region of Interest Settings IMD Global Settings Settings Info Enable IMD Display Object Bounding Box Display Enabled Zones Sensitivity Moderate Max. Object Size Min. Object Size 3 67  Changing default settings GUI Behavior The ‘Sensitivity’ dropdown list can be set to the following states: • Low • Low-to-Moderate • Moderate • Moderate-to-High • High When a mouseover event occurs over the ‘Sensitivity’ dropdown list, instructive information is shared in the Settings Info sub-window: ‘Try Low sensitivity to improve performance in outdoor or challenging environments. High- sensitivity works best for fast moving objects in stable lighting environments.’ The Min. Object Size slider changes the radius of a green circle rendered on the screen. Likewise, Max. Object Size will affect the radius of a red circle. These circles are drawn anytime a mouseover event occurs on the slider or adjacent input window. The user is free to move the circles to any location inside of the preview window. Move the slider or type in a value to set the minimum object size. Objects that are smaller than this value will be ignored by Intelligent motion detection.   Defaults Min Obj Size Max Obj Size
  • 12. Intelligent Motion Detection Add ROI Delete ROI Clear All Save Cancel Advanced Enable ROI Label Region of Interest Settings IMD Global Settings Settings Info Enable IMD Display Object Bounding Box Display Enabled Zones Sensitivity Moderate Max. Object Size Min. Object Size 100 8000  Enabling with no user-defined ROIs GUI Behavior If the user clicks ‘Save’ when no user-defined regions of interest have been selected, a DMVA Settings Manager dialog pops up and displays a message. This dialog has the focus of attention over any other control owned by the browser window. The dialog has redundant controls (same as those presented within the IMD Control Panel). If the user clicks ‘Save’, a new ROI is automatically defined by a 4-point polygon that shares the same coordinates as the entire frame. This ROI is enabled and labeled as ‘Field_of_View’. The dialog is closed and the page is redirected to DMVA home. If the user clicks ‘Cancel’, the dialog is closed and the focus returns to the IMD Control Panel. Intelligent motion detection is enabled.    Defaults Since no regions of interest have been explicitly defined, an alert event will be signaled when any object motion is detected within the entire field of view. Press Save to accept or Cancel to return to the IMD Control Panel. DMVA Settings Manager Save Cancel
  • 13. Intelligent Motion Detection Add ROI Delete ROI Clear All Save Cancel Advanced Enable ROI Label ROI_0 Region of Interest Settings IMD Global Settings Settings Info Enable IMD Display Object Bounding Box Display Enabled Zones Sensitivity Moderate Max. Object Size Min. Object Size 100 8000  Adding a region of interest (1) GUI Behavior If a mouseover event occurs over the ‘Add ROI’ button, an informative message is presented in ‘Settings Info’. If the user clicks this button, the top row in the scrolling list is highlighted in a brighter color. In this row, an enabled checkbox appears in the 1st column and a default ROI label, e.g. ROI_0, appears in the 2nd column. Click on this button to define a polygon to represent a new region of interest..    Defaults 
  • 14. Intelligent Motion Detection Add ROI Delete ROI Clear All Save Cancel Advanced Enable ROI Label ROI_0 Region of Interest Settings IMD Global Settings Settings Info Enable IMD Display Object Bounding Box Display Enabled Zones Sensitivity Moderate Max. Object Size Min. Object Size 100 8000  Adding a region of interest (2) GUI Behavior After the ‘Add ROI’ button is clicked, the mouse pointer changes from arrow to crosshairs when the mouse moves inside preview window. An instructive message is presented in ‘Settings Info’ that explains how to define the ROI. Clicks outside the preview window force a warning message inside Settings Info, e.g. Invalid point selected! Please pick a point inside the preview window. If 16 points have been defined while setting the ROI, the next single click will complete the polygon based on those points. Define the region of interest. Each left mouse button click within the live preview window defines a point in the polygon. Up to 16 points can be used. Double-clicking the left mouse button will automatically complete the polygon.    Defaults  
  • 15. Intelligent Motion Detection Add ROI Delete ROI Clear All Save Cancel Advanced Enable ROI Label ROI_1 ROI_0 Region of Interest Settings IMD Global Settings Settings Info Enable IMD Display Object Bounding Box Display Enabled Zones Sensitivity Moderate Max. Object Size Min. Object Size 100 8000  Adding a region of interest (3) GUI Behavior After the ROI has been established, the area inside the lines is shaded and the label displayed. Buttons ‘Delete ROI’ and ‘Clear All’ become enabled and the top row of the ROI list loses the focus (no highlighting). To add more ROIs, the user must click ‘Add ROI’ for each region. Once activated, the user must click the left mouse button inside the preview over a location not already defined by an existing ROI to create a new ROI. The elements in the ROI list will grow accordingly. Define the region of interest. Each left mouse button click within the live preview window defines a point in the polygon. Up to 16 points can be used. Double-clicking the left mouse button will automatically complete the polygon.    Defaults  ROI_0  
  • 16. Intelligent Motion Detection Add ROI Delete ROI Clear All Save Cancel Advanced Enable ROI Label DESK ROI_0 Region of Interest Settings IMD Global Settings Settings Info Enable IMD Display Object Bounding Box Display Enabled Zones Sensitivity Moderate Max. Object Size Min. Object Size 100 8000  Editing a region of interest GUI Behavior The user can change the label name at any point by click on the label in the ROI list. Any name under 64 characters is exceptable as long as the name is unique, case-insensitive, begins with a letter, etc. (same restrictions as filenames). When editing, the cursor is displayed and the entire row is highlighted. If the another ROI was being defined when the user tries to edit a label name, store the settings to a valid state, e.g. close the polygon as if a double-click were registered. Re-label the name of the region of interest. Limit names to 64 characters and avoid leading numbers and the following characters: ` , ~ ! @ # $ % ^ & * ( ) / ? ‘ { } |    Defaults  ROI_0  DESK
  • 17. Intelligent Motion Detection Add ROI Delete ROI Clear All Save Cancel Advanced Enable ROI Label DOOR DESK MAIN_FLOOR Region of Interest Settings IMD Global Settings Settings Info Enable IMD Display Object Bounding Box Display Enabled Zones Sensitivity Moderate Max. Object Size Min. Object Size 100 8000  Deleting a region of interest GUI Behavior If a mouseover event happens over the ‘Delete ROI’ button, Settings Info reports ‘Click to delete a highlighted ROI.’ Pressing this button without first selected a ROI label name to do nothing; Settings Info will report ‘You must select a ROI label name first to delete.’ Double-clicking a label name gives it the focus of attention, e.g. highlights it. If ‘Delete ROI’ is clicked after the row is highlighted, the corresponding overlay is also brightened. Clicking this button a second time will delete the element in the list as well as the overlay region in the preview window. If the focus of attention is diverted to the keyboard or another location on the page not over the button, the delete operation is canceled. No overlap tolerated. Max. ROI = 16 Click again to delete a highlighted region of interest.    Defaults  ROI_0  DESK  DOOR
  • 18. 18 DMVA Event Monitor Excerpts from DMVA Graphical User Interface (GUI) Specification
  • 19. DMVA Event Monitor Timestamp Event Type Event Details Tu Oct 7 09 13:29:39.1 IMD Zone = LOBBY Duration = 00:1:39.2 Tu Oct 7 09 13:34:14.3 IMD Zone = LOBBY Duration = 00:0:16.3 Tu Oct 7 09 13:38:24.5 IMD Zone = LOBBY Duration = 00:0:59.5 Tu Oct 7 09 13:45:19.0 COUNT Num = 2 Direction = LATERAL Tu Oct 7 09 13:54:35.8 TRIP ZONE Direction = Zone 12 Tu Oct 7 09 13:55:19.3 IMD Zone = DESK Duration = 00:19:49.2 Event List DMVA Message Window Timestamp Metadata Overlay DMVA Live Event Monitor GUI Behavior The DMVA Event Monitor page is the main DMVA page, e.g. homepage. It features a video window, a scrolling list of event summaries, function indicators/selectors, controls for display and event recording, and a message window. The video window shows both live (default) and archived video, which can be marked up by graphical overlays. The ‘Display’ subwindow allows the user to select the desired overlay. A row of indicators/selectors sits below. Indicator “lamp” for an active analytic function glows “green”; Tamper, which runs at all times, can also glow “red” during active tamper events. Lamps also used to 1 of the 4 analytic functions that can run simultaneously with tamper. After function is selected, user can click ‘Configure’ to bring up the dedicated configuration page. During each event, the compressed audio and/or video, metadata, and detailed event log file can be saved. Tamper alerts are displayed in the DMVA Message window. Overall DMVA health and status information is shown here as well. Tamper alert: Camera movement detected Tues Oct 7 2009 13:56:09.2! LOBBY DESK Tamper     Analytics Display Object Bounding Box Enabled Zones Event Recording Video Stream Audio Stream Metadata Stream Event Log CancelConfigure Advanced IMD CountingTripZone SMOS
  • 20. DMVA Event Monitor Timestamp Event Type Event Details Tu Oct 7 09 13:29:39.1 IMD Zone = LOBBY Duration = 00:1:39.2 Tu Oct 7 09 13:34:14.3 IMD Zone = LOBBY Duration = 00:0:16.3 Tu Oct 7 09 13:38:24.5 IMD Zone = LOBBY Duration = 00:0:59.5 Tu Oct 7 09 13:45:19.0 COUNT Num = 2 Direction = LATERAL Tu Oct 7 09 13:54:35.8 TRIP ZONE Direction = Zone 12 Tu Oct 7 09 13:55:19.3 IMD Zone = DESK Duration = 00:19:49.2 Event List DMVA Message Window Timestamp Metadata Overlay DMVA Live Event Monitor Cllick ‘Advanced’ to fine-tune camera tamper detection parameters. LOBBY DESK Tamper     Analytics Display Object Bounding Box Enabled Zones Event Recording Video Stream Audio Stream Metadata Stream Event Log CancelConfigure Advanced IMD CountingTripZone SMOS GUI Behavior If the Advanced button is pushed, a dialog box appears. For now, it will show camera tamper options. Press Save to accept or Cancel to return to the DMVA Event Monitor. Save Cancel Advanced DMVA Settings Detect lights on-to-off events Detect lights off-to-on events Camera Tamper Detection Sensitivity Moderate  Reset Sec. 30
  • 21. DMVA Event Monitor Timestamp Event Type Event Details Tu Oct 7 09 13:29:39.1 IMD Zone = LOBBY Duration = 00:1:39.2 Tu Oct 7 09 13:34:14.3 IMD Zone = LOBBY Duration = 00:0:16.3 Tu Oct 7 09 13:38:24.5 IMD Zone = LOBBY Duration = 00:0:59.5 Tu Oct 7 09 13:45:19.0 COUNT Num = 2 Direction = LATERAL Tu Oct 7 09 13:54:35.8 TRIP ZONE Direction = Zone 12 Tu Oct 7 09 13:55:19.3 IMD Zone = DESK Duration = 00:19:49.2 Event List DMVA Live Event Monitor GUI Behavior DMVA generates an output response for every frame of video. For each event, an summary record is created that establishes the duration of the event and associates the timestamp, type, and relevant zone information (context dependent based on the enabled analytic function). When a user selects an event from the list and ‘Video Stream’ has also been selected (at time of event as well as while user is interacting) under ‘Event Recording’, the compressed video of the event is retrieved and replayed in the window. If ‘Metadata Stream’ has also been select, bounding box information is available for display (depends on whether ‘Metadata Overlay’ has been checked or not). Video and metadata recorded for this event. LOBBY DESK     Tamper Analytics IMD CountingTripZone SMOS CancelConfigure Advanced Timestamp Metadata Overlay Object Bounding Box Enabled Zones Video Stream Audio Stream Metadata Stream Event Log Display Event Recording       DMVA Message Window
  • 22. Image Signal Processing (ISP) video processing smart plug-ins 22 Video stabilization: Camera shakes due to weather, traffic → degrades video quality TI’s advanced video stabilization reduces jitter, improves visual quality Face detection tracking, recognition: For identification, access control, privacy mask TI’s face detection HW tracks multiple faces (>30) with detection in <20 milliseconds TI’s Video Noise Filters preserve detail, enhance visual quality, enable low light imaging Image noise: Degrades low light image quality, details are lost Problem Solution John Smith 
  • 23. Problem Solution Image Signal Processing (ISP) video processing smart plug-ins TI Confidential - NDA 23 Lens Distortion, Fish Eye Correction Fish eye is lenses can get to 360 degree view TI’s Lens Distortion Correction technology enables to use cheaper fish eye lenses Auto White Balance, Auto Exposure For exposure correction and compensate for type of illumination TI’s H3A hardware provides for Auto Exposure, Auto White Balance and Auto Focus Security application: Provide clear picture in high dynamic range scenes TI’s WDR brings details in the shadow & compensate for low cost sensors
  • 25. What problem we are trying to solve? • Two eyes help humans to perceive depth. Still, humans are prone to illusions. Images from: http://www.qianqin.de/2007/10/15/clockwise-or-counterclockwise/ • Most security cameras have only one “eye”, so they cannot easily distinguish between – shadows and a person – two people standing behind each other • Depth perception can reduce false detections by 5-10x
  • 26. 50 Years of Intelligent Technology Development at TI • TI started very early – In 1964, TI started developing a guidance system for a laser-guided artillery shell[1], By 1969, the first guidance kits were delivered [2] • TI’s analytics algorithms are internally developed, robust video and non- video based intelligence technologies enhanced over a long period of time. • Today, TI is a leading technology supplier to the automotive market where self-driving cars require far more accuracy than for IPCameras. 26 [2] Note: In 1997 TI exited the defense business, selling it’s defense unit to Raytheon [1] http://www.ti.com/corp/docs/company/history/timeline/defense/1960/docs/65-ti_demos_laser_guidance.htm Front Camera Surround View Smart Rear Camera Long Range Radar Others Fusion Night Vision Mirror Replacement
  • 27. Time of Flight technology • Time of Flight technology measures depth based on how soon the modulated pulses reflect back to source • Works great at night when stereo is ineffective • More on TOF technology: http://www.ti.com/ww/en/analog/3dtof/in dex.shtml
  • 28. Digital Light Processing (DLP) technology • DLP technology relies on structured lighting to illuminate a grid on the scene and observe the distortions in the grid to perceive depth • DLP offers higher resolution than TOF technology • More on DLP technology: http://www.ti.com/solution/3d_machine_ vision Left Hand Electric Drill Metal Screw Color maps of measured objects
  • 29. Comparison of 3D Imaging Technologies Time of Flight (ToF) Stereoscopic Vision Fixed Structured Light Programmable Structured Light (DLP) Operational Principle IR pulse, measure light transit time Two 2D sensors emulate human eyes Single pattern visible or IR illumination, detects distortion Multiple pattern visible or IR illumination, detects distortion Point Cloud Generation Direct out of chipset High SW Processing Medium SW Processing SW Processing scales with # of patterns Latency Low Medium Medium Medium Active Illumination Yes No Yes Yes – Customizable Spectrum Low Light Performance Good Weak Good Good Bright Light Performance Medium Good Medium / Weak Depends on illumination power Medium / Weak Depends on illumination power Power Consumption Medium/High Scales w/ distance Low Medium Medium Scales with distance Range Short to long range Depends on laser power & modulation Mid range Depends on spacing between cameras Very short to mid range Depends on illumination power Very short to mid range Depends on illumination power Resolution QQVGA, QVGA -> Roadmap to VGA Camera Dependent Camera Dependent WVGA to 1080p -> Roadmap to WQXGA Depth Accuracy mm to cm Depends on resolution of sensor mm to cm Difficulty with smooth surface mm to cm µm to cm Scanning Speed Fast Limited by sensor speed Medium Limited by software complexity Fast Limited by camera speed Fast / Medium Limited by camera speed Applications Location X X X Identification X X X X Measurement & Inspection X X X X Biometrics X Augmented Reality X X X