Autonomous driving revolution- trends, challenges and machine learning
Download as PPTX, PDF6 likes1,628 views
The document discusses trends in autonomous driving, including challenges when big data meets machine learning in cars. It outlines how sensor systems collect big data from cameras, radar, ultrasound and lidar. Machine learning is then used to perceive the real world through techniques like object recognition, 3D scene understanding, semantic segmentation and reinforcement learning. Autonomous vehicles will also need powerful embedded computing and connectivity through vehicle-to-vehicle and cloud networks.
Autonomous driving revolution- trends, challenges and machine learning
- 1. Autonomous driving revolution: trends, challenges and machine learning Junli Gu 1
- 2. Outline • Vehicle revolution • Challenges: when Big Data meets Machine Learning in the car • Sensor System collects Big Data • Machine Learning perceives the real world • Car is a Digital Agent • Conclusions 2
- 3. Transportation evolution history 3 P1: Analog device P2: Digitalization P3: Intelligence 1900 2020 P0: Primitive From IHS
- 4. A revolution from analog to digital to intelligence 4 • All parts are digital controllable • Complex sensor systems • World perception • Controlled by computers • A car will become an agent • With autonomous behavior
- 5. Big Data meets Machine Learning in the Car 5 World Perception GPS, mapping, localization Motion Control Path Planning • Real world is a Big Data problem • Driving in human world required intelligent perception of the world
- 6. Hybrid sensor system collects Big Data • A sensor system composed of Cameras, Radar, Ultrasound, Lidar • Sensor fusion technology is not mature yet • Different sensor data is mostly computed separately 6
- 7. Case study in combining sensors • Google self driving cars • Velodyne 64-beam laser + 4 radars+ 1 camera+ GPS • generates a 3D geometry map of the environment • Radar info is too thin to differentiate objects of same size, same speed • Cameras see rich semantics. What you see is what you get. But no depth 7 What radar sees What camera sees
- 8. Machine learning perceives the real world • Large scale of 2D object recognition is better than human • What objects are around • 3D scene understanding and modeling • Where is the object • Semantic segmentation • Extent of the obstacles • Reinforcement learning • Policy (reward or penalty based learning) • End to end learning • From raw data direct to behavior, like a robot 8
- 9. Deep learning based 2D image recognition 9 97% 2016 • Large scale object recognition 97% accuracy • Lane detection, pedestrian detection, vehicle detection • Animal detection, and road surface detection etc. • Case: Mobile eye from traditional algorithm => Deep learning DNN model Big Data Human 95%
- 10. 3D scene understanding • Real world is not flat. It is 3D. • Depth information is critical for driving. • 3D model is far more complicated. 10 Reference: Lin Yuanqing, “Building Blocks for Visual 3D Scene Understanding towards Autonomous Driving”
- 11. Semantic segmentation • Understand the extent of the object and each pixel of it • Advanced requirement based on recognition and detection • Eventually 3D world segmentation 11 Above: “SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation” Right: “Object Scene Flow for Autonomous Vehicles”
- 12. Reinforcement learning • Goal: to win • Policy based learning (reward/penalty) • Learning task: next move at arbitrary states • Algorithm: reinforcement learning + DNN • Similar methodology can be applied to driving • Driving Policy: Given safety, spend least time to get to destination • The car figures out how to drive by following the reward or penalty 12 Google Alpha go
- 13. End to end learning • Use machine learning as the only step from raw data to control • Without human interference noise in the middle steps • DaveNet: AI teaches the car how to drive • NVidia GTC 2016 “after 3000 miles of supervised driving, their car can navigate safely on freeways and country roads, even on rainy days.” “End to End Learning for Self-Driving Cars”, arxiv.org ; Google “Human-level control through deep reinforcement learning” 13
- 14. Other challenges in the Car • Embedded computers in the car • Cloud solution • Vehicle external connections (V2E) • Vehicle internal connection 14
- 15. Embedded computing systems • Real time computing for the sensor array’s Big Data input • Heterogeneous SoC + HPC level computing (Tera flops) • Reliability is another key 15 ARM DSP GPU accelerator General purpose Control friendly Customized processor machine learning Computing throughput Mobile eye Google TPU NVidia drive PX Qualcomm snapdragon
- 16. Cloud based solution • Agent+ system: cloud updates with traffic condition, bad weather, etc. (Toyota) • Connect smart home with car (Volks Wagen, with LG software) • Challenges: network reliability and real time response 16 cloud Collect traffic info broadcast info Collect smart home
- 17. V2E (vehicle to everything) network • Collectively become smarter: share learning on the network • Dephi: V2V(vehicle), V2B(building), V2P(pedestrain) • Tesla: fleet learning. Baidu: vehicle to road 17
- 18. High speed internal interconnect • All digital components communicate • High speed due to many sensor • Reliability and redundant design 18 Credit to Texas Instruments
- 19. Conclusions • Cars are transforming from analog to digitalization and intelligence • Future car will be a very complex digital autonomous device, similar to Robots • When Big Data meets Machine learning in the car • Complex sensor system collects the Big Data info • Machine learning algorithms will be the key to perceive the real world • Computing in the car is another challenge • Cars will be connected to each other and the cloud • Whoever addresses the technical challenge will harvest the influence
Editor's Notes
- #2: Abstract: Autonomous driving has gain enormous attention and momentum in the past year, due to its potential huge impact on industry. This talk will summarize the current trends, my personal understanding of the revolution. Then the talk will highlight the technical challenges and share some insights in how machine learning might lead us to the path. I deliver the talk today on behalf of myself, not telsa. So I will not talk about tesla project. At the current stage, there are many different voice or efforts. so I will focus on providing a high level introduction and summary of the trends. We have a panel discuassion following the talk, we have a few tesla experts, baidu expert and faraday future expert.
- #5: All components are connected Drivers are no longer needed Great industry impact, revolutionary impact
- #6: Object Scenaros Event Time Every country, every region The car will be part of the real wolrd, it takes the data, it generates new event and data
- #8: One of the earliest efforts in self driving car is google car. From information category perspective, we need both 2D visual informaiton, with 3D depth info, speed info to build a complete picutre of the world
- #9: After sensors collects the data, a much more challenging task is to perceive the world based on the data The biggest challenge is that there is no existing algorithm that describes the world, due to it is high complex and high dimension People believe that machine learning will play a critical role in perceiving the world. In this seciton I will provide a summary of the categories of machine leraning and AI related algorithm that has been tried or will have good potential in autonomous driving. In order for car to drive itself, we need a lot of detailed information, which include 2d object recognition to tell us what is around us, 3D scene understiang with tell us where is the object and distance from us We will also need the extent of the obstacles, the sizes of it, so that we can segment it from the free space Recently google apha go has showed latest research direction of reinforcement learning and end to end learning. Which I will also cover to explain their potential in autonomous driving
- #10: Deep learning is the most succefully mahcine lernaing algorithm in 2D image recognition The algorithm is based on a very complicated DNN model, usually millions to billions of parameters, take big data input, and generatee object recognition results, detection results Deep learning has achieved a lot of break through in object recognition accuracy, the latest google inception model has achieved 97% accuracy which is better than a human expert, 95%. Deep learning algorithm provides a great legacy for us to delpoy to do lance detection, pedestrain detection etc. Mobile eye solution used to use traditional computer vision algorithms, nowadays they also start to use deep learning
- #11: The world is not flat. Human see everything within a 3D world. Vehicles can not drive based on flat world modeling
- #17: Tesla fleet learning is so far the most sucessful cloud solution, used in currect tesla cars Cloud-based computer vision is about delivering powerful algorithms as a service, over the web. In a cloud-based architecture, the algorithms live in a data center and applications talk to the vision backend via an API layer. And while certain mission-critical applications cannot have a cloud-component agent+ system, based on cloud (cloud updates with traffic condition, bad weather, instructions etc)
- #18: Eg Baidu is promoting, dephi Compared to cloud, this is a p2p network