Autonomous Vehicle Development with Unity
3 likes1,404 views
The document discusses the challenges and solutions in developing autonomous vehicles, emphasizing the need for simulation to prepare these systems for unforeseen scenarios. Unity's SimViz initiative aims to provide tools and environments to support simulation and visualization for autonomous vehicle development, focusing on data-centric machine learning applications. The roadmap outlines plans for enhancing simulation capabilities, including a variety of sensor models and environments to facilitate effective training and integration.
Autonomous Vehicle Development with Unity
- 1. “Autonomous Vehicles is a very easy technology to develop. Self-driving cars are, in fact, a given.” -- Nobody Ever
- 2. How can we increase the brainpower and optimize the efforts applied to one of the hardest problems in tech?
- 3. Jose De Oliveira Lead Engineer for Autonomous Vehicles Unity
- 4. Synthetic Environments for Autonomous Vehicles Development
- 5. Agenda • Context • Why simulate • Unity SimViz • SimViz Roadmap? • SimViz Announcement
- 6. Agenda • Context • Why simulate • Unity SimViz • SimViz Roadmap • SimViz Announcement
- 7. Product Design Sales & Marketing Service & Training HMI Autonomous Vehicles Automotive Areas
- 8. Agenda • Context • Why simulate • Unity SimViz • SimViz Roadmap • SimViz Announcement
- 9. • ML is core for AV and is data-centric • Collecting data is expensive • Annotating data is both expensive and error prone • Real-world data can only prepare for what has already happened The autonomous vehicles data problem
- 11. Simulation can get autonomous systems ready for unforeseen scenarios and edge conditions in a fast and cost effective way. Why Simulate?
- 12. Agenda • Context • Why simulate • Unity SimViz • SimViz Roadmap • SimViz Announcement
- 13. Sim.Viz (/simviZ/), noun 1. Concatenation of the words “simulation” and "visualization" (plus some alliteration for flair). SimViz: ! • Unity’s initiatives, features and products to support simulation and visualization applications in the fields of autonomous vehicles (AV) and beyond.! !
- 14. External Data! (HD Maps,V2X)! Autonomous System (Real or Simulated)! Simulation Environment! Control! Planning! Perception! Coordination! Vehicle Dynamics! Environment! Sensor Model! Parameters (weather, time)! Dynamic Elements! Static Elements! Scenarios! Simulation systems
- 15. External Data! (HD Maps,V2X)! Autonomous System (Real or Simulated)! Simulation Environment! Control! Planning! Perception! Coordination! Vehicle Dynamics! Environment! Sensor Model! Parameters (weather, time)! Dynamic Elements! Static Elements! Scenarios! Simulation systems SimViz!
- 16. Agenda • Context • Why simulate • Unity SimViz • SimViz Roadmap • SimViz Announcement
- 17. How can Unity help AV development • Incentivize development community: environments and tools for simulations • Build AV and ML support into the platform • Promote an ecosystem of companies that extend the platform and create simulation solutions Licensed image from Adobe Stock
- 18. Asset Store Auto Packages EasyRoads3D HD Pack (Andasoft)! Enviro - Sky and Weather (Hendrik Haupt)! https://assetstore.unity.com/lists/real-world-simulation-54066!
- 19. Robust ecosystem
- 20. Sensors! Content! Integration! SimViz Solution Templates Wording Sensor SDK! Reference models! Leverage 2018+ features! Communication layer! Performance gains! Linux improvements! ML Agents! Support to open formats! Pipelines for data ingestion! Dynamic elements models! Starter content!
- 21. Unity ML Agents Workflow Set Up Game for Training! Train Agents! Embed Agents!
- 22. Q4! Q1! Q2! Q3! Q4! SimViz Reference Package! • Sensor template! • Content template! ROS Integration! Platform Integration! • DxM (Data Exchange Module)! • OpenDrive Support! • Sensor Ready Materials! Scalable Deployment! • Headless runtime! • AI based pedestrian model! • Sensor Validation Standards! • ML agent integration! SimViz Solution Roadmap 2018! 2019!
- 23. Agenda • Context • Why simulate • Unity SimViz • SimViz Roadmap • SimViz Announcement
- 24. SimViz Solution Template: Environments Artificial Scenes included: ! o City streets o Open Highway o Residential streets
- 25. SimViz Solution Template: Sensors Sensor Models included:! o RGB-D Camera o LiDAR o Radar o IMU o GPS SynCity Lite
- 26. Integration of Scenes and Sensor Templates