Validation Framework for Autonomous Aerial Vehicles
0 likes105 views
This document discusses research efforts on validation and verification (V&V) of autonomous vehicles. It outlines the rise of autonomous technologies like self-driving cars and drones and the challenges they pose. It then introduces a validation framework developed by the author and his team at Florida Polytechnic University. This framework uses abstraction layers, simulation, and scenario testing to efficiently and progressively test autonomous systems. It aims to decompose real-world data into basic elements and automatically generate edge cases to thoroughly evaluate decision making capabilities. The framework represents a novel approach to the slow, costly, and incomplete V&V methods used today.
Validation Framework for Autonomous Aerial Vehicles
- 1. Validation Framework for Autonomous Aerial Vehicles Research Efforts on the V&V of Autonomous Vehicles 03/03/2020 RKA 2016-06-22 M. Ilhan Akbas, Ph.D. Assistant Professor of Electrical and Computer Engineering Embry-Riddle Aeronautical University
- 2. Outline • About Our Research • Rise of Autonomy and Challenges • Validation Framework • Related Research Themes • Summary
- 3. About Our Research • M. Ilhan Akbas – BS in EE, MS and PhD on Communication Technologies – Post Grad Work on Modeling & Simulation Projects – Industry Experience in Defense, NATO Projects • Research on AVs, CPS, Intelligent Mobility • Projects from NSF, Cyber Florida, State • Published in SAE, IEEE Trans., ACM • Teaching AV, M&S, Soft. Eng, Networks, Security
- 4. Outline • About Our Research • Rise of Autonomy and Challenges • Testing Framework • Related Research Themes • Summary
- 5. Self-Driving Cars McKinsey defines the DNA of Industry 4.0 convergent disruption as the proliferation of big data, adv. analytics, HMI and dig.-to-physical transfers.
- 6. Autonomous Drones McKinsey defines the DNA of Industry 4.0 convergent disruption as the proliferation of big data, adv. analytics, HMI and dig.-to-physical transfers.
- 7. Market Disruptions Autonomous Cars • Changing car ownership, MaaS. • Insurance, maintenance, towing etc. • Disabled and elderly will have access to mobility. • Parking practices will change. Autonomous Drones • Ground delivery vs drone delivery. • Air taxis, MaaS. • Efficient services such as surveillance, mapping. • Airline/helicopter piloting changes.
- 8. Potential Impact: ● 24x7 Access ● Reuse of Infrastructure (Utilization) ● On-call Access Simplifications: ● Limited Paths ● Managed Intersections and Pedestrians ● Lower Speeds Public Transport
- 9. Potential Impact: ● Warehouse Operations ● Outdoor Logistics Operations ● Long Haul and Last Mile Simplifications: ● Controlled Environments ● Human Leverage (“follow-me” model) Logistics
- 10. Potential Impact: ● Enable driverless elderly communities ● Increase Safety and Health Simultaneously ● Efficient Architectural Designs Simplifications: ● Lower Speeds ● Controlled Environments ● Less Complex Interaction Models Planned Communities
- 11. Potential Impact: ● 24x7 Operation ● Higher Level of Capability (e.g. Fruit Picking Robots) Simplifications: ● Lower Speeds ● Less Complex Interaction Models Agriculture
- 12. Autonomous Car Market Estimates Market estimates vary but several groups estimate the market over $80 Billion in just under fifteen years
- 13. Autonomous Car Market Estimates Market estimates vary but several groups estimate the market over $80 Billion in just under fifteen years
- 14. Autonomous Drone Market Estimates Market estimates vary but several groups estimate the market over $40 Billion in just under fifteen years
- 15. Autonomous Drone Market Estimates Market estimates vary but several groups estimate the market over $35 Billion in just under fifteen years
- 16. Research Goal Our research goal is focused on building a framework to identify complex rare scenarios for AV Validation
- 17. Current Efforts in AV Validation • Real World Testing • Low probability tests are difficult to produce • Extremely slow and costly • Test Tracks & Labs • Recreates various situations • Can’t be the sole solution for scenario analysis • Image Dataset Based Testing • Datasets of images for assisted control • Still slow and costly • Simulation Based Testing • Current solutions are not progressive All current verification efforts are slow, costly, hard to repeat and not progressive NY Times - 1991
- 18. Challenges Autonomous Cars • 2D environment • Established regulations • Congested operation domain • More prescriptive (roads, signs etc.) Autonomous Drones • 3D environment • Shifting regulations • Challenging last 10 feet of delivery • Possible high number of adversarial actors Autonomous technology is applicable in both domains with similar challenges and different constraints
- 19. Outline • About Me • Rise of Autonomy and Challenges • Validation Framework • Other Research Themes • Summary
- 20. Technology Core • Hard-wired vehicles operated by people replaced by software-defined & networked computers operated by intelligent agents • Fundamental technology is a traditional networked sensor and signal processing chain with a decision support system PERCEPTION SIGNAL PROCESSING ACTUATION Signal Processing Signal Processing Signal Processing Signal Processing DataFusion Artificial Intelligence Network Acceleration Steering Brake... ... ... ... Passive Active GPS Network/IoT
- 21. Critical Issues • Conceptual Model: What are the conceptual models for decision making and perception stages? • Test Regime: What is the test regime that can build confidence for the operation of the AV? • Completeness: How do we understand the state space sufficiently? • Accumulative Learning: How do we know the next version of the AV is safer?
- 23. AV Testing Solution Smart Cities Controlled Environment Lab Our research aims to create a full framework to conduct research, development and testing of AVs
- 24. Similarities to Hardware V&V • Semiconductor chips are composed of complex components • Most semiconductor chips can be simulated in the low kilo-hertz range while run in giga-hertz range • There is a need to compress long learning cycle into a reasonable development cycle • There is a need for robustness to environmental conditions • There is a need for completeness and accumulated learning
- 25. Solutions from Hardware V&V • Use of abstraction to decompose the problem • Development of various abstraction levels (transistor, gate, RTL, micro-architecture, architecture, network) • At the highest level: • Formal verification • Constrained-random test generation • Real-world test injection • Coverage analysis Our solution aims to use the powerful methods of hardware and software V&V for Autonomous Vehicles
- 26. Scenario Testing Solution • Characterization • Decompose data sets from physical world into atomic blocks • Scenario generation • Build virtual models using atomic elements • Coverage analysis • Track cross-product cases • Certification • Based on coverage completion The solution is focused on the Decision Making of Autonomous Vehicles
- 27. Scenario Abstraction Scenario Simulation Scenario Test Generation Constraints Scenario Abstraction Scenario Database - Converge Matrix Industry/ Regulators Test Lab Scenario Scenario Abstraction from Real Life Test Lab Diagnostic Scenario Abstraction The framework serves as a ground truth for testing other sources of error such as environmental conditions or sensor failures
- 28. Formulation of Test Scenarios Assertion functionality on top of this inner core provides the concepts of good/bad or pass/fail
- 30. • Semantic language is used to define scenarios • Actors, environment are created/validated • Low fidelity simulator is the first step for implementation Implementation
- 31. Florida Poly AV Testing VisionAV Testing Framework Roadmap Simulation & Scenario Testing Hardware in the Loop Testing AV Testing Individual Sensor Testing TimeTime Next: Automatic test pattern generation to find edge cases, collaboration with environmental and HiL Simulators
- 32. Outline • About Me • Rise of Autonomy and Challenges • Testing Framework • Related Research Themes • Summary
- 33. Transportation OS• Transportation System is inefficient • Utilization of Market Economics and Network Ideas Transportation Operating System What if we could dynamically use the transportation system resources?
- 34. Human/AV communication Incentive Mechanisms for Mobile Crowdsensing • Spatial and temporal coverage for sampling in a target area and isolated sub-regions is important • An incentive mechanism that dynamically assigns compensation for data collection in the sub-regions How can we incentivize users of ITS for coverage?
- 35. Human/AV communication • Humans have operating languages, AVs need it as well Cooperation of AVs with Human Operators What about the tourists and residents?
- 36. Outline • About Me and Florida Poly • Rise of Autonomy and Challenges • Florida Poly Solution • Florida Poly Research Themes • Summary
- 37. Thank You! M. Ilhan Akbas, PhD Assistant Professor of Electrical and Computer Engineering Embry-Riddle Aeronautical University • akbasm@erau.edu • https://sites.google.com/site/miakbas/ • https://www.linkedin.com/in/miakbas/