Self-Flying Drones: On a Mission to Navigate Dark, Dangerous and Unknown Worlds
3 likes352 views
The document discusses advancements in self-flying drones and their potential applications in navigating challenging environments, such as nuclear site inspections and GPS-denied areas. It highlights the importance of autonomy, robust perception, and multi-modal sensor fusion for effective robotic operation, particularly in degraded visual environments. The work aims to develop sophisticated algorithms for mapping, exploration, and radiation detection to enhance safety and efficiency in hazardous conditions.
![Nuclearized Robotics
Christos Papachristos, Autonomous Robots Lab, University of Nevada, Reno
´ Robotic Detection of Radiation
Other Radiation types:
´ Neutron Detection is relevant with homeland
security and industrial monitoring (e.g.
detection of nuclear weapons, personnel
monitoring, water content in soil).
´ Gas-filled (e.g. He-3), Scintillation, Solid-State.
´ Alpha Detection is very challenging. Alpha
particles are the heaviest and highly charged,
they quickly give up their energy to any
medium they pass.
´ Special detection methods are required, gas-
filled detector [ZnS(Ag)] combined with
Aluminized Mylar film.
´ Requires contact & robotic manipulation.](https://image.slidesharecdn.com/christospapachristossiliconmountain-171006180027/85/Self-Flying-Drones-On-a-Mission-to-Navigate-Dark-Dangerous-and-Unknown-Worlds-36-320.jpg)
Self-Flying Drones: On a Mission to Navigate Dark, Dangerous and Unknown Worlds
- 2. Self-Flying Drones: On a mission to navigate Dark, Dangerous and Unknown Worlds Christos Papachristos Autonomous Robots Lab, University of Nevada, Reno
- 3. Broader Vision ´ In all non-too-distant visions of the future, robots are part of everyday lives, fulfilling the roles of humanity’s need for comfortable transportation, everyday safety and security, a tireless and reliable workforce, or even that of a convenient company. The robots of such a future –from single appliance to entire cities– can operate on their own. ´ Robotics can promote sustainable and scalable growth, even out societal disparity, improve our quality of life, accelerate scientific progress, and more. ´ To reach this scale, a concrete baseline is necessary to provide the foundations of high- level perception, navigation, task-handling, reasoning, etc. ´ Autonomy is the key. The absolute baseline is Robust Perception and Autonomous Planning. Christos Papachristos, Autonomous Robots Lab, University of Nevada, Reno
- 4. Motivation ´ On one hand Aerial Robots are exceptional candidates for jobs that require going to remote locations to inspect, map, and monitor their environment. These locations can be: ´ Particularly difficult to reach and/or GPS-denied. ´ Engulfed in complete darkness or White-washed. ´ Hazy due to atmospheric conditions or dust within enclosed spaces. ´ Hazardous for human health. ´ Autonomous Aerial Robotic Operation in GPS-denied Degraded Visual Environments ´ Indicative Application domains: ´ Nuclear Site Decommissioning ´ Remote Infrastructure Inspection ´ Oil & Gas Industry Inspection ´ Surveillance, Security Monitoring Christos Papachristos, Autonomous Robots Lab, University of Nevada, Reno
- 5. Putting the Pieces together ´ Robotic Autonomy : The ability to operate without the need for human action and reasoning and make own choices. ´ Generate moves sequence from A to B. ´ Objectives: Exploration, Inspection, … ´ Rules: Collision-free, Power-limitations, … ´ Optimize response. ´ Guarantee constraints. ´ Robot Configuration. ´ Sensor Suite. ´ Processing Components. ´ Multi-modal Perception: Visual, Inertial, LIDAR, Thermal, … ´ Robust State Estimation: GPS-denied, DVE(s), … A baseline for Autonomous Mobile Robots Christos Papachristos, Autonomous Robots Lab, University of Nevada, Reno
- 6. Christos Papachristos, Autonomous Robots Lab, University of Nevada, Reno Putting the Pieces together ´ Starting off with the basics for a simple yet fully-autonomous aerial robot:
- 7. Visual – Inertial SLAM Christos Papachristos, Autonomous Robots Lab, University of Nevada, Reno ´ The Simultaneous Localization & Mapping problem Vision-based feature detection and tracking Detect – Track – Recover Structure from Motion ´ Formulate as Estimation process with Bayesian Reasoning. ´ Correlate robot pose Uncertainty to measurement Uncertainty (landmark 3D positions). ´ Information Fusion via Joint Distribution of processes that contain Uncertainty.
- 8. Visual – Inertial SLAM Christos Papachristos, Autonomous Robots Lab, University of Nevada, Reno ´ Visual-only SFM Limitations 2D-projective transformation introduces problem of scale Inertial (IMU) data have absolute scale ´ Use an Extended Kalman Filter – Prediction Step
- 9. Visual – Inertial SLAM Christos Papachristos, Autonomous Robots Lab, University of Nevada, Reno ´ Visual-only SFM Limitations 2D-projective transformation introduces problem of scale Inertial (IMU) data have absolute scale ´ Use an Extended Kalman Filter – Update Step Visual-Inertial Localization – Altogether: ´ System Model: Propagation of Estimate & Uncertainty based on Rigid Body model and accelerometer & gyroscope data. ´ Measurement Model: Correction based on landmark-states observation (camera-based feature detection).
- 10. Visual – Inertial SLAM Christos Papachristos, Autonomous Robots Lab, University of Nevada, Reno ´ Visual – Inertial Localization and Mapping Using a Stereo Camera ´ Reliable stereo camera model gives better landmark estimation statistics. ´ Improved 3D pose estimation. ´ Consistent stereo depth map. ´ “Dense” Reconstruction / Mapping Left Camera Right Camera
- 11. Volumetric Mapping Christos Papachristos, Autonomous Robots Lab, University of Nevada, Reno ´ Visual-Inertial Localization and Mapping & Dense Reconstruction From Dense to Volumetric Mapping Voxel-grid from PointCloud (octomap) ´ Volumetric representation of the known environment. ´ Makes distinction between occupied & free voxels based on a probabilistic hit/miss model of a depth sensor. ´ Efficient representation in memory ( octree structure, nodes store “occupied” probability ). ´ Fast node (voxel) lookup (usually hash-table based) given its 3D coordinates.
- 12. Volumetric Mapping ´ Octomap(s) Volumetric Mapping & Robotic Autonomy Ray collision checks for landmark visibility l1 l2 l3 l4 Christos Papachristos, Autonomous Robots Lab, University of Nevada, Reno Fast node-lookup benefits ray-checking ´ Ray-casting / Ray-checking is the process of checking along a 3D line segment (ray) if occupied, free, or unmapped voxels are crossed. ´ Checking if a transition from initial 3D configuration to a desired one (waypoint) will encounter an obstacle. ´ Checking if a 3D landmark lies in Line-of-Sight or “occluded”.
- 13. Path-Planning Christos Papachristos, Autonomous Robots Lab, University of Nevada, Reno ´ Core Path-planning Principles : Random Sampling Expanding Random Trees in the known (mapped) and free configuration space. ´ Only Collision-free transitions are permitted for every segment. ´ Collision-free navigation along path.
- 14. Path-Planning Christos Papachristos, Autonomous Robots Lab, University of Nevada, Reno ´ Core Path-planning Principles : Receding-Horizon Strategy ´ A finite number of path-planning moves (e.g. the first segment only) is performed. ´ Real-time feedback from mapping updates the environment knowledge. Based on this updated state, path-planning is re-evaluated. ´ The first moves is performed again, with each iteration followed by a new map update.
- 15. Autonomy & Active Perception Christos Papachristos, Autonomous Robots Lab, University of Nevada, Reno ´ The Objective: Explore a location while mapping with consistency. Given a bounded volume 𝑉" , find a collision free path 𝜎 starting at an initial configuration 𝜉%&%' ∈ Ξthat leads to identifying the free and occupied parts 𝑉*+,, " and 𝑉-.. " when being executed, such that there does not exist any collision free configuration from which any piece of 𝑉" {𝑉*+,, " , 𝑉-.. " } could be perceived. Problem 1: Volumetric Exploration Given a 𝑉1 ⊂ 𝑉" , find a collision free path 𝜎1 starting at an initial configuration 𝜉3 ∈ Ξ and ending in a configuration 𝜉*%&45 ∈ Ξ that aims to improve the robot’s localization and mapping confidence by following paths of optimized expected robot pose and tracked landmarks covariance. Problem 2: Belief Uncertainty-aware planningCombined Problem The overall problem is that of exploring an unknown bounded 3D volume 𝑉" ⊂ ℝ7 , while aiming to minimize the localization and mapping uncertainty as evaluated through a metric over the robot pose and landmarks probabilistic belief. Problem Definition
- 16. Autonomy & Active Perception Christos Papachristos, Autonomous Robots Lab, University of Nevada, Reno ´ Objective 1: Volumetric Exploration Receding-Horizon Exploration & Mapping Path-planner (rhemplanner) Two-level Path-planning paradigm: ´ Addresses the combined problem in a hierarchical approach. ´ At every iteration, a finite depth random tree is spanned. Each vertex is annotated with a collected Information Gain – a metric of how much new space is going to be explored. Planning Layer 1: Volumetric Exploration
- 17. Autonomy & Active Perception Christos Papachristos, Autonomous Robots Lab, University of Nevada, Reno ´ Objective 1: Volumetric Exploration Receding-Horizon Exploration & Mapping Path-planner (rhemplanner) ´ Tree-based exploration: At every iteration, a finite depth random tree is spanned. Each vertex is annotated with the collected Information Gain – a metric of how much new space is going to be explored. ´ Within it, evaluation regarding the path that overall leads to the highest information gain is conducted. This corresponds to the best path for the given iteration (a sequence of next- best-views as sampled). ´ Receding Horizon: For the extracted best path, only the first viewpoint is actually executed. ´ The system moves to it, map is updated, process is repeated. Executed Step
- 18. Autonomy & Active Perception Christos Papachristos, Autonomous Robots Lab, University of Nevada, Reno ´ Objective 1: Volumetric Exploration Receding-Horizon Exploration & Mapping Path-planner (rhemplanner) ´ Probabilistic Re-observation term: Maximize newly explored space and try to re-observe the parts where confidence whether they are occupied is low.
- 19. Autonomy & Active Perception Christos Papachristos, Autonomous Robots Lab, University of Nevada, Reno ´ Objective 2: Uncertainty – Aware Path-planning Receding-Horizon Exploration & Mapping Path-planner (rhemplanner) Two-level Path-planning paradigm: ´ Hierarchical structure: ´ Given an exploration 1st view- point, another 2nd layer random tree is spanned locally “around” that vertex. Each possible path leading to the end-configuration is annotated with a Belief Gain – a metric of how much the robot belief has improved / deteriorated. ´ The mechanism to propagate robot’s belief has to be established. Planning Layer 2: Uncertainty-Optimization
- 20. Autonomy & Active Perception Christos Papachristos, Autonomous Robots Lab, University of Nevada, Reno ´ Objective 2: Uncertainty – Aware Path-planning Receding-Horizon Exploration & Mapping Path-planner (rhemplanner) ´ Exploit the EKF pipeline used for SLAM to propagate belief. ´ Propagate State and Uncertainty along all paths. ´ Assume closed-loop dynamics, simulate inertial measurements. Predict Step
- 21. ´ Exploit the EKF pipeline used for SLAM to propagate belief. ´ Propagate State and Uncertainty along all paths. ´ Use octomap representation to predict landmark visibility / occlusion. ´ Perform virtual updates for all landmarks expected to be seen. Autonomy & Active Perception Christos Papachristos, Autonomous Robots Lab, University of Nevada, Reno ´ Objective 2: Uncertainty – Aware Path-planning Receding-Horizon Exploration & Mapping Path-planner (rhemplanner) Update Step l1 l2 l3 l4
- 22. Autonomy & Active Perception Christos Papachristos, Autonomous Robots Lab, University of Nevada, Reno ´ Objective 2: Uncertainty – Aware Path-planning Receding-Horizon Exploration & Mapping Path-planner (rhemplanner) ´ Finally, compute the propagated covariance matrix for every path: ´ The D-optimality metric is a measure of how “small” the corresponding ellipsoid is: ´ Choose the path that minimizes the D-optimality metric – i.e. minimizes the Uncertainty on arrival. ´ This path might turn out to be the original straight segment – Optimum is only selected out of a finite number of randomly sampled trajectories.
- 23. Autonomy & Active Perception Christos Papachristos, Autonomous Robots Lab, University of Nevada, Reno ´ Receding-Horizon Uncertainty-aware Exploration & Mapping Path-planner (rhemplanner)
- 24. Autonomy & Active Perception Christos Papachristos, Autonomous Robots Lab, University of Nevada, Reno ´ Receding-Horizon Uncertainty-aware Exploration & Mapping Path-planner (rhemplanner)
- 25. Degraded Visual Environments Christos Papachristos, Autonomous Robots Lab, University of Nevada, Reno Degraded Visual Environments
- 26. Degraded Visual Environments Christos Papachristos, Autonomous Robots Lab, University of Nevada, Reno ´ Uncertainty-Aware Exploration & Mapping Tying back to the original motivation ´ Particularly for DVEs, pure volumetric exploration is not sufficient. ´ The selected viewpoints and their sequence will heavily influence the localization of the robot. ´ A 1st generation of Multi-Modal sensor fusion for GPS-denied localization and mapping in DVEs. NIR Cameras & IR LED(s) Time-of-Flight 3D Camera IMU
- 27. Degraded Visual Environments Christos Papachristos, Autonomous Robots Lab, University of Nevada, Reno ´ Uncertainty-Aware Exploration & Mapping Tying back to the original motivation ´ A 1st generation of Multi-Modal sensor fusion for GPS-denied localization and mapping in DVEs. ´ Same Active Perception approach for reliable autonomy subject to the challenges of DVEs. NIR Cameras & IR LED(s) Time-of-Flight 3D Camera IMU
- 28. Degraded Visual Environments Christos Papachristos, Autonomous Robots Lab, University of Nevada, Reno ´ Multi-Modal Mapping Unit Tightly-integrated Multi-Modal sensor (featuring Hardware-synchronization, Expansions, …) ´ Inertial Sensors (accelerometers, gyroscopes) ´ Vision (synchronized with flashing LEDs) ´ Depth Cameras (Time-of-Flight) ´ GPS integration-ready ´ Support for multi-Camera setups
- 29. Degraded Visual Environments Christos Papachristos, Autonomous Robots Lab, University of Nevada, Reno ´ Field Experiments The “real” test – A driving force for improvement ´ Autonomous Robotic Navigation, Exploration, Inspection and Mapping in GPS-denied DVEs. ´ Technology developed in-house. Demonstrated in Field Experiments. ´ The 1st generation of Multi-Modal sensor fusion: ´ Visual-Inertial / Depth – odometry loosely-coupled via EKF.
- 30. Degraded Visual Environments Christos Papachristos, Autonomous Robots Lab, University of Nevada, Reno ´ Multi-Modal Mapping Unit Tightly-integrated Multi-Modal sensor (featuring Hardware-synchronization, Expansions, …)
- 31. Nuclearized Robotics Christos Papachristos, Autonomous Robots Lab, University of Nevada, Reno ´ Field Experiments The “real” test – A driving force for improvement Nuclearized Robots
- 32. Nuclearized Robotics Christos Papachristos, Autonomous Robots Lab, University of Nevada, Reno ´ Motivation The Nuclear Cleanup Mission ´ Nuclear facility decommissioning. ´ Soil and water cleanup. ´ Liquid radioactive waste processing & disposition. ´ Solid radioactive waste treatment, storage and disposal. ´ Nuclear materials and spent nuclear fuel management. Figure from DOE – EM
- 33. Nuclearized Robotics Christos Papachristos, Autonomous Robots Lab, University of Nevada, Reno ´ Motivation DOE – EM Facilities Characterization ´ Flying & roving robots to help characterize DOE – EM facilities. Goal set for demonstration of developed technologies in nuclear analog facilities of DOE – EM. ´ Identification and Semantic Classification of tanks, pipes, and other important structures to intelligently focus the robot exploration and inspection tasks. ´ Radiation, Chemical, and Heat spatial maps are fused with 3D models of the environment. ´ Integrated Planning & Multi-Modal perception for comprehensive mapping of nuclear facilities, Active Perception to improve (while benefiting from) radiation, chemical, heat estimation.
- 34. Nuclearized Robotics Christos Papachristos, Autonomous Robots Lab, University of Nevada, Reno ´ Robotic Detection of Ionizing Radiation Target Platform: Autonomous Micro Aerial Vehicles ´ Detection of Gamma Radiation – common need and requirement of the decommissioning efforts. Key technologies: ´ Miniature CeBr3, CsI, NaI scintillators with built-in temperature compensated bias generator and pre-amplifier alongside a Silicon Photomultiplier tube. ´ Miniature solid-state low voltage detectors. ´ Gamma cameras (heavy for aerial robots).
- 35. Nuclearized Robotics Christos Papachristos, Autonomous Robots Lab, University of Nevada, Reno ´ Robotic Detection of Ionizing Radiation Gamma Radiation Detection ´ Sensor calibration is required too: ´ Radiation detectors can present significant polarity characteristics. Calibration requires exhaustive tests for different sensor orientations. ´ Differential installment of two gamma detectors will potentially enhance source localization. ´ Integration of spectroscopy through relevant algorithms and a multi-channel analyzer. Calibration against known characterized sources allows estimation of detected gamma photon energy. ´ Estimation / Characterization of types of sources contributing to a region’s radioactivity. ´ “Hunt” for specific expected radioactive source types.
- 36. Nuclearized Robotics Christos Papachristos, Autonomous Robots Lab, University of Nevada, Reno ´ Robotic Detection of Radiation Other Radiation types: ´ Neutron Detection is relevant with homeland security and industrial monitoring (e.g. detection of nuclear weapons, personnel monitoring, water content in soil). ´ Gas-filled (e.g. He-3), Scintillation, Solid-State. ´ Alpha Detection is very challenging. Alpha particles are the heaviest and highly charged, they quickly give up their energy to any medium they pass. ´ Special detection methods are required, gas- filled detector [ZnS(Ag)] combined with Aluminized Mylar film. ´ Requires contact & robotic manipulation.
- 37. Nuclearized Robotics Christos Papachristos, Autonomous Robots Lab, University of Nevada, Reno ´ Robotic Detection of Radiation Autonomous Exploration & Mapping Aerial Robot for DVE and Nuclear Sites Gamma Detector Multi-Modal Perception Unit Scintillation Detector(s) Solid-State Detector(s)
- 38. Nuclearized Robotics Christos Papachristos, Autonomous Robots Lab, University of Nevada, Reno ´ Robotic Detection of Radiation Autonomous Exploration & Mapping Aerial Robot for DVE and Nuclear Sites
- 40. Further Research & Applications Christos Papachristos, Autonomous Robots Lab, University of Nevada, Reno ´ Robotic Detection of Change ´ Perform real-time change detection. ´ Expand to efficient 3D-to-3D change detection approaches. ´ Incorporate change-driven “curiosity” in planning algorithms.
- 41. Further Research & Applications Christos Papachristos, Autonomous Robots Lab, University of Nevada, Reno ´ Curious Robots ´ Employ a human-paradigm for interest: Collect more meaningful data without necessarily having any explicit mission objective. ´ Ability to focus perceptual attention towards regions that have “Visual Saliency”
- 42. Further Research & Applications Christos Papachristos, Autonomous Robots Lab, University of Nevada, Reno ´ Augmented – Reality Robotics ´ Provide Real-Time feedback of data annotated with Mission-Relevant information. ´ Take the actual flying away from the human, but still maintain the ability to redirect the robot’s attention towards areas of interest.
- 43. Thank you! Please ask your question!