![Evaluation on KITTI odometry estimation dataset
Geiger et. al, “Vision meets
Robotics: The KITTI dataset”,
IJRR2013
Odometry estimation algorithms
- Keyframe-based GICP odometry
- LeGO-LOAM [Tixiao, IROS2018]
- SuMa [Behley, RSS2018]
Three algorithms with totally different architectures
Parameter settings
(1) Manually tuned (2) Fixed param (3) Adaptive param
256 offline SMBO trials
For seq. 00
Training/validation set
Seq. 00-05 : for training
Seq. 06-10 : for validation](https://image.slidesharecdn.com/iros2021presentation-231115084931-6d8bec3e/85/Adaptive-Hyper-Parameter-Tuning-for-Black-box-LiDAR-Odometry-IROS2021-12-320.jpg)
Adaptive Hyper-Parameter Tuning for Black-box LiDAR Odometry [IROS2021]
- 1. Adaptive Hyper-Parameter Tuning for Black-box LiDAR Odometry Kenji Koide, Masashi Yokozuka, Shuji Oishi, and Atsuhiko Banno National Institute of Advanced Industrial Science and Technology (AIST), Japan
- 2. Odometry Estimation LiDAR Odometry Visual Odometry Engel et al., Direct Sparse Odometry Pan et al., MULLS: Versatile LiDAR SLAM via Multi-metric Linear Least Square
- 3. Tuning is important Odometry estimation/SLAM frameworks involve many hyper-parameters (e.g., downsample resolution, map resolution, keyframe interval...) Many parameters need to be tuned depending on the sensor and environment (e.g., Indoor/Outdoor, Mechanical Rotating/Solid-State LiDAR) w/o parameter tuning Estimation quality largely depends on the choice of the parameters
- 4. Tuning is difficult https://google-cartographer-ros.readthedocs.io/en/latest/tuning.html Google Cartographer Tuning Guide says: "Tuning Cartographer is unfortunately really difficult. The system has many parameters many of which affect each other." MULLS, SOTA LiDAR SLAM framework, involves over 80 params It's well documented, but you still need to understand in detail how it works https://github.com/YuePanEdward/MULLS Some other frameworks don't even provide documentation... Odometry estimation methods are surprisingly complex, parameter tuning is difficult
- 5. Automatic and adaptive parameter selection for black-box LiDAR odometry Indoor Outdoor Forest Adaptive Parameter Selection Environment descriptor Param Set A Param Set B Param Set C LiDAR Odometry Accuracy improvement by parameter selection No knowledge on the inner working Data-driven meta-algorithm as a potential improvement for any odometry estimation methods
- 6. Data-driven black-box LiDAR odometry analysis
- 7. Offline parameter-error function modeling Surrogate function for error prediction Params Env. descriptor Odometry error Data-driven function modeling 1. Sample a random parameter set 2. Run LiDAR odometry algorithm 3. For each sub-trajectory: • Extract an environment descriptor • Evaluate the odometry error (RTE) 4. Repeat 1~3 5. Fit a KNN regressor s.t. Sequential Model-based Optimization SMBO finds the param that maximizes the expected improvement (EI):
- 8. Environment descriptor NDT voxel histogram-based descriptor 1. Calc normal distribution voxels M. Magnusson et. al, “Appearance-based loop detection from 3D laser data using the normal distributions transform,” ICRA2009 3. Create histogram and apply PCA (N=10) The framework is agnostic to the descriptor; other hand-crafted as well as learned features can be used 2. Classify voxels into linear/planar/sphere 𝑒𝑖𝑔 Σ = 𝜆1, 𝜆2, 𝜆3 𝜆1 > 𝜆2 > 𝜆3 𝑁0 𝐿 , 𝑁0 𝑃 , 𝑁0 𝑆 𝑁1 𝐿 , 𝑁1 𝑃 , 𝑁1 𝑆 𝑁2 𝐿 , 𝑁2 𝑃 , 𝑁2 𝑆
- 9. Online parameter selection Params Env. descriptor Odometry error Surrogate function (KNN regressor) Best parameter set for the current environment 1. Extract the descriptor for the current input cloud 2. Find the parameter set that minimizes the predicted error 𝑆 is nonlinear and non-convex run SMBO on 𝑺 Parameter selection is performed every second ① ② ③
- 10. Simple toy example Simulated environment (A) cave, (B) open space, (C) outdoor street Odometry estimation algorithm Keyframe-based NDT odometry with 2 params - NDT resolution - Keyframe interval Need to be tuned depending on the environment NDT resolution Keyframe interval Large Small Better convergence Better accuracy Small odometry drift Better stability Parameter Accuracy vs stability trade-off Parameter settings (1) Manually tuned (2) Fixed param (3) Adaptive param 256 offline SMBO trials
- 11. Simple toy example Parameters are selected depending on the environment without detailed knowledge of the algorithm A meta tuning algorithm that can potentially improve the accuracy of any odometry estimation methods
- 12. Evaluation on KITTI odometry estimation dataset Geiger et. al, “Vision meets Robotics: The KITTI dataset”, IJRR2013 Odometry estimation algorithms - Keyframe-based GICP odometry - LeGO-LOAM [Tixiao, IROS2018] - SuMa [Behley, RSS2018] Three algorithms with totally different architectures Parameter settings (1) Manually tuned (2) Fixed param (3) Adaptive param 256 offline SMBO trials For seq. 00 Training/validation set Seq. 00-05 : for training Seq. 06-10 : for validation
- 13. Sampled parameters and corresponding errors of GICP Point location: sampled parameter set Point color: odometry estimation error Different sequences require different parameters Max corresponding distance Keyframe interval - Seq. 00 requires a large max correspondence distance to prevent estimation corruption
- 14. Sampled parameters and corresponding errors of GICP Point location: sampled parameter set Point color: odometry estimation error Different sequences require different parameters - Seq. 00 requires a large max correspondence distance to prevent estimation corruption - The best keyframe interval largely varies depending on the environment Max corresponding distance Keyframe interval
- 15. Sampled parameters and corresponding errors of GICP Point location: sampled parameter set Point color: odometry estimation error Different sequences require different parameters - Seq. 00 requires a large max correspondence distance to prevent estimation corruption - The best keyframe interval largely varies depending on the environment Max corresponding distance Keyframe interval There is no parameter set that works well for all the seqs A conservative param for seq. 00 Deteriorated accuracy Param set for another seq Estimation corruption Params must be adaptively tuned depending on the environment
- 16. Evaluation on KITTI odometry estimation dataset Fixed parameter set : Improved accuracy on the training set Deteriorated accuracy on the test set Adaptive parameter set : Improved accuracy on both the training and test sets
- 17. Conclusions • An adaptive parameter tuning framework for black-box LiDAR odometry is proposed • The proposed framework uses a data-driven surrogate function modeling for error prediction • Offline parameter sampling and online parameter selection are efficiently done with SMBO (Sequential Model-based Optimization) • The proposed framework successfully improved the accuracy of different algorithms in a practical situation