Iterative Visual Recognition for Learning Based Randomized Bin-picking
1 like3,215 views
The document presents a method for iterative visual recognition in randomized bin-picking to automate assembly processes. It discusses challenges such as identifying multiple object poses and occlusions using a wrist-mounted 3D vision sensor. The proposed approach improves success rates and utilizes prior visual information to enhance object pose estimation.
Iterative Visual Recognition for Learning Based Randomized Bin-picking
- 1. 1 (C) Sonoyama ENGINEERING SCIENCE OSAKA UNIVERSITY Iterative Visual Recognition for Learning Based Randomized Bin-Picking Kensuke Harada1,2, Weiwei Wan2, Tokuo Tsuji3 Kohei Kikuchi4, Kazuyuki Nagata2, and Hiromu Onda2 1. Osaka University 2. National Inst. of Advanced Industrial Science and Technology 3. Kanazawa University 4. Toyota Motors Co. Ltd. Int. Symposium on Experimental Robotics (ISER) 2016
- 2. 2 (C) Sonoyama ENGINEERING SCIENCE OSAKA UNIVERSITY Why Randomized Bin-Picking? •Parts Automatically Supplied to an Assembly Cell •First Step to Automate the Assembly Process Parts Production Company Randomized Bin-Picking Assembly Cell Int. Symposium on Experimental Robotics (ISER) 2016
- 3. 3 (C) Sonoyama ENGINEERING SCIENCE OSAKA UNIVERSITY Related Works • 2D Grasp – Morales et al. (’01) , Fryndenal et al.(‘98), Domae et al. (‘14) • Grasp Planning – Dupis et al. (‘08), Harada et al. (‘14) • Learning Based Method – Harada et al. (CASE ’16) – Levine et al. (ISER ’16) Int. Symposium on Experimental Robotics (ISER) 2016
- 4. 4 (C) Sonoyama ENGINEERING SCIENCE OSAKA UNIVERSITY Overview of Learning Based Randomized Bin-Picking Swept volume includes point cloud of neighboring objects Finger will collide with neighboring objects Classify success/failure cases based on distribution of point cloud included in the swept volume Int. Symposium on Experimental Robotics (ISER) 2016 IEEE CASE 2016
- 5. 5 (C) Sonoyama ENGINEERING SCIENCE OSAKA UNIVERSITY Learning Method Overview Feature Vector Bins to store point cloud Feature vector Discretized Point Cloud Distribution in the Swept Volume Int. Symposium on Experimental Robotics (ISER) 2016 Random Forest (RF) is used
- 6. 6 (C) Sonoyama ENGINEERING SCIENCE OSAKA UNIVERSITY Int. Symposium on Experimental Robotics (ISER) 2016 Success rate is more than 90% within our trial Excluding the case where visual recognition make mistake Success rate changes depending on the density of objects
- 7. 7 (C) Sonoyama ENGINEERING SCIENCE OSAKA UNIVERSITY Two Problems in Bin-Picking:#1 Identification of Multiple Objects’ Pose (Iteration of Heavy Computation) Remember Previous Identification Result Identify Object Pose based on Difference of Image Objects Configuration Just Partially Different from the Configuration in the Previous Picking Trial Previous Method Proposed Method Int. Symposium on Experimental Robotics (ISER) 2016
- 8. 8 (C) Sonoyama ENGINEERING SCIENCE OSAKA UNIVERSITY Two Problems in Bin-Picking:#2 Using Camera Attached at Wrist Realizing Maximum Visibility of the Pile Picking Fails due to Occlusion Previous Method Proposed Method Int. Symposium on Experimental Robotics (ISER) 2016
- 9. 9 (C) Sonoyama ENGINEERING SCIENCE OSAKA UNIVERSITY In this research • We propose a method for iteratively identifying the objects’ pose to solve the two problems. • We use 3D vision sensor attached at the wrist.
- 10. 10 (C) Sonoyama ENGINEERING SCIENCE OSAKA UNIVERSITY Definition of Sensor Pose Candidate • Regular Polyhedron • Sensor Located at Line Perpendicular to a Face • Discretized Distance: l Int. Symposium on Experimental Robotics (ISER) 2016
- 11. 11 (C) Sonoyama ENGINEERING SCIENCE OSAKA UNIVERSITY Determination of Sensor Pose 1st Trial • Maximizing Visibility • IK Solvable • Collision Free Int. Symposium on Experimental Robotics (ISER) 2016
- 12. 12 (C) Sonoyama ENGINEERING SCIENCE OSAKA UNIVERSITY Determination of Sensor Pose After 2nd Trial • Use Occupancy Grid Map Assumption • Object Coniguration : Small Change After Picking Trial Int. Symposium on Experimental Robotics (ISER) 2016
- 13. 13 (C) Sonoyama ENGINEERING SCIENCE OSAKA UNIVERSITY Determination of Sensor Pose After 2nd Trial • Use Occupancy Grid Map • Maximizing Number of Occupied Grids Int. Symposium on Experimental Robotics (ISER) 2016
- 14. 14 (C) Sonoyama ENGINEERING SCIENCE OSAKA UNIVERSITY Object Pose Detection Segmentation / Identification Point Cloud in Current PickingPreviously Captured Point Cloud Picking Int. Symposium on Experimental Robotics (ISER) 2016
- 15. 15 (C) Sonoyama ENGINEERING SCIENCE OSAKA UNIVERSITY Object Pose Detection Difference between Two Points Compare Two Point Clouds Small Difference Merge Do not Merge Int. Symposium on Experimental Robotics (ISER) 2016 Large Difference
- 16. 16 (C) Sonoyama ENGINEERING SCIENCE OSAKA UNIVERSITY Object Pose Detection Identify Objects’ Pose Segment with Large Shape Difference Int. Symposium on Experimental Robotics (ISER) 2016
- 17. 17 (C) Sonoyama ENGINEERING SCIENCE OSAKA UNIVERSITY Experimental Setup • Segmentation using Euclidian Distance • CVFH + CRF Estimation • ICP Algorithm
- 18. 18 (C) Sonoyama ENGINEERING SCIENCE OSAKA UNIVERSITY 1st 2nd 3rd
- 19. 19 (C) Sonoyama ENGINEERING SCIENCE OSAKA UNIVERSITY Results • Calculation Time Improved According to the Number of Estimated Objects • Increase of Success Rate （Under Investigation） • Segmentation Sometimes Fails Int. Symposium on Experimental Robotics (ISER) 2016
- 20. 20 (C) Sonoyama ENGINEERING SCIENCE OSAKA UNIVERSITY Conclusions • We propose a method for iteratively identifying the objects’ pose to solve the two problems. • We use previously captured visual information to estimate the objects’ pose. • We determined the sensor pose to maximize the visibility of the pile.