![Uncalibrated Image-based Control of Robots Azad Shademan PhD Candidate Computing Science, University of Alberta Edmonton, Alberta, CANADA [email_address]](https://image.slidesharecdn.com/shademan20081105-1226979894177972-9/85/Knowdiff-visiting-lecturer-140-Azad-Shademan-Uncalibrated-Image-Based-Robotic-Visual-Servoing-1-320.jpg)
![Visual-motor Equation x 1 x 2 x 3 x 4 q=[q 1 … q 6 ] Visual-Motor Equation This Jacobian is important for motion control.](https://image.slidesharecdn.com/shademan20081105-1226979894177972-9/85/Knowdiff-visiting-lecturer-140-Azad-Shademan-Uncalibrated-Image-Based-Robotic-Visual-Servoing-12-320.jpg)
![Conclusions Reviewed position-based and image-based visual servoing schemes. Presented two global methods to learn the visual-motor function. KNN suffers from the bias in local estimations. LLS (global) works better than the KNN (global) and local updates. The Jacobian of more complex visual tasks can also be learned using LLS method. [email_address]](https://image.slidesharecdn.com/shademan20081105-1226979894177972-9/85/Knowdiff-visiting-lecturer-140-Azad-Shademan-Uncalibrated-Image-Based-Robotic-Visual-Servoing-39-320.jpg)
Knowdiff visiting lecturer 140 (Azad Shademan): Uncalibrated Image-Based Robotic Visual Servoing
- 1. Uncalibrated Image-based Control of Robots Azad Shademan PhD Candidate Computing Science, University of Alberta Edmonton, Alberta, CANADA [email_address]
- 2. Vision-Based Control A A A B B B current desired Left Image Right Image
- 3. Vision-Based Control Left Image Right Image B B B
- 4. Where is the camera located? Eye-to-Hand e.g.,hand/eye coordination Eye-in-Hand
- 5. Vision-Based Control Feedback from visual sensor (camera) to control a robot Also called visual servoing Visual servoing is the task of minimizing a visually specified objective by giving appropriate control commands to a robot Is it any difficult ? Images are 2D, the robot workspace is 3D 2D data 3D geometry
- 6. Visual Servo Control law Position-Based: Robust and real-time pose estimation + robot’s world-space (Cartesian) controller Image-Based: Desired image features seen from camera Control law entirely based on image features Hybrid: Depth information is added to image data to increase stability
- 7. Position-Based Robust and real-time relative pose estimation E xtended K alman F ilter to solve the nonlinear relative pose equations. Cons: EKF is not the optimal estimator. Performance and the convergence of pose estimates are highly sensitive to EKF parameters.
- 8. Position-Based Desired pose Estimated pose
- 9. Position-Based Measurement noise State variable Process noise yaw pitch roll Measurement equation (projection) is nonlinear and must be linearized.
- 10. K x k-1,k-1 z k R k P k,k P k,k-1 C k Q k-1 x k,k x k,k-1 P k-1,k-1 Kalman Gain Measurement noise covariance A priori error cov. @ k-1 Process noise covariance Initial/previous state Linearization Measurement State update State prediction Error cov. prediction Error cov. update
- 11. Image-Based Desired Image feature Extracted image feature
- 12. Visual-motor Equation x 1 x 2 x 3 x 4 q=[q 1 … q 6 ] Visual-Motor Equation This Jacobian is important for motion control.
- 13. Visual-motor Jacobian Image space velocity Joint space velocity A A B B
- 14. Image-Based Control Law Measure the error in image space Calculate/Estimate the inverse Jacobian Update new joint values
- 15. Image-Based Control Law Desired Image feature Extracted image feature
- 16. Jacobian calculation Analytic form available if model is known. Known model Calibrated Must be estimated if model is not known Unknown model Uncalibrated
- 17. Calibrated: Interaction Matrix Analytic form depends on depth estimates. Camera/Robot transform required. No flexibility. Camera Velocity
- 18. Uncalibrated: Visual-Motor Jacobian A naïve method: Orthogonal projections
- 19. Uncalibrated: Visual-Motor Jacobian A naïve method: Orthogonal projections
- 20. Uncalibrated: Visual-Motor Jacobian A naïve method: Orthogonal projections …
- 21. Uncalibrated: Visual-Motor Jacobian A popular local estimator: Recursive secant method (Broyden update):
- 22. Relaxed model assumptions Traditionally: Local methods No global planning (-) Difficult to show asymptotic stability condition is ensured (-) Main problem of traditional methods is the locality. Calibrated vs. Uncalibrated Model derived analytically Global asymptotic stability (+) Optimal planning is possible (+) A lot of prior knowledge on the model (-) Global Model Estimation ( Research result ) Optimal trajectory planning (+) Global stability guarantee (+)
- 23. Synopsis of Global Visual Servoing Model Estimation (Uncalibrated) Visual-Motor Kinematics Model Global Model Extending Linear Estimation (Visual-Motor Jacobian) to Nonlinear Estimation Our contributions: K-NN Regression-Based Estimation Locally Least Squares Estimation
- 24. Local vs. Global Key idea: using only the previous estimation to estimate the Jacobian RLS with forgetting factor Hosoda and Asada ’94 1 st Rank Broyden update: Jägersand et al. ’97 Exploratory motion: Sutanto et al. ‘98 Quasi-Newton Jacobian estimation of moving object: Piepmeier et al. ‘04 Key idea: using all of the interaction history to estimate the Jacobian Globally-Stable controller design Optimal path planning Local methods don’t!
- 25. K-NN Regression-based Method q 1 q 2 3 NN q 1 q 2 x 1 ?
- 26. (X,q) L ocally L east S quares Method q 1 q 2 x 1 ? K-neighbour(q)
- 27. Eye-to-hand Experiments Puma 560 Stereo vision Features: projection of the end-effector’s position on image planes (4-dim) 3 DOF for control
- 28. Measuring the Estimation Error
- 30. Visual Task Specification Image Features: Geometric primitives (points, lines, etc.) Higher order image moments Shape parameters … Visual Tasks Point-to-point (point alignment) Point-to-line (colinearity) Point-to-plane (coplanarity) …
- 31. Eye-in-hand
- 36. Mean-Squared-Error Task 1 Task 2
- 37. Task Errors
- 39. Conclusions Reviewed position-based and image-based visual servoing schemes. Presented two global methods to learn the visual-motor function. KNN suffers from the bias in local estimations. LLS (global) works better than the KNN (global) and local updates. The Jacobian of more complex visual tasks can also be learned using LLS method. [email_address]
- 40. Thank you!
- 41. Visual Ambiguity: Single Camera