Challenges in Perception for Learning, Cognition and Control Approaches
- 1. Challenges in Perception for Learning, Cognition and Control Approaches Darius Burschka Machine Vision Perception Group (MVP) Technische Universität München, Germany
- 2. Darius Burschka – Machine Vision and Perception Group (TUM) Structure of a CNN for Robotics Applications Raw sensor information (depth image from an RGB-D sensor) is taken as input to connect with the robot control command directly Ideal case: learn the coupling between perception and control commands from demonstrations
- 3. Darius Burschka – Machine Vision and Perception Group (TUM) Curse of Complexity in Direct Processing In theory, one could use all the extracted features with a classifier such as a softmax classifier, but this can be computationally challenging. For instance images of size 96x96 pixels with 400 learned features over 8x8 inputs. Each convolution results in an output of size (96−8+1)∗(96−8+1)=7921, and since we have 400 features, this results in a vector of 892∗400= 3,168,400 features per example. Learning a classifier with inputs having 3+ million features can be unwieldy, and can also be prone to over-fitting.
- 4. Darius Burschka – Machine Vision and Perception Group (TUM) Deep-Learning for Perception in Robotics What do all these tasks have in common? Source: L. Tai, “Deep-Learning in Mobile Robotics…”
- 5. Darius Burschka – Machine Vision and Perception Group (TUM) Learning Approaches vs. Conventional Tools Image source: Berkley Grouping/Segmentation, Labeling, Identification because differences in the properties of pixels to entire image are considered due to the structure of CNNs Problem: not clear, what the processing is based on! ? Hand-designed features help to estimate metric distances between them using calibration parameters
- 6. Darius Burschka – Machine Vision and Perception Group (TUM) Deep-Learning for Control in Robotics Source: L. Tai, “Deep-Learning in Mobile Robotics…”
- 7. Darius Burschka – Machine Vision and Perception Group (TUM) Can we avoid Metric Information in Representation of Environments?
- 8. Darius Burschka – Machine Vision and Perception Group (TUM) Fast Uncalibrated Monocular Estimation of Independent Motion Components
- 9. Representation of the Environment in Collision Space
- 10. Darius Burschka – Machine Vision and Perception Group (TUM) Examples of non-metric Control (RoboMobil)
- 11. Darius Burschka – Machine Vision and Perception Group (TUM) Examples of non-metric Control (RoboMobil)
- 12. Darius Burschka – Machine Vision and Perception Group (TUM) Conclusions • Image labeling and image retrieval (indexing) represent a large application field for perception and image processing (Google, Apple, etc.), but robotic requires often a metric mapping from the sensor onto control values • Work on alternatives for coupling between the sensor and the actuator should be one of the perception goals • Currently used data representations need to be re-defined for learning applications.