NLP for Robotics
- 1. Processing Natural Language in Robotics Applications01001000 01100101 01101100 01101100 01101111 00100000 01010111 01101111 01110010 01101100 01100100 00100001 00100000Alan Shen
- 2. Related workWubble Voice Command Demo (Gazebo Simulation)Arizona Robotics Research Group - University of Arizonahttp://www.youtube.com/watch?v=atB9mh6u1Nghttp://ua-ros-pkg.googlecode.com
- 3. Related workHumanoid robot speech recognition and object trackinghttp://www.youtube.com/watch?v=0jW9LgtiiM8
- 4. GoalsImplement sentence recognition in text form (console/gui)Provided some basis for speech processing “Pick up the blue cup”Pick up the blue cup
- 5. Existing toolsPossible python library implementationOpen RaveText ProcessingSpeech ProcessingRobot ActionPrairie Dog LibrariesPython Tagging Libraries (eg: NLTK)
- 6. Parsing ordersAmbiguous interpretations (robot command sentences may suffer less from this) “Robot, make her duck” “Get the elevator”Mapping verbs to targets: “Follow that person” “Get in the elevator” “Pick up that object”
- 7. Parts of speech taggingGiven a word, what is its part of speech?𝐴𝑟𝑔𝑀𝑎𝑥: 𝑃(𝑇𝑎𝑔𝑠|𝑊𝑜𝑟𝑑𝑠)
- 8. Parts of speech tagging𝐴𝑟𝑔𝑀𝑎𝑥: 𝑃(𝑇𝑎𝑔𝑠|𝑊𝑜𝑟𝑑𝑠)Given a known corpus, maybe it’s easier to predict a word given a tag:Bayes: 𝑃𝐴𝐵=𝑃𝐵𝐴𝑃(𝐴)𝑃(𝐵)
- 9. Parts of speech tagging𝐴𝑟𝑔𝑀𝑎𝑥: 𝑃(𝑇𝑎𝑔𝑠|𝑊𝑜𝑟𝑑𝑠)Bayes: 𝑃𝐴𝐵=𝑃𝐵𝐴𝑃(𝐴)𝑃(𝐵)𝐴𝑟𝑔𝑀𝑎𝑥: 𝑃𝑊𝑜𝑟𝑑𝑠𝑇𝑎𝑔𝑠𝑃(𝑇𝑎𝑔𝑠)𝑃(𝑊𝑜𝑟𝑑𝑠)Don’t need to normalize…
- 10. Parts of speech tagging𝐴𝑟𝑔𝑀𝑎𝑥: 𝑃(𝑇𝑎𝑔𝑠|𝑊𝑜𝑟𝑑𝑠)Bayes: 𝑃𝐴𝐵=𝑃𝐵𝐴𝑃(𝐴)𝑃(𝐵)𝐴𝑟𝑔𝑀𝑎𝑥:𝑃𝑊𝑜𝑟𝑑𝑠𝑇𝑎𝑔𝑠𝑃𝑇𝑎𝑔𝑠
- 11. Calculating ArgMax and likely tagsSpeech and Language Processing - Jurafsky and Martin
- 12. Further workOnce actions and their targets are mapped, generate actionsDirectly executing OpenRave commands in sequence?Commands ignored if OpenRave is busyDetermine method of controllingPrairieDog movementNeed to ensure that text command interfaceis compatible with both arm and wheel controlsPopulate objects in robot’s word dictionaryEg: block, door, elevator, person