Closing the gap between neuroscience and machine learning
0 likes387 views
The document discusses the intersection of neuroscience and machine learning, emphasizing the differences between human and computer intelligence, particularly in terms of hardware and software. It introduces 'sensenet', an open-source simulator aimed at bridging this gap through touch and sensorimotor simulations, with components to aid in developing new AI algorithms. The project's goal is to engage both neuroscience and computer science communities in the study of touch and sensorimotor research.
Closing the gap between neuroscience and machine learning
- 1. Closing the gap between neuroscience and machine learning Jason Toy @jtoy
- 2. Neuroscience • Scientific study of the nervous system • Why? How do humans and living beings function and learn. What are we? • Many sub disciplines: cognitive science, computational neuroscience, neurochemistry, neuro anatomy, clinical neuroscience, etc
- 5. Machine learning • study of computer science that often uses statistical methods to give the computers the ability to learn • Silo'd away • Why? Can we make computers intelligent like humans using current computer science paradigms
- 7. Differences between computers and human intelligence • Hardware • Software • Need new algorithms? • New hardware? • What is missing?
- 8. Differences between computers and human intelligence - hardware • Programs in computer vs embodied cognitions • Digital vs analog? Hybrid models? • Parallel computation vs sequential?
- 9. Differences between computers and human intelligence - software • Single objective function vs dynamic goals • text/images vs multimodal sensory • ML models learn strong features vs casual models for explaining and understanding
- 10. Reinforcement learning • Multi arm bandits - slot machines • Marketing and landing page optimization • Robots
- 14. sensorimotor vs RL ?
- 15. Sensenet: My contribution & attempt at closing the gap • Sensenet: An open source touch and sensorimotor simulator for neuroscience and machine learning research • https://github.com/jtoy/sensenet
- 16. Why sensorimotor? • Most ML algorithms are passive and static: CNNs, classification • Single objective function models in classification/RL • vision vs embodied sense • Next gen algorithms in the sensorimotor system - touch / motion / mapping / location / rotations
- 17. Goal of SenseNet Project • Aid in discovery of new AI algorithms through touch/sensorimotor simulations • Get computer science and neuroscience communities interested in touch and sensorimotor research
- 20. SenseNet components • 4 main components: • Physics and Touch Simulator • Simple extendable API for RL, sensorimotor, and touch abstractions • Standardized benchmarks and reference • Dataset of 3D objects dataset for manipulation
- 21. Physics and Touch Simulator • SenseNet provides different physics and touch environments as a reference • Hard and soft body collision detection • Tunable and tweakable physics and touch apparatuses • Current touch sensor acts as edge detector (similar to merkel cells)
- 22. Bullet Physics
- 24. Get to human like sensors
- 25. SenseNet API • Physics in C++, everything else in python • Researchers interact with the API in python • Reinforcement learning API / OpenAI gym comp • Developing API to add touch sensors and skin • Can use standard robotics tools to add agents (URDF/stl)
- 26. Environments / Agents • W Full Hand Index finger only Touch Wand ~7 agent types currently
- 27. SenseNet API & Touch Agents • Current API: reinforcement learning API (gym) • In development: • APIs to enable multiple reward functions. • Skin API, sensorimotor API • Embodied Agents: Hand, single finger, touch wand, disembodied touch agents, custom agents
- 29. SenseNet Dataset
- 30. SenseNet Dataset
- 32. Sensenet experiments • Can we train a simulated hand to touch and classify objects • Can we train a simulated hand to touch and classify textures • Can we train a simulated hand to touch and manipulate objects
- 34. Summary • Computer .......... humans • deep learning is awesome, but ........ • " We urge researchers in neuroscience and AI to find a common language, allowing a free flow of knowledge that will allow continued progress" - Demis Hassabis, Founder of DeepMind
- 35. Jason Toy @jtoy jtoy@jtoy.net Want to contribute? Questions? Feedback http://github.com/jtoy/sensenet