Recognizing Locations on Objects by Marcus Lewis
1 like1,225 views
The document discusses a model for recognizing locations relative to objects, inspired by the principles of grid cells found in the brain. It proposes a solution consisting of three steps: recalling past sensory features, normalizing those features, and voting on the body's location relative to the object. The model highlights how the brain processes and organizes spatial information to achieve location recognition.
Recognizing Locations on Objects by Marcus Lewis
- 1. Recognizing locations on objects Marcus Lewis — November 2017 mlewis@numenta.com
- 2. Agenda • The problem, in context: Detecting locations relative to objects • Inspiration: Principles of grid cells • A solution: Recall, Normalize, Vote
- 3. Part 1 The problem, in context: Detecting locations relative to objects
- 4. Consider the inputs to the brain. Sensors move. The brain is very good at making sense of sensorimotor sequences. Sensors are often independent. Familiar objects often cause novel combinations of sensory features. The body is an array of sensors. With both vision and touch, the brain receives an array of sensory features.
- 6. ? Features + Locations relative to object solvable Sensorimotor sequences Model of objects
- 11. These locations aren’t relative to the body.
- 12. These locations aren’t relative to the body. They’re relative to the object
- 13. They’re relative to the object These locations aren’t relative to the body.
- 14. ? Features + Locations relative to object solvable Sensorimotor sequences Model of objects
- 15. solvable? Features + Locations relative to body solvable This Talk Sensorimotor sequences Model of objects Features + Locations relative to object
- 16. Given the sensor’s location relative to the body… …calculate its location relative to the object.
- 18. Part 2 Inspiration: Principles of grid cells
- 21. Each grid cell fires at multiple equally spaced locations. Electrode Cell Entorhinal cortex A Room
- 22. Firing fields of 16 different cells. Electrode Cell 1 Cell 2 Cell 3 1 2 3 4 8765 1211109 16151413 1 2 3 8765 1211109 16151413 1 2 3 4 8765 1211109 16151413 1 65 109 151413 1 2 3 4 8765 1 2 3 4 8765 1211109 161514 2 3 4 87 1211 16 1 2 3 4 8765 4 1 5 4 13 Entorhinal cortex A Room
- 23. Room 1 After the rat learns a room, grid cells anchor to the room consistently. Room 2 They anchor to other rooms differently. “Purely Translational Realignment in Grid Cell Firing Patterns Following Nonmetric Context Change” Marozzi, Ginzberg, Alenda, Jeffery (2015)
- 24. A cool grid cell trick: Modules Module 2Module 1 + + Module 3 Entorhinal cortex = Big space of unique locations
- 25. Principles of grid cells The cell activity moves between a fixed set of locations. These locations are mapped onto the environment. =+ Body Motion2. Updating it from motion cues The brain determines the location by… 1. Recalling it from sensory cues + Feature =
- 27. Part 3 A solution: Recall, Normalize, Vote
- 28. Sensory 1 Object Input Object Input Location Sensory 2Location Column 1 Column 2 Layer 2 Layer 4
- 29. Column 1 Column 2 Object Input Feature relative to Object Object Input Feature relative to Object Body relative to Object Feature relative to Body Feature relative to Body New: Recall past locations of sensed features. New: Vote on the body’s location relative to the object. Layer 2 Layer 4 Layer 6A Sensory 1 Sensory 2
- 30. BB A B A BB A B A A A B A A B A A B A B B A A A Object 1 Object 3 Object 2 Object 4 B A B B A A A Demo:
- 31. BB A B A Visualizing the solution All objects are learned at points in a large “location” space.
- 32. A BB BB A B A All objects are learned at points in a large “location” space. Visualizing the solution
- 33. B A A BB BB A B A All objects are learned at points in a large “location” space. Visualizing the solution
- 34. B A A BB All objects are learned at points in a large “location” space. B A B B A A A Visualizing the solution
- 35. B A B B A A A B A A BB All objects are learned at points in a large “location” space. A B A Visualizing the solution
- 36. B A B B A A A B A A BB All objects are learned at points in a large “location” space. A A B B A Visualizing the solution
- 37. B A B B A A A B A A BB All objects are learned at points in a large “location” space. A A B BB A A Visualizing the solution
- 38. B A A A A B B BB B A A This “location” space now contains maps of multiple objects. Visualizing the solution Map of Object 1 Map of Object 2 Distance between isn’t meaningful
- 39. B A A A A B B BB B A A BB A B A To recognize a location, each sensor votes on the body’s location. Visualizing the solution
- 40. B A A A A B B BB B A A BB A B A To recognize a location, each sensor votes on the body’s location. Visualizing the solution
- 41. B A A A A B B BB B A A BB A B A To recognize a location, each sensor votes on the body’s location. Visualizing the solution
- 42. B A A A A B B BB B A A BB A B A 2 To recognize a location, each sensor votes on the body’s location. Visualizing the solution
- 43. B A A A A B B BB B A A BB A B A 3 2 2 2 2 To recognize a location, each sensor votes on the body’s location. Visualizing the solution
- 44. B A A A A B B BB B A A BB A B A 3 To recognize a location, each sensor votes on the body’s location. Visualizing the solution
- 45. solvable? Features + Locations relative to body solvable This Talk Sensorimotor sequences Model of objects Features + Locations relative to object
- 46. “Do I remember any objects that contain this arrangement of features?” The holistic system does this:
- 47. Summary Location is probably one of the brain’s main primitives. Grid cells suggest an approach to location: • Cortex has a “space” of locations that it can process. • Cortex maps objects into this space. Cortex could recognize these locations in 3 steps: Recall, Normalize, Vote B A A BB
- 48. Thank you.