Jeff Hawkins NAISys 2020: How the Brain Uses Reference Frames, Why AI Needs to do the Same
1 like2,341 views
The document discusses how the neocortex models the world through distributed sensory-motor systems using reference frames, which are crucial for understanding human intelligence and its implications for AI. It highlights the similarities between cortical columns and AI structures, emphasizing the need for AI to adopt similar principles of learning and knowledge representation. The author's research proposes frameworks to enhance AI performance using concepts derived from biological intelligence.
Jeff Hawkins NAISys 2020: How the Brain Uses Reference Frames, Why AI Needs to do the Same
- 1. How the Brain Uses Reference Frames to Model the World Why AI Needs to do the Same JEFF HAWKINS NAISYS 2020
- 2. To understand the brain we need “new ways of thinking about it”. “What is conspicuously lacking is a framework of ideas within which to interpret all these different approaches.” Crick 1979
- 3. Organ of intelligence - Sensory perception: vision, touch, hearing - Motor: limbs, tools, language - Abstract thought: math, science, philosophy Attributes - Learns continuously - Learns rapidly - Efficient: 20 watts for brain, 5ms process step - Flexible: learns thousands of diverse tasks 1) The neocortex learns a model of the world. 2) It is a highly distributed model. Each cortical column is a complete sensory-motor modeling system. 3) Cortical columns use reference frames to store knowledge and generate behavior. 4) Reference frames in the cortex are derivatives of grid cells and place cells. Today’s AI is not as capable, not even close. Outline of talk
- 4. The neocortex learns a model of the world Your model of the world is huge - Thousands of physical objects, how they look, feel, and sound - How objects are composed other objects - Where objects are located relative to each other - How objects behave - Concepts such as math, democracy We use the model to create goal-oriented behaviors The model is predictive - Prediction is the primary training signal
- 5. Cortical columns Approx. 150,000 columns (1 x 2.5mm) Mountcastle 1979: - All columns look similar because they perform the same intrinsic function. - What a column does is determined by what it is connected to. - Understanding what a column does will have “great generalizing significance”. 1) Columns are complex 100K neurons, 500M synapses Dozens of cell types, hundreds of minicolumns Whatever they do is also complex 2) All columns have a motor output 3) All columns are remarkably similar Mountcastle 1997
- 7. L2/3 L4 L6 Location reference frame Object A single column learns completes models of objects by integrating features and locations over time. “A Theory of How Columns in the Neocortex Enable Learning the Structure of the World” (Hawkins, et. al., 2017) Multiple columns can infer objects in a single sensation by “voting” on object identity. ? Sensed feature
- 8. Reference Frames in the Old Brain “Grid cells” in entorhinal cortex - Reference frames for environments “Place cells” in hippocampus - Sensory driven representation of location Grid and place cell equivalents exist in every cortical column - Create reference frames for objects Moser, 2005 Hawkins et. al., 2018 Lewis et. al., 2018 Hawkins et. al., 2017 Okeefe, 1978
- 9. Entorhinal Cortex Room Neocortex Grid cells Represent location of body in a reference frame relative to room. Cortical grid cells Represent location of sensor in a reference frame relative to object.
- 10. L2/3 L4 L6 Object Sensed feature Cortical “Grid” Cells Lewis et. al., 2018 Location (reference frame) Cortical “Place” Cells
- 11. Felleman, van Essen, 1991 Retina Simple features Complex features Object v1 v2 Hierarchy - Most connections are not hierarchical. - More than 40% of all possible connections exist. - Primary and secondary regions are largest. - Primary sensory regions exhibit multi-modal responses. Retina Skin The Thousand Brains Theory of Intelligence - There are thousands of complementary models - Most connections are for voting. (blue) We are aware of the consensus - Hierarchical connections pass complete objects
- 12. Doeller, C. F., Barry, C., & Burgess, N. (2010). Evidence for grid cells in a human memory network. Nature Growing empirical evidence for grid cells in the neocortex Constantinescu, A., O’Reilly, J., Behrens, T. (2016) Organizing Conceptual Knowledge in Humans with a Gridlike Code. Science
- 13. Biological intelligence requires learning a model of the world. 1) Each cortical column is a complete sensory-motor modeling system. 2) Columns vote to reach a consensus. 3) Cortical columns use reference frames to represent knowledge. - objects, body, concepts I believe machine intelligence must work on the same principles. Summary
- 14. ROADMAP TO MACHINE INTELLIGENCE Performance Robustness Point neuron Continuous learning CNN Invariant representations Fast learning Sparsity Active Dendrites Reference frames Model voting Today: We released a white paper and press release. Using sparse-sparse CNNs we demonstrate large performance gains (50x) with competitive accuracy.
- 15. 2019: A Framework for Intelligence and Cortical Function Based on Grid Cells in the Neocortex 2019: Locations in the Neocortex: A Theory of Sensorimotor Object Recognition Using Cortical Grid Cells 2019: Flexible Representation and Memory of Higher-Dimensional Cognitive Variables with Grid Cells 2017: A Theory of How Columns in the Neocortex Enable Learning the Structure of the World 2017: The HTM Spatial Pooler—A Neocortical Algorithm for Online Sparse Distributed Coding 2016: Why Neurons Have Thousands of Synapses, A Theory of Sequence Memory in Neocortex 2016: Continuous Online Sequence Learning with an Unsupervised Neural Network Model Neuroscience Papers Poster 43 “Sparsity in the Neocortex and its Implications for Machine Learning” Contact Subutai Ahmad via slack or email to arrange a zoom discussion. sahmad@numenta.com jhawkins@numenta.com Available Early 2021Thank You