Triangulating general intelligence and cortical microcircuits
- 1. Twitter: @dileeplearning @vicariousaiDileep George Triangulating general intelligence and cortical microcircuits
- 2. 1. Build Artificial General Intelligence 2. Understand how the brain works
- 3. -600 -450 -300 -150 0 old brain new brain sponges jellyfish flatworm fish reptiles birds amphibians mammaliaforms primates “Direct fit to nature” Behaviors for survival in a niche World modeling “Model free” “Model based”
- 5. Less structure More structure RBMs Conv-nets Fully connected Neural nets Flat Domain specific architectures Neo-Cortex Fewer assumptions More assumptions Data hungry Data efficient More innate structureTabula Rasa Tabula Rasa vs Innate tradeoff
- 6. To build general intelligence, the question we need to ask is this: What are the basic set of assumptions that are specific enough to make learning feasible in a reasonable amount of time while being general enough to be applicable to a large class of problems? Looking at the brain helps to speed up the search for the goldilocks set of constraints, and to tease apart the interdependent representational scaffolding by which they need to be operationalized in a machine.
- 7. Properties of the brain Properties of the world Real-world datasets Algorithmic model Triangulation strategy
- 8. Properties of the brain Properties of the world Real-world datasets Algorithmic model Triangulation strategy Cognitive Science Neurophysiology Neuroanatomy Physics of the world Real-world test sets Computational/Algorithmic model Computational principles
- 9. Properties of the brain Properties of the world Real-world datasets Algorithmic model Triangulation strategy Physics of the world Real-world test sets If you ignore the “world” vertex Doesn’t address the overall task of vision Doesn’t make contact with real-world data
- 10. Properties of the brain Properties of the world Real-world datasets Algorithmic model Triangulation strategy Physics of the world Real-world test sets If you ignore the “brain vertex” Pure machine learning / statistics Need not generalize like the brain Hard to connect to brain’s circuits
- 11. Properties of the brain Properties of the world Real-world datasets Algorithmic model Triangulation strategy Physics of the world Real-world test sets If you ignore the computational vertex Too hard to build working models No algorithmic principles to guide you
- 12. Properties of the brain Properties of the world Real-world datasets Algorithmic model Triangulation strategy • “RCN”: Recursive Cortical Network • Compositionality • Controllability: Top-down attention • 300x more data-efficient than deep CNNs • Oct 2017
- 14. CAPTCHAs are important because they exemplify the strong generalization we seek
- 17. Biological Feature Computational/Algorithmic reason Representation in RCN Blobs and interblobs Curvelet-like smoothness of natural signals & contour-surface factorization Structure of contour-surface factor Lateral connections between inter-blob columns Higher-order contour-continuity in natural signals Cloned structure of lateral connections for higher-order interactions Object-based attention Compositionality Only positive weights & Object- background factorization Hierarchy Efficient learning and inference Hierarchically structured Border-ownership coding Required when objects are represented in a factorized &n hierarchical manner Two clones of each feature for border ownership coding Feedback connections Inference requires explaining away when the representation is compositional Message passing algorithms automatically do explaining away Different dynamics for contour and surface features Convergence of message-passing depends on the schedule Biologically inspired message-scheduling works better Everything is interconnected. Trying to build a joint model can simplify things because of this.
- 18. • Factorized contour-surface representations • Lateral connections between contour features • Feedback and explaining away
- 20. What is the general computational principle behind contour-surface factorization?
- 21. Piecewise continuous. Discontinuities themselves are piecewise continuous in a lower dimension. 3D volumetric medical image Moving disk in 3D spatio-temporal space. Shock fronts in fluid dynamics
- 22. What do you lose through contour-surface factorization? you lose the ability to easily recognize QR codes
- 23. • Factorized contour-surface representations • Lateral connections between contour features • Feedback and explaining away
- 24. Lateral connections in the visual cortex Behavior Neurophysiology Property of natural signals Bosking et al 1997 Lawlor & Zucker 2013
- 25. generations from the contour network Lateral factor
- 26. generations from the contour network
- 27. • Factorized contour-surface representations • Lateral connections between contour features • Feedback and explaining away Required for dynamic contextual integration
- 28. m? Local inferences are often wrong even in ‘simple-looking’ in real-world images Explaining-away local evidence helps to find the globally best solution
- 29. m!
- 30. m!!
- 31. m?
- 32. coungk Explaining-away local evidence helps to find the globally best solution
- 33. Parsing a scene = MAP inference Backward pass will hallucinate some objects. These are then explained away with further message passing.
- 34. CAPTCHA parsing • RCN is NOT trained on examples from each CAPTCHA • We train on clean font examples • Generalizes to different deformations and clutter without specific training
- 35. One-shot and few-shot recognition on MNIST Robustness to novel perturbations
- 36. Reconstruction under novel perturbations
- 37. Omniglot : One-shot generation samples
- 38. Two standard benchmarks for real-world text recognition ICDAR Street View Text
- 39. Exceeds Deep CNN accuracy with 300x data efficiency
- 42. Task Competing model Classification CNN RCN Parsing Multi-CNN Reconstruction VAE, DRAW Generation Bayesian Program Learning Occlusion Reasoning & Binding RCN is an integrated vision model
- 44. Semi-transparent shiny Grasping succes rate > 99% on all these variations
- 45. Properties of the brain Properties of the world Real-world datasets Algorithmic model Triangulation strategy • “RCN”: Recursive Cortical Network • Compositionality • Controllability: Top-down attention • 300x more data-efficient than deep CNNs • Oct 2017 Microcircuit model
- 46. Properties of the brain Properties of the world Real-world datasets Algorithmic model Triangulation strategy • “RCN”: Recursive Cortical Network • Compositionality • Controllability: Top-down attention • 300x more data-efficient than deep CNNs • Oct 2017
- 47. Graphical Model -> Factor Graph -> Neural Circuit
- 49. A B C D a b c d e E ma!d md!a md!b mb!d mc!d md!eme!d md!c Belief b d OR OR OR F G ÷ expexp -1 X X X ⇤ log ⇤ mb!d mc!d me!d md!a mb!d mc!d md!a = log emc!d+mb!d+me!d emc!d+mb!d+me!d (mb!d) (mc!d)(eme!d 1) H a b c e Each variable is binary A scalar value is passed along each edge (log ratio)
- 50. microcolumn FB_pool Pools Contour Feature copies Contour Features Appearance Features macrocolumn FF_feat FF_copy FF_pool FF_lat BEL_lat BEL_feat FF_EXP_AWY FB_comp FeaturesLateralsPools FF_pool FF_feat a b c d e f g h i OR OR POOL CONTOUR-LAT CONTOUR-SURFACE APPEARANCE ExpAwy FF_lat RCN factor graph for one level
- 51. mf!a = max i=1...M mbi!f log M mf!bm = min(ma!f log M, max j6=m mbj !f ) f a b1 b2 bM · · · POOL zm = T log ⇣ mcm!d T ⌘ hm = X j6=m zj + Tf ⇣ X j6=m zj T ⌘ md!cm = T log ⇣ ⇣hm T ⌘ + ⇣ hm T ⌘ e me!d T ⌘ with f(x) = log(1 e x ) and (x) = 1 1 + e x md!e = X m zm + Tf ⇣ X m zm T ⌘ · · ·c1 c2 cM e dOR a b1 b2 bM · · · d md!a = X m mbm!d md!bj = ma!d + X m6=j mbm!d mf!h2 c = max h1 (mh1!f + L(h1, h2)) mf!h1 c = max h1 (mh2!f + L(h1, h2)) Lh1 h2 sa = T log X i e mai!f T sb = T log X i e mbi!f T sb = T log X i e mai!f +mbi!f T mf!e = T log ⇣ e sa+sb sd T 1 ⌘ c mf!ai c = mbi!f + T log ✓ 1 + e me!f T c ✓ e sb mbi!f T 1 ◆◆ mf!bi c = mai!f + T log ✓ 1 + e me!f T c ✓ e sb mai!f T 1 ◆◆ a1 . . . aM b1 . . . bM e A B C D E
- 52. From Retina TRN FF_copy FF_pool FB_pool FF_lat BEL_lat FF_lat FB_comp blob interblob blob BEL_lat FF_EXP_AWY BEL_POOL_FF BEL_feat a b c d e f g h i Layer 2/3 Layer 1 Layer 4 Layer 5 Layer 6 First Order Thalamus Higer Order Thalamus FeedforwardFeedbackExplaining Away microcolumn FeaturesLateralsPoolsFeaturesLaterals To other cortical columns Direct Feedforward Gated feedforward a b c d e f h i Feedback copyforfeedback copyforfeedback A B
- 53. Feedback/higher-level context to this feature Pooling centered on this feature Sequences through this feature Lateral connections of this feature Feedforward representation of this feature Lateral connections with feedback Belief computation of this feature Feedback to components of this feature microcolumn All the neurons in a cortical column represent the same feature Different laminae represent the computations corresponding to that features’ participation: • laterally with other features in same level • hierarchically with features in other levels • sequences passing through this feature • feed-forward, feedback computation • Belief computation Many inter-laminar, within-column connections can be innate. Inter-columnar connections are learned Cortical microcolumn as a binary random variable
- 54. Thalamus as “Explaining away and gating”
- 55. a b c L4 L5 L3 L2 L6 L1 TRN HO e L4 L5 L3 L2 L6 L1 TRN FO Sensory input ma!dmd!a md!b mb!d mc!d md!c TRN HO L6 L5 b L5 d L6 e a b c d e OR OR OR a c Sherman & Guillery thalamic pathway
- 56. ma!dmd!a md!b mb!d mc!d md!c TRN HO L6 L5 b L5 d L6 e a b c d e OR OR OR a c
- 57. interactions between inter-blobs and blobs
- 64. Properties of the brain Properties of the world Real-world datasets Algorithmic model Triangulation strategy
- 65. Visit www.vicarious.com Follow Vicarious on Twitter @vicariousai Email: dileep@vicarious.com Follow me on Twitter @dileeplearning More information & Contact Thank you for listening!