SlideShare a Scribd company logo

Machine learning and cognitive neuroimaging: new tools can answer new questions

Gael Varoquaux, profile picture
Gael Varoquaux

The document discusses the impact of machine learning on cognitive neuroimaging, emphasizing its ability to address complex questions and improve models of brain activity through encoding and decoding approaches. It highlights the importance of rich encoding models, the challenges of interpreting overlapping activations, and the role of machine learning in enhancing the understanding of cognitive functions. Various statistical methods and models, including multi-voxel pattern analysis, are explored to improve predictions related to brain activity and its implications for cognitive neuroscience.

Healthcare
Related topics:
Cognitive Science Insights
1 of 65
Downloaded 78 times
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
Machine learning and cognitive neuroimaging: new tools can answer new questions Gaël Varoquaux How machine learning is shaping cognitive neuroimaging [Varoquaux and Thirion 2014]
Cognitive neuroscience: linking psychology and neuroscience (neural implementations) Vision: A computational investigation into the human representation and processing of visual information [Marr 1982] G Varoquaux 2
Machine learning: computational statistics for prediction (out-of-sample properties) Paradigm shift the dimensionality of data grows, enables richer models Open-ended questions ⇒ large # features From parameter inference to prediction x y G Varoquaux 3
Machine learning: computational statistics for prediction (out-of-sample properties) Paradigm shift the dimensionality of data grows, enables richer models Open-ended questions ⇒ large # features From parameter inference to prediction x y Understanding, not predicting Danger of solving the wrong problem Lost in formalization G Varoquaux 3
Statistics Machine learning Statistical machine learning Hypothesis testing Prediction T-test Tests on prediction Cross-validation G Varoquaux 4
Statistics Machine learning Statistical machine learning Hypothesis testing Prediction T-test Tests on prediction Cross-validation In sample Out of sample G Varoquaux 4
Statistics Machine learning Statistical machine learning Hypothesis testing Prediction T-test Tests on prediction Cross-validation In sample Out of sample Parametric Non-parametric G Varoquaux 4
Statistics Machine learning Statistical machine learning Hypothesis testing Prediction T-test Tests on prediction Cross-validation In sample Out of sample Parametric Non-parametric Non-parametric tests Probabilistic modeling Few parameters Many parameters G Varoquaux 4
Statistics Machine learning Statistical machine learning Hypothesis testing Prediction T-test Tests on prediction Cross-validation In sample Out of sample Parametric Non-parametric Non-parametric tests Probabilistic modeling Few parameters Many parameters Univariate Multivariate G Varoquaux 4
Statistics Machine learning Statistical machine learning Hypothesis testing Prediction T-test Tests on prediction Cross-validation In sample Out of sample Parametric Non-parametric Non-parametric tests Probabilistic modeling Few parameters Many parameters Univariate Multivariate GLM = correlations Naive Bayes Univariate selection Differences mostly cultural: it’s a continuum G Varoquaux 4
Cognitive neuroimaging and machine learning G Varoquaux 5
Cognitive neuroimaging and machine learning Predicting the task: decoding G Varoquaux 5
Cognitive neuroimaging and machine learning Predicting neural response: encoding G Varoquaux 5
Cognitive neuroimaging and machine learning Unsupervised learning on brain activity G Varoquaux 5
Cognitive neuroimaging and machine learning Unsupervised learning on behavior G Varoquaux 5
Cognitive neuroimaging and machine learning G Varoquaux 5
Rest of this talk 1 Encoding 2 Decoding G Varoquaux 6
1 Encoding Towards richer models of brain activity G Varoquaux 7
1 Uncovering neural coding Insights on breaking down cognitive functions into atomic steps [Hubel and Wiesel 1962] Neurons receptive to Gabors (edges) G Varoquaux 8
1 Uncovering neural coding Insights on breaking down cognitive functions into atomic steps [Hubel and Wiesel 1962] Neurons receptive to Gabors (edges) [Logothetis... 1995] Shapes in inferior temporal cortex G Varoquaux 8
1 Uncovering neural coding: richer models Insights on breaking down cognitive functions into atomic steps [Hubel and Wiesel 1962] Neurons receptive to Gabors (edges) [Logothetis... 1995] Shapes in inferior temporal cortex Machine learning: computer-vision models mapped to brain activity [Yamins... 2014] G Varoquaux 8
1 Uncovering neural coding: in fMRI Model-based fMRI [O’Doherty... 2007] [Harvey... 2013] High-level descriptions [Mitchell... 2008] Natural stimuly [Kay... 2008] G Varoquaux 9
Machine learning for encoding models Richer models of encoding capture fine descriptions of behavior / stimuli Require to forgo the contrast methodolgy Is this a good or a bad thing? G Varoquaux 10
1 Models of the visual system Image V1 cortex V2 cortex Inferior temporal cortex Fusiform face area Jack? Is there a “face” region? A “foot” region? A “left big toe” region? G Varoquaux 11
1 Uncovering neural coding: cognitive oppositions Is there a “face” region? A “foot” region? A “left big toe” region? vs G Varoquaux 12
1 Uncovering neural coding: cognitive oppositions Is there a “face” region? A “foot” region? A “left big toe” region? vs G Varoquaux 12
1 Uncovering neural coding: cognitive oppositions Is there a “face” region? A “foot” region? A “left big toe” region? vs - G Varoquaux 12
1 Uncovering neural coding: cognitive oppositions Is there a “face” region? A “foot” region? A “left big toe” region? vs -Mapping relies on cognitive subtraction Bound to mental process decomposition G Varoquaux 12
1 Decomposing visual stimuli Low-level visual cortex is tuned to natural image statistics [Olshausen et al. 1996] What drives high-level representations? G Varoquaux 13
1 Decomposing visual stimuli Low-level visual cortex is tuned to natural image statistics [Olshausen et al. 1996] What drives high-level representations? Convolutional Net G Varoquaux 13
Data-driven encoding models Image V1 cortex V2 cortex Inferior temporal cortex Fusiform face area Jack? [Khaligh-Razavi and Kriegeskorte 2014, Güçlü and van Gerven 2015] FMRI beyond a handfull of contrasts ⇒ Sets us free from the paradigm G Varoquaux 14
2 Decoding From brain activity to behavior G Varoquaux 15
2 Increased sensitivity “Given the goal of detecting the presence of a particular mental representation in the brain, the primary advantage of MVPA methods over individual-voxel-based methods is increased sensitivity.” — [Norman... 2006] G Varoquaux 16
2 Increased sensitivity An omnibus test “Given the goal of detecting the presence of a particular mental representation in the brain, the primary advantage of MVPA methods over individual-voxel-based methods is increased sensitivity.” — [Norman... 2006] Is there “information” about a stimuli in a given region? G Varoquaux 16
2 Increased sensitivity An omnibus test “Given the goal of detecting the presence of a particular mental representation in the brain, the primary advantage of MVPA methods over individual-voxel-based methods is increased sensitivity.” — [Norman... 2006] “However, these maps are not guaranteed to include all the voxels that are involved in representing the categories of interest.” — [Norman... 2006] G Varoquaux 16
Non-linear cognitive model Linear predictive models Representations Stimuli 2 Increased sensitivity An omnibus test Decoding used to test / compare encoding models [Naselaris... 2011] G Varoquaux 17
2 Generalization as a test: cross-validation x y x y High-dimensional models G Varoquaux 18
2 Generalization as a test: cross-validation x y x y High-dimensional models ⇒ Important to test on independent data, to control for model complexity G Varoquaux 18
2 Generalization as a test: cross-validation High-dimensional models ⇒ Important to test on independent data, to control for model complexity ­40% ­20% ­10%  0% +10% +20% +40% Leave one sample out Leave one subject/session 20% left­out,   3 splits 20% left­out,   10 splits 20% left­out,   50 splits ­22% +19% +3% +43% ­10% +10% ­21% +17% ­11% +11% ­24% +16% ­9% +9% ­24% +14% ­9% +8% ­23% +13%   Intra subject   Inter subject No silver bullet Poster 3829, Oral Th 12:45 G Varoquaux 18
2 Behavioral predictions as a test Increase “cognitive resolution” One voxel’s information is not enough to distinguish many cognitive states ⇒ analysis combining info across voxels G Varoquaux 19
2 Behavioral predictions as a test Increase “cognitive resolution” One voxel’s information is not enough to distinguish many cognitive states ⇒ analysis combining info across voxels Interpreting overlapping activations Psychology not interested in where a task is creating activation, but if two tasks are creating activations in same areas G Varoquaux 19
2 Inference in cognitive neuroimaging What is the neural support of a function? What is function of a given brain module? G Varoquaux 20
2 Inference in cognitive neuroimaging What is the neural support of a function? What is function of a given brain module? Brain mapping = task-evoked activity G Varoquaux 20
2 Inference in cognitive neuroimaging [Poldrack 2006, Henson 2006] What is the neural support of a function? What is function of a given brain module? Reverse inference Brain mapping = task-evoked activity + crafting “contrasts” to isolate effects G Varoquaux 20
2 Inference in cognitive neuroimaging [Kanwisher... 1997, Gauthier... 2000, Hanson and Halchenko 2008] What is the neural support of a function? What is function of a given brain module? Reverse inference Is there a face area? G Varoquaux 20
2 Inference in cognitive neuroimaging [Poldrack... 2009, Schwartz... 2013] What is the neural support of a function? What is function of a given brain module? Reverse inference Decoding: Find regions that predict observed cognition G Varoquaux 20
2 Decoding for reverse inference [Poldrack... 2009, Schwartz... 2013] Prediction = proxy for implication Need large cognitive coverage G Varoquaux 21
2 Decoding for reverse inference [Poldrack... 2009, Schwartz... 2013] Prediction = proxy for implication Need large cognitive coverage Interpretation of the “grandmother neuron” “more than a neuron re- sponds to one concept and [...] neurons do not neces- sarily respond to only one concept are given by the data itself [Quian Quiroga and Kreiman 2010] G Varoquaux 21
2 Brain decoding with linear models Design matrix × Coefficients = Coefficients are brain maps Target G Varoquaux 22
2 Brain decoding to recover predictive regions? Face vs house visual recognition [Haxby... 2001] SVM error: 26% G Varoquaux 23
2 Brain decoding to recover predictive regions? Face vs house visual recognition [Haxby... 2001] Sparse model error: 19% G Varoquaux 23
2 Brain decoding to recover predictive regions? Face vs house visual recognition [Haxby... 2001] Ridge error: 15% Best predictor outlines the worst regions Best maps predict worst G Varoquaux 23
2 Decoders as estimators [Gramfort... 2013] Inverse problem Minimize the error term: ˆw = argmin w l(y − X w) Ill-posed: Many different w will give the same prediction error Choice driven by (implicit) priors of the decoder SVM sparse ridge TV- 1 G Varoquaux 24
2 Decoders as estimators [Gramfort... 2013] Inverse problem Minimize the error term: ˆw = argmin w l(y − X w) Ill-posed: Many different w will give the same prediction error Choice driven by (implicit) priors of the decoder SVM sparse ridge TV- 1 Inferences rely, explicitely or implicitely, on the regions estimated by the decoder G Varoquaux 24
Wrapping up G Varoquaux 25
@GaelVaroquaux Machine learning for cognitive neuroimaging The description of cognition is hard ⇒ Encoding Rich models depend less on paradigms
@GaelVaroquaux Machine learning for cognitive neuroimaging The description of cognition is hard ⇒ Encoding Decoding as an omnibus test For rich encoding models To interpret overlaping activation Cross-validation error bars
@GaelVaroquaux Machine learning for cognitive neuroimaging The description of cognition is hard ⇒ Encoding Decoding as an omnibus test Decoding for reverse inference Requires large cognitive coverage
@GaelVaroquaux Machine learning for cognitive neuroimaging The description of cognition is hard ⇒ Encoding Decoding as an omnibus test Decoding for reverse inference Estimation of predictive regions is difficult Infinite number of maps predict as well
@GaelVaroquaux Machine learning for cognitive neuroimaging The description of cognition is hard ⇒ Encoding Decoding as an omnibus test Decoding for reverse inference Estimation of predictive regions is difficult Software: nilearn In Python http://nilearn.github.io ni [Varoquaux and Thirion 2014] How machine learning is shaping cognitive neuroimaging
References I I. Gauthier, M. J. Tarr, J. Moylan, P. Skudlarski, J. C. Gore, and A. W. Anderson. The fusiform “face area” is part of a network that processes faces at the individual level. J cognitive neuroscience, 12:495, 2000. A. Gramfort, B. Thirion, and G. Varoquaux. Identifying predictive regions from fMRI with TV-L1 prior. In PRNI, page 17, 2013. U. Güçlü and M. A. van Gerven. Deep neural networks reveal a gradient in the complexity of neural representations across the ventral stream. The Journal of Neuroscience, 35(27): 10005–10014, 2015. S. J. Hanson and Y. O. Halchenko. Brain reading using full brain support vector machines for object recognition: there is no “face” identification area. Neural Computation, 20:486, 2008. B. Harvey, B. Klein, N. Petridou, and S. Dumoulin. Topographic representation of numerosity in the human parietal cortex. Science, 341(6150):1123–1126, 2013.
References II J. V. Haxby, I. M. Gobbini, M. L. Furey, ... Distributed and overlapping representations of faces and objects in ventral temporal cortex. Science, 293:2425, 2001. R. Henson. Forward inference using functional neuroimaging: Dissociations versus associations. Trends in cognitive sciences, 10:64, 2006. D. H. Hubel and T. N. Wiesel. Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. The Journal of physiology, 160:106, 1962. N. Kanwisher, J. McDermott, and M. M. Chun. The fusiform face area: a module in human extrastriate cortex specialized for face perception. J Neuroscience, 17:4302, 1997. K. N. Kay, T. Naselaris, R. J. Prenger, and J. L. Gallant. Identifying natural images from human brain activity. Nature, 452:352, 2008.
References III S.-M. Khaligh-Razavi and N. Kriegeskorte. Deep supervised, but not unsupervised, models may explain it cortical representation. PLoS Comput Biol, 10(11):e1003915, 2014. N. K. Logothetis, J. Pauls, and T. Poggio. Shape representation in the inferior temporal cortex of monkeys. Current Biology, 5:552, 1995. D. Marr. Vision: A computational investigation into the human representation and processing of visual information. The MIT press, Cambridge, 1982. T. M. Mitchell, S. V. Shinkareva, A. Carlson, K.-M. Chang, V. L. Malave, R. A. Mason, and M. A. Just. Predicting human brain activity associated with the meanings of nouns. science, 320: 1191, 2008. T. Naselaris, K. N. Kay, S. Nishimoto, and J. L. Gallant. Encoding and decoding in fMRI. Neuroimage, 56:400, 2011.
References IV K. A. Norman, S. M. Polyn, G. J. Detre, and J. V. Haxby. Beyond mind-reading: multi-voxel pattern analysis of fmri data. Trends in cognitive sciences, 10:424, 2006. J. P. O’Doherty, A. Hampton, and H. Kim. Model-based fMRI and its application to reward learning and decision making. Annals of the New York Academy of Sciences, 1104:35, 2007. B. Olshausen ... Emergence of simple-cell remainsceptive field properties by learning a sparse code for natural images. Nature, 381:607, 1996. R. Poldrack. Can cognitive processes be inferred from neuroimaging data? Trends in cognitive sciences, 10:59, 2006. R. A. Poldrack, Y. O. Halchenko, and S. J. Hanson. Decoding the large-scale structure of brain function by classifying mental states across individuals. Psychological Science, 20:1364, 2009.
References V R. Quian Quiroga and G. Kreiman. Postscript: About grandmother cells and jennifer aniston neurons. Psychological Review, 117: 297, 2010. Y. Schwartz, B. Thirion, and G. Varoquaux. Mapping cognitive ontologies to and from the brain. In NIPS, 2013. G. Varoquaux and B. Thirion. How machine learning is shaping cognitive neuroimaging. GigaScience, 3:28, 2014. D. L. Yamins, H. Hong, C. F. Cadieu, E. A. Solomon, D. Seibert, and J. J. DiCarlo. Performance-optimized hierarchical models predict neural responses in higher visual cortex. Proc Natl Acad Sci, page 201403112, 2014.

More Related Content

PDF
Scikit-learn and nilearn: Democratisation of machine learning for brain imaging
Gael Varoquaux
 
PDF
Better neuroimaging data processing: driven by evidence, open communities, an...
Gael Varoquaux
 
PDF
Python for brain mining: (neuro)science with state of the art machine learnin...
Gael Varoquaux
 
PDF
Functional-connectome biomarkers to meet clinical needs?
Gael Varoquaux
 
PDF
Towards psychoinformatics with machine learning and brain imaging
Gael Varoquaux
 
PDF
Multimodal Deep Learning
Universitat Politècnica de Catalunya
 
PDF
Deep learning for detecting anomalies and software vulnerabilities
Deakin University
 
PDF
Visual reasoning
Deakin University
 
Scikit-learn and nilearn: Democratisation of machine learning for brain imaging
Gael Varoquaux
 
Better neuroimaging data processing: driven by evidence, open communities, an...
Gael Varoquaux
 
Python for brain mining: (neuro)science with state of the art machine learnin...
Gael Varoquaux
 
Functional-connectome biomarkers to meet clinical needs?
Gael Varoquaux
 
Towards psychoinformatics with machine learning and brain imaging
Gael Varoquaux
 
Multimodal Deep Learning
Universitat Politècnica de Catalunya
 
Deep learning for detecting anomalies and software vulnerabilities
Deakin University
 
Visual reasoning
Deakin University
 

What's hot (6)

PDF
Deep Learning Representations for All (a.ka. the AI hype)
Universitat Politècnica de Catalunya
 
PPT
Machine Learning ICS 273A
butest
 
PDF
Deep learning 1.0 and Beyond, Part 1
Deakin University
 
PDF
Deep Neural Networks for Multimodal Learning
Marc Bolaños Solà
 
PDF
Self-supervised Visual Learning 2020 - Xavier Giro-i-Nieto - UPC Barcelona
Universitat Politècnica de Catalunya
 
PDF
Open-ended Visual Question-Answering
Universitat Politècnica de Catalunya
 
Deep Learning Representations for All (a.ka. the AI hype)
Universitat Politècnica de Catalunya
 
Machine Learning ICS 273A
butest
 
Deep learning 1.0 and Beyond, Part 1
Deakin University
 
Deep Neural Networks for Multimodal Learning
Marc Bolaños Solà
 
Self-supervised Visual Learning 2020 - Xavier Giro-i-Nieto - UPC Barcelona
Universitat Politècnica de Catalunya
 
Open-ended Visual Question-Answering
Universitat Politècnica de Catalunya
 
Ad

Viewers also liked (20)

PDF
Scientist meets web dev: how Python became the language of data
Gael Varoquaux
 
PDF
Scikit-learn: the state of the union 2016
Gael Varoquaux
 
PDF
Succeeding in academia despite doing good_software
Gael Varoquaux
 
PDF
On the code of data science
Gael Varoquaux
 
PDF
Scikit-learn for easy machine learning: the vision, the tool, and the project
Gael Varoquaux
 
PDF
Inter-site autism biomarkers from resting state fMRI
Gael Varoquaux
 
PDF
Brain maps from machine learning? Spatial regularizations
Gael Varoquaux
 
PPT
Intelligent Machine On Cognitive Methodological Development
Naga Balaji
 
PPT
The Future and Innovation
John Cousins
 
PDF
Computer system architecture
Pradhan Rishi Sharma
 
PDF
A hand-waving introduction to sparsity for compressed tomography reconstruction
Gael Varoquaux
 
PDF
Advanced network modelling 2: connectivity measures, goup analysis
Gael Varoquaux
 
PDF
Processing biggish data on commodity hardware: simple Python patterns
Gael Varoquaux
 
PPTX
computer system Architecture (Food detector)
Mehul Boghra
 
PDF
Social-sparsity brain decoders: faster spatial sparsity
Gael Varoquaux
 
PDF
Open Source Scientific Software
Gael Varoquaux
 
PPTX
Cognitive and Socio-Emotional Resilience in Dyslexia
fumikohoeft
 
PDF
Connectomics: Parcellations and Network Analysis Methods
Gael Varoquaux
 
PDF
Scikit learn: apprentissage statistique en Python
Gael Varoquaux
 
PDF
CS4109 Computer System Architecture
ktosri
 
Scientist meets web dev: how Python became the language of data
Gael Varoquaux
 
Scikit-learn: the state of the union 2016
Gael Varoquaux
 
Succeeding in academia despite doing good_software
Gael Varoquaux
 
On the code of data science
Gael Varoquaux
 
Scikit-learn for easy machine learning: the vision, the tool, and the project
Gael Varoquaux
 
Inter-site autism biomarkers from resting state fMRI
Gael Varoquaux
 
Brain maps from machine learning? Spatial regularizations
Gael Varoquaux
 
Intelligent Machine On Cognitive Methodological Development
Naga Balaji
 
The Future and Innovation
John Cousins
 
Computer system architecture
Pradhan Rishi Sharma
 
A hand-waving introduction to sparsity for compressed tomography reconstruction
Gael Varoquaux
 
Advanced network modelling 2: connectivity measures, goup analysis
Gael Varoquaux
 
Processing biggish data on commodity hardware: simple Python patterns
Gael Varoquaux
 
computer system Architecture (Food detector)
Mehul Boghra
 
Social-sparsity brain decoders: faster spatial sparsity
Gael Varoquaux
 
Open Source Scientific Software
Gael Varoquaux
 
Cognitive and Socio-Emotional Resilience in Dyslexia
fumikohoeft
 
Connectomics: Parcellations and Network Analysis Methods
Gael Varoquaux
 
Scikit learn: apprentissage statistique en Python
Gael Varoquaux
 
CS4109 Computer System Architecture
ktosri
 
Ad

Similar to Machine learning and cognitive neuroimaging: new tools can answer new questions (14)

PDF
Atlases of cognition with large-scale human brain mapping
Gael Varoquaux
 
PDF
Simple big data, in Python
Gael Varoquaux
 
PDF
Machine learning for functional connectomes
Gael Varoquaux
 
PDF
Coding for science and innovation
Gael Varoquaux
 
PDF
Brain reading, compressive sensing, fMRI and statistical learning in Python
Gael Varoquaux
 
PDF
Measuring mental health with machine learning and brain imaging
Gael Varoquaux
 
PDF
Exploring EEG for object detection and retrieval
Universitat Politècnica de Catalunya
 
PDF
Can we recover meaning full spatial information from multivariate pattern ana...
Gael Varoquaux
 
PDF
Cognitive Vision - After the hype
potaters
 
PDF
Estimating Functional Connectomes: Sparsity’s Strength and Limitations
Gael Varoquaux
 
PPTX
Our Best Ideas in Our Hands with Adaptive Virtual Reality - MCAA - ESOF 2016
Niki Lambropoulos PhD
 
PDF
Individual Brain Charting, a high-resolution fMRI dataset for cognitive mappi...
Ana Luísa Pinho
 
PDF
Where is my mind?
Nicolas Rougier
 
PDF
Data Visualisation
Daniel Marcous
 
Atlases of cognition with large-scale human brain mapping
Gael Varoquaux
 
Simple big data, in Python
Gael Varoquaux
 
Machine learning for functional connectomes
Gael Varoquaux
 
Coding for science and innovation
Gael Varoquaux
 
Brain reading, compressive sensing, fMRI and statistical learning in Python
Gael Varoquaux
 
Measuring mental health with machine learning and brain imaging
Gael Varoquaux
 
Exploring EEG for object detection and retrieval
Universitat Politècnica de Catalunya
 
Can we recover meaning full spatial information from multivariate pattern ana...
Gael Varoquaux
 
Cognitive Vision - After the hype
potaters
 
Estimating Functional Connectomes: Sparsity’s Strength and Limitations
Gael Varoquaux
 
Our Best Ideas in Our Hands with Adaptive Virtual Reality - MCAA - ESOF 2016
Niki Lambropoulos PhD
 
Individual Brain Charting, a high-resolution fMRI dataset for cognitive mappi...
Ana Luísa Pinho
 
Where is my mind?
Nicolas Rougier
 
Data Visualisation
Daniel Marcous
 

More from Gael Varoquaux (10)

PDF
Evaluating machine learning models and their diagnostic value
Gael Varoquaux
 
PDF
Machine learning with missing values
Gael Varoquaux
 
PDF
Dirty data science machine learning on non-curated data
Gael Varoquaux
 
PDF
Representation learning in limited-data settings
Gael Varoquaux
 
PDF
Similarity encoding for learning on dirty categorical variables
Gael Varoquaux
 
PDF
Simple representations for learning: factorizations and similarities
Gael Varoquaux
 
PDF
A tutorial on Machine Learning, with illustrations for MR imaging
Gael Varoquaux
 
PDF
Computational practices for reproducible science
Gael Varoquaux
 
PDF
Building a cutting-edge data processing environment on a budget
Gael Varoquaux
 
PDF
Scikit-learn: apprentissage statistique en Python. Créer des machines intelli...
Gael Varoquaux
 
Evaluating machine learning models and their diagnostic value
Gael Varoquaux
 
Machine learning with missing values
Gael Varoquaux
 
Dirty data science machine learning on non-curated data
Gael Varoquaux
 
Representation learning in limited-data settings
Gael Varoquaux
 
Similarity encoding for learning on dirty categorical variables
Gael Varoquaux
 
Simple representations for learning: factorizations and similarities
Gael Varoquaux
 
A tutorial on Machine Learning, with illustrations for MR imaging
Gael Varoquaux
 
Computational practices for reproducible science
Gael Varoquaux
 
Building a cutting-edge data processing environment on a budget
Gael Varoquaux
 
Scikit-learn: apprentissage statistique en Python. Créer des machines intelli...
Gael Varoquaux
 

Recently uploaded (20)

PPTX
PEDIATRIC OSCE, MBBS, by Dr. Sangit Chhantyal(IOM)..pptx
SangeetChhantyal
 
PPTX
Rheumatic heart diseases with Type 2 Diabetes Mellitus
drsharmasekhar54321
 
PPTX
Immunity....(shweta).................pptx
Shweta Singh
 
PPTX
1. Drug Distribution System.pptt b pharmacy
SurajRawal10
 
PPTX
General Pharmacology by Nandini Ratne, Nagpur College of Pharmacy, Hingna Roa...
Nandini Ratne
 
PDF
Assessment of Complications in Patients Maltreated with Fixed Self Cure Acryl...
Aun Haider
 
PDF
Khaled Sary- Trailblazers of Transformation Middle East's 5 Most Inspiring Le...
insightscare
 
DOCX
Copies if quanti.docxsegdfhfkhjhlkjlj,klkj
bacunawajade
 
PDF
Structure Composition and Mechanical Properties of Australian O.pdf
drdivyasinha5
 
PPTX
Infection prevention and control for medical students
w2tz2qrqxd
 
PPTX
PE and Health 7 Quarter 3 Lesson 1 Day 3,4 and 5.pptx
AnnaMae39
 
PPT
KULIAH UG WANITA Prof Endang 121110 (1).ppt
eddieSaputra4
 
PPTX
First Aid and Basic Life Support Training.pptx
polistadesiree
 
PPTX
3. Adherance Complianace.pptx pharmacy pci
SurajRawal10
 
PPTX
Trichuris trichiura infection
banisterbribi
 
PPTX
HEMODYNAMICS - I DERANGEMENTS OF BODY FLUIDS.pptx
shahanakhankhan36
 
PDF
Dr. Jasvant Modi - Passionate About Philanthropy
Dr. Jasvant Modi
 
PDF
NUTRITION THROUGHOUT THE LIFE CYCLE CHILDHOOD -AGEING
LoverTeddy
 
PPTX
Medical aspects of impairment including all the domains mentioned in ICF
muskaan650751
 
PPTX
Importance of Immediate Response (1).pptx
aryethscandelaria
 
PEDIATRIC OSCE, MBBS, by Dr. Sangit Chhantyal(IOM)..pptx
SangeetChhantyal
 
Rheumatic heart diseases with Type 2 Diabetes Mellitus
drsharmasekhar54321
 
Immunity....(shweta).................pptx
Shweta Singh
 
1. Drug Distribution System.pptt b pharmacy
SurajRawal10
 
General Pharmacology by Nandini Ratne, Nagpur College of Pharmacy, Hingna Roa...
Nandini Ratne
 
Assessment of Complications in Patients Maltreated with Fixed Self Cure Acryl...
Aun Haider
 
Khaled Sary- Trailblazers of Transformation Middle East's 5 Most Inspiring Le...
insightscare
 
Copies if quanti.docxsegdfhfkhjhlkjlj,klkj
bacunawajade
 
Structure Composition and Mechanical Properties of Australian O.pdf
drdivyasinha5
 
Infection prevention and control for medical students
w2tz2qrqxd
 
PE and Health 7 Quarter 3 Lesson 1 Day 3,4 and 5.pptx
AnnaMae39
 
KULIAH UG WANITA Prof Endang 121110 (1).ppt
eddieSaputra4
 
First Aid and Basic Life Support Training.pptx
polistadesiree
 
3. Adherance Complianace.pptx pharmacy pci
SurajRawal10
 
Trichuris trichiura infection
banisterbribi
 
HEMODYNAMICS - I DERANGEMENTS OF BODY FLUIDS.pptx
shahanakhankhan36
 
Dr. Jasvant Modi - Passionate About Philanthropy
Dr. Jasvant Modi
 
NUTRITION THROUGHOUT THE LIFE CYCLE CHILDHOOD -AGEING
LoverTeddy
 
Medical aspects of impairment including all the domains mentioned in ICF
muskaan650751
 
Importance of Immediate Response (1).pptx
aryethscandelaria
 

Machine learning and cognitive neuroimaging: new tools can answer new questions

  • 1. Machine learning and cognitive neuroimaging: new tools can answer new questions Gaël Varoquaux How machine learning is shaping cognitive neuroimaging [Varoquaux and Thirion 2014]
  • 2. Cognitive neuroscience: linking psychology and neuroscience (neural implementations) Vision: A computational investigation into the human representation and processing of visual information [Marr 1982] G Varoquaux 2
  • 3. Machine learning: computational statistics for prediction (out-of-sample properties) Paradigm shift the dimensionality of data grows, enables richer models Open-ended questions ⇒ large # features From parameter inference to prediction x y G Varoquaux 3
  • 4. Machine learning: computational statistics for prediction (out-of-sample properties) Paradigm shift the dimensionality of data grows, enables richer models Open-ended questions ⇒ large # features From parameter inference to prediction x y Understanding, not predicting Danger of solving the wrong problem Lost in formalization G Varoquaux 3
  • 5. Statistics Machine learning Statistical machine learning Hypothesis testing Prediction T-test Tests on prediction Cross-validation G Varoquaux 4
  • 6. Statistics Machine learning Statistical machine learning Hypothesis testing Prediction T-test Tests on prediction Cross-validation In sample Out of sample G Varoquaux 4
  • 7. Statistics Machine learning Statistical machine learning Hypothesis testing Prediction T-test Tests on prediction Cross-validation In sample Out of sample Parametric Non-parametric G Varoquaux 4
  • 8. Statistics Machine learning Statistical machine learning Hypothesis testing Prediction T-test Tests on prediction Cross-validation In sample Out of sample Parametric Non-parametric Non-parametric tests Probabilistic modeling Few parameters Many parameters G Varoquaux 4
  • 9. Statistics Machine learning Statistical machine learning Hypothesis testing Prediction T-test Tests on prediction Cross-validation In sample Out of sample Parametric Non-parametric Non-parametric tests Probabilistic modeling Few parameters Many parameters Univariate Multivariate G Varoquaux 4
  • 10. Statistics Machine learning Statistical machine learning Hypothesis testing Prediction T-test Tests on prediction Cross-validation In sample Out of sample Parametric Non-parametric Non-parametric tests Probabilistic modeling Few parameters Many parameters Univariate Multivariate GLM = correlations Naive Bayes Univariate selection Differences mostly cultural: it’s a continuum G Varoquaux 4
  • 11. Cognitive neuroimaging and machine learning G Varoquaux 5
  • 12. Cognitive neuroimaging and machine learning Predicting the task: decoding G Varoquaux 5
  • 13. Cognitive neuroimaging and machine learning Predicting neural response: encoding G Varoquaux 5
  • 14. Cognitive neuroimaging and machine learning Unsupervised learning on brain activity G Varoquaux 5
  • 15. Cognitive neuroimaging and machine learning Unsupervised learning on behavior G Varoquaux 5
  • 16. Cognitive neuroimaging and machine learning G Varoquaux 5
  • 17. Rest of this talk 1 Encoding 2 Decoding G Varoquaux 6
  • 18. 1 Encoding Towards richer models of brain activity G Varoquaux 7
  • 19. 1 Uncovering neural coding Insights on breaking down cognitive functions into atomic steps [Hubel and Wiesel 1962] Neurons receptive to Gabors (edges) G Varoquaux 8
  • 20. 1 Uncovering neural coding Insights on breaking down cognitive functions into atomic steps [Hubel and Wiesel 1962] Neurons receptive to Gabors (edges) [Logothetis... 1995] Shapes in inferior temporal cortex G Varoquaux 8
  • 21. 1 Uncovering neural coding: richer models Insights on breaking down cognitive functions into atomic steps [Hubel and Wiesel 1962] Neurons receptive to Gabors (edges) [Logothetis... 1995] Shapes in inferior temporal cortex Machine learning: computer-vision models mapped to brain activity [Yamins... 2014] G Varoquaux 8
  • 22. 1 Uncovering neural coding: in fMRI Model-based fMRI [O’Doherty... 2007] [Harvey... 2013] High-level descriptions [Mitchell... 2008] Natural stimuly [Kay... 2008] G Varoquaux 9
  • 23. Machine learning for encoding models Richer models of encoding capture fine descriptions of behavior / stimuli Require to forgo the contrast methodolgy Is this a good or a bad thing? G Varoquaux 10
  • 24. 1 Models of the visual system Image V1 cortex V2 cortex Inferior temporal cortex Fusiform face area Jack? Is there a “face” region? A “foot” region? A “left big toe” region? G Varoquaux 11
  • 25. 1 Uncovering neural coding: cognitive oppositions Is there a “face” region? A “foot” region? A “left big toe” region? vs G Varoquaux 12
  • 26. 1 Uncovering neural coding: cognitive oppositions Is there a “face” region? A “foot” region? A “left big toe” region? vs G Varoquaux 12
  • 27. 1 Uncovering neural coding: cognitive oppositions Is there a “face” region? A “foot” region? A “left big toe” region? vs - G Varoquaux 12
  • 28. 1 Uncovering neural coding: cognitive oppositions Is there a “face” region? A “foot” region? A “left big toe” region? vs -Mapping relies on cognitive subtraction Bound to mental process decomposition G Varoquaux 12
  • 29. 1 Decomposing visual stimuli Low-level visual cortex is tuned to natural image statistics [Olshausen et al. 1996] What drives high-level representations? G Varoquaux 13
  • 30. 1 Decomposing visual stimuli Low-level visual cortex is tuned to natural image statistics [Olshausen et al. 1996] What drives high-level representations? Convolutional Net G Varoquaux 13
  • 31. Data-driven encoding models Image V1 cortex V2 cortex Inferior temporal cortex Fusiform face area Jack? [Khaligh-Razavi and Kriegeskorte 2014, Güçlü and van Gerven 2015] FMRI beyond a handfull of contrasts ⇒ Sets us free from the paradigm G Varoquaux 14
  • 32. 2 Decoding From brain activity to behavior G Varoquaux 15
  • 33. 2 Increased sensitivity “Given the goal of detecting the presence of a particular mental representation in the brain, the primary advantage of MVPA methods over individual-voxel-based methods is increased sensitivity.” — [Norman... 2006] G Varoquaux 16
  • 34. 2 Increased sensitivity An omnibus test “Given the goal of detecting the presence of a particular mental representation in the brain, the primary advantage of MVPA methods over individual-voxel-based methods is increased sensitivity.” — [Norman... 2006] Is there “information” about a stimuli in a given region? G Varoquaux 16
  • 35. 2 Increased sensitivity An omnibus test “Given the goal of detecting the presence of a particular mental representation in the brain, the primary advantage of MVPA methods over individual-voxel-based methods is increased sensitivity.” — [Norman... 2006] “However, these maps are not guaranteed to include all the voxels that are involved in representing the categories of interest.” — [Norman... 2006] G Varoquaux 16
  • 36. Non-linear cognitive model Linear predictive models Representations Stimuli 2 Increased sensitivity An omnibus test Decoding used to test / compare encoding models [Naselaris... 2011] G Varoquaux 17
  • 37. 2 Generalization as a test: cross-validation x y x y High-dimensional models G Varoquaux 18
  • 38. 2 Generalization as a test: cross-validation x y x y High-dimensional models ⇒ Important to test on independent data, to control for model complexity G Varoquaux 18
  • 39. 2 Generalization as a test: cross-validation High-dimensional models ⇒ Important to test on independent data, to control for model complexity ­40% ­20% ­10%  0% +10% +20% +40% Leave one sample out Leave one subject/session 20% left­out,   3 splits 20% left­out,   10 splits 20% left­out,   50 splits ­22% +19% +3% +43% ­10% +10% ­21% +17% ­11% +11% ­24% +16% ­9% +9% ­24% +14% ­9% +8% ­23% +13%   Intra subject   Inter subject No silver bullet Poster 3829, Oral Th 12:45 G Varoquaux 18
  • 40. 2 Behavioral predictions as a test Increase “cognitive resolution” One voxel’s information is not enough to distinguish many cognitive states ⇒ analysis combining info across voxels G Varoquaux 19
  • 41. 2 Behavioral predictions as a test Increase “cognitive resolution” One voxel’s information is not enough to distinguish many cognitive states ⇒ analysis combining info across voxels Interpreting overlapping activations Psychology not interested in where a task is creating activation, but if two tasks are creating activations in same areas G Varoquaux 19
  • 42. 2 Inference in cognitive neuroimaging What is the neural support of a function? What is function of a given brain module? G Varoquaux 20
  • 43. 2 Inference in cognitive neuroimaging What is the neural support of a function? What is function of a given brain module? Brain mapping = task-evoked activity G Varoquaux 20
  • 44. 2 Inference in cognitive neuroimaging [Poldrack 2006, Henson 2006] What is the neural support of a function? What is function of a given brain module? Reverse inference Brain mapping = task-evoked activity + crafting “contrasts” to isolate effects G Varoquaux 20
  • 45. 2 Inference in cognitive neuroimaging [Kanwisher... 1997, Gauthier... 2000, Hanson and Halchenko 2008] What is the neural support of a function? What is function of a given brain module? Reverse inference Is there a face area? G Varoquaux 20
  • 46. 2 Inference in cognitive neuroimaging [Poldrack... 2009, Schwartz... 2013] What is the neural support of a function? What is function of a given brain module? Reverse inference Decoding: Find regions that predict observed cognition G Varoquaux 20
  • 47. 2 Decoding for reverse inference [Poldrack... 2009, Schwartz... 2013] Prediction = proxy for implication Need large cognitive coverage G Varoquaux 21
  • 48. 2 Decoding for reverse inference [Poldrack... 2009, Schwartz... 2013] Prediction = proxy for implication Need large cognitive coverage Interpretation of the “grandmother neuron” “more than a neuron re- sponds to one concept and [...] neurons do not neces- sarily respond to only one concept are given by the data itself [Quian Quiroga and Kreiman 2010] G Varoquaux 21
  • 49. 2 Brain decoding with linear models Design matrix × Coefficients = Coefficients are brain maps Target G Varoquaux 22
  • 50. 2 Brain decoding to recover predictive regions? Face vs house visual recognition [Haxby... 2001] SVM error: 26% G Varoquaux 23
  • 51. 2 Brain decoding to recover predictive regions? Face vs house visual recognition [Haxby... 2001] Sparse model error: 19% G Varoquaux 23
  • 52. 2 Brain decoding to recover predictive regions? Face vs house visual recognition [Haxby... 2001] Ridge error: 15% Best predictor outlines the worst regions Best maps predict worst G Varoquaux 23
  • 53. 2 Decoders as estimators [Gramfort... 2013] Inverse problem Minimize the error term: ˆw = argmin w l(y − X w) Ill-posed: Many different w will give the same prediction error Choice driven by (implicit) priors of the decoder SVM sparse ridge TV- 1 G Varoquaux 24
  • 54. 2 Decoders as estimators [Gramfort... 2013] Inverse problem Minimize the error term: ˆw = argmin w l(y − X w) Ill-posed: Many different w will give the same prediction error Choice driven by (implicit) priors of the decoder SVM sparse ridge TV- 1 Inferences rely, explicitely or implicitely, on the regions estimated by the decoder G Varoquaux 24
  • 56. @GaelVaroquaux Machine learning for cognitive neuroimaging The description of cognition is hard ⇒ Encoding Rich models depend less on paradigms
  • 57. @GaelVaroquaux Machine learning for cognitive neuroimaging The description of cognition is hard ⇒ Encoding Decoding as an omnibus test For rich encoding models To interpret overlaping activation Cross-validation error bars
  • 58. @GaelVaroquaux Machine learning for cognitive neuroimaging The description of cognition is hard ⇒ Encoding Decoding as an omnibus test Decoding for reverse inference Requires large cognitive coverage
  • 59. @GaelVaroquaux Machine learning for cognitive neuroimaging The description of cognition is hard ⇒ Encoding Decoding as an omnibus test Decoding for reverse inference Estimation of predictive regions is difficult Infinite number of maps predict as well
  • 60. @GaelVaroquaux Machine learning for cognitive neuroimaging The description of cognition is hard ⇒ Encoding Decoding as an omnibus test Decoding for reverse inference Estimation of predictive regions is difficult Software: nilearn In Python http://nilearn.github.io ni [Varoquaux and Thirion 2014] How machine learning is shaping cognitive neuroimaging
  • 61. References I I. Gauthier, M. J. Tarr, J. Moylan, P. Skudlarski, J. C. Gore, and A. W. Anderson. The fusiform “face area” is part of a network that processes faces at the individual level. J cognitive neuroscience, 12:495, 2000. A. Gramfort, B. Thirion, and G. Varoquaux. Identifying predictive regions from fMRI with TV-L1 prior. In PRNI, page 17, 2013. U. Güçlü and M. A. van Gerven. Deep neural networks reveal a gradient in the complexity of neural representations across the ventral stream. The Journal of Neuroscience, 35(27): 10005–10014, 2015. S. J. Hanson and Y. O. Halchenko. Brain reading using full brain support vector machines for object recognition: there is no “face” identification area. Neural Computation, 20:486, 2008. B. Harvey, B. Klein, N. Petridou, and S. Dumoulin. Topographic representation of numerosity in the human parietal cortex. Science, 341(6150):1123–1126, 2013.
  • 62. References II J. V. Haxby, I. M. Gobbini, M. L. Furey, ... Distributed and overlapping representations of faces and objects in ventral temporal cortex. Science, 293:2425, 2001. R. Henson. Forward inference using functional neuroimaging: Dissociations versus associations. Trends in cognitive sciences, 10:64, 2006. D. H. Hubel and T. N. Wiesel. Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. The Journal of physiology, 160:106, 1962. N. Kanwisher, J. McDermott, and M. M. Chun. The fusiform face area: a module in human extrastriate cortex specialized for face perception. J Neuroscience, 17:4302, 1997. K. N. Kay, T. Naselaris, R. J. Prenger, and J. L. Gallant. Identifying natural images from human brain activity. Nature, 452:352, 2008.
  • 63. References III S.-M. Khaligh-Razavi and N. Kriegeskorte. Deep supervised, but not unsupervised, models may explain it cortical representation. PLoS Comput Biol, 10(11):e1003915, 2014. N. K. Logothetis, J. Pauls, and T. Poggio. Shape representation in the inferior temporal cortex of monkeys. Current Biology, 5:552, 1995. D. Marr. Vision: A computational investigation into the human representation and processing of visual information. The MIT press, Cambridge, 1982. T. M. Mitchell, S. V. Shinkareva, A. Carlson, K.-M. Chang, V. L. Malave, R. A. Mason, and M. A. Just. Predicting human brain activity associated with the meanings of nouns. science, 320: 1191, 2008. T. Naselaris, K. N. Kay, S. Nishimoto, and J. L. Gallant. Encoding and decoding in fMRI. Neuroimage, 56:400, 2011.
  • 64. References IV K. A. Norman, S. M. Polyn, G. J. Detre, and J. V. Haxby. Beyond mind-reading: multi-voxel pattern analysis of fmri data. Trends in cognitive sciences, 10:424, 2006. J. P. O’Doherty, A. Hampton, and H. Kim. Model-based fMRI and its application to reward learning and decision making. Annals of the New York Academy of Sciences, 1104:35, 2007. B. Olshausen ... Emergence of simple-cell remainsceptive field properties by learning a sparse code for natural images. Nature, 381:607, 1996. R. Poldrack. Can cognitive processes be inferred from neuroimaging data? Trends in cognitive sciences, 10:59, 2006. R. A. Poldrack, Y. O. Halchenko, and S. J. Hanson. Decoding the large-scale structure of brain function by classifying mental states across individuals. Psychological Science, 20:1364, 2009.
  • 65. References V R. Quian Quiroga and G. Kreiman. Postscript: About grandmother cells and jennifer aniston neurons. Psychological Review, 117: 297, 2010. Y. Schwartz, B. Thirion, and G. Varoquaux. Mapping cognitive ontologies to and from the brain. In NIPS, 2013. G. Varoquaux and B. Thirion. How machine learning is shaping cognitive neuroimaging. GigaScience, 3:28, 2014. D. L. Yamins, H. Hong, C. F. Cadieu, E. A. Solomon, D. Seibert, and J. J. DiCarlo. Performance-optimized hierarchical models predict neural responses in higher visual cortex. Proc Natl Acad Sci, page 201403112, 2014.