SlideShare a Scribd company logo

Graph Signal Processing: an interpretable framework to link neurocognitive architectures with brain activity

N
Nicolas Farrugia

This document discusses the integration of graph signal processing (GSP) in neurocognitive architectures to interpret brain activity, emphasizing the role of graph theory in analyzing brain connectivity. It outlines the motivations for using GSP, its applications in neuroimaging, and methods for analyzing brain oscillations and connectivity through various metrics. The content highlights predictive approaches using GSP-derived feature vectors and the potential for generating new hypotheses linking temporal and graph frequencies.

Science
1 of 56
Downloaded 24 times
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
Graph Signal Processing: an interpretable framework to link neurocognitive architectures with brain activity Nicolas Farrugia June 4th, 2019 IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 1 / 26
Prelude Signal processing in Neuroscience Interpreting (oscillatory) brain activity using Fourier or Wavelet transforms. Network Neuroscience Applying graph theory metrics to study functional and structural connectivity. Graph Signal Processing for Cognitive Neurosciences The best of both worlds ? IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 2 / 26
Prelude Signal processing in Neuroscience Interpreting (oscillatory) brain activity using Fourier or Wavelet transforms. Network Neuroscience Applying graph theory metrics to study functional and structural connectivity. Graph Signal Processing for Cognitive Neurosciences The best of both worlds ? IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 2 / 26
Prelude Signal processing in Neuroscience Interpreting (oscillatory) brain activity using Fourier or Wavelet transforms. Network Neuroscience Applying graph theory metrics to study functional and structural connectivity. Graph Signal Processing for Cognitive Neurosciences The best of both worlds ? IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 2 / 26
1 Motivations Functional role of brain oscillations From brain regions to network neuroscience Networks and connectome harmonics / gradients 2 Graph Signal Processing for Neuroimaging Introduction to Graph Signal Processing Inference using GSP derived measures Predictive approaches using GSP derived feature vectors GSP and interpretability IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 3 / 26
Spectral (time-frequency) analysis in cognitive neuroscience Hans Berger and early measurements of electro-encephalography -> Observations on alpha (7.5 to 12.5 Hz) and beta (12.5 to 25 Hz) frequency bands (and later, gamma frequency band from 25 to 40 Hz) IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 4 / 26
Spectral (time-frequency) analysis in cognitive neuroscience Fast-forward to 21st century... Functional role of delta-theta phase in sensory prediction. Arnal and Giraud, 2012, Trends in Cognitive Sciences IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 4 / 26
Spectral (time-frequency) analysis in cognitive neuroscience Fast-forward to 21st century... Phase and cross-frequency coupling: Sauseng et al. 2008, Neuroscience and Biobehavioral reviews See also : non-sinusoidal oscillations (Cole and Woytek, 2017, TICS) IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 4 / 26
1 Motivations Functional role of brain oscillations From brain regions to network neuroscience Networks and connectome harmonics / gradients 2 Graph Signal Processing for Neuroimaging Introduction to Graph Signal Processing Inference using GSP derived measures Predictive approaches using GSP derived feature vectors GSP and interpretability IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 5 / 26
From regions to networks... Primary visual cortex IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 6 / 26
From regions to networks... Coactivation of brain areas - networks Resting-state networks - spontaneous and task activity. IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 6 / 26
The network neuroscience revolution Fornito et al. 2015, Nat. Neuro. IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 7 / 26
From networks to spectral graph analysis Connectome harmonics - spectral analysis of structural connectivity. Atasoy et al. 2016, Nat. Comms IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 8 / 26
From networks to spectral graph analysis Connectivity gradients - Spectral analysis of functional connectivity. Margulies et al. 2016, PNAS IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 8 / 26
From networks to spectral graph analysis Functional interpretation of connectivity gradients. Margulies et al. 2016, PNAS IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 8 / 26
1 Motivations Functional role of brain oscillations From brain regions to network neuroscience Networks and connectome harmonics / gradients 2 Graph Signal Processing for Neuroimaging Introduction to Graph Signal Processing Inference using GSP derived measures Predictive approaches using GSP derived feature vectors GSP and interpretability IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 9 / 26
What is Graph Signal Processing ? GSP in four steps Define a graph with N nodes, Compute Graph Laplacian L (e.g. L = D − A) and find its eigenvectors U Define a signal x with N values, one per node. Graph Fourier transform is defined by ˆx = U x Analogy with (Discrete) Fourier Transform Shuman et al. 2013, IEEE Signal Processing Magazine IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 10 / 26
Why using GSP for Neuroimaging ? A straightforward use case Brain signals on graphs built on structural (white matter) connectivity. Huang et al. 2018 IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 11 / 26
Why using GSP for Neuroimaging ? A straightforward use case Brain signals on graphs built on structural (white matter) connectivity. Elegance Network-centric tools adapted to the problem Applicable to both inference-based and predictive approaches Interpretations in term of graph frequencies IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 11 / 26
Why using GSP for Neuroimaging ? A straightforward use case Brain signals on graphs built on structural (white matter) connectivity. Elegance Network-centric tools adapted to the problem Applicable to both inference-based and predictive approaches Interpretations in term of graph frequencies IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 11 / 26
1 Motivations Functional role of brain oscillations From brain regions to network neuroscience Networks and connectome harmonics / gradients 2 Graph Signal Processing for Neuroimaging Introduction to Graph Signal Processing Inference using GSP derived measures Predictive approaches using GSP derived feature vectors GSP and interpretability IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 12 / 26
Inference using GSP metrics What measures are authors currently using ? Useful measures on graph signals Smoothness (x Lx), also called Dirichlet energy, Mutual information / F measure of GFT decompositions, Concentration in certain frequency bands (Alignement / Liberality) IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 13 / 26
Inference using GSP metrics Huang et al. 2018 IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 13 / 26
Inference using GSP metrics Statistical methods : group-wise inference Dependent variables derived from GSP statistically related to behavior. Smoothness / Dirichlet energy (Smith et al. 2017) Alignement and Liberality related to attention switching (Huang et al., 2018) IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 13 / 26
Inference using GSP metrics Resting state networks predicted by connectome harmonics. Atasoy et al. 2016, Nat. Comms IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 13 / 26
1 Motivations Functional role of brain oscillations From brain regions to network neuroscience Networks and connectome harmonics / gradients 2 Graph Signal Processing for Neuroimaging Introduction to Graph Signal Processing Inference using GSP derived measures Predictive approaches using GSP derived feature vectors GSP and interpretability IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 14 / 26
Predictive approach using GSP Proposed approach Graph nodes defined by a brain parcellation Graph edges defined by functional or structural connectivity Feature Extraction on brain activity using GSP (e.g. GFT or Graph Wavelets) Feature selection Supervised learning (using an interpretable method) Ménoret, Farrugia, Pasdeloup and Gripon, 2017, GlobalSIP Brahim and Farrugia, 2019, GRETSI Pilavci and Farrugia, 2019, ICASSP IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 15 / 26
Predictive approach using GSP Proposed approach Graph nodes defined by a brain parcellation Graph edges defined by functional or structural connectivity Feature Extraction on brain activity using GSP (e.g. GFT or Graph Wavelets) Feature selection Supervised learning (using an interpretable method) Ménoret, Farrugia, Pasdeloup and Gripon, 2017, GlobalSIP Brahim and Farrugia, 2019, GRETSI Pilavci and Farrugia, 2019, ICASSP IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 15 / 26
Predictive approach using GSP Proposed approach Graph nodes defined by a brain parcellation Graph edges defined by functional or structural connectivity Feature Extraction on brain activity using GSP (e.g. GFT or Graph Wavelets) Feature selection Supervised learning (using an interpretable method) Ménoret, Farrugia, Pasdeloup and Gripon, 2017, GlobalSIP Brahim and Farrugia, 2019, GRETSI Pilavci and Farrugia, 2019, ICASSP IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 15 / 26
Predictive approach using GSP Proposed approach Graph nodes defined by a brain parcellation Graph edges defined by functional or structural connectivity Feature Extraction on brain activity using GSP (e.g. GFT or Graph Wavelets) Feature selection Supervised learning (using an interpretable method) Ménoret, Farrugia, Pasdeloup and Gripon, 2017, GlobalSIP Brahim and Farrugia, 2019, GRETSI Pilavci and Farrugia, 2019, ICASSP IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 15 / 26
Graph Fourier Transform on resting state data Brahim and Farrugia, submitted IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 16 / 26
Graph Fourier Transform on resting state data Resuls on ADHD200 dataset. 5 10 15 20 25 30 35 40 45 Features 0.0 0.2 0.4 0.6 0.8 1.0 Accuracy STD+SG STD FC EC CC NS Mean Mean+SG 5 10 15 20 25 30 35 40 45 Features 0.0 0.5 1.0 1.5 2.0 2.5 -log(P) 1.0 2.5 Brahim and Farrugia, submitted IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 16 / 26
Graph Fourier Transform on resting state data ADHD-200 Approaches Acc (%) Sen (%) Spe (%) STD 52.5±0.037 55±0.043 55±0.066 STD+GSP 85±0.019 85±0.036 85±0.036 FC 65±0.032 60±0.059 70±0.063 ABIDE NYU Langone Approaches Acc (%) Sen (%) Spe (%) STD 66.65±0.007 63.75±0.015 74.44±0.010 STD+GSP 70.36±0.007 67.68±0.011 75.67±0.012 FC 62.83±0.006 56.61±0.014 70.22±0.015 Table: Maximum classification rates of the different approaches for ADHD-200 and NYU databases using SVM classifier with linear kernel (max ± standard error of the mean). Brahim and Farrugia, submitted IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 16 / 26
Graph Fourier Transform on resting state data Feature Selection (results from ABIDE NYU Langone) Top: Ratios of selected ROIs across 10 folds, using STD (Accuracy 66.65±0.007 ). Bottom : Average of selected GFT modes across 10 folds, using STD + GFT (Accuracy 70.36±0.007) Brahim and Farrugia, submitted IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 16 / 26
Spectral Graph Wavelet Transform (SGWT) Wavelet atoms are obtained by: ψs,a = ΣN n=1g(sλn )ˆδ(n)un (1) where g(.) is the kernel function, s is the scale, ˆδ is the graph Fourier transform of dirac delta and un are the columns of U. SGWT of f is obtained using the inner product Wf (s, a) =< ψs,a , f >. Optimizing spectral coverage using Warped Translate kernel Pilavci and Farrugia, 2019, ICASSP IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 17 / 26
Spectral Graph Wavelet Transform (SGWT) We tested this approach using an averaged functional connectivity graph parcellated on 523 regions of BASC Atlas Usings signals from : A synthetic regression problem on a brain graph fMRI data from Chang et al., predicting affective rating of emotional pictures Pilavci and Farrugia, 2019, ICASSPIMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 17 / 26
Spectral Graph Wavelet Transform (SGWT) Pilavci and Farrugia, 2019, ICASSP IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 17 / 26
1 Motivations Functional role of brain oscillations From brain regions to network neuroscience Networks and connectome harmonics / gradients 2 Graph Signal Processing for Neuroimaging Introduction to Graph Signal Processing Inference using GSP derived measures Predictive approaches using GSP derived feature vectors GSP and interpretability IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 18 / 26
GSP and Interpretability Inference Generate and test hypothesis on graph frequency bands (similar to M/EEG oscillations) Spatial contributions of Graph Fourier modes Prediction / data-driven Visualize GSP features or contributions of Grapˆh Fourier modes Use of interpretable ML methods with GSP features as inputs IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 19 / 26
GSP and Interpretability Inference Generate and test hypothesis on graph frequency bands (similar to M/EEG oscillations) Spatial contributions of Graph Fourier modes Prediction / data-driven Visualize GSP features or contributions of Grapˆh Fourier modes Use of interpretable ML methods with GSP features as inputs IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 19 / 26
Generating new hypothesis using GSP (1/2) Hypothesis using Frequencies What would such an hypothesis look like ? In the field of brain oscillations, empirical knowledge has been built using frequency bands. Should we start to make hypothesis on graph frequencies ? Measures of (temporal) excursions from liberality or alignement (Huang et al. 2018) - linking temporal frequencies and graph frequencies Can we directly make hypothesis on Graph fourier modes ? IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 20 / 26
Generating new hypothesis using GSP (1/2) Hypothesis using Frequencies What would such an hypothesis look like ? In the field of brain oscillations, empirical knowledge has been built using frequency bands. Should we start to make hypothesis on graph frequencies ? Measures of (temporal) excursions from liberality or alignement (Huang et al. 2018) - linking temporal frequencies and graph frequencies Can we directly make hypothesis on Graph fourier modes ? IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 20 / 26
Generating new hypothesis using GSP (1/2) Hypothesis using Frequencies What would such an hypothesis look like ? In the field of brain oscillations, empirical knowledge has been built using frequency bands. Should we start to make hypothesis on graph frequencies ? Measures of (temporal) excursions from liberality or alignement (Huang et al. 2018) - linking temporal frequencies and graph frequencies Can we directly make hypothesis on Graph fourier modes ? IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 20 / 26
Generating new hypothesis using GSP (2/2) Hypothesis using Harmonics In previous slides we have shown that propagation (diffusion) patterns: Were functionally meaningful, Could predict spontaneous activity. Analysis of experimental data could exploit decompositions based on Harmonics Could we exploit (connectome) harmonics to understand spatiotemporal dynamics ? IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 21 / 26
Generating new hypothesis using GSP (2/2) Hypothesis using Harmonics In previous slides we have shown that propagation (diffusion) patterns: Were functionally meaningful, Could predict spontaneous activity. Analysis of experimental data could exploit decompositions based on Harmonics Could we exploit (connectome) harmonics to understand spatiotemporal dynamics ? IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 21 / 26
Generating new hypothesis using GSP (2/2) Hypothesis using Harmonics In previous slides we have shown that propagation (diffusion) patterns: Were functionally meaningful, Could predict spontaneous activity. Analysis of experimental data could exploit decompositions based on Harmonics Could we exploit (connectome) harmonics to understand spatiotemporal dynamics ? IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 21 / 26
What is missing: Dynamic (hyper?) graphs Estimating a graph measure at each time point may not be valid (Basset and Sizemore 2017. Ongoing debate on time(?)-varying functional connectivity using sliding windows(e.g. Hindriks et al. 2016) Building dynamic graph and associated measures using slepians ? (Huang et al. 2018), Preti and Van De Ville 2017 IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 22 / 26
What is missing: Dynamic (hyper?) graphs Estimating a graph measure at each time point may not be valid (Basset and Sizemore 2017. Ongoing debate on time(?)-varying functional connectivity using sliding windows(e.g. Hindriks et al. 2016) Building dynamic graph and associated measures using slepians ? (Huang et al. 2018), Preti and Van De Ville 2017 IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 22 / 26
What is missing: Dynamic (hyper?) graphs Estimating a graph measure at each time point may not be valid (Basset and Sizemore 2017. Ongoing debate on time(?)-varying functional connectivity using sliding windows(e.g. Hindriks et al. 2016) Building dynamic graph and associated measures using slepians ? (Huang et al. 2018), Preti and Van De Ville 2017 IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 22 / 26
What is missing: Dynamic (hyper?) graphs Estimating a graph measure at each time point may not be valid (Basset and Sizemore 2017. Ongoing debate on time(?)-varying functional connectivity using sliding windows(e.g. Hindriks et al. 2016) Building dynamic graph and associated measures using slepians ? (Huang et al. 2018), Preti and Van De Ville 2017 IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 22 / 26
Conclusions and perspectives The promising part: Connectome harmonics are relevant, Certain graph frequency may be useful to generate new hypothesis, GSP may help levarage explanatory power in small datasets. The difficult part... Brain signals (oscillations) are highly non-linear and non-stationnary, Dynamic graphs are needed to integrate both the complex spatial dynamics and non-stationary aspects, Unique opportunity of collaborative research between Neuroscience and AI! IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 23 / 26
Conclusions and perspectives The promising part: Connectome harmonics are relevant, Certain graph frequency may be useful to generate new hypothesis, GSP may help levarage explanatory power in small datasets. The difficult part... Brain signals (oscillations) are highly non-linear and non-stationnary, Dynamic graphs are needed to integrate both the complex spatial dynamics and non-stationary aspects, Unique opportunity of collaborative research between Neuroscience and AI! IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 23 / 26
Conclusions and perspectives The promising part: Connectome harmonics are relevant, Certain graph frequency may be useful to generate new hypothesis, GSP may help levarage explanatory power in small datasets. The difficult part... Brain signals (oscillations) are highly non-linear and non-stationnary, Dynamic graphs are needed to integrate both the complex spatial dynamics and non-stationary aspects, Unique opportunity of collaborative research between Neuroscience and AI! IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 23 / 26
Thanks! Questions? Thanks to Vincent Gripon, Abdelbasset Brahim, Yusuf Yigit Pilavci, Mathilde Ménoret and Bastien Pasdeloup. IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 24 / 26
Prediction and Inference "Inference typically focuses on isolating variables deemed individually important above some chance level, often based on p values." "Prediction commonly aims at identifying variable sets that together enable accurate guessing of outcomes based on other or future data." Bzdok and Ioannidis. 2019, Trends in Neurosciences IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 25 / 26
Graph Fourier Transform on task fMRI data Feature Extraction using GSP Supervised learning with dimensionality reduction using Graph Fourier Transform and Graph Sampling (Menoret et al. 2017) Table: Comparison of Graph Sampling (Semilocal graph), PCA, ICA and ANOVA. Classification accuracy with 50 components for the Simulated and Haxby datasets. Method Simulation Haxby Easy Difficult Face-House Cat-Face PCA 88.8% 65.5% 82.7% 63.6% ICA 90.2% 65.3% 84.4% 67.0% ANOVA 92.1% 67.3% 85.5% 65.5% Graph sampling 90.9% 72.5% 88.2% 69.0% Ménoret, Farrugia, Pasdeloup and Gripon, 2017, GlobalSIP IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 26 / 26

More Related Content

PDF
Lec17 sparse signal processing & applications
United States Air Force Academy
 
PPTX
Attention in Deep Learning
健程 杨
 
PDF
Cognitive computing
Priyanshi Jain
 
PPTX
What Is A Neural Network? | How Deep Neural Networks Work | Neural Network Tu...
Simplilearn
 
PPTX
Statistics - SoftMax Equation
Andrew Ferlitsch
 
PPTX
Deep neural networks
Si Haem
 
PDF
A tutorial in Connectome Analysis (1) - Marcus Kaiser
Lake Como School of Advanced Studies
 
PDF
A survey of deep learning approaches to medical applications
Joseph Paul Cohen PhD
 
Lec17 sparse signal processing & applications
United States Air Force Academy
 
Attention in Deep Learning
健程 杨
 
Cognitive computing
Priyanshi Jain
 
What Is A Neural Network? | How Deep Neural Networks Work | Neural Network Tu...
Simplilearn
 
Statistics - SoftMax Equation
Andrew Ferlitsch
 
Deep neural networks
Si Haem
 
A tutorial in Connectome Analysis (1) - Marcus Kaiser
Lake Como School of Advanced Studies
 
A survey of deep learning approaches to medical applications
Joseph Paul Cohen PhD
 

What's hot (20)

PPTX
Introduction Of Artificial neural network
Nagarajan
 
PDF
Deep learning and Healthcare
Thomas da Silva Paula
 
PDF
CNN Attention Networks
Taeoh Kim
 
PDF
Deep Learning for Graphs
DeepLearningBlr
 
PPTX
CNN Machine learning DeepLearning
Abhishek Sharma
 
PPTX
Machine learning for wireless networks @Bestcom2016
Merima Kulin
 
PPTX
Comparison of Learning Algorithms for Handwritten Digit Recognition
Safaa Alnabulsi
 
PPTX
NS-CUK Joint Journal Club: V.T.Hoang, Review on "Heterogeneous Graph Attentio...
ssuser4b1f48
 
PDF
Neural Networks: Introducton
Mostafa G. M. Mostafa
 
PDF
Controlled dropout: a different dropout for improving training speed on deep ...
Byung Soo Ko
 
PPTX
Neural network
Ramesh Giri
 
PDF
Notes from Coursera Deep Learning courses by Andrew Ng
dataHacker. rs
 
PDF
Pr045 deep lab_semantic_segmentation
Taeoh Kim
 
PPTX
neural networks
joshiblog
 
PPTX
Deep Learning Tutorial
Amr Rashed
 
PDF
4 Dimensionality reduction (PCA & t-SNE)
Dmytro Fishman
 
PDF
Nature-Inspired Optimization Algorithms
Xin-She Yang
 
PPTX
Graph Representation Learning
Jure Leskovec
 
PDF
Artificial neural network for machine learning
grinu
 
PDF
Handwritten Digit Recognition
ijtsrd
 
Introduction Of Artificial neural network
Nagarajan
 
Deep learning and Healthcare
Thomas da Silva Paula
 
CNN Attention Networks
Taeoh Kim
 
Deep Learning for Graphs
DeepLearningBlr
 
CNN Machine learning DeepLearning
Abhishek Sharma
 
Machine learning for wireless networks @Bestcom2016
Merima Kulin
 
Comparison of Learning Algorithms for Handwritten Digit Recognition
Safaa Alnabulsi
 
NS-CUK Joint Journal Club: V.T.Hoang, Review on "Heterogeneous Graph Attentio...
ssuser4b1f48
 
Neural Networks: Introducton
Mostafa G. M. Mostafa
 
Controlled dropout: a different dropout for improving training speed on deep ...
Byung Soo Ko
 
Neural network
Ramesh Giri
 
Notes from Coursera Deep Learning courses by Andrew Ng
dataHacker. rs
 
Pr045 deep lab_semantic_segmentation
Taeoh Kim
 
neural networks
joshiblog
 
Deep Learning Tutorial
Amr Rashed
 
4 Dimensionality reduction (PCA & t-SNE)
Dmytro Fishman
 
Nature-Inspired Optimization Algorithms
Xin-She Yang
 
Graph Representation Learning
Jure Leskovec
 
Artificial neural network for machine learning
grinu
 
Handwritten Digit Recognition
ijtsrd
 
Ad

Similar to Graph Signal Processing: an interpretable framework to link neurocognitive architectures with brain activity (20)

PDF
Evaluating Graph Signal Processing for Neuroimaging Through Classification an...
Nicolas Farrugia
 
PDF
Graph Signal Processing for Machine Learning A Review and New Perspectives - ...
lauratoni4
 
PDF
Graph Signal Processing for Machine Learning A Review and New Perspectives - ...
lauratoni4
 
PDF
Better neuroimaging data processing: driven by evidence, open communities, an...
Gael Varoquaux
 
PDF
End-to-End Stroke Imaging Analysis Using Effective Connectivity and Interpret...
AlessandroCrimi3
 
PDF
DTI brain networks analysis
emapesce
 
PDF
Neural networks in the brain
Ilya Zakharov
 
PDF
Brain Signal Analysis Advances in Neuroelectric and Neuromagnetic Methods 1St...
lashencawva
 
PPTX
[NS][Lab_Seminar_241028]BrainGNN: Interpretable Brain Graph Neural Network fo...
thanhdowork
 
PDF
Graph Neural Networks in Brain Connectivity Analysis
SohithReddy9
 
PPTX
common spatial pattern of the brain.pptx
Neeraj Singh
 
PDF
A survey based on eeg classification
ijitjournal
 
PDF
Bradley Voytek - Berkeley Cognitive Neuroscience 2018
UC San Diego
 
PDF
Classification of electroencephalography using cooperative learning based on...
IJECEIAES
 
PPTX
Invited_Reyes_BME_Basel_March2025v2.pptx
Mauricio Reyes
 
PDF
E44082429
IJERA Editor
 
PDF
Overview of Machine Learning and Deep Learning Methods in Brain Computer Inte...
IJCSES Journal
 
PDF
Ontheclassificationof ee gsignalbyusingansvmbasedalgorythm
Karthik S
 
PPTX
Jessica Rudd, PhD Student, Analytics and Data Science, Kennesaw State Univers...
MLconf
 
PDF
fNIRS and Brain Computer Interface for Communication
InsideScientific
 
Evaluating Graph Signal Processing for Neuroimaging Through Classification an...
Nicolas Farrugia
 
Graph Signal Processing for Machine Learning A Review and New Perspectives - ...
lauratoni4
 
Graph Signal Processing for Machine Learning A Review and New Perspectives - ...
lauratoni4
 
Better neuroimaging data processing: driven by evidence, open communities, an...
Gael Varoquaux
 
End-to-End Stroke Imaging Analysis Using Effective Connectivity and Interpret...
AlessandroCrimi3
 
DTI brain networks analysis
emapesce
 
Neural networks in the brain
Ilya Zakharov
 
Brain Signal Analysis Advances in Neuroelectric and Neuromagnetic Methods 1St...
lashencawva
 
[NS][Lab_Seminar_241028]BrainGNN: Interpretable Brain Graph Neural Network fo...
thanhdowork
 
Graph Neural Networks in Brain Connectivity Analysis
SohithReddy9
 
common spatial pattern of the brain.pptx
Neeraj Singh
 
A survey based on eeg classification
ijitjournal
 
Bradley Voytek - Berkeley Cognitive Neuroscience 2018
UC San Diego
 
Classification of electroencephalography using cooperative learning based on...
IJECEIAES
 
Invited_Reyes_BME_Basel_March2025v2.pptx
Mauricio Reyes
 
E44082429
IJERA Editor
 
Overview of Machine Learning and Deep Learning Methods in Brain Computer Inte...
IJCSES Journal
 
Ontheclassificationof ee gsignalbyusingansvmbasedalgorythm
Karthik S
 
Jessica Rudd, PhD Student, Analytics and Data Science, Kennesaw State Univers...
MLconf
 
fNIRS and Brain Computer Interface for Communication
InsideScientific
 
Ad

Recently uploaded (20)

PPTX
The KM-GBF monitoring framework – status & key messages.pptx
pensoftservices
 
PPTX
famous lake in india and its disturibution and importance
judesebina
 
PDF
An interstellar mission to test astrophysical black holes
Sérgio Sacani
 
PDF
Placing the Near-Earth Object Impact Probability in Context
Sérgio Sacani
 
PPTX
TOTAL hIP ARTHROPLASTY Presentation.pptx
hararghemssc
 
PDF
Biophysics 2.pdffffffffffffffffffffffffff
MiraMusleh
 
PDF
AlphaEarth Foundations and the Satellite Embedding dataset
gdgforscience
 
PDF
HPLC-PPT.docx high performance liquid chromatography
darshanambiga1633
 
PDF
Sciences of Europe No 170 (2025)
Sciences of Europe
 
PPTX
GEN. BIO 1 - CELL TYPES & CELL MODIFICATIONS
breeysal7
 
PDF
bbec55_b34400a7914c42429908233dbd381773.pdf
agambirev
 
PPTX
Introduction to Fisheries Biotechnology_Lesson 1.pptx
Esther Pongcol
 
PDF
SEHH2274 Organic Chemistry Notes 1 Structure and Bonding.pdf
ivan1108lau
 
PDF
IFIT3 RNA-binding activity primores influenza A viruz infection and translati...
saracelre
 
PDF
ELS_Q1_Module-11_Formation-of-Rock-Layers_v2.pdf
savannaesar07
 
PPTX
2. Earth - The Living Planet Module 2ELS
markjustinebarolobau
 
PPTX
DRUG THERAPY FOR SHOCK gjjjgfhhhhh.pptx.
vipvivek704
 
PPTX
ECG_Course_Presentation د.محمد صقران ppt
lelouchml1995
 
PPTX
neck nodes and dissection types and lymph nodes levels
DipuSarma
 
PPTX
EPIDURAL ANESTHESIA ANATOMY AND PHYSIOLOGY.pptx
PrabakaranMP1
 
The KM-GBF monitoring framework – status & key messages.pptx
pensoftservices
 
famous lake in india and its disturibution and importance
judesebina
 
An interstellar mission to test astrophysical black holes
Sérgio Sacani
 
Placing the Near-Earth Object Impact Probability in Context
Sérgio Sacani
 
TOTAL hIP ARTHROPLASTY Presentation.pptx
hararghemssc
 
Biophysics 2.pdffffffffffffffffffffffffff
MiraMusleh
 
AlphaEarth Foundations and the Satellite Embedding dataset
gdgforscience
 
HPLC-PPT.docx high performance liquid chromatography
darshanambiga1633
 
Sciences of Europe No 170 (2025)
Sciences of Europe
 
GEN. BIO 1 - CELL TYPES & CELL MODIFICATIONS
breeysal7
 
bbec55_b34400a7914c42429908233dbd381773.pdf
agambirev
 
Introduction to Fisheries Biotechnology_Lesson 1.pptx
Esther Pongcol
 
SEHH2274 Organic Chemistry Notes 1 Structure and Bonding.pdf
ivan1108lau
 
IFIT3 RNA-binding activity primores influenza A viruz infection and translati...
saracelre
 
ELS_Q1_Module-11_Formation-of-Rock-Layers_v2.pdf
savannaesar07
 
2. Earth - The Living Planet Module 2ELS
markjustinebarolobau
 
DRUG THERAPY FOR SHOCK gjjjgfhhhhh.pptx.
vipvivek704
 
ECG_Course_Presentation د.محمد صقران ppt
lelouchml1995
 
neck nodes and dissection types and lymph nodes levels
DipuSarma
 
EPIDURAL ANESTHESIA ANATOMY AND PHYSIOLOGY.pptx
PrabakaranMP1
 

Graph Signal Processing: an interpretable framework to link neurocognitive architectures with brain activity

  • 1. Graph Signal Processing: an interpretable framework to link neurocognitive architectures with brain activity Nicolas Farrugia June 4th, 2019 IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 1 / 26
  • 2. Prelude Signal processing in Neuroscience Interpreting (oscillatory) brain activity using Fourier or Wavelet transforms. Network Neuroscience Applying graph theory metrics to study functional and structural connectivity. Graph Signal Processing for Cognitive Neurosciences The best of both worlds ? IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 2 / 26
  • 3. Prelude Signal processing in Neuroscience Interpreting (oscillatory) brain activity using Fourier or Wavelet transforms. Network Neuroscience Applying graph theory metrics to study functional and structural connectivity. Graph Signal Processing for Cognitive Neurosciences The best of both worlds ? IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 2 / 26
  • 4. Prelude Signal processing in Neuroscience Interpreting (oscillatory) brain activity using Fourier or Wavelet transforms. Network Neuroscience Applying graph theory metrics to study functional and structural connectivity. Graph Signal Processing for Cognitive Neurosciences The best of both worlds ? IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 2 / 26
  • 5. 1 Motivations Functional role of brain oscillations From brain regions to network neuroscience Networks and connectome harmonics / gradients 2 Graph Signal Processing for Neuroimaging Introduction to Graph Signal Processing Inference using GSP derived measures Predictive approaches using GSP derived feature vectors GSP and interpretability IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 3 / 26
  • 6. Spectral (time-frequency) analysis in cognitive neuroscience Hans Berger and early measurements of electro-encephalography -> Observations on alpha (7.5 to 12.5 Hz) and beta (12.5 to 25 Hz) frequency bands (and later, gamma frequency band from 25 to 40 Hz) IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 4 / 26
  • 7. Spectral (time-frequency) analysis in cognitive neuroscience Fast-forward to 21st century... Functional role of delta-theta phase in sensory prediction. Arnal and Giraud, 2012, Trends in Cognitive Sciences IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 4 / 26
  • 8. Spectral (time-frequency) analysis in cognitive neuroscience Fast-forward to 21st century... Phase and cross-frequency coupling: Sauseng et al. 2008, Neuroscience and Biobehavioral reviews See also : non-sinusoidal oscillations (Cole and Woytek, 2017, TICS) IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 4 / 26
  • 9. 1 Motivations Functional role of brain oscillations From brain regions to network neuroscience Networks and connectome harmonics / gradients 2 Graph Signal Processing for Neuroimaging Introduction to Graph Signal Processing Inference using GSP derived measures Predictive approaches using GSP derived feature vectors GSP and interpretability IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 5 / 26
  • 10. From regions to networks... Primary visual cortex IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 6 / 26
  • 11. From regions to networks... Coactivation of brain areas - networks Resting-state networks - spontaneous and task activity. IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 6 / 26
  • 12. The network neuroscience revolution Fornito et al. 2015, Nat. Neuro. IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 7 / 26
  • 13. From networks to spectral graph analysis Connectome harmonics - spectral analysis of structural connectivity. Atasoy et al. 2016, Nat. Comms IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 8 / 26
  • 14. From networks to spectral graph analysis Connectivity gradients - Spectral analysis of functional connectivity. Margulies et al. 2016, PNAS IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 8 / 26
  • 15. From networks to spectral graph analysis Functional interpretation of connectivity gradients. Margulies et al. 2016, PNAS IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 8 / 26
  • 16. 1 Motivations Functional role of brain oscillations From brain regions to network neuroscience Networks and connectome harmonics / gradients 2 Graph Signal Processing for Neuroimaging Introduction to Graph Signal Processing Inference using GSP derived measures Predictive approaches using GSP derived feature vectors GSP and interpretability IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 9 / 26
  • 17. What is Graph Signal Processing ? GSP in four steps Define a graph with N nodes, Compute Graph Laplacian L (e.g. L = D − A) and find its eigenvectors U Define a signal x with N values, one per node. Graph Fourier transform is defined by ˆx = U x Analogy with (Discrete) Fourier Transform Shuman et al. 2013, IEEE Signal Processing Magazine IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 10 / 26
  • 18. Why using GSP for Neuroimaging ? A straightforward use case Brain signals on graphs built on structural (white matter) connectivity. Huang et al. 2018 IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 11 / 26
  • 19. Why using GSP for Neuroimaging ? A straightforward use case Brain signals on graphs built on structural (white matter) connectivity. Elegance Network-centric tools adapted to the problem Applicable to both inference-based and predictive approaches Interpretations in term of graph frequencies IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 11 / 26
  • 20. Why using GSP for Neuroimaging ? A straightforward use case Brain signals on graphs built on structural (white matter) connectivity. Elegance Network-centric tools adapted to the problem Applicable to both inference-based and predictive approaches Interpretations in term of graph frequencies IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 11 / 26
  • 21. 1 Motivations Functional role of brain oscillations From brain regions to network neuroscience Networks and connectome harmonics / gradients 2 Graph Signal Processing for Neuroimaging Introduction to Graph Signal Processing Inference using GSP derived measures Predictive approaches using GSP derived feature vectors GSP and interpretability IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 12 / 26
  • 22. Inference using GSP metrics What measures are authors currently using ? Useful measures on graph signals Smoothness (x Lx), also called Dirichlet energy, Mutual information / F measure of GFT decompositions, Concentration in certain frequency bands (Alignement / Liberality) IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 13 / 26
  • 23. Inference using GSP metrics Huang et al. 2018 IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 13 / 26
  • 24. Inference using GSP metrics Statistical methods : group-wise inference Dependent variables derived from GSP statistically related to behavior. Smoothness / Dirichlet energy (Smith et al. 2017) Alignement and Liberality related to attention switching (Huang et al., 2018) IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 13 / 26
  • 25. Inference using GSP metrics Resting state networks predicted by connectome harmonics. Atasoy et al. 2016, Nat. Comms IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 13 / 26
  • 26. 1 Motivations Functional role of brain oscillations From brain regions to network neuroscience Networks and connectome harmonics / gradients 2 Graph Signal Processing for Neuroimaging Introduction to Graph Signal Processing Inference using GSP derived measures Predictive approaches using GSP derived feature vectors GSP and interpretability IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 14 / 26
  • 27. Predictive approach using GSP Proposed approach Graph nodes defined by a brain parcellation Graph edges defined by functional or structural connectivity Feature Extraction on brain activity using GSP (e.g. GFT or Graph Wavelets) Feature selection Supervised learning (using an interpretable method) Ménoret, Farrugia, Pasdeloup and Gripon, 2017, GlobalSIP Brahim and Farrugia, 2019, GRETSI Pilavci and Farrugia, 2019, ICASSP IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 15 / 26
  • 28. Predictive approach using GSP Proposed approach Graph nodes defined by a brain parcellation Graph edges defined by functional or structural connectivity Feature Extraction on brain activity using GSP (e.g. GFT or Graph Wavelets) Feature selection Supervised learning (using an interpretable method) Ménoret, Farrugia, Pasdeloup and Gripon, 2017, GlobalSIP Brahim and Farrugia, 2019, GRETSI Pilavci and Farrugia, 2019, ICASSP IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 15 / 26
  • 29. Predictive approach using GSP Proposed approach Graph nodes defined by a brain parcellation Graph edges defined by functional or structural connectivity Feature Extraction on brain activity using GSP (e.g. GFT or Graph Wavelets) Feature selection Supervised learning (using an interpretable method) Ménoret, Farrugia, Pasdeloup and Gripon, 2017, GlobalSIP Brahim and Farrugia, 2019, GRETSI Pilavci and Farrugia, 2019, ICASSP IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 15 / 26
  • 30. Predictive approach using GSP Proposed approach Graph nodes defined by a brain parcellation Graph edges defined by functional or structural connectivity Feature Extraction on brain activity using GSP (e.g. GFT or Graph Wavelets) Feature selection Supervised learning (using an interpretable method) Ménoret, Farrugia, Pasdeloup and Gripon, 2017, GlobalSIP Brahim and Farrugia, 2019, GRETSI Pilavci and Farrugia, 2019, ICASSP IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 15 / 26
  • 31. Graph Fourier Transform on resting state data Brahim and Farrugia, submitted IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 16 / 26
  • 32. Graph Fourier Transform on resting state data Resuls on ADHD200 dataset. 5 10 15 20 25 30 35 40 45 Features 0.0 0.2 0.4 0.6 0.8 1.0 Accuracy STD+SG STD FC EC CC NS Mean Mean+SG 5 10 15 20 25 30 35 40 45 Features 0.0 0.5 1.0 1.5 2.0 2.5 -log(P) 1.0 2.5 Brahim and Farrugia, submitted IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 16 / 26
  • 33. Graph Fourier Transform on resting state data ADHD-200 Approaches Acc (%) Sen (%) Spe (%) STD 52.5±0.037 55±0.043 55±0.066 STD+GSP 85±0.019 85±0.036 85±0.036 FC 65±0.032 60±0.059 70±0.063 ABIDE NYU Langone Approaches Acc (%) Sen (%) Spe (%) STD 66.65±0.007 63.75±0.015 74.44±0.010 STD+GSP 70.36±0.007 67.68±0.011 75.67±0.012 FC 62.83±0.006 56.61±0.014 70.22±0.015 Table: Maximum classification rates of the different approaches for ADHD-200 and NYU databases using SVM classifier with linear kernel (max ± standard error of the mean). Brahim and Farrugia, submitted IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 16 / 26
  • 34. Graph Fourier Transform on resting state data Feature Selection (results from ABIDE NYU Langone) Top: Ratios of selected ROIs across 10 folds, using STD (Accuracy 66.65±0.007 ). Bottom : Average of selected GFT modes across 10 folds, using STD + GFT (Accuracy 70.36±0.007) Brahim and Farrugia, submitted IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 16 / 26
  • 35. Spectral Graph Wavelet Transform (SGWT) Wavelet atoms are obtained by: ψs,a = ΣN n=1g(sλn )ˆδ(n)un (1) where g(.) is the kernel function, s is the scale, ˆδ is the graph Fourier transform of dirac delta and un are the columns of U. SGWT of f is obtained using the inner product Wf (s, a) =< ψs,a , f >. Optimizing spectral coverage using Warped Translate kernel Pilavci and Farrugia, 2019, ICASSP IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 17 / 26
  • 36. Spectral Graph Wavelet Transform (SGWT) We tested this approach using an averaged functional connectivity graph parcellated on 523 regions of BASC Atlas Usings signals from : A synthetic regression problem on a brain graph fMRI data from Chang et al., predicting affective rating of emotional pictures Pilavci and Farrugia, 2019, ICASSPIMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 17 / 26
  • 37. Spectral Graph Wavelet Transform (SGWT) Pilavci and Farrugia, 2019, ICASSP IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 17 / 26
  • 38. 1 Motivations Functional role of brain oscillations From brain regions to network neuroscience Networks and connectome harmonics / gradients 2 Graph Signal Processing for Neuroimaging Introduction to Graph Signal Processing Inference using GSP derived measures Predictive approaches using GSP derived feature vectors GSP and interpretability IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 18 / 26
  • 39. GSP and Interpretability Inference Generate and test hypothesis on graph frequency bands (similar to M/EEG oscillations) Spatial contributions of Graph Fourier modes Prediction / data-driven Visualize GSP features or contributions of Grapˆh Fourier modes Use of interpretable ML methods with GSP features as inputs IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 19 / 26
  • 40. GSP and Interpretability Inference Generate and test hypothesis on graph frequency bands (similar to M/EEG oscillations) Spatial contributions of Graph Fourier modes Prediction / data-driven Visualize GSP features or contributions of Grapˆh Fourier modes Use of interpretable ML methods with GSP features as inputs IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 19 / 26
  • 41. Generating new hypothesis using GSP (1/2) Hypothesis using Frequencies What would such an hypothesis look like ? In the field of brain oscillations, empirical knowledge has been built using frequency bands. Should we start to make hypothesis on graph frequencies ? Measures of (temporal) excursions from liberality or alignement (Huang et al. 2018) - linking temporal frequencies and graph frequencies Can we directly make hypothesis on Graph fourier modes ? IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 20 / 26
  • 42. Generating new hypothesis using GSP (1/2) Hypothesis using Frequencies What would such an hypothesis look like ? In the field of brain oscillations, empirical knowledge has been built using frequency bands. Should we start to make hypothesis on graph frequencies ? Measures of (temporal) excursions from liberality or alignement (Huang et al. 2018) - linking temporal frequencies and graph frequencies Can we directly make hypothesis on Graph fourier modes ? IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 20 / 26
  • 43. Generating new hypothesis using GSP (1/2) Hypothesis using Frequencies What would such an hypothesis look like ? In the field of brain oscillations, empirical knowledge has been built using frequency bands. Should we start to make hypothesis on graph frequencies ? Measures of (temporal) excursions from liberality or alignement (Huang et al. 2018) - linking temporal frequencies and graph frequencies Can we directly make hypothesis on Graph fourier modes ? IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 20 / 26
  • 44. Generating new hypothesis using GSP (2/2) Hypothesis using Harmonics In previous slides we have shown that propagation (diffusion) patterns: Were functionally meaningful, Could predict spontaneous activity. Analysis of experimental data could exploit decompositions based on Harmonics Could we exploit (connectome) harmonics to understand spatiotemporal dynamics ? IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 21 / 26
  • 45. Generating new hypothesis using GSP (2/2) Hypothesis using Harmonics In previous slides we have shown that propagation (diffusion) patterns: Were functionally meaningful, Could predict spontaneous activity. Analysis of experimental data could exploit decompositions based on Harmonics Could we exploit (connectome) harmonics to understand spatiotemporal dynamics ? IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 21 / 26
  • 46. Generating new hypothesis using GSP (2/2) Hypothesis using Harmonics In previous slides we have shown that propagation (diffusion) patterns: Were functionally meaningful, Could predict spontaneous activity. Analysis of experimental data could exploit decompositions based on Harmonics Could we exploit (connectome) harmonics to understand spatiotemporal dynamics ? IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 21 / 26
  • 47. What is missing: Dynamic (hyper?) graphs Estimating a graph measure at each time point may not be valid (Basset and Sizemore 2017. Ongoing debate on time(?)-varying functional connectivity using sliding windows(e.g. Hindriks et al. 2016) Building dynamic graph and associated measures using slepians ? (Huang et al. 2018), Preti and Van De Ville 2017 IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 22 / 26
  • 48. What is missing: Dynamic (hyper?) graphs Estimating a graph measure at each time point may not be valid (Basset and Sizemore 2017. Ongoing debate on time(?)-varying functional connectivity using sliding windows(e.g. Hindriks et al. 2016) Building dynamic graph and associated measures using slepians ? (Huang et al. 2018), Preti and Van De Ville 2017 IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 22 / 26
  • 49. What is missing: Dynamic (hyper?) graphs Estimating a graph measure at each time point may not be valid (Basset and Sizemore 2017. Ongoing debate on time(?)-varying functional connectivity using sliding windows(e.g. Hindriks et al. 2016) Building dynamic graph and associated measures using slepians ? (Huang et al. 2018), Preti and Van De Ville 2017 IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 22 / 26
  • 50. What is missing: Dynamic (hyper?) graphs Estimating a graph measure at each time point may not be valid (Basset and Sizemore 2017. Ongoing debate on time(?)-varying functional connectivity using sliding windows(e.g. Hindriks et al. 2016) Building dynamic graph and associated measures using slepians ? (Huang et al. 2018), Preti and Van De Ville 2017 IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 22 / 26
  • 51. Conclusions and perspectives The promising part: Connectome harmonics are relevant, Certain graph frequency may be useful to generate new hypothesis, GSP may help levarage explanatory power in small datasets. The difficult part... Brain signals (oscillations) are highly non-linear and non-stationnary, Dynamic graphs are needed to integrate both the complex spatial dynamics and non-stationary aspects, Unique opportunity of collaborative research between Neuroscience and AI! IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 23 / 26
  • 52. Conclusions and perspectives The promising part: Connectome harmonics are relevant, Certain graph frequency may be useful to generate new hypothesis, GSP may help levarage explanatory power in small datasets. The difficult part... Brain signals (oscillations) are highly non-linear and non-stationnary, Dynamic graphs are needed to integrate both the complex spatial dynamics and non-stationary aspects, Unique opportunity of collaborative research between Neuroscience and AI! IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 23 / 26
  • 53. Conclusions and perspectives The promising part: Connectome harmonics are relevant, Certain graph frequency may be useful to generate new hypothesis, GSP may help levarage explanatory power in small datasets. The difficult part... Brain signals (oscillations) are highly non-linear and non-stationnary, Dynamic graphs are needed to integrate both the complex spatial dynamics and non-stationary aspects, Unique opportunity of collaborative research between Neuroscience and AI! IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 23 / 26
  • 54. Thanks! Questions? Thanks to Vincent Gripon, Abdelbasset Brahim, Yusuf Yigit Pilavci, Mathilde Ménoret and Bastien Pasdeloup. IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 24 / 26
  • 55. Prediction and Inference "Inference typically focuses on isolating variables deemed individually important above some chance level, often based on p values." "Prediction commonly aims at identifying variable sets that together enable accurate guessing of outcomes based on other or future data." Bzdok and Ioannidis. 2019, Trends in Neurosciences IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 25 / 26
  • 56. Graph Fourier Transform on task fMRI data Feature Extraction using GSP Supervised learning with dimensionality reduction using Graph Fourier Transform and Graph Sampling (Menoret et al. 2017) Table: Comparison of Graph Sampling (Semilocal graph), PCA, ICA and ANOVA. Classification accuracy with 50 components for the Simulated and Haxby datasets. Method Simulation Haxby Easy Difficult Face-House Cat-Face PCA 88.8% 65.5% 82.7% 63.6% ICA 90.2% 65.3% 84.4% 67.0% ANOVA 92.1% 67.3% 85.5% 65.5% Graph sampling 90.9% 72.5% 88.2% 69.0% Ménoret, Farrugia, Pasdeloup and Gripon, 2017, GlobalSIP IMT-Atlantique GSP and interpretability in Neuroimaging June 4th, 2019 26 / 26