SlideShare a Scribd company logo

MAIN Conf Talk: Learning representations from neural signals

A
agramfort

The document discusses automatic sleep stage classification from polysomnography (PSG) data using deep learning methods. PSG data contains multimodal time series signals including electroencephalography (EEG), electrooculography (EOG), and electromyography (EMG). The objective is to learn a function that can classify each time point into a sleep stage (awake, REM, stage 1, stage 2, etc.) using the raw PSG signals as input. Deep neural networks have shown promising results on this task compared to traditional machine learning and signal processing methods. The document reviews recent literature on using convolutional neural networks and other deep learning approaches for sleep stage classification from EEG data.

Science
1 of 107
Downloaded 15 times
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
Alexandre Gramfort alexandre.gramfort@inria.fr Learning representations from neural signals MAIN conf. - Dec. 2019
Alex Gramfort Learning representations from neural signals Neurons as current generators 2 Current Large cortical pyramidal cells organized in macro-assemblies with their dendrites normally oriented to the local cortical surface White matter Gray matter
Alex Gramfort Learning representations from neural signals Neurons as current generators 2 Current Large cortical pyramidal cells organized in macro-assemblies with their dendrites normally oriented to the local cortical surface White matter Gray matter Q = I × d (10 to 100 nAm) with the equivalent current dipole (ECD) model B Magnetic Field Neural Current (post synaptic) Equivalent Current Dipole E Electric Field Neural Current (post synaptic) Equivalent Current Dipole
Alex Gramfort Learning representations from neural signals Electro- & Magneto-encephalography 3 B Neural Current (post synaptic) Equivalent Current Dipole dB/dz MEG recordings First EEG recordings in 1929 by H. Berger Hôpital La Timone Marseille, France E Neural Current (post synaptic) Equivalent Current Dipole V EEG recordings
Alex Gramfort Learning representations from neural signals Stereotaxic EEG (sEEG) 4 Intracranial electrodes; 5 to 15 contacts per electrode Around 10 electrodes are implanted Stereotaxic Implantation
MAIN Conf Talk: Learning representations from neural signals
μ rhythm
[T. Dupré laTour, L.Tallot, L. Grabot,V. Doyère,V. vanWassenhove,Y. Grenier,A. Gramfort, (2017) PLOS Computational biology] μ rhythm
[T. Dupré laTour, L.Tallot, L. Grabot,V. Doyère,V. vanWassenhove,Y. Grenier,A. Gramfort, (2017) PLOS Computational biology] μ rhythm CFC: High frequency bursts coupled with slow waves
[T. Dupré laTour, L.Tallot, L. Grabot,V. Doyère,V. vanWassenhove,Y. Grenier,A. Gramfort, (2017) PLOS Computational biology] μ rhythm CFC: High frequency bursts coupled with slow waves Neural signals exhibit diverse and complex morphologies
Alex Gramfort Learning representations from neural signals “Textbook” brain rythms 6 Gamma (> 25 Hz) Beta (12-25 Hz) Alpha (8-12 Hz) Theta (4-8 Hz) Delta (1-4 Hz)
Alex Gramfort Learning representations from neural signals Fourier Fallacy [Jasper 48] 7 Power Spectra:Waveform:Topography:
Alex Gramfort Learning representations from neural signals Fourier Fallacy [Jasper 48] 7 «Even though it may be possible to analyze the complex forms of brain waves into a number of different sine-wave frequencies, this may lead only to what might be termed a “Fourier fallacy”, if one assumes ad hoc that all of the necessary frequencies actually occur as periodic phenomena in cell groups within the brain. » Jasper, 1948 Power Spectra:Waveform:Topography:
Alex Gramfort Learning representations from neural signals Fourier Fallacy [Jasper 48] 7 «Even though it may be possible to analyze the complex forms of brain waves into a number of different sine-wave frequencies, this may lead only to what might be termed a “Fourier fallacy”, if one assumes ad hoc that all of the necessary frequencies actually occur as periodic phenomena in cell groups within the brain. » Jasper, 1948 Harmonics Power Spectra:Waveform:Topography:
Alex Gramfort Learning representations from neural signals Outline 8 S. Chambon,V.Thorey, P. J.Arnal, E. Mignot,A. Gramfort. (2018), DOSED: a deep learning approach to detect multiple sleep micro-events in EEG signal,Arxiv 2500163 S. Chambon, M. Galtier, P.Arnal, G.Wainrib, A. Gramfort (2018),A deep learning architecture for temporal sleep stage classification using multivariate and multimodal time series, IEEE Trans. Neural Systems and Rehabilitation Engineering T. Dupré la Tour,  T. Moreau, M. Jas,A. Gramfort (2018), Multivariate Convolutional Sparse Coding for Electromagnetic Brain Signals, Proc. NeurIPS Conf. M. Jas,T. Dupré la Tour, U. Simsekli,A. Gramfort (2017), Learning the Morphology of Brain Signals Using Alpha-Stable Convolutional Sparse Coding, Proc. NeurIPS Conf. Polysomnography exam: I Electroencephalography (EEG) I Electrooculography (EOG) I Electromyography (EMG) I · · · Preliminary exam in sleep studies: I sleep disorders I psychiatric disorders... Polysomnographie “Deep” supervised learning on sleep EEG data: Unsupervised convolutional sparse coding:
Deep supervised learning on sleep EEG data 1 Spindle K-complex
Alex Gramfort Learning representations from neural signals Polysomnography (PSG) • Clinical exam • Electrophysiological signals 10 Electro-oculography (EOG) Electro-encephalography (EEG) Electro-myography (EMG)
Alex Gramfort Learning representations from neural signals Polysomnography (PSG) • Clinical exam • Electrophysiological signals 10 Electro-oculography (EOG) Electro-encephalography (EEG) Electro-myography (EMG) Routinely annotated by sleep experts
Alex Gramfort Learning representations from neural signals 2 types of annotations Hypnogram
 of sleep stages Micro-events: Spindles, K-complex etc. 11
Alex Gramfort Learning representations from neural signals 2 types of annotations Hypnogram
 of sleep stages Micro-events: Spindles, K-complex etc. 11 Classification problem
Alex Gramfort Learning representations from neural signals 2 types of annotations Hypnogram
 of sleep stages Micro-events: Spindles, K-complex etc. 11 Classification problem Joint detection and classification problem
MAIN Conf Talk: Learning representations from neural signals
Alex Gramfort Learning representations from neural signals Objective 13 PSG data
Alex Gramfort Learning representations from neural signals Objective 13 x 2 X PSG data
Alex Gramfort Learning representations from neural signals Objective 13 Learn: ˆf : X ! Y Y = {Awake, REM, Stage 1, Stage 2, etc.} x 2 X PSG data
Alex Gramfort Learning representations from neural signals Objective 13 Learn: ˆf : X ! Y Y = {Awake, REM, Stage 1, Stage 2, etc.} x 2 X PSG data Multiclass prediction problem Input is multiple time series
Alex Gramfort Learning representations from neural signals and many more…. An old yet timely problem 14 C. Berthomier, X. Drouot, M. Herman-Stoïca, et al. ,“Automatic analysis of single-channel sleep EEG: validation in healthy individuals.,” Sleep, vol. 30, no. 11, pp. 1587–1595, 2007. O.Tsinalis, P. M. Matthews, andY. Guo,“Automatic Sleep Stage Scoring Using Time-Frequency Analysis and Stacked Sparse Autoencoders,” Annals of Biomedical Engineering, vol. 44, no. 5, pp. 1587–1597, 2016. J. B. Stephansen, A.Ambati, E. B. Leary, et al.,“The use of neural networks in the analysis of sleep stages and the diagnosis of narcolepsy,” CoRR, vol. abs/1710.02094, 2017. A.Vilamala, K. H. Madsen, and L. K. Hansen,“Deep convolutional neural networks for interpretable analysis of EEG sleep stage scoring,” 2017 IEEE Workshop on Machine Learning for Signal Processing (MLSP) S.Biswal, J.Kulas, H.Sun, B.Goparaju, M.B.Westover, M.T.Bianchi and J. Sun,“SLEEPNET: automated sleep staging system via deep learning,” arXiv:1707.08262, 2017. H. Dong,A. Supratak,W. Pan, C.Wu, P. M. Matthews, andY. Guo,“Mixed Neural Network Approach for Temporal Sleep Stage Classification,” arXiv:1610.06421v1, vol. 1, 2016. O.Tsinalis, P.M. Matthews,Y. Guo and S.Zafeiriou,“Automatic Sleep Stage Scoring with Single-Channel EEG Using Convolutional Neural Networks,” arXiv:1610.01683, pp. 1–10, 2016. A. Supratak, H. Dong, C.Wu, andY. Guo,“Deepsleepnet: a model for automatic sleep stage scoring based on raw single-channel EEG,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2017. T. Lajnef, S. Chaibi, P. Ruby, et al. ,“Learning Machines and Sleeping Brains:Automatic Sleep Stage Classification using Decision-Tree Multi-Class SupportVector Machines,” Journal of Neuroscience Methods, 2015.
Alex Gramfort Learning representations from neural signals and many more…. An old yet timely problem 14 C. Berthomier, X. Drouot, M. Herman-Stoïca, et al. ,“Automatic analysis of single-channel sleep EEG: validation in healthy individuals.,” Sleep, vol. 30, no. 11, pp. 1587–1595, 2007. O.Tsinalis, P. M. Matthews, andY. Guo,“Automatic Sleep Stage Scoring Using Time-Frequency Analysis and Stacked Sparse Autoencoders,” Annals of Biomedical Engineering, vol. 44, no. 5, pp. 1587–1597, 2016. J. B. Stephansen, A.Ambati, E. B. Leary, et al.,“The use of neural networks in the analysis of sleep stages and the diagnosis of narcolepsy,” CoRR, vol. abs/1710.02094, 2017. A.Vilamala, K. H. Madsen, and L. K. Hansen,“Deep convolutional neural networks for interpretable analysis of EEG sleep stage scoring,” 2017 IEEE Workshop on Machine Learning for Signal Processing (MLSP) S.Biswal, J.Kulas, H.Sun, B.Goparaju, M.B.Westover, M.T.Bianchi and J. Sun,“SLEEPNET: automated sleep staging system via deep learning,” arXiv:1707.08262, 2017. H. Dong,A. Supratak,W. Pan, C.Wu, P. M. Matthews, andY. Guo,“Mixed Neural Network Approach for Temporal Sleep Stage Classification,” arXiv:1610.06421v1, vol. 1, 2016. O.Tsinalis, P.M. Matthews,Y. Guo and S.Zafeiriou,“Automatic Sleep Stage Scoring with Single-Channel EEG Using Convolutional Neural Networks,” arXiv:1610.01683, pp. 1–10, 2016. A. Supratak, H. Dong, C.Wu, andY. Guo,“Deepsleepnet: a model for automatic sleep stage scoring based on raw single-channel EEG,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2017. T. Lajnef, S. Chaibi, P. Ruby, et al. ,“Learning Machines and Sleeping Brains:Automatic Sleep Stage Classification using Decision-Tree Multi-Class SupportVector Machines,” Journal of Neuroscience Methods, 2015. To be compared
Alex Gramfort Learning representations from neural signals and many more…. An old yet timely problem 14 Obj: learn from raw multimodal and multivariate signals C. Berthomier, X. Drouot, M. Herman-Stoïca, et al. ,“Automatic analysis of single-channel sleep EEG: validation in healthy individuals.,” Sleep, vol. 30, no. 11, pp. 1587–1595, 2007. O.Tsinalis, P. M. Matthews, andY. Guo,“Automatic Sleep Stage Scoring Using Time-Frequency Analysis and Stacked Sparse Autoencoders,” Annals of Biomedical Engineering, vol. 44, no. 5, pp. 1587–1597, 2016. J. B. Stephansen, A.Ambati, E. B. Leary, et al.,“The use of neural networks in the analysis of sleep stages and the diagnosis of narcolepsy,” CoRR, vol. abs/1710.02094, 2017. A.Vilamala, K. H. Madsen, and L. K. Hansen,“Deep convolutional neural networks for interpretable analysis of EEG sleep stage scoring,” 2017 IEEE Workshop on Machine Learning for Signal Processing (MLSP) S.Biswal, J.Kulas, H.Sun, B.Goparaju, M.B.Westover, M.T.Bianchi and J. Sun,“SLEEPNET: automated sleep staging system via deep learning,” arXiv:1707.08262, 2017. H. Dong,A. Supratak,W. Pan, C.Wu, P. M. Matthews, andY. Guo,“Mixed Neural Network Approach for Temporal Sleep Stage Classification,” arXiv:1610.06421v1, vol. 1, 2016. O.Tsinalis, P.M. Matthews,Y. Guo and S.Zafeiriou,“Automatic Sleep Stage Scoring with Single-Channel EEG Using Convolutional Neural Networks,” arXiv:1610.01683, pp. 1–10, 2016. A. Supratak, H. Dong, C.Wu, andY. Guo,“Deepsleepnet: a model for automatic sleep stage scoring based on raw single-channel EEG,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2017. T. Lajnef, S. Chaibi, P. Ruby, et al. ,“Learning Machines and Sleeping Brains:Automatic Sleep Stage Classification using Decision-Tree Multi-Class SupportVector Machines,” Journal of Neuroscience Methods, 2015. To be compared
Alex Gramfort Learning representations from neural signals Proposed approach Features • Convolutional Neural Net (CNN): Frequency content Spatial context • Multivariate signals: spatial filtering (like ICA would do) • Multimodal inputs: different pipelines for EEG, EOG and EMG Temporal context • Concatenate features of neighboring samples [A deep learning architecture for temporal sleep stage classification using multivariate and multimodal time series. S. Chambon, M. N. Galtier, P. J.Arnal, G.Wainrib,A. Gramfort. IEEE TSRNE 2018] 15
Alex Gramfort Learning representations from neural signals Problem formulation 16 ˆf 2 argminf2F Ex,y2X⇥Y [`(y, f(x))] Risk minimization:
Alex Gramfort Learning representations from neural signals Problem formulation 16 ˆf 2 argminf2F Ex,y2X⇥Y [`(y, f(x))] Risk minimization: F• : a class of models (neural networks architecture).
Alex Gramfort Learning representations from neural signals Problem formulation 16 ˆf 2 argminf2F Ex,y2X⇥Y [`(y, f(x))] Risk minimization: F• : a class of models (neural networks architecture). `• : loss.We use the categorical cross-entropy. • Minimized on a training set using backpropagation and online stochastic gradient descent.
shape = (C, T) Spatial Filtering shape = (C’, T) Spatial Filtering EEG/EOG EMG Input Conv/Relu/MP Softmax Conv/Relu/MP 𝜙C 𝜓 𝜙 𝜙T
shape = (C, T) Spatial Filtering shape = (C’, T) Spatial Filtering EEG/EOG EMG Input Conv/Relu/MP Softmax Conv/Relu/MP 𝜙C 𝜓 𝜙 𝜙T shape = (C, T) Spatial Filtering shape = (C’, T) Spatial Filtering EEG/EOG EMG Input Conv/Relu/MP Softmax Conv/Relu/MP 𝜙C 𝜓 𝜙 𝜙T
shape = (C, T) Spatial Filtering shape = (C’, T) Spatial Filtering EEG/EOG EMG Input Conv/Relu/MP Softmax Conv/Relu/MP 𝜙C 𝜓 𝜙 𝜙T shape = (C, T) Spatial Filtering shape = (C’, T) Spatial Filtering EEG/EOG EMG Input Conv/Relu/MP Softmax Conv/Relu/MP 𝜙C 𝜓 𝜙 𝜙T
shape = (C, T) Spatial Filtering shape = (C’, T) Spatial Filtering EEG/EOG EMG Input Conv/Relu/MP Softmax Conv/Relu/MP 𝜙C 𝜓 𝜙 𝜙T shape = (C, T) Spatial Filtering shape = (C’, T) Spatial Filtering EEG/EOG EMG Input Conv/Relu/MP Softmax Conv/Relu/MP 𝜙C 𝜓 𝜙 𝜙T shape = (C, T) Spatial Filtering shape = (C’, T) Spatial Filtering EEG/EOG EMG Input Conv/Relu/MP Softmax Conv/Relu/MP 𝜙C 𝜓 𝜙 𝜙T
shape = (C, T) Spatial Filtering shape = (C’, T) Spatial Filtering EEG/EOG EMG Input Conv/Relu/MP Softmax Conv/Relu/MP 𝜙C 𝜓 𝜙 𝜙T
shape = (C, T) Spatial Filtering shape = (C’, T) Spatial Filtering EEG/EOG EMG Input Conv/Relu/MP Softmax Conv/Relu/MP 𝜙C 𝜓 𝜙 𝜙T Block 1 Conv, ReLU, Max Pooling Input Block 2
Alex Gramfort Learning representations from neural signals Model parameters 23
Alex Gramfort Learning representations from neural signals Model parameters 23 Spatial filtering
Alex Gramfort Learning representations from neural signals Model parameters 23 Spatial filtering Conv. layer 1
Alex Gramfort Learning representations from neural signals Model parameters 23 Spatial filtering Conv. layer 1 Conv. layer 2
zt 𝜙 𝜓 0 30 yt
zt k zt zt+k Time (s) 𝜙𝜙 𝜙 𝜓 Sequence of inputsEEGChannels F4 F3 C3 C4 O1 O2 -60 -30 0 30 60 90 yt
zt k zt zt+k Time (s) 𝜙𝜙 𝜙 𝜓 Sequence of inputsEEGChannels F4 F3 C3 C4 O1 O2 -60 -30 0 30 60 90 yt We pool the representations from neighboring windows
Experiments…
Alex Gramfort Learning representations from neural signals Experimental setup • Data • MASS - session 3: O’Reilly et al. 2014 • http://www.ceams-carsm.ca/en/MASS • 61 records, ~10h / record • Total number of samples about 60,000 (like for MNIST) • Preprocessing • lowpass filtering (30Hz) • downsampling 128Hz 27
Alex Gramfort Learning representations from neural signals Experimental setup • Cross validation • 5 splits • 41 records for training, 10 for validation, 10 for testing • Baselines • Gradient boosting: Hand crafted features (mean, variance… relative power in frequency bands) [Lajnev et al. 2015] • [Tsinalis et al. 2016]: deep convolutional network • [Supratak et al. 2017]: deep convolutional network processing low and high frequency contents specifically 28
Alex Gramfort Learning representations from neural signals Influence of spatial context 29
Influence of temporal context 30
Influence of temporal context 30 • Temporal context helps • But less when using more channels
Alex Gramfort Learning representations from neural signals Performance vs SoTA 31 Performance: Computational efficiency:
Alex Gramfort Learning representations from neural signals Performance vs SoTA 31 Performance: Computational efficiency: • Networks outperform hand crafted features with gradient boosting • Combining all modalities helps (multivariate) • SoTA is obtained with less parameters (fast)
Alex Gramfort Learning representations from neural signals “Opening” the black box 32 Confusion matrix: N1 is hard
Alex Gramfort Learning representations from neural signals “Opening” the black box 32 Confusion matrix: N1 is hard “Hiding” frequencies from the network: network has learnt scoring rules
Start time End time Type Joint detection and classification of micro-events [A deep learning architecture to detect events in EEG signals during sleep. S. Chambon,V.Thorey, P. J.Arnal, E. Mignot,A. Gramfort. MLSP 2018] [DOSED: a deep learning approach to detect multiple sleep micro-events in EEG signal,. S. Chambon,V.Thorey, P. J.Arnal, E. Mignot,A. Gramfort. Arxiv 2500163]
Start time End time Type Joint detection and classification of micro-events Objective: Predict what and when at the same time [A deep learning architecture to detect events in EEG signals during sleep. S. Chambon,V.Thorey, P. J.Arnal, E. Mignot,A. Gramfort. MLSP 2018] [DOSED: a deep learning approach to detect multiple sleep micro-events in EEG signal,. S. Chambon,V.Thorey, P. J.Arnal, E. Mignot,A. Gramfort. Arxiv 2500163]
Alex Gramfort Learning representations from neural signals Spindles K-complex Spindles and K-complexes in N2 stage • Different types, durations, scales • Associated to certain sleep stages and sleep disorders Micro-events 34 Why is it hard? Why should you care?
Alex Gramfort Learning representations from neural signals State of the art • Signal processing: • Band-pass filtering + thresholding • Decomposition into components • Shallow learning algorithms: • Clustering • Binary classifier 35 [Parekh et al. 2017, Lajnef et al. 2017] [Ray et al. 2015,Wamsley et al. 2012,Wendt et al. 2012, Mölle et al. 2011, Nir et al. 2011, Ferrarelli et al. 2007] [Patti et al. 2017] [Patti et al. 2015, Lachner-Piza et al. 2018] None of these works use deep learning
Alex Gramfort Learning representations from neural signals DOSED algorithm Inspired by: YOLO and SSD 1. Predict: type, start & end 2. Allows for multiple types of events 3. Detection and classification in one pass 4. Convolutional network 36 [Redmon et al. 2016, Liu et al. 2015]
Alex Gramfort Learning representations from neural signals Event detector 𝜓loc : conv. layerInput 𝜙: CNN 𝜓clf : conv. layer Setup Model Default events 37
Alex Gramfort Learning representations from neural signals d1 d2 dNd Loss: localization + classification !38
Alex Gramfort Learning representations from neural signals d1 d2 dNd Default events overlapping with true events Loss: localization + classification !39
Alex Gramfort Learning representations from neural signals d1 d2 dNd Remaining default events Loss: localization + classification !40 Default events overlapping with true events
Alex Gramfort Learning representations from neural signals Evaluation •By event metrics
 
 •Based on Intersection over Union (IoU)
 •Precision, Recall, F1 Predicted event True event 41 [Warby et al. 2014] IoU < 𝜕 => False positive IoU > 𝜕 => True positive
Alex Gramfort Learning representations from neural signals 3 datasets SS2
 [O’Reilly et al. 2014] SSC
 [Andlauer et al. 2013] WSC [Young et al. 2008] # Subjects 19 26 30 Age 23.6 ± 3.7 52.2 ± 14.3 65.2 ± 8.2 Aplit
 (train, val, test) 10, 5, 4 15, 5, 6 19, 5, 6 Events spindles
 K-complexes spindles spindles 42
General benchmark
Alex Gramfort Learning representations from neural signals Illustration 44 True event Predicted event
Alex Gramfort Learning representations from neural signals Illustration 44 True event Predicted event
Alex Gramfort Learning representations from neural signals Joint detection 45 Joint detection does not deteriorate performance
Convolutional Sparse Coding (CSC) for learning the morphology of neural signals 2 Code: https://alphacsc.github.io Multivariate Convolutional Sparse Coding for Electromagnetic Brain Signals, (2018), T. Dupré laTour, T. Moreau, M. Jas,A. Gramfort, Proc. NeurIPS Conf. Learning the Morphology of Brain Signals Using Alpha-Stable Convolutional Sparse Coding, (2017), M. Jas,T. Dupré laTour, U. Simsekli,A. Gramfort, Proc. NeurIPS Conf.
Alex Gramfort Learning representations from neural signals Signal representations ▪Sparse representations: wavelet basis ▪Sparse coding / dictionary learning ▪Shift-invariant representations ▪In neurophysiology: ▪ Matching of time-invariant filters (MOTIF) ▪ Multivariate orthogonal matching pursuit ▪ Matching pursuit and heuristics ▪ Sliding window machine ▪ Adaptive waveform learning 47 [Morlet 70’, Meyer 80’, Mallat 90’ etc.] [Olshausen and Field, 1996, Elad and Aharon, 2006] [Lewicki and Sejnowski, 1999, Grosse et al, 2007] [Jost et al, 2006] [Gips et al, 2017] [Hitziger et al, 2017] [Barthélemy et al, 2012] [Brokmeier and Principe, 2016]
Alex Gramfort Learning representations from neural signals Convolutional sparse coding 48 [Grosse et al, 2007]
Alex Gramfort Learning representations from neural signals Convolutional sparse coding 49 [Grosse et al, 2007]
Alex Gramfort Learning representations from neural signals Convolutional sparse coding 50 [Grosse et al, 2007]
Alex Gramfort Learning representations from neural signals Optimization strategy 51 Block-coordinate descent:
Alex Gramfort Learning representations from neural signals Optimization strategy 51 Block-coordinate descent: [Kavukcuoglu et al, 2010] [Chalasani et al, 2013] [Bristow et al, 2013] [Wohlberg, 2016] [Jas et al, 2017] [Dupré la Tour et al, 2018] ▪ Z-step ▪ GCD ▪ FISTA ▪ ADMM ▪ ADMM + FFT ▪ L-BFGS ▪ LGCD
Alex Gramfort Learning representations from neural signals Optimization strategy 51 Block-coordinate descent: [Grosse et al, 2007] [Heide et al, 2015] [Wohlberg, 2016] [Jas et al, 2017] [Dupré la Tour et al, 2018) ▪ D-step ▪ FFT ▪ ADMM + FFT ▪ ADMM + FFT ▪ L-BFGS (dual) ▪ PGD [Kavukcuoglu et al, 2010] [Chalasani et al, 2013] [Bristow et al, 2013] [Wohlberg, 2016] [Jas et al, 2017] [Dupré la Tour et al, 2018] ▪ Z-step ▪ GCD ▪ FISTA ▪ ADMM ▪ ADMM + FFT ▪ L-BFGS ▪ LGCD
Speed benchmarks
Speed benchmarks
Alex Gramfort Learning representations from neural signals Learned atoms 53 Data: ~80 Hz
Alex Gramfort Learning representations from neural signals Learned atoms 53 Data: ~80 Hz CSC reveals CFC
Alex Gramfort Learning representations from neural signals Learned atoms 53 Data: How about if I have many channels? ~80 Hz CSC reveals CFC
Alex Gramfort Learning representations from neural signals From ICA to CSC 54 https://pypi.python.org/pypi/python-picard/0.1 Independent Component Analysis (ICA)
Alex Gramfort Learning representations from neural signals From ICA to CSC 54 https://pypi.python.org/pypi/python-picard/0.1 Independent Component Analysis (ICA) X S
Alex Gramfort Learning representations from neural signals From ICA… 55 https://pypi.python.org/pypi/python-picard/0.1 X = A S = + +… a1 aks1 sk + +… https://pierreablin.github.io/picard/auto_examples/plot_ica_eeg.html https://www.martinos.org/mne/stable/auto_tutorials/plot_artifacts_correction_ica.html
Alex Gramfort Learning representations from neural signals … … to CSC 56 https://pypi.python.org/pypi/python-picard/0.1 X = + u1 v1 ⇤ z1 uk vk ⇤ zk convolution
Alex Gramfort Learning representations from neural signals … ⇤ Topography waveform temporal activations … to CSC 56 https://pypi.python.org/pypi/python-picard/0.1 X = + u1 v1 ⇤ z1 uk vk ⇤ zk convolution
Alex Gramfort Learning representations from neural signals … ⇤ Topography waveform temporal activations … to CSC 56 https://pypi.python.org/pypi/python-picard/0.1 X = + u1 v1 ⇤ z1 uk vk ⇤ zk convolution CSC allows to learn jointly • topography (ie. localization) • signal waveform • when waveform occurs
Alex Gramfort Learning representations from neural signals Multivariate CSC 57 [Multivariate Convolutional Sparse Coding for Electromagnetic Brain Signals, (2018), T. Dupré laTour,  T. Moreau, M. Jas,A. Gramfort, Proc. NeurIPS Conf.]
Alex Gramfort Learning representations from neural signals Multivariate CSC 57 [Multivariate Convolutional Sparse Coding for Electromagnetic Brain Signals, (2018), T. Dupré laTour,  T. Moreau, M. Jas,A. Gramfort, Proc. NeurIPS Conf.]
Alex Gramfort Learning representations from neural signals Multivariate CSC 57 [Multivariate Convolutional Sparse Coding for Electromagnetic Brain Signals, (2018), T. Dupré laTour,  T. Moreau, M. Jas,A. Gramfort, Proc. NeurIPS Conf.] Rank 1 constraint:
Alex Gramfort Learning representations from neural signals CSC on MEG 58 https://pypi.python.org/pypi/python-picard/0.1 •MEG vectorview •Median nerve stim. [Multivariate Convolutional Sparse Coding for Electromagnetic Brain Signals, (2018), T. Dupré laTour,  T. Moreau, M. Jas,A. Gramfort, Proc. NeurIPS Conf.]
Alex Gramfort Learning representations from neural signals CSC on MEG 58 https://pypi.python.org/pypi/python-picard/0.1 •MEG vectorview •Median nerve stim. CSC reveals mu- shaped waveforms [Multivariate Convolutional Sparse Coding for Electromagnetic Brain Signals, (2018), T. Dupré laTour,  T. Moreau, M. Jas,A. Gramfort, Proc. NeurIPS Conf.]
Alex Gramfort Learning representations from neural signals CSC on MEG 58 https://pypi.python.org/pypi/python-picard/0.1 •MEG vectorview •Median nerve stim. See the frequency harmonics CSC reveals mu- shaped waveforms [Multivariate Convolutional Sparse Coding for Electromagnetic Brain Signals, (2018), T. Dupré laTour,  T. Moreau, M. Jas,A. Gramfort, Proc. NeurIPS Conf.]
Alex Gramfort Learning representations from neural signals CSC on MEG 59 Results on auditory task:
https://alphacsc.github.io
https://pypi.python.org/pypi/python-picard/0.1 https://alphacsc.github.io/auto_examples/multicsc/plot_sample_evoked_response.html
https://pypi.python.org/pypi/python-picard/0.1 https://alphacsc.github.io/auto_examples/multicsc/plot_sample_evoked_response.html
https://pypi.python.org/pypi/python-picard/0.1 https://alphacsc.github.io/auto_examples/multicsc/plot_sample_evoked_response.html Use permutation statistics to know if atoms are condition specific and if so when
Conclusion • SoTA convolutional network for sleep scoring on PSG data • First deep architecture for event detection on EEG data with SoTA performance • CSC model: Fast solver and multivariate • CSC comes with Python code
http://www.martinos.org/mne MNE software for processing MEG and EEG data, A. Gramfort, M. Luessi, E. Larson, D. Engemann, D. Strohmeier, C. Brodbeck, L. Parkkonen, M. Hämäläinen, Neuroimage 2013
http://www.martinos.org/mnehttp://www.martinos.org/mne Harvard U. Montreal MNE developer in 2010 MNE developers in 2018 Berkeley Paris NYU Cambridge Aalto Ilmenau Julich Sheffield U.Wash. UCSF Graz Marseille More than 80 people have contributed to MNE
>>> import mne >>> raw = mne.io.read_raw_edf('SC4001E0-PSG.edf') >>> annot = mne.read_annotations('SC4001EC-Hypnogram.edf') >>> raw.set_annotations(annot) >>> events, event_id = mne.events_from_annotations(raw) >>> epochs = mne.Epochs(raw, events, event_id, tmin=0., tmax=30., baseline=None) >>> epochs['Sleep stage W’].plot_psd(fmax=20., picks=[0, 1])) https://github.com/mne-tools/mne-python/pull/5718 Working with EEG sleep data in 7 lines of Python code
https://swipe4ica.github.io/#/ https://github.com/SwipesForScience
Thanks ! GitHub : @agramfort Twitter : @agramfort Support ERC SLAB,ANR THALAMEEG ANR-14-NEUC-0002-01 NIH R01 MH106174, DFG HA 2899/21-1. http://alexandre.gramfort.netContact S. Chambon,V.Thorey, P. J.Arnal, E. Mignot,A. Gramfort. (2018), DOSED: a deep learning approach to detect multiple sleep micro-events in EEG signal,Arxiv 2500163 T. Dupré la Tour,  T. Moreau, M. Jas,A. Gramfort, Multivariate Convolutional Sparse Coding for Electromagnetic Brain Signals, (2018), Proc. NeurIPS Conf. M. Jas,T. Dupré la Tour, U. Simsekli,A. Gramfort, Learning the Morphology of Brain Signals Using Alpha-Stable Convolutional Sparse Coding, (2017), Proc. NeurIPS Conf. Tom Dupré laTour Mainak Jas Thomas Moreau Umut Simsekli Stanislas Chambon ValentinThorey Pierrick Arnal Emmanuel Mignot Joint work with: S. Chambon, M. Galtier, P.Arnal, G.Wainrib, A. Gramfort (2018),A deep learning architecture for temporal sleep stage classification using multivariate and multimodal time series, IEEE Trans. Neural Systems and Rehabilitation Engineering

More Related Content

PDF
International Journal of Computational Engineering Research(IJCER)
ijceronline
 
PPT
Nencki321 day2
Centre of New Technologies, University of Warsaw
 
PDF
Analysis of EEG data Using ICA and Algorithm Development for Energy Comparison
ijsrd.com
 
PPT
Blue Brain_Nikhilesh+Krishna Raj
Krishna Raj .S
 
PDF
International Journal of Computational Engineering Research(IJCER)
ijceronline
 
PDF
A0510107
IOSR Journals
 
PDF
Classification of EEG Signals for Brain-Computer Interface
Azoft
 
PDF
Review:Wavelet transform based electroencephalogram methods
ijtsrd
 
International Journal of Computational Engineering Research(IJCER)
ijceronline
 
Nencki321 day2
Centre of New Technologies, University of Warsaw
 
Analysis of EEG data Using ICA and Algorithm Development for Energy Comparison
ijsrd.com
 
Blue Brain_Nikhilesh+Krishna Raj
Krishna Raj .S
 
International Journal of Computational Engineering Research(IJCER)
ijceronline
 
A0510107
IOSR Journals
 
Classification of EEG Signals for Brain-Computer Interface
Azoft
 
Review:Wavelet transform based electroencephalogram methods
ijtsrd
 

What's hot (20)

PDF
Neuron based time optimal controller of horizontal saccadic eye movements
Alireza Ghahari
 
PDF
ANALYSIS OF BRAIN COGNITIVE STATE FOR ARITHMETIC TASK AND MOTOR TASK USING EL...
sipij
 
PDF
A hybrid classification model for eeg signals
Aboul Ella Hassanien
 
PDF
Enhancing extreme learning machine: Novel extentions and applications to opti...
Apdullah YAYIK, Ph.D.
 
PDF
Report
Roger Rush
 
PDF
Senior Thesis
Anand Sekar
 
PDF
Cr31618621
IJERA Editor
 
PPTX
Development of portable eeg for treatment & diagnosis of disorders
andfaulkner
 
PPTX
Introduction to Spiking Neural Networks: From a Computational Neuroscience pe...
Jason Tsai
 
PDF
Bio radation abuse_protection_110
Arun Agrawal
 
PPTX
Neurally Driven Prosthetics
Marshall Benson
 
PDF
Feature Extraction Techniques and Classification Algorithms for EEG Signals t...
Editor IJCATR
 
PDF
IJET-V2I6P20
IJET - International Journal of Engineering and Techniques
 
PPTX
NYAI - Intersection of neuroscience and deep learning by Russell Hanson
Rizwan Habib
 
PDF
Sensors 20-00904-v2
Hoopeer Hoopeer
 
PDF
Improvement of a Bidirectional Brain-Computer Interface for Neural Engineerin...
HayleyBoyd5
 
PDF
Tracking times in temporal patterns embodied in intra-cortical data for cont...
IJECEIAES
 
PDF
Classification of EEG Signal for Epileptic Seizure DetectionusingEMD and ELM
IJTET Journal
 
PPTX
Introduction to Spiking Neural Networks: From a Computational Neuroscience pe...
Jason Tsai
 
PPTX
The Brain as a Whole: Executive Neurons and Sustaining Homeostatic Glia
InsideScientific
 
Neuron based time optimal controller of horizontal saccadic eye movements
Alireza Ghahari
 
ANALYSIS OF BRAIN COGNITIVE STATE FOR ARITHMETIC TASK AND MOTOR TASK USING EL...
sipij
 
A hybrid classification model for eeg signals
Aboul Ella Hassanien
 
Enhancing extreme learning machine: Novel extentions and applications to opti...
Apdullah YAYIK, Ph.D.
 
Report
Roger Rush
 
Senior Thesis
Anand Sekar
 
Cr31618621
IJERA Editor
 
Development of portable eeg for treatment & diagnosis of disorders
andfaulkner
 
Introduction to Spiking Neural Networks: From a Computational Neuroscience pe...
Jason Tsai
 
Bio radation abuse_protection_110
Arun Agrawal
 
Neurally Driven Prosthetics
Marshall Benson
 
Feature Extraction Techniques and Classification Algorithms for EEG Signals t...
Editor IJCATR
 
IJET-V2I6P20
IJET - International Journal of Engineering and Techniques
 
NYAI - Intersection of neuroscience and deep learning by Russell Hanson
Rizwan Habib
 
Sensors 20-00904-v2
Hoopeer Hoopeer
 
Improvement of a Bidirectional Brain-Computer Interface for Neural Engineerin...
HayleyBoyd5
 
Tracking times in temporal patterns embodied in intra-cortical data for cont...
IJECEIAES
 
Classification of EEG Signal for Epileptic Seizure DetectionusingEMD and ELM
IJTET Journal
 
Introduction to Spiking Neural Networks: From a Computational Neuroscience pe...
Jason Tsai
 
The Brain as a Whole: Executive Neurons and Sustaining Homeostatic Glia
InsideScientific
 
Ad

Similar to MAIN Conf Talk: Learning representations from neural signals (20)

PPT
What is (computational) neuroscience?
SSA KPI
 
PPTX
EEG Presentation Kassing/Konold
Driphtwood
 
PDF
Eeg
kuricheti
 
PPTX
Presentation(lokesh)
Lokesh Agrawal
 
PDF
Computer Aided Detection of Obstructive Sleep Apnea from EEG Signals
sipij
 
PDF
Computer Aided Detection of Obstructive Sleep Apnea from EEG Signals
sipij
 
PDF
Computer Aided Detection of Obstructive Sleep Apnea from EEG Signals
sipij
 
PDF
Computer Aided Detection of Obstructive Sleep Apnea from EEG Signals
sipij
 
PPTX
Use of eeg signal for mental analysis of a Person
Dipesh Pandey
 
PDF
Textbook of electroencephalography
Professor Yasser Metwally
 
PPTX
Eeg seminar
DrRAVIKANTKUMAR
 
PDF
Accessing Human Brain
freelaylay
 
PPTX
Basics of EEG
KarthiKeyan858190
 
PPTX
Seminar on clinical neurophysiology for medical students 2018.
Edina Timea Varga
 
PPTX
Seminar on clinical neurophysiology for medical students 2018.
Edina Timea Varga
 
PDF
Magnetoencephalography
Olusola Adeyemi
 
PPTX
Electroencephalography (eeg)
Ranjeet Singha
 
DOCX
Brain-Computer-Interfaces
Dominic Portain
 
PPT
EEG_circut.ppt
AbdErrezakChahoub
 
PPT
EEG_circuit.ppt
FatmaSetyaningsih2
 
What is (computational) neuroscience?
SSA KPI
 
EEG Presentation Kassing/Konold
Driphtwood
 
Eeg
kuricheti
 
Presentation(lokesh)
Lokesh Agrawal
 
Computer Aided Detection of Obstructive Sleep Apnea from EEG Signals
sipij
 
Computer Aided Detection of Obstructive Sleep Apnea from EEG Signals
sipij
 
Computer Aided Detection of Obstructive Sleep Apnea from EEG Signals
sipij
 
Computer Aided Detection of Obstructive Sleep Apnea from EEG Signals
sipij
 
Use of eeg signal for mental analysis of a Person
Dipesh Pandey
 
Textbook of electroencephalography
Professor Yasser Metwally
 
Eeg seminar
DrRAVIKANTKUMAR
 
Accessing Human Brain
freelaylay
 
Basics of EEG
KarthiKeyan858190
 
Seminar on clinical neurophysiology for medical students 2018.
Edina Timea Varga
 
Seminar on clinical neurophysiology for medical students 2018.
Edina Timea Varga
 
Magnetoencephalography
Olusola Adeyemi
 
Electroencephalography (eeg)
Ranjeet Singha
 
Brain-Computer-Interfaces
Dominic Portain
 
EEG_circut.ppt
AbdErrezakChahoub
 
EEG_circuit.ppt
FatmaSetyaningsih2
 
Ad

More from agramfort (7)

PDF
MNE sapien labs 2019
agramfort
 
PDF
SfN 2018: Machine learning and signal processing for neural oscillations
agramfort
 
PDF
ICML 2018 Reproducible Machine Learning - A. Gramfort
agramfort
 
PDF
MNE group analysis presentation @ Biomag 2016 conf.
agramfort
 
PDF
Teaching ML with scikit-learn at Telecom ParisTech
agramfort
 
PDF
Anomaly/Novelty detection with scikit-learn
agramfort
 
PDF
Paris machine learning meetup 17 Sept. 2013
agramfort
 
MNE sapien labs 2019
agramfort
 
SfN 2018: Machine learning and signal processing for neural oscillations
agramfort
 
ICML 2018 Reproducible Machine Learning - A. Gramfort
agramfort
 
MNE group analysis presentation @ Biomag 2016 conf.
agramfort
 
Teaching ML with scikit-learn at Telecom ParisTech
agramfort
 
Anomaly/Novelty detection with scikit-learn
agramfort
 
Paris machine learning meetup 17 Sept. 2013
agramfort
 

Recently uploaded (20)

PDF
Communicating Health Policies to Diverse Populations (www.kiu.ac.ug)
publication11
 
PDF
Warm, water-depleted rocky exoplanets with surfaceionic liquids: A proposed c...
Sérgio Sacani
 
PPTX
Substance Disorders- part different drugs change body
NizaAnne
 
PPTX
Seminar Hypertension and Kidney diseases.pptx
johnsalaoudine
 
PPTX
A powerpoint on colorectal cancer with brief background
lilmiceman
 
PPTX
endocrine - management of adrenal incidentaloma.pptx
csushrut2511
 
PPTX
PMR- PPT.pptx for students and doctors tt
DiyaSajid2
 
PPTX
Probability.pptx pearl lecture first year
ayeshanadeem0508
 
PPTX
SCIENCE 4 Q2W5 PPT.pptx Lesson About Plnts and animals and their habitat
irishjeancui
 
PPTX
Introcution to Microbes Burton's Biology for the Health
fliwertyfw154
 
PDF
Looking into the jet cone of the neutrino-associated very high-energy blazar ...
Sérgio Sacani
 
PDF
Science Form five needed shit SCIENEce so
ArielLucifer1
 
PPT
THE CELL THEORY AND ITS FUNDAMENTALS AND USE
nctpcctv
 
PPT
Mutation in dna of bacteria and repairss
Hassan Yousaf
 
PPTX
Hypertension_Training_materials_English_2024[1] (1).pptx
johnsalaoudine
 
PPT
Heredity-grade-9 Heredity-grade-9. Heredity-grade-9.
tipaniiisheyn
 
PPT
Computional quantum chemistry study .ppt
SrilakshmivenkataNar
 
PDF
GROUP 2 ORIGINAL PPT. pdf Hhfiwhwifhww0ojuwoadwsfjofjwsofjw
jjsaplad10
 
PPTX
gene cloning powerpoint for general biology 2
G17RodriguezJanelleA
 
PDF
Worlds Next Door: A Candidate Giant Planet Imaged in the Habitable Zone of ↵ ...
Sérgio Sacani
 
Communicating Health Policies to Diverse Populations (www.kiu.ac.ug)
publication11
 
Warm, water-depleted rocky exoplanets with surfaceionic liquids: A proposed c...
Sérgio Sacani
 
Substance Disorders- part different drugs change body
NizaAnne
 
Seminar Hypertension and Kidney diseases.pptx
johnsalaoudine
 
A powerpoint on colorectal cancer with brief background
lilmiceman
 
endocrine - management of adrenal incidentaloma.pptx
csushrut2511
 
PMR- PPT.pptx for students and doctors tt
DiyaSajid2
 
Probability.pptx pearl lecture first year
ayeshanadeem0508
 
SCIENCE 4 Q2W5 PPT.pptx Lesson About Plnts and animals and their habitat
irishjeancui
 
Introcution to Microbes Burton's Biology for the Health
fliwertyfw154
 
Looking into the jet cone of the neutrino-associated very high-energy blazar ...
Sérgio Sacani
 
Science Form five needed shit SCIENEce so
ArielLucifer1
 
THE CELL THEORY AND ITS FUNDAMENTALS AND USE
nctpcctv
 
Mutation in dna of bacteria and repairss
Hassan Yousaf
 
Hypertension_Training_materials_English_2024[1] (1).pptx
johnsalaoudine
 
Heredity-grade-9 Heredity-grade-9. Heredity-grade-9.
tipaniiisheyn
 
Computional quantum chemistry study .ppt
SrilakshmivenkataNar
 
GROUP 2 ORIGINAL PPT. pdf Hhfiwhwifhww0ojuwoadwsfjofjwsofjw
jjsaplad10
 
gene cloning powerpoint for general biology 2
G17RodriguezJanelleA
 
Worlds Next Door: A Candidate Giant Planet Imaged in the Habitable Zone of ↵ ...
Sérgio Sacani
 

MAIN Conf Talk: Learning representations from neural signals

  • 2. Alex Gramfort Learning representations from neural signals Neurons as current generators 2 Current Large cortical pyramidal cells organized in macro-assemblies with their dendrites normally oriented to the local cortical surface White matter Gray matter
  • 3. Alex Gramfort Learning representations from neural signals Neurons as current generators 2 Current Large cortical pyramidal cells organized in macro-assemblies with their dendrites normally oriented to the local cortical surface White matter Gray matter Q = I × d (10 to 100 nAm) with the equivalent current dipole (ECD) model B Magnetic Field Neural Current (post synaptic) Equivalent Current Dipole E Electric Field Neural Current (post synaptic) Equivalent Current Dipole
  • 4. Alex Gramfort Learning representations from neural signals Electro- & Magneto-encephalography 3 B Neural Current (post synaptic) Equivalent Current Dipole dB/dz MEG recordings First EEG recordings in 1929 by H. Berger Hôpital La Timone Marseille, France E Neural Current (post synaptic) Equivalent Current Dipole V EEG recordings
  • 5. Alex Gramfort Learning representations from neural signals Stereotaxic EEG (sEEG) 4 Intracranial electrodes; 5 to 15 contacts per electrode Around 10 electrodes are implanted Stereotaxic Implantation
  • 8. [T. Dupré laTour, L.Tallot, L. Grabot,V. Doyère,V. vanWassenhove,Y. Grenier,A. Gramfort, (2017) PLOS Computational biology] μ rhythm
  • 9. [T. Dupré laTour, L.Tallot, L. Grabot,V. Doyère,V. vanWassenhove,Y. Grenier,A. Gramfort, (2017) PLOS Computational biology] μ rhythm CFC: High frequency bursts coupled with slow waves
  • 10. [T. Dupré laTour, L.Tallot, L. Grabot,V. Doyère,V. vanWassenhove,Y. Grenier,A. Gramfort, (2017) PLOS Computational biology] μ rhythm CFC: High frequency bursts coupled with slow waves Neural signals exhibit diverse and complex morphologies
  • 11. Alex Gramfort Learning representations from neural signals “Textbook” brain rythms 6 Gamma (> 25 Hz) Beta (12-25 Hz) Alpha (8-12 Hz) Theta (4-8 Hz) Delta (1-4 Hz)
  • 12. Alex Gramfort Learning representations from neural signals Fourier Fallacy [Jasper 48] 7 Power Spectra:Waveform:Topography:
  • 13. Alex Gramfort Learning representations from neural signals Fourier Fallacy [Jasper 48] 7 «Even though it may be possible to analyze the complex forms of brain waves into a number of different sine-wave frequencies, this may lead only to what might be termed a “Fourier fallacy”, if one assumes ad hoc that all of the necessary frequencies actually occur as periodic phenomena in cell groups within the brain. » Jasper, 1948 Power Spectra:Waveform:Topography:
  • 14. Alex Gramfort Learning representations from neural signals Fourier Fallacy [Jasper 48] 7 «Even though it may be possible to analyze the complex forms of brain waves into a number of different sine-wave frequencies, this may lead only to what might be termed a “Fourier fallacy”, if one assumes ad hoc that all of the necessary frequencies actually occur as periodic phenomena in cell groups within the brain. » Jasper, 1948 Harmonics Power Spectra:Waveform:Topography:
  • 15. Alex Gramfort Learning representations from neural signals Outline 8 S. Chambon,V.Thorey, P. J.Arnal, E. Mignot,A. Gramfort. (2018), DOSED: a deep learning approach to detect multiple sleep micro-events in EEG signal,Arxiv 2500163 S. Chambon, M. Galtier, P.Arnal, G.Wainrib, A. Gramfort (2018),A deep learning architecture for temporal sleep stage classification using multivariate and multimodal time series, IEEE Trans. Neural Systems and Rehabilitation Engineering T. Dupré la Tour,  T. Moreau, M. Jas,A. Gramfort (2018), Multivariate Convolutional Sparse Coding for Electromagnetic Brain Signals, Proc. NeurIPS Conf. M. Jas,T. Dupré la Tour, U. Simsekli,A. Gramfort (2017), Learning the Morphology of Brain Signals Using Alpha-Stable Convolutional Sparse Coding, Proc. NeurIPS Conf. Polysomnography exam: I Electroencephalography (EEG) I Electrooculography (EOG) I Electromyography (EMG) I · · · Preliminary exam in sleep studies: I sleep disorders I psychiatric disorders... Polysomnographie “Deep” supervised learning on sleep EEG data: Unsupervised convolutional sparse coding:
  • 16. Deep supervised learning on sleep EEG data 1 Spindle K-complex
  • 17. Alex Gramfort Learning representations from neural signals Polysomnography (PSG) • Clinical exam • Electrophysiological signals 10 Electro-oculography (EOG) Electro-encephalography (EEG) Electro-myography (EMG)
  • 18. Alex Gramfort Learning representations from neural signals Polysomnography (PSG) • Clinical exam • Electrophysiological signals 10 Electro-oculography (EOG) Electro-encephalography (EEG) Electro-myography (EMG) Routinely annotated by sleep experts
  • 19. Alex Gramfort Learning representations from neural signals 2 types of annotations Hypnogram
 of sleep stages Micro-events: Spindles, K-complex etc. 11
  • 20. Alex Gramfort Learning representations from neural signals 2 types of annotations Hypnogram
 of sleep stages Micro-events: Spindles, K-complex etc. 11 Classification problem
  • 21. Alex Gramfort Learning representations from neural signals 2 types of annotations Hypnogram
 of sleep stages Micro-events: Spindles, K-complex etc. 11 Classification problem Joint detection and classification problem
  • 23. Alex Gramfort Learning representations from neural signals Objective 13 PSG data
  • 24. Alex Gramfort Learning representations from neural signals Objective 13 x 2 X PSG data
  • 25. Alex Gramfort Learning representations from neural signals Objective 13 Learn: ˆf : X ! Y Y = {Awake, REM, Stage 1, Stage 2, etc.} x 2 X PSG data
  • 26. Alex Gramfort Learning representations from neural signals Objective 13 Learn: ˆf : X ! Y Y = {Awake, REM, Stage 1, Stage 2, etc.} x 2 X PSG data Multiclass prediction problem Input is multiple time series
  • 27. Alex Gramfort Learning representations from neural signals and many more…. An old yet timely problem 14 C. Berthomier, X. Drouot, M. Herman-Stoïca, et al. ,“Automatic analysis of single-channel sleep EEG: validation in healthy individuals.,” Sleep, vol. 30, no. 11, pp. 1587–1595, 2007. O.Tsinalis, P. M. Matthews, andY. Guo,“Automatic Sleep Stage Scoring Using Time-Frequency Analysis and Stacked Sparse Autoencoders,” Annals of Biomedical Engineering, vol. 44, no. 5, pp. 1587–1597, 2016. J. B. Stephansen, A.Ambati, E. B. Leary, et al.,“The use of neural networks in the analysis of sleep stages and the diagnosis of narcolepsy,” CoRR, vol. abs/1710.02094, 2017. A.Vilamala, K. H. Madsen, and L. K. Hansen,“Deep convolutional neural networks for interpretable analysis of EEG sleep stage scoring,” 2017 IEEE Workshop on Machine Learning for Signal Processing (MLSP) S.Biswal, J.Kulas, H.Sun, B.Goparaju, M.B.Westover, M.T.Bianchi and J. Sun,“SLEEPNET: automated sleep staging system via deep learning,” arXiv:1707.08262, 2017. H. Dong,A. Supratak,W. Pan, C.Wu, P. M. Matthews, andY. Guo,“Mixed Neural Network Approach for Temporal Sleep Stage Classification,” arXiv:1610.06421v1, vol. 1, 2016. O.Tsinalis, P.M. Matthews,Y. Guo and S.Zafeiriou,“Automatic Sleep Stage Scoring with Single-Channel EEG Using Convolutional Neural Networks,” arXiv:1610.01683, pp. 1–10, 2016. A. Supratak, H. Dong, C.Wu, andY. Guo,“Deepsleepnet: a model for automatic sleep stage scoring based on raw single-channel EEG,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2017. T. Lajnef, S. Chaibi, P. Ruby, et al. ,“Learning Machines and Sleeping Brains:Automatic Sleep Stage Classification using Decision-Tree Multi-Class SupportVector Machines,” Journal of Neuroscience Methods, 2015.
  • 28. Alex Gramfort Learning representations from neural signals and many more…. An old yet timely problem 14 C. Berthomier, X. Drouot, M. Herman-Stoïca, et al. ,“Automatic analysis of single-channel sleep EEG: validation in healthy individuals.,” Sleep, vol. 30, no. 11, pp. 1587–1595, 2007. O.Tsinalis, P. M. Matthews, andY. Guo,“Automatic Sleep Stage Scoring Using Time-Frequency Analysis and Stacked Sparse Autoencoders,” Annals of Biomedical Engineering, vol. 44, no. 5, pp. 1587–1597, 2016. J. B. Stephansen, A.Ambati, E. B. Leary, et al.,“The use of neural networks in the analysis of sleep stages and the diagnosis of narcolepsy,” CoRR, vol. abs/1710.02094, 2017. A.Vilamala, K. H. Madsen, and L. K. Hansen,“Deep convolutional neural networks for interpretable analysis of EEG sleep stage scoring,” 2017 IEEE Workshop on Machine Learning for Signal Processing (MLSP) S.Biswal, J.Kulas, H.Sun, B.Goparaju, M.B.Westover, M.T.Bianchi and J. Sun,“SLEEPNET: automated sleep staging system via deep learning,” arXiv:1707.08262, 2017. H. Dong,A. Supratak,W. Pan, C.Wu, P. M. Matthews, andY. Guo,“Mixed Neural Network Approach for Temporal Sleep Stage Classification,” arXiv:1610.06421v1, vol. 1, 2016. O.Tsinalis, P.M. Matthews,Y. Guo and S.Zafeiriou,“Automatic Sleep Stage Scoring with Single-Channel EEG Using Convolutional Neural Networks,” arXiv:1610.01683, pp. 1–10, 2016. A. Supratak, H. Dong, C.Wu, andY. Guo,“Deepsleepnet: a model for automatic sleep stage scoring based on raw single-channel EEG,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2017. T. Lajnef, S. Chaibi, P. Ruby, et al. ,“Learning Machines and Sleeping Brains:Automatic Sleep Stage Classification using Decision-Tree Multi-Class SupportVector Machines,” Journal of Neuroscience Methods, 2015. To be compared
  • 29. Alex Gramfort Learning representations from neural signals and many more…. An old yet timely problem 14 Obj: learn from raw multimodal and multivariate signals C. Berthomier, X. Drouot, M. Herman-Stoïca, et al. ,“Automatic analysis of single-channel sleep EEG: validation in healthy individuals.,” Sleep, vol. 30, no. 11, pp. 1587–1595, 2007. O.Tsinalis, P. M. Matthews, andY. Guo,“Automatic Sleep Stage Scoring Using Time-Frequency Analysis and Stacked Sparse Autoencoders,” Annals of Biomedical Engineering, vol. 44, no. 5, pp. 1587–1597, 2016. J. B. Stephansen, A.Ambati, E. B. Leary, et al.,“The use of neural networks in the analysis of sleep stages and the diagnosis of narcolepsy,” CoRR, vol. abs/1710.02094, 2017. A.Vilamala, K. H. Madsen, and L. K. Hansen,“Deep convolutional neural networks for interpretable analysis of EEG sleep stage scoring,” 2017 IEEE Workshop on Machine Learning for Signal Processing (MLSP) S.Biswal, J.Kulas, H.Sun, B.Goparaju, M.B.Westover, M.T.Bianchi and J. Sun,“SLEEPNET: automated sleep staging system via deep learning,” arXiv:1707.08262, 2017. H. Dong,A. Supratak,W. Pan, C.Wu, P. M. Matthews, andY. Guo,“Mixed Neural Network Approach for Temporal Sleep Stage Classification,” arXiv:1610.06421v1, vol. 1, 2016. O.Tsinalis, P.M. Matthews,Y. Guo and S.Zafeiriou,“Automatic Sleep Stage Scoring with Single-Channel EEG Using Convolutional Neural Networks,” arXiv:1610.01683, pp. 1–10, 2016. A. Supratak, H. Dong, C.Wu, andY. Guo,“Deepsleepnet: a model for automatic sleep stage scoring based on raw single-channel EEG,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2017. T. Lajnef, S. Chaibi, P. Ruby, et al. ,“Learning Machines and Sleeping Brains:Automatic Sleep Stage Classification using Decision-Tree Multi-Class SupportVector Machines,” Journal of Neuroscience Methods, 2015. To be compared
  • 30. Alex Gramfort Learning representations from neural signals Proposed approach Features • Convolutional Neural Net (CNN): Frequency content Spatial context • Multivariate signals: spatial filtering (like ICA would do) • Multimodal inputs: different pipelines for EEG, EOG and EMG Temporal context • Concatenate features of neighboring samples [A deep learning architecture for temporal sleep stage classification using multivariate and multimodal time series. S. Chambon, M. N. Galtier, P. J.Arnal, G.Wainrib,A. Gramfort. IEEE TSRNE 2018] 15
  • 31. Alex Gramfort Learning representations from neural signals Problem formulation 16 ˆf 2 argminf2F Ex,y2X⇥Y [`(y, f(x))] Risk minimization:
  • 32. Alex Gramfort Learning representations from neural signals Problem formulation 16 ˆf 2 argminf2F Ex,y2X⇥Y [`(y, f(x))] Risk minimization: F• : a class of models (neural networks architecture).
  • 33. Alex Gramfort Learning representations from neural signals Problem formulation 16 ˆf 2 argminf2F Ex,y2X⇥Y [`(y, f(x))] Risk minimization: F• : a class of models (neural networks architecture). `• : loss.We use the categorical cross-entropy. • Minimized on a training set using backpropagation and online stochastic gradient descent.
  • 34. shape = (C, T) Spatial Filtering shape = (C’, T) Spatial Filtering EEG/EOG EMG Input Conv/Relu/MP Softmax Conv/Relu/MP 𝜙C 𝜓 𝜙 𝜙T
  • 35. shape = (C, T) Spatial Filtering shape = (C’, T) Spatial Filtering EEG/EOG EMG Input Conv/Relu/MP Softmax Conv/Relu/MP 𝜙C 𝜓 𝜙 𝜙T shape = (C, T) Spatial Filtering shape = (C’, T) Spatial Filtering EEG/EOG EMG Input Conv/Relu/MP Softmax Conv/Relu/MP 𝜙C 𝜓 𝜙 𝜙T
  • 36. shape = (C, T) Spatial Filtering shape = (C’, T) Spatial Filtering EEG/EOG EMG Input Conv/Relu/MP Softmax Conv/Relu/MP 𝜙C 𝜓 𝜙 𝜙T shape = (C, T) Spatial Filtering shape = (C’, T) Spatial Filtering EEG/EOG EMG Input Conv/Relu/MP Softmax Conv/Relu/MP 𝜙C 𝜓 𝜙 𝜙T
  • 37. shape = (C, T) Spatial Filtering shape = (C’, T) Spatial Filtering EEG/EOG EMG Input Conv/Relu/MP Softmax Conv/Relu/MP 𝜙C 𝜓 𝜙 𝜙T shape = (C, T) Spatial Filtering shape = (C’, T) Spatial Filtering EEG/EOG EMG Input Conv/Relu/MP Softmax Conv/Relu/MP 𝜙C 𝜓 𝜙 𝜙T shape = (C, T) Spatial Filtering shape = (C’, T) Spatial Filtering EEG/EOG EMG Input Conv/Relu/MP Softmax Conv/Relu/MP 𝜙C 𝜓 𝜙 𝜙T
  • 38. shape = (C, T) Spatial Filtering shape = (C’, T) Spatial Filtering EEG/EOG EMG Input Conv/Relu/MP Softmax Conv/Relu/MP 𝜙C 𝜓 𝜙 𝜙T
  • 39. shape = (C, T) Spatial Filtering shape = (C’, T) Spatial Filtering EEG/EOG EMG Input Conv/Relu/MP Softmax Conv/Relu/MP 𝜙C 𝜓 𝜙 𝜙T Block 1 Conv, ReLU, Max Pooling Input Block 2
  • 40. Alex Gramfort Learning representations from neural signals Model parameters 23
  • 41. Alex Gramfort Learning representations from neural signals Model parameters 23 Spatial filtering
  • 42. Alex Gramfort Learning representations from neural signals Model parameters 23 Spatial filtering Conv. layer 1
  • 43. Alex Gramfort Learning representations from neural signals Model parameters 23 Spatial filtering Conv. layer 1 Conv. layer 2
  • 45. zt k zt zt+k Time (s) 𝜙𝜙 𝜙 𝜓 Sequence of inputsEEGChannels F4 F3 C3 C4 O1 O2 -60 -30 0 30 60 90 yt
  • 46. zt k zt zt+k Time (s) 𝜙𝜙 𝜙 𝜓 Sequence of inputsEEGChannels F4 F3 C3 C4 O1 O2 -60 -30 0 30 60 90 yt We pool the representations from neighboring windows
  • 48. Alex Gramfort Learning representations from neural signals Experimental setup • Data • MASS - session 3: O’Reilly et al. 2014 • http://www.ceams-carsm.ca/en/MASS • 61 records, ~10h / record • Total number of samples about 60,000 (like for MNIST) • Preprocessing • lowpass filtering (30Hz) • downsampling 128Hz 27
  • 49. Alex Gramfort Learning representations from neural signals Experimental setup • Cross validation • 5 splits • 41 records for training, 10 for validation, 10 for testing • Baselines • Gradient boosting: Hand crafted features (mean, variance… relative power in frequency bands) [Lajnev et al. 2015] • [Tsinalis et al. 2016]: deep convolutional network • [Supratak et al. 2017]: deep convolutional network processing low and high frequency contents specifically 28
  • 50. Alex Gramfort Learning representations from neural signals Influence of spatial context 29
  • 52. Influence of temporal context 30 • Temporal context helps • But less when using more channels
  • 53. Alex Gramfort Learning representations from neural signals Performance vs SoTA 31 Performance: Computational efficiency:
  • 54. Alex Gramfort Learning representations from neural signals Performance vs SoTA 31 Performance: Computational efficiency: • Networks outperform hand crafted features with gradient boosting • Combining all modalities helps (multivariate) • SoTA is obtained with less parameters (fast)
  • 55. Alex Gramfort Learning representations from neural signals “Opening” the black box 32 Confusion matrix: N1 is hard
  • 56. Alex Gramfort Learning representations from neural signals “Opening” the black box 32 Confusion matrix: N1 is hard “Hiding” frequencies from the network: network has learnt scoring rules
  • 57. Start time End time Type Joint detection and classification of micro-events [A deep learning architecture to detect events in EEG signals during sleep. S. Chambon,V.Thorey, P. J.Arnal, E. Mignot,A. Gramfort. MLSP 2018] [DOSED: a deep learning approach to detect multiple sleep micro-events in EEG signal,. S. Chambon,V.Thorey, P. J.Arnal, E. Mignot,A. Gramfort. Arxiv 2500163]
  • 58. Start time End time Type Joint detection and classification of micro-events Objective: Predict what and when at the same time [A deep learning architecture to detect events in EEG signals during sleep. S. Chambon,V.Thorey, P. J.Arnal, E. Mignot,A. Gramfort. MLSP 2018] [DOSED: a deep learning approach to detect multiple sleep micro-events in EEG signal,. S. Chambon,V.Thorey, P. J.Arnal, E. Mignot,A. Gramfort. Arxiv 2500163]
  • 59. Alex Gramfort Learning representations from neural signals Spindles K-complex Spindles and K-complexes in N2 stage • Different types, durations, scales • Associated to certain sleep stages and sleep disorders Micro-events 34 Why is it hard? Why should you care?
  • 60. Alex Gramfort Learning representations from neural signals State of the art • Signal processing: • Band-pass filtering + thresholding • Decomposition into components • Shallow learning algorithms: • Clustering • Binary classifier 35 [Parekh et al. 2017, Lajnef et al. 2017] [Ray et al. 2015,Wamsley et al. 2012,Wendt et al. 2012, Mölle et al. 2011, Nir et al. 2011, Ferrarelli et al. 2007] [Patti et al. 2017] [Patti et al. 2015, Lachner-Piza et al. 2018] None of these works use deep learning
  • 61. Alex Gramfort Learning representations from neural signals DOSED algorithm Inspired by: YOLO and SSD 1. Predict: type, start & end 2. Allows for multiple types of events 3. Detection and classification in one pass 4. Convolutional network 36 [Redmon et al. 2016, Liu et al. 2015]
  • 62. Alex Gramfort Learning representations from neural signals Event detector 𝜓loc : conv. layerInput 𝜙: CNN 𝜓clf : conv. layer Setup Model Default events 37
  • 63. Alex Gramfort Learning representations from neural signals d1 d2 dNd Loss: localization + classification !38
  • 64. Alex Gramfort Learning representations from neural signals d1 d2 dNd Default events overlapping with true events Loss: localization + classification !39
  • 65. Alex Gramfort Learning representations from neural signals d1 d2 dNd Remaining default events Loss: localization + classification !40 Default events overlapping with true events
  • 66. Alex Gramfort Learning representations from neural signals Evaluation •By event metrics
 
 •Based on Intersection over Union (IoU)
 •Precision, Recall, F1 Predicted event True event 41 [Warby et al. 2014] IoU < 𝜕 => False positive IoU > 𝜕 => True positive
  • 67. Alex Gramfort Learning representations from neural signals 3 datasets SS2
 [O’Reilly et al. 2014] SSC
 [Andlauer et al. 2013] WSC [Young et al. 2008] # Subjects 19 26 30 Age 23.6 ± 3.7 52.2 ± 14.3 65.2 ± 8.2 Aplit
 (train, val, test) 10, 5, 4 15, 5, 6 19, 5, 6 Events spindles
 K-complexes spindles spindles 42
  • 69. Alex Gramfort Learning representations from neural signals Illustration 44 True event Predicted event
  • 70. Alex Gramfort Learning representations from neural signals Illustration 44 True event Predicted event
  • 71. Alex Gramfort Learning representations from neural signals Joint detection 45 Joint detection does not deteriorate performance
  • 72. Convolutional Sparse Coding (CSC) for learning the morphology of neural signals 2 Code: https://alphacsc.github.io Multivariate Convolutional Sparse Coding for Electromagnetic Brain Signals, (2018), T. Dupré laTour, T. Moreau, M. Jas,A. Gramfort, Proc. NeurIPS Conf. Learning the Morphology of Brain Signals Using Alpha-Stable Convolutional Sparse Coding, (2017), M. Jas,T. Dupré laTour, U. Simsekli,A. Gramfort, Proc. NeurIPS Conf.
  • 73. Alex Gramfort Learning representations from neural signals Signal representations ▪Sparse representations: wavelet basis ▪Sparse coding / dictionary learning ▪Shift-invariant representations ▪In neurophysiology: ▪ Matching of time-invariant filters (MOTIF) ▪ Multivariate orthogonal matching pursuit ▪ Matching pursuit and heuristics ▪ Sliding window machine ▪ Adaptive waveform learning 47 [Morlet 70’, Meyer 80’, Mallat 90’ etc.] [Olshausen and Field, 1996, Elad and Aharon, 2006] [Lewicki and Sejnowski, 1999, Grosse et al, 2007] [Jost et al, 2006] [Gips et al, 2017] [Hitziger et al, 2017] [Barthélemy et al, 2012] [Brokmeier and Principe, 2016]
  • 74. Alex Gramfort Learning representations from neural signals Convolutional sparse coding 48 [Grosse et al, 2007]
  • 75. Alex Gramfort Learning representations from neural signals Convolutional sparse coding 49 [Grosse et al, 2007]
  • 76. Alex Gramfort Learning representations from neural signals Convolutional sparse coding 50 [Grosse et al, 2007]
  • 77. Alex Gramfort Learning representations from neural signals Optimization strategy 51 Block-coordinate descent:
  • 78. Alex Gramfort Learning representations from neural signals Optimization strategy 51 Block-coordinate descent: [Kavukcuoglu et al, 2010] [Chalasani et al, 2013] [Bristow et al, 2013] [Wohlberg, 2016] [Jas et al, 2017] [Dupré la Tour et al, 2018] ▪ Z-step ▪ GCD ▪ FISTA ▪ ADMM ▪ ADMM + FFT ▪ L-BFGS ▪ LGCD
  • 79. Alex Gramfort Learning representations from neural signals Optimization strategy 51 Block-coordinate descent: [Grosse et al, 2007] [Heide et al, 2015] [Wohlberg, 2016] [Jas et al, 2017] [Dupré la Tour et al, 2018) ▪ D-step ▪ FFT ▪ ADMM + FFT ▪ ADMM + FFT ▪ L-BFGS (dual) ▪ PGD [Kavukcuoglu et al, 2010] [Chalasani et al, 2013] [Bristow et al, 2013] [Wohlberg, 2016] [Jas et al, 2017] [Dupré la Tour et al, 2018] ▪ Z-step ▪ GCD ▪ FISTA ▪ ADMM ▪ ADMM + FFT ▪ L-BFGS ▪ LGCD
  • 82. Alex Gramfort Learning representations from neural signals Learned atoms 53 Data: ~80 Hz
  • 83. Alex Gramfort Learning representations from neural signals Learned atoms 53 Data: ~80 Hz CSC reveals CFC
  • 84. Alex Gramfort Learning representations from neural signals Learned atoms 53 Data: How about if I have many channels? ~80 Hz CSC reveals CFC
  • 85. Alex Gramfort Learning representations from neural signals From ICA to CSC 54 https://pypi.python.org/pypi/python-picard/0.1 Independent Component Analysis (ICA)
  • 86. Alex Gramfort Learning representations from neural signals From ICA to CSC 54 https://pypi.python.org/pypi/python-picard/0.1 Independent Component Analysis (ICA) X S
  • 87. Alex Gramfort Learning representations from neural signals From ICA… 55 https://pypi.python.org/pypi/python-picard/0.1 X = A S = + +… a1 aks1 sk + +… https://pierreablin.github.io/picard/auto_examples/plot_ica_eeg.html https://www.martinos.org/mne/stable/auto_tutorials/plot_artifacts_correction_ica.html
  • 88. Alex Gramfort Learning representations from neural signals … … to CSC 56 https://pypi.python.org/pypi/python-picard/0.1 X = + u1 v1 ⇤ z1 uk vk ⇤ zk convolution
  • 89. Alex Gramfort Learning representations from neural signals … ⇤ Topography waveform temporal activations … to CSC 56 https://pypi.python.org/pypi/python-picard/0.1 X = + u1 v1 ⇤ z1 uk vk ⇤ zk convolution
  • 90. Alex Gramfort Learning representations from neural signals … ⇤ Topography waveform temporal activations … to CSC 56 https://pypi.python.org/pypi/python-picard/0.1 X = + u1 v1 ⇤ z1 uk vk ⇤ zk convolution CSC allows to learn jointly • topography (ie. localization) • signal waveform • when waveform occurs
  • 91. Alex Gramfort Learning representations from neural signals Multivariate CSC 57 [Multivariate Convolutional Sparse Coding for Electromagnetic Brain Signals, (2018), T. Dupré laTour,  T. Moreau, M. Jas,A. Gramfort, Proc. NeurIPS Conf.]
  • 92. Alex Gramfort Learning representations from neural signals Multivariate CSC 57 [Multivariate Convolutional Sparse Coding for Electromagnetic Brain Signals, (2018), T. Dupré laTour,  T. Moreau, M. Jas,A. Gramfort, Proc. NeurIPS Conf.]
  • 93. Alex Gramfort Learning representations from neural signals Multivariate CSC 57 [Multivariate Convolutional Sparse Coding for Electromagnetic Brain Signals, (2018), T. Dupré laTour,  T. Moreau, M. Jas,A. Gramfort, Proc. NeurIPS Conf.] Rank 1 constraint:
  • 94. Alex Gramfort Learning representations from neural signals CSC on MEG 58 https://pypi.python.org/pypi/python-picard/0.1 •MEG vectorview •Median nerve stim. [Multivariate Convolutional Sparse Coding for Electromagnetic Brain Signals, (2018), T. Dupré laTour,  T. Moreau, M. Jas,A. Gramfort, Proc. NeurIPS Conf.]
  • 95. Alex Gramfort Learning representations from neural signals CSC on MEG 58 https://pypi.python.org/pypi/python-picard/0.1 •MEG vectorview •Median nerve stim. CSC reveals mu- shaped waveforms [Multivariate Convolutional Sparse Coding for Electromagnetic Brain Signals, (2018), T. Dupré laTour,  T. Moreau, M. Jas,A. Gramfort, Proc. NeurIPS Conf.]
  • 96. Alex Gramfort Learning representations from neural signals CSC on MEG 58 https://pypi.python.org/pypi/python-picard/0.1 •MEG vectorview •Median nerve stim. See the frequency harmonics CSC reveals mu- shaped waveforms [Multivariate Convolutional Sparse Coding for Electromagnetic Brain Signals, (2018), T. Dupré laTour,  T. Moreau, M. Jas,A. Gramfort, Proc. NeurIPS Conf.]
  • 97. Alex Gramfort Learning representations from neural signals CSC on MEG 59 Results on auditory task:
  • 102. Conclusion • SoTA convolutional network for sleep scoring on PSG data • First deep architecture for event detection on EEG data with SoTA performance • CSC model: Fast solver and multivariate • CSC comes with Python code
  • 103. http://www.martinos.org/mne MNE software for processing MEG and EEG data, A. Gramfort, M. Luessi, E. Larson, D. Engemann, D. Strohmeier, C. Brodbeck, L. Parkkonen, M. Hämäläinen, Neuroimage 2013
  • 104. http://www.martinos.org/mnehttp://www.martinos.org/mne Harvard U. Montreal MNE developer in 2010 MNE developers in 2018 Berkeley Paris NYU Cambridge Aalto Ilmenau Julich Sheffield U.Wash. UCSF Graz Marseille More than 80 people have contributed to MNE
  • 105. >>> import mne >>> raw = mne.io.read_raw_edf('SC4001E0-PSG.edf') >>> annot = mne.read_annotations('SC4001EC-Hypnogram.edf') >>> raw.set_annotations(annot) >>> events, event_id = mne.events_from_annotations(raw) >>> epochs = mne.Epochs(raw, events, event_id, tmin=0., tmax=30., baseline=None) >>> epochs['Sleep stage W’].plot_psd(fmax=20., picks=[0, 1])) https://github.com/mne-tools/mne-python/pull/5718 Working with EEG sleep data in 7 lines of Python code
  • 107. Thanks ! GitHub : @agramfort Twitter : @agramfort Support ERC SLAB,ANR THALAMEEG ANR-14-NEUC-0002-01 NIH R01 MH106174, DFG HA 2899/21-1. http://alexandre.gramfort.netContact S. Chambon,V.Thorey, P. J.Arnal, E. Mignot,A. Gramfort. (2018), DOSED: a deep learning approach to detect multiple sleep micro-events in EEG signal,Arxiv 2500163 T. Dupré la Tour,  T. Moreau, M. Jas,A. Gramfort, Multivariate Convolutional Sparse Coding for Electromagnetic Brain Signals, (2018), Proc. NeurIPS Conf. M. Jas,T. Dupré la Tour, U. Simsekli,A. Gramfort, Learning the Morphology of Brain Signals Using Alpha-Stable Convolutional Sparse Coding, (2017), Proc. NeurIPS Conf. Tom Dupré laTour Mainak Jas Thomas Moreau Umut Simsekli Stanislas Chambon ValentinThorey Pierrick Arnal Emmanuel Mignot Joint work with: S. Chambon, M. Galtier, P.Arnal, G.Wainrib, A. Gramfort (2018),A deep learning architecture for temporal sleep stage classification using multivariate and multimodal time series, IEEE Trans. Neural Systems and Rehabilitation Engineering