![• Present of S : Sn=(Xn , Yn)
• Past of S : Sn
–
=(Xn
–
, Yn
–
)
X
S
Y
Y = (Y1, Y2, … , Yn, … ,YN)
X=(X1, X2, … , Xn, … ,XN)
• Dynamic Process S=( X1,...,XM , Y ) = ( X,Y )
• With reference to a target system Y :
• Dynamical System S={S1,...,SM}
S={X1,...,XM-1 , Y} = {X,Y}X={X1,...,XM-1}
X1
S
Y
…
X2
nY
n
n-1, n-2, …
nX
Considering the flow of time allows to study
directed dynamical interactions within
and between processes
,...],[ 21 −−
−
= nnn XXX
,...],[ 21 −−
−
= nnn YYY
Y
X
presentpast
Introduction
Information Dynamics:
Theory
Synergy and
Redundancy
Conclusions
Information Dynamics:
Application
DYNAMICAL SYSTEMS AND PROCESSES
Building probability distributions from the past of the system](https://image.slidesharecdn.com/marinazzosynergycns2020-200727114717/85/Information-transfer-in-dynamical-models-implemented-on-the-brain-structural-connectome-2-320.jpg)
Information transfer in dynamical models implemented on the brain structural connectome
- 1. Department of Data Analysis – University of Ghent Synergistic information in a dynamical model implemented on the human structural connectome reveals spatially distinct associations with age Daniele Marinazzo CNS Info Theory Workshop July 2020 http://users.ugent.be/~dmarinaz/ daniele.marinazzo@ugent.be @dan_marinazzo
- 2. • Present of S : Sn=(Xn , Yn) • Past of S : Sn – =(Xn – , Yn – ) X S Y Y = (Y1, Y2, … , Yn, … ,YN) X=(X1, X2, … , Xn, … ,XN) • Dynamic Process S=( X1,...,XM , Y ) = ( X,Y ) • With reference to a target system Y : • Dynamical System S={S1,...,SM} S={X1,...,XM-1 , Y} = {X,Y}X={X1,...,XM-1} X1 S Y … X2 nY n n-1, n-2, … nX Considering the flow of time allows to study directed dynamical interactions within and between processes ,...],[ 21 −− − = nnn XXX ,...],[ 21 −− − = nnn YYY Y X presentpast Introduction Information Dynamics: Theory Synergy and Redundancy Conclusions Information Dynamics: Application DYNAMICAL SYSTEMS AND PROCESSES Building probability distributions from the past of the system
- 3. )(log)()( n y nnY ypypYHH n −== • Information generated by Y (system information): • Predictive Information about Y : PREDICTION ENTROPY ),|(log),,(),|( ,, | −− −−−− −− −== nnn yxy nnnnnnXY yxypyxypYXYHU nnn • Unexplained information about the present of Y given the past of S )( ),|( log),,(),;( ,, n nnn yxy nnnnnnY yp yxyp yxypYXYIP nnn −− −−−− −− == Information contained in the past of S=(X,Y) that can be used to predict the present of the target Y Information contained in the present of Y when the past of S=(X,Y) is known Information contained in the present of the process Y YH XYU | YP ),|(),;()( −−−− += nnnnnnn YXXHYXYIYH XYYY UPH |+= • Entropy decomposition: Introduction Information Dynamics: Theory Synergy and Redundancy Conclusions Information Dynamics: Application PREDICTIVE INFORMATION
- 4. • Expansion of the predictive information )|;();(),;( −−−−− +== nnnnnnnnY YXYIYYIYXYIP Information Storage: Information Transfer: TRANSFER ENTROPYYSSELF ENTROPY YXT → • Information storage in the system Y : SELF ENTROPY (SE) )|()( − −= nnn YYHYH )( )|( log),( , n nn yy nnY yp yyp yypS nn − − − = • Information transfer from X to Y : TRANSFER ENTROPY (TE) ),|()|( −−− −= nnnnn YXYHYYH )|( ),|( log),,( ,, − −− −− → −− = nn nnn yxy nnnYX yyp yxyp yxypT nnn Information contained in the past of X that can be used to predict the present of Y above and beyond the information contained in the past of Y Information contained in the past of Y that can be used to predict its present YH XYU | YS YXT → YP Introduction Information Dynamics: Theory Synergy and Redundancy Conclusions Information Dynamics: Application PREDICTIVE INFORMATION DECOMPOSITION
- 5. INFORMATION DYNAMICS AT TRANSITIONS Kuramoto oscillators on a lattice (Heyvaert 2018) Information transfer is minimized both in the completely ordered and in the disordered state
- 6. EXAMPLE OF TRANSITIONS : ISING MODEL T→∞ T~Tcrit T→0
- 7. ISING MODEL: BEYOND FERROMAGNETISM Polarization of news and opinions, financial crashes, epileptic seizures, learning, etc
- 8. INFORMATION DYNAMICS AT TRANSITIONS Pairwise and global Mutual Information peak at the critical temperature (Matsuda et al. IJTP 1996).
- 9. ISING SPINS ON THE STRUCTURAL CONNECTOME TE and correlations are maximised in the critical regime, but similarity with the SC has a minimum (forget the underlying structure to allow a dynamical repertoire)
- 10. GRANGER CAUSALITY AND TRANSFER ENTROPY GC and TE are equivalent for Ising spins with (low) coupling J (and for Gaussian variables)
- 11. IN- AND OUT- INFO FLOW IN ISING SYSTEMS
- 12. IN- AND OUT- INFO FLOW IN ISING SYSTEMS The in-degree distribution of the connectivity as a function of cin is a power law for small coupling, exponential for strong coupling
- 13. DIMINISHING MARGINAL RETURNS In many situations it can be expected that each node of a network may handle a limited amount of information. This structural constraint suggests that information flow networks should exhibit some topological evidences of the law of diminishing marginal returns, a fundamental principle of economics which states that when the amount of a variable resource is increased, while other resources are kept fixed, the resulting change in the output will eventually diminish. Investigated in Economics, Sociology, Anthropology Here we define the ratio R=cout/cin as a proxy of the presence of this phenomenon (when R>1)
- 14. INFO BOTTLENECKS IN THE CRITICAL REGIME R does not coincide with the node strength, especially for finer connectomes
- 15. INFO BOTTLENECKS IN THE CRITICAL REGIME Marinazzo, D., Pellicoro, M., Wu, G., Angelini, L., Cortés, J. M., & Stramaglia, S. (2014). Information Transfer and Criticality in the Ising Model on the Human Connectome. PLoS ONE, 9(4), e93616. doi:10.1371/journal.pone.0093616 R R R Time between spin flips
- 16. INFORMATION DYNAMICS AT TRANSITIONS But what comes before? Transition to order or peak info? Global Transfer Entropy peaks in the paramagnetic phase (Barnett et al. PRL 2013). This involves computing Transfer Entropy across all the system. Pairwise and global Mutual Information peak at the critical temperature (Matsuda et al. IJTP 1996).
- 17. INFORMATION DYNAMICS AT TRANSITIONS • Question 1: global transfer entropy requires dynamical data. Can we have precursors based also on static data (i.e. behavioral scores across several subjects)? • Question 2: Do we really have to measure all the variables, or we can build precursors based on a small number (e.g. 3) of variables?
- 18. PARTIAL INFORMATION DECOMPOSITION Distinct non-negative measures of redundancy and synergy, accounting for the possibility that redundancy and synergy may coexist as separate elements of information modification. The interaction TE is actually a measure of the ‘net’ synergy manifested in the transfer of information from the two sources to the target. PID components cannot be obtained through classic information theory simply subtracting conditional MI terms: one more relation is needed. Shannon information theory does not univocally determine this decomposition Redundancy is defined as the minimum of the information provided by each individual source to the target This choice satisfies the desirable property that the redundant TE is independent of the correlation between the source processes. 𝑇𝐸 𝑋1,𝑋2→𝑌 = 𝑈 𝑋1→𝑌 + 𝑈 𝑋2→𝑌+ 𝑅 𝑋1,𝑋2→𝑌+ 𝑆 𝑋1,𝑋2→𝑌 𝑇𝐸 𝑋1→𝑌 = 𝑈 𝑋1→𝑌+𝑅 𝑋1,𝑋2→𝑌 𝑇𝐸 𝑋2→𝑌 = 𝑈 𝑋2→𝑌+𝑅 𝑋1,𝑋2→𝑌 𝑅 𝑋1,𝑋2→𝑌 = min{𝑇𝐸 𝑋1→𝑌, 𝑇𝐸 𝑋2→𝑌}
- 19. INTRODUCTION DECOMPOSITION FOR MUTUAL INFORMATION AND TRANSFER ENTROPY Not conditioning on the past (instantaneous) Conditioning on the past (lagged)
- 20. INFORMATION DYNAMICS AT TRANSITIONS The synergy peak approaches the critical value as the amount of synergy decreases instantaneous lagged Is the synergy which peaks in the paramagnetic phase
- 21. NOW ON THE HUMAN STRUCTURAL CONNECTOME Work by Davide Nuzzi (UniBa) DTI of 196 subjects, age range 5-85 y • does the synergy still peak before the critical point in a nonuniform network? • are the hubs of structural connectivity also hubs of synergy? • is there association with age?
- 22. NOW ON THE HUMAN STRUCTURAL CONNECTOME Hubs of structural connectivity are not among the nodes towards which synergy is highest
- 23. NOW ON THE HUMAN STRUCTURAL CONNECTOME Positive and negative associations of synergy with age, in localized clusters In some regions this association is continuous with age, in other ones it's limited to the first ~30 years
- 24. CONCLUSIONS The total amount of information transfer is maximized at criticality when Ising spins are connected according to the structural connectome The ratio between input and output information has also a maximum, related with the law of diminishing marginal returns The physical quantity that actually acts as a transition precursor is the synergy This valuable marker can be found considering as few as three variables, and lagged correlations are not necessary to this scope Implementation on the human connectome shows differential associations with age
- 25. http://users.ugent.be/~dmarinaz/ daniele.marinazzo@ugent.be @dan_marinazzo THANKS Marinazzo, D., Wu, G., Pellicoro, M., Angelini, L., & Stramaglia, S. (2012). Information Flow in Networks and the Law of Diminishing Marginal Returns: Evidence from Modeling and Human Electroencephalographic Recordings. PLoS ONE, 7(9), e45026. doi:10.1371/journal.pone.0045026 Marinazzo, D., Pellicoro, M., Wu, G., Angelini, L., Cortés, J. M., & Stramaglia, S. (2014). Information Transfer and Criticality in the Ising Model on the Human Connectome. PLoS ONE, 9(4), e93616. doi:10.1371/journal.pone.0093616 Stramaglia, S., Pellicoro, M., Angelini, L., Amico, E., Aerts, H., Cortés, J. M., … Marinazzo, D. (2017). Ising model with conserved magnetization on the human connectome: Implications on the relation structure-function in wakefulness and anesthesia. Chaos: An Interdisciplinary Journal of Nonlinear Science, 27(4), 047407. doi:10.1063/1.4978999 Marinazzo, D., Angelini, L., Pellicoro, M., & Stramaglia, S. (2019). Synergy as a warning sign of transitions: The case of the two-dimensional Ising model. Physical Review E, 99(4). doi:10.1103/physreve.99.040101 Nuzzi, D., Pellicoro, M., Angelini, L., Marinazzo, D., & Stramaglia, S. (2020). Synergistic information in a dynamical model implemented on the human structural connectome reveals spatially distinct associations with age. Network Neuroscience, 1–15. doi:10.1162/netn_a_00146