Introduction to Trentool

Editor's Notes

• #2: Here’s a short introduction about a tool I encountered during a workshop in Frankfurt.\nTrentool is used to measure information transfer directionality between two or more signal streams, and has been successfully used in a lot of areas from meteorology to neuroscience to laser physics.\n
• #3: We start at the basic models and work our way upwards:\nwe have our traditional coherence\nand the partial coherence for multivariate data.\nBoth of these models can be extended to measure causality,\nand we arrive at Granger causality and partial directed coherence.\n
• #4: We start at the basic models and work our way upwards:\nwe have our traditional coherence\nand the partial coherence for multivariate data.\nBoth of these models can be extended to measure causality,\nand we arrive at Granger causality and partial directed coherence.\n
• #5: That’s why I’m presenting a new model today, which is based on a completely different mathematical model: entropy.\n
• #6: Let’s say we have two signal streams, which possibly exchange some information with each other.\nThe information content in each data stream can be represented by its respective entropy.\nOne important aspect: The combined entropy of the two sources can be divided into two features: conditional entropy and mutual information.\n
• #7: In a given data stream, it’s entirely possible that certain aspects repeat over the course of time. We can test for repeating patterns by taking the whole history of the data stream and predicting the next data point with that. The probability for some new information to appear during this step is called transition probability.\n
• #8: Next point is mutual information.\nFirst, when we combine the entropy of both data streams, we get the receivers' diversity H(X) + H(Y).\nThen, we can calculate the conditional entropy, which you can imagine as the equivocation of the receiver about its source.\nBy subtracting these two measures, we get the amount of information that both datastreams share with each other, but doesn’t increase the opponent entropy.\nIn general, mutual information can be thought as a measure of disagreement between information receivers.\nAnd this is what is popularly known as transfer entropy.\n
• #9: When we combine conditional entropy and mutual information, we get conditional mutual information (or the apparent transfer entropy).\nYou can imagine it like this:\nHow much are the transition probabilities changed by knowing the information history of the other data stream?\nIf both sources ‘do their own thing’, the transition probabilities don’t change at all, and transfer entropy is zero.\nBut if there is some information transfer, then this comparison develops a peak, and shows exactly how much entropy has been transferred.\nWe can derive two additional measures from that, conditional transfer entropy and predictive information.\nI’ll explain the practical importance of conditional transfer entropy in the next slide.\n
• #10: First, we have apparent transfer entropy. It has severe issues with logical connections.\nIt can’t discover causality in multivariate couplings, it overlooks intermediate sources, and it has an issue with common causes.\n\nConditional transfer entropy, on the other hand, can deal with the latter two issues.\nHowever, multivariate interactions (green) can only be found when we attempt to interpret as many signals as possible. This is called the complete approach.\n\nFor the purpose of cognitive science, conditional Transfer Entropy is usually sufficient.\nSo, let’s take a look at the properties of Transfer Entropy when we apply it to our field.\n
• #11: Contrary to most other methods, transfer entropy doesn’t assume anything about its data.\nIf the source data is noise-free and linear, the underlying kernel function will be low-dimensional and very fast to reduce.\nThe method does underestimate the information flow in short data sequences, but this bias can be calculated and corrected.\nIn practice, even data sources of as little as 100 samples can be examined for their causality.\n
• #12: Experience with transfer entropy in neuroscience (mostly ECoG) shows that most causal relationships occur in the single digit timespan.\nGood news for us: neurons are fairly weakly coupled, and this means that bigger amounts of entropy are transferred with each step.\nComplex networks like neuronal networks open up additional issues, but these are independent from Transfer Entropy, and I’ll cover these in a bit.\n\n
• #13: We also have two problems, both of which are very important for applied research:\nFirst, a bias from input amplitude, which makes it hard to test for significance.\nSecond, a very strong causality result when two data streams contain the some of the same information at the same time.\nOf course, this causes problems with volume conduction in EEG.\n
• #14: Coming back to our issue with complex systems.\n Here, we encounter another, more basic problem: generalized synchronization\n Neurons are fairly slow and quite complicated structures, so let’s break it down to something more basic.\n A physicist, Ingo Fischer, built a setup of semiconductor lasers.\n Each laser is set up to modulate another laser, and they’re all arranged in a ring.\n Semiconductor lasers react very fast (on the order of nanoseconds) and are a bit noisy,\n so they can form a chaotic feedforward coupling, just as neurons.\n When you insert a probe into this system, you can perform a causality analysis on the output signals.\n Here’s the result for different amounts of lasers in the ring:\n Two lasers are fairly closely correlated, and also transfer entropy provides a nice contrast.\n With eight lasers, both transfer entropy and correlation have dropped to 0.1\n and in the asymptotic case of infinite lasers, both measurements drop to 0.\n But even in an infinitely large ring, two elements are just as strongly coupled as in the small setup!\n It’s just that neither correlation- nor entropy-based models can detect this coupling anymore.\n
• #15: Ok, enough with the theory, let’s take a look at something practical: Trentool.\nTrentool is an implementation for Transfer Entropy,\nand most importantly, it covers the two big problems: volume conduction and significance testing.\nIt uses a normalized version of conditional transfer entropy, which removes the input amplitude bias, and enables significance testing.\nVolume conduction is dealt with by analyzing causality for each possible delay, and removing the sub-milisecond-results from the other results.\nTherefore, it’s become the tool of choice for my own causal measurements.\nNow we’ll take a look under the hood and see what it does with its data.\n
• #16: A nice surprise was that Trentool works with Fieldtrip-styled data. It accepts Fieldtrip raw data, and its configuration is manipulated with a single cfg variable.\n
• #17: The parameter optimization estimates whether the data is stochastic or not, and uses one of these two components to estimate the kernel embedding dimension and the probable search range for delays.\nCao criterion: for deterministic data\nRagwitz criterion: for stochastic data\n
• #18: The data is shifted sample for sample, and then run through transfer entropy calculation.\nThe permutation test turns out a few values:\np-Value\nSignificance (uncorrected)\nStatistic Value (mean or t-value)\n
• #19: Shift test:\nDetecting volume conduction in multiple electrode settings\n
• #20: second permutation test between conditions\n
• #21: additional Final results:\nsignificance for each signal delay (corrected for multiple comparisons)\nestimate for volume conduction\n
• #22: The use case for multiple datasets is only slightly different, and the permutation test adapts the according significance values automatically.\n
• #23: This how it looks like in practice!\nThis is noisy data, so the estimation of kernel dimension always goes to the maximum. With a dimensionality and a search range of 50ms, the causality tests took about 30 minutes on a quad-core workstation.\n
• #24: Results for information transfer from X to Y: significant information transfer until 16ms, with a possible peak at 14ms\n
• #25: Same datasets, opposite direction: no significant effect\n
• #26: Ok, that’s it for today. Thank you!\n
• #27: \n