SlideShare a Scribd company logo

An Introduction to Causal Discovery, a Bayesian Network Approach

C
COST action BM1006

This gene ranked 152nd based on correlation alone. Using causal reasoning and Bayesian networks, the researchers were able to better identify genes that could causally influence the disease state, rather than just being correlated. This integrative approach combining genetic and gene expression data provided more insights into disease causality than traditional correlation-based methods alone.

Health & Medicine
1 of 71
1
2
3
4
5
6
7
8
9
10
11
12
13
Most read
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
Most read
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
Most read
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
Introduction to causal discovery: A Bayesian Networks approach Ioannis Tsamardinos1, 2 Sofia Triantafillou1, 2 Vincenzo Lagani1 1Bioinformatics Laboratory, Institute of Computer Science, Foundation for Research and Tech., Hellas 2Computer Science Department University of Crete
Democritus said that he would rather discover a single cause than be the king of Persia “Beyond such discarded fundamentals as ‘matter’ and ‘force’ lies still another fetish amidst the inscrutable arcana of modern science, namely the category of cause and effect” [K. Pearson]
What is causality? • What do you understand when I say : – smoking causes lung cancer?
What is (probabilistic) causality? • What do you understand when I say : – smoking causes lung cancer? • A causes B: – A causally affects B – Probabilistically – Intervening onto values of A will affect the distribution of B – in some appropriate context
Statistical Association (Unconditional Dependency) • Dep( X, Y | ) • X and Y are associated – Observing the value of X may change the conditional distribution of the (observed) values of Y: P(Y | X) P(Y) – Knowledge of X provides information for Y – Observed X is predictive for observed Y and vice versa – Knowing X changes our beliefs for the distribution of Y – Makes no claims about the distribution of Y, if instead of observing, we intervene on the values of X • Several means for measuring it
Association is NOT Causation • Yellow teeth and lung cancer are associated • Can I bleach my teeth and reduce the probability of getting lung cancer? • Is Smoking really causing Lung Cancer?
BUT “If A and B are correlated, A causes B OR B causes A OR they share a latent common cause“ [Hans Reichenbach]
Is Smoking Causing Lung Cancer? All possible models* common common cause cause Lung Lung Smoking Smoking Cancer Cancer Lung Smoking Cancer *assuming: 1. Smoking precedes Lung Cancer 2. No feedback cycles 3. Several hidden common causes can be modeled by a single hidden common cause
A way to learn causality 1. Take 200 people 2. Randomly split them in control and treatment groups 3. Force control group to smoke, force treatment group not to smoke 4. Wait until they are 60 years old 5. Measure correlation [ Randomized Control Trial ] [Sir Ronald Fisher]
Manipulation All possible models* common common RCT cause RCT cause Lung Lung Smoking Smoking Cancer Cancer RCT Lung Smoking Cancer
Manipulation removes other causes All possible models* common common RCT cause RCT cause Lung Lung Smoking Smoking Cancer Cancer RCT Lung Smoking Cancer
Manipulation removes other causes All possible models* common common RCT cause RCT cause Lung Lung Smoking Smoking Cancer Cancer RCT Lung Smoking Cancer Association persists only when relationship is causal
RCTs are hard • Can we learn anything from observational data?
RCTs are hard • Can we learn anything from observational data? “If A and B are correlated, A causes B OR B causes A OR they share a latent common cause“ [Hans Reichenbach]
Conditional Association (Conditional Dependency) • Dep( X, Y | Z) • X and Y are associated conditioned on Z – For some values of Z (some context) – Knowledge of X still provides information for Y – Observed X is still predictive for observed Y and vice versa • Statistically estimable
Conditioning and Causality Burglar Earthquake Alarm Call [example by Judea Pearl]
Conditioning and Causality Burglar Earthquake Alarm Call Dep(Burglar, Call| ) Burglar provides information for Call
Conditioning and Causality Learning the value of Burglar Earthquake intermediate and Alarm common causes renders variables Call independent Ind (Burglar, Call| Alarm) Burglar provides no information for Call once Alarm is known
Conditioning and Causality Burglar Earthquake Alarm Call Ind (Burglar, Earthquake| )
Conditioning and Causality Burglar Earthquake Alarm Call Ind (Burglar, Earthquake| )
Conditioning and Causality Learning the value of Burglar Earthquake common effects Alarm renders variables dependent Call Dep (Burglar, Earthquake| Alarm)
Observing data from a causal model Smoking Yellow Teeth Lung Cancer What would you observe? • Dep(Lung Cancer, Yellow Teeth| ) • Dep(Smoking, Lung Cancer | ) • Dep(Lung Cancer, Yellow Teeth | ) • Ind(Lung Cancer, Yellow Teeth| Smoking)
Observing data from a causal model Smoking Yellow Teeth Lung Cancer What would you observe? • Dep(Lung Cancer, Yellow Teeth| ) • Dep(Smoking, Lung Cancer | ) • Dep(Lung Cancer, Yellow Teeth | ) • Ind(Lung Cancer, Yellow Teeth| Smoking)
Observing data from a causal model Smoking Yellow Teeth Lung Cancer What would you observe? • Dep(Lung Cancer, Yellow Teeth| ) • Dep(Smoking, Lung Cancer | ) • Dep(Lung Cancer, Yellow Teeth | ) • Ind(Lung Cancer, Yellow Teeth| Smoking)
Observing data from a causal model Smoking Yellow Teeth Lung Cancer What would you observe? • Dep(Lung Cancer, Yellow Teeth| ) • Dep(Smoking, Lung Cancer | ) • Ind(Lung Cancer, Yellow Teeth | ) • Dep(Lung Cancer, Yellow Teeth| Smoking)
Markov Equivalent Networks Smoking Smoking Yellow Teeth Lung Cancer Yellow Teeth Lung Cancer Smoking • Same conditional Independencies • Same skeleton Yellow Teeth Lung Cancer • Same v-structures (subgraphs X Y Z no X-Z)
Causal Bayesian Networks* Graph G JPD J Smoking Lung Cancer Smoking Yellow Yes No Teeth Yellow Teeth Yes Yes 0,01 0,04 Yes No 0,01 0,04 No Yes 0,000045 0,044955 No No 0,000855 0,854145 Lung Cancer Assumptions about the nature of causality connect the graph G with the observed distribution J and allow reasoning *almost there
Causal Markov Condition (CMC) Smoking Lung Cancer Smoking Yellow Yes No Teeth Yellow Teeth Yes Yes 0,01 0,04 Yes No 0,01 0,04 No Yes 0,000045 0,044955 No No 0,000855 0,854145 Lung Cancer Every variable is (conditionally) independent of its non-effects (non-descendants in the graph) given its direct causes (parents)
Causal Markov Condition Smoking P(Yellow Teeth, Smoking, Lung Cancer) = P(Smoking) P(Yellow Teeth| Smoking) P(Lung Cancer| Smoking) Yellow Teeth Lung Cancer
Factorization with the CMC P(Yellow Teeth, Smoking, Lung Cancer) = P(Smoking) P(Yellow Teeth| Smoking) P(Lung Cancer| Smoking) Smoking P(Smoking) = 0.1 P(Yellow Teeth|Smoking) = 0.5 P(Yellow Teeth| Smoking) = 0.05 Yellow Teeth P(Lung Cancer|Smoking) = 0.2 P(Lung Cancer| Smoking) = 0.001 Lung Cancer
Using a Causal Bayesian Network Smoking 1. Factorize the jpd Medicine Y 2. Answer questions like: 1. P(Lung Cancer| Levels of Protein X) = ? Levels of Protein X Lung 2. Ind(Smoking, Fatigue| Levels of Cancer Protein X)? Yellow- stained 3. What will happen if I design a Fingers drug that blocks the function of protein X (predict effect of Fatigue interventions)?
Bayesian Networks I don’t like all these assumptions I kind of liked reducing the parameters of the distribution Drop the Causal part!
Using a Bayesian Network Smoking 1. Factorize the jpd Medicine Y 2. Answer questions like: 1. P(Lung Cancer| Levels of Protein X) = ? Levels of 2. Ind(Smoking, Fatigue| Protein X Lung Cancer Levels of Protein X)? Yellow-stained Fingers Fatigue
Observing a causal model Raw data Subject # Smoking Yellow Lung Smoking Teeth Cancer 1 0 0 0 2 1 1 0 3 1 1 0 Yellow Teeth Lung Cancer 4 0 1 0 5 0 0 0 6 1 1 0 7 1 1 1 8 0 1 0 9 0 0 0 What would you observe? 10 1 1 0 • Dep(Lung Cancer, Yellow Teeth| ) 11 1 1 0 • Dep(Smoking, Lung Cancer | ) . . . . • Dep(Lung Cancer, Yellow Teeth | ) . . . . • Ind(Lung Cancer, Yellow Teeth| Smoking) . . . . 10000 1 1 1
Learning Set of Equivalent Networks Constraint-Based Approach Score-Based (Bayesian) Test conditional Find the DAG with the independencies in maximum a posteriori data and find a DAG that probability given the encodes them data
Learning the network Constraint-Based Approach Bayesian Approach •SGS [Spirtes, Glymour, & Scheines 2000] •K2 [Cooper and Herskowitz 1992] • PC [Spirtes, Glymour, & Scheines 2000] •GBPS [Spirtes and Meek 1995] •GES [Chickering and Meek 2002] • TPDA [Cheng et al., 1997] •Sparse Candidate [Friedman et al. 1999] •CPC [Ramsey et al, 2006] •Optimal Reinsertion [Moore and Wong 2003] Hybrid •Rec [Xie, X, Geng, Zhi, JMLR 2008] •Exact Algorithms [Koivisto et al., 2004] , •MMHC [Tsamardinos et al. 2006] [Koivisto, 2006] , [Silander & Myllymaki, 2006] • CB [Provan et al. 1995] • BENEDICT [Provan and de Campos 2001] •ECOS [Kaname et al. 2010] Many more!!!
Assumptions* • Tests of Conditional Independences / Scoring methods may not be appropriate for the type of data at hand • Faithfulness • No feedback cycles • No determinism • No latent variables • No measurement error • No averaging effects *aka why the algorithms may not work
Faithfulness • ALL conditional (in)dependencies stem from the CMC Time in +0.6 +0.5 the sun Sunscreen Skin Cancer -0.3 Time in •Markov Condition does not imply: the sun •Ind(Skin Cancer, Sunscreen) •Unfaithful if: Skin Cancer •Ind(Skin Cancer, Sunscreen) Sunscreen
Collinearity and Determinism X Y W Z X Z W Y • Assume Y and Z are information equivalent (e.g., one-to- one deterministic relation) • Cannot distinguish the two graphs • A specific type of violation of Faithfulness
No Feedback Cycles Studying Good Grades • Studying causes Good Grades causes more studying (at a later time!)… • Hard to define without explicitly representing time • If all relations are linear, we can assume we sample from the distribution of the equilibrium of the system when external factors are kept constant – Path-diagrams (Structural Equation Models with no measurement model part) allow such feedback loops • If there is feedback and relations are not linear, there may be chaos, literally (mathematically) and metaphorically
No Latent Confounders Heat (Not recorded) • Dep (Ice Cream, Polio) • No CAUSAL Bayesian Network on the Ice cream Polio modeled variables ONLY captures causal Ice cream Polio relations correctly Ice cream • Both Bayesian Polio Networks capture associations correctly (not always the case)
Effects of Measurement Error X Y W True Model X Y W Possible Induced Y X W Model • X, Y, Z the actual physical quantities • X , Y , Z the measured quantities (+ noise) • If Y is measured with more error than X then Dep(X ;W | Y )
Effects of Averaging • Almost all omics technologies measure average quantities over millions of cells • The quantities in the models though refer to single-cells X Y W True Model Possibly Induced Xi Yi Wi Model
Success Stories • Identify genes that cause a phenotype. – [Schadt et al., Nature Genetics, 2005] • Reconstruct causal pathways. – [K. Sachs, et al. Science , (2005)] • Identify causal effects. – [Maathius et al., Nature Methods, 2010] • Predict association among variables never measured together. – [Tsamardinos et al., JMLR, 2012] • Select features that are most predictive of a target variable. – [Aliferis et. al., JMLR, 2010 ]
An integrative genomics approach to infer causal associations between gene expression and disease L –Locus of DNA variation R – gene expression C- Phenotype (Omental Fat Pad Mass trait) Biological knowledge: Nothing causally affects L L R C Causal model L R C Reactive model L R C Independent model
An integrative genomics approach to infer causal associations between gene expression and disease 1. Identify loci susceptible for causing the disease • 4 QTLs 2. Identify gene expression traits correlated with the disease • 440 genes 3. Identify genes with eQTLs that coincide with the QTLs • 113 genes, 267 eQTLs 4. Identify genes that support causal models 5. Rank genes by causal effect One of them ranked 152 out of the 440 based on mere correlation
An integrative genomics approach to infer causal associations between gene expression and disease 1. Identify loci susceptible for causing the disease • 4 QTLs 2. Identify gene expression traits correlated with the disease • 440 genes 3. Identify genes with eQTLs that coincide with the QTLs • 113 genes, 267 eQTLs 4. Identify genes that support causal models 5. Rank genes by causal effect 4 of the top ranked genes where experimentally validated as causal
Causal Protein-Signaling Networks Derived from Multi-parameter Single-Cell Data CD3 CD28 LFA-1 L PI3K RAS Cytohesin JAB-1 Zap70 A VAV SLP-76 LckT PKC PLC PIP3 Akt PIP2 PKA Raf MAPKKK MAPKKK Mek1/2 • Protein Signaling Pathways Erk1/2 resemble Causal Bayesian Networks MEK4/7 MEK3/6 • Use Causal Bayesian Networks learning JNK p38 to reconstruct a Protein Signaling pathway [K. Sachs, et al. Science , (2005)]
Reconstructed vs. Actual Network Phospho-Proteins Phospho-Lipids PKC Perturbed in data Expected Pathway PKA Reported Raf Plc Reversed Jnk P38 Missed Mek PIP3 Erk  15/17 Classic PIP2 Akt  17/17 Reported  3 Missed
Predicting causal effects in large-scale systems from observational data What will happen if you knock down Gene X? Gene A Gene B … Gene X 1 0.1 0.5 1.2 Intervention calculus when DAG is Absent 2 0.56 2.32 0.7 … 1. For every DAG G faithful to the data n 7 0.4 2.4 c 2. Causal effect G of X on V in DAG G is 1. 0, if V PaX (G) 2. coefficient of X in V∼X + PaX (G), otherwise 3. Causal effect c of X on V is the minimum of all cG
Predicting causal effects in large-scale systems from observational data IDA Evaluation Experimental Observational Data* Data** Rank causal effects Apply IDA. Take top m percent Take top q genes Rosetta Compendium data: Compare • 5,361 genes • 234 single-gene deletion mutants* • 63 wild-type measurements**
Predicting causal effects in large-scale systems from observational data • m=10%
Integrative Causal Analysis • Make inferences from multiple heterogeneous datasets – That measure quantities under different experimental conditions – Measure different (overlapping) sets of quantities – In the context of prior knowledge • General Idea: – Find all CAUSAL models that simultaneously fit all datasets and are consistent with prior knowledge – Reason with the set of all such models
Reason with Set of Solutions X Y Z W Y-Z ? X Y Z W Variables X Y Z W Y X W Z X Y Z W samples ρXW.Y = 0 X Y Z W |ρΧΥ | > | ρΧZ | X Y Z W ρ XW.Z= 0 X Y Z W
Make Predictions X Y Z W Y-Z X Y Z W Variables X Y Z W Y X W Z X Y Z W samples ρXW.Y = 0 X Y Z W |ρΧΥ | > | ρΧZ | X Y Z W ρ XW.Z= 0 X Y Z W Nothing causes X
Incorporating Prior Knowledge X Y Z W Y-Z X Y Z W Variables X Y Z W Y X W Z X Y Z W samples ρXW.Y = 0 X Y Z W |ρΧΥ | > | ρΧZ | X Y Z W ρ XW.Z= 0 X Y Z W Nothing causes X
Further Inductions X Y Z W X Y Z W X Y Z W Variables X Y Z W Y X W Z samples ρXW.Y = 0 X Y Z W |ρΧΥ | > | ρΧZ | X Y Z W ρ XW.Z= 0 X Y Z W Nothing causes X Changing Y will have an effect on Z
Proof-of-concept Results I Tsamardinos, S Triantafillou and V Lagani, Towards Integrative Causal Analysis of Heterogeneous Datasets and Studies, Journal of Machine Learning Research, to appear 20 datasets Actual correlation Biological, Financial, Text, Medical, Social 698897 predictions 98% accuracy vs. 16% for random guessing Predicted correlation 0.79 R2 between predicted and sample correlation
Causality and Feature Selection • Question: Find a minimal set of molecular quantities that collectively carries all the information for optimal prediction / diagnosis (target variable) (Molecular Signature) – Minimal: throw away irrelevant or superfluous features – Collectively: May need to consider interactions – Optimal: Requires constructing a classification / regression model and estimating its performance • Answer*:It is the direct causes, the direct effects, and the direct causes of the direct effects of the target variable in the BN (called the Markov Blanket in this context) * Adopting all causal assumptions
Markov Blanket CD3 CD28 LFA-1 L PI3K RAS Cytohesin JAB-1 Zap70 A VAV T SLP-76 Lck PKC PLC PIP3 Akt PIP2 PKA Raf Quantity of Interest MAPKKK MAPKKK Mek1/2 MEK4/7 MEK3/6 Erk1/2 JNK p38
Markov Blanket CD3 CD28 LFA-1 L PI3K RAS Cytohesin JAB-1 Zap70 A VAV T SLP-76 Lck PKC PLC PIP3 Akt PIP2 PKA Raf Quantity of Interest MAPKKK MAPKKK Mek1/2 Signature / MEK4/7 MEK3/6 Erk1/2 Markov JNK p38 Blanket
Markov Blanket Algorithms • Efficient and accurate algorithms applicable to datasets with hundreds of thousands of variables – Max-Min Markov Blanket, [Tsamardinos, Aliferis, Statnikov, KDD 2003] – HITON [Aliferis, Tsamardinos, Statnikov, AMIA 2003] – [Aliferis, Statnikov, Tsamardinos, et. al. JMLR 2010] • State-of-the-art in variable selection
Objective •Identifying a set of transcripts able to predict IKAROS gene expression. •The selected set should be: – Maximally informative: able to predict IKAROS expression with optimal accuracy – Minimal: containing no redundant or uninformative transcripts
Data • Genome-wide transciptome data from HapMap individual of European descent [Montgomery et al., 2010] – Lymphoblast cells – 60 distinct individuals – Approximately. ~140K transcripts • RKPM values freely available from ArrayExpress – www.ebi.ac.uk/arrayexpress/experiments/E-MTAB-197
Methods • Constraint-based, local learning feature selection method for identifying multiple signatures – [Tsamardinos, Lagani and Pappas, 2012] • Support Vector Machine (SVM) for providing testable predictions – [Chang and Lin, 2011] • Nested cross validation procedure for: – setting algorithms’ parameters – providing unbiased performance estimations – [Statnikov, Aliferis, Tsamardinos, et al., 2005]
Results • 22 different signatures found to be equally maximally predictive – Mean Absolute Error: 1.93 – R2: 0.7159 – Correlation of predictions and true expressions: – 0.8461 (p-value < 0.0001) • Example signature: – ENST00000246549, ENST00000545189, ENST00000265495, ENST00000398483, ENST00000496570 • Corresponding to genes: – FFAR2, ZNF426, ELF2, MRPL48, DNMT3A
Predicted vs. Observed IKZF1 values
Beyond This Tutorial Textbooks: • Pearl, J. Causality: models, reasoning and inference (Cambridge University Press: 2000). • Spirtes, P., Glymour, C. & Scheines, R. Causation, Prediction, and Search. (The MIT Press: 2001). • Neapolitan, R. Learning Bayesian Networks. (Prentice Hall: 2003).
Beyond This Tutorial Different principles for discovering causality • Shimizu, S., Hoyer, P.O., Hyvärinen, A. & Kerminen, A. A Linear Non-Gaussian Acyclic Model for Causal Discovery. Journal of Machine Learning Research 7, 2003-2030 (2006). Hoyer, P., Janzing, D., Mooij, J., Peters, J. & Schölkopf, B. Nonlinear causal discovery with additive noise models. Neural Information Processing Systems (NIPS) 21, 689-696 (2009). Causality with Feedback cycles • Hyttinen, A., Eberhardt, F., Hoyer, P.O., Learning Linear Cyclic Causal Models with Latent Variables. Journal of Machine Learning Research, 13(Nov):3387-3439, 2012. Causality with Latent Variables • Richardson, T. & Spirtes, P. Ancestral Graph Markov Models. The Annals of Statistics 30, 962-1030 (2002). • Leray, P., Meganck, S., Maes, S. & Manderick, B. Causal graphical models with latent variables: Learning and inference. Innovations in Bayesian Networks 156, 219-249 (2008).
Conclusions • Causal Discovery is possible from observational data or by limited experiments • Beware of violations assumptions and equivalences • Causality provides a formal language for conceptualizing data analysis problems • Necessary to predict the effect of interventions • Deep connections to Feature Selection • Allows integrative analysis in novel ways • Advanced theory and algorithms exist for different sets of (less restrictive) assumptions • Way to go still, particularly in disseminating to non-experts
References • S. B. Montgomery et al. Transcriptome genetics using second generation sequencing in a Caucasian population. Nature 464, 773-777 (1 April 2010) • I. Tsamardinos, V. Lagani and D. Pappas. Discovering multiple, equivalent biomarker signatures. In Proceeding of the 7th conference of the Hellenic Society for Computational Biology & Bioinformatics (HSCBB) 2012 • C. Chang and C. Lin. LIBSVM: a library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2:27:1--27:27, 2011 • A. Statnikov, C.F. Aliferis, I. Tsamardinos, D. Hardin, and S.Levy. A comprehensive evaluation of multicategory classification methods for microarray gene expression cancer diagnosis. Bioinformatics, 21:631-643, 2005.

More Related Content

PDF
Variational Autoencoders VAE - Santiago Pascual - UPC Barcelona 2018
Universitat Politècnica de Catalunya
 
PDF
Discovery of Linear Acyclic Models Using Independent Component Analysis
Shiga University, RIKEN
 
PPTX
Semi supervised, weakly-supervised, unsupervised, and active learning
Yusuke Uchida
 
PDF
A direct method for estimating linear non-Gaussian acyclic models
Shiga University, RIKEN
 
ODP
Probabilistic programming
Eli Gottlieb
 
PDF
SSII2019TS: Shall We GANs?​ ～GANの基礎から最近の研究まで～
SSII
 
PPTX
【論文読み会】PiCO_Contrastive Label Disambiguation for Partial Label Learning.pptx
ARISE analytics
 
PDF
Causal discovery and prediction mechanisms
Shiga University, RIKEN
 
Variational Autoencoders VAE - Santiago Pascual - UPC Barcelona 2018
Universitat Politècnica de Catalunya
 
Discovery of Linear Acyclic Models Using Independent Component Analysis
Shiga University, RIKEN
 
Semi supervised, weakly-supervised, unsupervised, and active learning
Yusuke Uchida
 
A direct method for estimating linear non-Gaussian acyclic models
Shiga University, RIKEN
 
Probabilistic programming
Eli Gottlieb
 
SSII2019TS: Shall We GANs?​ ～GANの基礎から最近の研究まで～
SSII
 
【論文読み会】PiCO_Contrastive Label Disambiguation for Partial Label Learning.pptx
ARISE analytics
 
Causal discovery and prediction mechanisms
Shiga University, RIKEN
 

What's hot (20)

PDF
[DL輪読会]Recent Advances in Autoencoder-Based Representation Learning
Deep Learning JP
 
PPT
activelearning.ppt
butest
 
PDF
Introduction to Diffusion Models
Sangwoo Mo
 
PDF
GAN（と強化学習との関係）
Masahiro Suzuki
 
PDF
因果探索: 基本から最近の発展までを概説
Shiga University, RIKEN
 
PDF
Denoising Diffusion Probabilistic Modelsの重要な式の解説
Tomonari Masada
 
PDF
Non-Gaussian Methods for Learning Linear Structural Equation Models: Part I
Shiga University, RIKEN
 
PPTX
PRML Chapter 8
Sunwoo Kim
 
PDF
IIBMP2016 深層生成モデルによる表現学習
Preferred Networks
 
PDF
PRML学習者から入る深層生成モデル入門
tmtm otm
 
PDF
論文紹介-Multi-Objective Deep Reinforcement Learning
Shunta Nomura
 
PDF
[DL輪読会] Spectral Norm Regularization for Improving the Generalizability of De...
Deep Learning JP
 
PDF
[Ｒ勉強会][データマイニング] R言語による時系列分析
Koichi Hamada
 
PDF
Towards Causal Representation Learning
Suyeong Park
 
PDF
独立成分分析 ICA
Daisuke Yoneoka
 
PDF
統計的因果推論への招待 -因果構造探索を中心に-
Shiga University, RIKEN
 
PDF
人工知能2018 強化学習の応用
Hirotaka Hachiya
 
PPTX
DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulation
ssuser2e0133
 
PDF
Graph Neural Network in practice
tuxette
 
PDF
Independent Component Analysis
Tatsuya Yokota
 
[DL輪読会]Recent Advances in Autoencoder-Based Representation Learning
Deep Learning JP
 
activelearning.ppt
butest
 
Introduction to Diffusion Models
Sangwoo Mo
 
GAN（と強化学習との関係）
Masahiro Suzuki
 
因果探索: 基本から最近の発展までを概説
Shiga University, RIKEN
 
Denoising Diffusion Probabilistic Modelsの重要な式の解説
Tomonari Masada
 
Non-Gaussian Methods for Learning Linear Structural Equation Models: Part I
Shiga University, RIKEN
 
PRML Chapter 8
Sunwoo Kim
 
IIBMP2016 深層生成モデルによる表現学習
Preferred Networks
 
PRML学習者から入る深層生成モデル入門
tmtm otm
 
論文紹介-Multi-Objective Deep Reinforcement Learning
Shunta Nomura
 
[DL輪読会] Spectral Norm Regularization for Improving the Generalizability of De...
Deep Learning JP
 
[Ｒ勉強会][データマイニング] R言語による時系列分析
Koichi Hamada
 
Towards Causal Representation Learning
Suyeong Park
 
独立成分分析 ICA
Daisuke Yoneoka
 
統計的因果推論への招待 -因果構造探索を中心に-
Shiga University, RIKEN
 
人工知能2018 強化学習の応用
Hirotaka Hachiya
 
DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulation
ssuser2e0133
 
Graph Neural Network in practice
tuxette
 
Independent Component Analysis
Tatsuya Yokota
 
Ad

Similar to An Introduction to Causal Discovery, a Bayesian Network Approach (20)

PDF
The book of Why
mustafa sarac
 
PDF
Epinor presentation 24.09.2015.
EPINOR
 
PDF
7. Bias and Causation medical Statistics.pdf
hilalkhuram0911
 
PPTX
Association causation
Vishnu Yenganti
 
PDF
Association and Causation
Dr. Animesh Gupta
 
PDF
weon preconference 2013 vandenbroucke counterfactual theory causality epidem...
Bsie
 
PDF
Type of epidemiology and epidemiological methods
pratikshapaudel3
 
PPS
Russo Madrid Medicine Oct07
Utrecht University and University College London
 
PPT
Epidemiology Study Design
Mohammad Ismail Zubair MD. MSc
 
PPTX
causation association and its types. Bradford hill's criteria
Dr. Anu Marhatta
 
PPTX
Dolls case control study
Mrinmoy Bharadwaz
 
PDF
Causal Inference in Data Science and Machine Learning
Bill Liu
 
PPTX
CASUAL INFERENCE.pptx
kihembopamelah
 
PPTX
Data science as a science
jtleek
 
PDF
Causation and association of disease
NurAlam132
 
PPTX
Friis chapter 1 slides
todd.damrow
 
PPTX
causation.main.association to causaty slide
MBAamer1
 
PPT
Bradford Hill Criteria.ppt
externalReviewer
 
PPTX
4.1. introduction
A M
 
PPT
Cohort study design.ppt Epidemiology medical
SoravSorout
 
The book of Why
mustafa sarac
 
Epinor presentation 24.09.2015.
EPINOR
 
7. Bias and Causation medical Statistics.pdf
hilalkhuram0911
 
Association causation
Vishnu Yenganti
 
Association and Causation
Dr. Animesh Gupta
 
weon preconference 2013 vandenbroucke counterfactual theory causality epidem...
Bsie
 
Type of epidemiology and epidemiological methods
pratikshapaudel3
 
Russo Madrid Medicine Oct07
Utrecht University and University College London
 
Epidemiology Study Design
Mohammad Ismail Zubair MD. MSc
 
causation association and its types. Bradford hill's criteria
Dr. Anu Marhatta
 
Dolls case control study
Mrinmoy Bharadwaz
 
Causal Inference in Data Science and Machine Learning
Bill Liu
 
CASUAL INFERENCE.pptx
kihembopamelah
 
Data science as a science
jtleek
 
Causation and association of disease
NurAlam132
 
Friis chapter 1 slides
todd.damrow
 
causation.main.association to causaty slide
MBAamer1
 
Bradford Hill Criteria.ppt
externalReviewer
 
4.1. introduction
A M
 
Cohort study design.ppt Epidemiology medical
SoravSorout
 
Ad

More from COST action BM1006 (11)

PDF
Knowledge management for integrative omics data analysis
COST action BM1006
 
PDF
Reverse-engineering techniques in Data Integration
COST action BM1006
 
PDF
from B-cell Biology to Data Integration
COST action BM1006
 
PDF
Mechanisms of Asthma and Allergy (MeDALL): from population based birth cohort...
COST action BM1006
 
PDF
Integrative Analysis of Epigenomics and miRNA data in Immune System Models
COST action BM1006
 
PDF
Proteomics analysis: Basics and Applications
COST action BM1006
 
PDF
Metabolomics Data Analysis
COST action BM1006
 
PDF
Metabolomics: data acquisition, pre-processing and quality control
COST action BM1006
 
PDF
RNA-seq Analysis
COST action BM1006
 
PDF
ChipSeq Data Analysis
COST action BM1006
 
PDF
X-omics Data Integration Challenges
COST action BM1006
 
Knowledge management for integrative omics data analysis
COST action BM1006
 
Reverse-engineering techniques in Data Integration
COST action BM1006
 
from B-cell Biology to Data Integration
COST action BM1006
 
Mechanisms of Asthma and Allergy (MeDALL): from population based birth cohort...
COST action BM1006
 
Integrative Analysis of Epigenomics and miRNA data in Immune System Models
COST action BM1006
 
Proteomics analysis: Basics and Applications
COST action BM1006
 
Metabolomics Data Analysis
COST action BM1006
 
Metabolomics: data acquisition, pre-processing and quality control
COST action BM1006
 
RNA-seq Analysis
COST action BM1006
 
ChipSeq Data Analysis
COST action BM1006
 
X-omics Data Integration Challenges
COST action BM1006
 

Recently uploaded (20)

PPTX
Important Obstetric Emergency that must be recognised
khusyairi0
 
PDF
NEET PG 2025 | 200 High-Yield Recall Topics Across All Subjects
Shivankan Kakkar
 
PPTX
Electromyography (EMG) in Physiotherapy: Principles, Procedure & Clinical App...
BALAJI SOMA
 
PPTX
SKIN Anatomy and physiology and associated diseases
ReshmaMurali14
 
PDF
Handout_ NURS 220 Topic 10-Abnormal Pregnancy.pdf
zuluuke
 
PPTX
Neuropathic pain.ppt treatment managment
BaharBazooyar
 
PDF
Therapeutic Potential of Citrus Flavonoids in Metabolic Inflammation and Ins...
publication11
 
PPTX
JUVENILE NASOPHARYNGEAL ANGIOFIBROMA.pptx
rounakuser
 
PDF
CT Anatomy for Radiotherapy.pdf eryuioooop
DrHabtamu1
 
PPTX
POLYCYSTIC OVARIAN SYNDROME.pptx by Dr( med) Charles Amoateng
AMOATENGCHARLESNANAA
 
PDF
Medical Evidence in the Criminal Justice Delivery System in.pdf
academicstrivee
 
DOCX
RUHS II MBBS Microbiology Paper-II with Answer Key | 6th August 2025 (New Sch...
Shivankan Kakkar
 
PPTX
Uterus anatomy embryology, and clinical aspects
Mandeep Sharma
 
PPTX
ca esophagus molecula biology detailaed molecular biology of tumors of esophagus
akashdeepsohisge
 
PPTX
Gastroschisis- Clinical Overview 18112311
Nathan Lupiya
 
PPTX
Chapter-1-The-Human-Body-Orientation-Edited-55-slides.pptx
JANDIESAGAYNO2
 
PPTX
Respiratory drugs, drugs acting on the respi system
scsbtzbq9b
 
PPTX
1 General Principles of Radiotherapy.pptx
DrHabtamu1
 
PDF
Khadir.pdf Acacia catechu drug Ayurvedic medicine
BansariPatel112358
 
PDF
Deadly Stampede at Yaounde’s Olembe Stadium Forensic.pdf
academicstrivee
 
Important Obstetric Emergency that must be recognised
khusyairi0
 
NEET PG 2025 | 200 High-Yield Recall Topics Across All Subjects
Shivankan Kakkar
 
Electromyography (EMG) in Physiotherapy: Principles, Procedure & Clinical App...
BALAJI SOMA
 
SKIN Anatomy and physiology and associated diseases
ReshmaMurali14
 
Handout_ NURS 220 Topic 10-Abnormal Pregnancy.pdf
zuluuke
 
Neuropathic pain.ppt treatment managment
BaharBazooyar
 
Therapeutic Potential of Citrus Flavonoids in Metabolic Inflammation and Ins...
publication11
 
JUVENILE NASOPHARYNGEAL ANGIOFIBROMA.pptx
rounakuser
 
CT Anatomy for Radiotherapy.pdf eryuioooop
DrHabtamu1
 
POLYCYSTIC OVARIAN SYNDROME.pptx by Dr( med) Charles Amoateng
AMOATENGCHARLESNANAA
 
Medical Evidence in the Criminal Justice Delivery System in.pdf
academicstrivee
 
RUHS II MBBS Microbiology Paper-II with Answer Key | 6th August 2025 (New Sch...
Shivankan Kakkar
 
Uterus anatomy embryology, and clinical aspects
Mandeep Sharma
 
ca esophagus molecula biology detailaed molecular biology of tumors of esophagus
akashdeepsohisge
 
Gastroschisis- Clinical Overview 18112311
Nathan Lupiya
 
Chapter-1-The-Human-Body-Orientation-Edited-55-slides.pptx
JANDIESAGAYNO2
 
Respiratory drugs, drugs acting on the respi system
scsbtzbq9b
 
1 General Principles of Radiotherapy.pptx
DrHabtamu1
 
Khadir.pdf Acacia catechu drug Ayurvedic medicine
BansariPatel112358
 
Deadly Stampede at Yaounde’s Olembe Stadium Forensic.pdf
academicstrivee
 

An Introduction to Causal Discovery, a Bayesian Network Approach

  • 1. Introduction to causal discovery: A Bayesian Networks approach Ioannis Tsamardinos1, 2 Sofia Triantafillou1, 2 Vincenzo Lagani1 1Bioinformatics Laboratory, Institute of Computer Science, Foundation for Research and Tech., Hellas 2Computer Science Department University of Crete
  • 2. Democritus said that he would rather discover a single cause than be the king of Persia “Beyond such discarded fundamentals as ‘matter’ and ‘force’ lies still another fetish amidst the inscrutable arcana of modern science, namely the category of cause and effect” [K. Pearson]
  • 3. What is causality? • What do you understand when I say : – smoking causes lung cancer?
  • 4. What is (probabilistic) causality? • What do you understand when I say : – smoking causes lung cancer? • A causes B: – A causally affects B – Probabilistically – Intervening onto values of A will affect the distribution of B – in some appropriate context
  • 5. Statistical Association (Unconditional Dependency) • Dep( X, Y | ) • X and Y are associated – Observing the value of X may change the conditional distribution of the (observed) values of Y: P(Y | X) P(Y) – Knowledge of X provides information for Y – Observed X is predictive for observed Y and vice versa – Knowing X changes our beliefs for the distribution of Y – Makes no claims about the distribution of Y, if instead of observing, we intervene on the values of X • Several means for measuring it
  • 6. Association is NOT Causation • Yellow teeth and lung cancer are associated • Can I bleach my teeth and reduce the probability of getting lung cancer? • Is Smoking really causing Lung Cancer?
  • 7. BUT “If A and B are correlated, A causes B OR B causes A OR they share a latent common cause“ [Hans Reichenbach]
  • 8. Is Smoking Causing Lung Cancer? All possible models* common common cause cause Lung Lung Smoking Smoking Cancer Cancer Lung Smoking Cancer *assuming: 1. Smoking precedes Lung Cancer 2. No feedback cycles 3. Several hidden common causes can be modeled by a single hidden common cause
  • 9. A way to learn causality 1. Take 200 people 2. Randomly split them in control and treatment groups 3. Force control group to smoke, force treatment group not to smoke 4. Wait until they are 60 years old 5. Measure correlation [ Randomized Control Trial ] [Sir Ronald Fisher]
  • 10. Manipulation All possible models* common common RCT cause RCT cause Lung Lung Smoking Smoking Cancer Cancer RCT Lung Smoking Cancer
  • 11. Manipulation removes other causes All possible models* common common RCT cause RCT cause Lung Lung Smoking Smoking Cancer Cancer RCT Lung Smoking Cancer
  • 12. Manipulation removes other causes All possible models* common common RCT cause RCT cause Lung Lung Smoking Smoking Cancer Cancer RCT Lung Smoking Cancer Association persists only when relationship is causal
  • 13. RCTs are hard • Can we learn anything from observational data?
  • 14. RCTs are hard • Can we learn anything from observational data? “If A and B are correlated, A causes B OR B causes A OR they share a latent common cause“ [Hans Reichenbach]
  • 15. Conditional Association (Conditional Dependency) • Dep( X, Y | Z) • X and Y are associated conditioned on Z – For some values of Z (some context) – Knowledge of X still provides information for Y – Observed X is still predictive for observed Y and vice versa • Statistically estimable
  • 16. Conditioning and Causality Burglar Earthquake Alarm Call [example by Judea Pearl]
  • 17. Conditioning and Causality Burglar Earthquake Alarm Call Dep(Burglar, Call| ) Burglar provides information for Call
  • 18. Conditioning and Causality Learning the value of Burglar Earthquake intermediate and Alarm common causes renders variables Call independent Ind (Burglar, Call| Alarm) Burglar provides no information for Call once Alarm is known
  • 19. Conditioning and Causality Burglar Earthquake Alarm Call Ind (Burglar, Earthquake| )
  • 20. Conditioning and Causality Burglar Earthquake Alarm Call Ind (Burglar, Earthquake| )
  • 21. Conditioning and Causality Learning the value of Burglar Earthquake common effects Alarm renders variables dependent Call Dep (Burglar, Earthquake| Alarm)
  • 22. Observing data from a causal model Smoking Yellow Teeth Lung Cancer What would you observe? • Dep(Lung Cancer, Yellow Teeth| ) • Dep(Smoking, Lung Cancer | ) • Dep(Lung Cancer, Yellow Teeth | ) • Ind(Lung Cancer, Yellow Teeth| Smoking)
  • 23. Observing data from a causal model Smoking Yellow Teeth Lung Cancer What would you observe? • Dep(Lung Cancer, Yellow Teeth| ) • Dep(Smoking, Lung Cancer | ) • Dep(Lung Cancer, Yellow Teeth | ) • Ind(Lung Cancer, Yellow Teeth| Smoking)
  • 24. Observing data from a causal model Smoking Yellow Teeth Lung Cancer What would you observe? • Dep(Lung Cancer, Yellow Teeth| ) • Dep(Smoking, Lung Cancer | ) • Dep(Lung Cancer, Yellow Teeth | ) • Ind(Lung Cancer, Yellow Teeth| Smoking)
  • 25. Observing data from a causal model Smoking Yellow Teeth Lung Cancer What would you observe? • Dep(Lung Cancer, Yellow Teeth| ) • Dep(Smoking, Lung Cancer | ) • Ind(Lung Cancer, Yellow Teeth | ) • Dep(Lung Cancer, Yellow Teeth| Smoking)
  • 26. Markov Equivalent Networks Smoking Smoking Yellow Teeth Lung Cancer Yellow Teeth Lung Cancer Smoking • Same conditional Independencies • Same skeleton Yellow Teeth Lung Cancer • Same v-structures (subgraphs X Y Z no X-Z)
  • 27. Causal Bayesian Networks* Graph G JPD J Smoking Lung Cancer Smoking Yellow Yes No Teeth Yellow Teeth Yes Yes 0,01 0,04 Yes No 0,01 0,04 No Yes 0,000045 0,044955 No No 0,000855 0,854145 Lung Cancer Assumptions about the nature of causality connect the graph G with the observed distribution J and allow reasoning *almost there
  • 28. Causal Markov Condition (CMC) Smoking Lung Cancer Smoking Yellow Yes No Teeth Yellow Teeth Yes Yes 0,01 0,04 Yes No 0,01 0,04 No Yes 0,000045 0,044955 No No 0,000855 0,854145 Lung Cancer Every variable is (conditionally) independent of its non-effects (non-descendants in the graph) given its direct causes (parents)
  • 29. Causal Markov Condition Smoking P(Yellow Teeth, Smoking, Lung Cancer) = P(Smoking) P(Yellow Teeth| Smoking) P(Lung Cancer| Smoking) Yellow Teeth Lung Cancer
  • 30. Factorization with the CMC P(Yellow Teeth, Smoking, Lung Cancer) = P(Smoking) P(Yellow Teeth| Smoking) P(Lung Cancer| Smoking) Smoking P(Smoking) = 0.1 P(Yellow Teeth|Smoking) = 0.5 P(Yellow Teeth| Smoking) = 0.05 Yellow Teeth P(Lung Cancer|Smoking) = 0.2 P(Lung Cancer| Smoking) = 0.001 Lung Cancer
  • 31. Using a Causal Bayesian Network Smoking 1. Factorize the jpd Medicine Y 2. Answer questions like: 1. P(Lung Cancer| Levels of Protein X) = ? Levels of Protein X Lung 2. Ind(Smoking, Fatigue| Levels of Cancer Protein X)? Yellow- stained 3. What will happen if I design a Fingers drug that blocks the function of protein X (predict effect of Fatigue interventions)?
  • 32. Bayesian Networks I don’t like all these assumptions I kind of liked reducing the parameters of the distribution Drop the Causal part!
  • 33. Using a Bayesian Network Smoking 1. Factorize the jpd Medicine Y 2. Answer questions like: 1. P(Lung Cancer| Levels of Protein X) = ? Levels of 2. Ind(Smoking, Fatigue| Protein X Lung Cancer Levels of Protein X)? Yellow-stained Fingers Fatigue
  • 34. Observing a causal model Raw data Subject # Smoking Yellow Lung Smoking Teeth Cancer 1 0 0 0 2 1 1 0 3 1 1 0 Yellow Teeth Lung Cancer 4 0 1 0 5 0 0 0 6 1 1 0 7 1 1 1 8 0 1 0 9 0 0 0 What would you observe? 10 1 1 0 • Dep(Lung Cancer, Yellow Teeth| ) 11 1 1 0 • Dep(Smoking, Lung Cancer | ) . . . . • Dep(Lung Cancer, Yellow Teeth | ) . . . . • Ind(Lung Cancer, Yellow Teeth| Smoking) . . . . 10000 1 1 1
  • 35. Learning Set of Equivalent Networks Constraint-Based Approach Score-Based (Bayesian) Test conditional Find the DAG with the independencies in maximum a posteriori data and find a DAG that probability given the encodes them data
  • 36. Learning the network Constraint-Based Approach Bayesian Approach •SGS [Spirtes, Glymour, & Scheines 2000] •K2 [Cooper and Herskowitz 1992] • PC [Spirtes, Glymour, & Scheines 2000] •GBPS [Spirtes and Meek 1995] •GES [Chickering and Meek 2002] • TPDA [Cheng et al., 1997] •Sparse Candidate [Friedman et al. 1999] •CPC [Ramsey et al, 2006] •Optimal Reinsertion [Moore and Wong 2003] Hybrid •Rec [Xie, X, Geng, Zhi, JMLR 2008] •Exact Algorithms [Koivisto et al., 2004] , •MMHC [Tsamardinos et al. 2006] [Koivisto, 2006] , [Silander & Myllymaki, 2006] • CB [Provan et al. 1995] • BENEDICT [Provan and de Campos 2001] •ECOS [Kaname et al. 2010] Many more!!!
  • 37. Assumptions* • Tests of Conditional Independences / Scoring methods may not be appropriate for the type of data at hand • Faithfulness • No feedback cycles • No determinism • No latent variables • No measurement error • No averaging effects *aka why the algorithms may not work
  • 38. Faithfulness • ALL conditional (in)dependencies stem from the CMC Time in +0.6 +0.5 the sun Sunscreen Skin Cancer -0.3 Time in •Markov Condition does not imply: the sun •Ind(Skin Cancer, Sunscreen) •Unfaithful if: Skin Cancer •Ind(Skin Cancer, Sunscreen) Sunscreen
  • 39. Collinearity and Determinism X Y W Z X Z W Y • Assume Y and Z are information equivalent (e.g., one-to- one deterministic relation) • Cannot distinguish the two graphs • A specific type of violation of Faithfulness
  • 40. No Feedback Cycles Studying Good Grades • Studying causes Good Grades causes more studying (at a later time!)… • Hard to define without explicitly representing time • If all relations are linear, we can assume we sample from the distribution of the equilibrium of the system when external factors are kept constant – Path-diagrams (Structural Equation Models with no measurement model part) allow such feedback loops • If there is feedback and relations are not linear, there may be chaos, literally (mathematically) and metaphorically
  • 41. No Latent Confounders Heat (Not recorded) • Dep (Ice Cream, Polio) • No CAUSAL Bayesian Network on the Ice cream Polio modeled variables ONLY captures causal Ice cream Polio relations correctly Ice cream • Both Bayesian Polio Networks capture associations correctly (not always the case)
  • 42. Effects of Measurement Error X Y W True Model X Y W Possible Induced Y X W Model • X, Y, Z the actual physical quantities • X , Y , Z the measured quantities (+ noise) • If Y is measured with more error than X then Dep(X ;W | Y )
  • 43. Effects of Averaging • Almost all omics technologies measure average quantities over millions of cells • The quantities in the models though refer to single-cells X Y W True Model Possibly Induced Xi Yi Wi Model
  • 44. Success Stories • Identify genes that cause a phenotype. – [Schadt et al., Nature Genetics, 2005] • Reconstruct causal pathways. – [K. Sachs, et al. Science , (2005)] • Identify causal effects. – [Maathius et al., Nature Methods, 2010] • Predict association among variables never measured together. – [Tsamardinos et al., JMLR, 2012] • Select features that are most predictive of a target variable. – [Aliferis et. al., JMLR, 2010 ]
  • 45. An integrative genomics approach to infer causal associations between gene expression and disease L –Locus of DNA variation R – gene expression C- Phenotype (Omental Fat Pad Mass trait) Biological knowledge: Nothing causally affects L L R C Causal model L R C Reactive model L R C Independent model
  • 46. An integrative genomics approach to infer causal associations between gene expression and disease 1. Identify loci susceptible for causing the disease • 4 QTLs 2. Identify gene expression traits correlated with the disease • 440 genes 3. Identify genes with eQTLs that coincide with the QTLs • 113 genes, 267 eQTLs 4. Identify genes that support causal models 5. Rank genes by causal effect One of them ranked 152 out of the 440 based on mere correlation
  • 47. An integrative genomics approach to infer causal associations between gene expression and disease 1. Identify loci susceptible for causing the disease • 4 QTLs 2. Identify gene expression traits correlated with the disease • 440 genes 3. Identify genes with eQTLs that coincide with the QTLs • 113 genes, 267 eQTLs 4. Identify genes that support causal models 5. Rank genes by causal effect 4 of the top ranked genes where experimentally validated as causal
  • 48. Causal Protein-Signaling Networks Derived from Multi-parameter Single-Cell Data CD3 CD28 LFA-1 L PI3K RAS Cytohesin JAB-1 Zap70 A VAV SLP-76 LckT PKC PLC PIP3 Akt PIP2 PKA Raf MAPKKK MAPKKK Mek1/2 • Protein Signaling Pathways Erk1/2 resemble Causal Bayesian Networks MEK4/7 MEK3/6 • Use Causal Bayesian Networks learning JNK p38 to reconstruct a Protein Signaling pathway [K. Sachs, et al. Science , (2005)]
  • 49. Reconstructed vs. Actual Network Phospho-Proteins Phospho-Lipids PKC Perturbed in data Expected Pathway PKA Reported Raf Plc Reversed Jnk P38 Missed Mek PIP3 Erk  15/17 Classic PIP2 Akt  17/17 Reported  3 Missed
  • 50. Predicting causal effects in large-scale systems from observational data What will happen if you knock down Gene X? Gene A Gene B … Gene X 1 0.1 0.5 1.2 Intervention calculus when DAG is Absent 2 0.56 2.32 0.7 … 1. For every DAG G faithful to the data n 7 0.4 2.4 c 2. Causal effect G of X on V in DAG G is 1. 0, if V PaX (G) 2. coefficient of X in V∼X + PaX (G), otherwise 3. Causal effect c of X on V is the minimum of all cG
  • 51. Predicting causal effects in large-scale systems from observational data IDA Evaluation Experimental Observational Data* Data** Rank causal effects Apply IDA. Take top m percent Take top q genes Rosetta Compendium data: Compare • 5,361 genes • 234 single-gene deletion mutants* • 63 wild-type measurements**
  • 52. Predicting causal effects in large-scale systems from observational data • m=10%
  • 53. Integrative Causal Analysis • Make inferences from multiple heterogeneous datasets – That measure quantities under different experimental conditions – Measure different (overlapping) sets of quantities – In the context of prior knowledge • General Idea: – Find all CAUSAL models that simultaneously fit all datasets and are consistent with prior knowledge – Reason with the set of all such models
  • 54. Reason with Set of Solutions X Y Z W Y-Z ? X Y Z W Variables X Y Z W Y X W Z X Y Z W samples ρXW.Y = 0 X Y Z W |ρΧΥ | > | ρΧZ | X Y Z W ρ XW.Z= 0 X Y Z W
  • 55. Make Predictions X Y Z W Y-Z X Y Z W Variables X Y Z W Y X W Z X Y Z W samples ρXW.Y = 0 X Y Z W |ρΧΥ | > | ρΧZ | X Y Z W ρ XW.Z= 0 X Y Z W Nothing causes X
  • 56. Incorporating Prior Knowledge X Y Z W Y-Z X Y Z W Variables X Y Z W Y X W Z X Y Z W samples ρXW.Y = 0 X Y Z W |ρΧΥ | > | ρΧZ | X Y Z W ρ XW.Z= 0 X Y Z W Nothing causes X
  • 57. Further Inductions X Y Z W X Y Z W X Y Z W Variables X Y Z W Y X W Z samples ρXW.Y = 0 X Y Z W |ρΧΥ | > | ρΧZ | X Y Z W ρ XW.Z= 0 X Y Z W Nothing causes X Changing Y will have an effect on Z
  • 58. Proof-of-concept Results I Tsamardinos, S Triantafillou and V Lagani, Towards Integrative Causal Analysis of Heterogeneous Datasets and Studies, Journal of Machine Learning Research, to appear 20 datasets Actual correlation Biological, Financial, Text, Medical, Social 698897 predictions 98% accuracy vs. 16% for random guessing Predicted correlation 0.79 R2 between predicted and sample correlation
  • 59. Causality and Feature Selection • Question: Find a minimal set of molecular quantities that collectively carries all the information for optimal prediction / diagnosis (target variable) (Molecular Signature) – Minimal: throw away irrelevant or superfluous features – Collectively: May need to consider interactions – Optimal: Requires constructing a classification / regression model and estimating its performance • Answer*:It is the direct causes, the direct effects, and the direct causes of the direct effects of the target variable in the BN (called the Markov Blanket in this context) * Adopting all causal assumptions
  • 60. Markov Blanket CD3 CD28 LFA-1 L PI3K RAS Cytohesin JAB-1 Zap70 A VAV T SLP-76 Lck PKC PLC PIP3 Akt PIP2 PKA Raf Quantity of Interest MAPKKK MAPKKK Mek1/2 MEK4/7 MEK3/6 Erk1/2 JNK p38
  • 61. Markov Blanket CD3 CD28 LFA-1 L PI3K RAS Cytohesin JAB-1 Zap70 A VAV T SLP-76 Lck PKC PLC PIP3 Akt PIP2 PKA Raf Quantity of Interest MAPKKK MAPKKK Mek1/2 Signature / MEK4/7 MEK3/6 Erk1/2 Markov JNK p38 Blanket
  • 62. Markov Blanket Algorithms • Efficient and accurate algorithms applicable to datasets with hundreds of thousands of variables – Max-Min Markov Blanket, [Tsamardinos, Aliferis, Statnikov, KDD 2003] – HITON [Aliferis, Tsamardinos, Statnikov, AMIA 2003] – [Aliferis, Statnikov, Tsamardinos, et. al. JMLR 2010] • State-of-the-art in variable selection
  • 63. Objective •Identifying a set of transcripts able to predict IKAROS gene expression. •The selected set should be: – Maximally informative: able to predict IKAROS expression with optimal accuracy – Minimal: containing no redundant or uninformative transcripts
  • 64. Data • Genome-wide transciptome data from HapMap individual of European descent [Montgomery et al., 2010] – Lymphoblast cells – 60 distinct individuals – Approximately. ~140K transcripts • RKPM values freely available from ArrayExpress – www.ebi.ac.uk/arrayexpress/experiments/E-MTAB-197
  • 65. Methods • Constraint-based, local learning feature selection method for identifying multiple signatures – [Tsamardinos, Lagani and Pappas, 2012] • Support Vector Machine (SVM) for providing testable predictions – [Chang and Lin, 2011] • Nested cross validation procedure for: – setting algorithms’ parameters – providing unbiased performance estimations – [Statnikov, Aliferis, Tsamardinos, et al., 2005]
  • 66. Results • 22 different signatures found to be equally maximally predictive – Mean Absolute Error: 1.93 – R2: 0.7159 – Correlation of predictions and true expressions: – 0.8461 (p-value < 0.0001) • Example signature: – ENST00000246549, ENST00000545189, ENST00000265495, ENST00000398483, ENST00000496570 • Corresponding to genes: – FFAR2, ZNF426, ELF2, MRPL48, DNMT3A
  • 67. Predicted vs. Observed IKZF1 values
  • 68. Beyond This Tutorial Textbooks: • Pearl, J. Causality: models, reasoning and inference (Cambridge University Press: 2000). • Spirtes, P., Glymour, C. & Scheines, R. Causation, Prediction, and Search. (The MIT Press: 2001). • Neapolitan, R. Learning Bayesian Networks. (Prentice Hall: 2003).
  • 69. Beyond This Tutorial Different principles for discovering causality • Shimizu, S., Hoyer, P.O., Hyvärinen, A. & Kerminen, A. A Linear Non-Gaussian Acyclic Model for Causal Discovery. Journal of Machine Learning Research 7, 2003-2030 (2006). Hoyer, P., Janzing, D., Mooij, J., Peters, J. & Schölkopf, B. Nonlinear causal discovery with additive noise models. Neural Information Processing Systems (NIPS) 21, 689-696 (2009). Causality with Feedback cycles • Hyttinen, A., Eberhardt, F., Hoyer, P.O., Learning Linear Cyclic Causal Models with Latent Variables. Journal of Machine Learning Research, 13(Nov):3387-3439, 2012. Causality with Latent Variables • Richardson, T. & Spirtes, P. Ancestral Graph Markov Models. The Annals of Statistics 30, 962-1030 (2002). • Leray, P., Meganck, S., Maes, S. & Manderick, B. Causal graphical models with latent variables: Learning and inference. Innovations in Bayesian Networks 156, 219-249 (2008).
  • 70. Conclusions • Causal Discovery is possible from observational data or by limited experiments • Beware of violations assumptions and equivalences • Causality provides a formal language for conceptualizing data analysis problems • Necessary to predict the effect of interventions • Deep connections to Feature Selection • Allows integrative analysis in novel ways • Advanced theory and algorithms exist for different sets of (less restrictive) assumptions • Way to go still, particularly in disseminating to non-experts
  • 71. References • S. B. Montgomery et al. Transcriptome genetics using second generation sequencing in a Caucasian population. Nature 464, 773-777 (1 April 2010) • I. Tsamardinos, V. Lagani and D. Pappas. Discovering multiple, equivalent biomarker signatures. In Proceeding of the 7th conference of the Hellenic Society for Computational Biology & Bioinformatics (HSCBB) 2012 • C. Chang and C. Lin. LIBSVM: a library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2:27:1--27:27, 2011 • A. Statnikov, C.F. Aliferis, I. Tsamardinos, D. Hardin, and S.Levy. A comprehensive evaluation of multicategory classification methods for microarray gene expression cancer diagnosis. Bioinformatics, 21:631-643, 2005.