Data preprocessing and unsupervised learning methods in Bioinformatics
- 1. Data Preprocessing Unsupervised learning Elena Sügis elena.sugis@ut.ee Introduction to Bioinformatics, LVSC2016
- 2. ?? ?
- 6. Experiments
- 7. Data comes in different forms
- 9. Simple data analysis pipeline Data Black Magic Result high quality data machine learning method awesome result
- 10. Simple data analysis pipeline Data Black Magic Result poor quality data machine learning method not so awesome result
- 13. 80 %
- 15. Interpretation Summarize/ plot raw data Impute missing values Normalize/ Standardize Handle outliers Data analysis Import datavalidation
- 16. Meet your data
- 17. Interpretation Summarize/ plot raw data Impute missing values Normalize/ Standardize Handle outliers Data analysis Import datavalidation
- 18. Missing Values Origins: • Malfunctioning measurement equipment • Very low intensity signal • Deleted due to inconsistency with other recorded data • Data removed/not entered by mistake
- 19. Missing Values How to deal with them: • Filter out • Replace missing values by 0 • Replace by the mean, median value • K nearest neighbor imputation (KNN imputation) • Expectation—Maximization (EM) based imputations
- 21. KNN • We are given a gene expression matrix M • Let X=(x1, x2, …, xi, …, xn) be a vector in the matrix M with a missing value at xi at the dimension i • Find in the gene expression data matrix matrix vectors X1 , X2 , …, Xk , such that they are the k closest vectors to X in M (with a chosen distance measure) among the vectors that do not have a missing value at dimension i • Replace the missing value xi with the mean (or median) of X1 i, X2 i, …, Xk i , i.e., mean (median) of the values at dimension i of vectors X1 , X2 , …, Xk
- 22. KNN Healthy people Patients Gene expression matrix
- 23. Imputed missing values Healthy people Patients Gene expression matrix
- 24. Interpretation Summarize/ plot raw data Impute missing values Normalize/ Standardize Handle outliers Data analysis Import datavalidation
- 26. Normalization & Standardization Objective: adjust measurements so that they can be appropriately compared among samples Key ideas: • Remove technological biases • Make samples comparable Methods: • Z-scores (centering and scaling) • Logarithmization • Quantile normalization • Linear model based normalization
- 27. Z-scores Centering a variable is subtracting the mean of the variable from each data point so that the new variable's mean is 0. Scaling a variable is multiplying each data point by a constant in order to alter the range of the data. where: µ is the mean of the population. σ is the standard deviation of the population. z = x −µ σ
- 28. transforms the data by a linear projection onto a lower-dimensional space that preserves as much data variation as possible Principal Component Analysis
- 29. Principal Component Analysis Objective: Reduce dimensionality while preserving as much variance as possible http://setosa.io/ev/principal-component-analysis/
- 30. Visualize normalized data Groups Healthy Patients group1 Patients group2
- 31. Visual inspection after normalization
- 32. Visual Inspection. PCA Highlight groups Patients Healthy people
- 33. Arrrgh!!! Why aren’t you together ?!?!
- 34. Visual Inspection. PCA Color by experiment/dataset/day DAY1 DAY2
- 35. Batch Effects Measurements are affected by: • Laboratory conditions • Reagent lots • Personnel differences are technical sources of variation that have been added to the samples during handling. They are unrelated to the biological or scientific variables in a study. Major problem : might be correlated with an outcome of interest and lead to incorrect conclusions
- 36. Fighting The Batch Effects Experimental design solutions: • Shorter experiment time • Equally distributed samples between multiple laboratories and across different processing times, etc. • Provide info about changes in personnel, reagents, storage and laboratories Statistical solutions: • ComBat • SVA(Surrogate variable analysis, SVD+linear models) • PAMR (Mean-centering) • DWD (Distance-weighted discrimination based on SVM) • Ratio_G (Geometric ratio-based) J.T. Leek, Nature Reviews Genetics 11, 733-739 (October 2010,) Chao Chen, PlosOne, 2011
- 38. Interpretation Summarize/ plot raw data Impute missing values Normalize/ Standardize Handle outliers Data analysis Import datavalidation
- 42. Interpretation Summarize/ plot raw data Impute missing values Normalize/ Standardize Handle outliers Data analysis Import datavalidation
- 43. IF YOU TORTURE THE DATA LONG ENOUGH IT WILL CONFESS TO ANYTHING Ronald Coase, Economist, Nobel Prize winner
- 44. Clustering is finding groups of objects such that: similar (or related) to the objects in the same group and different from (or unrelated) to the objects in other groups What is cluster analysis?
- 45. Properties • Classes/labels for each instance are derived only from the data • For that reason, cluster analysis is referred to as unsupervised classification
- 46. • Intuition building Finding hidden internal structure of the high-dimensional data • Hypothesis generation Finding and characterizing similar groups of objects in the data • Knowledge discovery in data Ex. Underlying rules, reoccurring patterns, topics, etc. • Summarizing / compressing large data • Data visualization Why to cluster biological data?
- 47. Intuition building cardiopulmonary/ metabolic disorders neurological diseases sensory conditions cerebral vascular accident cancer http://bmcgeriatr.biomedcentral.com/articles/10.1186/1471-2318-11-45 presence of one or more additional diseases or disorders co- occurring with a primary disease or disorder
- 48. • Intuition building Finding hidden internal structure of the high-dimensional data • Hypothesis generation Finding and characterizing similar groups of objects in the data • Knowledge discovery in data Ex. Underlying rules, reoccurring patterns, topics, etc. • Summarizing / compressing large data • Data visualization Why to cluster biological data?
- 50. • Intuition building Finding hidden internal structure of the high-dimensional data • Hypothesis generation Finding and characterizing similar groups of objects in the data • Knowledge discovery in data Ex. Underlying rules, reoccurring patterns, topics, etc. • Summarizing / compressing large data • Data visualization Why to cluster biological data?
- 51. Knowledge discovery in data Ex. Underlying rules, reoccurring patterns, topics, etc.
- 52. • Intuition building Finding hidden internal structure of the high-dimensional data • Hypothesis generation Finding and characterizing similar groups of objects in the data • Knowledge discovery in data Ex. Underlying rules, reoccurring patterns, topics, etc. • Summarizing / compressing large data • Data visualization Why to cluster biological data?
- 54. Partitional vs Hierarchical Creates a nested and hierarchical set of partitions/clusters Each sample(point) is assigned to a unique cluster
- 55. Fuzzy vs Non-Fuzzy Fuzzy vs Non-Fuzzy Each object belongs to each cluster with some weight (the weight can be zero) Each object belongs to exactly one cluster Each object belongs to each cluster with some weight Each object belongs to exactly one cluster
- 56. Hierarchical clusteringHierarchical clustering Hierarchical clustering is usually depicted as a dendrogram (tree) Hierarchical clustering is usually depicted as a dendrogram (tree)
- 57. • Each subtree corresponds to a cluster • Height of branching shows distance Hierarchical clustering • Each subtree corresponds to a cluste • Height of branching shows distance Hierarchical clustering
- 58. Hierarchical clustering (0) Algorithm for Agglomerative Hierarchical Clustering: Join the two closest objects Algorithm for Agglomerative Hierarchical Clustering: Join the two closest objects Hierarchical clustering
- 59. Join the two closest objects Hierarchical clustering (1) Join the two closest objects Hierarchical clustering
- 60. Keep joining the closest pairs Hierarchical clustering (2) Keep joining the closest pairs Hierarchical clustering
- 61. Hierarchical clustering (3) Keep joining the closest pairs Keep joining the closest pairs Hierarchical clustering
- 62. Hierarchical clustering (4) Keep joining the closest pairs Keep joining the closest pairs Hierarchical clustering
- 63. Hierarchical clustering (5) Keep joining the closest pairs Keep joining the closest pairs Hierarchical clustering
- 64. Hierarchical clustering (10) After 10 steps we have 4 clusters left After 10 steps we have 4 clusters left Hierarchical clustering
- 65. Q: Which clusters do we merge next?Hierarchical clustering (10) After 10 steps we have 4 clusters left
- 66. Hierarchical clustering (10) Several ways to measure distance between clusters: • Single linkage (MIN) Several ways to measure distance between clusters: • Single linkage(MIN) Hierarchical clustering
- 67. Hierarchical clustering (10) Several ways to measure distance between clusters: • Single linkage (MIN) • Complete linkage (MAX) Several ways to measure distance between clusters: • Single linkage(MIN) • Complete linkage(MAX) Hierarchical clustering
- 68. Hierarchical clustering (10) Several ways to measure distance between clusters: • Single linkage (MIN) • Complete linkage (MAX) • Average linkage • Weighted • Unweighted • ... Several ways to measure distance between clusters: • Single linkage (MIN) • Complete linkage (MAX) • Average linkage • Weighted • Unweighted ... • Ward’s method Hierarchical clustering
- 69. Hierarchical clustering (11) In this example and at this stage we have the same result as in partitional clustering In this example and at this stage we have the same result as in partitional clustering Hierarchical clustering
- 70. Hierarchical clustering (12) In the final step the two remaining clusters are joined into a single cluster In the final step the two remaining clusters are joined into a single cluster Hierarchical clustering
- 71. Hierarchical clustering (13) In the final step the two remaining clusters are joined into a single cluster In the final step the two remaining clusters are joined into a single cluster Hierarchical clustering
- 72. Examples of Hierarchical Clustering in Bioinformatics Examples of Hierarchical Clustering in Bioinformatics PhylogenyGene expression clustering
- 73. K-means clustering • Partitional, non-fuzzy • Partitions the data into K clusters • K is given by the user Algorithm: • Choose K initial centers for the clusters • Assign each object to its closest center • Recalculate cluster centers • Repeat until converges
- 79. Elbow method Estimate the number of clusters
- 80. K-means clustering summary • One of the fastest clustering algorithms • Therefore very widely used • Sensitive to the choice of initial centres • many algorithms to choose initial centres cleverly • Assumes that the mean can be calculated • can be used on vector data • cannot be used on sequences (what is the mean of A and T?)
- 81. K-medoids clustering • The same as K-means, except that the center is required to be at an object • Medoid - an object which has minimal total distance to all other objects in its cluster • Can be used on more complex data, with any distance measure • Slower than K-means
- 91. Examples of K-means and K-medoids in Bioinformatics Gene expression clustering Sequence clustering Examples of K-means and K-medoids in Bioinformatics
- 92. Distance measuresDistance measures Distance of vectors and • Euclidean distance • Manhattan distance • Correlation distance Distance of sequences and • Hamming distance => 3 • Levenshtein distance x = (x1, . . . , xn) y = (y1, . . . , yn) d(x, y) = v u u t nX i=1 (xi yi) 2 d(x, y) = nX i=1 |xi yi| d(x, y) = 1 r(x, y) is Pearson correlation coefficient r(x, y) ACCTTG TACCTG ACCTTG TACCTG .ACCTTG TACC.TG => 2
- 94. Interpretation Summarize/ plot raw data Impute missing values Normalize/ Standardize Handle outliers Data analysis Import datavalidation
- 95. Put it into words & Discover https://mastrianascience.wikispaces.com/file/view/scientific_method_wordle.png/253466620/694x406/scientific_method_wordle.
- 96. Gene ontology • Molecular Function - elemental activity or task • Biological Process - broad objective or goal • Cellular Component - location or complex What found genes are doing
- 97. Functional annotations & Significance statistical significance of having drawn a sample consisting of a specific number of k successes out of n total draws from a population of size N containing K successes.
- 99. Practice time!