Dimensionality reduction and visualization techniques for high-dimensional genomic data - Data Science Conference DSC3.0 talk - Dusan Randjelovic

Editor's Notes

• #2: Good morning! Thank you all for coming to this talk. My name is Dusan and for the past 2 years I have been occasionaly playing with some interesting genomics datasets. I work as Bioinformatics Analyst, in a company called “Seven Bridges Genomics”, so by now you are already guessing that this talk will be more on the science side of data science. I will however focus on some particular methods for analysis of biological data - which are quite interesting and widely applicable in general data science - primarily methods of dimensionality reduction. I hope you’ll find this overview helpful and if you come to like computational biology and genomics after this talk it’s just a bonus :).
• #3: First - I’ll talk about some specifics of genomics and introduce just enough cell biology needed so that you could understand the use-cases. And then I will talk about dimensionality reduction and show-case some high-dimensional datasets in genomics.
• #5: There is a saying that data scientist is “Person who is better at programming than any… ”, but in case of genomics data science I would like to argue that those two disciplines are not enough. Genomics data scientist depends heavily on the domain knowledge.
• #6: In order to understand the results of analysis, to pose the right questions or even to recognise features in a dataset you need to have some basic knowledge about the processes in a cell and in data-generation. So the domain is actually crucial here. Apaet from this - there is usually not only one approach to each study, but several experiments on different levels (DNA, RNA level, some biomedical measurements, phenotype quantification, etc.) - and this so called multi-omics approach, that is so powerful for clinicians, gives headaches to genomics data scientists. Another difficulty here is that population scale studies and per-sample studies deal with equally unknown phenomena - there is so many associations and correlation but discipline is so young and there are not many theoretical models to help guide new analysis.
• #7: As I said we will need some basic cell biology here, but I’ll try to be brief :) - you probably remember most of this from high-school anyway, right? Cells of complex organisms have a nucleus in which there are some long molecules called DNA, packed in chromosoms. DNA molecules serve as a blueprint for making other molecules that are involved in every function of a cell, like RNA molecules or indirectly proteins. Whole DNA material is called a genome and some small regions of it are famous genes. What is important here is that DNA is structured and is the same in every cell. It is composed of billions of smaller molecules adenine, thymine, cytosin and guanine and could be represented as long sequence of letters ACTG -> unique for each individual.
• #8: Overall, complex interactions of DNA, RNA, proteins and environment makes what we call a phenotype -> some physical charateristics, like eye color or a disease.
• #9: If we would be somehow able digitize these molecules, we would get a picture of processes in a cell that cause diseases or are responsible for some phenotype. That digitization is possible and is called sequencing. From a data science perspective: when you sequence a genome, the end-result is a dataset that says on which positions among 3 billion letters of your genome there is a variation or mutation - something different than some reference genome. You could see these millions of differences as features of your dataset to explore! Another common digitization is to count RNA molecules transcribed from genes, which usually gives you datasets with 10s of thousands features.
• #10: Sequencing is what gave birth to genomics, the study of the whole genome, and since 2003 this technology breaks Moore’s law and is currently one of the greatest sources of big data.
• #12: In order to profile complex biological processes we measure as much as we can. It’s fortunate that we could measure all that features of some process at once - but with complexity and lots of features comes a Curse.
• #13: And it is called the Curse of dimensionality. What is meant by this are some geometric and probabilistic consequences of dealing with high domensional feature space. For example: if we have some number of samples and we measure 1, 2, or 3 features of those same samples, what we notice is that by taking more features into account we are sampling from increasing feature-space - and our samples are less and less representative of that feature-space. Main definition of curse of dimensionality is that we need exponentially more samples if we are increasing number of features.
• #14: On the other hand - even if we have enough samples - there are some geometrical consequences of going higher in dimensionality. In machine learning, wheter unsupervised or supervised - we are usually interested in finding distances between datapoints, in order to establish some similarity metric. And distances and neighbourhoods in high-dimensional feature-space are problematic. If you look at the right image you’ll see that greatest circle inside a square covers 78% of it, which could also be seen as greatest neighbourhood inside 2dim feature-space. If we increase the dimensionality, we see that a sphere covers 52% of a cube, and going further we have only 0.24% of 10dim hypersphere covering 10dim hypercube. This really changes the meaning of near and far - since most data points are far away in the corners.
• #15: So if we go to 100 or 1000 dimensions we have sparcity introduced merely by geometry of such a space. Locality is also broken, and # of samples needed grows rapidly. Additionaly, most algorithms have some optimal number of features to work with, as seen on the classifier performance curve from the right.
• #16: So what do we do to avoid the Curse? - We try to reduce the dimensions.
• #17: We could and should always try to reduce dimensionality of a dataset if we suspect that intrinsic dimensionality - the one that completely describes the effect that we are measuring - is lower than number of features. For example, on the image here you see small images of letter A and since images are usually described by values inside a pixels, we could imagine this dataset being of for example 64x64 features. But if we are interested in transformations of A present in these pictures we could see that only 2 transformations are applied: scaling and rotation - so instead of 64x64 features we could have only 2 to represent the whole dataset. Reduction of dimensionality is done when we are doing feature selection or filtering of only interesting features or feature engineering - construction of new, better features from the ones we measure. It is sometimes imperative to reduce dimensions merely because of computational complexity or to compress the dataset. But, most interesting purpose of dimensionality reduction is to do visualization and exploratory analysis.
• #18: Dimensionality reduction techniques are unsupervised machine learning techniques to learn the embedding of high-dimensional dataset in lower dimensions, usually 2 or 3 if we aim to explore and visualize the dataset. The last note from this more formal explanation from the slide is about keeping geometry intact as much as possible - that part is the hardest since datasets could have some weird topological or metrical properties to it.
• #19: Number of dimensionality reduction methods and techniques is rapidly rising, especially of non-linear ones. Methods could be divided by nature of realtionships among features to linear and non-linear. We have division on convex and non-convex methods if objective function that is optimized is convex or not. Important distinction is between global and local methods (ones that preserve global dissimilarities in a dataset and ones that keep local similarity better). No matter which method, similar to clustering or classification, what we need among datapoints is some established similarity measure.
• #20: When we speak about similarity two other terms usually come in: neighborhood and distance. Neighborhood of a datapoint could be defined by all the points that fall under some radius (left image on the slide) or specified number of points in decreasing order. But to measure distances in order to calculate neighborhoods or in order to set some similarity measure we have many choices. Euclidean distance in higher dimensions is most common but in some case some others are used, like geodesic distance or even some non-metric distances as we will see in examples.
• #21: Most non-linear techniques could be seen as finding lower dimensional manifold on which datapoints lay. Similarity between datapoints is than measured along this manifold. Manifold learning is sometimes synonim for dimensionality reduction.
• #22: As you saw there are different criteria to divide techniques and only one of taxonomies is given here on the slide. Going further I’ll talk into more details about PCA, ISOMap and tSNE.
• #23: And as promised I’ll show-case these methods on some real genomics datasets.
• #24: First one is Principal component analysis technique done on Simons Diversity dataset, as part of finding out about stratification of human population by genomic variants or mutations.
• #25: Simons Diversity dataset contains 300 genomes from 142 populations. It’s 35TB of raw sequencing data and processed data. On Seven Bridges, we host this complete dataset. We have also done reproduction of published studies.
• #26: We know that different human populations have common phenotypes, which should also be found on genome level. If we take all mutations/variations from simons dataset and reduce them to 2 dimensions based on how variable are they in samples - we should get similar samples clustered together. We use global disimilarity between datapoints here. What we actually did, we took only non-African samples, and only one type of mutations, and all of that only on one chromosome, and we still got some nice separation in 2 dimensions! On the plot you could see different colors for different populations but that is only for plotting, this was done unsupervised by PCA. This was done on SBG platform with R/Bioconductor tool SNPrelate
• #27: PCA is a linear technique that finds directions along which variance of the data is maximized (so called eigenvectors). Eigenvectors are basis of M matrix here on slide and are all mutually ortogonal/normal and could be found as solutions to second eqn. By decomposing initial data matrix X this way we get ordered principal components: independent features in decreasing order of contribution to overall variance. How many principal components we take for further analysis could be determined by second plot here: proportinal variance explained.
• #28: Second case is a bit more complex - since it deals with highly non-linear data. Goal is to infer cell populations from single cell RNA-seq data.
• #29: In single cell RNA-seq experiment we are looking at RNA molecules and counting them in each cell. What you get as a result after processing raw sequencing data is matrix like in lower right corner: cells as columns and different molecules that correspond to genes as rows. If you remember me saying: cells have same DNA but depending on cell type and function different cells have different RNAs, and we should be able to cluster cells by their expression profiles. Usually clustering is easier if done in reduced space.
• #30: Multiple-hypothesis testing -> if you torture your data enough, it will confess
• #31: I have used this scikit-learn based framework to look at different projections of one particular single cell study. What was challenging here was to confirm cell types after reduction and clustering and for that I have looked at how well some cell’s expression profile corelated with some known molecular pathway (blue to red scale right).
• #32: One non-linear technique capable of dealing with this kind of data is ISOmap, which uses geodesic distances along the manifold to model similarity between points. Plastic example of how this distance is more useful could be seen on artificial dataset on the left image: x and y are near in euclidean sense, but far apart if you measure along the manifold.
• #33: And last one is tSNE used to visualize tissue-specific expression profiles.
• #34: Dataset is somewhat similar to single cell RNA-seq one - but in this case we have RNAs from different tissues of 100s of people. We expect to see tissues separated in some lower dim space.
• #35: Original study used PCA and k-means in low dim. And hierarchical clustering in high dimensional space, but when tSNE reanalysis is done it showed better separation of tissues in tSNE space.
• #36: tSNE is non-convex method which means every time it will give slightly different results. Similarity between data points is not even a metric, it is conditional probability that some nearby point is neighbouring. In lower space we try to keep those probabilities but with t-distr. instead of normal. tSNE preserves local similarity between datapoints and is very effective and popular technique, but difficult to interpret.