![Motivation: Cancer Immunology
meets Machine Learning
Q: Which mutated protein fragments will cause a
targeted immune response against tumor cells?
• Protein fragments must be presented on the cell surface to
be recognized by the immune system.
• Immunologists have collected ~300k examples of protein
fragment “affinity” for the proteins that present them to
the immune system.
• Build a model from protein fragment sequence to binding
affinity! Amino acid sequence -> [0,1]](https://image.slidesharecdn.com/pydata2015-recurrentneuralnetworks-151111222043-lva1-app6891/85/Python-libraries-for-Deep-Learning-with-Sequences-5-320.jpg)
Python libraries for Deep Learning with Sequences
- 1. Python Libraries for Deep Learning with Sequences Alex Rubinsteyn PyData NYC 2015
- 2. HammerLab @ Mount Sinai http://www.hammerlab.org/research/
- 3. Not biologists! Most lab members have a background in Computer Science and/or Mathematics: • Distributed data management • Machine learning & statistics • Programming languages • Data visualization
- 4. Motivation: Cancer Immunology meets Machine Learning T-cells recognize specific amino acid sequences presented on the surface of tumor cells.
- 5. Motivation: Cancer Immunology meets Machine Learning Q: Which mutated protein fragments will cause a targeted immune response against tumor cells? • Protein fragments must be presented on the cell surface to be recognized by the immune system. • Immunologists have collected ~300k examples of protein fragment “affinity” for the proteins that present them to the immune system. • Build a model from protein fragment sequence to binding affinity! Amino acid sequence -> [0,1]
- 6. Learning with fixed length sequences is easy! • 1-of-k encoding turns sequence of n characters into k*n length binary vector • Vector embedding (e.g. word2vec) can learn better representations from the data directly.
- 7. Varying length sequences are hard! • (Common approach) bag of words / n-grams: Doesn’t work when order actually matters! • “Artisanal” features & sliding windows: how to combine varying numbers of windows? • Structured prediction (e.g. CRFs): work for specific tasks BUT non-compositional (hard to combine with NNs) & inference is often slow
- 8. Simple RNNs • Neural network with two kinds of inputs: input from current time step (Xt) & previous output (ht-1 ) • Train using “Back-propagation through time” (BPTT) • Works for short time-scale dependencies between inputs and outputs Source: http://colah.github.io/posts/2015-08-Understanding-LSTMs/
- 9. Simple RNNs • …but what happens across many time steps? • Hard to track relationship between inputs & outputs • “Vanishing” & “Exploding” gradients Source: http://colah.github.io/posts/2015-08-Understanding-LSTMs/
- 10. LSTM • Holds internal state (ct) across timesteps • Gated update of state based on input (xt), previous output (ht-1), and previous state (ct-1) Generating Sequences With Recurrent Neural Networks (Graves 2012)
- 11. LSTM Equations Generating Sequences With Recurrent Neural Networks (Graves 2012)
- 13. Source Data • Grimm’s Fairy Tales from Project Gutenberg • ~500k characters • 69 distinct characters “A certain king had a beautiful garden, and in the garden stood a tree which bore golden apples. These apples were always counted, and about the time when they began to grow ripe it was found that every night one of them was gone. The king became very angry at this, and ordered the gardener to keep watch all night under the tree.”
- 14. Character-Level Language Modeling • Task: Predict the next character from the previous k characters. • Input: Sequence of binary vectors representing characters. • Output: Probability distribution over characters. • Loss Function: Probabilities of all wrong characters.
- 18. Output, epoch #1 “'I the kas would ontile were her werl now I heast of the sonenund lest enct her ouk that pistered the with of fean, wile that the fing wared me in the parled to the bees if the sther gound.' Then whan shund again, the seeps of the wame went on the coot; be he as deated sime out of the the, and boked yither”
- 19. Output, epoch #2 “The bettle seet resent throw in his sell and seard ney woor and see betore.' 'qoat rawed, with one lyor wand her, he lettle she sauded out of one to the shore sanded off the would be wonderth put her that she once sen which neved becound, and the toot and saw a loodser, and said her one will be soon arl dead.' But I will go and kered to the bear reppined.'”
- 20. Output, epoch #10 “So the old woman was clacked upon his head, and mise could not run off and berought to courterus, for the fisherman was that the young grey back, and the cat was before to the church, and the grandmasters were off back.”
- 21. Output, epoch #15 “The king came and said, 'Give me a moon.' 'Ah,' said the wolf, 'the king was so well to go out to him her way in the sea, for a wood would have nothing to wait all yourself. 'That was bring the church of the chamber,' said the old woman, 'and take some his horse.'”
- 22. Output, epoch #40 “The bear was standing by the side of the stream, and said to him, 'What a way into the forest the golden house if you have been the world when they shall have sent me forth, and I will soon find you down in an earth.' The man was going on the house, and sent the fire, and she said, 'If you have a fat good companion for the princess.'”
- 23. Other RNN Tasks
- 24. Machine Translation Source: http://cs224d.stanford.edu/lectures/CS224d-Lecture8.pdf
- 26. Computational Biology • Predicting mRNA splicing from DNA sequence • Predicting at which residues a protein will be cut by the proteasome (or other proteases) • Predicting binding of long peptides to Class II MHC molecules for immune recognition by Helper T-cells.
- 27. Thanks!