2014 11-12 sbsm032rstatsprogramming.key
0 likes381 views
This document contains information about various programming languages and data handling skills for bioinformatics. It includes sections on choosing a programming language, with pros and cons of languages like Excel, R, Unix command line, Java, Perl, Python and Ruby listed. Other sections cover practical skills like accessing and subsetting data in R, using regular expressions for text searches, creating functions, and using for loops. Examples of code are provided throughout.
![• creating a vector
• give me a vector containing numbers from 5 to 11 (3 variants)
> myvector <- 5:11
> myvector <- seq(from=5, to=11, by=1)
> myvector <- c(5, 6, 7, 8, 9, 10, 11)
> myvector
[1] 5 6 7 8 9 10 11
• accessing a subset
• of a vector
> bigvector <- 150:100
> bigvector
[1] 150 149 148 147 146 145 144 143 142 141 140 139 138 137 136 135 [20] 131 130 129 128 127 126 125 124 123 122 121 120 119 118 117 116 [39] 112 111 110 109 108 107 106 105 104 103 102 101 100
> mysubset <- bigvector[myvector]
> mysubset
[1] 146 145 144 143 142 141 140
!
> subset(bigvector, bigvector > 120)
[1] 150 149 148 147 146 145 144 143 142 141 140 139 138 137 136 135 [20] 131 130 129 128 127 126 125 124 123 122 121](https://image.slidesharecdn.com/2014-11-12sbsm032rstatsprogramming-141112081114-conversion-gate01/85/2014-11-12-sbsm032rstatsprogramming-key-33-320.jpg)
![Regular expressions (regex):
Text search on steroids.
Regular expression Finds
David David
Dav(e|(id)) David, Dave
Dav(e|(id)|(ide)|o) David, Dave, Davide, Davo
At{1,2}enborough Attenborough,
Atenborough
Atte[nm]borough Attenborough,
Attemborough
At{1,2}[ei][nm]bo{0,1}ro((ugh)|w){0,1}
Atimbro,
attenbrough,
ateinborow
Easy counting, replacing all with “Sir David Attenborough”](https://image.slidesharecdn.com/2014-11-12sbsm032rstatsprogramming-141112081114-conversion-gate01/85/2014-11-12-sbsm032rstatsprogramming-key-38-320.jpg)
![Regex Special symbols
Regular expression Finds Example
[aeiou] any single vowel “e”
[aeiou]* between 0 and infinity
vowels vowels, e.g.’ “eeooouuu"
[aeoiu]{1,3} between 1 and 3 vowels “oui”
!
a|i one of the 2 characters “"
((win)|(fail)) one of the two
words in () fail](https://image.slidesharecdn.com/2014-11-12sbsm032rstatsprogramming-141112081114-conversion-gate01/85/2014-11-12-sbsm032rstatsprogramming-key-39-320.jpg)
![More Regex Special symbols
Synonymous with
[:digit:] [0-9]
[A-z] [A-z], ie [A-Za-z]
s whitespace
. any single character
.+ one to many of anything
b* between 0 and infinity letter ‘b’
[^abc] any character other than a, b or c.
( (
[:punct:] any of these: ! " # $ % & ' ( ) * + , - . /
: ; < = > ? @ [ ] ^ _ ` { |
• Google “Regular expression cheat sheet”
• ?regexp](https://image.slidesharecdn.com/2014-11-12sbsm032rstatsprogramming-141112081114-conversion-gate01/85/2014-11-12-sbsm032rstatsprogramming-key-40-320.jpg)
![Regular expressions (regex):
matching “LMTSOMIKTIP” and “LMVSONYYIKTIP” but
Text not search “LMVSQMIKTIP”
on steroids.
Regular expression Finds
David David
Dav(e|(id)) David, Dave
Dav(e|(id)|(ide)|o) David, Dave, Davide, Davo
At{1,2}enborough Attenborough,
Atenborough
Atte[nm]borough Attenborough,
Attemborough
At{1,2}[ei][nm]bo{0,1}ro((ugh)|w){0,1}
Atimbro,
attenbrough,
ateinborow
Easy counting, replacing all with “Sir David Attenborough”](https://image.slidesharecdn.com/2014-11-12sbsm032rstatsprogramming-141112081114-conversion-gate01/85/2014-11-12-sbsm032rstatsprogramming-key-42-320.jpg)
![Functions
• R has many. e.g.: plot(), t.test()
• Making your own:
tree_age_estimate <- function(diameter, species) {
[...do the magic... maybe something like:
growth.rate <- growth.rates[ species ]
age.estimate <- diameter / growth.rate
...]
!
return(age.estimate)
}
> tree_age_estimate(25, “White Oak”)
+ 66
> tree_age_estimate(60, “Carya ovata”)
+ 190](https://image.slidesharecdn.com/2014-11-12sbsm032rstatsprogramming-141112081114-conversion-gate01/85/2014-11-12-sbsm032rstatsprogramming-key-45-320.jpg)
![“for”
Loop
> possible_colours <- c('blue', 'cyan', 'sky-blue', 'navy blue',
'steel blue', 'royal blue', 'slate blue', 'light blue', 'dark
blue', 'prussian blue', 'indigo', 'baby blue', 'electric blue')
!
> possible_colours
[1] "blue" "cyan" "sky-blue" "navy blue"
[5] "steel blue" "royal blue" "slate blue" "light blue"
[9] "dark blue" "prussian blue" "indigo" "baby blue"
[13] "electric blue"
!
> for (colour in possible_colours) {
+ print(paste("The sky is oh so, so", colour))
+ }
!
[1] "The sky is so, oh so blue"
[1] "The sky is so, oh so cyan"
[1] "The sky is so, oh so sky-blue"
[1] "The sky is so, oh so navy blue"
[1] "The sky is so, oh so steel blue"
[1] "The sky is so, oh so royal blue"
[1] "The sky is so, oh so slate blue"
[1] "The sky is so, oh so light blue"
[1] "The sky is so, oh so dark blue"
[1] "The sky is so, oh so prussian blue"
[1] "The sky is so, oh so indigo"
[1] "The sky is so, oh so baby blue"
[1] "The sky is so, oh so electric blue"](https://image.slidesharecdn.com/2014-11-12sbsm032rstatsprogramming-141112081114-conversion-gate01/85/2014-11-12-sbsm032rstatsprogramming-key-49-320.jpg)
2014 11-12 sbsm032rstatsprogramming.key
- 1. SBSM035 - Stats/ Bioinformatics/ Programming y.wurm@qmul.ac.uk http://yannick.poulet.org
- 2. © Alex Wild & others
- 4. Atta leaf-cutter ants © National Geographic
- 5. Atta leaf-cutter ants © National Geographic
- 6. Atta leaf-cutter ants © National Geographic
- 8. Oecophylla Weaver ants © ameisenforum.de
- 9. © ameisenforum.de Fourmis tisserandes
- 10. © ameisenforum.de Oecophylla Weaver ants
- 11. © wynnie@flickr © forestryimages.org
- 12. Tofilski et al 2008 Forelius pusillus
- 13. Forelius pusillus hides the nest entrance at night Tofilski et al 2008
- 14. Forelius pusillus hides the nest entrance at night Tofilski et al 2008
- 15. Forelius pusillus hides the nest entrance at night Tofilski et al 2008
- 16. Forelius pusillus hides the nest entrance at night Tofilski et al 2008
- 17. Forelius pusillus hides the nest entrance at night Avant Workers staying outside die « preventive self-sacrifice » Tofilski et al 2008
- 18. Dorylus driver ants: ants with no home © BBC
- 19. Animal biomass (Brazilian rainforest) Mammals Birds Reptiles Other insects Amphibians from Fittkau & Klinge 1973 ! Earthworms ! ! Spiders Soil fauna excluding earthworms, ants & termites Ants & termites
- 20. We use modern technologies to understand insect societies. • evolution of social behaviour • molecules involved in social behaviour • consequences of environmental change
- 22. Big data is invading biology
- 23. This changes 454 everything. Illumina Solid... Any lab can sequence anything!
- 24. Big data is invading biology • Genomics • Biodiversity assessments • Stool microbiome sequencing • Personalized medicine • Cancer genomics • Sensor networks - e.g tracking microclimates, recording sounds • Aerial surveys (Drones) - e.g. crop productivity; rainforest cover • Camera traps
- 27. Choosing a programming language Good: Bad: Excel quick & dirty easy to make mistakes doesn’t scale R numbers, stats, genomics programming Unix command-line == shell == bash Can’t escape it. Quick & Dirty. HPC. programming, complicated things Java 1990s user interfaces overcomplicated. Perl 1980s. Everything. Python scripting, text ugly Ruby scripting, text Javascript/Node scripting, flexibility(web & client), community only little bio-stuff
- 28. First steps towards data handling • Basic stats - done! • Programming in R • UNIX command-line bioinformaticians
- 30. Practicals • Aim: get relevant data handling skills • Doing things by hand: • impossible? • slow, • error-prone, • Automate! • Basic programming • in R • no stats!
- 31. Practicals: contents • Done: • data accessing/subsetting • New: • search/replace • regular expressions • New: • functions • loops Text search on steroids Reusable pieces of work Repeating the same thing many times
- 33. • creating a vector • give me a vector containing numbers from 5 to 11 (3 variants) > myvector <- 5:11 > myvector <- seq(from=5, to=11, by=1) > myvector <- c(5, 6, 7, 8, 9, 10, 11) > myvector [1] 5 6 7 8 9 10 11 • accessing a subset • of a vector > bigvector <- 150:100 > bigvector [1] 150 149 148 147 146 145 144 143 142 141 140 139 138 137 136 135 [20] 131 130 129 128 127 126 125 124 123 122 121 120 119 118 117 116 [39] 112 111 110 109 108 107 106 105 104 103 102 101 100 > mysubset <- bigvector[myvector] > mysubset [1] 146 145 144 143 142 141 140 ! > subset(bigvector, bigvector > 120) [1] 150 149 148 147 146 145 144 143 142 141 140 139 138 137 136 135 [20] 131 130 129 128 127 126 125 124 123 122 121
- 37. Regular expressions (regex): Text search on steroids.
- 38. Regular expressions (regex): Text search on steroids. Regular expression Finds David David Dav(e|(id)) David, Dave Dav(e|(id)|(ide)|o) David, Dave, Davide, Davo At{1,2}enborough Attenborough, Atenborough Atte[nm]borough Attenborough, Attemborough At{1,2}[ei][nm]bo{0,1}ro((ugh)|w){0,1} Atimbro, attenbrough, ateinborow Easy counting, replacing all with “Sir David Attenborough”
- 39. Regex Special symbols Regular expression Finds Example [aeiou] any single vowel “e” [aeiou]* between 0 and infinity vowels vowels, e.g.’ “eeooouuu" [aeoiu]{1,3} between 1 and 3 vowels “oui” ! a|i one of the 2 characters “" ((win)|(fail)) one of the two words in () fail
- 40. More Regex Special symbols Synonymous with [:digit:] [0-9] [A-z] [A-z], ie [A-Za-z] s whitespace . any single character .+ one to many of anything b* between 0 and infinity letter ‘b’ [^abc] any character other than a, b or c. ( ( [:punct:] any of these: ! " # $ % & ' ( ) * + , - . / : ; < = > ? @ [ ] ^ _ ` { | • Google “Regular expression cheat sheet” • ?regexp
- 41. Your turn Make a regular expression • matching “LMTSOMIKTIP” and “LMVSONYYIKTIP” but not “LMVSQMIKTIP” ! • matching all variants of “ok” (e.g., “O.K.”, “Okay”…)
- 42. Regular expressions (regex): matching “LMTSOMIKTIP” and “LMVSONYYIKTIP” but Text not search “LMVSQMIKTIP” on steroids. Regular expression Finds David David Dav(e|(id)) David, Dave Dav(e|(id)|(ide)|o) David, Dave, Davide, Davo At{1,2}enborough Attenborough, Atenborough Atte[nm]borough Attenborough, Attemborough At{1,2}[ei][nm]bo{0,1}ro((ugh)|w){0,1} Atimbro, attenbrough, ateinborow Easy counting, replacing all with “Sir David Attenborough”
- 44. Functions
- 45. Functions • R has many. e.g.: plot(), t.test() • Making your own: tree_age_estimate <- function(diameter, species) { [...do the magic... maybe something like: growth.rate <- growth.rates[ species ] age.estimate <- diameter / growth.rate ...] ! return(age.estimate) } > tree_age_estimate(25, “White Oak”) + 66 > tree_age_estimate(60, “Carya ovata”) + 190
- 46. Your turn • Create a function that takes as input a length in centimetres and returns the length in feet+inches.
- 47. Function
- 48. Loops
- 49. “for” Loop > possible_colours <- c('blue', 'cyan', 'sky-blue', 'navy blue', 'steel blue', 'royal blue', 'slate blue', 'light blue', 'dark blue', 'prussian blue', 'indigo', 'baby blue', 'electric blue') ! > possible_colours [1] "blue" "cyan" "sky-blue" "navy blue" [5] "steel blue" "royal blue" "slate blue" "light blue" [9] "dark blue" "prussian blue" "indigo" "baby blue" [13] "electric blue" ! > for (colour in possible_colours) { + print(paste("The sky is oh so, so", colour)) + } ! [1] "The sky is so, oh so blue" [1] "The sky is so, oh so cyan" [1] "The sky is so, oh so sky-blue" [1] "The sky is so, oh so navy blue" [1] "The sky is so, oh so steel blue" [1] "The sky is so, oh so royal blue" [1] "The sky is so, oh so slate blue" [1] "The sky is so, oh so light blue" [1] "The sky is so, oh so dark blue" [1] "The sky is so, oh so prussian blue" [1] "The sky is so, oh so indigo" [1] "The sky is so, oh so baby blue" [1] "The sky is so, oh so electric blue"
- 50. Your turn •What does the following code do (decompose on pen and paper)
- 51. Your turn • Create a loop that multiplies the numbers from ‘x’ to ‘y’