Bioinformatica: Leggere file con Perl, e introduzione alle espressioni regolari (BMR Genomics) - Lezione 29 agosto 2014
0 likes471 views
This document summarizes an introduction to bioinformatics training on Perl programming. It covers simulating shotgun sequencing using Perl, reading text files line by line in Perl, and parsing file formats like SAM. It also introduces key concepts in Perl like pattern matching using regular expressions, including metacharacters, quantifiers, character classes, alternatives, and capturing portions of patterns.
![SAM to FASTQ
1. Print user manual
2. Get arguments from command line via @ARGV
3. Open the SAM file, we’ll use “SAM” as file handle
while ($line = <SAM>) {!
chomp($line);!
@sam_fields = split(/t/, $line);!
print “@$sam_fields[0]n”;
print “$sam_fields[9]n+n”; "
print “$sam_fields[10]n”;!
}](https://image.slidesharecdn.com/bmrperl-readfiles-140905035233-phpapp01/85/Bioinformatica-Leggere-file-con-Perl-e-introduzione-alle-espressioni-regolari-BMR-Genomics-Lezione-29-agosto-2014-22-320.jpg)
![Character classes
• List of character, and meta-, enclosed by [ ]
In this context the ^ means not
• [aeiou] any vowel
• [A-Z] any upper case
• [^a-z] non lowercase letters
• [ACGTacgt] any DNA letter](https://image.slidesharecdn.com/bmrperl-readfiles-140905035233-phpapp01/85/Bioinformatica-Leggere-file-con-Perl-e-introduzione-alle-espressioni-regolari-BMR-Genomics-Lezione-29-agosto-2014-43-320.jpg)
Bioinformatica: Leggere file con Perl, e introduzione alle espressioni regolari (BMR Genomics) - Lezione 29 agosto 2014
- 1. Bioinformatics Training Introduction to Perl Programming Andrea Telatin
- 2. Today’s menu • Homeworks: Shotgun simulation • Reading a file (line by line): SAM format • Pattern matching
- 3. Simulating a Shotgun Homework review
- 4. Simulating a Shotgun Homework review
- 5. Print a random substring INPUT: $sequenza = ‘ACGATCGTACGTAGCTGACTGATCGTACGTAACGTACGTAGCTGACTGATCGA TCGTAGCTAGCAGCTGATCGACTGACTGACTGATCGATCGTACGTAGCTAGCTG ACTGATC’; $lunghezza = length($sequenza); $lunghezza_read = 12; $posizione = int(rand($lunghezza - $lunghezza_read); $seq = substr($sequenza, $posizione, $lunghezza_read); print “>Randomn$seqn”;
- 6. Print a random substring $sequenza = ‘ACGATCGTACGTAGCTGACTGATCGTACGTAACGTACGTAGCTGACTGATCGA TCGTAGCTAGCAGCTGATCGACTGACTGACTGATCGATCGTACGTAGCTAGCTG ACTGATC’; $lunghezza = length($sequenza); $lunghezza_read = 12; $posizione = int(rand($lunghezza - $lunghezza_read); $seq = substr($sequenza, $posizione, $lunghezza_read); print “>Randomn$seqn”; COMPUTATION
- 7. $sequenza = ‘ACGATCGTACGTAGCTGACTGATCGTACGTAACGTACGTAGCTGACTGATCGA TCGTAGCTAGCAGCTGATCGACTGACTGACTGATCGATCGTACGTAGCTAGCTG ACTGATC’; $lunghezza = length($sequenza); $lunghezza_read = 12; $posizione = int(rand($lunghezza - $lunghezza_read); $seq = substr($sequenza, $posizione, $lunghezza_read); print “>Randomn$seqn”; Print a random substring OUTPUT
- 8. Finished example $seq = ‘ACACTAGGTACGTAGCTGATCGTACGTAGCTGACG…’; $readnum = 100; $readlen = 40;! ! for ($i = 1; $i<=$readnum; $i++) {! $pos = int(rand(length($seq)-$readlen));! $readseq = substr($seq, $pos, $readlen);! print ">READ_$i pos=$posn$readseqn";! }
- 9. Finished example $seq = ‘ACACTAGGTACGTAGCTGATCGTACGTAGCTGACG…’; $readnum = 100; $readlen = 40;! ! for ($i = 1; $i<=$readnum; $i++) {! $pos = int(rand(length($seq)-$readlen));! $readseq = substr($seq, $pos, $readlen);! print ">READ_$i pos=$posn$readseqn";! } “HARD CODED” INPUT http://codepad.org/2OinaEX1
- 10. Taking arguments from the shell
- 11. We already met @ARGV print STDERR “Instructions…n”; ($seq, $readlen, $readnum) = @ARGV; if ($readnum==0) { ! ! die “Please specify all the params.n” } ! for ($i = 1; $i<=$readnum; $i++) {! $pos = int(rand(length($seq)-$readlen));! $readseq = substr($seq, $pos, $readlen);! print ">READ_$i pos=$posn$readseqn";! } USER SUPPLIED INPUT
- 12. Read a Text File
- 13. Reading a text file • Requires the program to know the filename, that in the BaSH jargon means its path (either relative or absolute) • In Perl we assign a nickname to the opened file, called file handle. • We use the open(FH, filename) function • The function returns “true” on success.
- 14. Reading a File if (open(FILE, ‘path/to/filename’) == 0) { die “ Cant read filename.n”; }! After opening it, we can read it line by line with a while loop: while ($currentline = <FILE>) { print “$currentline”; }! Warning: $currentline keeps its return char, if not needed remember to remove it with chomp($riga)!
- 15. Reading a File if (open(FILE, ‘path/to/filename’) == 0) { die “ Cant read filename.n”; }! After opening it, we can read it line by line with a while loop: while ($currentline = <FILE>) { print “$currentline”; }! Warning: $currentline keeps its return char, if not needed remember to remove it with chomp($riga)! File Handle File Name TYPICAL “READ FILE” LOOP
- 16. A Perl clone: cat.pl if (open(I, “$filename”) == 0) { die “Cant read “$filename”.n”; } while ($line = <I>) { chomp($line); $c++; print “$linen”; }! print STDERR “I read $c lines.n”;
- 17. A Perl clone: cat.pl if (open(I, “$filename”) == 0) { die “Cant read “$filename”.n”; } while ($line = <I>) { chomp($line); $c++; print “$linen”; }! print STDERR “I read $c lines.n”; With built-in “wc” too
- 18. Parsing a file SAM as a common example
- 19. SAM format review • Header: starts by @ • Alignments in tabular format:
- 20. SAM format review • Header: starts by @ • Alignments in tabular format:
- 21. Two simple examples • SAM to FASTQ (extracts reads from SAM file) • Subsample SAM
- 22. SAM to FASTQ 1. Print user manual 2. Get arguments from command line via @ARGV 3. Open the SAM file, we’ll use “SAM” as file handle while ($line = <SAM>) {! chomp($line);! @sam_fields = split(/t/, $line);! print “@$sam_fields[0]n”; print “$sam_fields[9]n+n”; " print “$sam_fields[10]n”;! }
- 23. Subsample SAM In the simplest implementation we just want to print one line every pack of lines. I.e. 1 out of 10 (= 10%).
- 24. Subsample SAM 1. Print user manual ($sam_file, $denom) = @ARGV;! ! if (open(SAM, "$sam_file")==0) {! die "Unable to read SAM file: "$sam_file".n";! }! ! while ($line = <SAM>) {! $c++;! if ($c % $denom == 0) {! print $line;! }! }
- 25. Subsample SAM 1. Print user manual ($sam_file, $denom) = @ARGV;! ! if (open(SAM, "$sam_file")==0) {! die "Unable to read SAM file: "$sam_file".n";! }! ! while ($line = <SAM>) {! $c++;! if ($c % $denom == 0) {! print $line;! }! } is there something to fix here?
- 26. Another example Adding a little bit of spice
- 27. Alignments per chromosome • We basically want to count how many alignments were found per reference sequence • We might be interested, also, in normalising the number of alignments by chromosome length
- 28. SAM format review • Header: starts by @ • Alignments in tabular format:
- 29. COFFE BREAK to let you think about the program :)
- 30. Let’s do it together
- 31. Let’s do it together 1. We need to store the chromosome length. That information is on the header How can we store it? 2. We need a counter for each chromosome. We don’t know in advance how many are there. Any idea for this?" !
- 32. Let’s do it together while ($line = <SAM>) {! chomp($line);! if (substr($line, 0, 1) eq ‘@‘) {! ! ! #header parsing! ! } else {! ! ! #alignments parsing! }! }" Global structure
- 33. Let’s do it together ($field, $content) = split(/t/, $line);! if ($field eq ‘@SQ’) {! ($seqname, $len) = split(/t/, $content);! $seqname = substr($seqname, 2);! …! ! $size{$seqname} = $len;! }" Header parsing
- 34. Let’s do it together 1. We need to store the chromosome length. That information is on the header How can we store it? 2. We need a counter for each chromosome. We don’t know in advance how many are there. Any idea for this?" ! Header parsing
- 36. What is it? • The key feature making Perl so powerful in processing text (i.e. parsing) • A mini language inside the language • Something like wild cards (but much more powerful) in BaSH • Used to find data and substituting it
- 37. The match operator • Patterns are usually delimited by / • Returns true if the text contains the pattern • Example: if ($dna=~/GGATCC/) { print “BamHI sensitive sequencen”; }
- 38. The substitution operator • Syntax is s/PATTERN/SUBSTITUTION/ • Returns true if the text contains the pattern • The modifier “g” at the end substitutes all the occurrences • Example: $dna=~s/GGATCC/-cut-/g;
- 39. Before starting… • http://regex101.com and many other online tools can do that too… • Let’s create our own regex tester :)
- 40. Metacharacters • ^ Requires the pattern to be found at the begin • $ Requires the pattern to be found at the end • . Matches any character (except for n) • s Matches a whitespace (space, tab…) • S Matches a non-whitespace (everything else)
- 41. Metacharacters (2) • d Matches any digit (D any non-digit) • w Matches any alphanum (W …) • . Matches any character (except for n) • s Matches a whitespace (space, tab…) • S Matches a non-whitespace (everything else)
- 42. Quantifiers • Added at the end of an entity • ? zero or one • * zero or more • + at least one (one or more) • {n} n or more • {n,m} between n and m
- 43. Character classes • List of character, and meta-, enclosed by [ ] In this context the ^ means not • [aeiou] any vowel • [A-Z] any upper case • [^a-z] non lowercase letters • [ACGTacgt] any DNA letter
- 44. Alternatives • (also) in this context the pipe char is the “or” • /TTC|GTA/ matches either TTC or GTA • Grouped by parentheses: /mi chiamo (Andrea|Paolo)/
- 45. Capturing portions • The parentheses are used to capture a portion of pattern