Text-mining practical
Download as PPT, PDF3 likes941 views
The document discusses text mining and named entity recognition techniques for finding disease genes from biomedical abstracts. It describes retrieving over 60,000 abstracts on prostate cancer and schizophrenia, identifying human genes from those texts, and prioritizing genes that are commonly mentioned between the two diseases. The task involves recognizing gene names in the abstracts, creating a "black list" to filter out incorrectly identified names, and extracting gene co-mentions to find shared disease genes.
Text-mining practical
- 1. Text-mining practical Lars Juhl Jensen
- 2. unix primer
- 5. cat
- 6. less
- 7. head -10
- 8. tail -10
- 10. cut -f 2
- 11. sort
- 12. sort -nr
- 13. uniq -c
- 15. write to file
- 17. using pipes
- 19. putting it all together
- 20. cut -f 4 infile | sort | uniq -c | sort -nr | head -100 > outfile
- 21. the task
- 24. human genes
- 26. what I have done
- 28. two diseases
- 29. prostate cancer
- 30. schizophrenia
- 31. two sets of documents
- 32. 62,755 abstracts
- 33. 65,588 abstracts
- 34. one directory with each set
- 35. one file with each abstract
- 36. dictionary
- 38. human genes
- 39. 22,523 entities
- 40. synonyms
- 45. 2,726,495 names
- 46. tagdir program
- 51. what you will do
- 56. co-mentioning
- 57. within abstracts
- 58. ank genes for each disease
- 59. find shared gene
- 61. a helping hand
- 62. “black list”
- 63. 100+ matches
- 64. 10+ matches
- 66. wrap up
- 67. Protein kinase B
- 68. PKB
- 69. Akt
- 70. AKT1
- 71. same protein
- 72. synonyms matter
- 73. “black list” is crucial
- 74. text mining is useful
- 75. not black magic
- 76. Thanks for your attention! 76