3. introduction to text mining
- 1. Introduction to Text Mining
- 2. Agenda • Defining Text Mining • Structured vs. Unstructured Data • Why Text Mining • Some Text Mining Ambiguities • Pre-processing the Text
- 3. Text Mining • The discovery by computer of new, previously unknown information, by automatically extracting information from a usually large amount of different unstructured textual resources Previously unknown means: • Discovering genuinely new information • Discovering new knowledge vs. merely finding patterns is like the difference between a detective following clues to find the criminal vs. analysts looking at crime statistics to assess overall trends in car theft Unstructured means: • Free naturally occurring text • As opposed HTML, XML….
- 4. Text Mining Vs. Data Mining • Data in Data mining is a series of numbers. Data for text mining is a collection of documents. • Data mining methods see data in spreadsheet format. Text mining methods see data in document format
- 5. Structured vs. Unstructured Data • Structured data • Loadable into “spreadsheets” • Arranged into rows and columns • Each cell filled or could be filled • Data mining friendly • Unstructured daa • Microsoft Word, HTML, PDF documents, PPTs • Usually converted into XML semi structured • Not structured into cells • Variable record length, notes, free form survey-answers • Text is relatively sparse, inconsistent and not uniform • Also images, video, music etc.
- 6. Why Text Mining? • Leveraging text should improve decisions and predictions • Text mining is gaining momentum • Sentiment analysis (twitter, facebook) • Predicting stock market • Predicting churn • Customer influence • Customer service and help desk • Not to mention Watson
- 7. Why Text Mining is Hard? • Language is ambiguous • Context is needed to clarify • The same words can have different meaning (homographs) • Bear (verb) – to support or carry • Bear (noun) – a large animal • Different words can mean the same (synonyms) • Language is subtle • Concept / word extraction usually results in huge number of dimensions • Thousands of new fields • Each field typically has low information content (sparse) • Misspellings, abbreviations, spelling variants • Renders search engines, SQL queries.. ineffective.
- 8. Some Text Mining Ambiguities • Homonomy: same word, different meaning • Mary walked along the bank of the river • HarborBank is the richest bank in the citys • Synonymy: Synonyms, different words, similar or same meaning, can substitute one word for other without changing meaning • Miss Nelson became a kind of big sister to Benjamin • Miss Nelson became a kind of large sister to Benjamin • Polysemy: same word or form, but different, albeit related meaning • The bank raised its interest rates yesterday • The store is next to the newly constructed bank • The bank appeared first in Italy I the Renaissance • Hyponymy: Concept hierarchy or subclass • Animal (noun) – cat, dog • Injury – broken leg, intusion
- 9. Seven Types of Text Mining • Search and Information Retrieval – storage and retrieval of text documents, including search engines and keyword search • Document Clustering – Grouping and categorizing terms, snippets, paragraphs or documents using clustering methods • Document Classification – grouping and categorizing snippets, paragraphs or document using data mining classification methods, based on methods trained on labelled examples • Web Mining – Data and Text mining on the internet with specific focus on scaled and interconnectedness of the web • Information Extraction – Identification and extraction of relevant facts and relationships from unstructured text • Natural Language Processing – Low level language processing and understanding of tasks (eg. Tagging part of speech) • Concept extraction – Grouping of words and phrases into semantically similar groups
- 10. Text Mining – Some Definitions • Document – a sequence of words and punctuation, following the grammatical rules of the language. • Term – usually a word, but can be a word-pair or phrase • Corpus – a collection of documents • Lexicon – set of all unique words in corpus
- 11. Pre-processing the Text • Text Normalization • Parts of Speech Tagging • Removal of stop words Stop words – common words that don’t add meaningful content to the document • Stemming • Removing suffices and prefixes leaving the root or stem of the word. • Term weighting • POS Tagging • Tokenization
- 12. Text Normalization • Case • Make all lower case (if you don’t care about proper nouns, titles, etc) • Clean up transcription and typing errrors • do n’t, movei • Correct misspelled words • Phonetically • Use fuzzy matching algorithms such as Soundex, Metaphone or string edit distance • Dictionaries • Use POS and context to make good guess
- 13. Parts of Speech Tagging • Useful for recognizing names of people, places, organizations, titles • English language • Minimum set includes noun, verb, adjective, adverb, prepositions, congjunctions POS Tags from Penn Tree Bank Tag Description Tag Description Tag Description CC Coordinating Conjunction CD Cardinal Number DT Determiner EX Existential there FW Foreign Word IN Preposition or subordinating conjuction JJ Adjective JJR Adjective, comparative JJS Adjective, superlative LS List Item Marker MD Modal NN Noun, singular or mass NNS Noun Plural NNPS Proper Noun Plural PDT Prederminer POS Possessive Ending PRP Personal pronoun PRPS Possessive pronoun RB Adverb RBR Adverb, comparative RBS Adverb, superlative RP Particle SYM Symbol TO To UH Interjection VB Verb, base form VBD Verb, past tens
- 14. Example of Tagging • In this talk, Mr. Pole discussed how Target was using Predictive Analytics including descriptions of using potential value models, coupon models, and yes predicting when a woman is due • In/IN this/DT talk/NN, Mr./NNP Pole/NNP discussed/VBD how/WRB Target/NNP was/VBD using/VBG Predictive/NNP Analytics/NNP including/VBG descriptions/NNS of/IN using/VBG potential/JJ value/NN models/NNS, coupon/NN models/NNS, and yes predicting/VBG when/WRB a/DT woman/NN is due/JJ
- 15. Tokenization • Converts streams of characters into words • Main clues (in English): Whitespace • No single algorithms ‘works’ always • Some languages do not have white space (Chinese, Japanese)
- 16. Stemming • Normalizes / unifies variations of the same data • ‘walking’, ‘walks’, ‘walked’, ‘walked’ walk • Inflectional stemming • Remove plurals • Normalize verb tenses • Remove other affixes • Stemming to root • Reduce word to most basic element • More aggressive than inflectional • ‘denormalization’ norm • ‘Apply’, ‘applications’, ‘reapplied’ apply
- 17. Common English Stop Words • a, an, and, are, as, at, be, but, buy, for, if, in, into, is, it, no, not, of, on, or, such, that, the, their, then, these, they, this, to, was, will, with • Stop words are very common and rarely provide useful information for information extraction and concept extraction • Removing stop words also reduce dimensionality
- 18. Dictionaries and Lexicons • Highly recommended, can be very time consuming • Reduces set of key words to focus on • Words of interest • Dictionary words • Increase set of keywords to focus on • Proper nouns • Acronyms • Titles • Numbers • Key ways to use dictionary • Local dictionary (specialized words) • Stop words and too frequent words • Stemming – reduce stems to dictionary words • Synonyms – replace synonyms with root words in the list • Resolve abbreviations and acronyms
- 19. Sentiment Analysis Workflow Content Retrieval Content Extraction Corpus Generation Corpus Transformation Corpus Filtering Sentiment Calculation WebDataRetrievalCorpusPre Processing Sentiment Analysis
- 20. Sentiment Indicators • 𝑝𝑜𝑙𝑎𝑟𝑖𝑡𝑦 = 𝑝−𝑛 𝑝+𝑛 • 𝑠𝑢𝑏𝑗𝑒𝑐𝑡𝑖𝑣𝑖𝑡𝑦 = 𝑝+𝑛 𝑁 • 𝑝𝑜𝑠𝑖𝑡𝑖𝑣𝑒 𝑠𝑒𝑡𝑛𝑖𝑚𝑒𝑛𝑡𝑠 𝑝𝑒𝑟 𝑟𝑒𝑓𝑒𝑟𝑒𝑛𝑐𝑒 = 𝑝 𝑁 • 𝑛𝑒𝑔𝑎𝑡𝑖𝑣𝑒 𝑠𝑒𝑡𝑛𝑖𝑚𝑒𝑛𝑡𝑠 𝑝𝑒𝑟 𝑟𝑒𝑓𝑒𝑟𝑒𝑛𝑐𝑒 = 𝑛 𝑁 • 𝑠𝑒𝑡𝑛𝑖𝑚𝑒𝑛𝑡 𝑑𝑖𝑓𝑓𝑒𝑟𝑒𝑛𝑐𝑒𝑠 𝑝𝑒𝑟 𝑟𝑒𝑓𝑒𝑟𝑒𝑛𝑐𝑒 = 𝑝 − 𝑛 𝑁