NLP
- 1. Natural language processing (NLP) Presented By : Mohamed El-Serngawy
- 2. Agenda Definition & Introduction Steps in NLP Statistical NLP Real World Application Demos with free NLP Application
- 3. Definition & Introduction Natural language processing (NLP) is a field of computer science and linguistics concerned with the interactions between computers and human (natural) languages Why Natural Language Processing ? Huge amounts of data ◦ Internet = at least 20 billions pages ◦ Intranet Applications for processing large amounts of texts require NLP expertise
- 4. Definition & Introduction We look at how we can exploit knowledge about the world, in combination with linguistic facts, to build computational natural language systems. Natural language generation systems convert information from computer databases into readable human language, Natural language understanding systems convert samples of human language into more formal representations such as parse trees or first- order logic structures that are easier for computer programs to manipulate
- 5. Steps in NLP Phonetics, Phonology: how Word are prononce in termes of sequences of sounds Morphological Analysis: Individual words are analyzed into their components and non word tokens such as punctuation are separated from the words. Syntactic Analysis: Linear sequences of words are transformed into structures that show how the words relate to each other. Semantic Analysis: The structures created by the syntactic analyzer are assigned meanings. Discourse integration: The meaning of an individual sentence may depend on the sentences that precede it and may influence the meanings of the sentences that follow it. Pragmatic Analysis: The structure representing what was said is reinterpreted to determine what was actually meant.
- 6. Phonetics Study of the physical sounds of human speech ◦ /i:/, /ɜ:/, /ɔ:/, /ɑ:/ and /u:/ ◦ 'there' => /ðeə/ ◦ 'there on the table' => /ðeər ɒn ðə teɪbl / •Transcription of sounds (IPA)
- 7. Phonetic Articulory phonetics : production • Auditory phonetics : speech perception – McGurk effect • Acoustics phonetics: properties of sound waves (frequency and harmonics)
- 8. Morphological Analysis Suppose we have an english interface to an operating system and the following sentence is typed: ◦ I want to print Bill’s .init file. Morphological analysis must do the following things: ◦ Pull apart the word “Bill’s” into proper noun “Bill” and the possessive suffix “’s” ◦ Recognize the sequence “.init” as a file extension that is functioning as an adjective in the sentence. This process will usually assign syntactic categories to all the words in the sentence. Consider the word “prints”. This word is either a plural noun or a third person singular verb ( he prints ).
- 9. Syntactic Analysis Syntactic analysis must exploit the results of morphological analysis to build a structural description of the sentence. The goal of this process, called parsing, is to convert the flat list of words that forms the sentence into a structure that defines the units that are represented by that flat list. The important thing here is that a flat sentence has been converted into a hierarchical structure and that the structure correspond to meaning units when semantic analysis is performed. Reference markers are shown in the parenthesis in the parse tree Each one corresponds to some entity that has been mentioned in the sentence.
- 10. Syntactic Analysis Syntactic Processing : Almost all the systems that are actually used have two main components: ◦ A declarative representation, called a grammar, of the syntactic facts about the language. ◦ A procedure, called parser, that compares the grammar against input sentences to produce parsed structures.
- 11. Syntactic Analysis Grammars and Parsers : The most common way to represent grammars is as a set of production rules. A simple Context-free phrase structure grammar fro English: S → NP VP NP → the NP1 NP → PRO NP → PN NP → NP1 NP1 → ADJS N ADJS → ε | ADJ ADJS VP → V VP → V NP N → file | printer PN → Bill PRO → I ADJ → short | long | fast V → printed | created | want First rule can be read as “ A sentence is composed of a noun phrase followed by Verb Phrase”; Vertical bar is OR ; ε represnts empty string. Symbols that are further expanded by rules are called nonterminal symbols. Symbols that correspond directly to strings that must be found in an input sentence are called terminal symbols.
- 12. Syntactic Analysis S NP PN Bill VP V printed NP the NP1 ADJS E N file Bill Printed the file A Parse tree for a sentence :
- 13. Syntactic Analysis A parse tree : John ate the apple. 1. S -> NP VP 2. VP -> V NP 3. NP -> NAME 4. NP -> ART N 5. NAME -> John 6. V -> ate 7. ART-> the 8. N -> apple S NP VP NAME John V ate NP ART N the apple
- 14. Semantic Analysis Semantic analysis must do two important things: ◦ It must map individual words into appropriate objects in the knowledge base or database ◦ It must create the correct structures to correspond to the way the meanings of the individual words combine with each other.
- 15. Semantic Analysis Lexical processing : The first step in any semantic processing system is to look up the individual words in a dictionary ( or lexicon) and extract their meanings. Many words have several meanings, and it may not be possible to choose the correct one just by looking at the word itself. The process of determining the correct meaning of an individual word is called word sense disambiguation or lexical disambiguation. It is done by associating, with each word in lexicon, information about the contexts in which each of the word’s senses may appear. Sometimes only very straightforward info about each word sense is necessary. For example, baseball field interpretation of diamond could be marked as a LOCATION. Some useful semantic markers are : ◦ PHYSICAL-OBJECT ◦ ANIMATE-OBJECT ◦ ABSTRACT-OBJECT
- 16. Semantic Analysis Word Net (common sense KnowledgBase) : A database of lexical relations. Inspired by current psycholinguistic theories of human lexical memory. Synset: a set of synonyms, representing one underlying lexical concept ◦ Example: fool {chump, fish, fool, gull, mark, patsy, fall guy, sucker, schlemiel, shlemiel, soft touch, mug} Relations link the synsets: hypernym, Has- Member, Member-Of, Antonym, etc. 16
- 17. Semantic Analysis Example pu-erh.cs.utexas.edu$ wn bike -partn Part Meronyms of noun bike 2 senses of bike Sense 1 motorcycle, bike HAS PART: mudguard, splashguard Sense 2 bicycle, bike, wheel HAS PART: bicycle seat, saddle HAS PART: bicycle wheel HAS PART: chain HAS PART: coaster brake HAS PART: handlebar HAS PART: mudguard, splashguard HAS PART: pedal, treadle, foot lever HAS PART: sprocket, sprocket wheel 17 • Example Pu-erh.cs.utexas.edu$wn bike Information available for noun bike -hypen Hypernyms -hypon, -treen Hyponyms & Hyponym Tree -synsn Synonyms (ordered by frequency) -partn Has Part Meronyms -meron All Meronyms -famln Familiarity & Polysemy Count -coorn Coordinate Sisters -simsn Synonyms (grouped by similarity of meaning) -hmern Hierarchical Meronyms -grepn List of Compound Words -over Overview of Senses Information available for verb bike -hypev Hypernyms -hypov, -treev Hyponyms & Hyponym Tree -synsv Synonyms (ordered by frequency) -famlv Familiarity & Polysemy Count -framv Verb Frames -simsv Synonyms (grouped by similarity of meaning) -grepv List of Compound Words -over Overview of Senses
- 18. Discourse Integration Specifically we do not know whom the pronoun “I” or the proper noun “Bill” refers to. To pin down these references requires an appeal to a model of the current discourse context, from which we can learn that the current user is USER068 and that the only person named “Bill” about whom we could be talking is USER073. Once the correct referent for Bill is known, we can also determine exactly which file is being referred to.
- 19. Pragmatic Analysis The final step toward effective understanding is to decide what to do as a results. One possible thing to do is to record what was said as a fact and be done with it. For some sentences, whose intended effect is clearly declarative, that is precisely correct thing to do. But for other sentences, including this one, the intended effect is different. We can discover this intended effect by applying a set of rules that characterize cooperative dialogues. The final step in pragmatic processing is to translate, from the knowledge based representation to a command to be executed by the system. The results of the understanding process is
- 20. Pragmatic Analysis Knowledge about the kind of actions that speakers intend by their use of sentences ◦ REQUEST: HAL, open the pod bay door. ◦ STATEMENT: HAL, the pod bay door is open. ◦ INFORMATION QUESTION: HAL, is the pod bay door open? Speech act analysis (politeness, irony, greeting, apologizing...)
- 21. Statistical NLP Statistical NLP aims to perform statistical inference for the field of NLP Statistical inference consists of taking some data generated in accordance with some unknown probability distribution and making inferences.
- 22. Motivations for Statistical NLP Cognitive modeling of the human language processing has not reached a stage where we can have a complete mapping between the language signal and the information contents. Complete mapping is not always required. Statistical approach provides the flexibility required for making the modeling of a language more accurate.
- 23. Real World Application Automatic summarization Foreign language reading aid Foreign language writing aid Information extraction Information retrieval (IR) - IR is concerned with storing, searching and retrieving information. It is a separate field within computer science (closer to databases), but IR relies on some NLP methods (for example, stemming). Some current research and applications seek to bridge the gap between IR and NLP. Machine translation - Automatically translating from one human language to another. Named entity recognition (NER) - Given a stream of text, determining which items in the text map to proper names, such as people or places. Although in English, named entities are marked with capitalized words, many other languages do not use capitalization to distinguish named entities. Natural language generation Natural language search
- 24. Real World Application Natural language understanding Optical character recognition anaphora resolution Query expansion Question answering - Given a human language question, the task of producing a human-language answer. The question may be a closed-ended (such as "What is the capital of Canada?") or open-ended (such as "What is the meaning of life?"). Speech recognition - Given a sound clip of a person or people speaking, the task of producing a text dictation of the speaker(s). (The opposite of text to speech.) Spoken dialogue system Stemming Text simplification Text-to-speech Text-proofing
- 25. Demos with free NLP Application DEMO
- 26. THANKS