MultiMWE: Building a Multi-lingual Multi-Word Expression (MWE) Parallel Corpora
0 likes133 views
The document discusses the development of a multilingual multi-word expression (MWE) corpus and its applications in natural language processing (NLP) and machine translation (MT). It outlines previous research on MWEs, the methodology for corpus creation, and the interaction of MWEs with various NLP tasks, emphasizing their importance in improving translation accuracy. Additionally, several examples and experimental results demonstrate the effectiveness of integrating MWEs into MT systems.
MultiMWE: Building a Multi-lingual Multi-Word Expression (MWE) Parallel Corpora
- 1. MultiMWE: Building a Multi-lingual Multi-Word Expression (MWE) Parallel Corpora Lifeng Han, Gareth Jones and Alan Smeaton ADAPT research seminar, 2020 March Accepted long paper in 12th Edition of Language Resources and Evaluation Conference (LREC2020)
- 2. Outline • Multiword Expressions (MWEs) in ADAPT • - started earlier, EACL-MWE2017 shared task by TCD+DCU • Background of MWEs • - intro MWE, tasks in NLP, corpus, with Machine Translation (MT) • MultiMWE corpus • - how we made it (workflow) • MultiMWE corpus applications • - in MT, but it can go further in Multilingual/Crosslingual NLP task
- 3. vMWEs in ADAPT • “Detection of Verbal Multi-Word Expressions via Conditional Random Fields with Syntactic Dependency Features and Semantic Re-Ranking". 2017. Alfredo Maldonado, Lifeng Han, Erwan Moreau, Ashjan Alsulaimani, Koel Dutta Chowdhury, Carl Vogel, Qun Liu. The 13th Workshop on Multiword Expressions @ EACL 2017 • - exploits universal syntactic dependency features through a Conditional Random Fields (CRF) sequence model • - Closed Track at the PARSEME VMWE Shared Task 2017, ranking 2nd place in most languages on full VMWE-based evaluation and 1st in three languages on token-based evaluation (Polish, Swedish, French) • - an option to re-rank the 10 best CRF-predicted sequences via semantic vectors, boosting its scores above other systems in the competition. • Chapter: “Semantic reranking of CRF label sequences for verbal multiword expression identification”. 2018. Erwan Moreau, Ashjan Alsulaimani, Alfredo Maldonado, Lifeng Han, Carl Vogel, Koel Dutta Chowdhury. In Stella Markantonatou, Carlos Ramisch, Agata Savary & Veronika Vincze (eds.), Multiword expressions at length and in depth: Extended papers from the MWE 2017 workshop, 177–207. Berlin: Language Science Press. DOI:10.5281/zenodo.1469559
- 4. Definition • a MWE shall be a term including several words to express a specific concept, which is able to be decomposed, and the words combined together as an MWE are syntactically, semantically, pragmatically, or statistically idiosyncratic in nature.{Sag2002MWE,Tim2010mwe,huning2015MWEs} • MWE (Sag2002): • lexical phrases: fixed or semi-fixed expressions and syntactically-flexible expressions • institutionalized phrases
- 5. Examples • Categories of MWE: • idioms (…kick the bucket) • Metaphor (…apple of my eye) • compound nouns (tooth + paste -> toothpaste) • word combinations from various part-of-speech set: verb-particle, adjective-verb (dry cleaning), preposition-verb (output), adjective-noun (full moon), preposition-noun (underground), etc. • … • Appearance of MWE: • Continuous (kick off, football magazine) • Dis-continuous (pick someone up) ref: https://www.learnenglish.de/grammar/nouncompound.html
- 6. MWE events • MWE detection, acquisition, construction, treatment, disambiguation • MWE interaction with other NLP tasks e.g. MT: • EUROPHRAS 4th Workshop on Multi-word Units in Machine Translation and Translation Technology (MUMTTT 2019) — 27 September 2019 — Málaga, • ACL-SIGLEX orgnized MWE related events (since 2003, yearly) • 2003 ACL workshop Multiword Expressions: Analysis, Acquisition and Treatment – Sapporo, Japan • 2020 COLING Joint Workshop on Multiword Expressions and Electronic Lexicons (MWE-LEX 2020) – September 14, 2020 – Barcelona, Spain.
- 7. Joint topics on MWEs and e-lexicons: Extracting and enriching MWE lists from traditional human-readable lexicons for NLP use Formats for NLP-applicable MWE lexicons Interlinking MWE lexicons with other language resources Using MWE lexicons in NLP tasks (identification, parsing, translation, MWE discovery in the service of lexicography Multiword terms in specialised lexicons Representing semantic properties of MWEs in lexicons Paving the way towards encoding lexical idiosyncrasies in constructions
- 8. MWE related corpora • MWE aware English Dependency corpus • from LDC(https://catalog.ldc.upenn.edu/LDC2017T01), • - annotated English compound words in the corpus as one kind of MWE • - to facilitate the constituency and dependency parsing task • English MWEs in web reviews data (schneider-etal-2014) • http://www.cs.cmu.edu/~ark/LexSem/ • hand-annotated online review data with comprehensive MWEs including English noun, verb, and preposition super-senses (tags include communication, group, stative, location, possession, etc.). • these MWE corpus construction works are monolingual tasks and focus on English only
- 9. MWE related corpora • multilingual MWE corpus construction from the PARSEME • - The IC1207 COST Action, is an interdisciplinary scientific network devoted to the role of multi-word expressions (MWEs) in parsing • - includes 18 European languages • - corpus focuses on one kind of MWE (verbal MWE) • - not parallel, and the size of the data varies very much from language to language (some languages have only hundreds of sentences).
- 10. MWE interaction with MT • Báihuà (⽩白話): contemporary Chinese • 去 (qù) ... 了了(le): is a simple dis-continuous Chinese MWE used to express a past tense action (went to do something, went to somewhere) • The purpose ‘上課 (shangke)’ is not translated. ZH source: ZH pinyin: Xiǎo míng qù xué xiào shàng kè le EN reference: Xiao Ming went to school to attend classes EN MT output: Xiao Ming went to school note: Three zh->en examples from googleMT 2018/10
- 11. MWE interaction with MT • Poem: Shīgē(詩歌). 花 as ambiguous word. • Well aligned MWEs shall help disambiguation. 花 vs ⼈人 (flower - people) ZH source: ZH pinyin: Nián nián suì suì huā xiāng sì, suì suì nián nián rén bù tóng. EN reference: The flowers are similar each year, while people are changing year after year. EN MT output: One year spent similar, each year is different
- 12. MWE interaction with MT • Wényán (⽂文⾔言): ancient Chinese. • 安知 as MWE means ‘…do not know…’. Or 安知…哉 means ‘how can … know …?’ ZH source: ZH pinyin: Yàn què ān zhī hóng hú zhī zhì zāi? EN reference (literal): How can a finch know the ambition of a big bird (or swan)? EN MT output: What is the meaning of Yanque Anzhihong? EN reference: The nonsense forks do not know the ambitions of the very motivated people.
- 13. MWE interaction with MT • Some earlier work about MWE+MT: • {lambert2005mwe} bilingual MWE pairs to modify the word alignment procedure of MT on an English-Spanish corpus. grouping the MWEs as one token before training • {Ren2009mwe} integrated bilingual Chinese-English MWEs into the SMT toolkit Moses • {Bouamor2012IdentifyingBM} which designed models to extract continuous MWEs for French- English translation • {Skadina2016MultiwordEI} which discussed various MWEs in English-Latvian MT • {EBRAHIM2017111mwe} focused on phrasal verb MWEs in Arabic-English phrase-based SMT. • MWE+NMT: • {Li2016neuralname} a character level sequence to sequence modeling to translate named entities and then integrated this into an overall NMT system on a Chinese-to-English • {ugawa2018neural} focuses on the difficulty of translation of compound words in the source language, by introducing an encoder for the input word at the NE tag level at each time step.
- 14. MultiMWE corpus • MWE extraction pipeline from (Rikters and Bojar, 2017) https://github.com/M4t1ss/MWE-Tools • extend the extraction work into language pairs such as German-English and Chinese-English • extracted MWEs from our experiments are freely available for MT and NLP researchers • However, the extracted MWE candidates from this framework is only the continuous type. In follow up work, we will design some patterns or other models to extract discontinuous MWEs
- 15. MultiMWE corpus • Extract the monolingual MWEs + alignment + filtering Corpus construction workflow together with corpus flow diagram
- 16. MultiMWE corpus • Some pattern examples used for Chinese monolingual MWE extraction <pat><w pos="a"/><w pos="a"/><w pos="a"/><w pos="n"/></pat> <pat><w pos="a"/><w pos="a"/><w pos="a"/><w pos="n"/><w pos="n"/></pat> <pat><w pos="a"/><w pos="a"/><w pos="a"/><w pos="n"/><w pos="n"/><w pos="n"/></pat> <pat><w pos="a"/><w pos="a"/><w pos="n"/></pat> <pat><w pos="a"/><w pos="a"/><w pos="n"/><w pos="a"/><w pos="n"/></pat> <pat><w pos="a"/><w pos="a"/><w pos="n"/><w pos="n"/></pat> <pat><w pos="a"/><w pos="a"/><w pos="n"/><w pos="n"/><w pos="n"/></pat> <pat><w pos="a"/><w pos="a"/><w pos="n"/><w pos="n"/><w pos="n"/><w pos="n"/></pat> <pat><w pos="a"/><w pos="c"/><w pos="a"/><w pos="n"/></pat> <pat><w pos="a"/><w pos="c"/><w pos="a"/><w pos="n"/><w pos="n"/></pat> <pat><w pos="a"/><w pos="c"/><w pos="a"/><w pos="n"/><w pos="n"/><w pos="n"/></pat> <pat><w pos="a"/><w pos="c"/><w pos="n"/><w pos="n"/></pat> <pat><w pos="a"/><w pos="c"/><w pos="n"/><w pos="n"/><w pos="n"/></pat> <pat><w pos="a"/><w pos="n"/></pat> <pat><w pos="a"/><w pos="n"/><w pos="a"/><w pos="a"/><w pos="n"/></pat> <pat><w pos="a"/><w pos="n"/><w pos="a"/><w pos="n"/></pat> <pat><w pos="a"/><w pos="n"/><w pos="a"/><w pos="n"/><w pos="n"/></pat>
- 17. MultiMWE corpus begin{itemize} item Morphological tagging of De and En. item Tagged De/En into XML format. item Design MWE-patterns for De/En item Extract Monolingual MWEs with MWEtoolkit item Generate De-En lexicon translation probability files with Giza++ and Moses item Align Bilingual MWEs with MPAligner end{itemize} workflow: take de-en language pair as one example
- 18. MultiMWE corpus • Chinese-English MWEs, similar de-en, except: • item PoS pattern design. • item Stop-word list preparation. • item Zh-En translation probability files.
- 19. MultiMWE corpus • Added Chinese Patterns for MWEs from the LCMC Tags, in addition to the mapped pos tags from en language. Lancaster Corpus of Mandarin Chinese https://www.lancaster.ac.uk/fass/projects/corpus/LCMC/
- 20. MultiMWE corpus Chinese English Estimation cat ears 0.780979 long tail 0.820427 small dustpan 0.856796 artistic works 0.6281 group table 0.708438 computer expert 0.801311 golf club 0.976473 acne products 0.695547 different conditions 0.887839 evergreen plant 0.610852 note: ( )->( ) simplified to traditional Chinese character, used in paper content Extracted zh-en pairs Without running Head of the file Most scores above 0.6 簸箕 (Bòji) MWE non-decomposable 电脑 (Diànnǎo): electricity+brain
- 21. MultiMWE corpus German English Estimation europäische Kommission european commission 0.970964 upcoming events upcoming events 1 europäischen Kommission european commission 0.990844 praktische Informationen practical information 0.948533 östlichen Teils eastern part 0.793047 private Konzession private concession 0.921197 französische Staat french state 0.853861 europäischen Rat european council 0.984224 größeren Infrastrukturprojekten major infrastructure projects 0.853873 zwischengeschalteten Banken intermediary banks 0.754617 Pruned de-en with threshold 0.70 with examples from the Head of the file
- 23. MultiMWE Application • Theoretically and technically MultiMWE corpus shall be useful for most cross-lingual/multilingual NLP tasks • Here we introduce the experiments we did on MT • De->En and Zh->En • All attention based NMT model with THUMT toolkit. • 5 millions of training sentences for each language pair • Development and testing set from WMT workshop www.statmt.org
- 25. MultiMWE Application • ⼝口⽔水戰 (Kǒushuǐ zhàn): war of words. translated into ``water fighting" by the Baseline, while translated into ``oral combat" in a proper way by MWE enhanced model. Though not reflected by BLEU. • Baseline translation is due to that this is a metaphor expression in Chinese using ⼝口⽔水+戰 that is a combination of saliva(səˈlʌɪvə)+war • Modifier MWE 所謂 is used to express ``supposed" or ``so-called”, missed out by baseline. Examples of MWE translations in MT outputs Src Ref the leaders of Russia and Turkey met on Tuesday to shake hands and declare a formal end to an eight - month long war of words and economic sanctions . Base Russian and Turkish leaders met Tuesday , shaking hands and declaring the official end of eight months of water fighting and economic sanctions . B+MWE Russian and Turkish leaders met on Tuesday and both shook hands and announced a formal end of eight months of oral combat and economic sanctions . Src Ref the offence was even greater , coming from a supposed friend . Base attacks from a friend are even harder to accept . B+MWE the attack from so-called friends is harder to accept . Src: source; Ref: reference. B+MWE: Baseline+MWE. Simplified Chinese ( , ) mapping into Traditional ( , ), used in paper.
- 26. Other Selected references • Chinese character for MT, and Named Entity as MWE: • Han, L. and Kuang, S. (2018). Incorperating chinese radicals into neural machine translation: Deeper than character level. Presented in ESSLLI-2018, https://arxiv.org/abs/1805.01565 • Han, A. L.-F., Wong, D. F., and Chao, L. S. (2013). Chinese named entity recognition with conditional random fields in the light of chinese characteristics. In Language Processing and Intelligent Information Systems, pages 57–68. Springer. • The MultiMWE corpus, + paper references: https://github.com/poethan/MWE4MT • MWE and tools: • Sag, I. A., Baldwin, T., Bond, F., Copestake, A., and Flickinger, D. (2002). Multiword expressions: A pain in the neck for nlp. In Alexander Gelbukh, editor, Com- putational Linguistics and Intelligent Text Processing, pages 1–15, Berlin, Heidelberg. Springer Berlin Heidel- berg. • Rikters, M. and Bojar, O. (2017). Paying Attention to Multi-Word Expressions in Neural Machine Translation. In Proceedings of the 16th Machine Translation Summit (MT Summit 2017), Nagoya, Japan. • Pinnis, M. (2013). Context independent term mapper for European languages. In Proceedings of the International Conference Recent Advances in Natural Language Pro- cessing RANLP 2013, pages 562–570, Hissar, Bulgaria, September. INCOMA Ltd. Shoumen, BULGARIA. • Ramisch, C. (2015). Multiword Expressions Acquisition: A Generic and Open Framework, volume XIV of The- ory and Applications of Natural Language Processing. Springer.