John Richardson - 2015 - KyotoEBMT System Description for the 2nd Workshop on Asian Translation
0 likes348 views
The document describes improvements made to the KyotoEBMT machine translation system. It discusses using forest parsing of input sentences to handle parsing errors and syntactic divergences. It also describes using the Nile alignment tool along with constituent parsing to improve word alignments from the training corpus. New features were added and the reranking was improved by incorporating a neural machine translation-based bilingual language model.
John Richardson - 2015 - KyotoEBMT System Description for the 2nd Workshop on Asian Translation
- 1. KyotoEBMT System Descrip3on for the 2nd Workshop on Asian Transla3on John Richardson Raj Dabre Chenhui Chu Fabien Cromières Toshiaki Nakazawa Sadao Kurohashi 1
- 2. Outline • Overview of the system • Improvements since WAT2014 • Results for WAT2015 • Conclusion 2
- 4. KyotoEBMT Overview • Example-‐Based MT paradigm • Need parallel corpus • Few language-‐specific assumpJons • sJll a few language-‐specific rules • Tree-‐to-‐Tree Machine TranslaJon • Maybe the least commonly used variant of x-‐to-‐x • SensiJve to parsing quality of both source and target languages • Maximize the chances of preserving informaJon • Dependency trees • Less commonly used than ConsJtuent trees • Most natural for Japanese • Should contain all important semanJc informaJon 4
- 5. KyotoEBMT pipeline • Somehow classic pipeline • 1-‐ Preprocessing of the parallel corpus • 2-‐ Processing of input sentence • 3-‐ Decoding/Tuning/Reranking • Tuning and reranking done with kbMira • seems to work beVer than PRO for us 5
- 6. Other specifici3es • No “phrase-‐table” • all translaJon rules computed on-‐the-‐fly for each input • cons: • possibly slower (but not so slow) • compuJng significance/ sparse features more complicated • pros: • full-‐context available for compuJng features • no limit on the size of matched rules • possibility to output perfect translaJon when input is very similar to an example • “Flexible” translaJon rules • OpJonal words • AlternaJve inserJon posiJons • Decoder can process flexible rules more efficiently than a long list of alternaJve rules • some “flexible rules” may actually encode >millions of “standard rules” 6
- 7. Flexible Rules Extracted on-‐the-‐fly 7 プラスチック (plasJc) は 石油 から 製造 さ れる 例えば(for example) the 水素 は 現在 天然ガス や 石油 から 製造 さ れる hydrogen is produced from natural gas and petroleum at present raw X(plasJc) is petroleum produced from Y(for example) ? Y(for example) Y(for example) raw* opJonal word Input: Matched Example: Flexible translaJon rule created on-‐the-‐fly: X: Simple case (X has an equivalent in the source example) Y: ambiguous inserJon posiJon “raw”: null-‐aligned -‐> opJonal (encode many translaJon opJons at once)
- 8. Other specifici3es • No “phrase-‐table” • all translaJon rules computed on-‐the-‐fly for each input • cons: • possibly slower (but not so slow) • compuJng significance/ sparse features more complicated • pros: • full-‐context available for compuJng features • no limit on the size of matched rules • possibility to output perfect translaJon when input is very similar to an example • “Flexible” translaJon rules • OpJonal words • AlternaJve inserJon posiJons • Decoder can process flexible rules more efficiently than a long list of alternaJve rules • some “flexible rules” may actually encode >millions of “standard rules” 8
- 10. KyotoEBMT improvements • Our system is very sensiJve to parsing errors • ConJnuous improvements to • Juman • KNP • SKP • Added support for parse forests • (compact representaJons) Forest parses 10
- 11. Forest Input • A parJal soluJon to the issues of Tree-‐to-‐Tree MT • Can help with parsing errors • Can help with syntacJc divergences • In WAT2014, • we used 20-‐best input parses • n-‐best list of all inputs merged and reranked • Now, with forest: • an exponenJal number of input parses can be encoded • the selecJon of parses is done during decoding 11
- 12. KyotoEBMT improvements • System is also very sensiJve to alignment errors • We used to correct alignments by using dependency trees (Nakazawa and Kurohashi, 2012) • Now we further improve them with Nile (Riesa et al., 2011) Forest parses 12
- 13. Alignment Improvements • Used Nile (Riesa et al., 2011) to improve the alignment • As suggested by (Neubig and Duh, 2014) • Require us to parse into consJtuent trees as well • Ckylark parser for Japanese (Oda+, 2015) • Berkeley Parser for Chinese/English • Nile becomes the third element of an alignment pipeline Giza++ (Nakazawa and Kurohashi, 2012) Nile with dependency trees with consJtuent trees 13 F:0.63 F:0.69 F:0.75 JC alignment F -‐> (Giza++ / Nile only -‐> F:0.70)
- 14. KyotoEBMT improvements • Many small improvements • BeVer handling of flexible rules • Bug fixes • 10 new features • alignment score • context similarity score based on word2vec vectors • … Forest parses 14
- 15. KyotoEBMT improvements • Reranking • Previously used features: • 7-‐gram language model • RNNLM language model • Now also using a Neural MT based bilingual Language Model Forest parses 15
- 16. Bilingual Neural Network Language Model • Combine Neural MT with EBMT • We use the state-‐of-‐the-‐art model described by (Bahdanau et al., 2015) • Model seen as a Language Model condiJonalized on the input • Remarks: • Processing Japanese and Chinese as sequences of characters gave good results • No need to limit vocabulary (~4000/6000 characters for J/C) • Avoid segmentaJon issues • Faster training • Neural MT models alone produced bad translaJons • eg. Character BLEU for C-‐>J almost half that of KyotoEBMT • Reranking performances saturates before MT performances 16 0 10 20 30 40 50 60 10 20 40 Reranked BLEU/ NeuralMT char-‐BLEU vs Epochs for J-‐>C Neural MT cBLEU reranked BLEU KyotoEBMT cBLEU
- 17. KyotoEBMT improvements • Improved working methods (that maVers!) • automaJc nightly tesJng for variaJons in BLEU/ asserJon errors/ memory leaks • Overall improvements across all the pipeline • EsJmaJng the global contribuJon of each element is tough, but here are the final results, … Forest parses 17
- 18. Results 18
- 19. Results for WAT2015 Reranking BLEU RIBES HUMAN J-‐>E NO 21.31 (+0.71) 70.65 (+0.53) 16.50 YES 22.89 (+1.82) 72.46 (+2.56) 32.50 E-‐>J NO 30.69 (+0.92) 76.78 (+1.57) 40.50 YES 33.06 (+1.97) 78.95 (+2.99) 51.00 J-‐>C NO 29.99 (+2.78) 80.71 (+1.58) 16.00 YES 31.40 (+3.83) 82.70 (+3.87) 12.50 C-‐>J NO 36.30 (+2.73) 81.97 (+1.87) 16.75 YES 38.53 (+3.78) 84.07 (+3.81) 18.50 19
- 20. Results for WAT2015 Reranking BLEU RIBES HUMAN J-‐>E NO 21.31 (+0.71) 70.65 (+0.53) 16.50 YES 22.89 (+1.82) 72.46 (+2.56) 32.50 E-‐>J NO 30.69 (+0.92) 76.78 (+1.57) 40.50 YES 33.06 (+1.97) 78.95 (+2.99) 51.00 J-‐>C NO 29.99 (+2.78) 80.71 (+1.58) 16.00 YES 31.40 (+3.83) 82.70 (+3.87) 12.50 C-‐>J NO 36.30 (+2.73) 81.97 (+1.87) 16.75 YES 38.53 (+3.78) 84.07 (+3.81) 18.50 +0.71 +1.82 +0.92 +1.97 +2.78 +3.83 +2.73 +3.78 The various improvements lead to good changes in BLEU. Almost +4 BLEU for the JC/CJ 20
- 21. Results for WAT2015 Reranking BLEU RIBES HUMAN J-‐>E NO 21.31 (+0.71) 70.65 (+0.53) 16.50 YES 22.89 (+1.82) 72.46 (+2.56) 32.50 E-‐>J NO 30.69 (+0.92) 76.78 (+1.57) 40.50 YES 33.06 (+1.97) 78.95 (+2.99) 51.00 J-‐>C NO 29.99 (+2.78) 80.71 (+1.58) 16.00 YES 31.40 (+3.83) 82.70 (+3.87) 12.50 C-‐>J NO 36.30 (+2.73) 81.97 (+1.87) 16.75 YES 38.53 (+3.78) 84.07 (+3.81) 18.50 Mystery! Only for J-‐>C, we find that reranking decreased Human EvaluaJon score. (While sJll improving BLEU/RIBES) 21
- 22. Results for WAT2015 Reranking BLEU RIBES HUMAN J-‐>E NO 21.31 (+0.71) 70.65 (+0.53) 16.50 YES 22.89 (+1.82) 72.46 (+2.56) 32.50 E-‐>J NO 30.69 (+0.92) 76.78 (+1.57) 40.50 YES 33.06 (+1.97) 78.95 (+2.99) 51.00 J-‐>C NO 29.99 (+2.78) 80.71 (+1.58) 16.00 YES 31.40 (+3.83) 82.70 (+3.87) 12.50 C-‐>J NO 36.30 (+2.73) 81.97 (+1.87) 16.75 YES 38.53 (+3.78) 84.07 (+3.81) 18.50 22
- 23. Code is available and Open-‐sourced • Version 1.0 released • 1 year arer version 0.1 • 2 years arer development started • Downloadable at: hVp://nlp.ist.i.kyoto-‐u.ac.jp/kyotoebmt/ • GPL Licence 23
- 24. Conclusion • KyotoEBMT is a (Dependency) Tree-‐to-‐Tree MT system with state-‐of-‐ the-‐art results • Open-‐sourced (hVp://nlp.ist.i.kyoto-‐u.ac.jp/kyotoebmt/) • Improvements across the whole pipeline lead us to close to +4 BLEU improvements • Some future works: • Make more use of the target structure • Use of deep learning features in the decoder • eg. as in (Devlin et al., 2014) • … 24
- 25. Thank you! 25