A statistical approach to machine translation
Download as PPTX, PDF0 likes820 views
This document describes a statistical approach to machine translation. It discusses using probability to determine the most likely source sentence S given a target sentence T. It presents methods for computing language model probabilities, translation probabilities, and searching for the optimal S. Two pilot experiments are described to estimate parameters for the translation model from bilingual text data. Evaluation of the second experiment showed the decoded sentences were either exact, alternate, different, wrong or ungrammatical compared to the reference translations.
A statistical approach to machine translation
- 1. A Statistical Approach to Machine Translation Peter F. Brown, John Cocke, Stephen A. Della Pietra, Vincent J. Della Pietra, Fredrick Jeinek, John D. Lafferty, Robert L. Mercer, and Paul S. Roossin Published in: · Journal Computational Linguistics archive Volume 16 Issue 2, June 1990 Pages 79-85 MIT Press Cambridge, MA, USA
- 2. Introduction • (S, T): Every pair of sentences • Pr(T|S): probability of T when presented with S • Given a sentence T in the target language, seek the sentence S from which the translator produced T. • Chance of error is minimized by choosing the most probable sentence S given T – In other words, to maximize Pr(S|T) – Pr(S|T) = Pr(S) Pr(T|S) / Pr(T) : Bayes’ theorem – Pr(S): language model probability of S – Pr(T|S): translation probability of T given S
- 3. • Statistical Translation System requires: – A method for computing language model probabilities – A method for computing translation probabilities – A method for searching among possible source sentences S for the one that gives the greatest value for Pr(S)Pr(T|S)
- 4. The Language Model • a word string S1, S2, … Sn • Pr(S1, S2, … Sn) = Pr(S1) Pr(S2|S1)…Pr(Sn|S1S2…Sn-1) • Given a history S1S2…Sj-1, you must know the probability of object word Sj • Histories are too long, probability parameters cannot be separated • To reduce parameters: – Categorize each history into same class – Probability of object depends on the history in same class
- 5. The Translation Model • A word can be translated into more than one word – Fertility: num. of T words that an S word produces • Notation for alignment: – (Jean aime Marie | John(1) loves(2) Mary(3) ) • ( T | S) • Num. in S words are positions in T • Computing the probability of the alignment: – (Le chien est battu par Jean | John(6) does beat(3, 4) the(1) dog(2))
- 6. The Translation Model • In English adjectives precede nouns, in French adj. follows nouns – Distortion: T word appears far from S word in alignment • Distortion probability: – Pr(i|j, l) • i: a target position • J: a source position • l: the target length
- 7. Searching • Searching for the sentence S to maximize Pr(S)Pr(T|S) • Uses stack search – A list of partial alignment hypothesis – (Jean aime Marie| * ) • *: a place holder for unknown sequence of S – In iteration, extends most promising entries and adds to its hypothesis – Ends when complete alignment is the most promising significantly
- 8. Parameter Estimation • Both LM and TM have many parameters • To estimate, needs pairs of translations • For this experiment, they used Canadian parliament’s records translated in English/French – Param. & LM/TM can be estimated from this
- 9. Two Pilot Experiments • First experiment: to estimate params for TM • 9000 most common words in English/French from Hansard data
- 10. Two Pilot Experiment • Second experiment: – French to English – 1,000 most frequently used English words – 1,700 most frequently used French words ( covered by the 1k English words) – Estimated 17 million parameters of translation model from 117,000 pairs of sentences – Est. bigram language model from 570,000 sentences from the English part of Hansard – Evaluation: • Exact: Decoded sentence was exactly the same • Alternate: same meaning, slightly different words • Different: not covey the same meaning as the translation • Wrong: makes a sense but not interpreted • Ungrammatical: grammatically deficient