Semantics in Statistical Machine Translation

Transcription

1 Semantics in Statistical Machine Translation Mihael Arcan DERI, NUI Galway Copyright All rights reserved.

2 Overview 1. Statistical Machine Translation (SMT) 2. Translations in SMT 3. Ambiguity in SMT 4. Experiment Dice coefficient 5. Experiment Google N-Gram Database 6. Reordering the translations 7. Evaluation Bleu score 8. Future Work and Conclusion 2

3 Statistical Machine Translation (SMT) What is statistical machine translation Translation model Ensures adequacy of a language Language model Ensures the fluency of the language (5-grams) 3

4 Translations in SMT Problems that we have to handle Data Sparseness Ambiguity Data Sparseness Source Language Unsere Wohnung ist gegen Wasserschäden versichert. Target Language our accommodation shall be insured against Wasserschäden. Source Language Ja, ich vertrete die selbe Ansicht. Target Language Yes, I represents the selbe view. 4

5 Ambiguity in Machine Translation Source Langauge In dieser Woche muss er seinen Kollegen vertreten. Reference This week he has to fill in for his colleague. Google MT This week he has represented his colleagues. Monnet Moses this week, he needs to his colleagues. Source Langauge Frau Müller vertritt unsere Firma auf der Messe in Muenchen. Reference Ms Müller represents our company at the trade fair in Munich. Google MT Mrs. Mueller represents our company at the fair in Munich. Monnet Moses mrs müller is our firm on the fair in munich. Monnet Moses (#13) mrs müller represents our firm on the fair in munich. Source Langauge Nach der Sitzung mussten sie sich die Beine vertreten. Reference After the meeting they had to strech their legs. Google MT After the session she had to stretch their legs. Monnet Moses after the meeting, they were the legs. 5

6 Semantics - Dice coefficient Dice coefficient Example: nacht vs. night [ni,ig,gh,ht] vs. [na,ac,ch,ht] s = (2 * 1) / (4 + 4) = 0.25 Would you like to insure your house against fire too? vs. Would you like to insure your house against fire? s = (2 * 9) / (9 + 10) =

7 Semantics - Dice coefficient The Google Web 1T 5-Gram Database SQLite Index & Web Interface

8 our housing is against water damage assured Google n-gram (assure) our Google n-gram (insure) against, damage do you wish to assure the house also against fire? Google n-gram (assure) to, you, wish the Google n-gram (insure) to, house, against, you, wish

9 How the translation order changed Unsere Wohnung ist gegen Wasserschäden versichert. our housing is against water damage assured our house is against water damage insured our housing is against water damage insured our house is opposed to water damage insured our housing is opposed to water damage assured our home is to water damage insured our housing is opposed to water damage insured our home against water damage insured our housing is against water damage our housing against water damage insured our home is against water damage assured our house is against water damage our home is against water damage insured our house is against water damage assured our house is against water damage assured our home is insured against water damage our house is against water damage insured our housing is opposed to water damage assurance our home to water damage insuredour housing is insured against water damage the housing is against water damage insured

10 Evaluation If we would measure if a sentence is gramatical correct, we could solve the language modeling problem Example: our home to water damage insured Evaluating translations automatically is hard many ways to say something Blue score Bilingual Evaluation Understudy Why? it achieves a high correlation with human judgements of quality

11 Evaluation of Translations Source Language Unsere Wohnung ist gegen Wasserschäden versichert. Reference Our flat is covered against water damage. Target Language (Google MT) Our home is insured against flood damage Target Language (Monnet Moses) our housing is against water damage assured Target Language (Monnet Moses + reordering) our house is against water damage insured

12 Future Work and Conclusion Conclusion size of the resources resources ambiguity can be solved with additional information Future Work structural information - Ontologies