English to Bangla Phrase-Based Statistical Machine Translation

Size: px

Start display at page:

Download "English to Bangla Phrase-Based Statistical Machine Translation"

Darcy Johns
7 years ago
Views:

1 Master Thesis English to Bangla Phrase-Based Statistical Machine Translation Md. Zahurul Islam Supervisors: Dr. Andreas Eisele PD. Dr. Valia Kordoni Prof. Hans Uszkoreit Department of Computational linguistics Saarland University Germany Dr. Jörg Tiedemann Department of Humanities Computing University of Groningen The Netherlands & Submitted to Department of Computational linguistics at Saarland University Erasmus Mundus European Masters Program in Language and Communication Technologies (LCT) August 2009

2 Master Thesis English to Bangla Phrase-Based Statistical Machine Translation Md. Zahurul Islam Supervisors: Dr. Jörg Tiedemann Department of Humanities Computing University of Groningen The Netherlands & Dr. Andreas Eisele DFKI GmbH, Language Technology Lab and Department of Computational linguistics Saarland University Germany Submitted to Faculty of Arts at University of Groningen Research Master Linguistics Erasmus Mundus European Masters Program in Language and Communication Technologies (LCT) August 2009

12 S > NP +VP

13 !"#$ %&#'!"(

14 )*#+,-! Pr(w) Pr(e w) Pr(j e) Pr(k j) Pr(o k)

15 Pr(w).P r (e w).p r (j e).p r (k j).p r (o k) Pr(w)

16 e I 1 = e 1... e i... e I f1 J = f 1... f j... f J ˆf ĵ 1 = argmin J,f J 1 J f j 1 ) Pr (f J 1 e I 1.L (f J 1,f J 1 ) ( ) L f1 J,f J 1 f1 J f J 1 ) L (f 1 J,f J = 1 { 0 f J 1 = f J 1 1 ) =1 δ (f 1 J,f J 1

17 ( δ f J 1,f J 1 ) ˆf Ĵ1 = argmax { Pr ( f1 J e I )} 1 f1 J e I 1 Pr ( f1 J e1) I Pr ( f1 J e I ) Pr ( ) ( ) f J 1 = 1 Pr e I 1 f1 J Pr ( ) e I 1 Pr ( e I 1) e I 1 argmax { Pr ( f1 J e I { ( ) ( 1)} = argmax Pr f J 1 Pr e I 1 f1 J )} Pr ( ) f1 J Pr ( ) ( ) e I 1 f1 J Pr e I 1 f1 J Pr ( ) f1 J

18 f1 J e I 1 e 1,e 2... e I e I 1 f1 J φ ( f1 J e1) I f1 J e I 1 d (a j b j 1 ) a j i th e i b j 1 i 1 th e i 1 d (a j b j 1 )=α aj bj 1 1 Pr ( f1 J e I ) I,J 1 = φ (f j,e i ) d (a j b j 1 ) i,j=1 α

19 e = e I 1 f = f1 J ˆf = argmax f {Pr(f e)} ˆf = argmax f {Pr(e f).p r (f)} Pr(e f) Pr(f) Pr(e f) = a Pr(e, a f) a a = a I 1 = a 1... a i... a I,a i {0,... J} a i =0 p (a i a i 1,J) a i a j 1 p (( a i a i 1 ) J)

20 { ( ˆF = argmax f Bangla Pr e I 1 f1 J ) ( )}.P r f J 1 n f (n) A f1 J e I 1 S = ( ) e I 1,f1 J Cost ( e I 1,f1 J ) = φ (e i,f j ) d (a j b j 1 ) Pr ( f1 J ) s f J 1 A n n

22 ./0# ,67,893: )"#;<'* =*/.* B7,34: -)C*#$' DE*#F.' 2>GA,H7,I9A J7,669,34A: K#LMN* 27?,3,869,77: O*F! 2>J7,678: #$#.#.#& 'PQ 2I9,339,339,RR9 A4,S9: TU*V $D*.*WF 2R64,3J7A,84 I4J7,3J7X46: YQ' 25J9A: TZ*![Q -1\*. 2R64H7,84A,869 8G]73: s = s I 1 = s 1... s i... s I t = t J 1 = t 1... t j... t J { ( ˆt = argmax t J 1 Pr t J 1 s I )} 1 { ( ˆt = argmax t J 1 Pr s I 1 t J )} ( ) 1.P r t J 1 N gram Pr ( t J 1 ) Pr ( s I 1 t J 1 )

23 %/W.;*/^F %/W _,&*/F #'/Y &/F 'PQF _,&*/F )( `F #'/Y.aE*F &/F.#a.$*.,-! )*#+ )*#+/!,b.ae*.ae*b c*;* defq c*/;efq )f" gh.) )f"h.)!"#$ `)i %#$ %$F*

30 `;#1 `j^ %&'*F %('W! K#f;*F 'E*ME )E).* 'E*ME )E).* )*fe!*$k^;o*/) `;#1 '!"' U!*iU(.*f*Fl #$m K/n*)F g#'uu `( 'b #D.*/) ;#o' `(#1 #D/./) p#' `)i ;F/! -.1* M*b#' M/qr &*/F's D/b/t T!E*uE*! -M `1* ;*v'zw* ;*v'zw* -.x#p %F) #;iy$ -.x#p %F) $z!". $#FU*. -;*{*#F;* -;*{*#F;* # ;* Q.

31 K#=. Z' #O;*F. )*', #;,$"' `;#1 U&F `j/^* %&'*F %('W! K#f;*F 'E*ME )E).* 'E*ME )E).* )*fe!*$k^;o*/) `;#1 '!"' U!*iU(.*f*Fl #$m Z' #O;*F. K/n*)F g#'uu `( 'b #D.*/) ;#o' `(#1 #D/./) p#' `)i ;F/! -.1* M*b#' M/qr &*/F's $*$y D$P'ZP D/b/t T!E*uE*! -M -}*^;~ `1* Ä #}q Äs )*', #;,$"'

32 &#F)!</'F.kY'* -^* -& ;#$U' `F!z!Qb T#!/)P' 2uÅ Ç É Ñ: %$F*.$Ö T*.#Ü;!qE #)/áàl ;/F#t `)i ;$ $Z"#FF TO*/)F #D.*) %/F* O*/^*O*/) ;F/! &*F*F Z'E K#=. =F 'E*U'*^ {E*#1{;~ )* Z*!Qb &#F.iuE*' P=!/FF./Ü `;/â ;*Z ;/F#t s ä -Fj/^U'~ )* %(';*'"/' Wz#D! Z*!Qb 'ke'!$ $Z"FQF.iã* O*/^*O*/) ;*M<;F D/b/t `)i #'/b*w;*fq/pf W#Få*U 'E*U'*^ #$#'$*$ É/bZ -Fj/^U'~ )* Z*!Qb 'ke'!$ $Z"FQ %(';*'"/'F ;*M<;*#F!*F )E*&*/F

33 `;#1 `$' #'/P<#U;* %&'*F %('W! K#f;*F K#=/. 'E*ME -^'/P/'F #D/./) `( ;#o' 'b `#1 p#' `)i çf" -M OQ#! TPU<' ;F* -.1* M/qr D/! &*/F M*/! K#O/M*W#1F $Q$*i.* ;F* M*bs %/t -M u*å)pdl )l<'* ;F* D/é `1* ;F* D/b/t `;#1 s #'/àf #'/àf

34 `;#1 -U*&&*F `$' #'/P<#U;* %&'*F %('W! K#f;*F K#=/. 'E*ME -^'/P/'F #D/./) `( ;#o' 'b `#1 p#' `)i çf" -M OQ#! TPU<' ;F* -.1* M/qr D/! &*/F M*/! K#O/M*W#1F $Q$*i.* ;F* M*b s $*D$"P %D/$#P'*Z*P %/t -M )l<'* ;F* D/é `1* ;F* D/^*;*{ `;#1 Ä#$qÄs #;,-$*' #'/àf

35 `;#1 è*f `F #'/P<#U;* %&'*F %('W! K#f;*F :K#=/. 'E*ME )*#lze `#1 ;#o' 'b `)i `1* -;)^ `( Wê/W*/^F ë$#; `1* D/! &*/F M/qr K#O/M*W#1F $Q$*i.* ;F*F Z'E s $*D$"P %D/$#P'*Z*P %/t -M )l<'* ;F* D/é `1* ;F* D/^*;*{ `;#1 Ä #$q Ä s #;,-$*' #'/àf -;*' -;*' -í/â T/b*Z'Qb D/! &*/F -;*/1< -M -./1ì -&/! &*/F' s %&'*F -0!*/PF -M K#!#Fî -)#'#=1 -;*'.$/bf Çï #P' ;#^Ü,Kñ= É&F -)UQF O*W $*^)*DQ.P.E/PF #)#0 ;/F.F*.#F KE*/.*#./bU' s

36 n

37 ACC = 1 N N {1 r i = c i, 0 } i=1 T op J = 1 N N {1 c i,j c i,j = ri, 0 } i=1 LCS(c, r) = 1/2 (length (c)+length (r) editdistance (c, r)) i th R i = LCS (c i,r i ) length (r i ) P i = LCS (c i,r i ) length (c i ) F i =2 R i P i R i + P i

41 `;#1 &#FY*^'* ;/F `;#1 è*f `F #'/P<#U;* %&'*F %(#' K#f;*F %&'*F %('W! K#f;*F gó^ )*#l/zef K#=. K#=/. 'E*ME )*#lze `(#1 `(#1 #D/./) M! ;#o' Uò ;/F '* `)i `(#1F O/MNF ;*Fl $*â #)àmn #ôf.öõ ;F/! M/qrs u*å)pdl )l<'* ;F* D/é `(#1 `;#1 KúQ;*F ;/F/t #'/àf #;,-$*' #'/àf #;t" ù1'* `(#1/! #)àmn -&/! -;*1</! -M/! T/MN*Z'QMN $Q$*i.* ;F/! -&/F#t^s s `(#1.F*.#F.#$#!F.P.)z/ûF $/fe O*^.)<*/&í* `;#1 Çï #P' &M<*MN;*/^F -O*î*/! K#!#Fî."#)f* TÖ*) -PMN #)#0 ;/Fs `#1 ;#o' 'b `)i `1* -;)^ `( Wê/W*/^F ë$#; `1* D/! &*/F M/qr K#O/M*W#1F $Q$*i.* ;F*F Z'E s $*D$"P %D/$#P'*Z*P u*å)pdl -M )l<'* ;F* D/é `1* ;F* D/^*;*{ `;#1 Ä #$q Ä s -;*' -;*' -í/â T/b*Z'Qb D/! &*/F -;*/1< -M -./1ì -&/! &*/F' %&'*F -0!*/PF -M K#!#Fî -)#'#=1 -;*'.$/bf Çï #P' ;#^Ü,Kñ= É&F -)UQF O*W $*^)*DQ.P.E/PF #)#0 ;/F.F*.#F KE*/.*#./bU' s

43 φ (e f) `F -;*' -;*'#1 )*(/F #)#0É `F -;*' -;*'#1 )*(/F #)#0É `F -;*' -;*'#1 )*(/F #)#0É `F -;*' -;*'#1 )*(/F #)#0É ;F* #)#0 D/! &*/F 0.5 D/! &*/F )*#l#ze; '*$ )E)D*F ;/F ü

1.- L a m e j o r o p c ió n e s c l o na r e l d i s co ( s e e x p li c a r á d es p u é s ).

1.- L a m e j o r o p c ió n e s c l o na r e l d i s co ( s e e x p li c a r á d es p u é s ). PROCEDIMIENTO DE RECUPERACION Y COPIAS DE SEGURIDAD DEL CORTAFUEGOS LINUX P ar a p od e r re c u p e ra r nu e s t r o c o rt a f u e go s an t e un d es a s t r e ( r ot u r a d e l di s c o o d e l a