Improving Chronological Sentence Ordering by Precedence Relation

Transcription

1 Improvn Cronolol Sntn Orrn y Prn Rlton Nok OKAZAKI T Unvrsty o Tokyo Hono, Bunkyo-ku, Tokyo , Jpn okzk@mvtu-tokyojp Yutk MATSUO AIST Aom, Koto-ku, Tokyo , Jpn ymtsuo@rstojp Mtsuru ISHIZUKA T Unvrsty o Tokyo Hono, Bunkyo-ku, Tokyo , Jpn szuk@mvtu-tokyojp Astrt It s nssry to n propr rrnmnt o sntns n orr to nrt wll-ornz summry rom multpl oumnts In ts ppr w sr n ppro to ornt sntn orrn or summrzn nwsppr rtls Sn tr s no urnt tt ronolol orrn o xtrt sntns, w s wly us y onvntonl summrzton systm, rrns sntn n prsuppos normton o t sntn, w mprov ronolol orrn y rsolvn ntnt sntns o rrn sntns Comnn t rnmnt lortm wt topl smntton n ronolol orrn, w rss our xprmnt to tst t tvnss o t propos mto T rsults rvl tt t propos mto mprovs ronolol sntn orrn 1 Introuton T rowt o omputrz oumnts nls us to n rlvnt normton sly own to tnolol vns n Inormton Rtrvl Altou t s onvnnt tt w n otn rt numr o oumnts wt sr nn, ts stuton lso prsnts t normton polluton prolm: Wo s wlln to tk t tous urn o rn ll tos txt oumnts? Automt txt summrzton (Mn, 2001), s on soluton to t prolm, provn usrs wt onns vrson o t ornl txt Most xstn summrzton systms mk us o sntn or prrp xtrton, w ns snnt txtul smnts n sour oumnts, n ompl tm n summry Atr w slt snnt sntns s mtrl or summry, w must n propr rrnmnt o t sntns n t sntn y ltn unnssry prts or nsrtn nssry xprssons Altou tr s n rt l o rsr on xtrton sn t rly st o nturl lnu prossn (Lun, 1958), rsr on post-prossn o utomt summrzton s rltvly smll n numr It s ssntl to py ttnton to sntn orrn n s o mult-oumnt summrzton Sntn poston n t ornl oumnt, w yls oo lu to sntn rrnmnt or snl-oumnt summrzton, s not nou or mult-oumnt summrzton us w must onsr ntr-oumnt orr t t sm tm In ts ppr w propos n ppro to ornt txt struturn or summrzn nwsppr rtls W mprov ronolol orrn, w s wly us y onvntonl summrzton systm, omplmntn prsuppos normton o sntn T rst o ts ppr s ornz s ollows W rst rvw t sntn orrn prolm n prsnt our ppro to nrt n ptl orrn n t lt o orn rlton T susqunt ston (Ston 3) rsss vluton mtrs n xprmnt rsults In Ston 4 w suss utur work n onlu ts ppr 2 Sntn Orrn 21 Sntn orrn prolm Our ol s to trmn t most prol prmutton o vn sntns n to nrt wll-strutur txt Wn umn s sk to mk n rrnmnt o sntns, or s my prorm ts tsk wtout ulty just s w wrt out touts n txt Howvr, w must onsr wt omplss ts tsk sn omputrs r unwr o orr o tns y ntur Dsours orn s typ y rtorl rlton (Mnn n Tompson, 1988) n orn rlton (Hos, 1990) s o lp to ts quston Hum (Hum, 1748) lm tt qults rom w ssoton rss n y w t mn s onvy rom on to notr r tr: rsmln; ontuty n tm or pl; n us n t Tt s to sy w soul ornz txt rom r-

2 Cronolol orr ) Dolly t lon sp ws orn n 1996 ) T tr s o rnt kn n Dolly n prnnt or out v monts ) Dolly v rt to two lrn n r l Rnmnt [--] Sntns Prrr orrn Fur 1: A ronolol orrn s not nou mnt normton on t ss o topl rlvny, ronolol squn, n us-t rlton It s splly tru n sntn orrn o nwsppr rtls us w must rrn lr numr o tm-srs vnts onrnn svrl tops Brzly t l (Brzly t l, 2002) rss t prolm o sntn orrn n t ontxt o mult-oumnt summrzton n t mpt o sntn orrn on rlty o summry Ty propos two nv sntnorrn tnqus su s mjorty orrn (xmns most rqunt orrs n t ornl oumnts) n ronolol orrn (orrs sntn y t pulton t) Sown tt usn nv orrn lortms os not prou ststory orrns, Brzly t l lso nvstts trou xprmnts wt umns, ow to nty pttrns o orrns tt n mprov t lortm Bs on t xprmnts, ty propos notr lortm tt utlzs ronolol orrn wt topl smntton to sprt sntns rrrn to top rom ons rrrn to notr Lpt (Lpt, 2003) proposs notr ppro to normton orrn s on prolst mol tt ssums t prolty o ny vn sntn s trmn y ts jnt sntn n lrns onstrnts on sntn orr rom orpus o omn sp txts Lpt stmts trnstonl prolty twn sntns y som ttruts su s vrs (prn rltonsps o vrs n t orpus), nouns (ntty-s orn y kpn trk o t nouns) n pnns (strutur o sntns) 22 Improvn ronolol orrn Anst t kroun o ts stus, w propos t us o ntn sntns to rrn sntns Lt us onsr n xmpl sown n Fur 1 Tr r tr sntns,, n rom w w t n orr [--] y ronolol orrn Wn w r ts sntns n ts orr, w n sntn to norrtly poston Ts s us sntn s wrttn on t prsupposton tt t rr my know tt Dolly l In otr wors, t s mor ttn to ssum sntn to n lorton o sntn As on my sly mn, tr r som prnt sntns pror to sntn n t ornl oumnt Lk o prsupposton osurs wt sntn s syn n onuss t rrs Hn, w soul rn t ronolol orr n rvs t orr to [--], puttn sntn or sntn W sow lok rm o our orrn lortm sown n Fur 2 Gvn nn sntns not y [ ], or xmpl, t lortm vntully prous n orrn, [ ] W onsr topl smntton n ronolol orrn to unmntl to sntn orrn s wll s onvntonl orrn tnqus (Brzly t l, 2002) n mk n ttmpt to rn t orrn W rstly ronz tops n sour oumnts to sprt sntns rrrn to top rom ons rrrn to notr In Fur 2 xmpl w otn two topl smnts (lustrs) s n output rom t topl lustrn In t son ps w orr sntns o smnt y t ronolol orr I two sntns v t sm ronolol orr, w lort t orr on t ss o sntn poston n rsmln rlton Fnlly, w rn orrn y rsolvn ntnt sntns n output t nl orrn In t rst o ts ston w v tl srpton o ps 23 Topl lustrn T rst tsk s to torz sntns y tr tops W ssum nwsppr rtl to wrttn out on top Hn, to lssy tops n sntns, w v only to lssy rtls

3 Unorr sntns Topl lustrn y oumnts Clustr #1 Clustr #2 Cronolol orrn wt rsmln rlton Orrn rnmnt y prn rlton Orr sntns Fur 2: T outln o t orrn lortm y tr tops Gvn l rtls n w oun m kns o trms n t rtls Lt D oumnt-trm mtrx (l m), wos lmnt D j rprsnts rquny o trm #j n oumnt #, W us D to not trm vtor (-omponnt row vtor) o oumnt # Atr msurn stn or ssmlrty twn two rtls #x n #y: stn(d x, D y ) = 1 D x D y D x D y, (1) w pply t nrst nor mto (Covr n Hrt, 1967) to mr pr o lustrs wn tr mnmum stn s lowr tn vn prmtr α = 03 (trmn mprlly) At lst w lssy sntns orn to topl lustrs, ssumn tt sntn n oumnt lonn to lustr lso lons to t sm lustr 24 Cronolol orrn It s ult or omputrs to n rsmln or us-t rlton twn two pnomn wl w o not v onlusv vn wtr pr o sntns tr rtrrly rom multpl oumnts s som rlton A nwsppr usully ls wt novl vnts tt v ourr sn t lst pulton Hn, pulton t (tm) o rtl turns out to oo stmtor o rsmln rlton (, w osrv trn or srs o rlvnt vnts n tm pro), ontuty n tm, n us-t rlton (, n vnt ours s rsult o prvous vnts) Altou rsolvn tmporl xprssons n sntns (, ystry, t nxt yr, t) (Mn n Wlson, 2000; Mn t l, 2003) my v mor prs stmton o ts rltons, t s not n sy tsk For ts rson w orr sntns o smnt (lustr) y t ronolol ' Artl #1 Artl #2 Artl #3 ronolol orr Fur 3: Bkroun o orrn rnmnt y prn rlton orr, ssnn tm stmp or sntn y ts pulton t (, t t wn t rtl ws wrttn) Wn tr r sntns vn t sm tm stmp, w lort t orr on t ss o sntn poston n sntn onntvty W rstor n ornl orrn two sntns v t sm tm stmp n lon to t sm rtl I sntns v t sm tm stmp n r not rom t sm rtl, w rrn sntn w s mor smlr to prvously orr sntns to ssur sntn onntvty 25 Orrn rnmnt y prn rlton Atr w otn n orrn o topl smnt y ronolol orrn, w mprov t s sown n Fur 1 s on ntn sntns Fur 3 sows t kroun o orrn rnmnt y prn rlton Just s n t xmpl n Fur 1, w v tr sntns,, n n ronolol orr At rst w t sntn out o t sntns n k ts ntnt sntns Sn tt tr r no sntns pror to sntn n rtl #1, w pt to put sntn r Tn w t sntn out o rmnn sntns n k ts ntnt sntns W n svrl sntns or sntn n rtl #2 ts tm Grspn wt t ntnt sntns r syn, w onrm rst o ll wtr wt ty r syn s mnton y prvously rrn sntns (, sntn ) I t s mnton, w put sntn r n xtn t orrn to [-] Otrws, w sr susttuton or wt t prn sntns r syn rom t rmnn sntns (, sntn n ts xmpl) In t Fur 3 xmpl, w n out tt sntn s not rrrn to wt sntn s syn ut sntn s pproxmtly rrrn to tt

4 Strt (1) 0 (2) 0 (5) 0 (3)7 6 (4) 2 1 (1) (2) (3) (4) (5) No prnt sntns or sntn Coos No prnt sntns or sntn Coos Tr r prnt sntns or sntn Sr sortst pt rom to n W oun sntn to t losst to t prnt sntns o Sr sortst pt rom to n No prnt sntns or Coos W n pt rom to n v s t sortst 8 (6) (7) 4 (8) 0 (6) (7) (8) Tr r prnt sntns or sntn Sr sortst pt rom to,, n W n t rt pt rom to s t sortst Coos t rst, sntn En T rn orrn s Fur 4: Orrn rnmnt y prn rlton s sortst pt prolm Puttn sntn or, w nlly t t rn orrn [--] Supposn tt sntn mntons smlr normton s ut xprsss mor tn, t s notn unusul tt n xtrton mto os not oos sntn ut sntn Bus mto or mult-oumnt summrzton (, MMR (Cronll n Golstn, 1998)) mks ort to qur normton ovr n rus runnt normton t t sm tm, t s qut nturl tt t mto os not oos ot sntn n n trms o runny n prrs sntn s n trms o normton ovr Fur 4 llustrts ow t lortm rns vn ronolol orrn [-----] W n stn s ssmlrty vlu o prnt normton o sntn Wn sntn s ntnt sntns n tr ontnt s not mnton y prvously rrn sntns, ts stn wll Wn sntn s no prnt sntns, w n t stn to 0 In t xmpl sown n Fur 4 xmpl w o not n poston o sntns n us ty o not v prnt sntns (, ty r l sntns) On t otr n, sntn s som prnt sntns n ts ornl oumnt Prprn trm vtor o t prnt sntns, w lult ow mu t prnt ontnt s ovr y otr sntns usn stn n ov In Fur 4 xmpl t stn rom sntn n to s (stn = 07) W sr sortst pt rom sntn to sntns n y strst sr n orr to n sutl sntns or sntn Gvn tt sntn n Fur 4 srs smlr ontnt s t prnt sntns o sntn n s l sntn, w tr t sortst pt rom sntn to sntns n v sntn W xtn t rsultnt orrn to [---], nsrtn sntn or sntn Tn w onsr sntn, w s not l sntn n (stn = 04) Prprn trm vtor o t prnt sntns o sntn, w sr sortst pt rom sntn to sntns,,, n T sr rsult sows tt w soul lv sntn ts tm us t prnt ontnt sms to sr n sntns,,, n ttr tn In ts wy w t t nl orrn, [-----]

5 3 Evluton In ts ston w sr our xprmnt to tst t tvnss o t propos mto An orrn to vlut: ( s 5, s 6, s 7, s 8, s 1, s 2, s 9, s 3, s 4 ) Corrton y mov oprton 31 Exprmnt n vluton mtrs W onut n xprmnt o sntn orrn trou mult-oumnt summrzton to tst t tvnss o t propos mto W utlz t TSC-3 (Hro t l, to ppr n 2004) tst ollton, w onssts o 30 sts o mult-oumnt summrzton tsks For mor normton out TSC-3 tsk, s t worksop prons Prormn n mportnt sntn xtrton (Okzk t l, to ppr n 2004) up to t sp numr o sntns (pproxmtly 10% o summrzton rt), w m mtrl or summry (, xtrt sntns) or tsk W orr t sntns y sx mtos: umn-m orrn (HO) s t st nor; rnom orrn (RO) s t lowst nor; ronolol orrn (CO) (, ps 2 only); ronolol orrn wt topl smntton (COT) (, pss 1 n 2); propos mto wtout topl smntton (PO) (, pss 2 n 3); n propos mto wt topl smntton (POT)) W sk umn jus to vlut sntn orrn o ts summrs T rst vluton tsk s sujtv rn wr umn ju mrks n orrn o summry sntns on sl o 4: 4 (prt), 3 (ptl), 2 (poor), n 1 (unptl) W v lr rtron o sorn to t jus s ollows A prt summry s txt tt w nnot mprov ny urtr y r-orrn An ptl summry s on tt mks sns n s unnssry to rvs vn tou tr my som room or mprovmnt n trms o rlty A poor summry s on tt loss tr o t story t som pls n rqurs mnor mnmnt to rn t up to t ptl lvl An unptl summry s on tt lvs mu to mprov n rqurs ovrll rstruturn rtr tn prtl rvson Atonlly, w norm t jus tt summrs wr m o t sm st o xtrt sntns n only sntn orrn m rns twn t summrs n orr to vo ny sturn n rtn In ton to t rtn, t s usul tt w xmn ow los n orrn s to n ptl on wn t orrn s rr s poor Consrn tt svrl sntn-orrn pttrns r ptl or vn summry, w T orrt orrn: ( s 5, s 6, s 7, s 9, s 2, s 8, s 1, s 3, s 4 ) A ju s suppos to sow ow to mprov n orrn T ju's rn s ntrupt or t ponts mrk wt lk rls Fur 5: Corrton o n orrn tnk tt t s vlul to msur t r o orrton us ts mtr vrtully rqurs umn orrtor to prpr orrt nswr or orrn n s or r mn Tror, umn ju s suppos to llustrt ow to mprov n orrn o summry wn or s mrks t summry wt poor n t rtn tsk W rstrt ppll oprtons o orrton to mov oprton so s to kp t mnmum orrton o t orrn W n mov oprton r s rmovn sntn n nsrtn t sntn nto n pproprt pl (s Fur 5) Supposn sntn orrn to rnk, w n lult rnk orrlton ont o prmutton o n orrn π n prmutton o t rrn orrn σ Lt {s 1,, s n } st o summry sntns nt wt nx numrs rom 1 to n W n prmutton π S n to not n orrn o sntns wr π() rprsnts n orr o sntn s Smlrly, w n prmutton σ S n to not t orrt orrn For xmpl, t π n σ n Fur 5 wll : ( π = ( σ = ), (2) ) (3) Sprmn s rnk orrlton τ s (π, σ) n Knll s rnk orrlton τ k (π, σ) r known s mous rnk orrlton mtrs 6 τ s (π, σ) = 1 n(n + 1)(n 1) τ k (π, σ) = n 1 1 n(n 1)/2 n (π() σ()) 2 =1 (4) n =1 j=+1 sn(π(j) π()) sn(σ(j) σ()), (5)

6 RO CO COT PO POT HO Tl 1: Dstruton o rtn sor o orrns n prnt urs wr sn(x) = 1 or x > 0 n 1 otrws Ts mtrs rn rom 1 (n nvrs rnk) to 1 (n ntl rnk) v 0 ( non-orrlt rnk) In t xmpl sown n Equtons 2 n 3 w otn τ s (π, σ) = 085 n τ k (π, σ) = 072 W propos notr mtr to ssss t r o sntn ontnuty n rn, τ (π, σ): τ (π, σ) = 1 n ( ) q πσ 1 (), πσ 1 ( 1) + 1, n =1 (6) wr: π(0) = σ(0) = 0; q(x, y) = 1 wn x quls y n 0 otrws Ts mtr rns rom 0 (no ontnuty) to 1 (ntl) T summry n Fur 5 my ntrrupt ju s rn tr sntn S 7, S 1, S 2 n S 9 s or s srs nxt sntn to r Hn, w osrv our sontnuts n t orrn n lult sntn ontnuty τ (π, σ) = (9 4)/9 = Rsults Tl 1 sows struton o rtn sor o mto n prnt urs Jus mrk out 75% o umn-m orrn (HO) s tr prt or ptl wl ty rjt s mny s 95% o rnom orrn (RO) Cronolol orrn (CO) not yl ststory rsult losn tr o 63% summrs ltou CO prorm mu ttr tn RO Topl smntton oul not ontrut to orrn mprovmnt o CO s wll: COT s sltly wors tn CO Atr tkn n npt look t t lur orrns, w oun t topl lustrn not prorm wll urn ts tst W suppos t topl lustrn oul not prov t mrts wt ts tst ollton us t ollton onssts o rlvnt rtls rtrv y som qury n pols wll y umn so s not to nlu unrlt rtls to top On t otr n, t propos mto (PO) mprov ronolol orrn mu ttr tn topl smntton Not tt t sum o prt n ptl rto jump up rom 36% (CO) to 55% (PO) Ts sows t orrn rnmnt y prn rlton mprovs ronolol orrn y pusn poor orrn to n ptl lvl Tl 2 rports losnss o orrns to t orrt ons wt vr sors (AVG) n t stnr vtons (SD) o t tr mtrs τ s, τ k n τ It pprs tt vr urs sows smlr tnny to t rtn tsk wt tr msurs: HO s t st; PO s ttr tn CO; n RO s ntly t worst W ppl on-wy nlyss o vrn (ANOVA) to tst t t o our rnt mtos (RO, CO, PO n HO) ANOVA prov t t o t rnt mtos (p < 001) or tr mtrs W lso ppl Tuky tst to ompr t rn twn ts mtos Tuky tst rvl tt RO ws ntly t worst wt ll mtrs Howvr, Sprmn s rnk orrlton τ S n Knll s rnk orrlton τ k l to prov t snnt rn twn CO, PO n HO Only sntn ontnuty τ prov PO s ttr tn CO; n HO s ttr tn CO (α = 005) T Tuky tst prov tt sntn ontnuty s ttr onormty to t rtn rsults n r srmnton to mk omprson Tl 3 sows losnss o orrns to ons m y umn (ll rsults o HO soul 1 y nssty) Altou w oun RO s lrly t worst s wll s otr rsults, w nnot n t snnt rn twn CO, PO, n HO wt ll mtrs Ts rsult prsnts to t ulty o utomt vluton y prprn on orrt orrn 4 Conlusons In ts ppr w sr our ppro to ornt sntn orrn or summrzn nwsppr rtls W onut n xprmnt o sntn orrn trou mult-oumnt summrzton T propos mto w utlzs prn rlton o sntn rv oo rsults, rsn poor ronolol orrns to n ptl lvl y 20% W lso propos n vluton mtr tt msurs sntn ontnuty n mnmnt-s vluton tsk T mnmnt-s vluton outprorm t vluton tt omprs n orrn wt n nswr m y umn T sntn ontnuty mtr ppl to t mnmnt-s tsk sow mor r-

7 Sprmn Knll Contnuty Mto AVG SD AVG SD AVG SD RO CO COT PO POT HO Tl 2: Comprson wt orrt orrn Sprmn Knll Contnuty Mto AVG SD AVG SD AVG SD RO CO COT PO POT HO Tl 3: Comprson wt umn-m orrn mnts wt t rtn rsult W pln to o urtr stuy on t sntn orrn prolm n utur work, xplorn ow to pply our lortm to oumnts otr tn nwsppr or ntrt orrn prolm wt xtrton prolm to mprov otr W lso ronz t nssty to stls n utomt vluton mto o sntn orrn Aknowlmnts W m us o Mn Nwsppr n Yomur Nwsppr rtls n summrzton tst ollton o TSC-3 Rrns R Brzly, E El, n K MKown 2002 Inrrn strts or sntn orrn n multoumnt summrzton Journl o Artl Intlln Rsr (JAIR), 17:35 55 J Cronll n J Golstn 1998 T us o MMR, vrsty-s rrnkn or rorrn oumnts n proun summrs In Prons o t 21st Annul Intrntonl ACM-SIGIR Conrn on Rsr n Dvlopmnt n Inormton Rtrvl, ps T M Covr n P E Hrt 1967 Nrst nor pttrn lsston IEEE Trnstons on Inormton Tory, IT-13:21 27 T Hro, T Fukusm, M Okumur, n H Nn to ppr n 2004 Txt summrzton lln 3: txt summrzton vluton t ntr worksop4 In Workn not o t 4t NTCIR Worksop Mtn J Hos 1990 Ltrtur n Conton, CSLI Ltur Nots 21 CSLI D Hum 1748 Plosopl Essys onrnn Humn Unrstnn M Lpt 2003 Prolst txt struturn: xprmnts wt sntn orrn In Prons o t 41st Mtn o t Assoton o Computtonl Lnusts, ps H P Lun 1958 T utomt rton o ltrtur strts IBM Journl o Rsr n Dvlopmnt, 2(2): I Mn n G Wlson 2000 Roust tmporl prossn o nws In Prons o t 38t Annul Mtn o ACL 2000, ps I Mn, B Smn, n J Zn 2003 Inrrn tmporl orrn o vnts n nws Prons o t Humn Lnu Tnoloy Conrn (HLT-NAACL) 03 I Mn 2001 Auomt Summrzton Jon Bnjmns W Mnn n S Tompson 1988 Rtorl strutur tory: Towr untonl tory o txt ornzton Txt, 8: N Okzk, Y Mtsuo, n M Iszuk to ppr n 2004 TISS: An ntrt summrzton systm or TSC-3 In Workn not o t 4t NTCIR Worksop Mtn