DEIM Forum 2016 C4-1. skip-gram. , word2vec, [1] 1994 TREC-3 (The Third Text Retrieval Conference) [3] [4] [5] [6] Ballesteros [4] [6]

Transcription

1 DEIM Forum 2016 C Language Technologies Institute, Carnegie Mellon University 5000 Forbes Ave, Pittsburgh, PA USA madao@akane.waseda.jp, hiroakih@cs.cmu.edu, tetsuyasakai@acm.org skip-gram 1., word2vec, [1] 1994 TREC-3 (The Third Text Retrieval Conference) [3] [4] [5] 1 2 [6] [6] Mikolov word2vec [7] [8] word2vec skip-gram [9] Mikolov [8] word2vec [6] word2vec Ballesteros [4] 1 local feedback [10] local context analysis [11] T T

2 [12] Mikolov [9] Skip-gram Skip-gram 1 2 ( Mikolov [8] ) 1 Skip-gram 1 T T t=1 c< = j< = c,j =0 log p(w t+j w t ) (1) T (w 1, w 2, w 3,..., w T ) c word2vec vec( Berlin ) vec( Germany )+vec( F rance ) vec( P aris ) word2vec 2. 3 word2vec 1 [8] 2 Mikolov [8] Mikolov [8] 2 2 [13] min W n W x i z i 2 (2) i=1 W x i i z i x i i n Mikolov [8] [8] 2 2 [6] 5000 Google [14] Robertson Offer Weight [15]

3 ( ) (r + 0.5)(N n R + r + 0.5) OW = r log (n r + 0.5)(R r + 0.5) N n R r (3) R Google [14] Google NTCIR-5,6 CLIR [16] ( 20 ) [6] C0 ID T2 NTCIR-5,6 CLIR [8] Wikipedia [17] Wikipedia 460 Wikipedia 90 word2vec Mikolov [8] Wikipedia 3 2 NTCIR-5,6 CLIR [16] 4. 4 indexing Indri [19] Indri Ponte combination of the language modeling [20] Turtle inference network retrieval frameworks [21] Indri NTCIR NTCIREVAL [22] NTCIREVAL MSnDCG@1000 ndcg normalized discounted cumulative gain [23] DCG DCG ndcg ndcg 1 MSnDCG Burges [24] ndcg [23] MSnDCG MSnDCG & Google 1

4 MSnDCG MSnDCG 5. 3 Tukey HSD Tukey HSD [25] 2 p 2 p & & MSnDCG 3 p & & p & & MSnDCG 4 1 x = y 5 x = y 0 & 4 5% p 5% MSnDCG NTCIR President of Peru, Alberto Fujimori, scandal, bride MSnDCG Peru Fujimori of

5 NTCIR Ichiro, Rookie of the Year, Major League MSnDCG 0 MSnDCG Ichiro NTCIR India and Pakistan territorial conflict, nuclear weapons 017 MSnDCG MSnDCG security test MSnDCG 2 weapon indian 2 MSnDCG NTCIR Embryonic Stem Cells () () MSnDCG 1 NTCIR Television Broadcasting, Digitalization MSnDCG 3 digital signal uhf digitalization digital 5. 5 stopwords stopwords MSnDCG Tukey HSD p 5 stopwords MSnDCG stopwords stopwords 2 p stopwords stopwords stopwords MSnDCG MSnDCG 0 6 stopwords p 5% stopwords 97 16

6 MSnDCG Google () () MSnDCG 2 MSnDCG 6 7 NTCIR-5 MSnDCG ( ) NTCIR-6 MSnDCG ( ) Mikolov [8] Mikolov stopwords [1] Hull, David A., and Grefenstette, Gregory. Querying across languages: a dictionary-based approach to multilingual information retrieval. Proceedings of the 19th annual international ACM SIGIR conference on Research and development in information retrieval. ACM, [2] Grefenstette, Gregory (ed.): Cross-Language Information Retrieval, Kluwer Academic Publishers, [3] Voorhees, Ellen, M. and Harman, Donna, K.: TREC: Experiment and Evaluation in Information Retrieval, The MIT Press, [4] Ballesteros, Lisa, and Croft, W. Bruce. Phrasal translation and query expansion techniques for cross-language informa-

7 tion retrieval. ACM SIGIR Forum. Vol. 31. No. SI. ACM, [5] Sakai, T., Koyama, M., Izuha, T., Kumano, A., Manabe, T. and Kokubu, T.: Toshiba BRIDJE at NTCIR-6 CLIR: The Head/Lead Method and Graded Relevance Feedback, NTCIR-6 Proceedings, pp.36-43, May [6],. DEIM Forum, [7] word2vec, [8] Mikolov, Tomas, Le, Quoc V. and Sutskever, Ilya. Exploiting similarities among languages for machine translation. arxiv preprint arxiv: (2013). [9] Mikolov, Tomas, et al. Distributed representations of words and phrases and their compositionality. Advances in neural information processing systems [10] Attar, Rony, and Fraenkel, Aviezri S. Local feedback in fulltext retrieval systems. Journal of the ACM (JACM) 24.3 (1977): [11] Xu, Jinxi, and Croft, W. Bruce. Query expansion using local and global document analysis. Proceedings of the 19th annual international ACM SIGIR conference on Research and development in information retrieval. ACM, [12], word2vec,, [13] Gardner, William A. Learning characteristics of stochasticgradient-descent algorithms: A general study, analysis, and critique. Signal Processing 6.2 (1984): [14] Google, [15] Robertson, Stephen E., and Sparck Jones, Karen. Simple, proven approaches to text retrieval. University of Cambridge. Computer Laboratory, [16] Kishida, Kazuaki, et al. Overview of CLIR task at the fifth NTCIR workshop. Proc. Fifth NTCIR Workshop [17] Wikipedia, [18] Pre-trained word and phrase vectors, om/p/word2vec/. [19] Indri, [20] Ponte, Jay M., and Croft, W. Bruce. A language modeling approach to information retrieval. Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval. ACM, [21] Turtle, Howard, and Croft, W. Bruce. Evaluation of an inference network-based retrieval model. ACM Transactions on Information Systems (TOIS) 9.3 (1991): [22] Sakai, T. NTCIREVAL: A Generic Toolkit for Information Access Evaluation. FIT Volume 2. RD-004. pp [23],,, [24] Burges, Chris, et al. Learning to rank using gradient descent. Proceedings of the 22nd international conference on Machine learning. ACM, [25] Sakai, T., Metrics, Statistics, Tests, PROMISE Winter School 2013:Bridging between Information Retrieval and Databases (LNCS 8173). pp Springer