Document Retrieval for Large Scale Content Analysis using Contextualized Dictionaries

Size: px
Start display at page:

Download "Document Retrieval for Large Scale Content Analysis using Contextualized Dictionaries"

Transcription

1 TKE 2014 Document Retrieval for Large Scale Content Analysis using Contextualized Dictionaries Gregor Wiedemann Andreas Niekler NLP Group Department of Computer Science University of Leipzig Augustusplatz Leipzig

2 Outline 1) Motivation 2) Dictionary creation with topic models 3) Contextualizing dictionaries 4) Retrieval with dictionaries 5) Evaluation 2

3 Motivation social science business intelligence content analysis media studies political science... How is European identity framed in newspapers? How (often) do policy makers refer to concepts of social or distributive justice? Is there a neoliberal economization of political justifications in the public policy debate? 3

4 How to find relevant documents Information Retrieval: obtaining documents relevant to an information need by querying a collection standard query: small key word set Puzzling question: How can analysts represent their (rather abstract) information need? small keyword set Idea: compilation of a reference collection of paradigmatic documents paradigmatic document = document knowingly containing information / language use of interest for CA purpose 4

5 Example use case Political science study on neoliberalism Is there a neoliberal economization of political justifications in the public policy debate? Target collection Reference collection 36 works of confessed neoliberals (Mont Pelerin Society) e.g. Milton Friedman, F.A. Hayek etc. 400,000 news paper articles from DIE ZEIT ( ) Hayek 5

6 Approach 3 steps of retrieval for content analysis purpose 1) extraction of ranked dictionary from reference collection 2) extraction of co-occurrence data from reference collection 3) relevancy scoring of documents in target collection with dictionary + co-occurrence data 6

7 dictionary creation dictionary automatically or manually compiled set of (several hundred) keywords representing conceptual / domain knowledge rank information automatic dictionary creation: unequal importance of terms term extraction task from reference collection e.g. TF/IDF, LL-measure of frequencies between two compara,... 7

8 dictionary creation via topic models Topic Models: statistical models (e.g. LDA, Blei et. al 2003) to extract latent semantic structure from collections distribution over K topics in documents distribution over words in topics p(w zk ) idea: term probability can be used to score weight of dictionary terms K tw n =log (tf (w n )) k=1 p(w n z k ) 8

9 example study Proba bility Top 10 Words mensch, freiheit, gesellschaft, gesetz, regeln, allgemein, grupp, ziel, bestimmt, regier [human, freedom, society, law, rules, common, group, aim, certain, govern] einkomm, gut, zeit, haushalt, konsum, kost, straftat, preis, wert, gleichung [income, goods, time, budget, consume, cost, offense, price, value, equation] steu, gut, offent, period, steuersatz, beschrank, staatlich, einnahm, steuerzahl, besteuer [tax, goods, public, period, tax rate, constraint, state, revenue, tax payer, taxation] polit, analys, regeln, okonom, theori, modell, verhalt, ansatz, frag, polit [political, analysis, rules, economic, theory, model, behaviour, approach, question, politics] Term... Extracting dictionary of 500 highest ranked terms Weigth einkomm [income] preis [price] gut [goods] polit [political] zeit [time] hoh [high] kost [cost] regeln [rules] mensch [human] offent [public] person [person] regier [government] wert [value] inflation [inflation] analys [analysis] bestimmt [certain] allgemein [common]

10 contextualizing dictionaries need for more subtle meaning representation in content analysis IR co-occurrence data captures meaning of terms (distributional semantics hypothesis) Term-Term-Matrix C: computation of significant co-ooccurrences of dictionary terms from reference collection sentence window dictionary of length N C dice measure (0;1) reflects syntagmatic relations 10

11 contextualizing dictionaries filtering for reference corpus specific co-occurrences computation of significant co-ooccurrences of dictionary terms from (large) randomly composed corpus (e.g. Leipzig Corpora Collection) term-term matrix D C '=max(c D,0) 11

12 Example study example term C C' öffentlich [public] beitrag [contribution] eltern [parents] gut privat meinung ausgabe schule leisten wichtig insbesondere sozial größen kind alter schule humankapital altruismus gut ausgabe meinung privat theorie leisten insbesondere größen sozial buch kind alter humankapital schule altruismus common co-occurrences in general langague get lower values in C' specific co-occurrences from reference collection stay high C' 12

13 Retrieval with dictionaries Vector Space Model Translate dictionary ranks into boosting factors 13

14 Retrieval with dictionaries Applying length normalization Applying contextual similarity Final scoring formula 14

15 Retrieval with dictionaries Applying length normalization Applying contextual similarity Final scoring formula 15

16 Example study score length year title 347, Pro und kontra Mehrwertsteuer [Pro's and con's of VAT] 321, Oelkrise und Konjunktur [Oil crisis and economy] 290, Energie muß billig sein [Energy has to be cheap] 289, Die Steuern senken [Lower the taxes] 287, Korrektur der Einkommensteuer [Correction of VAT] 281, Die Bauern im Nacken [The farmers at the neck] 279, Was ist uns die Mark wert? [What is the Mark worth to us?] 272, Steuern mit der Steuer [Governing with taxes] 264, Ohne Abkühlung keine Stabilität [No stability without slowdown] 262, Das sicherste Mittel [The most secure instrument] 261, Entlastung wovon? [Relief of what?] 254, Das Fernsehen und die Angst [Television and fear] 254, Nicht ernst gemeint: die Quote [Quotas not meant serious] 251, Eine Konfliktstrategie der Union [A conflict strategy of the EU] Selecting the top 2,000 ranks of retrieved documents for further Content Analysis 16

17 Evaluation I 2 purposes for evaluation 1) determining optimal α 2) assessing quality of - score context vs. score VSM - topic models vs. tf-idf for dictionary creation no gold standard data set for this retrieval task 2 alternative appoaches Approach 1: Generating pseudorels by data fusion (Nuray/Can 2006): create set of pseudorelevant documents from best ranked documents of most distinctive retrieval systems consider tf-idf / topic model + different α values as different systems evaluate mean average precisions (MAP) of all systems 17

18 Evaluation I 4 most distinctive systems: Dtf-idf + score VSM [α=0] Dtf-idf + score context [tf(w,s)=0] DTopicModel + score VSM [α=0] DTopicModel + score context [tf(w,s)=0] 54 documents as pseudorels 18

19 Evaluation II Approach 2: Precision at k evaluation of example retrieval by domain experts Conclusion 1. improvement of retrieval results by mixing of unigram and co-occurrence information from reference collections 2. further improvement by dictionary extraction with topic models approach enables domain experts to query large collections for texts representing rather abstract domain knowledge 19

20 Literature Alsumait, L., Barbara, D., Gentle, J., Domeniconi, C.: Topic signicance ranking of LDA generative models. In: ECML/PKDD '09: Part I. pp (2009) Biemann, C., Heyer, G., Quastho, U., Richter, M.: The leipzig corpora collection. Monolingual corpora of standard size. In: Corpus Linguistic 2007 (2007) Billhardt, H., Borrajo, D., Maojo, V.: Using term co-occurrence data for document indexing and retrieval. In: In Proceedings of the 22nd IRSG. pp (2000) Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent dirichlet allocation. Journal of Machine Learning Research 3, (2003) Bordag, S.: A comparison of co-occurrence and similarity measures as simulations of context. In: Proceedings of the 9th CICLing. pp (2008) Krippendor, K.: Content analysis: An introduction to its methodology. SAGE, 3 edn. (2013) Nuray, R., Can, F.: Automatic ranking of information retrieval systems using data fusion. Information Processing & Management 42(3), (2006) Peat, H., Willet, P.: The limitations of term co-occurrence data for query expansion in document retrieval systems. ASIS Journal 42(5), 378{383 (1991) 20

Leipzig Corpus Miner A Text Mining Infrastructure for Qualitative Data Analysis

Leipzig Corpus Miner A Text Mining Infrastructure for Qualitative Data Analysis TKE 2014 Leipzig Corpus Miner A Text Mining Infrastructure for Qualitative Data Analysis Andreas Niekler aniekler@informatik.uni-leipzig.de Gregor Wiedemann gregor.wiedemann@uni-leipzig.de Gerhard Heyer

More information

Brauchen die Digital Humanities eine eigene Methodologie?

Brauchen die Digital Humanities eine eigene Methodologie? Deutsche DH, Passau 26.03.2014 Brauchen die Digital Humanities eine eigene Methodologie? 26. März 2014 Heyer / Niekler / Wiedemann 1 Übersicht Aspekte der Operationalisierung geistes- und sozialwissenschaftlicher

More information

Leipzig Corpus Miner - A Text Mining Infrastructure for Qualitative Data Analysis

Leipzig Corpus Miner - A Text Mining Infrastructure for Qualitative Data Analysis Leipzig Corpus Miner - A Text Mining Infrastructure for Qualitative Data Analysis Andreas Niekler, Gregor Wiedemann, Gerhard Heyer To cite this version: Andreas Niekler, Gregor Wiedemann, Gerhard Heyer.

More information

Which universities lead and lag? Toward university rankings based on scholarly output

Which universities lead and lag? Toward university rankings based on scholarly output Which universities lead and lag? Toward university rankings based on scholarly output Daniel Ramage and Christopher D. Manning Computer Science Department Stanford University Stanford, California 94305

More information

Data Mining Yelp Data - Predicting rating stars from review text

Data Mining Yelp Data - Predicting rating stars from review text Data Mining Yelp Data - Predicting rating stars from review text Rakesh Chada Stony Brook University rchada@cs.stonybrook.edu Chetan Naik Stony Brook University cnaik@cs.stonybrook.edu ABSTRACT The majority

More information

Topic models for Sentiment analysis: A Literature Survey

Topic models for Sentiment analysis: A Literature Survey Topic models for Sentiment analysis: A Literature Survey Nikhilkumar Jadhav 123050033 June 26, 2014 In this report, we present the work done so far in the field of sentiment analysis using topic models.

More information

Bridging CAQDAS with text mining: Text analyst s toolbox for Big Data: Science in the Media Project

Bridging CAQDAS with text mining: Text analyst s toolbox for Big Data: Science in the Media Project Bridging CAQDAS with text mining: Text analyst s toolbox for Big Data: Science in the Media Project Ahmet Suerdem Istanbul Bilgi University; LSE Methodology Dept. Science in the media project is funded

More information

Identifying Focus, Techniques and Domain of Scientific Papers

Identifying Focus, Techniques and Domain of Scientific Papers Identifying Focus, Techniques and Domain of Scientific Papers Sonal Gupta Department of Computer Science Stanford University Stanford, CA 94305 sonal@cs.stanford.edu Christopher D. Manning Department of

More information

IT services for analyses of various data samples

IT services for analyses of various data samples IT services for analyses of various data samples Ján Paralič, František Babič, Martin Sarnovský, Peter Butka, Cecília Havrilová, Miroslava Muchová, Michal Puheim, Martin Mikula, Gabriel Tutoky Technical

More information

Search and Information Retrieval

Search and Information Retrieval Search and Information Retrieval Search on the Web 1 is a daily activity for many people throughout the world Search and communication are most popular uses of the computer Applications involving search

More information

Wikipedia and Web document based Query Translation and Expansion for Cross-language IR

Wikipedia and Web document based Query Translation and Expansion for Cross-language IR Wikipedia and Web document based Query Translation and Expansion for Cross-language IR Ling-Xiang Tang 1, Andrew Trotman 2, Shlomo Geva 1, Yue Xu 1 1Faculty of Science and Technology, Queensland University

More information

Effective Mentor Suggestion System for Collaborative Learning

Effective Mentor Suggestion System for Collaborative Learning Effective Mentor Suggestion System for Collaborative Learning Advait Raut 1 U pasana G 2 Ramakrishna Bairi 3 Ganesh Ramakrishnan 2 (1) IBM, Bangalore, India, 560045 (2) IITB, Mumbai, India, 400076 (3)

More information

How One Word Can Make all the Difference

How One Word Can Make all the Difference How One Word Can Make all the Difference Using Subject Metadata for Automatic Query Expansion and Reformulation Vivien Petras School of Information Management & Systems UC Berkeley Overview Introduction

More information

Folksonomies versus Automatic Keyword Extraction: An Empirical Study

Folksonomies versus Automatic Keyword Extraction: An Empirical Study Folksonomies versus Automatic Keyword Extraction: An Empirical Study Hend S. Al-Khalifa and Hugh C. Davis Learning Technology Research Group, ECS, University of Southampton, Southampton, SO17 1BJ, UK {hsak04r/hcd}@ecs.soton.ac.uk

More information

A SYSTEM FOR AUTOMATIC QUERY EXPANSION IN A BROWSER-BASED ENVIRONMENT

A SYSTEM FOR AUTOMATIC QUERY EXPANSION IN A BROWSER-BASED ENVIRONMENT Mario Kubek Technical University of Ilmenau, Germany mario.kubek@tu-ilmenau.de Hans Friedrich Witschel University of Leipzig, Germany witschel@informatik.uni-leipzig.de A SYSTEM FOR AUTOMATIC QUERY EXPANSION

More information

Term extraction for user profiling: evaluation by the user

Term extraction for user profiling: evaluation by the user Term extraction for user profiling: evaluation by the user Suzan Verberne 1, Maya Sappelli 1,2, Wessel Kraaij 1,2 1 Institute for Computing and Information Sciences, Radboud University Nijmegen 2 TNO,

More information

1 o Semestre 2007/2008

1 o Semestre 2007/2008 Departamento de Engenharia Informática Instituto Superior Técnico 1 o Semestre 2007/2008 Outline 1 2 3 4 5 Outline 1 2 3 4 5 Exploiting Text How is text exploited? Two main directions Extraction Extraction

More information

Homework 2. Page 154: Exercise 8.10. Page 145: Exercise 8.3 Page 150: Exercise 8.9

Homework 2. Page 154: Exercise 8.10. Page 145: Exercise 8.3 Page 150: Exercise 8.9 Homework 2 Page 110: Exercise 6.10; Exercise 6.12 Page 116: Exercise 6.15; Exercise 6.17 Page 121: Exercise 6.19 Page 122: Exercise 6.20; Exercise 6.23; Exercise 6.24 Page 131: Exercise 7.3; Exercise 7.5;

More information

2-3 Automatic Construction Technology for Parallel Corpora

2-3 Automatic Construction Technology for Parallel Corpora 2-3 Automatic Construction Technology for Parallel Corpora We have aligned Japanese and English news articles and sentences, extracted from the Yomiuri and the Daily Yomiuri newspapers, to make a large

More information

Projektgruppe. Categorization of text documents via classification

Projektgruppe. Categorization of text documents via classification Projektgruppe Steffen Beringer Categorization of text documents via classification 4. Juni 2010 Content Motivation Text categorization Classification in the machine learning Document indexing Construction

More information

IRIS - English-Irish Translation System

IRIS - English-Irish Translation System IRIS - English-Irish Translation System Mihael Arcan, Unit for Natural Language Processing of the Insight Centre for Data Analytics at the National University of Ireland, Galway Introduction about me,

More information

Diagnosis Code Assignment Support Using Random Indexing of Patient Records A Qualitative Feasibility Study

Diagnosis Code Assignment Support Using Random Indexing of Patient Records A Qualitative Feasibility Study Diagnosis Code Assignment Support Using Random Indexing of Patient Records A Qualitative Feasibility Study Aron Henriksson 1, Martin Hassel 1, and Maria Kvist 1,2 1 Department of Computer and System Sciences

More information

LINGUISTIC SUPPORT IN "THESIS WRITER": CORPUS-BASED ACADEMIC PHRASEOLOGY IN ENGLISH AND GERMAN

LINGUISTIC SUPPORT IN THESIS WRITER: CORPUS-BASED ACADEMIC PHRASEOLOGY IN ENGLISH AND GERMAN ELN INAUGURAL CONFERENCE, PRAGUE, 7-8 NOVEMBER 2015 EUROPEAN LITERACY NETWORK: RESEARCH AND APPLICATIONS Panel session Recent trends in Bachelor s dissertation/thesis research: foci, methods, approaches

More information

Recommender Systems: Content-based, Knowledge-based, Hybrid. Radek Pelánek

Recommender Systems: Content-based, Knowledge-based, Hybrid. Radek Pelánek Recommender Systems: Content-based, Knowledge-based, Hybrid Radek Pelánek 2015 Today lecture, basic principles: content-based knowledge-based hybrid, choice of approach,... critiquing, explanations,...

More information

Intelligent Search for Answering Clinical Questions Coronado Group, Ltd. Innovation Initiatives

Intelligent Search for Answering Clinical Questions Coronado Group, Ltd. Innovation Initiatives Intelligent Search for Answering Clinical Questions Coronado Group, Ltd. Innovation Initiatives Search The Way You Think Copyright 2009 Coronado, Ltd. All rights reserved. All other product names and logos

More information

SYSTRAN Chinese-English and English-Chinese Hybrid Machine Translation Systems for CWMT2011 SYSTRAN 混 合 策 略 汉 英 和 英 汉 机 器 翻 译 系 CWMT2011 技 术 报 告

SYSTRAN Chinese-English and English-Chinese Hybrid Machine Translation Systems for CWMT2011 SYSTRAN 混 合 策 略 汉 英 和 英 汉 机 器 翻 译 系 CWMT2011 技 术 报 告 SYSTRAN Chinese-English and English-Chinese Hybrid Machine Translation Systems for CWMT2011 Jin Yang and Satoshi Enoue SYSTRAN Software, Inc. 4444 Eastgate Mall, Suite 310 San Diego, CA 92121, USA E-mail:

More information

Word Completion and Prediction in Hebrew

Word Completion and Prediction in Hebrew Experiments with Language Models for בס"ד Word Completion and Prediction in Hebrew 1 Yaakov HaCohen-Kerner, Asaf Applebaum, Jacob Bitterman Department of Computer Science Jerusalem College of Technology

More information

Domain Classification of Technical Terms Using the Web

Domain Classification of Technical Terms Using the Web Systems and Computers in Japan, Vol. 38, No. 14, 2007 Translated from Denshi Joho Tsushin Gakkai Ronbunshi, Vol. J89-D, No. 11, November 2006, pp. 2470 2482 Domain Classification of Technical Terms Using

More information

Stefan Engelberg (IDS Mannheim), Workshop Corpora in Lexical Research, Bucharest, Nov. 2008 [Folie 1]

Stefan Engelberg (IDS Mannheim), Workshop Corpora in Lexical Research, Bucharest, Nov. 2008 [Folie 1] Content 1. Empirical linguistics 2. Text corpora and corpus linguistics 3. Concordances 4. Application I: The German progressive 5. Part-of-speech tagging 6. Fequency analysis 7. Application II: Compounds

More information

Development of an Enhanced Web-based Automatic Customer Service System

Development of an Enhanced Web-based Automatic Customer Service System Development of an Enhanced Web-based Automatic Customer Service System Ji-Wei Wu, Chih-Chang Chang Wei and Judy C.R. Tseng Department of Computer Science and Information Engineering Chung Hua University

More information

Cross-Language Information Retrieval by Domain Restriction using Web Directory Structure

Cross-Language Information Retrieval by Domain Restriction using Web Directory Structure Cross-Language Information Retrieval by Domain Restriction using Web Directory Structure Fuminori Kimura Faculty of Culture and Information Science, Doshisha University 1 3 Miyakodani Tatara, Kyoutanabe-shi,

More information

Analysis of Social Media Streams

Analysis of Social Media Streams Fakultätsname 24 Fachrichtung 24 Institutsname 24, Professur 24 Analysis of Social Media Streams Florian Weidner Dresden, 21.01.2014 Outline 1.Introduction 2.Social Media Streams Clustering Summarization

More information

Getting Off to a Good Start: Best Practices for Terminology

Getting Off to a Good Start: Best Practices for Terminology Getting Off to a Good Start: Best Practices for Terminology Technologies for term bases, term extraction and term checks Angelika Zerfass, zerfass@zaac.de Tools in the Terminology Life Cycle Extraction

More information

Search Engines. Stephen Shaw <stesh@netsoc.tcd.ie> 18th of February, 2014. Netsoc

Search Engines. Stephen Shaw <stesh@netsoc.tcd.ie> 18th of February, 2014. Netsoc Search Engines Stephen Shaw Netsoc 18th of February, 2014 Me M.Sc. Artificial Intelligence, University of Edinburgh Would recommend B.A. (Mod.) Computer Science, Linguistics, French,

More information

From Terminology Extraction to Terminology Validation: An Approach Adapted to Log Files

From Terminology Extraction to Terminology Validation: An Approach Adapted to Log Files Journal of Universal Computer Science, vol. 21, no. 4 (2015), 604-635 submitted: 22/11/12, accepted: 26/3/15, appeared: 1/4/15 J.UCS From Terminology Extraction to Terminology Validation: An Approach Adapted

More information

Real-Time Identification of MWE Candidates in Databases from the BNC and the Web

Real-Time Identification of MWE Candidates in Databases from the BNC and the Web Real-Time Identification of MWE Candidates in Databases from the BNC and the Web Identifying and Researching Multi-Word Units British Association for Applied Linguistics Corpus Linguistics SIG Oxford Text

More information

Approaches of Using a Word-Image Ontology and an Annotated Image Corpus as Intermedia for Cross-Language Image Retrieval

Approaches of Using a Word-Image Ontology and an Annotated Image Corpus as Intermedia for Cross-Language Image Retrieval Approaches of Using a Word-Image Ontology and an Annotated Image Corpus as Intermedia for Cross-Language Image Retrieval Yih-Chen Chang and Hsin-Hsi Chen Department of Computer Science and Information

More information

A Statistical Text Mining Method for Patent Analysis

A Statistical Text Mining Method for Patent Analysis A Statistical Text Mining Method for Patent Analysis Department of Statistics Cheongju University, shjun@cju.ac.kr Abstract Most text data from diverse document databases are unsuitable for analytical

More information

Quantifying Political Legitimacy from Twitter

Quantifying Political Legitimacy from Twitter Quantifying Political Legitimacy from Twitter Haibin Liu and Dongwon Lee College of Information Sciences and Technology The Pennsylvania State University, University Park, PA {haibin,dongwon}@psu.edu Abstract.

More information

Incorporating Window-Based Passage-Level Evidence in Document Retrieval

Incorporating Window-Based Passage-Level Evidence in Document Retrieval Incorporating -Based Passage-Level Evidence in Document Retrieval Wensi Xi, Richard Xu-Rong, Christopher S.G. Khoo Center for Advanced Information Systems School of Applied Science Nanyang Technological

More information

Cover Page. "Assessing the Agreement of Cognitive Space with Information Space" A Research Seed Grant Proposal to the UNC-CH Cognitive Science Program

Cover Page. Assessing the Agreement of Cognitive Space with Information Space A Research Seed Grant Proposal to the UNC-CH Cognitive Science Program Cover Page "Assessing the Agreement of Cognitive Space with Information Space" A Research Seed Grant Proposal to the UNC-CH Cognitive Science Program Submitted by: Dr. Gregory B. Newby Assistant Professor

More information

An Incrementally Trainable Statistical Approach to Information Extraction Based on Token Classification and Rich Context Models

An Incrementally Trainable Statistical Approach to Information Extraction Based on Token Classification and Rich Context Models Dissertation (Ph.D. Thesis) An Incrementally Trainable Statistical Approach to Information Extraction Based on Token Classification and Rich Context Models Christian Siefkes Disputationen: 16th February

More information

Hybrid Strategies. for better products and shorter time-to-market

Hybrid Strategies. for better products and shorter time-to-market Hybrid Strategies for better products and shorter time-to-market Background Manufacturer of language technology software & services Spin-off of the research center of Germany/Heidelberg Founded in 1999,

More information

Statistics on Long Distance Transportation by Buses and coaches

Statistics on Long Distance Transportation by Buses and coaches Statistics on Long Distance Transportation by Buses and coaches Germany ECE-Workshop - Maastricht, 29. November 2006 Uwe Reim Federal Statistical Office of Germany Folie 1 German official statistics: Legal

More information

HIERARCHICAL HYBRID TRANSLATION BETWEEN ENGLISH AND GERMAN

HIERARCHICAL HYBRID TRANSLATION BETWEEN ENGLISH AND GERMAN HIERARCHICAL HYBRID TRANSLATION BETWEEN ENGLISH AND GERMAN Yu Chen, Andreas Eisele DFKI GmbH, Saarbrücken, Germany May 28, 2010 OUTLINE INTRODUCTION ARCHITECTURE EXPERIMENTS CONCLUSION SMT VS. RBMT [K.

More information

VCU-TSA at Semeval-2016 Task 4: Sentiment Analysis in Twitter

VCU-TSA at Semeval-2016 Task 4: Sentiment Analysis in Twitter VCU-TSA at Semeval-2016 Task 4: Sentiment Analysis in Twitter Gerard Briones and Kasun Amarasinghe and Bridget T. McInnes, PhD. Department of Computer Science Virginia Commonwealth University Richmond,

More information

Using Wikipedia to Translate OOV Terms on MLIR

Using Wikipedia to Translate OOV Terms on MLIR Using to Translate OOV Terms on MLIR Chen-Yu Su, Tien-Chien Lin and Shih-Hung Wu* Department of Computer Science and Information Engineering Chaoyang University of Technology Taichung County 41349, TAIWAN

More information

FOR IMMEDIATE RELEASE

FOR IMMEDIATE RELEASE FOR IMMEDIATE RELEASE Hitachi Developed Basic Artificial Intelligence Technology that Enables Logical Dialogue Analyzes huge volumes of text data on issues under debate, and presents reasons and grounds

More information

Social Business Intelligence Text Search System

Social Business Intelligence Text Search System Social Business Intelligence Text Search System Sagar Ligade ME Computer Engineering. Pune Institute of Computer Technology Pune, India ABSTRACT Today the search engine plays the important role in the

More information

High quality, low maintenance content tagging @ ZEIT Online Breno Faria, Christoph Goller

High quality, low maintenance content tagging @ ZEIT Online Breno Faria, Christoph Goller High quality, low maintenance content tagging @ ZEIT Online Breno Faria, Christoph Goller About Us IntraFind Software AG Elasticsearch Partner (we also do consulting) Specialist for Information Retrieval

More information

RRSS - Rating Reviews Support System purpose built for movies recommendation

RRSS - Rating Reviews Support System purpose built for movies recommendation RRSS - Rating Reviews Support System purpose built for movies recommendation Grzegorz Dziczkowski 1,2 and Katarzyna Wegrzyn-Wolska 1 1 Ecole Superieur d Ingenieurs en Informatique et Genie des Telecommunicatiom

More information

Mining a Corpus of Job Ads

Mining a Corpus of Job Ads Mining a Corpus of Job Ads Workshop Strings and Structures Computational Biology & Linguistics Jürgen Jürgen Hermes Hermes Sprachliche Linguistic Data Informationsverarbeitung Processing Institut Department

More information

An Improved VSM-based Post-Requirements Traceability Recovery Approach Using Context Analysis

An Improved VSM-based Post-Requirements Traceability Recovery Approach Using Context Analysis An Improved VSM-based Post-Requirements Traceability Recovery Approach Using Context Analysis Jiale Zhou, Yue Lu, Kristina Lundqvist Mälardalen Real-Time Research Centre, Mälardalen University, Västerås,

More information

Content-Based Discovery of Twitter Influencers

Content-Based Discovery of Twitter Influencers Content-Based Discovery of Twitter Influencers Chiara Francalanci, Irma Metra Department of Electronics, Information and Bioengineering Polytechnic of Milan, Italy irma.metra@mail.polimi.it chiara.francalanci@polimi.it

More information

A History of the «Concise Oxford Dictionary»

A History of the «Concise Oxford Dictionary» Lodz Studies in Language 34 A History of the «Concise Oxford Dictionary» Bearbeitet von Malgorzata Kaminska 1. Auflage 2014. Buch. 342 S. Hardcover ISBN 978 3 631 65268 8 Format (B x L): 14,8 x 21 cm Gewicht:

More information

Dublin City University at CLEF 2004: Experiments with the ImageCLEF St Andrew s Collection

Dublin City University at CLEF 2004: Experiments with the ImageCLEF St Andrew s Collection Dublin City University at CLEF 2004: Experiments with the ImageCLEF St Andrew s Collection Gareth J. F. Jones, Declan Groves, Anna Khasin, Adenike Lam-Adesina, Bart Mellebeek. Andy Way School of Computing,

More information

CATALYST REPORT DebateHub and Collective Intelligence Dashboard Testing University of Naples Federico II

CATALYST REPORT DebateHub and Collective Intelligence Dashboard Testing University of Naples Federico II CATALYST REPORT DebateHub and Collective Intelligence Dashboard Testing University of Naples Federico II 1. Overview: Information about Community Partner Involved 1.1. Resumé This test was carried out

More information

Network Big Data: Facing and Tackling the Complexities Xiaolong Jin

Network Big Data: Facing and Tackling the Complexities Xiaolong Jin Network Big Data: Facing and Tackling the Complexities Xiaolong Jin CAS Key Laboratory of Network Data Science & Technology Institute of Computing Technology Chinese Academy of Sciences (CAS) 2015-08-10

More information

Analysis of Tweets for Prediction of Indian Stock Markets

Analysis of Tweets for Prediction of Indian Stock Markets Analysis of Tweets for Prediction of Indian Stock Markets Phillip Tichaona Sumbureru Department of Computer Science and Engineering, JNTU College of Engineering Hyderabad, Kukatpally, Hyderabad-500 085,

More information

Interactive Recovery of Requirements Traceability Links Using User Feedback and Configuration Management Logs

Interactive Recovery of Requirements Traceability Links Using User Feedback and Configuration Management Logs Interactive Recovery of Requirements Traceability Links Using User Feedback and Configuration Management Logs Ryosuke Tsuchiya 1, Hironori Washizaki 1, Yoshiaki Fukazawa 1, Keishi Oshima 2, and Ryota Mibe

More information

Big Data Summarization Using Semantic. Feture for IoT on Cloud

Big Data Summarization Using Semantic. Feture for IoT on Cloud Contemporary Engineering Sciences, Vol. 7, 2014, no. 22, 1095-1103 HIKARI Ltd, www.m-hikari.com http://dx.doi.org/10.12988/ces.2014.49137 Big Data Summarization Using Semantic Feture for IoT on Cloud Yoo-Kang

More information

dm106 TEXT MINING FOR CUSTOMER RELATIONSHIP MANAGEMENT: AN APPROACH BASED ON LATENT SEMANTIC ANALYSIS AND FUZZY CLUSTERING

dm106 TEXT MINING FOR CUSTOMER RELATIONSHIP MANAGEMENT: AN APPROACH BASED ON LATENT SEMANTIC ANALYSIS AND FUZZY CLUSTERING dm106 TEXT MINING FOR CUSTOMER RELATIONSHIP MANAGEMENT: AN APPROACH BASED ON LATENT SEMANTIC ANALYSIS AND FUZZY CLUSTERING ABSTRACT In most CRM (Customer Relationship Management) systems, information on

More information

Adaptive Business Intelligence

Adaptive Business Intelligence Adaptive Business Intelligence Bearbeitet von Zbigniew Michalewicz, Martin Schmidt, Matthew Michalewicz, Constantin Chiriac 1. Auflage 2006. Buch. xiii, 246 S. Hardcover ISBN 978 3 540 32928 2 Format (B

More information

Risk Analysis Overview

Risk Analysis Overview What Is Risk? Uncertainty about a situation can often indicate risk, which is the possibility of loss, damage, or any other undesirable event. Most people desire low risk, which would translate to a high

More information

Extracting translation relations for humanreadable dictionaries from bilingual text

Extracting translation relations for humanreadable dictionaries from bilingual text Extracting translation relations for humanreadable dictionaries from bilingual text Overview 1. Company 2. Translate pro 12.1 and AutoLearn 3. Translation workflow 4. Extraction method 5. Extended

More information

Latent Semantic Indexing with Selective Query Expansion Abstract Introduction

Latent Semantic Indexing with Selective Query Expansion Abstract Introduction Latent Semantic Indexing with Selective Query Expansion Andy Garron April Kontostathis Department of Mathematics and Computer Science Ursinus College Collegeville PA 19426 Abstract This article describes

More information

Disambiguating Implicit Temporal Queries by Clustering Top Relevant Dates in Web Snippets

Disambiguating Implicit Temporal Queries by Clustering Top Relevant Dates in Web Snippets Disambiguating Implicit Temporal Queries by Clustering Top Ricardo Campos 1, 4, 6, Alípio Jorge 3, 4, Gaël Dias 2, 6, Célia Nunes 5, 6 1 Tomar Polytechnic Institute, Tomar, Portugal 2 HULTEC/GREYC, University

More information

Semantic Search in E-Discovery. David Graus & Zhaochun Ren

Semantic Search in E-Discovery. David Graus & Zhaochun Ren Semantic Search in E-Discovery David Graus & Zhaochun Ren This talk Introduction David Graus! Understanding e-mail traffic David Graus! Topic discovery & tracking in social media Zhaochun Ren 2 Intro Semantic

More information

Information Sharing and Data Protection in the Area of Freedom, Security and Justice

Information Sharing and Data Protection in the Area of Freedom, Security and Justice Information Sharing and Data Protection in the Area of Freedom, Security and Justice Towards Harmonised Data Protection Principles for Information Exchange at EU-level Bearbeitet von Franziska Boehm 1.

More information

Anti-Spam Filter Based on Naïve Bayes, SVM, and KNN model

Anti-Spam Filter Based on Naïve Bayes, SVM, and KNN model AI TERM PROJECT GROUP 14 1 Anti-Spam Filter Based on,, and model Yun-Nung Chen, Che-An Lu, Chao-Yu Huang Abstract spam email filters are a well-known and powerful type of filters. We construct different

More information

TF-IDF. David Kauchak cs160 Fall 2009 adapted from: http://www.stanford.edu/class/cs276/handouts/lecture6-tfidf.ppt

TF-IDF. David Kauchak cs160 Fall 2009 adapted from: http://www.stanford.edu/class/cs276/handouts/lecture6-tfidf.ppt TF-IDF David Kauchak cs160 Fall 2009 adapted from: http://www.stanford.edu/class/cs276/handouts/lecture6-tfidf.ppt Administrative Homework 3 available soon Assignment 2 available soon Popular media article

More information

SYSTRAN 混 合 策 略 汉 英 和 英 汉 机 器 翻 译 系 统

SYSTRAN 混 合 策 略 汉 英 和 英 汉 机 器 翻 译 系 统 SYSTRAN Chinese-English and English-Chinese Hybrid Machine Translation Systems Jin Yang, Satoshi Enoue Jean Senellart, Tristan Croiset SYSTRAN Software, Inc. SYSTRAN SA 9333 Genesee Ave. Suite PL1 La Grande

More information

A Framework for Ontology-Based Knowledge Management System

A Framework for Ontology-Based Knowledge Management System A Framework for Ontology-Based Knowledge Management System Jiangning WU Institute of Systems Engineering, Dalian University of Technology, Dalian, 116024, China E-mail: jnwu@dlut.edu.cn Abstract Knowledge

More information

Exploiting Bilingual Translation for Question Retrieval in Community-Based Question Answering

Exploiting Bilingual Translation for Question Retrieval in Community-Based Question Answering Exploiting Bilingual Translation for Question Retrieval in Community-Based Question Answering Guangyou Zhou, Kang Liu and Jun Zhao National Laboratory of Pattern Recognition Institute of Automation, Chinese

More information

W. Heath Rushing Adsurgo LLC. Harness the Power of Text Analytics: Unstructured Data Analysis for Healthcare. Session H-1 JTCC: October 23, 2015

W. Heath Rushing Adsurgo LLC. Harness the Power of Text Analytics: Unstructured Data Analysis for Healthcare. Session H-1 JTCC: October 23, 2015 W. Heath Rushing Adsurgo LLC Harness the Power of Text Analytics: Unstructured Data Analysis for Healthcare Session H-1 JTCC: October 23, 2015 Outline Demonstration: Recent article on cnn.com Introduction

More information

Insights into Six Decades of Scientific Practice

Insights into Six Decades of Scientific Practice DTA-/CLARIN-D-Konferenz Historische Textkorpora für die Geistes- und Sozialwissenschaften Title Insights into Six Decades of Scientific Practice Speaker Coauthors Gerhard Heyer, NLP chair (heyer@informatik.uni-leipzig.de)

More information

Artificial Intelligence and Transactional Law: Automated M&A Due Diligence. By Ben Klaber

Artificial Intelligence and Transactional Law: Automated M&A Due Diligence. By Ben Klaber Artificial Intelligence and Transactional Law: Automated M&A Due Diligence By Ben Klaber Introduction Largely due to the pervasiveness of electronically stored information (ESI) and search and retrieval

More information

Exploiting Comparable Corpora and Bilingual Dictionaries. the Cross Language Text Categorization

Exploiting Comparable Corpora and Bilingual Dictionaries. the Cross Language Text Categorization Exploiting Comparable Corpora and Bilingual Dictionaries for Cross-Language Text Categorization Alfio Gliozzo and Carlo Strapparava ITC-Irst via Sommarive, I-38050, Trento, ITALY {gliozzo,strappa}@itc.it

More information

Exam in course TDT4215 Web Intelligence - Solutions and guidelines -

Exam in course TDT4215 Web Intelligence - Solutions and guidelines - English Student no:... Page 1 of 12 Contact during the exam: Geir Solskinnsbakk Phone: 94218 Exam in course TDT4215 Web Intelligence - Solutions and guidelines - Friday May 21, 2010 Time: 0900-1300 Allowed

More information

F. Aiolli - Sistemi Informativi 2007/2008

F. Aiolli - Sistemi Informativi 2007/2008 Text Categorization Text categorization (TC - aka text classification) is the task of buiding text classifiers, i.e. sofware systems that classify documents from a domain D into a given, fixed set C =

More information

Enhancing the relativity between Content, Title and Meta Tags Based on Term Frequency in Lexical and Semantic Aspects

Enhancing the relativity between Content, Title and Meta Tags Based on Term Frequency in Lexical and Semantic Aspects Enhancing the relativity between Content, Title and Meta Tags Based on Term Frequency in Lexical and Semantic Aspects Mohammad Farahmand, Abu Bakar MD Sultan, Masrah Azrifah Azmi Murad, Fatimah Sidi me@shahroozfarahmand.com

More information

Clustering Connectionist and Statistical Language Processing

Clustering Connectionist and Statistical Language Processing Clustering Connectionist and Statistical Language Processing Frank Keller keller@coli.uni-sb.de Computerlinguistik Universität des Saarlandes Clustering p.1/21 Overview clustering vs. classification supervised

More information

Mining Signatures in Healthcare Data Based on Event Sequences and its Applications

Mining Signatures in Healthcare Data Based on Event Sequences and its Applications Mining Signatures in Healthcare Data Based on Event Sequences and its Applications Siddhanth Gokarapu 1, J. Laxmi Narayana 2 1 Student, Computer Science & Engineering-Department, JNTU Hyderabad India 1

More information

Automated Problem List Generation from Electronic Medical Records in IBM Watson

Automated Problem List Generation from Electronic Medical Records in IBM Watson Proceedings of the Twenty-Seventh Conference on Innovative Applications of Artificial Intelligence Automated Problem List Generation from Electronic Medical Records in IBM Watson Murthy Devarakonda, Ching-Huei

More information

A Survey on Product Aspect Ranking

A Survey on Product Aspect Ranking A Survey on Product Aspect Ranking Charushila Patil 1, Prof. P. M. Chawan 2, Priyamvada Chauhan 3, Sonali Wankhede 4 M. Tech Student, Department of Computer Engineering and IT, VJTI College, Mumbai, Maharashtra,

More information

Eng. Mohammed Abdualal

Eng. Mohammed Abdualal Islamic University of Gaza Faculty of Engineering Computer Engineering Department Information Storage and Retrieval (ECOM 5124) IR HW 5+6 Scoring, term weighting and the vector space model Exercise 6.2

More information

An Overview of a Role of Natural Language Processing in An Intelligent Information Retrieval System

An Overview of a Role of Natural Language Processing in An Intelligent Information Retrieval System An Overview of a Role of Natural Language Processing in An Intelligent Information Retrieval System Asanee Kawtrakul ABSTRACT In information-age society, advanced retrieval technique and the automatic

More information

Towards an Evaluation Framework for Topic Extraction Systems for Online Reputation Management

Towards an Evaluation Framework for Topic Extraction Systems for Online Reputation Management Towards an Evaluation Framework for Topic Extraction Systems for Online Reputation Management Enrique Amigó 1, Damiano Spina 1, Bernardino Beotas 2, and Julio Gonzalo 1 1 Departamento de Lenguajes y Sistemas

More information

Information Services for Smart Grids

Information Services for Smart Grids Smart Grid and Renewable Energy, 2009, 8 12 Published Online September 2009 (http://www.scirp.org/journal/sgre/). ABSTRACT Interconnected and integrated electrical power systems, by their very dynamic

More information

CINDOR Conceptual Interlingua Document Retrieval: TREC-8 Evaluation.

CINDOR Conceptual Interlingua Document Retrieval: TREC-8 Evaluation. CINDOR Conceptual Interlingua Document Retrieval: TREC-8 Evaluation. Miguel Ruiz, Anne Diekema, Páraic Sheridan MNIS-TextWise Labs Dey Centennial Plaza 401 South Salina Street Syracuse, NY 13202 Abstract:

More information

Sentiment-Oriented Contextual Advertising

Sentiment-Oriented Contextual Advertising Teng-Kai Fan Department of Computer Science National Central University No. 300, Jung-Da Rd., Chung-Li, Tao-Yuan, Taiwan 320, R.O.C. tengkaifan@gmail.com Chia-Hui Chang Department of Computer Science National

More information

Finding Advertising Keywords on Web Pages. Contextual Ads 101

Finding Advertising Keywords on Web Pages. Contextual Ads 101 Finding Advertising Keywords on Web Pages Scott Wen-tau Yih Joshua Goodman Microsoft Research Vitor R. Carvalho Carnegie Mellon University Contextual Ads 101 Publisher s website Digital Camera Review The

More information

SEO for Web 2.0 Enterprise Search Summit West September 2008

SEO for Web 2.0 Enterprise Search Summit West September 2008 SEO for Web 2.0 Enterprise Search Summit West September 2008 Agenda Search 0.0 Search 1.0 Web 2.0 Search 2.0 Data the next Intel inside Harnessing the Collective Intelligence Rich User Experience Software

More information

3 Paraphrase Acquisition. 3.1 Overview. 2 Prior Work

3 Paraphrase Acquisition. 3.1 Overview. 2 Prior Work Unsupervised Paraphrase Acquisition via Relation Discovery Takaaki Hasegawa Cyberspace Laboratories Nippon Telegraph and Telephone Corporation 1-1 Hikarinooka, Yokosuka, Kanagawa 239-0847, Japan hasegawa.takaaki@lab.ntt.co.jp

More information

Behavior Analysis in Crowded Environments. XiaogangWang Department of Electronic Engineering The Chinese University of Hong Kong June 25, 2011

Behavior Analysis in Crowded Environments. XiaogangWang Department of Electronic Engineering The Chinese University of Hong Kong June 25, 2011 Behavior Analysis in Crowded Environments XiaogangWang Department of Electronic Engineering The Chinese University of Hong Kong June 25, 2011 Behavior Analysis in Sparse Scenes Zelnik-Manor & Irani CVPR

More information

SEMANTIC ASSOCIATIONS FOR CONTEXTUAL ADVERTISING

SEMANTIC ASSOCIATIONS FOR CONTEXTUAL ADVERTISING Journal of Electronic Commerce Research, VOL 9, NO 1, 008 SEMANTIC ASSOCIATIONS FOR CONTEXTUAL ADVERTISING Massimiliano Ciaramita Yahoo! Research Barcelona massi@yahoo-inc.com Vanessa Murdock Yahoo! Research

More information

CS 6740 / INFO 6300. Ad-hoc IR. Graduate-level introduction to technologies for the computational treatment of information in humanlanguage

CS 6740 / INFO 6300. Ad-hoc IR. Graduate-level introduction to technologies for the computational treatment of information in humanlanguage CS 6740 / INFO 6300 Advanced d Language Technologies Graduate-level introduction to technologies for the computational treatment of information in humanlanguage form, covering natural-language processing

More information

Customizing an English-Korean Machine Translation System for Patent Translation *

Customizing an English-Korean Machine Translation System for Patent Translation * Customizing an English-Korean Machine Translation System for Patent Translation * Sung-Kwon Choi, Young-Gil Kim Natural Language Processing Team, Electronics and Telecommunications Research Institute,

More information

Stefan Engelberg (IDS Mannheim), Workshop Corpora in Lexical Research, Bucharest, Nov. 2008 [Folie 1]

Stefan Engelberg (IDS Mannheim), Workshop Corpora in Lexical Research, Bucharest, Nov. 2008 [Folie 1] Content 1. Empirical linguistics 2. Text corpora and corpus linguistics 3. Concordances 4. Application I: The German progressive 5. Part-of-speech tagging 6. Fequency analysis 7. Application II: Compounds

More information