WHAT IS THE TEMPORAL VALUE OF WEB SNIPPETS?

Transcription

1 WHAT IS THE TEMPORAL VALUE OF WEB SNIPPETS? Ricardo Campos 1, 2, 4 Gaël Dias 2, Alípio Jorge 3, 4 1 Tomar Polytechnic Institute, Tomar, Portugal 2 Centre of Human Language Tecnnology and Bioinformatics, University of Beira Interior, Covilhã, Portugal 3 Faculty of Sciences, University of Oporto, OPorto, Portugal 4 LIAAD-INESC Porto L.A, OPorto, Portugal TWAW st International Temporal Web Analytics Workshop in association with WWW 2011, Hyderabad - India, March 28, 2011 [ w w w. i p t. p t ] [ w w w. l i a a d. u p. p t ] h u l t i g. d i. u b i. p t ]

2 INTRODUCTION Web Logs Temporal Expressions Time in the WWW Queries Research Causes and Consequences Our Goal 2-39

3 WWW 2011 SIGIR Last Month INTRODUCTION What is Time? Web Logs TEMPORAL EXPRESSIONS Time in the WWW Queries How is Time Expressed? Research Causes and Consequences Our Goal Time is expressed in a number of different forms, depending on how the temporal intent is defined: Explicit Expressions Implicit Expressions Relative Expressions 3-39

4 INTRODUCTION Web Logs Time is everywhere in the WWW Temporal Expressions TIME IN THE WWW Queries Research Causes and Consequences Our Goal s Webpage s, Blogs Newswire Articles Web Snippets Web Logs Digital Libraries TIME 4-39

5 INTRODUCTION Web Logs And What About Queries? Temporal Expressions Time in the WWW QUERIES Temporal Information Research Our Goal Football World Cup Miss Universe

6 INTRODUCTION Web Logs Temporal Expressions TEMPORAL INFORMATION Time in the WWW Research Queries Our Goal Consequences of not using Temporal Information Such a lack has several consequences: prevents the distribution of relevant documents over time and users to be aware of possible historical perspectives of given subjects; prevents the modeling of user queries according to a specific period; This is mainly due to the fact that the retrieval models used, keep representing documents and queries using a simplistic representation where the temporal semantics of the documents are ignored. 6-39

7 INTRODUCTION Reasons for that Web Logs Temporal Expressions Time in the WWW Queries TEMPORAL INFORMATION Research Our Goal From a query point of view, this is because systems do not infer temporal intents expressed by users in a query (e.g., Miss Universe); From a document point of view, although times clues may be found in the texts, they are usually not taken into account, certainly due to the difficulties that exist to correlate temporal information to the corresponding topics. 7-39

8 INTRODUCTION Web Logs Temporal Expressions Time in the WWW Queries Research in Temporal Information Temporal Information RESEARCH Our Goal Temporal and Content Dynamics Crawling, Indexing and Ranking Detection of Topics and their Track of Changes over Time Web Archives Developing Web Search Improvement 8-39

9 INTRODUCTION Web Logs Why Web Search Improvement? Temporal Expressions Time in the WWW Queries Temporal Information RESEARCH Our Goal Temporal Representation of the Documents As pointed out by (Dakka, Gravano and Ipeirotis 2008) systems mainly use topic similarity ranking of the query results to return documents, not modeling time explicitly. Timelines Web Search Improvement Temporal Snippets Notwithstanding as mentioned by (Alonso, Gertz and Baeza- Yates, 2009) few works have fully used temporal information for exploration and search purposes. Temporal Web Search Engines + Temporal Clustering 9-39

10 INTRODUCTION Web Logs Temporal Expressions Time in the WWW Queries Temporal Information Research OUR GOAL Understanding the Temporal Nature of Implicit User Queries Return of relevant results from several specific periods Understanding the temporal intent of documents and queries is therefore of the utmost importance to produce high quality information retrieval systems Position it on time Define appropriate interfaces to explore the query 10-39

11 INTRODUCTION Web Logs Temporal Expressions Time in the WWW Queries In this paper we have two objectives Temporal Information Research OUR GOAL This paper is the result of part of this research: Study whether web snippets are a valuable source of data to help inferring the temporal intents of queries, either implicitly or explicitly formulated; Study, in parallel with what has been done by (Nunes, Ribeiro and David, 2008), the temporal value of web query logs

12 Summary Web Logs Web Logs 12-39

13 WEB SNIPPETS Twofold Approach Web Logs FRAMEWORK Temporal Value Temporal Value Discussion Implicit Temporal Queries Implicit Temporal Queries Discussion Study the existence of temporal information within web snippets. We are particularly focused on extracting year dates, which are a kind of temporal information that often appears in this type of collection. Check if such information can be used to temporally classify implicit queries; Collection Execution Automatic Date Identification Metrics Classification of Implicit Temporal Queries Q465 Q465R20 Q450R20 Rule based model TSnippets TTitle Concept Classification Ambiguous Broad Clear Q450 Q450R100 Experiment One: Temporal Data in TURL Future Dates Temporal Classification ATemporal Ambiguous Unambiguous Experiment Two: Use of to date Temporal Implicit Queries 13-39

14 Web Logs WEB SNIPPETS Construction of the DataSet FRAMEWORK Temporal Value Temporal Value Discussion Implicit Temporal Queries Implicit Temporal Queries Discussion Collection dataset comprises a series of snippets, titles and associated URLs. 20 queries * 27 categories 540 queries January 2010 October World Cup 2010; Calendar 2011; Hair Styles 2010; Oil Spill; Oil Spill; BP Oil Spill; Waka Waka; 12.69% Internet 9.89% Computer & Electronics 7.96% Entertainment 14-39

15 Web Logs WEB SNIPPETS Framework TEMPORAL VALUE Temporal Value Discussion Execution December 2010 Implicit Temporal Queries Implicit Temporal Queries Discussion Collection Execution Q465R Q450R Q450R100 16,648 62,842 16,

16 Web Logs WEB SNIPPETS Automatic Date Identification Automatic Date Collection Execution Identification Framework TEMPORAL VALUE Temporal Value Discussion Implicit Temporal Queries Implicit Temporal Queries Discussion Upon the retrieved results, particularly over each triple item <snippet, title, url>, we ran our self-defined rule-based model in order to mark dates expressed by means of numerical patterns, particularly year dates 62,842 16,129 16,648 yyyy, yyyy-yyyy, yyyy/yyyy, mm/dd/yyyy, mm.dd.yyyy, dd/mm/yyyy and dd.mm/yyyy. Q465R20 Q450R20 Q450R100 Snippets 95.8% 94.3% 93.1% Titles 97.9% 95.8% 95.3% URLs 85.1% 75.0% 87.4% 16-39

17 WEB SNIPPETS Web Logs Framework TEMPORAL VALUE Temporal Value Discussion Implicit Temporal Queries Implicit Temporal Queries Discussion Evaluation Metrics Collection Execution Automatic Date Identification Metrics In order to better understand and determine the temporal value of each item, we defined three basic measures taking into account the query q. TSnippets = TTitles = TURLs = # Snippets Retrieved with Dates # Snippets Retrieved # Titles Retrieved with Dates # Titles Retrieved # URLs Retrieved with Dates # URLs Retrieved 17-39

18 Web Logs WEB SNIPPETS Comparison of the Different Values Framework Temporal Value TEMPORAL VALUE DISCUSSION Implicit Temporal Queries Implicit Temporal Queries Discussion Collection Execution Automatic Date Identification Metrics Q465R20 Q450R20 Q450R100 Snippets 12.40% 9.50% 9.19% Titles 5.69% 2.98% 3.27% URLs 4.26% 1.89% 5.59% TSnippets, TTitle and TURL 5,59% 100 Results per 3,27% 9,19% TURL(.) 20 Results per 1,89% 2,98% Ttitle(.) Tsnippets(.) 9,50% 0,00% 1,00% 2,00% 3,00% 4,00% 5,00% 6,00% 7,00% 8,00% 9,00% 10,00% 18-39

19 Web Logs WEB SNIPPETS Items with More than One Date Framework Temporal Value TEMPORAL VALUE DISCUSSION Implicit Temporal Queries Implicit Temporal Queries Discussion Collection Execution Automatic Date Identification Metrics Relation between items with dates and with more than one date (Q450R20) Items With Dates Items with more than one date Snippets Title Url 19-39

20 Web Logs WEB SNIPPETS Framework Temporal Value Implicit Temporal Queries TEMPORAL VALUE DISCUSSION Implicit Temporal Queries Discussion Distribution of Dates, Classification, Future Dates Collection Execution Automatic Date Identification Metrics Distribution of dates in the Timeline (Q450R100) Snippets Title URL Dates Sports Automotive Society Politics e.g., Calendar e.g., Football e.g., Dacia Duster e.g., Baby e.g., Barack Obama Q465R20 Q450R20 Q450R100 Snippets 18.6% 6.5% 7.9% Titles 13.8% 18.9% 19.7% URLs 9.64% 8.8% 5.7% 20-39

21 Introduction Web Logs WEB SNIPPETS Framework Temporal Value IMPLICIT TEMPORAL QUERIES Temporal Value Discussion Implicit Temporal Queries Discussion Understand the Temporal Nature of Implicit Queries Collection Execution Automatic Date Identification Metrics Classification of Implicit Temporal Queries Given the temporal value of web snippets, we aim at understanding if this temporal information can be used to automatically disambiguate query terms, namely implicit temporal queries. Football World Cup 21-39

22 Web Logs WEB SNIPPETS Framework Temporal Value IMPLICIT TEMPORAL QUERIES Temporal Value Discussion Implicit Temporal Queries Discussion Understand the Temporal Nature of Implicit Queries Collection Execution Automatic Date Identification Metrics Classification of Implicit Temporal Queries This is a particular hard task in the extent that temporal information is not available at least in a direct way (e.g., WWW)

23 Web Logs WEB SNIPPETS is Ambiguous in Concept? Framework Temporal Value Temporal Value Discussion IMPLICIT TEMPORAL QUERIES Implicit Temporal Queries Discussion Collection Execution Automatic Date Identification Metrics Classification of Implicit Temporal Queries We adopt the methodology proposed by (Song, Luo, Nie, Yu and Hon, 2009): Ambiguous : a query that has more than one meaning, e.g., Scorpions, which may refer to a rock band, the arachnid and the zodiac sign Broad : a query that covers a variety of subtopics, e.g., quotes, which covers some subtopics such as love quotes, historical quotes, etc. 54 Clear : a query that has a specific meaning and covers a narrow topic. Usually is a successful search in which the user can find what he is looking for in the first page of results, e.g., Bank of America

24 Temporal Classification Frequency Introduction Web Logs WEB SNIPPETS Framework Temporal Value IMPLICIT TEMPORAL QUERIES Temporal Value Discussion Implicit Temporal Queries Discussion Framework to Temporally Classify Implicit Queries Automatic Date Classification of Implicit Metrics Collection Execution Identification Temporal Queries Implicit Ambiguous in Concept? Non-Ambiguous Ambiguous Temporally Ambiguous Time 24-39

25 Temporal Classification Frequency Introduction Web Logs WEB SNIPPETS Framework Temporal Value IMPLICIT TEMPORAL QUERIES Temporal Value Discussion Implicit Temporal Queries Discussion Framework to Temporally Classify Implicit Queries Automatic Date Classification of Implicit Metrics Collection Execution Identification Temporal Queries Implicit Ambiguous in Concept? Non-Ambiguous Ambiguous Temporally Ambiguous BP Oil Spill Temporally Unambiguous 2010 Time 25-39

26 Temporal Classification Frequency Introduction Web Logs WEB SNIPPETS Framework Temporal Value IMPLICIT TEMPORAL QUERIES Temporal Value Discussion Implicit Temporal Queries Discussion Framework to Temporally Classify Implicit Queries Automatic Date Classification of Implicit Metrics Collection Execution Identification Temporal Queries Implicit Ambiguous in Concept? Non-Ambiguous Ambiguous Temporally Ambiguous Temporally Unambiguous Make my Trip ATemporal Time 26-39

27 WEB SNIPPETS Web Logs Framework Temporal Value Temporal Value Discussion IMPLICIT TEMPORAL QUERIES Implicit Temporal Queries Discussion Temporal Ambiguity Collection Execution Automatic Date Identification Metrics Classification of Implicit Temporal Queries Each of the 176 queries is classified into one of these three categories based on the temporal value of the triple items <snippet, title, url> retrieved. Given the fact that dates occur in a different proportion in any of the items <snippet, title, url>, we value each differently through Q450R20 Q450R100 TSnippets 66.10% 50.91% TTitles 20.75% 18.14% TURLs 13.75% 30.95% 100% 100% We call this temporal value: temporal ambiguity: 27-39

28 Web Logs WEB SNIPPETS Framework Temporal Value Temporal Value Discussion Implicit Temporal Queries IMPLICIT TEMPORAL QUERIES DISCUSSION Temporal Classification of the 176 Clear Concept Queries Collection Execution Automatic Date Identification Metrics Classification of Implicit Temporal Queries If (TA(q) < 10%) then is ATemporal Else (If TA(q) >= 10%) { If (query results are associated with > 1 year) then is Temporal Ambiguous else is Temporal Unambiguous } 28-39

29 Web Logs WEB SNIPPETS Framework Temporal Value Temporal Value Discussion Implicit Temporal Queries IMPLICIT TEMPORAL QUERIES DISCUSSION Temporal Classification of the 176 Clear Concept Queries Collection Execution Automatic Date Identification Metrics Classification of Implicit Temporal Queries 40 queries 23% 4 queries 2% Conceptual Classification Number Queries 132 queries 75% Ambiguous 220 Temporal Classification Number Queries % Clear 176 ATemporal % Ambiguous 40 23% Unambiguous 4 2% Broad

30 Goal WEB QUERY LOGS FRAMEWORK Temporal Value Temporal Value Discussion We complement our knowledge about temporal information by understanding the explicit relationships existing between queries and dates; We are interested in seeking for explicit temporal queries, e.g., Iraq War 1991 or World Cup Understand two phenomena: (1) are users interested in future dates when looking for a given subject? (2) what is the type of information they are seeking when issuing a query together with a date? 30-39

31 WEB QUERY LOGS Construction of the DataSet FRAMEWORK Temporal Value Temporal Value Discussion Web Logs dataset comprises a series of queries: 21,011,240 queries 10,154,742 queries 143,590 queries 601 queries March 2006 May German Coins; American Flag in 1943; Ford 2009; Epson p2000; 21.96% Automotive 9.48% Entertainment 8.15% Sports 31-39

32 WEB QUERY LOGS Framework TEMPORAL VALUE Temporal Value Discussion 1,21% of Temporal Explicit Queries Temporal Value = 143,590 10,154,742 = 1.41% We may end up with an even lower value. We classify each query according to one of two categories: real date or false date; 601 We conclude that 87 queries (14.14% of the sample) were false positives; 143, ,286 Temporal Explicit Queries, i.e., 1.21%. We can conclude that dates are seldom used by the users to express his intents

33 Introduction WEB QUERY LOGS Framework Temporal Value TEMPORAL VALUE DISCUSSION Distribution of Dates and Future Dates Distribution of Dates in the Timeline Notwithstanding a decrease from 2006 onwards (we recall that this collection is from 2006), future dates still represent 3.49% of the sample collection

34 WEB QUERY LOGS Web Logs Drawbacks Framework Temporal Value TEMPORAL VALUE DISCUSSION Web Logs are extremely hard to access outside the big industrial labs and highly dependent on the user own intents: Queries that have never been typed, thus not existing in the web search log e.g. Blaise Pascal 1623 (his year birth date) Less year qualified queries that may be as relevant as the most frequents ones 34-39

35 Web Logs CONCLUSIONS EXTRACTION OF TEMPORAL INFORMATION and Web Logs Drawbacks Metadata Approach vs Content-based Approach Time has been gaining an increasing importance in IR in a large number of subareas; Documents are full of temporal expressions, however not always exploited; Inferring the user intentions and the period the user has in mind may therefore play an extremely important role; 35-39

36 Web Logs CONCLUSIONS VS Web Logs Extraction of Temporal Information WEB SNIPPETS AND WEB QUERY LOGS Drawbacks In this paper, we showed that query understanding (25% of the queries have an implicit temporal nature) is possible through the use of web snippets; Beyond affording a faster processing, web snippets are a very rich data source, containing a broad range of temporal information (namely years) that can be used to help on inferring the temporal intent of queries; In the opposite direction web query logs have a very small temporal value (at about 1%)

37 Web Logs CONCLUSIONS VS Web Logs Extraction of Temporal Information WEB SNIPPETS AND WEB QUERY LOGS Drawbacks Its also interesting to note that future dates are very common in web snippets, but seldom used in Queries; And that in web snippets, some of the items even have more than one date. Temporal value in web snippets mostly appears together with the categories of automotive, sports, politics, which is precisely what users are looking for when they issue explicit queries 37-39

38 Web Logs CONCLUSIONS Drawbacks and Future Work Extraction of Temporal Information and Web Logs DRAWBACKS Web snippets are computed by search engines, which we do not control basing our system upon results generated by a black box may prevent from obtaining a clear picture of the temporal values of web snippets Drawbacks and Future Work we need to evaluate the feasibility of developing a search engine, albeit of a small scale, which will also enable us to compare a full text analysis approach with a web snippet based one 38-39

39 Web Logs Thanks for your attention! Both experiments are available for download at VipAccess is online at HULTIG is online at LIAAD is online at Polytechnic Institute of Tomar is online at Gaël Dias is online at Alípio Jorge is online at