What to Mine from Big Data? Hang Li Noah s Ark Lab Huawei Technologies

Transcription

1 What to Mine from Big Data? Hang Li Noah s Ark Lab Huawei Technologies

2 Big Data Value

3 Two Main Issues in Big Data Mining

4 Agenda Four Principles for What to Mine Stories regarding to Principles Search and Browse Log Mining as Example Our Work on Big Data Mining Mining Query Subtopics from Search Log Data Summary

5 Four Principles for What to Mine 1. Identifying scenarios of mining as much as possible 2. Logging as much data as possible 3. Integrating as much data as possible 4. Understanding data as much as possible

6 Identifying scenarios of mining as much as possible

7 Immanuel Kant The world as we know it is our interpretation of the observable facts in the light of theories that we ourselves invent

8 Example of Bad Design of Toolbar A toolbar developed at a search engine It recorded user s search behavior data However, It did not record the time at which the user closed browser No indication of end of session

9 Logging as much data as possible

10 Examples of Useful Log Information User moves mouse on screen (user may unconsciously put mouse on focused area) may infer users interest on the page User uses mouse to scroll up and down may infer whether user is serious about page content (more scrolling suggests more seriousness) User clicks on next page may infer user s current focus User closes browser window/tab may infer user s current focus

11 Integrating as much data as possible

12 Model of User Search Behavior Data needs to be collected from different sources (toolbar, search engine log) E.g., toolbar usually does not record search results Often challenging to integrate data

13 Understanding Data as Much as Possible

14 AOL Search Data Leak (2006) AOL search data release (20M queries, 650K users, 3 months) New York Times article A Face Is Exposed for AOL Searcher No Queries landscapers in Lilburn, Ga several people with the last name Arnold homes sold in shadow lake subdivision gwinnett county georgia. ''dog that urinates on everything 60 single men Identified searcher is Thelma Arnold, a widow living in Georia

15 Mining Query Subtopics from Search Log Data Yunhua Hu, Yanan Qian 1, Hang Li, Daxin Jiang, Jian Pei 2, and Qinghua Zheng 1 Microsoft Research Asia, Beijing, China 1 SPKLSTN Lab, Xi'an Jiaotong University, China 2 Simon Fraser University, Burnaby, BC, Canada

16 Outline Introduction Our Method Experiments Conclusion 16

17 Demo

18 Mined Subtopics

19

20

21 Subtopics of Query Most queries are ambiguous or multifaceted in web search Harry Shum Harry Shum Microsoft Harry Shum Jr XBox XBox games XBox homepage XBox marketplace Major senses and facets of query (subtopics) 21

22 Our Work = Automatically Mining Subtopics of Queries from Search Log Data

23 Phenomenon 1: One Subtopic per Search (OSS) Query Multi-Clicked URLs (Multi-Clicks) Frequency "Harry Shum" " " Jointly Clicked URLs in the same searches tend to represent the same subtopics

24 Phenomenon 2: Subtopic Clarification by Additional Keyword (SCAK) Query "Harry Shum" Microsoft Harry Shum" "Harry Shum Jr" "Harry Shum Glee Clicked URLs " " " " " " " " " " " URLs clicked in searches of the query and its expanded queries tend to represent the same subtopics.

25 Outline Introduction Our Method Experiments Conclusion 25

26 Our Approach Mining subtopics of queries by leveraging the two phenomena Subtopics of query are represented by URLs Keywords in expanded queries Example of subtopic Subtopi Keywords (in bold face) 1 harry shum microsoft harry shum bing microsoft harry shum 2 harry shum jr harry shum glee harry shum junior URLs

27 Flow of Clustering Method 27

28 Preprocessing Tree structure to index queries ( Q+W and W+Q for Q ) Pruning: Only keep expanded queries with URL overlap 28

29 Similarity Calculation between URLs S 1 : Similarity based URLs on OSS S 2 : Similarity based on SCAK S 3 : Similarity between URL tokens Multi- Click 1 Multi- Click 2 " " Multi- Click 3 N/A N/A 0.64 N/A N/A N/A N/A N/A 0.96 N/A Similarity Matrix of S 1 Similarity Matrix of S 2 URLs Jr Glee Microsoft " "

30 Clustering Algorithm Agglomerative clustering algorithm Two URLs are similar if the similarity is larger than a threshold Each maximum connected subgraph (a group of urls) represents a subtopic Algorithm is efficient and easy to implement 30

31 Outline Introduction Our Method Experiments Conclusion 31

32 Data Set and Parameter Setting One open dataset + two proprietary datasets Evaluation metric: B-cubed precision, recall, and F1 Manually tune the parameters in 1/3 of DataSetA 32

33 Evaluation of Subtopic Mining Evaluation on different similarity functions Evaluation on different types of queries 33

34 Application in Search Result Clustering (1) Search result clustering approaches Baseline: Wang and Zhai s work in SIGIR 07 Our approach: "subtopics of query as seed clusters" + traditional URL clustering Evaluation on TREC and DataSetA 34

35 Application in Search Result Clustering (2) Manual evaluation on DataSetB from various perspectives Side-by-side evaluation on DataSetB 35

36 Application in Search Results Re-ranking (1) 36

37 Application in Search Results Re-ranking (2) 37

38 Outline Introduction Our Method Experiments Conclusion 38

39 Conclusion Discovered two phenomena in search log data to represent query subtopics Developed a clustering method for subtopic mining Applied the mined subtopics into two tasks: search result clustering and re-ranking 39

40 Strength and Limitation of Big Data Mining Big data really creates big value Importance of insight Log tail challenges Mining needs knowledge 40

41 Summary Two Major Issues: What to Mine and How to Mine Four Principles for What to Mine Stories regarding to Principles Search and Browse Log Mining as Example Our Work on Big Data Mining Mining Query Subtopics from Search Log Data

42 Thanks! 42