Automatic Web Tagging and Person Tagging Using Language Models

Transcription

1 Automatic Web Tagging and Person Tagging Using Language Models Qiaozhu Mei 1 and Yi Zhang 2 1 University of Illinois at Urbana-Champaign, USA qmei2@uiuc.edu 2 University of California at Santa Cruz, USA yiz@soe.ucsc.edu 1 Introduction Social tagging systems, such as Delicious, My Web 2.0, Flickr, YouTube, have been very successful and attracted hundreds of million users. User provided tags of an object/page can be used to help the user re-find the object through search or share the customized object with other people. Instead of waiting for a user to find and input the appropriate words to tag an object, we propose to automatically recommend tags for user to choose from, a process that requires much less cognitive effort than traditional tagging. In particular, we formalize the tag suggestion problem as a ranking problem and propose a new probabilistic language model to rank meaningful tags, including words or phrases, for bookmarks. Besides, we adapt the probabilistic language model to tag users. The user tags can be viewed as recommended queries for the user to search documents. They can also be used as meta data about the users, which could be beneficial for people search or person recommendation. The effectiveness of the proposed techniques are demonstrated on data collected from del.icio.us. 2 Tag Suggestion by Automatic Labeling Tagging Logs In this section, we introduce a probabilistic approach to automatically generate tag suggestions by labeling language models estimated from tagging logs. In the rest of this paper, d is a web document, a tag t is a short text segment and V is the vocabulary of all possible tags. A bookmark, r, is a sequence of tags r[t ] = t 1 t 2...t l, selected by a user r[u] to mark a web document r[d] in a social bookmarking system. 2.1 Candidate Tag Generation To find a set of candidate tags, the most straight forward way is to use the words and phrases in the web document. However, the vocabulary used in a web document could also be quite different from the tags used by real users. One method is to generate such candidate tags from the tagging logs. We analyzed the tag log

2 and found that a bookmark has the following characteristics: 1) each bookmark is usually a sequence of tags instead of a single tag; 2) people usually use meaningful phrases rather than single words as tags (e.g., funny ideas, learning lessons, ESL English ); 3) there is usually no explicit segmentation of tags in a bookmark; and 4) the sequence of tags doesn t follow syntax rules. Motivated by this observation, we extract phrases by ranking word ngrams based on statistical T test similar to [8]. Another method is to rely on an outside dictionary to extract candidate tags. Again, we need to guarantee that the candidate tags are likely to be used as tags by real web users. Therefore, it is reasonable to extract tags from a collection of user generated text collection rather than from other sources such as a dictionary. In our experiments, we use the titles of every entry in Wikipedia 3 as candidate tags. 2.2 A Probabilistic Approach to Tag Ranking To rank the tags based on their relevance to a web document, we borrow the ranking methods in retrieval. In this work, we follow the language modeling retrieval framework, and represent a tag and a web document both with a unigram language model. Formally, we extract a unigram language model, or a multinomial distribution of words from a web document d and a candidate tag t, denoted as {p(w d)} and {p(w t)} respectively. Since we do not rely on the actual content of d, we alternatively estimate p(w d) based on the social tagging logs. Specifically, we have r c(w, r[t ]) I[r[D] = d] p(w d) = (1) w r c(w, r[t ]) I[r[D] = d] where I[S] is an indicator function which is set to 1 if the statement S is true and 0 otherwise, and c(w, r[t ]) is the occurrence of word w in the tags r[t ]. Once we estimated a language model for d, the problem is reduced to selecting tags to label such a multinomial model of words. The easiest way is apparently using words with largest p(w d). This method may have problem because it is usually hard to interpret the meaning of a distribution of words from just the top words because the top words are usually obscure and only partially captures the information encoded by the whole distribution [11]. Instead, we expect labels that could cover the semantics conveyed by the whole distribution (e.g., a distribution with top words like tree, search, DFS, prune, construct is better to be labeled as tree algorithms rather than tree ). We then follow [11] and present a probabilistic approach to automatically label the document language model p(w d). The basic idea is that if we can also extract a language model from a tag t, we could use the Kullback-Leibler (KL) divergence to score each candidate tag. Specifically, 3 f(t, d) = D(d t) = w p(w t) p(w d). (2)

3 How to estimate p(w t) is however tricker. The simplest way is to estimate p(w t) based on the word frequency in a tag. However, a tag is usually too short (e.g., 1 or 2 words) to estimate a reliable language model. If we embed such an estimate in Equation 2, we need to smooth the tag language model so that w, p(w t) > 0 [15]. When a tag is very short, such a smoothed language model would not be trustable. Indeed, in this case f(t, d) will be similar to count the occurrence of t in all bookmarks of d. We need to find a reliable language model p(w t) in an alternatively way. One possibility is to approximate p(w t) from the collection of tagging logs C, and estimate a distribution p(w t, C) to substitute p(w t). Similar to [11], we can rewrite Equation 2 as f(t, d) = w = w p(w t, C) p(w C) w p(w d) p(w C) w p(w t, C) p(w t) p(w, t C) D(d C) + Bias(t, C). (3) p(w C)p(t C) From this rewriting, we see that the scoring function can be decomposed into three components. The second component D(d C) is irrelevant to t, and can be ignored in ranking. The third component p(w t,c) w p(w t) can be interpreted as the bias of using C to approximate the unknown language model p(w t). When the t and C are from the same domain (e.g., if we use C to extract candidate tags), we can fairly assume that such bias is ignorable. Therefore, we have f(t, d) rank = p(w, t C) p(w C)p(t C) = E d[p MI(w, t C)]. (4) w p(w,t C) Please note that log p(w C)p(t C) is actually the pointwise mutual information of t and w conditional on the tagging logs. f(t, d) can thus be interpreted with the expected mutual information of t and a word in the document language model. Estimating P M I(w, t C) is straight forward, since we use efficiently find p(w, t C), P (w C) and P (t C) that maximize the likelihood that each word is cooccurring with t in the tagging records. Specifically, we have r C c(w, r[t ]) I[t r[t ]] p(w, t C) = w t r C c(w, r[t ]) I[t (5) r[t ]] r C c(w, r[t ]) p(w C) = (6) w r c(w, r[t ]) r C I[t r[t ]] p(t C) = t r C I[t r[t ]]. (7) Once the candidate tags are extracted, all the mutual information P M I(w, t C) can be computed and stored offline. This also improves the efficiency in ranking. Computing the expectation of the mutual information could still be time consuming if the vocabulary is large. To further improve the runtime efficiency, we

4 ignore all the mutual information where P MI(w, t C) < 0. We then select the tags with largest f(t, d) as the suggested tags to a document d. 2.3 Further discussion: tagging users and beyond Since eventually we are labeling language models, this method can be applied to suggest tags for users, using the following user language model r c(w, r[t ]) I[r[U] = u] p(w u) = w r c(w, r[t ]) I[r[U] = u]. (8) The same ranking function can be used to generate labels for this user language model, by replacing p(w d) with p(w u) in Equation 4. 3 Experiments Data: To show the effectiveness of our proposed methods, we collect two weeks ( ) tagging records from a well known social bookmarking website Del.icio.us 4, There are 579,652 bookmarks, 20,138 users, and 111,381 distinct tagging words. To test different ways of candidate tag generation, we also collect all the titles of entries in Wikipedia, from the data snapshot of 10/18/2007. There are in total 5,836,166 entries extracted. Candidate Tags: We explore three different ways of candidate tag generation. For the first method, we simply use single words in the tagging logs as candidate tags. For the second method, we extract significant bigrams from tagging logs using Student s T-Test. This was done using the N-gram Statistics Package [1]. We select the top 15,000 bigrams with the largest T-Score as the candidate tags. The top ranked bigrams are presented in Table 1. For the third method, we use all titles of Wikipedia entries as candidate tags. Table 1. Top Bigrams from Tagging Logs Bigrams with Highest T-score css design software tools web webdesign mac osx programming reference web web2.0 art design rails ruby mp3 music tools web photo photography photography photos It is easy to see that most of the top bigrams extracted from the tagging logs are meaningful. However, they could overfit the log data, where some words are user specific (e.g., webdesign), and some bigrams contain redundant words (e.g., photo photography). Such problem does not show in Wikipedia entries. However, only 48k such entry titles appear in the tagging logs, out of 5,836k. 4

5 Tagging Web Documents: The first experiment designed is to suggest tags for web documents. We select web documents with the largest number of bookmarks in the tagging log collection, and automatically suggest tags for them. The results are presented in Table 2. We present the top words in the document language model p(w d) estimated from the tagging logs in the second column. The right three columns present system generated tag suggestions, using single words, significant bigrams, and wikipedia titles as candidate tags, respectively. There are several interesting discoveries from Table 2. First, simply using top words in p(w d) favors frequent terms, such as web2.0. These are not desirable, because when a user uses it in the future trying to retrieve the documents, he has to spend much extra effort to target the web document from the many documents bookmarked with web2.0. The labeling based method gives much much better tag suggestions. In column 3, pipes seems a better tag than yahoo because it captures the meaning of the url more precisely. youtube seems more precise than all other words in column 2 for the url which is a video on youtube. Palette is a very interesting generalization of the meaning of which does not appear in the top words in p(w d). This is because the method we introduce tries to capture the meaning of the whole language model (i.e., the expectation of similarity of a tag to all the words), which thus generates more precise tags. However, some tags are obscure or not meaningful. For example, color, photo, and editor are obscure as tags, and ajax, pipes, feeds are ambiguous which could mean quite different concepts. Some words are also too domain specific and not so meaningful to the common audience (e.g., dhtml, comunidad ). All this is because single words are used as candidate tags. When phrases (significant bigrams, wikipedia entry names) are used as candidate tags, we see that the system generates much more understandable suggestions. Bigrams based tags are much more precise and interpretable. Ajax code, mashup pipes, and api feeds remove the ambiguity of single words. Photography tools, editor flickr, and color design are also more precise than tools, photo, editor, and design. However, some extracted phrases, such as xml youtube, adobe color, color colors, and css ajax, are good tags but not real phrases. In real life, people may not use such expressions. By using wikipedia entry names as candidate tags, the suggestions are more meaningful and understandable(e.g., blog feeds, javascript library, internet video, etc). Tagging Users: We select 10 users with the largest number of bookmarks in the tagging log collection, and automatically suggest tags for them. The results are presented in Table 3. There are also interesting findings from the tag suggestions for web users. Based on user preference analysis, a suggested tag can help user to find interesting web documents that other people bookmarked with this tag. The preference of a user is presented with a language model estimated from his own tags in column 2. We see that our algorithm suggests interesting tags to the user, pre-

6 URLs LM p(w d) Tag = Word Tag = Bigram Tag = Wiki Entry yahoo pipes feeds mashup pipes rss feeds mashup pipes yahoo web2.0 yahoo web2.0 yahoo mashup mashup mashup rss web2.0 rss 386 bookmarks feeds rss mashup rss syndication programming syndication api feeds mashups pipes mashups pipes programming blog feeds ajax ajax ajax code ajax javascript dhtml code javascript dhtml web2.0 javascript javascript ajax javascript webdesign moo.fx javascript web2.0 moo.fx 349 bookmarks programming dragdrop css ajax javascript library code phototype programming web2.0 javascript framework webdev autosuggest javascript programming ajax framework color color adobe color color design colour color design colour webdesign palette color colour palette tools colorscheme color colors web color 158 bookmarks adobe colours colour desgin colours graphics picker inspiration palete cor flash cor webdesign color rgb web2.0 youtube xml youtube internet video video revver web2.0 youtube youtube youtube vodcast video web2.0 revver web primer web2.0 xml research video watch?v=6gmp4nk0eoe internet comunidad online presentation vodcast 157 bookmarks xml participation social video primer community ethnograpy youtube video p2p TV photo photo editor flickr photo photography resize editor online resize tools flickr online photo flickr editor editor editor photo editor 149 bookmarks online edit photography tools edit web2.0 editing photo tools editing flickr crop editor image crop Table 2. Tag Suggestions for Web Documents Users LM p(w d) Tag = Bigram Tag = Wiki Entry photography art photography art photography art photography portraits photoblog portraits digital flickr portraits User1 tools photoblog photography photography web art photo landscapes design flickr photography flickr geek weblog wordpress art contest humor geek hack network programming programming humor programming tweak photography hack hacking hacking User2 blog networking programming security webdesign geek html geek humor security geek hacking sysadmin funny reference security digitalcamera games arg games arg arg games puzzles games research tools games internet games User3 programming arg code puzzles sudoku games sudoku storytelling cryptography code generator code generator software community games community games web rubyonrails web javascript reference css development css css brower development webdev User4 development development editor xhtml rubyonrails development forum dhtml tools development firefox css3 design javascript tools dom Table 3. Tag Suggestions for Web Users

7 sented in column 3 and 4. Tag art photography matches user 1 s interests. If there s tags like digital flickr and art contest, he is also likely to be interested. This also indicates an opportunity of personalized online advertisements. The interests of user 2 are actually a mixture of several themes. From column 4, we clearly see that network programming and geek humor are good suggestions to such themes. However, if there is a tag humor programming from other users (although looks weird in reality), which perfectly matches different aspects of his interests, he is very likely to explore such a tag. Similarly, we see that user 3 likes games and programming related content, and user 4 likes web development. Our methods suggest very highly relevant and understandable tags to them. 4 Related Work Recently, researchers have started to realize the importance of social bookmarking. This leads to the exploration of tagging logs in different ways [4, 9, 3, 7, 6, 14]. Most work are focusing on utilizing social tags, instead of suggesting tags. Folksonomy [9], tagging visualization [3], and spam detection for tagging system [7] are some of such examples. [2] utilizes social tags to help summarization. [6] explores search and ranking in tagging systems. [14] first uses tagging logs to help web search, and [5] gives an empirical justification of helping search with tagging logs. [10] introduced a duality hypothesis of search and tagging, which gives a theoretical justification of using tags to help search tasks. However, none of this work explores the problem of suggesting tags for web documents, or for web users. To the best of our knowledge, automatic bookmark suggestion is not well addressed in existing literature. The only work we are aware of is collaborative tag suggestion described in [13]. They discussed the desirable properties of suggested tags, however their tagging approach is not based on any probabilistic models and rather ad hoc. The probabilistic language modeling framework for tagging is motivated by the well known language modeling approach in the information retrieval community. In particular, [11] has proposed to assign meaningful labels to multinomial topic models. We adapted the technique to the novel problem of tag suggestion, and generate meaningful tags for web documents and web users. This paper is also related to early work on suggesting index terms for library documents [12]. However, all such work are based on content of documents, and is not appropriate for social bookmarking systems, where the content of web pages are hard to keep track of but rich tagging log is available. 5 Conclusions In this work, we formally define the problem of tag suggestion for social bookmarking systems, and present a probabilistic approach to automatically generate and rank meaningful tags for web documents and web users. Empirical experiments show that our proposed methods are effective to extract relevant and

8 meaningful tag suggestions. Such a technique could be applied to other interesting mining problems, such as ad term suggestion for online advertisement systems, and people tagging in social network applications. There are quite a few potential future directions, such as tag suggestion over time, personalized tag suggestion, and collaborative tag suggestion are all among the good examples. Another line of future work is to design a way to quantitative evaluate tag suggestion algorithms. References 1. S. Banerjee and T. Pedersen. The design, implementation, and use of the ngram statistics package. pages , O. Boydell and B. Smyth. From social bookmarking to social summarization: an experiment in community-based summary generation. In Proceedings of the 12th international conference on Intelligent user interfaces, pages 42 51, M. Dubinko, R. Kumar, J. Magnani, J. Novak, P. Raghavan, and A. Tomkins. Visualizing tags over time. In Proceedings of the 15th international conference on World Wide Web, pages , S. Golder and B. A. Huberman. The structure of collaborative tagging systems. Journal of Information Science, P. Heymann, G. Koutrika, and H. Garcia-Molina. Can social bookmarking improve web search? In Proceedings of the international conference on Web search and web data mining, pages , A. Hotho, R. Jasschke, C. Schmitz, and G. Stumme. Information retrieval in folksonomies: Search and ranking. In Y. Sure and J. Domingue, editors, The Semantic Web: Research and Applications, volume 4011, pages , G. Koutrika, F. A. Effendi, Z. Gyöngyi, P. Heymann, and H. Garcia-Molina. Combating spam in tagging systems. In Proceedings of the 3rd international workshop on Adversarial information retrieval on the web, pages 57 64, C. D. Manning and H. Schtze. Foundations of statistical natural language processing. MIT Press, Cambridge, MA, USA, C. Marlow, M. Naaman, D. Boyd, and M. Davis. Position paper, tagging, taxonomy, flickr, article, toread. In Proceedings of Hypertext, pages 31 40, Q. Mei, J. Jiang, H. Su, and C. Zhai. Search and tagging: Two sides of the same coin? In UIUC Technical Report, Q. Mei, X. Shen, and C. Zhai. Automatic labeling of multinomial topic models. In Proceedings of KDD 07, pages , B. R. Schatz, E. H. Johnson, P. A. Cochrane, and H. Chen. Interactive term suggestion for users of digital libraries: using subject thesauri and co-occurrence lists for information retrieval. In Proceedings of the first ACM international conference on Digital libraries, pages , Z. Xu, Y. Fu, J. Mao, and D. Su. Towards the semantic web: Collaborative tag suggestions. In Proceedings of the Collaborative Web Tagging Workshop at the WWW 2006, Edinburgh, Scotland, May Y. Yanbe, A. Jatowt, S. Nakamura, and K. Tanaka. Can social bookmarking enhance search in the web? In JCDL, pages , C. Zhai and J. Lafferty. A study of smoothing methods for language models applied to ad hoc information retrieval. In SIGIR 01: Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval, pages , 2001.