Big Data Technology Recommendation Challenges in Web Media Sites

Transcription

1 Big Data Technology Recommendation Challenges in Web Media Sites Course Summary Edward Bortnikov & Ronny Lempel Yahoo! Labs, Haifa Recommender Systems: A Canonical Big Data Problem Pioneered by Amazon in the mid/late 90s Today applied everywhere Shopping sites, content sites, multimedia streaming sites, social networks The field has a dedicated conference and easily merits a dedicated academic course 2 1

2 Recommendation in Social Networks 3 Recommender Systems Example of Effectiveness 1988: Random House releases Touching the Void, a book by a mountain climber detailing a harrowing account of near death in the Andes It got good reviews but modest commercial success 1999: Into Thin Air, another mountain-climbing tragedy book, becomes a best-seller By virtue of Amazon s recommender system, Touching the Void started to sell again, requiring a new edition A revised paperback edition spent 14 weeks on the New York Times bestseller list From The Long Tail, by Chris Anderson 4 2

3 The Netflix Challenge Slides 4-6 courtesy of Yehuda Koren, member of Challenge winners Bellkor s Pragmatic Chaos We re quite curious, really. To the tune of one million dollars. Netflix Prize rules Goal: improve on Netflix existing movie recommender The open-to-the-public contest began October 2, 2006; winners announced September 2009 Prize Based on reduction in root mean squared error on test data $1 million grand prize for 10% improvement on Cinematch result $0K 2007 progress prize for 8.43% improvement $0K 2008 progress prize for 9.44% improvement Netflix gets full rights to use IP developed by the winners Example of Crowdsourcing Netflix basically got over 100 researcher years (and good publicity) for $1.1M 6 3

4 Netflix Movie Ratings Data Training data 100 million ratings 480,000 users 17,770 movies 6 years of data: Test data Last few ratings of each user (2.8 million) Dates of ratings are given user Training data movie score user Test data movie Types of Recommender Systems At a high level, two main techniques: Content-based recommendation: characterizes the affinity of users to certain features (content, metadata) of their preferred items Lots of classification technology under the hood Collaborative Filtering: exploits similar consumption and preference patterns between users See next slides Many state of the art systems combine both techniques 8 4

5 Collaborative Filtering Mathematical Abstraction Consider a consumption matrix R of users and items r i,k =1 whenever person i consumed item k In other cases, r i,k might be person i s rating on item k The matrix R is typically very sparse and often very large Items Real-life task: top-k recommendation From among the items that weren t yet consumed by each user, predict which ones the user would most enjoy Related task on ratings data: matrix completion Predict users ratings for items they have yet to rate, i.e. complete missing values R = users U x I 9 Collaborative Filtering Neighborhood Models Compute the similarity of items [users] to each other Items are considered similar when users tend to rate them similarly or to co-consume them Users are considered similar when they tend to co-consume items or rate items similarly Recommend to a user: Items similar to items he/she has already consumed [rated highly] Items consumed [rated highly] by similar users Key questions: How exactly to define pair-wise similarities How to combine them into quality recommendations 10

6 R = Collaborative Filtering Matrix Factorization Latent factor models (LFM): Map both users and items to some f-dimensional space R f, i.e. produce f-dimensional vectors v u and w i for each user and items Define rating estimates as inner products: q ij = <v i,w j > Main problem: finding a mapping of users and items to the latent factor space that produces good estimates users Items V W U x I U x f f x I Closely related to dimensionality reduction techniques of the ratings matrix R (e.g. Singular Value Decomposition) 11 This Class Real collaborative filtering applications run into many research challenges beyond those represented by analysis of the user-item matrix These challenges are often under-represented in the literature Some examples covered in the slides: Perpetual cold start problems Inferring implicit interactions and satisfaction Personalization vs. Contextualization Repeated consumption and repeated recommendation; diversity Set and sequence recommendation Incremental Collaborative Filtering Social networks and recommendation consumption Focus here is mostly on Web Media sites 12 6

7 Web Media Sites 13 Definition: Cold Start Problems Good recommendations require observed data on the user being recommended to [items being recommended] User cold start: when a new user arrives to a system, can the system make a good first impression Item cold start: how do we recommend newly arrived items with little historic consumption 14 7

8 Challenge: Perpetual Cold Start Problems Extreme cases exhibit perpetual cold-start scenarios: All users are cold & appear just once (e.g. certain online advertising scenarios) Every item is ephemeral with a short lifetime (e.g. news recommendations) 1 False-Positive Costs in Media Sites are Low False positive: recommending an irrelevant item Consequence, in media sites: (just) a bit of lost time As opposed to lots of lost time or money in other settings Opportunity: better handling of cold-start problems Item cold-start: show new item to a select group of users whose feedback should help in modeling it to everyone Several possible formulations of optimization problems User cold-start: more aggressive exploration Vs. playing it safe and perpetuating popular items But exploration should be optimized to effectively model the user 16 8

9 Challenge: Inferring Interactions and Satisfaction Dominant model in the literature: input consists of <user-item-rating> triplets, i.e. explicit ratings are available In many recommendation settings we only know which items users have consumed, not whether they liked them I.e. no explicit ratings data Several publications talk about binary consumption data What about items the user did not consume Was the user even aware of the items he/she did not consume What items did the recommender system expose the user to 17 Presentation Bias Effect on Media Consumption Pop Culture: items longevity creates familiarity Media sites: items are ephemeral, and users are mostly unaware of items the site did not expose them to Presentation bias obscures true taste users essentially select the best of the little that was shown Must correctly account for presentation bias when modeling: seen & not selected not seen & not selected 18 9

10 Aside: Skips in Search Must correctly account for presentation bias when modeling: seen and not selected not seen and not selected Search: negative interpretation of skipped search results (Joachims, KDD 2002) 19 Layouts of Recommendation Modules Interpreting interactions in vertical layouts is easy using the skips paradigm What about 2D, tabbed, horizontal layouts 20 10

11 Layouts of Recommendation Modules (cont.) What about multiple presentation formats Are we more confident in a skip of a salient item 21 Challenge: Inferring Interactions and Satisfaction Beyond consumption, do interactions imply satisfaction Web pages: what happens after the initial click Short online videos: what happens after pressing play TV programs: zapping patterns In some domains, can we even positively assess consumption Is anyone watching Time 22 11

12 Personalized Popular Contextual 23 Challenge: Contextualization vs. Personalization Web media sites often display links to additional stories on each article page Matching the article s context, matching the user, consumed by the user s friends, popular When creating a unified list for a given a user reading a specific page, how should contextualization and personalization be mixed Ignoring story context might create offending recommendations Related direction: Tensor Factorization, Karatzoglou et. al, RecSys

13 Challenge: Repeated Recommendations One typically doesn t buy the same book twice, nor do people typically read the same news story twice But people listen to the songs they like over and over again, and watch movies they like multiple times as well When and how frequently is it ok to recommend an item that was already consumed On the other hand, when should we stop showing a recommendation if the user doesn t act upon it Implication: a recommendation system may not only need to track aggregated consumption to-date It may need to track consumption timelines It may need to track recommendation history 2 3D: Three Aspects of Diversity Time 1. How diverse is the recommendation to user u at time t Search: result set attributes (e.g. diversity) in Search (Agrawal et al., WSDM 2009) Netflix tutorial at RecSys 2012: diversity is Relatively well understood 26 13

14 3D: Three Aspects of Diversity Time 2. How diverse are the recommendations, across all users, at time t Indication of aggressiveness of personalization and deviation from popularity baselines 27 3D: Three Aspects of Diversity Time 3. How diverse are the recommendations to user u over time Shouldn t recommend the same items day after day 28 14

15 Challenge: Recommending Sets and Sequences of Items In some domains, users consume multiple items in rapid succession (e.g. music playlists) Recent works: WWW 2012 (Aizenberg et al., sets) and KDD 2012 (Chen et al., sequences) From Independent utility of recommendations to set or sequence utility, predicting items that go well together Sometimes need to respect constraints An extension of diversity Tiling recommendations: in TV Watch-list generation, the broadcast schedule further complicates matters due to program overlaps Perhaps a new domain of constrained recommendations 29 Challenge: Incremental Collaborative Filtering Live system often cannot afford to recompute recommendations regularly over the entire history Problem: collaborative filtering models do not easily lend themselves to faithful incremental processing User-Item Interactions t 1 User-Item Interactions t 2 User-Item Interactions t 3 M i = CF-ALG(t i ) f, f { M 1, M 2 } CF_ALG(t 1 t 2 ) Is there a good model aggregation function f(m prev, M curr ) that is good enough 30 T 1

16 Social Networks and Recommendation Computation Some are hailing social networks as a silver bullet for recommender systems Tell me who your friends are and we ll tell you what you like Is it really the case that we like the same media as our friends Affinity trumps friendship! There are people out there who are more like us than our limited set of friends Once affinity is considered, the marginal value of social connections is often negligible Not to be confused with non-friendship social networks, where connections are affinity-related (e.g. Epinions) 31 Social Networks and Recommendation Consumption Previous slide nonewithstanding, social is a great motivator for consuming recommendations People like you rate Lincoln very highly Your friends Alice and Bob saw Lincoln last night and loved it Explaining recommendations for motivating and increasing consumption is an emerging practice Some commercial systems completely separate their explanation generation from their recommendation generation So Alice and Bob may not be why the system recommended Lincoln to you, but they will be leveraged to get you to watch it Privacy in the face of joint consumption of a personalized experience vs

17 Course Summary Three main topics: Batch processing of large amounts of data Incremental processing Stream and online processing Each of the main topics covered business needs, algorithms, and systems Complementing topics: Infrastructure: data centers Methodology: controlled experiments Business case example: recommender systems 33 Revisiting Data Science Virtuous Cycle (Web) Requirements for systems come from each step! Capture Crawl, ingest feeds, record instrumented interactions, Transfer Move the data to a system capable of storing and processing it Visualization Experimentation & Metrics Deploy/Serve Tap output of previous step to improve user experience Analyze/Model Here data mining & machine learning take place 34 17

18 Course Summary Plenty that Wasn t Covered Interactive analytics platforms Dremel, Impala In-memory distributed filesystems Spark, Tachyon Graph processing Pregel, Bagel, Giraph Standard Data Science toolkits statistics, machine learning, data mining, information extraction Data visualization Dimensionality reduction techniques Engagement metrics Specific applications 3 Related Courses (Non-Exhaustive List) Systems: Parallel and Distributed Programming 23631/ Distributed Systems Functional Distributed computing Database Systems Theory: Introduction to Statistics / Computational Learning Tools: Introduction to AI Data Mining and Business Intelligence 23676/04619 Machine Learning Approximation Algorithms Applications: Search Engine Technology Information Retrieval Web Search and Data Mining 36 18

19 Final Logistics Exams: 30/7, 6/10; exact hours and location TBD All offline material and non-communicating devices allowed Reception hours by appointment As you may be aware, this was the first rendition of this course Help us improve! Feel free to send feedback to the course s account 37 Conclusions Big Data is an umbrella name for a vast area of multidisciplinary research theoretical, applied and experimental - with plenty more to be done. Academia & open source: Many journals and conferences in the domain Many open-source projects that build fascinating systems Can sustain many graduate-level theses Industry: Companies of all sizes and in many different areas are discovering the need for competency in Big Data and Data Science 38 19