Mining Social Networks for Recommendation. Mohsen Jamali& Martin Ester Simon Fraser University Tutorial at ICDM 2011 December 12 th 2011

Transcription

1 Mining Social Networks for Recommendation Mohsen Jamali& Martin Ester Simon Fraser University Tutorial at ICDM 2011 December 12 th 2011

2 Introduction Flood of information Conventional (industrial / mass) media Social media Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

3 Introduction Outline Introduction Recommender systems Recommendation in social networks Mining social networks Memory based approaches Model based approaches Link prediction Social networks with distrust Summary Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

4 Recommender Systems Web search Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

5 Recommender Systems Search Input: query keywords Output: ranked list of results User needs to know what he is looking for. but content changes, keywords change Every user gets same result. but users have diverse interests e.g., student, software developer, politician,... Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

6 Recommender Systems Users want to have personalized results. But are not willing to spend a lot of time to specify their personal information needs. Recommender systems automatically identify information relevant for a given user, learning from available data. Data - user actions, - user profiles. Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

7 Recommender Systems Rating Matrix Items Target Item Target User Users Similar User Departed Star Wars Matrix Hurt Locker Titanic Terminator Joe ?? John Susan Pal Jean Ben 1 5 Nathan Ratings Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

8 Recommendation Tasks Rating prediction Predict the rating of target user for target item, e.g. predict Joe s rating for Titanic. Top-N recommendation Predict the top-n highest-rated items among the items not yet rated by target user. Link recommendation (only if social network) Predict the top-n users to which the target user is most likely to connect. Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

9 Applications Yahoo! news recommendations Recommendations of new articles for Today box on Yahoo's home page 9,000 recommendations per minute Sophisticated personalization algorithm Based on demographic user attributes, the places they've visited when they've come to Yahoo in the past, and the stories they've already seen during that particular visit. Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

10 Applications Yahoo! news recommendations (cont.) Team of editors prepare news packages, algorithm ranks packages for user. Has increased the click through rate by 270% since Has helped editors to get better understanding of the interests of different user segments. Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

11 Applications Facebook friend recommendations People you may know Based on mutual friends, work and education information, networks you re part of, contacts and many other factors. Since our formula is automatic, you might occasionally see people you don t know or don t want to be friends with. To remove them from view, just click the X next to their names. Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

12 Applications Pandora music recommendation Internet radio service Pandora.com Music Genome Project: trained music analysts score each song based on hundreds of distinct musical characteristics. Recommend songs with similar scores Recommend sequence of songs Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

13 Applications Item-based collaborative filtering Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

14 Applications Netflix (movie recommendation) $1M prize for 10% accuracy improvement Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

15 Privacy Issues Recommender systems use a lot of personal data: movies watched, current location,... The more personal data shared, the better (more personalized) the recommendations. Serious threat to data privacy. Users need to be able to make an informed choice. E.g., Google users can shut off personalization features by deleting their Web history. Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

16 The Filter Bubble Users get less exposure to conflicting viewpoints and are isolated intellectually in their own informational bubble. E.g., Google results for "BP" User 1: investment news about British Petroleum User 2: information about the Deepwater Horizon oil spill Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

17 The Filter Bubble [Pariser2011]:... creates the impression that our narrow self-interest is all that exists.... Google and Facebookare offering users "too much candy, and not enough carrots. Book reviewer Paul Boutin: did a similar experiment among people with differing search histories with nearly identical search results. Harvard law professor Jonathan Zittrain: "the effects of search personalization have been light." Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

18 Collaborative Filtering Set of items I, set of users U Users rate items. No need for information about content of items or attributes of users. Users with similar ratings on some items are likely to have similar ratings on further item. Items which are rated similarly by some users are likely to have similar ratings by further users. Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

19 Collaborative Filtering Target User Aggregator Prediction Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

20 Collaborative Filtering Nearest neighbor-based approach [Resnick et al., 1994] find users with history of agreement (similar rating profiles), aggregate their ratings to predict unknown rating. Issues How to define user similarity? How many similar users? How to aggregate the ratings? Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

21 Collaborative Filtering r u, i : (observed) rating of user ufor item i r u,i i r u : mean rating of user u ˆ r, u i: predicted rating of useru for itemi N(u) :set ofusers similar to useru (who have rated itemi) sim( u, v) : similarity of users uand v κ : normalization factor Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

22 Collaborative Filtering Different users use the ratings scale differently. normalize ratings by the mean rating The more similar a user v, the higher the weight of his rating. Rating prediction rˆ u, i = ru + κ sim( u, v) ( rv, i rv ) v N ( u) Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

23 Collaborative Filtering 23 Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM 2011 How to define similarity of users? : set of items rated by both users uand v Pearson correlation coefficient Cosine similarity = uv uv uv I i v i v I i u i u v i v I i u i u r r r r r r r r v u sim 2, 2,,, ) ( ) ( ) )( ( ), ( = uv uv uv I i i v I i i u i v I i i u r r r r v u sim 2, 2,,, ), ( I uv Intro Recommenders Recommendation in SNs Mining SNs Memory based Model based Link prediction Distrust Summary

24 Collaborative Filtering So far: user-based CF Item-based CF is dual approach [Sarwar et al., 2001]. r i : mean rating of item i N(i) :set ofitems (rated by user u) similar to itemi sim( i, j) : similarity of itemsi andj Rating prediction rˆ u, i = ri + κ sim( i, j) ( ru, j rj ) j N ( i) Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

25 Collaborative Filtering So far: memory-based CF Model-based CF: learn model in training phase, apply model in test phase to predict rating. R(u) : set of ratings of user u Rating scale [1..n] Probabilistic model n ˆ = u, = r= 1 u i =, i r P( ru i r R( u)) How to compute P( r, r R( u))? r Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

26 Collaborative Filtering Idea: ratings of items depend on their location in a latent factor space. [Koren et al., 2009] Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

27 Collaborative Filtering Probabilistic matrix factorization [Salakhutdinov et al., 2007] Assumption: observed ratings are governed by latent variables (factors). N: number of users M: number of items K: number of factors, K << M, K << N U u : latent factor (vector) of user u V i : latent factor (vector) of item i Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

28 Collaborative Filtering Probabilistic matrix factorization (MF)(cont.) T 2 P( r = r U, V ) = Ν( r U V, σ ) σ u, σ V, σ R : normal priors Assumption: item ratings are independent from each other I R u, i = u, i u i u, i u i R 1,if rui observed 0, otherwise Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

29 Collaborative Filtering MF(cont.) Parameter learning through maximum likelihood estimation Equivalent to minimizing the error function Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

30 Collaborative Filtering MF(cont.) MF-based CF typically outperforms NN-based CF [Korenet al., 2009]. MF can naturally incorporate biases and additional data sources [Koren et al., 2009]. But latent factors are hard to interpret. Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

31 Content-based Recommenders Set of items I, set of users U. Given user profiles, describing the users tastes, preferences and needs. Given item profiles, characterizing the content of item. Top-N recommendation by ranking items w.r.t. similarity of item profiles and user profile [Balabanovic 1997]. Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

32 Content-based Recommenders Item profile: typically frequencies of kselected keywords. f i, j: frequency of keyword iin item j n i : number of items containing keyword i Term frequency / inverse document frequency TF i, Profile for item i: j = max f i, z j f z, j IDF i = log M n w i, j = TF i, j IDF i content ( i) = ( w, i,..., w k, 1 i i ) Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

33 Content-based Recommenders User profile: typically importance or frequencies of keywords, e.g. aggregation of profiles of items liked by user. Similarity of item iand user u sim ( u, i) = k l = 1 k l = 1 w w 2 l, u l, u w l, i k l = 1 w 2 l, i Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

34 Hybrid Recommender Systems Combine collaborative and content-based method. Approach 1: combine separate recommenders Combine results, e.g. using linear combination or voting. [Pazzani 1999] Approach 2: add aspects of content-based method to CF. [Pazzani 1999] E.g., use profiles to compute similarities between users or items. Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

35 Hybrid Recommender Systems Approach 3: add aspects of CF to contentbased method [Soboroff et al., 1999]. E.g., perform dimensionality reduction on group of content-based profiles. Approach 4: Unified recommendation model E.g., combine topic model, i.e. Latent Dirichlet Allocation, with MF. Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

36 Hybrid Recommender Systems Latent Dirichlet Allocation (LDA) [Blei et al., 2003] Assumption: documents have latent topic distribution, topics have word distributions. Graphical model Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM 2011 θ(latent) topic distribution Z (latent) topic W (observed) word αprior for topic distribution βword distributions N number of words per document M number of documents 36

37 Hybrid Recommender Systems Collaborative topic regression [Wang et al., 2011] Idea: latent item factors (V) depend on topic distribution (θ) of item. Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

38 Performance Evaluation Cross-validation on offline dataset Withhold subset of ratings (test set) Test ( U I ),i.e. Test = {( u, i), ( v, j),...} Use remaining ratings to train recommender (training set) Compare the withheld ratings against the predicted ratings, compute error measure Standard evaluation in research Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

39 Performance Evaluation Cross-validation on offline dataset (cont.) Measures for rating prediction Mean absolute error 1 MAE = Test ( u, i) Test Root mean square error RMSE = ( rˆ u, i ( u, i) Test Test rˆ u r, i u, i u, i Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM 2011 r ) 2 39

40 Performance Evaluation Cross-validation on offline dataset (cont.) Measures for top-n recommendation Recall (or coverage) TopN: set of the top-n recommendations (by algorithm) TestTop: set of all elements of the test set that are among the top-n items for the user Recall = TopN TestTop TestTop Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

41 Performance Evaluation Limitations Measures only accuracy of recommendations Does not measure other aspects such as diversity Does not measure how recommendations change user behavior this is the ultimate goal of a recommender! Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

42 Performance Evaluation In industry Want to evaluate user satisfaction and business profit A/B testing in online system Evaluation measures click-through rate usage return rate of customers profit Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

43 Challenges Privacy-preservation How to make good recommendations without violating privacy concerns? Diversity of recommendations Recommenders tend to suggest more of the same. Explanation of recommendations Necessary to build trust into the recommender. Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

44 Challenges Recommendations for cold-start users i.e. users with very few ratings Typically, ~ 50% of users cold-start. CF fails, because there are no similar users (userbased CF) and no item ratings to aggregate (itembased CF). Recommendations for cold-start items i.e. items with very few ratings Typically, ~ 50% of items cold-start. CF fails, because there are not enough ratersfor the item. Jamali& Ester: Content-based Mining Social Networks for Recommendation, method Tutorial works. at ICDM

45 Social Networks Social network [Wasserman et al., 1994] Used widely in the social and behavioral sciences, in economics, marketing,... directed or undirected graph nodes: actors edges: social relationships or interactions Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

46 Social Networks Different types of social relationships Different types of interactions Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

47 Social Networks Explicit social network relationships provided by users Implicit social network relationships inferred from user actions network Co-worker network Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

48 Social Networks The formation and evolution of social networks is affected by many effects, including Self-interest, Social and resource exchange, Balance, Homophily, Proximity. [Monge & Contractor 2003] Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

49 Trust Networks Related concept: trust network [Golbeck 2005] Trust network allows users to systematically document their trust-relationships, see which users have declared trust in another user. Connected users do not necessarily have a social relationship. Trust in a user may be based, e.g., on articles or reviews authored by that user. Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

50 Online Social Networks Emergence of online social networks Among the top websites [Alexa2011] FaceBook Twitter LinkedIn Availability of very large datasets Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

51 Social Rating Networks Social rating network (SRN): social network, where users are associated with item ratings. Item ratings can be numeric [1..5] or Boolean (bookmark photo, like article,...). Examples: Epinions, Flixster, last.fm, flickr, Digg. Social action: create social relationship, rating action: rate an item. Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

52 Social Rating Networks Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

53 Effects in Social Rating Networks Social influence: ratings are influenced by ratings of friends, i.e. friends are more likely to have similar ratings than strangers. Correlational influence: ratings are influenced by ratings of actors with similar ratings, i.e. if some ratings are similar, further ratings are more likely also to be similar. Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

54 Effects in Social Rating Networks Selection (homophily): actors relate to actors with similar ratings, i.e. actors with similar ratings are more likely to become friends. Transitivity: actors relate to friends of their friends, i.e. actors are more likely to relate to indirect friends. Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

55 Recommendation in Social Networks Benefits of social network-based recommendation: - Exploit social influence, correlational influence, transitivity, selection. -Can deal with cold-start users, as long as they are connected to the social network. -Are more robust to fraud, in particular to profile attacks. Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

56 Recommendation in Social Networks Challenges Low probability of finding rater at small network distance. Noisy ratings at large network distances. Social network data is very sensitive. Edges in online social networks are of greatly varying reliability / strength. Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

57 Mining Social Networks Lots of research in various directions, e.g. community identification, maximization of social influence, etc. Here only mining methods relevant for recommendation: Analysis of social influence, Models of social rating networks, Inference of social networks. Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

58 Influence and Correlation [Anagnostopoulos et al., 2008] Goal: does a SN exhibit social influence? Discrete time period [0..T], consider only one action, e.g. using a certain tag. At every time step, each user flips a coin to decide whether he will get active. Probability of activation depends only on number a of already active friends: p α ln( a+ 1) + β ( a) = α ln( a+ 1 ) + β 1+ e Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM 2011 e 58

59 Influence and Correlation α measures social correlation Y a,t : number of users with aactive friends at time t-1 who get activated at time t N a,t : number of users with aactive friends at time t-1 who do not get activated at time t Y = = a Ya, t, Na Na, t t t Compute αand βthat maximize the data Y a likelihood p ( a) (1 ( p( a)) Na a Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

60 Influence and Correlation If social influence does not play a role, then the timing of activation should be independent of the timing of activation of other users. W = { w 1,..., w l } : set of active users at time T t i : activation time of user i Shuffle test Perform random permutation πof {1,..., l}. Set activation time of user ito t = t. i' : π ( i) Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

61 Influence and Correlation Compute α for original activation times. Compute α for shuffled activation times. If α and α are close to each other, then the model exhibits no social influence. α for original activation times vs. α for shuffled activation times on Flickr dataset Social correlation, but no social influence! Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

62 Feedback Effects Between Similarity and Social Influence [Crandall et al., 2008] Goal: characterize how social influence and selection work together to affect users actions and interactions. Wikipedia dataset Actions: editing article, Interactions: editing the discussion page of another user. Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

63 Feedback Effects Between Similarity and Social Influence How does the similarity of two users vary around the time of their first interaction? Sharp increase in similarity immediately before first interaction Continuing but slower increase after first interaction Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

64 Feedback Effects Between Similarity and Social Influence Generative model for social network Users are associated with history of their actions and corresponding time stamps. Options for generating next action for user u Sample from u sown history. Sample from history of a friend of u. Sample from history of any user. Perform a new action. Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

65 Feedback Effects Between Similarity and Social Influence Options for generating next interaction for user u Sample from users with similar history of actions. Use weighted Jaccard coefficient to measure similarity. Sample a random user to interact with. Estimate model parameters from data Some parameters can be observed. Others are estimated by maximum likelihood estimation. Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

66 Modeling Social Rating Networks [Jamali et al., 2011] Goal: generative model considering all four effects between social actions and rating actions, i.e. social influence, selection, transitivity and correlational influence. What about other effects? E.g., demographic attributes, location. Corresponding data not observed, modeled as random background effect. Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

67 Modeling Social Rating Networks Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

68 Modeling Social Rating Networks Generation of actions similar to [Crandall et al., 2008], but also transitivity and correlational infl. Temporal dynamics of effects, e.g. new user Epinions Flickr Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

69 Modeling Social Rating Networks a k : k th action, ordered by timestamps S t : state of SRN at time t ϴ: set of model parameters Data likelihood Likelihood of social action Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

70 Modeling Social Rating Networks Likelihood of rating action Parameter learning Maximum likelihood estimation EM algorithm Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

71 Modeling Social Rating Networks Experimental results Φ t,1 =0.91 in Epinionsand Φ t,1 =0.9 in Flickr Transitivity is the strongest effect for social actions. Φ r,1 =0.59 in Epinionsand Φ r,1 = 0.54 in Flickr Social influence is the strongest effect for rating actions. Comparison partners include CrossModel, similar to [Crandall et al., 2008] SocialOnly, similar to [Leskovec et al., 2008] Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

72 Modeling Social Rating Networks Growth of similarity after creation of social relationship Flickr Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

73 Modeling Social Rating Networks Average network distance of users before creation of social relationship Epinions Flickr Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

74 Inferring Social Relationships and Their Strength So far: (Boolean) social network given. Sometimes, no information about social relationships, only user actions Inference of social network from user actions [Gomez-Rodriguez et al., 2010] Inference of weighted social network [Myers et al., 2010] Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

75 Inferring Social Networks [Gomez-Rodriguez et al., 2010] Goal: infer social relationships from user actions with time stamps. Assumption: there is a latent, static network over which influence propagates. t u : activation time of user u, i.e. time when user u gets activated ( infected ) by a cascade Cascade cspecified through activation times of all users: c [ t 1,..., t n ], possibly t = = i Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

76 Inferring Social Networks Independent Cascade model: activated node activates each of his friends with a given probability P c ( u, v) : probability of cascade cspreading from user uto user v = tv t u ( u, v) decreases with increasing P c P c 1 α or Pc ( u, ) α ( u, v) e v Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

77 77 Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM 2011 Inferring Social Networks C: set of all given cascades G: inferred directed graph over node (user) set U={1,,n} T(G): set of all subtreesof G Problem: compute Gwith at most kedges that maximizes the likelihood = T v u c v u P T c P ), ( ), ( ) ( = = T v u c C c G T T C c G T T v u P T c P G C P ), ( ) ( ) ( ), ( max ) ( max ) ( ) ( G C P Intro Recommenders Recommendation in SNs Mining SNs Memory based Model based Link prediction Distrust Summary

78 Inferring Social Networks Improvement of log-likelihood over empty graph E: F ( G) = max log P( c T ) max log T T ( G) Equivalent problem c T T ( E) P( c T ) Problem is NP-hard. F C (G) is submodular, which means that a greedy algorithm gives a constant-factor approximation of the optimal solution. NetInf algorithm Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

79 Inferring Social Networks Precision and recall for synthetic datasets Spreading probability PL: power law, Exp: exponential Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

80 Inferring Weighted Social Networks [Myers et al., 2010] NetInfis very accurate for homogeneous networks, i.e. networks where all connected nodes influence ( infect ) each other with the same probability. For inhomogeneous networks, define A ij = P( node i infects node j node i is infected) Goal: learn the matrix A = [A ij ] from the observed set of cascades C={c 1,..., c n } Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

81 Inferring Weighted Social Networks If ibecomes infected, then jwill be infected with probability A ij. w(t): transmission time model probability distribution of the transmission time from one node to a friend c τ i : time of infection of node iby cascade c time of infection of i sfriend jby cascade c τ c c j = τ i + t, where t ~ w( t) Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

82 Inferring Weighted Social Networks Likelihood of observed cascades C given a weight matrix A P( C A) = c C 1 (1 w( τ τ ) A Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM 2011 ) (1 A ) c c < < = < i j ji ji c c c c c i: τ i j: τ j τ i i: τ i j: τ j First term: one factor for each infected node i, assuming that at least one of his friends jwho was infected earlier infected him Second term: one factor for each non-infected node i, assuming that none of the infected friends jinfected him 82

83 Inferring Weighted Social Networks Parameter learning through maximum likelihood estimation If inever infected j, then A ij := 0 and do not need to learn it. Translate into a convex optimization problem: Finds globally optimal solution. Can use efficient convex optimization methods. ConNIe algorithm Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

84 Inferring Weighted Social Networks Precision vs. recall and mean square errors vs. number of edges for synthetic datasets Transmission time model PL: power law, Exp: exponential, WB: Weibull. Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

85 Mining Social Networks for Recommendation Tutorial at ICDM 2011 MohsenJamali& Martin Ester Part 2 Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

86 Outline Introduction Recommender systems Recommendation in social networks Mining social networks Methods for Recommendation in Social Netorks Memory based approaches Model based approaches Link prediction Social Networks with distrust Summary Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

87 Recommendation in SNs: Problem Definition Input Rating matrix Real valued or binary Social network Weighted or binary Social rating network (SRN) A social network in which users can express ratings on items Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

88 Recommendation in SNs: Problem Definition Rating prediction problem For a given user uand the target item i: Predict the rating r u,i. Top-N item recommendation For a given user urecommend top N items desirable for him [Deshpande et al. 2004]. Mostly neglected in the literature Top-N recommendation has been investigated in traditional recommender system. In social networks, there are very few works [Jamali et al b]. Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

89 Data Sets for Recommendation in SNs Epinions Online product review Explicit notion of trust Users review and rate products in different categories. Users express trust on other reviewers. 50K users, 140K items, 650K ratings, 480K links 70K users, 105K items, 575K ratings, 500K links 50 % cold start Less than 5 ratings Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

90 Data Sets for Recommendation in SNs Flixster Social metworking service for rating movies Friendship relations 1M users, 50K items, 8M ratings, 26M links 85% of users have no ratings 50% of rater are cold start Less than 5 ratings Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

91 Models for Recommendation in SNs Memory based approaches Explore the social network for raters Aggregate the ratings to compute prediction Store the social rating network No Learning phase Slow in prediction Most pioneer works for recommendation in SN are memory based approaches. Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

92 Models for Recommendation in SNs Model based approaches Learn a model Store the model parameters only Extra time for learning Fast in Prediction Most models are based on matrix factorization Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

93 Memory based Approaches for Recommendation in Social Networks Explore the network to find raters in the neighborhood Aggregate the ratings of rater to compute the predicted rating. There are different methods to calculate the top trusted neighborhood of users. Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

94 Advogato [Levien et al., 2002] A trust metric to compute the top N trusted users Input n: the number of users to trust x: the source users who want to trust A maximum flow based approach Advogatocan be used to find the neighborhood in rating prediction. Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

95 Node Capacities: source user: n user at level l capacity at l-1 average out-degree from l-1 Advogato(cont.) Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

96 Advogato(cont.) To apply Ford-Fulkerson algorithm for maximum flow analysis, we should have Single source, single sink Capacities on edges Graph transformation Super sink Split nodes into two nodes Node capacity c Edge with c-1 from negative to positive node Edge with capacity 1 from negative to super sink Infinite capacity for regular edges Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

97 Advogato(cont.) Maximum flow computed from source-negative to super sink Nodes with flow to super link Top n trusted users Recommendation No distinction among the top trusted users Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

98 Trust Metric AppleSeed [Ziegler 2005] Intuition: Spreading activation model Source node uis activation through injection of energy e. Energy is fully propagated through edges Proportional to the edge weights Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

99 AppleSeed(cont.) Nodes are ranked according to the energy they receive Issue: Trust is considered to be additive Nodes with many weakly trusted paths are considered to be highly trusted Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

100 TidalTrust [Golbeck 2005] Modified breadth first search in the network Consider all raters v at the shortest distance from the source user u. Trust between u and v Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

101 Predicted rating TidalTrust(cont.) Only considers raters at the shortest distance: Loss of information Lower recall Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

102 MoleTrust [Massa et al., 2007] Similar to the idea in TidalTrust Considers raters up to a maximum-depth d. Backward exploration in trust computation Tuning d: Trade-off between accuracy and recall Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

103 Memory based Approaches for Recommendation in Social Networks How far to go into network? Tradeoff between Precision and Recall Far neighbors on the exact target item Trusted friends on similar items Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

104 TrustWalker [Jamali et al., 2009] Random walk based model Combines item-based recommendation and trust-based recommendation. Performs several random walks on the social network. Each random walk returns a rating on the exact target item or a similar item. Prediction = aggregate of all returned ratings Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

105 TrustWalker Single Random Walk Starts from source user u 0. At step k, at node u: If uhas rated i,return r u,i With Φ u,i,k, the random walk stops Randomly select item jrated by uand return r u,j. With 1-Φ u,i,k, continue the random walk to a direct neighbor of u. Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

106 TrustWalker(cont.) Item similarities Φ u,i,k Similarity of items rated by uand target item i. The step of random walk Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

107 TrustWalker(cont.) Special cases of TrustWalker Φ u,i,k = 1 Random walk never starts. Item-based Recommendation. Φ u,i,k = 0 Pure trust-based recommendation. Continues until finding the exact target item. Aggregates the ratings weighted by probability of reaching them. Existing methods approximate this. Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

108 TrustWalker Extension TrustWalkercan be applied to recommend a list of top-n items [Jamali et al., 2009.b] Every random walk stops at a user v. All items ranked highly by v are returned as the result of the random walk. Result of several random walks are merged into a list of top-n items. Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

109 Memory based Approaches: Experiments RMSE Results on Epinions Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

110 Memory based Approaches: Experiments Result for cold start users on Epinions Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

111 Memory based Approaches: Experiments Result for cold start users on Epinions Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

112 Memory based Approaches: Experiments Result for all users on Epinions Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

113 Memory based Approaches: Experiments Result for all users on Epinions Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

114 Memory based Approaches: Experiments RMSE Results on Flixster Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

115 Memory based Approaches: Summary Not learning a model No Learning phase Explore the network to find raters Need to store the SRN Slow in the prediction phase due to exploration Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

116 Model based approaches for Recommendation in SNs Recently have attracted attention Most common approach: Matrix factorization Latent features for users Latent features for items Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

117 SoRec [Ma et al., 2008] Matrix factorization model Factorize the ratings and links together Social network as a binary matrix Latent factors for items (as in MF) Twolatent factors for users One for the initiator One for the receiver Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

118 FIP [Yang et al., 2011] Factorizes both rating matrix and the social network Similar to the idea in SoRec Assumes undirected network FIP vs. SocRec SocRec: directed graph FIP uses features as priors Choice of the user factor determining the observed rating is arbitrary Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

119 Social Trust Ensemble [Ma et al., 2009] Social Trust Ensemble (STE) Linear Combination of Basic matrix factorization and Latent factors of the user and the item determine the observed rating. Social network based approach Latent factors of the neighbors and the latent factor of the item determine the observed rating. Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

120 Social Trust Ensemble (cont.) The STE model Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

121 Social Trust Ensemble (cont.) The STE model Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

122 Social Trust Ensemble (cont.) Issues with STE Latent factors of neighbors should influence the latent factor of unot his ratings STE does not handle trust propagation Learning is based on observed ratings only Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

123 SocialMF [Jamaliet al., 2010] Social influence behavior of a user uis affected by his direct neighbors N u. Latent characteristics of a user depend on his neighbors. T u,v is the normalized trust value. Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

124 SocialMF(cont.) Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

125 SocialMF(cont.) Properties of SocialMF Trust propagation Learning the user latent factor is possible with existence of the social network only No need to fully observed rating for learning Appropriate for cold start users and users with no ratings Similar ideas [Ma et al. 2011] Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

126 Results for Epinions Gain over STE: 6.2%. for K=5 and 5.7% for K=10 Mohsen Jamali, Social Matrix Factorization 126

127 Results for Flixster SocialMFgain over STE (5%) is 3 times the STE gain over BasicMF(1.5%) Mohsen Jamali, Social Matrix Factorization 127

128 Experiments on Cold Start Users Mohsen Jamali, Social Matrix Factorization 128

129 Analysis of Learning Runtime SocialMF: STE: SocialMFis faster by factor N K r t # of Users Latent Feature Size Avg. ratings per user Avg. neighbors per user Mohsen Jamali, Social Matrix Factorization 129

130 Generalized Stochastic Block Model for Recommendation in Social Networks [Jamali et al., 2011] Social influence and selection lead to formation of communities/groups Users may belong to different groups in their actions Teacher interacting with students or his/her son Digital Camera when being rated Clustering based methods for recommendation Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

131 GSBM (cont.) Extending mixed membership stochastic block model [Airoldi et al. 2008] Users probabilistically act as a member of one of the groups in their actions. Every item is considered to belong to a latent group when it is being rated. The relation between users and items is governed by the relation between groups. 131

132 GSBM Graphical Model 132

133 GSBM (cont.) Sample the social relation, g2 K1 K1 g1 P(g1-->g2) B T 133

134 GSBM : Experiments on Rating Prediction Epinions Flixster 134

135 GSBM: Experiments on Link Prediction Epinions Flixster 135

136 Link Prediction Emergence of online social network The need to get connected to other people led to link prediction Problem Definition Given a user pair (u,v), estimate the probability of creation of the link u v Given a user u, recommend a list of top users for u to connect to. Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

137 Link Prediction (cont.) Link prediction vs. Rating prediction Strength of a relation between a users and another user Strength of a relation between a users and an item Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

138 Link Prediction Methods Pair-wise similarity based approach Roots in social selection Users with highest similarity to uare recommended to u. Every user uis represented by his/her observed features, properties and past activities such as ratings and clicks. Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

139 Link Prediction- Similarity based Methods Defining similarity measure between A and B the ratio between the amount of information needed to state the commonality of A and B and the information needed to fully describe what A and B are [Lin 1998]: Special Cases: Cosine similarity Pearson correlation Jaccard s coefficient Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

140 Link Prediction Methods (cont.) Network Topology based methods Common neighbors Jaccard s coefficient [Adamicand Adar 2003] Preferential attachment [Newman 2001] Initially proposed for modeling network growth Measure similarity, based on direct neighbors Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

141 Link Prediction Path based Methods Katz [Katz 1953]: path l A,B: number of paths of length lfrom Ato B Hitting time [Liben-Nowell et al., 2003] score(a,b): Average number of steps for a random walk to reach Bstarting from A Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

142 Link Prediction Path based Methods Random walk with restart [Pan et al., 2004] A random walk starts from A. At each step, with probability α the random walk restarts score(a,b): probability of being at B during the random walk. SimRank[Jehet al., 2002] Two user are similar to the extent that they are joined to similar neighbours. Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

143 Link Prediction Methods (cont.) MF based models [Rennieet al., 2005] Social network as a binary matrix Similar to MF methods for rating prediction Factorize the network matrix into product of lower rank matrices (representing user factors) Advanced version in [Yang et al., 2011] Latest advances: Supervised random walks [Backstrom et al., 2011] Random walk based approach Considering properties of links and user attributes Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

144 Analyzing Social Networks with Distrust Relations between users on social media sites often reflect a mixture of positive and negative interactions [Leskovec et al., 2010]. User can express distrust on other users E.g. block some users in ebay, Google+ Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

145 Analyzing Social Networks with Distrust (cont.) Few works have addressed negative relations [Leskovec et al., 2010] [Kunegiset al., 2009] [Brzozowski et al., 2008] [Guhaet al., 2004] Prior work shifted the trust to avoid negative values [Kamvaret al., 2003] Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

146 Analyzing Social Networks with Distrust (cont.) How does distrust propagate? d b e a c f Distrust propagates only one step [Guhaet al., 2004] g Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

147 Analyzing Social Networks with Distrust (cont.) How does distrust propagate? a trust distrust c b Distrust propagates only one step [Guhaet al., 2004] d e f g a a a a?? d e f g Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

148 Analyzing Social Networks with Distrust (cont.) Signed network can be analyzed according to two different theories [Leskovec et al., 2010]: Structural balance theory Originated in social psychology Triangles with three positive signs (three mutual friends,t3) and those with one positive sign (two friends with a common enemy, T1) are more plausible. [Leskovec et al., 2010] Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

149 Analyzing Social Networks with Distrust (cont.) Signed network can be analyzed according to two different theories [Leskovec et al., 2010]: Theory of Status Positive(negative) directed link indicate that the creator of the link views the recipient as having higher(lower) status These relative levels of status can be propagated along multi-step paths of signed links Leads to different predictions than balance theory. Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

150 Analyzing Social Networks with Distrust (cont.) Which theory does better explain the relations among users in a signed social network? [Leskovec et al., 2010] In undirected networks, structural balance theory In directed signed networks, theory of status Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

151 Recommendation in Social Networks with Distrust How can distrust be exploited in recommendation? Very few works addressed this problem [Ma et al., 2009.b] Matrix factorization model Modified objective (error) function Maximizing the distance between factor of uand his/her distrusted neighbor v Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

152 Recommendation in Social Networks with Distrust (cont.) [Ma et al., 2009.b] D + : set of users that u distrusts : distrust score Experiments on Epinions show promising results Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

153 Summary State-of-the-art methods for recommendation in social networks Memory based approaches ModelTrust[Massa 2007], Modified BFS TidalTrust[Golbeck 2005], Modified BFS TrustWalker[Jamali et al., 2009], Random Walk Model based approaches SoRec[Ma et al., 2008], Matrix Factorization FIP [Yan et al., 2011], Matrix Factorization STE [Ma et al., 2009], Matrix Factorization SocialMF[Jamali et al., 2010], Matrix Factorization GSBM[Jamali et al., 2011], Stochastic BlockModel Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

154 Summary Link Prediction Pair-wise profile similarity approaches Information theoretic based definition of similarity Network topology based approaches Common neighbors Path based approaches Katz, Hitting time, RWR, SimRank Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

155 Summary Social Networks with distrust Propagation of distrust Theories behind distrust Recommendation with distrust [Ma et al., 2009.b] Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

156 Future Research Directions Exploring other machine learning models Privacy of recommendation in social networks How to preserve privacy while employing social networks? Improving the diversity of recommendations How to evaluate the diversity? Recommendation of cold-start items They are very important! Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

157 Future Research Directions Recommendation in mobile social networks Distributed algorithm How to exploit the user location? Recommendation in social networks with documents (posts) E.g., Twitter Integration with topic models Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM

158 Thank You! Jamali& Ester: Mining Social Networks for Recommendation, Tutorial at ICDM