Mobile Apps Jovian Lin, Ph.D.

Transcription

1 7 th January 2015 Seminar Room 2.4, Lv 2, SIS, SMU Recommendation Algorithms for Mobile Apps Jovian Lin, Ph.D.

2 1. Introduction 2

3 No. of apps is ever-increasing 1.3 million Android apps on Google Play (as of August 2014) 1.2 million ios apps in Apple s App Store (as of June 2014) No. of apps Time 3

4 No. of apps is ever-increasing Good J Bad L Myriad of unique and useful apps Difficult to find relevant apps for users (i.e., information overload) 4

5 Introduction Methodologies New Contributions Future Directions Recommender Systems Recommender Algorithms for Mobile Apps Jovian Lin 7th Jan SMU 5

6 2/32 Recommender Systems Collaborative Filtering Content-based Filtering 6

7 Recommender Systems Collaborative Filtering Content-based Filtering items new items ?? 2 5 4??? users 3 4 1? 5? 5????? 5 1? 4?? 7

8 Recommender Systems Collaborative Filtering Content-based Filtering 1 items new items 4 5?? 2 5 4??? COLD START PROBLEM 3 1? 5?? 4? 5??? users 5 1? 4?? Solution #1: Wait for ratings to come in. Solution #2: Use content-based filtering. 8

9 Recommender Systems Collaborative Filtering Content-based Filtering Recommends items based on similar content Drawback lack of variety L (e.g., textual descriptions) Current App Recommended 9

10 Other Existing App Recommender Techniques Collect information from device: Xu et al. (2011) investigated the usage behaviors of individual apps by using anonymized network measurements. Yan and Chen (2011) and Costa-Montenegro et al. (2012) analyzed usage patterns in their own recommender systems. Collect external information: Zheng et al. (2010) and Davidsson and Moritz (2011) used GPS information for context-aware app recommendation. Yin et al. (2013) calculated how likely a user will replace an existing app in his/her device. Bhandari et al. (2013) proposed a graph-based method for recommending serendipitous apps. 10

11 Our Scope Fortunately, the mobile app domain has some unique properties, for example: 1. Apps have their own Twitter accounts E.g., AngryBirds has an on Twitter 21.1% of apps have Twitter accts (dataset of 33,802 apps) 2. Apps contain various version information. E.g., Ver 1, Ver 1.1, Ver 1.2.1, Ver % of apps have at least 5 versions 11

12 Our Objective Recommender Systems Mobile App Domain Improve app recommender systems by capitalizing on the unique properties of the app domain 12

13 Overview 1. We use nascent signals in microblogs (e.g., Twitter) to alleviate the cold-start problem in mobile app recommendation (in SIGIR 13). 2. We use version features to improve mobile app recommendation (in SIGIR 14). 13

14 2. Methodologies 14

15 Part #1: Utilizing Data from Social Media We made two observations: 1. Apps contain references to their Twitter accounts. 15

16 16

17 17

18 on Twitter 18

19 Recommender Algorithms for Mobile Apps Jovian Lin 7th Jan SMU 19

20 Part #1: Utilizing Data from Social Media We made two observations: 1. Apps contain references to their Twitter accounts. 2. Early signals about apps can be present in social networks, even before ratings are received. 20

21 Part #1: Utilizing Data from Social Media We made two observations: 1. Apps contain references to their Twitter accounts. 2. Early signals about apps can be present in social networks, even before ratings are received. Has an account on Twitter since Feb 2008 By May 2012, Evernote s Twitter account already had 120,000 followers and 1,300 tweets. Evernote ios app Release Date: 8 May ratings 0 ratings First few ratings start coming in 118,827 ratings May 2012 Jun 2012 Jul 2012 Dec

22 Part #1: Utilizing Data from Social Media How is it done? 22

23 Part #1: Utilizing Data from Social Media We want to estimate the probability that a target user u will like an app a. p( + a, u ) like app user 23

24 Part #1: Utilizing Data from Social Media We want to estimate the probability that a target user u will like an app a. p( + a, u ) = p( + t, u) p( t a) t T(a) like app user Twitter-follower Uniform distribution over the various Twitter-followers (t) following app a. Pseudo-documents & Pseudo-words 24

25 Part #1: Utilizing Data from Social Media downloaded/consumed 25

26 Part #1: Utilizing Data from Social Media downloads User u 26

27 Part #1: Utilizing Data from Social Media downloads User u twitterid_31230 twitterid_2289 twitterid_999 twitterid_50401 twitter.com/angrybirds followers twitterid_2 twitterid_ twitterid_

28 Recommender Algorithms for Mobile Apps Jovian Lin 7th Jan SMU 28

29 Part #1: Utilizing Data from Social Media downloads User u twitterid_31230 twitterid_2289 twitterid_999 twitterid_50401 twitter.com/angrybirds followers twitterid_2 twitterid_ twitterid_

30 Part #1: Utilizing Data from Social Media downloads User u twitterid_31230 twitterid_2289 twitterid_999 twitterid_50401 followers twitterid_2 twitterid_ twitterid_

31 Part #1: Utilizing Data from Social Media downloads User u twitterid_31230 twitterid_2289 twitterid_999 twitterid_50401 followers twitterid_2 twitterid_ twitterid_

32 Part #1: Utilizing Data from Social Media User u twitterid_31230 twitterid_2289 twitterid_999 twitterid_50401 twitterid_2 twitterid_ twitterid_

33 Part #1: Utilizing Data from Social Media User u Pseudo-Document twitterid_31230 twitterid_2289 twitterid_999 twitterid_50401 Pseudo-Words twitterid_2 twitterid_ twitterid_

34 Part #1: Utilizing Data from Social Media Followed by: twitterid 10 twitterid 12 Twitter-follower ID Preference indicator disliked App a (twitterid 10, DISLIKED) (twitterid 12, DISLIKED) User u liked App b Followed by: twitterid 10 twitterid 12 twitterid 29 (twitterid 10, LIKED) (twitterid 12, LIKED) (twitterid 29, LIKED) liked Followed by: twitterid 29 twitterid 31 (twitterid 29, LIKED) (twitterid 31, LIKED) Pseudo-document u App c 34

35 Part #1: Utilizing Data from Social Media Twitter-follower ID Preference indicator (twitterid 10, DISLIKED) (twitterid 12, DISLIKED) (twitterid 10, LIKED) (twitterid 12, LIKED) (twitterid 29, LIKED) (twitterid 29, LIKED) (twitterid 31, LIKED) The concept of pseudo-documents and pseudo-words does not apply exclusively to Twitter followers. Pseudo-document u 35

36 Part #1: Utilizing Data from Social Media Genre ID Preference indicator (genreid 10, DISLIKED) (genreid 12, DISLIKED) (genreid 10, LIKED) (genreid 12, LIKED) (genreid 29, LIKED) (genreid 29, LIKED) (genreid 31, LIKED) The concept of pseudo-documents and pseudo-words does not apply exclusively to Twitter followers. Pseudo-document u 36

37 Part #1: Utilizing Data from Social Media Word ID Preference indicator (wordid 10, DISLIKED) (wordid 12, DISLIKED) (wordid 10, LIKED) (wordid 12, LIKED) (wordid 29, LIKED) (wordid 29, LIKED) (wordid 31, LIKED) The concept of pseudo-documents and pseudo-words does not apply exclusively to Twitter followers. Pseudo-document u 37

38 Part #1: Utilizing Data from Social Media Construct Latent Groups (twitterid 10, DISLIKED) (twitterid 12, DISLIKED) (twitterid 10, LIKED) (twitterid 12, LIKED) (twitterid 29, LIKED) (twitterid 29, LIKED) (twitterid 31, LIKED) Pseudo-document u 38

39 Part #1: Utilizing Data from Social Media (twitterid 10, DISLIKED) (twitterid 12, DISLIKED) Per-document topic distribution (twitterid 10, LIKED) (twitterid 12, LIKED) (twitterid 29, LIKED) LDA (twitterid 29, LIKED) (twitterid 31, LIKED) Pseudo-documents Per-topic word distribution 39

40 Part #1: Utilizing Data from Social Media (twitterid 10, DISLIKED) (twitterid 12, DISLIKED) Per-document topic distribution (twitterid 10, LIKED) (twitterid 12, LIKED) (twitterid 29, LIKED) LDA (twitterid 29, LIKED) (twitterid 31, LIKED) Pseudo-documents Per-topic word distribution p( + t, u) = p( +, t z) p( z u) z Z 40

41 Part #1: Utilizing Data from Social Media (twitterid 10, DISLIKED) (twitterid 12, DISLIKED) Per-document topic distribution (twitterid 10, LIKED) (twitterid 12, LIKED) (twitterid 29, LIKED) LDA (twitterid 29, LIKED) (twitterid 31, LIKED) Pseudo-documents Per-topic word distribution p( + t, u) = p( +, t z) p( z u) z Z Per-topic word distribution Per-document topic distribution 41

42 Part #1: Utilizing Data from Social Media (twitterid 10, DISLIKED) (twitterid 12, DISLIKED) Per-document topic distribution (twitterid 10, LIKED) (twitterid 12, LIKED) (twitterid 29, LIKED) LDA (twitterid 29, LIKED) (twitterid 31, LIKED) Pseudo-documents Per-topic word distribution p( + t, u) = p( +, t z) p( z u) z Z Probability that the presence of Twitter-follower t indicates that it is liked by user u. 42

43 Part #1: Utilizing Data from Social Media p( + t, u) p( t a) 43

44 Part #1: Utilizing Data from Social Media We want to estimate the probability that a target user u will like an app a. p( + a, u ) = p( + t, u) p( t a) t T(a) like app user Probability that the presence of Twitter-follower t indicates that it is liked by user u. Derived from Pseudo-Documents and Pseudo-Words. Uniform distribution over the various Twitter-followers (t) following app a. 44

45 Part #1: Utilizing Data from Social Media Dataset We collected data from the Apple itunes Store and Twitter during September 2012 to December Stats: 1,289,668 ratings 7,116 apps (with Twitter accounts) 10,133 users. Restrictions: Each user must give at least 10 ratings for apps. Each Twitter ID is related to at least 5 apps. 45

46 Part #1: Utilizing Data from Social Media Simulating the Cold-start 10-fold cross validation. Selected 10% of the apps to be the held out set for all users. Each user has the same within-fold apps. Guarantee that none of these apps are in the training set of any user. 46

47 Part #1: Utilizing Data from Social Media Evaluation Metric Evaluation Metric: Recall Zero ratings are uncertain: i. either the user doesn t know about the app; or ii. iii. the user doesn t like the app (and didn t rate it). Makes it difficult to accurately compute precision. But since the existing ratings are true positives, recall is a more pertinent measure it only considers the positively rated apps. Recall is also used in Wang & Blei (KDD 11). 47

48 Part #1: Utilizing Data from Social Media Three Research Questions 1. How does the performance of Twitter-followers feature compare with other features? 2. How does our method compare with other techniques? 3. Do the latent groups make any sense? What can we learn from them? 48

49 Part #1: Utilizing Data from Social Media RQ1: Twitter-followers Feature VS Other Features 0.7 Ablation Recall All = T = G = D = W = All features Twitter-followers Genres Developers Words Number of recommended apps (M) Pseudo-Docs (W) Pseudo-Docs (D) Pseudo-Docs (G) Pseudo-Docs (T) Pseudo-Docs (All) 49

50 Part #1: Utilizing Data from Social Media RQ1: Twitter-followers Feature VS Other Features (Ablation Study) Feature All features (TGDW) * All, excluding Twitter-followers (GDW) All, excluding Genres (TDW) All, excluding Developers (TGW) All, excluding Words (TGD) Twitter-followers (T) Genres (G) Developers (D) Words (W)

51 Part #1: Utilizing Data from Social Media RQ1: Twitter-followers Feature VS Other Features (Why Words suck?) 51

52 Part #1: Utilizing Data from Social Media RQ2: How does our method compare with other techniques? 0.7 Full Dataset Recall Pseudo-Docs (All) Pseudo-Docs (Twitter) CTR LDA VSM (Twitter) VSM (Words) Number of recommended apps (M) VSM (Words) VSM (Twitter) LDA CTR Pseudo-Docs (Twitter)* Pseudo-Docs (All)** 52

53 Part #1: Utilizing Data from Social Media RQ3: Do the latent groups make any sense? 53

54 Part #1: Utilizing Data from Social Media RQ3: Do the latent groups make any sense? Games, Photo & Video Paper Monsters, Stickman Cliff Diving, Lili, Snoopy s Street Fair, Gizmonauts, etc. Video games warrior, lover of life, eternal student of the universe I m a Multimedia developer working at Kent State Uni! I also do art services for the game industry Agalog Games is an independent ios game studio Hi! Samadhi Games LLC is an Indie Developer of ios, Android, etc 54

55 Part Part #2: #1: Utilizing Data from Version Social Updates Media 55

56 Introduction Methodologies New Contributions Future Directions Part #2: Utilizing Data from Version Updates Change is the only constant (at least in the app domain) Recommender Algorithms for Mobile Apps Jovian Lin 7th Jan SMU 56

57 Part #2: Utilizing Data from Version Updates books, movies, music, etc. : Static apps : Changes (with version updates) 57

58 Part #2: Utilizing Data from Version Updates Version 1.0 App X Version 2.0 Includes High Definition (HD) capabilities An app that was unfavorable in the past may become favorable for a user after a version update. 58

59 Part #2: Utilizing Data from Version Updates Apple has added a new section: Best New Game Updates in their App Store (as of June 2014). Highlights recently updated apps. Easier to discover apps that have just been significantly updated. The inclusion of this new section to apple s ios App Store illustrates the importance of version features. 59

60 Part #2: Utilizing Data from Version Updates App X A version of the app. Legend An ID of a topic. New! Version Version Version Version Version Users Alex Topics Bob Clark 5 Ph.D. Defense Mobile App Recommendation By: Jovian Lin 60

61 Part #2: Utilizing Data from Version Updates App X New! Version Version Version Version Version A version of the app. Legend An ID of a topic Users Alex Topics Bob Clark 5 So if Bob has a keen interest in Topic 5 the chance that he adopts Version 3.0 of App X will be higher. Ph.D. Defense Mobile App Recommendation By: Jovian Lin 61

62 Part #2: Utilizing Data from Version Updates Our Approach 1. Extracting Version Features 2. Generating Latent Topics 3. Identifying Important Latent Topics 4. User Personalization 5. Calculating Version Snippet Score 62

63 Part #2: Utilizing Data from Version Updates 63

64 Part #2: Utilizing Data from Version Updates 1. Extracting Version Features Version Snippets (snippet = document) Version Category Genre Mixture Ratings a rating corresponds to a version of an app. i.e., user u gives version v of app a a numerical rating of r 64

65 Part #2: Utilizing Data from Version Updates 2. Generating Latent Topics Interpret the text in version-snippets/documents. Use topic models (LDA) to achieve this. Transforms text in documents to interpretable representation. Investigate 3 variants of LDA. Each variant employs a different set of version features. Topic Model LDA Textual Description Version Category Genre Mixture Labeled LDA (LLDA) Modifying Corpus Injection LDA/LLDA 65

66 Part #2: Utilizing Data from Version Updates 2. Generating Latent Topics LDA LLDA Injection LDA/LLDA LDA generates: Per-document topic distribution i.e., p(z d) Per-topic word distribution i.e., p(w z) But LDA can t incorporate observed information like: Version-category Genre mixture 66

67 Part #2: Utilizing Data from Version Updates 2. Generating Latent Topics LDA LLDA Injection LDA/LLDA LLDA supervised topic model that uses observed labels as topics [Ramage et al., 2009]. But it can be hacked to become semi-supervised. Semi-supervised: Observed labels = version categories & genre mixture * Latent topics = discovered/generated from descriptions *number of observed labels varies with different documents. 67

68 Part #2: Utilizing Data from Version Updates 2. Generating Latent Topics LDA LLDA Injection LDA/LLDA Enhancing the corpus before using topic models. Generate pseudo-terms from metadata * and incorporate them into each document. Text from the original document Pseudo-terms Enhanced document *metadata = version categories & genre mixture. 68

69 Part #2: Utilizing Data from Version Updates 3. Identifying Important Latent Topics So far, we can model each document as a distribution of topics. But we do not know which topics are important for a recommendation. RECOMMEND Topic Retina/HD graphics App X Topics: - game centre - ipad support - Retina display App Y Topics: - game centre - ipad support 69

70 Part #2: Utilizing Data from Version Updates So far, we can model each document as a distribution of topics. But we do not know which topics are important for a recommendation. Furthermore: 3. Identifying Important Latent Topics Apps are classified into different genres. Each genre works differently to the same type of version update. Topic Retina/HD graphics More relevant Music genre Games genre 70

71 Part #2: Utilizing Data from Version Updates 3. Identifying Important Latent Topics Key components: (i) genres & (ii) topics We weigh every genre-topic pair with w x (g,z) g = genre z = latent topic x {LDA, inj+lda, LLDA, inj+llda} 71

72 Part #2: Utilizing Data from Version Updates 3. Identifying Important Latent Topics Key components: (i) genres & (ii) topics We weigh every genre-topic pair with w x (g,z) w x (g,z) is weighted based on the popularity of the genre-topic pair. Popularity is scored based on existing user ratings. In other words: Relevance/popularity/weight of a genre-topic pair is based on user ratings Topic Retina/HD graphics More relevant Music genre Games genre 72

73 Part #2: Utilizing Data from Version Updates 4. User Personalization Need to know each user s preference w.r.t. the set of topics. For all version v s of apps that user u has consumed: Sum up the probabilities of the set of topics of each v Normalize We get p(z u) i.e., probability of user u being interested in topic z 73

74 Part #2: Utilizing Data from Version Updates 5. Calculating Version Snippet Score We want: score x (d,u), where d = document a.k.a. version-snippet u = user x {LDA, inj+lda, LLDA, inj+llda} i.e., the 4 topic models To calculate score x (d,u): Convert document d into set of topics Integrate it with: genre-topic weights, i.e., w x (g,z) user personalization, i.e., p(z u) 74

75 Part #2: Utilizing Data from Version Updates Evaluation Dataset: App metadata (itunes App Store): App ID Title & Description Genres User ratings (itunes App Store s Reviews) Version descriptions of apps (App Annie) Collected: 9797 users 6524 apps 109,338 versions 1,000,809 ratings 75

76 Part #2: Utilizing Data from Version Updates Evaluation Metric: Recall Zero ratings are uncertain: i. either the user doesn t know about the app; or ii. iii. Evaluation the user doesn t like the app (and didn t rate it). Makes it difficult to accurately compute precision. But since the existing ratings are true positives, recall is a more pertinent measure it only considers the positively rated apps. Recall is also used in Wang & Blei (KDD 11). 76

77 Part #2: Utilizing Data from Version Updates Evaluation Baselines: 1. Collaborative filtering (CF) achieved using probabilistic matrix factorization (PMF). 2. Content-based filtering (CBF) achieved using LDA on textual app descriptions only. 3. Hybrid baselines using Gradient Tree Boosting (GTB): a. CF+CBF (collaborative & content) b. CF+VSR (collaborative & VSR) c. CBF+VSR (content & VSR) d. CF+CBF+VSR (collaborative & content & VSR) * VSR = version sensitive recommendation 77

78 Part #2: Utilizing Data from Version Updates Research Questions 1. Importance of Genre Information 2. Comparison of Different Topic Models 3. Comparison Against Other Techniques (Individual) 4. Comparison Against Other Techniques (Combined) 5. Analysis: Dissecting Specific Topics 78

79 Part #2: Utilizing Data from Version Updates Results #1: Importance of Genre Information (on inj+llda) Compare: WITHOUT genre info VS WITH genre info 1. Genre info is an important discriminatory factor 2. as each genre affects the same type of topic differently. 3. For example: Topic Retina/HD graphics Music genre Without genre information With genre information Games genre 79

80 Part #2: Utilizing Data from Version Updates Results #2: Comparison of Different Topic Models Local comparison between Various Topic Models Supervised LLDA vs LDA vs LLDA vs inj+lda vs inj+llda 80

81 Part #2: Utilizing Data from Version Updates Results #2: Comparison of Different Topic Models Local comparison between Various Topic Models 1. Recall improves as more information is used. 2. Best = inj+llda 3. Both LLDA (yellow & red) models outperform the LDA (blue & green) counterparts. Topic Model LDA Textual Description Version Category Genre Mixture Labeled LDA (LLDA) Modifying Corpus Inj+LDA / inj+llda 4. Because LLDA utilizes: semi-supervised, and use of observed data. 5. Enhancing corpus generally improves recall. inj+llda > LLDA inj+lda > LDA 81

82 Part #2: Utilizing Data from Version Updates Results #3: Comparison Against Other Techniques (Individual) :: Individual/Standalone Techniques :: VSR vs CF vs CBF 1. Our VSR underperformed CF. 2. But VSR outperformed CBF. 3. Noisy textual app descriptions affect CBF s performance (Lin et al. 2013). 82

83 Part #2: Utilizing Data from Version Updates Results #4: Comparison Against Other Techniques (Combined) :: Hybrid Techniques :: CBF vs CBF+VSR vs CF vs CF+CBF vs CF+VSR vs CF+CBF+VSR 1. CF+VSR > CF CBF+VSR > CBF Combining with VSR improves individual CF and CBF techniques alone. 2. CF+VSR > CF+CBF Version features are better content representations than app descriptions 3. CF+VSR CF+CBF+VSR 83

84 Part #2: Utilizing Data from Version Updates Analysis: Dissecting Specific Topics (of inj+llda topic model) Display-related topic retina, display, graphic, resolut Minor version-cat Genres: Utilities Productivity Travel-related topic map, traffic, rout, locat, trip, road, address, poi Genres: Navigation Traveling Observed Topic Because inj+llda incorporates observed labels such as genre info. pain, medic, drug, pregnanc, period, health, track Genres: Medical Health & Fitness 84

85 3. New Contributions 85

86 Part #1: Utilizing Data from Social Media We propose a method that utilizes signals from Twitter to provide recommendation in cold-start situations. We identify a crucial link between app s social account and followers of its account. Our method can be extended to other social networks with follower structure. Who knows Apple apple may even implement their own social network in their own App Store :P 86

87 Part #2: Utilizing Data from Version Updates We present a framework that incorporates features distilled from version descriptions into app recommendation. From experiments, we identified that genre is a key factor in discriminating the topic distribution. 87

88 4. Future Directions 88

89 Explore Alternatives (to utilize features) 1. For recommendation using social networks, we could: a. use second-degree relationships (i.e., followers of followers), b. use Twitter lists manually curated groups based on specific themes 2. For recommendation using version information, we could view recommendation for different genres as different recommender tasks, and use multi-task learning (MTL) to learn. 3. Treat versions as inter-dependent and come up with a decaying exponential approach to model versions built upon one another. 89