Improving Cross-domain Recommendation through Probabilistic Cluster-level Latent Factor Model Extended Version

Transcription

1 Improvng Cross-doman Recommendaton throgh Probablstc Clster-level Latent Factor Model Extended Verson Abstract Cross-doman recommendaton has been proposed to transfer ser behavor pattern by poolng together the ratng data from mltple domans to allevate the sparsty problem appearng n sngle ratng domans. However, prevos models only assme that mltple domans share a latent common ratng pattern based on the ser-tem co-clsterng. To captre dverstes a- mong dfferent domans, we propose a novel Probablstc Clster-level Latent Factor (PCLF) model to mprove the cross-doman recommendaton performance. Experments on several real world datasets demonstrate that or proposed model otperforms the state-of-theart methods for the cross-doman recommendaton task. 1 Introdcton Tradtonal recommender systems based on collaboratve flterng (CF) am to provde recommendatons for sers on a set of tems belongng to only a sngle doman (e.g., msc or move) based on the hstorcal ser-tem preference records. However, CF recommender systems often sffer from the sparsty problem, becase n many cases, sers rate only a lmted nmber of tems, even the tem space s often very large. And the sparse ratng matrx reslts n lowqalty predctons. Wth the ncreasng of ser-generated content, there exsts a consderable nmber of pblcly avalable ser-tem ratngs from dfferent domans, ths, nstead of treatng tems from each sngle doman ndependently, knowledge acqred n a sngle doman cold be transferred and shared n other related domans, whch has been referred to as Cross-Doman Recommendaton. Cross-doman recommendaton models have shown that knowledge transfer and sharng among the related domans can be benefcal to allevate the data sparsty problem n sngle-doman recommendatons. CBT (L, Yang, and Xe 2009a) s an early transfer learnng algorthm, whch stdes knowledge transferablty between two dstnct data. A clster-level ratng pattern (a.k.a., codebook) s constrcted from the axlary data va some ser-tem co-clsterng algorthm. Then the codebook s transferred to the target data va codebook expanson. A later extenson called RMGM (L, Yang, and Xe 2009b) combnes codebook constrcton and codebook expanson n CBT nto one sngle step wth soft membershp ndcator matrces. Consderng the exstence of more than one axlary data, TALMUD (Moreno et al. 2012) extends the codebook n CBT to mltple codebooks wth dfferent relatedness weght. These models all assme that mltple domans share the common latent ratng pattern. However, related domans do not necessarly share sch a common ratng pattern. The dversty among the related domans mght otwegh the advantages of the common ratng pattern, whch may reslt n performance degradatons. That s, the exstng models cannot consder the doman-specfc knowledge abot the ratng patterns to mprove the mtal strengths n cross-doman recommendaton. To learn the shared knowledge and not-shared effect of each doman smltaneosly, we propose a novel Probablstc Clster-level Latent Factor (PCLF) model to enhance the cross-doman recommendaton, whch can learn the common ratng pattern shared across domans wth the flexblty of controllng the optmal level of sharng, as well as captre the doman-specfc ratng patterns of sers and clsterng of tems n each doman. Meanwhle, n order to allevate the sparsty problem n ratng datasets, we also constrct the prors on sers and tems by ncorporatng the ser-specfc clsters and tem-specfc clsters. Experments on several real world datasets show that or proposed model otperforms the state-of-the-art methods for the cross-doman recommendaton task. Wth experments we offer evdence that or model can do better on allevatng the sparsty problem. 2 Problem Settng Sppose that we are gven Z ratng matrces from related domans for personalzed tem recommendaton. In the z-th doman ratng matrx there are a set of sers U z = { (z) 1,..., (z) M z } to rate a set of tems V z = {v (z) 1,..., v(z) N z }, where M z and N z represent the nmbers of rows (sers) and colmns (tems) respectvely. Here the set of sers and tems across mltple domans may overlap or be solated wth each other. In ths work we consder the more dffclt case that nether the sers or the tems n the mltple ratng matrces are overlappng. The ratng data n the z-th ratng matrx s a set of trplets D z = {( (z) 1, v(z) 1, r(z) 1 ),..., ((z) S z, v (z) S z, r (z) S z )},where S z s the nmber of avalable ratngs n the z-th ratng matrx.the ratngs n D 1,...,D Z shold be n the same ratng scales R

2 (e.g.,1-5). In or cross-doman collaboratve flterng settng, we consder how to predct the mssng ratngs n all domans of nterest by transferrng correlated knowledge across domans. 3 Or Proposed Model 3.1 Model Specfcaton Or Proposed Model s motvated by the followng observatons. In real-world scenaros, for tems, we see the domanspecfc clsters and the common clsters exst smltaneosly. For example, moves and msc can be both classfed by regons, bt a move category (e.g., scence fcton) may not be able to descrbe msc. Also regons (the common clsters) and the move category (the doman-specfc clsters) affect the ratng reslts of moves n a certan proporton. So tems among mltple domans may not always be able to be groped nto hgh qalty clsters, and ncldng a doman-specfc ratng pattern (ser clsters ratng on doman-specfc tem clsters) may be more accrate than sharng the common knowledge only. 3.2 Clster-Level Latent Factor We assme that the hdden clster-level strctres across domans can be extracted to learn the ratng-pattern of ser grops on the tem clsters for knowledge transfer and sharng, and to clearly demonstrate the co-clsters of sers and tems. User Clsters In real world, sers may have mltple personaltes, so a ser can smltaneosly belong to mltple ser clsters. Sppose there are K ser clsters among all Z domans, {C (1), C (2),..., C (K) }. The probablty of a ser belongng to an exact ser clster k can be descrbed as ). Ths, we can defne ser clster membershp vector for a ser as p = [ (1) ), (2) ),..., (K) ) ] (1) s.t. p 1 = 1 Item Clsters Accordng to the example n the prevos secton, we defne two types of tem clsters: common tem clsters: whch may represent the mtal featres of tems n all Z domans. Sppose there are T common tem clsters, {C, (1) C, (2)..., C }. (T ) Smlar to ser clsters, we can defne common tem clster membershp vector for an tem v as p = [ P (C (1) v),..., P (C (T ) v) ] (2) s.t. p 1 = 1 doman-specfc tem clsters: whch may descrbe the propertes of tems n each doman. Sppose there are L z doman-specfc tem clsters for the z-th doman, {C vspez, (1) C (2) vspez,..., C vspez}. (Lz) Also, we defne domanspecfc tem clster membershp vector for an tem v n the z-th doman as p vspez = [ P (C (1) v),..., P (C (Lz) v) ] (3) s.t. p vspez 1 = 1 vspez vspez Clster-Level Ratng Matrx Frthermore, a ser-tem co-clster can also have mltple ratngs wth dfferent probabltes. For ser clster ratng common tem clster j, that s P (r () C, C ). (j) And for ser clster ratng doman-specfc tem clster j n the z-th doman, that s P (r C (), C (j) vspez). Then we can constrct clster-level ratng matrx S. Common clster-level ratng matrx S com R K T defned as S com = r rp (r C (1), C ) (1) r rp (r (1) C, C ) (T ) r rp (r C (2), C ) (1) r rp (r C (2), C (T ) )..... r rp (r C (K), C ) (1) r rp (r (K) C, C ) (T ) Each element S com (, j) denotes the expectaton of ratng gven by ser clster to common tem clster j. Smlarly, we obtan the defnton of doman-specfc clster-level ratng matrx S spez R K Lz S spez = r rp (r (1) C, C vspez) (1) r rp (r (1) C, C vspez) (Lz) r rp (r (2) C, C vspez) (1) r rp (r (2) C, C (Lz) vspez)..... r rp (r (K) C, C vspez) (1) r rp (r (K) C, C vspez) (Lz) Here, each element S spez (, j) denotes the expectaton of ratng gven by ser clster to doman-specfc tem clster j n the z-th doman. Probablstc Clster-level Latent Factor Model Based on the assmpton that random varables and v are ndependent (L, Yang, and Xe 2009b), we can defne the ratng fncton f R (, v) for a ser on an tem v, whch can be defned by the two combned ratng fncton: the crossdoman ratng fncton f Rc (, v) and the doman-specfc ratng fncton f (z) (, v). For that, the cross-doman ratng fncton f Rc (, v) n terms of the two latent clster varables C and C can be defned as follows: f Rc (, v) = p S com p T (4) = r P (r C, C )P (C )P (C v) r k,t = r P (r C, C )P (C, C, v) r k,t = r rp (r, v) (5)

3 Then the doman-specfc ratng fncton f (z) (, v) n terms of the two latent clster varables C and C vspez (lz) n the z-th doman can be wrtten as follows: f (z) (, v) = p S spez p T vspez (6) = r P (r C, C vspez)p (lz) (C )P (C (l z) vspe v) r k,l z = r P (r C, C vspez)p (lz) (C, C (lz) vspez, v) r k,l z = r rp (r, v) (7) Eqatons above mples that f Rc (, v) gves ratngs from the perspectve of cross-doman recommendaton. Whle, f (z) (, v) gves ratngs from the perspectve of sngledoman recommendaton. Based on the dea that or model combnes cross-doman recommendaton wth sngledoman recommendaton, we ntrodce a set of varables to balance the effect between them. For the z-th doman, set W (z) 1 to be the weght of cross-doman recommendaton, and W (z) 2 to be the weght of sngle-doman recommendaton. s.t. W (z) 1 + W (z) 2 = 1. Ths, we can defne the ratng fncton f R (, v) for the z-th doman as f R (, v) = W (z) 1 f Rc (, v) + W (z) 2 f (z) (, v) (8) = W (z) 1 (p S com p T ) + W (z) 2 (p S spez p T vspez) The llstraton of or proposed PCLF model can be fond n Fgre 1. 4 Model Learnng In ths secton, we ntrodce how to tran or model on the pooled ratng data z D z. The Expectaton and Maxmzaton (EM) (Dempster, Lard, and Rbn 1977) algorthm s a well-known optmzaton algorthm, whch alternates between two steps: the expectaton step and the maxmzaton step. Here we adopt EM algorthm for model tranng. For ease of nderstandng and wthot loss of generalty, we set Z=2,.e., two domans for recommendaton. We need to learn eleven sets of model parameters,.e., ), P (C ), P (C (l1) ), P (C(l2) P ( C ), P (v C ), P (v (1) (l C 1) ), P (v(2) (l C 2) P (r C, C ), P (r C, C (l1) ), P (r C ), ), ). For k = 1,..., K; l 1 = 1,..., L 1 ; l 2 = 1,..., L 2 ; t = 1,..., T ; z U z; v z V z and r R. Expectaton step compted as:, C The jont posteror probabltes are, v, r ) = P (, C )P (v, C )P (r C p,q P (, C (p) )P (v )P (r C (p), C ) ) (9), C (l1) (1) P ( (1), C )P (v (1) p,q P ((1), C (p), v (1), r (1) ) = )P (v (1) (2) P ( (2), C )P (v (2) p,q P ((2), C (p), C (l1), v (2), r (2) ) = )P (v (2) )P (r(1) C )P (r(1) )P (r(2) C )P (r(2), C (l1) (p) C ) (p) C ) (10) ) ) (11) Where Eqaton (9) s compted sng the pooled ratng data z D z. Eqaton (10) s compted sng the ratng data n the frst ratng matrx D 1, and Eqaton (11) s compted sng the ratng data n the second ratng matrx D 2. And P ( (z), C ) = P (C P (v, C ) = P (C P (v (1), C (l1) ) = P (C(l1) P (v (2) ) = P (C(l2) )P ( (z) )P (v C )P (v(1) )P (v(2) C ) (l C 1) (l C 2) ) for z = 1, 2 ) ) Maxmzaton step For smplcty, let P 0 (k, t j), P 1 (k, l 1 j), P 2 (k, l 2 j) as shorthands for, C, v, r ),, C (l1) (1), v (1), r (1) ) and (2), v (2), r (2) ), respectvely. Then the model parameters are pdated as: t j ) = P 0(k, t j) + z l j P z(k, l j) 2 z S z [ t P (C ) = k j P 0(k, t j) z S z P (C (l1) ) = k j P 1(k, l 1 j) P (C (l2) ) = k j: j = + l 2 j: (2) j = (12) (13) (14) S 1 j P 2(k, l 2 j) (15) S 2 P ( C ) = (16) P 0(k, t j) + l 1 P (v C ) = P (v (1) C (l1) ) = j: (1) j = P 1(k, l 1 j) / P 2(k, l 2 j)] [ )(2 z k k j:v P j=v 0(k, t j) S z)] P (C ) z S z P j:v (1) j =v (1) 1 (k, l 1 j) P (C (l1) ) S 1 (17) (18)

4 Fgre 1: Illstraton of or proposed PCLF model n the context of two related domans. The ratng predcton reslt f R (, v) of a ser on an tem v n move doman s the combnaton of cross-doman recommendaton reslt f Rc (, v) and sngle-doman recommendaton reslt f (1) (, v) wth specfc weght W (1) 1 and W (1) 2. P (v (2) C (l 2) ) = P (r C, C k ) = P j:v (2) j =v (2) 2 (k, l 2 j) P (C (l2) ) S 2 j:r P j=r 0(k, t j) j P 0(k, t j) (19) (20) P (r C, C (l1) ) = j:r (1) j =r P 1(k, l 1 j) j P (21) 1(k, l 1 j) P (r C ) = j:r (2) j =r P 2(k, l 2 j) j P (22) 2(k, l 2 j) To avod the local maxmm problems, we se a general form of the EM algorthm named annealed EM algorthm (AEM) (Hofmann and Pzcha 1998), whch s an EM algorthm wth reglarzaton. We adopt the same method sed n FMM (S and Jn 2003) for applyng AEM to tranng procedre. Model Inference After tranng the model, we get those sets of model parameters. Hence, the followng clster-level ratng matrx can be obtaned: S com (, j) = r for = 1,..., K; j = 1,..., T S spe1 (, j) = r for = 1,..., K; j = 1,..., L 1 S spe2 (, j) = r for = 1,..., K; j = 1,..., L 2 rp (r C (), C ) (j) (23) rp (r C (), C (j) ) (24) rp (r C (), C (j) ) (25) Also, the followng parameters can be compted sng the learned parameters based on the Bayes rle: P (C P (C (l1) ) = v) = )P (C ) P ( C k P ( C ) ) P (v C t P (v C )P (C ) )P (C ) v (1) ) = P (v(1) (l C 1) l 1 P (v (1) (l C 1) )P (C(l1) ) )P (C(l1) (26) (27) ) (28) P (C (l2) v (2) ) = P (v(2) (l C 2) )P (C(l2) ) l 2 P (v (2) (l C 2) )P (C(l2) ) (29) Accordng to Eqaton (8), mssng vales n the frst ratng matrx can be generated by f R (, v) = W (1) 1 (p S com p T )+W (1) 2 (p S spe1 p T ) And mssng vales n the second ratng matrx can be generated by f R (, v) = W (2) 1 (p S com p T )+W (2) 2 (p S spe2 p T ) Moreover, or model can also predct the ratngs on an tem n one doman for a ser n another doman. 5 Experments In ths secton, we examne how or proposed model behaves on real-world ratng datasets and compare t wth several state-of-the-art sngle-doman recommendaton models and cross-doman recommendaton models:

5 NMF (Nonnegatve Matrx Factorzaton) (Seng and Lee 2001): a sngle-doman model whch employs nonnegatve matrx factorzaton method to learn the latent factors n each doman and provde the predcton performance separately. FMM (Flexble Mxtre Model) (S and Jn 2003): a sngle-doman model whch ses probablstc mxtre model to learn latent clster strctre n each sngle doman and then provde the sngle doman performance separately. RMGM (Ratng-Matrx Generatve Model) (L, Yang, and Xe 2009b): a cross-doman model whch can only transfer and share the common ratng pattern by the clsterlevel ratng matrx across mltple domans. PCLF model: or proposed model. 5.1 Datasets For the experments we have sed the followng benchmark real-world datasets for performance evalaton: MoveLens dataset 1 : contans more than 100,000 move ratngs wth the scales from 1 to 5 provded by 943 sers on 1,682 moves. We randomly choose 500 sers wth more than 16 ratngs and 1000 moves for experments. EachMove dataset 2 : contans 2.8 mllon move ratngs wth the scales from 1 to 6 provded by 72,916 sers on 1,628 moves. We also randomly choose 500 sers wth more than 20 ratngs and 1000 moves for experments. Here, we map 6 to 5 to normalze the ratng scales from 1 to 5. Book-Crossng dataset 3 : contans more than 1.1 mllon ratngs wth the scales from 0 to 9 provded by 278,858 sers on 271,379 books. We stll randomly select 500 sers and 1000 books wth more than 16 ratngs for each tem n the experments. We also normalze the ratng s- cales from 1 to 5 for far comparson. 5.2 Evalaton Protocol We examne the compared models on dfferent datasets nder dfferent confgratons. The frst 300 sers n ths three datasets are sed for tranng, respectvely, and the last 200 sers for testng. For each test ser, we consder to keep d- fferent szes of the observed ratngs as the ntalzaton of each ser n the experments,.e., 5, 10 or 15 ratngs of each test ser are gven to avod cold-start problem and the remanng ratngs are sed for evalaton. For example, each test ser keeps 10 observed ratngs n the MoveLens tranng set s expressed as ML-Gven10. We choose Book-Crossng vs EachMove and Book- Crossng vs MoveLens and MoveLens vs EachMove as three knds of related domans to dscover the relatedness a- mong dfferent domans. In the experments we condct the methods by repeatng the process 10 tmes and report the average reslts lebanon/ir-lab.htm 3 czegler/bx/ To check the performances of dfferent methods, we se MAE (Mean Absolte Error) as the evalaton metrc. MAE s compted as MAE = O r r / O, where O denotes the nmber of test ratngs, r s the tre vale and r s the predcted ratng. The smaller the vale of MAE s, the better the model performs. 5.3 Experment Reslts Snce we se EM algorthm for model tranng, and the performances of EM algorthm s senstve to ntalzaton. So, the nflence of a good ntalzaton to or model performances s sgnfcant. After testng dfferent ntalzaton methods, we fnally se random vales for ntalzng, C, v, r ),, C (l1) (1), v (1), r (1) ) and (2), v (2), r (2) ). Note that they shold also be respectvely normalzed. Then we can obtan the eleven sets of ntalzed parameters accordng to Eqaton (12)-(22). The parameters of dfferent models have been manally tned and we report here the best reslts obtaned based on the optmal combnaton of many parameter settngs. For or model, we observed that the performance s rather stable when K, L 1, L 2 and T are n the range of [10,50]. Table 1 shows the MAE performances of the compared models on Book-Crossng vs EachMove domans nder d- fferent confgratons, where we set the nmber of sers and tem clsters to K=20, L 1 =15, L 2 =15 and T =10 respectvely. Fgre 2 provdes the performances of or proposed model nder dfferent weght confgratons, whch clearly shows the tradeoff between cross-doman recommendaton and doman-specfc recommendaton. Ths, n the experments we choose W (1) 1 = W (2) 1 = 0.35, W (1) 2 = W (2) 2 = In the experments, we have 5, 10 and 15 ratngs of each test ser n the Book-Crossng and EachMove datasets that are gven for tranng whle the remanng ratngs are sed for test, and the compared models are evalated on the dfferent combned settngs as BC-Gven5 vs EM- Gven5, BC-Gven10 vs EM-Gven10 and BC-Gven15 vs EM-Gven15. And we condct the same combned settngs on Book-Crossng vs MoveLens and MoveLens vs Each- Move. The experment reslts are reported n Table 1,2,3. Best reslts are n bold. From the reslts we can see that the best performng method among all the models s or proposed model. We also observed that the cross-doman based models clearly otperforms the sngle doman based models, whch shows that the latent cross-doman common ratng pattern can ndeed aggregate more sefl nformaton than the sngledoman methods do ndvdally. Moreover, or proposed P- CLF model provdes even better reslts than the state-of-theart cross-doman recommendaton model RMGM, whch ndcates the benefts of combnng the cross-doman nformaton wth the doman-specfc knowledge to enhance the cross-doman recommendaton accracy. Frthermore, from Table 2 and Table 3 we can also dscover that the performances for the tem recommendaton n the MoveLens dataset are not dentcal even n terms of the

6 Dataset Model Gven 5 Gven 10 Gven 15 NMF BC FMM RMGM PCLF NMF EM FMM RMGM PCLF Table 1: MAE performances of the compared models on Book-Crossng vs EachMove related domans nder dfferent confgratons. The combned settngs BC-Gven5 vs EM-Gven5, BC-Gven10 vs EM-Gven10 and BC-Gven15 vs EM-Gven15 are condcted. Best reslts are n bold. Dataset Model Gven 5 Gven 10 Gven 15 NMF ML FMM RMGM PCLF NMF EM FMM RMGM PCLF Table 3: MAE performances of the compared models on MoveLens vs EachMove related domans nder dfferent confgratons. The combned settngs ML-Gven5 vs EM- Gven5, ML-Gven10 vs EM-Gven10 and ML-Gven15 vs EM-Gven15 are condcted. same sers and tems when combned wth dfferent related domans n the experments. The reslts show that dfferent domans may have varos levels of shared nformaton, whch are nderlyng across domans. 6 Conclson Fgre 2: The performances of model nder dfferent weght Dataset Model Gven 5 Gven 10 Gven 15 NMF BC FMM RMGM PCLF NMF ML FMM RMGM PCLF Table 2: MAE performances of the compared models on Book-Crossng vs MoveLens related domans nder dfferent confgratons. The combned settngs BC-Gven5 vs ML-Gven5, BC-Gven10 vs ML-Gven10 and BC-Gven15 vs ML-Gven15 are condcted. In ths work, we proposed a novel cross-doman collaboratve flterng method, named probablstc clster-level latent factor (PCLF) model. The PCLF model has taken nto accont both the sefl knowledge across mltple related domans and the strctres n each doman. On the one hand, the PCLF model s able to learn shared common ratng pattern across mltple ratng matrces to allevate the sparsty problems n ndvdal doman. On the other hand, or model can also draw the dscrmnatve nformaton from each doman to constrct a set of latent space to represent the doman-specfc ratng patterns of ser grops on the tem clsters from each doman, whch s proptos to the mprovement of recommendaton accracy. The expermental reslts show that or proposed PCLF model ndeed can beneft from the combnaton of two types of clster-level ratng patterns (.e., common & doman-specfc) and otperforms the state-of-the-art methods for cross-doman recommendaton task. There are stll several extensons to mprove or work. At present, we only condct experments on datasets wth explct ratng (e.g., from 1 to 5). We plan to explore the ablty to handle the mplct preferences of sers (e.g., vst, clck or comment) of or model. Also, most recommendaton systems are expected to handle enormos amonts of data ( Bg Data ) at a reasonable tme. So we wll evalate the scalablty of or model. References Dempster, A. P.; Lard, N. M.; and Rbn, D. B Maxmm lkelhood from ncomplete data va the EM algorthm. Jornal of the Royal Statstcal Socety. Seres B (Methodologcal) Gao, S.; Lo, H.; Chen, D.; L, S.; Gallnar, P.; and Go, J Cross-doman recommendaton va clster-level

7 latent factor model. In Machne Learnng and Knowledge Dscovery n Databases. Sprnger Hofmann, T., and Pzcha, J Statstcal models for co-occrrence data. Artfcal Intellgence Laboratory Memo L, B.; Yang, Q.; and Xe, X. 2009a. Can moves and books collaborate? cross-doman collaboratve flterng for sparsty redcton. In IJCAI, volme 9, L, B.; Yang, Q.; and Xe, X. 2009b. Transfer learnng for collaboratve flterng va a ratng-matrx generatve model. In Proceedngs of the 26th Annal Internatonal Conference on Machne Learnng, ACM. L, B Cross-doman collaboratve flterng: A bref srvey. In Tools wth Artfcal Intellgence (ICTAI), rd IEEE Internatonal Conference on, IEEE. Moreno, O.; Shapra, B.; Rokach, L.; and Shan, G Talmd: transfer learnng for mltple domans. In Proceedngs of the 21st ACM nternatonal conference on Informaton and knowledge management, ACM. Pan, S. J., and Yang, Q A srvey on transfer learnng. Knowledge and Data Engneerng, IEEE Transactons on 22(10): Pan, W Collaboratve recommendaton wth axlary data: A transfer learnng vew. arxv preprnt arxv: Seng, D., and Lee, L Algorthms for non-negatve matrx factorzaton. Advances n neral nformaton processng systems 13: S, L., and Jn, R Flexble mxtre model for collaboratve flterng. In ICML, volme 3,