Alternative Formulas for Rating Prediction Using Collaborative Filtering

Transcription

1 Altenative Fomulas fo Rating Pediction Using Collaboative Filteing Ama Saic, Misad Hadziadic, David Wilson College of Computing and Infomatics The Univesity of Noth Caolina at Chalotte, 901 Univesity City Blvd, Chalotte, NC 83, USA {asaic, misad, Abstact This pape poposes and evaluates seveal altenate design choices fo common pediction metics employed by neighbohood-based collaboative filteing appoach It fist exploes the ole of diffeent baseline use aveages as the foundation of similaity weighting and ating nomalization in pediction, evaluating the esults in compaison to taditional neighbohood-based metics using the MovieLens data set The appoach is futhe evaluated on the Netflix movie data set, using a baseline coelation fomula between movies, without meta-nowledge Fo the Netflix domain, the appoach is augmented with a significance weighting vaiant that esults in an impovement ove the oiginal metic The esulting appoach is shown to impove accuacy fo neighbohoodbased collaboative filteing, and it is geneal and applicable to establishing elationships among agents with a common list of items which establish thei pefeences Keywods: Collaboative filteing, Pesonalized Recommendation, Rating Pediction, Similaity measue 1 Intoduction Collaboative filteing ecommende systems employ atings-based use pofiles in ode to mae item ecommendations o pedictions about use atings fo items Suitable items fo ecommendation, such as suggested movies to watch, ae identified not because thei desciption matches them with a taget use, but athe because these items have been lied by uses who ae simila to the taget use in tems of how they have ated othe items Collaboative filteing can employ a vaiety of foundational algoithms, but the most pevalent ae the so-called neighbohood-based methods Neighbohood-based methods fist locate a subset of the use population, based on thei similaity to the cuent o active use Typically, then a weighted combination of the neighbos atings ae employed as the basis fo ating-pediction o ecommendation fo the active use Heloce et al [4] identified and tested seveal main aspects of the design space fo neighbohood-based collaboative filteing, including: similaity weighting, significance weighting, vaiance weighting, neighbohood selection, ating nomalization, and neighbo contibution weighting In

2 this pape, we focus on impoving the accuacy in neighbohood-based collaboative filteing along the dimensions of computing the neighbohood and computing the pediction (see fo instance [3], [6] o [9]) This pape poposes and evaluates seveal altenate design choices fo common pediction metics employed by neighbohoodbased collaboative filteing appoaches It fist exploes the ole of diffeent baseline use aveages as the foundation of similaity weighting and ating nomalization in pediction, evaluating the esults in compaison to taditional neighbohood-based metics using the MovieLens data set The appoach is futhe evaluated on the Netflix movie data set, using a baseline coelation fomula between movies, without meta-nowledge Fo the Netflix pize [8] domain, the appoach is augmented with a significance weighting vaiant that esults in an impovement ove the oiginal Netflix metic Evaluation esults show that ou appoach impoves accuacy fo neighbohood-based collaboative filteing The appoach is geneal and applicable to establishing elationships among agents with a common list of items that establish thei pefeences Collaboative filteing fomulas The baseline algoithm in the liteatue fo neighbohood-based collaboative filteing uses the Peason coelation [4] Fo ates M and N it is calculated using ( M M )( N N) ( M M ) ( N N), whee M and N espectively ae atings fo the item, 1 M M and dim 1 N N the mean values, and dim denotes the total numbe of items dim ated by both uses If dim( M, N ) 0, then is simply set to zeo All sums in the above fomula ae computed ove the atings which both uses have in common M is stictly speaing a function of N, and vice vesa, but witing it down explicitly would unnecessaily complicate the fomulas The pedictions ae then computed using the following fomula M M + N Rates \{ M } ( N N ) N Rates \{ M }, whee M is the mean of all atings fo a given ate This is the standad GoupLens appoach fo the so-called use-to-use atings pediction We consideed the ates/uses to be agents and compae them among each othe to establish lins

3 between them (at least conceptually) The viewpoint can be changed so that the items become agents, the mechanics and, moe impotantly, also the undelying logic, stay unchanged it all depends on what is consideed the agent the use o the ated item Hee we compae the items to each othe, with M and N in the above fomulas denoting two items and summation pefomed ove the common ates fo these items Also the summation in the pediction fomulas is pefomed ove simila items not ates This is commonly efeed to as item-to-item collaboative filteing In ode to disambiguate the notation in what follows, we define two diffeent coelation coefficients as ( M M )( N N) ( M M ) ( N N), ( M M )( N N) ( M M ) ( N N) Hee M denotes the mean values ove all atings of the use M, and M the mean value only ove the atings that the use has in common with the use N The summation in both fomulas goes ove all common atings In what follows we will discuss collaboative filteing and show how additional fomulas can be obtained Ou main goal will be to see what changes the basic ideas of collaboative filteing allow We stat with M M + N Rates\{ M } ( N N ) N Rates\{ M }, (1) which is the oiginal GoupLens fomula as pesented in the pape by Resnic, Iacovou, Sucha, Begstom and Riedl [9], estated in ou notation Towads the end of the pape we will also move away fom using only linea elements in the pediction fomula Thee ae seveal diffeent possibilities to alte this fomula, fo example using nomalization Ou main citicism of the fomula, howeve, is that it utilizes the mean ove all atings M to offset the pedictions, while computing the Peason coelation ove only the popety values (ie atings) that both agents, say uses, have in common Theefoe we test seveal fomulas using the mean calculated ove all the values with othe agents M as well as only ove the common atings M fo a given use Using the mean aveages ove all the atings the equivalent of the above M M + N Rates\{ M } ( N N ) N Rates \{ M } ()

4 Howeve, some ates might be eluctant to give the best o wost possible atings on the Liet scale Theefoe, at least fo use-to-use compaisons, a possible change fom which we might expect some impovement is to adust the offset fom the mean in the pediction fomula, which is in fact based on the atings given by othe uses If we ty to scale these contibutions using the same nom as in the Peason coelation (see fo instance [5]), the GoupLens fomula (1) becomes M M + N Rates \{ M } ( M M ) ( N N ) N Rates \{ M } ( N N) (3) Nomalization maes little sense if we ae compaing items, since in this case we cannot tal of ating tendencies, although the vaiance itself might of couse be useful By using the aveages ove all the atings instead, the above fomula is tansfomed to ( M M ) M M + ( N N) N Rates \{ M } ( N N) N Rates \{ M } (4) Nevetheless, the behavio of agents outside the ange of values common to both uses cannot be implied by thei behavio within this ange This is also the most liely eason why the GoupLens fomula uses M instead of M But this itself is not consistent, since the mean value calculated ove all the atings is used as the stating point to which the contibutions fom the othe uses ae added At fist glance, it might seem that thee is no othe way to do this but to usem Howeve, the following fomula ovecomes this impefection M M ( N N) N Rates \{ M } N Rates \{ M } + N Rates \{ M } N Rates \{ M } (5) by using a weighted aveage also in the fist tem of the fomula The contibution of the othe uses is added to the aveage of use M fo the same ange and the total esult scaled by the value of thei Peason coelation Thus, evey use contibutes a value to the total estimate, which is popotional to the absolute value of the coelation coefficient Altenatively, we can also ewite this fomula as

5 M ( M + sgn( )( N N)) N Movies\{ M } N Movies\{ M } The pediction fomula is theefoe a weighted aveage of single pedictions based on othe agents This is also valid fo item-to item collaboative filteing Afte nomalizing the atings in the fomula (6) we obtain M M N Rates \{ M } + N Rates \{ M } ( M M ) ( N N) N Rates \{ M } N Rates \{ M } ( N N) (6) All of the above fomulas ae easonable altenatives to the GoupLens fomula (1) One could thin that the altenative fomulas ae computationally moe expensive, but the use of M the means in Fomula (1), which is computed only ove the common values, actually equies the same numbe of passes though data as the Fomula (5) only the additional means have to be stoed along with the coelation values The Fomulas () and (4) ae in fact easie to compute since the values fo the means can be pe-computed (fo all use atings), stoed, and used in all subsequent calculations Possibly one fomula will calculate a pediction, while the othe one will not, in which case we use the aveage fo pediction This does not happen fequently 1 Expeimental esults The evaluation was pefomed using the MovieLens data containing 100,000 atings, which povides data splits suitable fo use as taining and test sets The data can be obtained fom the GoupLens webpage [1] The data was andomly split into 5 base and test sets, of 80,000 and 0,000 atings espectively, in ode to be able to empiically evaluate fomulas We used mean absolute eo (MEA) as pefomance measue fo compaisons The gaph in figue 1 indicates that best pedictions ae obtained fo Fomulas (5) and (6) The evaluation of theses fomulas was pefomed using the mean absolute eo as a measue All uses fo which the absolute value of coelation with the cuent use was below 01 wee ignoed in the computation, as well as all the uses having only one ating in common Afte checing whethe the pedicted value is out of ange, ie less than 1 o geate than 5 in the case of MovieLens data, and coecting it to the closest possible value, we conclude that the nomalized pediction fomulas ae favoed again ove othe altenatives This is shown in Figue, which bings only minimal gains, possibly because it does not occu vey often

6 Fig 1 Mean absolute eo using coss-validation on 5 diffeent splits fo fomulas (1) - (6) Also, thee is the possibility of vaying the cut-off value fo the coelation with a small absolute value, as well as discading those uses who have less than a fixed numbe of atings in common with the use fo whom we ae tying to mae the pediction in ode to emove noise Consequently, the inteesting question is how the diffeent fomulas would behave in these cases In the est of the text we loo into these two issues in moe detail Figue 3, fo example, shows the values obtained afte discading all the uses with a coelation of less than 05, and clipping the pedicted value if it falls outside the boundaies Obviously, inceasing this value does not necessaily decease the oveall mean eo Fig Clipping values in case out of ange pedictions fo fomulas (1) - (6)

7 Fig 3 Excluding small coelation values fom contibuting to the pediction, does not necessaily impove the MAE Finally, let us examine what happens when we exclude all the coelations that ae based on 10 o fewe common atings, and disallow any contibution fom these uses The impotant thing to note is that this has to do with the tust in the similaity measue and that it does impove esults (Figue 4) This will also, obviously, Fig 4 Pedictions calculated using coelation values based on at least 10 common atings fo fomulas (1) - (6)

8 decease the numbe of cases in which we ae able to mae a pediction Again, the pedicted values in this example wee coected if they wee out of ange, and coelation values wee discaded if they wee below 01 In the est of the text we loo in moe detail into these two issues 3 A moe eliable similaity measue Based on ou initial expeiments, we wanted to test fo inceased scale on eal-wold data We used item-to-item collaboative filteing on the Netflix data [] to veify that the appoach can esult in impovements ove the standad fomula on eal poblems, othe than that we ae following the oiginal appoach by Resnic et al [9] No clusteing o tansfomation of data was pefomed o additional infomation used (such ae the dates when the ating was made, the gene of the movie etc) Fomula (5) based item-to-item collaboative filteing did not impove the scoe, but it also pefomed ust as good as Netfilx s oiginal algoithm Bette esults fo the dataset wee obtained by eplacing absolute values by squaes, which effectively amounts to using a diffeent similaity measue: Specifically, we inset equation (5), which eplaces esulting fomula is sgn( ) into the, and also use item-to-item compaisons The M ( M + sgn( )( N N )) N Movies \{ M } N Movies\{ M } (7) which is a weighted aveage of pedictions based on othe items The atings wee not nomalized, mostly because we wee using item-to-item compaisons, since in that case (ie fo a given movie) one cannot expect that atings povided by diffeent uses will vay fom the mean by the same aveage amount Rathe, one would expect such ating tendencies to be valid fo ates and not the items Adusting the value of the weights by taing into account the numbe of common atings yields the following slightly involved but othewise staight-fowad fomula M N Movies\{ M } β ( M, N) ( M + sgn( )( N N )) N Movies \{ M } β ( M, N ) (8) whee β ( M, N ) tanh( λ dim ), the numbe of uses who have ated movies M and N is given by dim, and λ is a paamete to be detemined β ( M, N), which eplaces in fomula (5), includes a penalty function used to scale the similaity

9 measue (hee the squae of the Peason coelation) based on the numbe of atings the two movies have in common, to constuct a new similaity measue The valueλ detemines, oughly speaing, the minimal numbe of uses who ated both movies The contibutions coming fom othe movies with only few ates in common ae penalized This is, in pinciple, only a mino modification of discading only loosely (anti-)coelated movies Appopiate λ can be found empiically Fo the Netflix data values between 0001 and 0003 seems to wo well Figue 5 shows the contibution of the tanh ( λ dim( M, N)) tem foλ 000 This appoach esulted in a 9% impovement ove the oiginal oot mean squaed eo of achieved by the Cinematch algoithm Fig 5 Tust allocated based on the numbe of common atings The similaity measue is multiplied by this value to obtain the contibution of a movie in the pediction of ating fo anothe movie Additionally, we have also expeimented with soting the atings fo othe movies in deceasing ode based on the similaity values β and aboting summation afte enough of the othe movies wee consideed We used tanh ( λ dim ) < 5 as condition, but the effect was maginal N Movies \{ M} simila to that of ounding values when they ae out of ange affecting only the thid digit afte decimal point in the RMSE Howeve, it is possible to do so, and, it did esult in a small impovement albeit at the expense of having to detemine a suitable cutoff value We found the values between and 3 to wo well with the values of λ in the above ange The only dawbac is that one has to change the logic of the application The changes descibed peviously consisted of stoing additional values in calculations which aleady had to be pefomed in ode to compute pedictions based on fomula (1) They could theefoe be incopoated easily into any system using the appoach descibed by [9]

10 4 Conclusion Although additional tests should be pefomed, it seems ealistic that the above fomulas could be used successfully instead of the standad pediction fomula [4] It is notewothy that the eos obtained fo the fomulas (5) and (6) in ou test uns wee consistently below those fo the most commonly used pediction fomula (1) We conside them also to be somewhat moe appealing because of the way the aveages ae calculated We theefoe popose that fomula (5) be the default fomula fo collaboative filteing, and one optionally use penalty functions as in fomulas (7) and (8) Any system which is based on (1), can easily be modified to use (5), (6) o (7), and, if one is willing to empiically detemine the additional paamete, also (8) Thee is no achitectual eason not to simply eplace the fomula and leave the est of the system unalteed It is also somewhat supising that, at least fo the data sample used, the nomalization seems to have had only a limited effect Pehaps, one could obtain bette esults with othe noms, fo instance maxu U, o U U Expeiments with the Netflix database show that the modified fomulas allow fo impovements ove the oiginal collaboative filteing The conclusion hee is that one can impove the esults by using penalty functions based on the numbe of common atings in addition to the similaity measue Refeences 1 GoupLens - Bennett, J and Lanning, S: The Netflix Pize, In Poceedings of the KDD Cup and Woshop (007) 3 Heloce, J L, Konstan, J A, Boches, A, and Riedl, J: An algoithmic famewo fo pefoming collaboative Filteing, Poceedings of the nd annual intenational ACM SIGIR confeence on Reseach and development in infomation etieval, pp 30-37, Beeley (1999) 4 Heloce, J, Konstan, J, and Riedl, J: Empiical Analysis of Design Choices in Neighbohood-based Collaboative Filteing Algoithms Infomational Retieval, Vol 5, No 4, pp Spinge (00) 5 Jin, R and Si, L: A Study of Methods fo Nomalizing Use Ratings in Collaboative Filteing, The 7th Annual Intenational ACM SIGIR Confeence, pp , Sheffield, (004) 6 Konstan, J, Mille, B, Maltz, D, Heloce, J, Godon, L, and Riedl, J: Gouplens: Applying collaboative filteing to usenet news, Communications of the ACM, Vol 40, No 3, pp ACM, New Yo (1997) 7 MovieLens Netflix pize - wwwnetflixpizecom 9 Resnic, P, Iacovou, N, Sucha, M, Begstom, P and Riedl, J: GoupLens: An Open Achitectue fo Collaboative Filteing of Netnews, Poceedings of ACM Confeence on Compute Suppoted Coopeative Wo, pp , Chapel Hill (1994) 10 Sawa, B, Kaypis, G, Konstan, J and Riedl, J: Item-based collaboative filteing ecommendation algoithms, WWW 01: Poceedings of the 10th intenational confeence on Wold Wide Web, pp 85 95ACM, New Yo (001)