Multi-Objective Optimization for Sponsored Search

Multi-Objective Optimization for Sponored Search Yilei Wang 1,*, Bingzheng Wei 2, Jun Yan 2, Zheng Chen 2, Qiao Du 2,3 1 Yuanpei College Peking Univerity Beijing, China, 100871 (+86)15120078719 wangyileipku@gmail.com ABSTRACT Sponored earch ha been recognized a one of the major internet monetization olution for commercial earch engine. There are generally three type of participant in thi online advertiing problem, who are earch uer, advertier and publiher. Though previou tudie have propoed to optimize for different participant independently, it i underexplored how to optimize for all participant in a unified framework and in a ytematic way. In thi paper, we propoe to model the ad ranking problem in ponored earch a a Multi-Objective Optimization (MOO) problem for all participant. We how that many previou tudie are pecial cae of the MOO framework. Taking advantage from the Pareto olution et of MOO, we can eaily find more optimized olution with ignificant improvement in one objective and minor acrifice in other. Thi enable a more flexible way for u to tradeoff among different participant, i.e. objective function, in ponored earch. Beide the empirical tudie for comparing MOO with related previou ponored earch tudie, we provide the inightful application of MOO framework, which i a prediction model to help uer determine the tradeoff parameter among different objective function. Experimental reult how the outtanding performance of the propoed prediction model for p a- rameter election in ad ranking optimization. Categorie and Subject Decriptor H.3.5 [Information Storage and Retrieval]: Online Information Service Commercial ervice; H.3.3 [Information Storage and Retrieval]: Information Search Retrieval Retrieval model General Term Algorithm, Experimentation. Keyword Sponored earch, ad ranking, ad relevance, rev enue, clickthrough rate, multi-objective optimization, Pareto olution et. Permiion to make digital or hard copie of all or part of thi work for peronal or claroom ue i granted without fee provided that copie are not made or ditributed for profit or commercial advantage and that copie bear thi notice and the full citation on the firt page. To copy otherwie, to republih, to pot on erver or to reditribute to lit, require prior pecific permiion and/or a fee. ADKDD 12, Augut 12, Beijing, China Copyright 2012 ACM 978-1-4503-1545-6/12/08 $15.00 2 Microoft Reearch Aia No. 5 Dan Ling Street Beijing, China, 100080 (+86)10-59175012 {bingwei, junyan, zhengc, v-qiadu}@microoft.com 3 Beijing Intitute of Technology No. 5 Zhongguancun South Street Beijing, China, 100081 (+86)15810684632 joyce.du.k@gmail.com 1. INTRODUCTION Sponored earch i attracting much attention from both commercial earch engine and academia due to it increaing market potential [12]. Through diplaying advertiement (ad) to earch engine uer according to their querie in the earch reult page, the publiher, i.e. earch engine, can earn revenue from advertier when their ad are clicked by end uer. One important factor that influence the utility of all participant in thi online advertiing proce i the rank of the ad for each query. The problem of how to elect and rank ad on earch reult page not only impact on uer click tendency but alo relate to the expected revenue of both advertier and publiher. Unfortunately, the objective of earch uer, advertier and publiher are generally inconitent, which make it hard to optimize for all of them imultaneouly. A an example, if the publiher care more about their revenue and have little conideration to other, they may prefer to diplay more profitable ad. Since many cheating ite generally bid higher price than ordinary ad, the uer feeling may be hurt [5] and the ad click-through rate could be low. From a long term view, the uer atifaction contribute to both the earch engine traffic and advertier Return on Invetment (ROI) ince it determine whether they will keep on cooperation with publiher. They both contribute to promote the publiher revenue in long term. Thu, when ranking ad on the earch reult page, publiher hould optimize for uer, advertier and publiher themelve imultaneouly with a reaonable trade off, which i hard to balance in many real world application cenario. Previou tudie on ponored earch generally focu on promoting the performance for ingle objective, uch a ad-to-uer relevance [11], or publiher revenue [31]. However, few of thee work have conidered how to tradeoff among all the participant in ponored earch in a unified framework and a ytematic way. Limitation exit in their work that with the improvement of one objective, ome other objective may deteriorate. In thi paper, we propoe to utilize the Multi-Objective Optimization (MOO) framework for modeling the ponored earch ad ranking problem. MOO i generally ued when multiple conflicting objective function need to be optimized imultaneouly. In the game of ponored earch, uer, advertier and publiher have eparate objective and their objective are uually difficult to coordinate a tated above. By utilizing MOO framework, we can effectively collect variou ranking reult and elect the proper one to balance between the three participant. Through empirically invetigating the MOO framework in the ad ranking problem, we conclude the benefit of MOO for ponored earch a: 1. Many previou ad ranking optimization olution could be conidered a pecial cae of thi MOO framework. Beide, MOO provide more balanced reult in all objective. 2. From the Pareto olution et of MOO framework, we can eaily make adjutment on the relationhip among objective. 3. A an application of MOO framework, we can efficiently predict a proper parameter configuration to tradeoff among different objective. 4. MOO offer u the extenibility if more participant, i.e. objective function, are joining the ponored earch game. The remaining of thi paper i organized a follow. In Section 2, we introduce ome related work on ad ranking problem. In Section 3, we briefly review the background knowledge of optimiza- * Thi work i done when firt author wa an intern of MSR Aia

tion in ponored earch ad ranking problem. In Section 4, we introduce the background of MOO framework and how why and how to fit the ad ranking problem into MOO framework. In addition, an application example of MOO on predicting tradeoff p a- rameter for different participant by MOO i alo introduced in the ame ection. In Section 5, we empirically tudy MOO for ponored earch to verify it benefit. In Section 6, we give the concluive remark on our tudy of MOO framework for ponored earch and dicu ome future topic on MOO. 2. RELATED WORKS The ad ranking problem ha been widely tudied in ponored earch literature and mainly from three perpective, namely ad relevance optimization, ad Click-through Rate (CTR) prediction, and Relevance and Revenue optimization (RnR). Firt of all, the ad relevance i playing a key role for atifying the earch engine uer. Uer can be better atified if the ad are relevant to their immediate interet. M any previou tudie [11, 21] utilized learning to rank [17] trategy to predict the relevance between ad and earch querie for ad ranking. Thee work either rank the ad by the pure context ignal uch a by text tring matching [21] or add click-through information a additional ignal [4, 11, 29]. Some reearcher argue that ad relevance cannot reflect uer atifaction in an unbiaed mode [11]. In addition, the click-through ignal may be biaed if the ad have high bounce rate [26]. To reolve thee limitation, ome recent tudie in web earch literature [7, 15] are moving forward to utilize implicit feedback from uer uch a Dwell time and uer behavior on ad landing page to augment the ad relevance. However, even though we may aume good ad relevance can guarantee good uer atifaction, it doe not mean the advertier can win more click and the publiher can earn better revenue [11]. Beide the ad relevance, the ad click-through rate (CTR) i generally treated a another important factor to be optimized in ponored earch. It i generally ued a an indicator to how whether the advertier i optimized, given that the higher the CTR i, the higher probability the ad impact the market. Some previou work [22, 27] focued on the pure text feature, more recent tudie predict the ad CTR by click model [28]. Thee method [1, 3, 9, 32] utilize Bayeian Network to repreent ome aumption of the uer ad click behavior. Targeting real world application, ome recent work propoed to predict CTR for new ad or thoe ad with pare click hitory [1, 10, 22]. However, the high CTR doe not mean that uer are definitely atified with ad ince ome attractive ad, uch a Loe Weight now, may ignificantly contribute to CTR though they may be irrelevant to querie [11]. For publiher, the ad revenue i naturally an important objective. However, it ha been realized that purely optimizing for revenue may harm uer feeling and even lead to their property lo if ome deceptive ad are hown on the earch reult page. Thi will caue a lo of uer affinity and then a lo of long term revenue to publiher. Thu mot previou work focued on optimizing for relevance and revenue imultaneouly to avoid low uer affinity. One method i to ue query ubtitution to improve both revenue and relevance [20]. In another group of algorithm, revenue i the main objective to be optimized, with uer utility incorporated into the ranking function [5, 25, 31]. Uer experience are depicted a either different core or a part of the lo function in thee algorithm. Recently, ome work generalize the relationhip between relevance and revenue by a tradeoff parameter [30]. Publiher can modify thi parameter to reach a pecific tradeoff between relevance and revenue. However, revenue i influenced by both CTR and Cot-Per-Click (CPC). Only RnR optimization may lead to ome reult with low CTR and high CPC. If CTR optimization can be incorporated into the ranking function, we may improve CTR with mall acrifice in CPC to benefit advertier, which in turn attract more advertier in bidding to help improve publiher long term revenue. A a ummary, mot previou tudie generally optimize for one or two objective in the ad ranking problem while eentially there exit more objective to optimize. A ytematic tudy on how to optimize and tradeoff among all the participant in a global view i deirable. 3. BACKGROUND: OPTIMIZATION IN SPONSORED SEARCH AD RANKING Sponored earch i one of the major internet monetization olution for commercial Web earch engine [31]. It take up to 46% of the internet advertiing revenue in 2010 [12]. A the repone to a uer earch query, ponored ad are uually diplayed on top or right-panel of the earch reult page. Search engine, participating a ad publiher, generally earn revenue in ponored earch by Cot-Per-Click (CPC) [6]. That i to ay, earch engine hall charge the advertier if and only if the ad are clicked. Search engine elect and rank the ad mainly baed on the bid keyword and the bidding price of the ad. Uer, advertier and earch engine benefit from each other in thi ponored earch game. Among the three participant, which ad to diplay to a uer earch query i the key problem to bridge all participant and to balance their benefit. The optimization for ad ranking in ponored earch i thu highlighted to addre the problem of optimizing the utilitie of all three group of participant. Optimize ad relevance for uer The ad to uer relevance i a commonly ued indicator to how whether the uer could be atified in ponored earch. There are variou option for the objective function of ad relevance. In thi work, we conider the Dicounted Cumulative Gain (DCG) [13] a the objective of optimizing the ad relevance. The DCG at a particular rank poition p i defined a: p rel 2 i 1 DCGp, log (1 i) 2 where i the graded relevance of the reult at poition i. When optimizing the ranked ad, we ue the predicted ad relevance a [30]. For each query, we try to optimize the ad ranking for the larget DCG, and improve the uer atifaction to the ranked ad reult. Optimize click-through rate for advertier Generally, advertier care more about how to improve their profit. In the game of ponored earch, CTR i directly related to the potential revenue of advertier, o in thi work, CTR i choen a the objective for optimizing advertier utility. Practically, we regard all the advertier a a whole, namely the objective function for advertier aim to optimize the total utility of all the advertier. We aume that for each query, a fixed number of ad will be hown on the earch reult page. Under thi aumption, high CTR tand for more click. By umming up CTR of all ad hown on the earch reult page, we can effectively compare the performance baed on advertier objective function. The higher the ummed CTR i, the more poible advertier hall earn profit from the uer click. Thu we mathematically define the advertier objective a: CTR( Adq( rankq( i)),

where q tand for the given query, ( ) repreent the candidate ad et for q, ( ) tand for the index of the ad ranked at poition i, and there are totally ad hown on the earch reult page. For each query, we aim to reach the maximum ummed CTR, i.e. to optimize advertier profit from the ad ranking reult. Optimize revenue for publiher Revenue i the primary goal of publiher in ponored earch, and thu treated a the objective of optimizing for publiher in thi work. Intuitively, the product of CTR and CPC of an ad, which ha been ued in previou tudy [31], tand for the expected revenue publiher can receive from the ad. Similar to advertier objective, by umming up revenue of all ad which are hown on the earch reult page, we can effectively compare the performance of ad ranking reult baed on the publiher objective function. Thu we mathematically define the advertier objective a: CTR( Adq ( rankq ( i)) CPC( Adq ( rankq ( i)), where q, ( ), ( ) and are of the ame meaning a in advertier objective function. For each query, we try to optimize the ad ranking for improving publiher revenue a well. 4. SPONSORED SEARCH IN MOO In thi ection, we model the ad ranking problem in ponored earch within the MOO framework. In Section 4.1, we firt introduce ome background knowledge about multi-objective optimization framework. Then, in Section 4.2, we explain why and how to fit ponored earch ad ranking into the MOO framework. In Section 4.3, we how an exemplar application of MOO framework to how it powerfulne. 4.1 Background on MOO The Multi-Objective Optimization, which ha been widely tudied in economic, finance, product deign, and other [19], define a proce of optimizing two or more conflicting objective function imultaneouly ubject to contraint [23]. It ha been recognized a an efficient way to make a tradeoff for atifying multiple different objective. However, in term of online advertiing, the MOO framework i underexplore though we can how that the ponored earch ad ranking can well fit the MOO etting. Mathematically, uppoe 1 ( ), ( ),, ( ) i a et of objective function, which may conflict with each other, ( ), ( ),, ( ) i a et of inequality contraint function and 1 1 ( ), ( ),, ( ) i a et of equality contraint function, then Multi-Objective Optimization i defined a [18]: T MinF( x) [ F1( x), F2( x),..., Fk ( x)] x ubject to g j( x) 0, j 1,2,..., m, h ( x) 0, l 1,2,..., e l where k, m, e are the number of objective function, inequality contraint and equality contraint repectively. x i called deign variable which i the variable to optimize. The feaible deign pace X i made up of all poible value for x that meet the contraint. Then the feaible criterion pace Z i defined a the et ( ). After the optimization, the reult of MOO are generally not a ingle olution but a olution et. The Pareto Set i a key definition and the ultimate goal in MOO framework, which conit of Pareto Optimal defined a follow [18]: Definition 1. Pareto Optimal: A point,, i Pareto Optimal if and only if that there doe not exit another point uch that ( ) ( ), and ( ) ( ) for at leat one function. There exit multiple way for computation purpoe in the MOO framework, which can effectively collect mot of the Pareto Optimal. Some of the claical olution are lited a follow. Weighted Sum Method It i the mot common approach to olve MOO problem. It aign weight to different participant, generate a global criterion by umming up the weighted objective function and then elect the optimal olution with the bet performance on thi global criterion. A mathematical equation of the global utility in thi method i decribed a [18]: k U F( x ), where indicate the weight of the objective function. are predefined parameter. We will dicu their election by MOO in Section 4.3. Lexicographic Method Thi method order the objective function according to their importance and then optimize them in order ubject to the contraint that high priority objective do not deteriorate. The mathematical formulation for thi method i decribed a [18]: Min F ( x) x i ubject to F F j i i 1,2,..., k i i * j( x) j( x ), 1,2,..., 1, where i indicate the preference order and optimum of the objective function. ( ) repreent the Genetic Method Thi method i a imulation of biological evolution. Reproduction, croover and mutation are employed to generate new reult in feaible deign pace [24]. Only thoe reult which are not dominated by other reult can urvive under the election method. After everal round of iteration, the optimized reult will be kept in the urviving et and elected into the Pareto Set. Thi method i good for avoiding local optima and will converge to global optimal after ufficient round of evolution [18]. We ue thi method in our experiment implementation ince it require little knowledge of the problem and i eay to implement [24]. 4.2 Formulate Sponored Search by MOO The game in ponored earch can be formulated into MOO framework a follow. In thi problem, we have ome known quantitie uch a uer information, candidate ad et and cot per click (CPC) of every ad, the query input by uer, and the maximum number of ad allowed to be hown on the earch reult page. From thee quantitie, we can predict the relevance between the ad and the query by learning to rank algorithm [17]. Deign variable in thi problem i the ranking function. For each query q and it candidate ad et C, we try to find a ( ) which optimize the utilitie of all the three group of participant in thi game. In the following part, we ue ( ( )) to indicate the ad on the poition of the particular ranking function and ( ( )) to indicate the poition of the ad with number i in thi ranking function. It i worth noting that click-through rate (CTR) i eentially a function for ranking. For a pecific ad and the correponding query, the CTR of the ad i not only related to the content imilarity

between the ad and query, but alo related to the poition of thi ad. If one ad i ranked higher, it CTR will be higher than that in lower poition. Then the problem in ponored earch i formulated a follow: T MaxF( x) [ F1 ( x), F2 ( x), F3 ( x)] x where x rankq( C), reli 2 1 F1 ( x) DCG log 2(1 i). F ( x) CTR( Ad ( rank ( i)) 2 3 F ( x) CTR( Ad ( rank ( i)) CPC( Ad ( rank ( i)) ubject to rank ( i) rank ( j), if i j q q q q q q q q By formulating the ponored earch ad ranking problem into MOO framework, we can expect following benefit. Generalized Model Previou work can be eaily fitted into thi MOO framework. Since they either try to optimize one major objective or try to balance two of them, we can imply et the other objective function to contant zero to generate a ingle objective optimization. Max F ( x) [ F ( x), F ( x), F ( x)] T, if ( ) ( ), we can get a For 1 2 3 x ingle optimization for uer atifaction a the method in [1, 4, 7, 11, 15, 29]. If ( ), the optimization reduce to a tradeoff between publiher revenue and ad to uer relevance a in [5, 25, 30, 31]. Alo, if 1 ( ) ( ), we then optimize for CTR a tudied in [1, 2, 3, 9, 10, 22, 32]. The MOO framework can not only cover mot previou work, but alo provide more balanced reult a we can ee later in experiment part. CTR a an Individual Objective In previou work, CTR i more treated a a feature to predict ad relevance or publiher revenue rather than a utility to impact on ranking function. Although the preciion of CTR prediction i important for both other objective, it abolute value ha alway been omitted. In our MOO framework, we extract CTR a an individual objective function that depict advertier utility. Introducing thi objective into the model enable u to avoid bad ituation where publiher get the ame revenue with le click and higher CPC. In the ame condition, if we can improve CTR while keeping revenue and relevance the ame, advertier hall get more profit, which in turn attract more advertier to bid. Predictability The predictability of MOO framework ha two meaning. The firt i the predictability for quality of an algorithm. Since all the poible reult have been found in MOO framework, we can judge the quality of an ad rank algorithm according to whether it can reach part of the Pareto Set and how good the reult are. It provide a tandard that we can predict and compare the effectivene of a particular method. The other meaning of predictability indicate that MOO framework can help u with the parameter election for three participant in the game. Detail on how to perform parameter prediction hall be dicued in next ubection. Extenibility MOO framework provide u with a potential extenibility. In previou work, method are fixed to optimize the pecific objective. For example, in [30], a pecific regreion function i introduced to balance revenue and relevance. However, thi model itelf i only uitable for thi particular problem. If more objective are introduced into the problem and need a tradeoff, thi model i unable to handle. They will need to propoe a totally new regreion function to olve the new problem. Thi deficiency can be eliminated in our MOO framework. We can extend thi framework with more objective. When new participant are joining in thi game in ponored earch, no modification i required on the framework itelf. The only change in our framework i the definition of the Pareto Set, which need to compare the value of the additional objective. Then we can olve the problem a before. Thi i a precient work for future if more participant are joined in the game of ponored earch, MOO framework may how it powerfulne with thi extenibility. 4.3 Exemplar Application: MOO for Sponored Search Parameter Selection A mentioned before, weighted um method i the mot common approach to olve the MOO problem. It i very intuitive that high weight lead to a high value of the correponding objective function. However, it i hard to chooe a uitable parameter configuration to meet our pecific requirement becaue there i no direct linear relationhip between the weight and the value of objective function. The difficultie are mainly reflected in the following apect. Firt, it take time to verify the effectivene of parameter in live earch engine. Even in offline experiment, large amount of trial and comparion hould be made before we can get to know the function of one parameter configuration. Second, the function of parameter are unpredictable. Small decreae in weight may be unneceary to get a mall decreae in the correponding objective. More ophiticated method hould be adopted for electing new parameter. To how the powerfulne of MOO in thi problem, we formulate the problem a follow: Problem: Given a requirement which i repreented a ( ): 1 ( ), i 1,,..,k-1, find a et of optimized parameter, i 1,,..,k, uch that with the global criterion k i 1 ( ), the optimized olution x meet the requirement or a cloe a to the requirement. Generally, we have an enumeration method to olve the problem: Solution: a. Enumerate variou parameter configuration on a query and find the relationhip between the proportion of objective function and the parameter. b. Regard uch relationhip a a training ample of the parameter election. c. Collect a training et large enough to train a model. d. Ue machine learning method uch a neuron network to learn the relationhip between requirement and parameter. Thi training proce i traightforward for parameter election. We can effectively predict the parameter for a pecific requirement with thi model. However, thi training ample generation proce can be quite low if with many objective function. For example, if 10 parameter value are needed to be enumerated on each objective, for three objective function, we may get 10*10*10 = 1000 parameter configuration on one query. MOO offer u a fater way to generate training et for predicting parameter for a pecific requirement. By collecting all the reult from Pareto Set, we can eaily judge which parameter configuration are redundant. Since each parameter configuration i correponding to a hyper-plane, only thoe parameter which cover two or more Pareto Optimal will be crucial to repreent a parameter boundary of one reult. Generally, there are 5 reult in Pareto Set o that at mot 4 parameter will be teted on one objective. Then for the quetion of the ame cale a lat example, only 4*4*4 =

64 parameter configuration need be enumerated, which will be 20 time fater than the former olution. Thu training et can be contructed much fater when with large amount of input. The detailed experiment reult can be found in experiment part. 5. EMPIRICAL STUDIES In thi ection, we empirically how the power of the MOO frame work for ad ranking problem in ponored earch. In Section 5.1, we decribe the experiment configuration including dataet and evaluation metric. Then in Section 5.2 we compare the MOO with other ingle objective optimization method to how the advantage of it. In Section 5.3, ome potential application of MOO will be dicued to give ome inightful idea by uing MOO for ponored earch. 5.1 Experiment Configuration 5.1.1 Dataet and the learning model for prediction For experiment purpoe, we collected day ad click-through log from a commonly ued commercial earch engine, which contain the earch querie of uer, ad returned by the publiher and the click hitory of thee ad. There are 3,601,305 querie altogether with at leat 8 returned ad for each query extracted. After randomly ampling from the data, a 5 level grade i aigned to each query-ad pair by editorial judgment baed on the degree of relevance, where core 1 tand for totally irrelevant and core 5 tand for trong relevant. We ued 90% of the querie for training and 10% of them for teting. In our experiment, we limit the returned ad number to 3, namely for each query, we aume exactly 3 ad are returned on the reult page. The commonly ued Gradient Booting Deciion Tree (GBDT) [8], which ha been the winner algorithm in earch ranking problem, and the General Click Model (GCM) [32], which ha been recognized a one of the algorithm to have outtanding performance in ad CTR prediction problem, are ued in our experiment to predict the relevance of query-ad pair and the CTR of ad repectively. We employ 13 feature ued in the etimation of ad relevance and 20 feature ued in ad CTR prediction, 6 of which are uer-pecific feature and the ret are URL-pecific feature [32]. Mot of the feature we ued in our experiment are the ame a their counterpart in [30, 32] uch a the uer country, local hour, ad category, ad poition, TF, TF*IDF, BM25, etc. 5.1.2 Evaluation metric The output of the ad ranking problem i a ranked lit of ad for a given earch query. Each reult contain three ad and their rank core. To evaluate the performance of the ranking reult, we utilize the following three commonly ued evaluation metric. Normalized Dicounted Cumulative Gain (NDCG) We ue NDCG [13] to evaluate the performance of ad ranking from uer objective, ay, the relevance between earch query and the ad. NDCG i derived from DCG, which i defined in Section 3, to compare DCG acro query with different length of reult lit a, DCGp NDCGp IDCG where indicate an ideal, ubcript p tand for the number of ad, the ame a in DCG definition. When evaluating the ranking reult, we conider the human-labeled grade a the true relevance and calculate the NDCG to ae the relevance performance of the ranking reult. We call the NDCG a Relevance Score hereafter. p Normalized CTR Score From advertier perpective, we ue a o called Normalized CTR Score a the evaluation metric in our experiment. Given a query q, it candidate ad et ( ) and the rank reult et R, we define the normalized CTR Score for a ranked reult lit a: CTR( Adq( rankq( i))) CTR core max CTR( Ad ( rank ( i))) rankr q q where CTR( Adq( rankq( i))) repreent the ummed CTR of the returned three ad for the given query. We ue hitorical CTR of an ad a the ground truth for thi evaluation. The ummed CTR divided by the maximum ummed CTR i the normalization proce to compare reult from different querie [30]. Normalized Revenue Score From publiher perpective, we ue Normalized Revenue Score [30] a the evaluation metric in our experiment. Given a query q, it candidate ad et ( ) and the rank reult et R, we define the Normalized Revenue Score for a pecific ranked reult lit a: CTR( Adq ( rankq ( i))) CPC( Adq ( rankq ( i))) Revenue core max CTR( Ad ( rank ( i))) CPC( Ad ( rank ( i))) where rankr q q q q CTR( Adq ( rankq ( i))) CPC( Adq ( rankq ( i))) repreent the ummed expected revenue of the returned ad for the given query. We ue hitorical CTR of an ad a the ground truth a well. Normalization proce i imilar to Normalized CTR core, which i ued to fairly compare reult from different querie. 5.2 Algorithm Comparion In thi experiment, we ued the evolutionary trategy, which i a kind of genetic method, for MOO framework. Each olution repreent a different election and ranking of the candidate ad. They are preented a triple, and the three value in the triple are Relevance core, CTR core and Revenue core. 5.2.1 Method effectivene verification In the experiment, three ingle objective optimization for relevance, CTR and revenue are regarded a the baeline algorithm. In thi part, we compare thee three algorithm with RnR optimization and MOO in all of the evaluation metric. Table 1 lit the average relevance core, CTR core and revenue core of thee algorithm. From thi table, we can conclude that from tatitical evaluation, optimizing objective function with predicted query - ad relevance and predicted CTR of an ad i proved to be conitent with the ground truth, which conider human label a the true relevance and hitory CTR a the true CTR of an ad. Alo we can find that although MOO fail in the competition with RnR optimization in relevance and revenue, it ha a large improvement Table 1. Reult for different optimization objective Relevance core CTR core Revenue core Relevance optimization 0.9478 0.8786 0.9050 CTR optimization 0.8714 0.8892 0.8280 Revenue optimization 0.8934 0.8330 0.9258 RnR optimization 0.9116 0.8441 0.9199 MOO 0.9059 0.8778 0.9079

Percentage of Covered Reult 100% 50% 0% 1 4 7 10 40 70 100 Number of Iteration Figure 2. Iteration number in MOO and percentage of it covered ingle objective optimization reult (a) (c) Figure 1. MO O reult and reult from other algorithm Circle tand for reult from MOO and filled point tand for reult from ingle objective optimization or RnR optimization in CTR, which help it to be the mot balanced reult among thee optimization. In the following part, we will give more detailed analyi on the advantage of MOO from different apect. 5.2.2 Completene of MOO reult A decribed above, previou work, which aim to optimize ingle objective function, are all pecial cae of MOO framework. The Pareto Set from MOO contain all the reult found in ingle objective optimization. We how thi concluion from ome cae tudie in Figure 1, which how all the ranking reult in Pareto Set from MOO framework. In Figure 1(a) and Figure 1, filled point repreent the reult of ingle objective optimization for relevance, CTR and revenue repectively. Since triple are cattered on a 3D pace, we project it on Relevance-Revenue and CTR-Revenue 2D plane to how the complete view of thoe reult. We can oberve that the reult from ingle objective optimization are indeed contained in the Pareto Set. Generally, the ingle objective optimization earche in the feaible pace and chooe the maximum value of the pecific objective function it optimize for. They certainly belong to Pareto Set becaue they are not dominated by other rank reult. We alo compare the reult from MOO and a tradeoff optimization for revenue and relevance (RnR). Both of thee two algorithm return a et of the reult. In Figure 1(c), we ue filled Point to depict reult from RnR. Similar to above figure, RnR reult are alo contained in the Pareto Set from MOO. MOO alo provide u with ome reult inferior on Relevance-Revenue Plane but uperior in CTR core. In our experiment, with the ame complexity a ingle objective optimization, we manage to get the approximate Pareto Set, which cover 80% of the ingle objective optimization reult, within 50 iteration. The relationhip between the MOO iteration number and percentage of it covered ingle objective optimization reult are hown in Figure 2. Since the reult from MOO framework i a complete et of optimized reult, we can directly ue MOO to (a) Figure 3. Reult from MOO and other algorithm. Filled circle tand for balanced reult in all objective function. replace thoe ingle algorithm to reduce the duplicate computation when we try to run three different algorithm. 5.2.3 Potential balanced reult in MOO Beide covering the reult from ingle objective optimization, MOO alo bring u more promiing reult which cannot be found by previou work. From the decription of goal in ponored earch, we need a balance between different objective function. MOO i well uited for thi goal ince it provide u with a comprehenive view of the poible rank reult. We can find balanced reult in the Pareto Set baed on the MOO framework in almot every query in the experiment. Figure 3(a) i a cae tudy of the reult from a random picked query in experiment. Marker in Figure 3(a) are the ame a in previou ection, except for a filled circle. We can eaily oberve from thi projection that reult with maximum value in either revenue or relevance have low value in the other objective. However, the filled circle ha a more balanced value in thee two objective. Moreover, from another projection of thee reult which i drawn in Figure 3, we can undertand that thi reult alo ha a relatively high value in CTR core. The filled circle will never be found by ingle objective optimization becaue it doe not reach the maximum value in either of the objective function. But compared to thoe ingle objective optimized reult, thi reult i more likely to be competitive to atify all the three participant in the game of ponored earch. 5.3 Potential Application of MOO The MOO framework ha many more potential application. Thee application help u in different tak. In thi part, we how the powerfulne of MOO framework in variou field. 5.3.1 Lexicographic olution Different publiher may have different target for revenue, CTR and relevance. For a riing earch provider, the fame in uer i appreciated mot. In thi cae, advertier and the publiher own revenue hould be conidered after uer atifaction when thi earch engine i involved into the game of ponored earch. In another ituation, for a earch engine which ha already had a

(a) (a) (c) Figure 4. Lexicographic olution Filled point tand for reult from ingle objective optimization and filled circle tand for reult from Lexicographic olution (c) (d) table cutomer bae, how to attract more advertier may become it mot urgent tak. It hould concentrate on improving Clickthrough Rate to atify advertier and get more contract with other advertier. In either of the ituation, the ad ranking problem in ponored earch reduce to a major objective problem with ome other minor objective function. Under MOO framework, thi problem can be eaily olved by lexicographic method, which i introduced in Section 4.1. With the ame evolutionary method, we can equally olve thi problem and achieve the optimized reult. The only thing we need do i to redefine the le than relationhip between rank reult. We how a cae tudy of thi application in Figure 4(a). In thi cae, ingle objective optimization for relevance i depicted a filled point. We ee a filled circle in thi figure which tand for the reult from major objective olution. Single objective optimization only conider reaching the maximum of Relevance core. However, many rank reult can equally reach the maximum of Relevance core. In major objective olution, optimization for revenue i followed after we have optimized relevance, the major objective. A we can ee from the figure, it i obviouly a better reult if only relevance and revenue are conidered important. Figure 4 and 4(c) how other two cae of thi improvement. A a concluion, lexicographic olution i imilar to ingle objective optimization but with ucceive tep after major objective ha been optimized. Minor objective can be alo optimized, which will improve the total utility of all the participant. In our experiment, we find 32.2% cae of thi kind of improvement in MOO reult, which optimize the minor objective by 7.1 % on average. (e) (f) Figure 5. Solution with contraint 5.3.2 Solution with contraint Sometime earch engine do not care much to get the optimized reult in ome pecific objective function. They imply et contraint on thee objective. After filtering olution which do not meet the contraint, they try to find optimized reult in other objective function. In that way, we can find more promiing reult compared to major objective olution. It i a imple method to find a good ad rank reult. However, the election of the con- Filled circle tand for the optimum reult with the certain contraint and traight line in the middle tand for contraint traint i not a imple a it eem to be. If the contraint i too weak, the filtering tep may be uele ince jut few reult are excluded from candidate in thi tep, which mean we totally give up to conider thi objective. On the contrary, if the contraint i too trong, many good reult in other objective may have been filtered before they are found. A good contraint hould keep the mot competitive reult while being effective to reduce the computation complexity after filtering tep. MOO framework can be efficiently ued to elect a proper contraint. To explain thi application, we firt how thi application in a cae tudy. The method for contraint election in one query by MOO framework can be generalized a follow. Firt, we can elect a ingle tet cae of data and depict all the reult olved from MOO framework in a graph. Then a proper contraint, which help u elect the mot propective reult, can be found in the graph. Further, we hould make a mall adjutment in thi contraint to ee if we can get a large improvement in other objective if the contraint i a little lower. Thu a proper contraint can be et on thi objective to help u find the olution we want. In the cae tudy hown in Figure 5(a) and 5, we chooe relevance a the contraint and revenue a the main objective that we want to optimize. If the contraint on relevance core i et a Figure 5(a), we can get only two candidate for further optimization, in which the optimized revenue core i 0.915. In the other hand, we can eaily find in Figure 5 that if we only require relevance core to

Ratio of top rank reult in all tet cae Table 2. Empirical reult for acrifice and improvement among objective 1% 2% 3% 4% 5% 1% 2% 3% 4% 5% 1% 2% 3% 4% 5% Improve CTR Improve Revenue Improve either of the other two objective Reduce Rel Reduce CTR Reduce Rev 1% 67.6% 60.7% 54.0% 48.1% 42.7% 62.9% 54.8% 48.4% 43.0% 38.2% 88.3% 81.8% 75.2% 69.1% 63.3% 2% 74.2% 67.6% 61.6% 56.1% 50.7% 66.2% 58.5% 52.2% 46.5% 41.7% 90.2% 84.7% 79.2% 73.9% 68.6% Improve Relevance Improve Revenue Improve either of the other two objective 1% 68.6% 57.3% 45.1% 34.9% 28.0% 57.4% 48.8% 42.0% 36.2% 31.3% 89.0% 79.3% 68.2% 58.3% 50.0% 2% 71.4% 59.6% 47.5% 36.7% 29.8% 65.3% 57.1% 50.4% 44.5% 39.1% 91.2% 82.5% 72.8% 64.0% 56.3% Improve Relevance Improve CTR Improve either of the other two objective 1% 62.8% 49.8% 37.4% 29.1% 23.7% 55.3% 47.4% 40.8% 35.5% 30.8% 84.9% 73.6% 61.7% 53.0% 45.9% 2% 65.7% 52.8% 41.1% 32.3% 26.6% 62.4% 54.7% 48.0% 42.2% 36.9% 87.7% 77.8% 67.5% 58.9% 51.8% be larger than.95, which i jut a. % decreae, the publiher expectation of revenue increae from 0.915 to 0.966, which experience a 5.5% improvement. It i a large profit to publiher. Figure 5(c), 5(d), 5(e) and 5(f) how other two example of contraint election. For application, a more general concluion hould be reached. From the maive tet cae, we conclude a tatitical reult, which how to u the relationhip between acrifice in one objective and the improvement in other objective. Thi can be regarded a an empirical reult o that whenever we want a quick adjutment of the relationhip between objective function, we can imply look up in thi table and modify the contraint a the data ay. Table 2 how to u the tatitical reult, where the firt column indicate the objective we want to acrifice and the econd column indicate the percentage of decreae on thi objective. The following part of the table how the poibility of expected improvement on other objective. Bolded reult indicate the improvement with at leat 40% poibility in ome pecific objective and at leat 60% poibility in either of the other two objective. Thi table i much more effective when we decreae the objective from optimized core 1.00. 5.3.3 Parameter election With the help of MOO framework, we can elect the tradeoff parameter for different participant in a fater and more intelligent way a decribed in Section 4.3. We firt how a cae tudy to find an optimal parameter et for one query. It i a viualized and parallel method o that both perception and computation of thi method are eay to reach. In Figure 6(a) and 6, all circle tand for reult in Pareto Set and each filled circle or other y m- bol tand for the optimized reult found by a different preet parameter configuration. We can compare thee reult and viually elect the parameter that meet our requirement. For example, if we want a balanced reult we may chooe the parameter repreented by the cro. Alo, if we want a higher revenue and relevance we may chooe the one correponding to the triangle. Similar to thi pecial cae, we generate large amount pair of requirement and their correponding parameter. Then we adopt neuron network on thee training data to train a model to predict parameter on new requirement. We evaluate the predicted p a- rameter together with everal fixed parameter configuration in tet data et of 27,926 querie. The evaluation metric in thi experiment i KL ditance [16] between the predicted reult and the requirement. In our experiment, we generate many tet cae with different requirement, in the competition with 176 parameter configuration, 19.7% of our predicted parameter perform the bet, and the other 80.3% are all ranked in the top 10 reult. The 1 0.5 0 (a) 1 2 3 4 5 6 7 8 9 10 # Rank Figure 7. Performance of predicted parameter in competition with 176 parameter configuration Figure 6. Parameter election Filled circle, triangle and croe all tand for optimum reult for different tradeoff parameter performance of thi prediction i hown in Figure 7. In the experiment we alo oberved that parameter do not have direct linear relationhip with requirement. For example, if the requirement i 1:0.6:0.6, the bet predicted parameter are 0.78:0.13:0.09. Thu, with the help of thi prediction work, whenever we have a requirement on the proportion of thee objective function, we can eaily predict a parameter configuration for it and find an optimized reult by thi weighted um global criterion. A decribed in Section 4.3, the training proce i much fater with MOO framework than the naïve enumeration method. Suppoe the training et can be contructed within 5 hour with MOO, it may take up to 4 day to get a ame cale training et by enumeration method. Thi i a great improvement in training peed. Since umming up weighted objective function i much fater than getting all the olution of MOO, thee parameter can be ued to replace MOO in ome pecific application like tated above. 6. CONCLUSIONS AND FUTURE WORK In thi paper, we propoed a Multi-Objective Optimization (MOO) framework to olve the ad rank problem in ponored earch. The

contribution of thi framework are four-fold. Firtly, it i jutified that all previou work can be reduced to MOO by electing only one or two of the objective. We add CTR a an individual objective o that all participant have their own objective function. It i more intuitive and cloer to real cenario. Secondly, the MOO framework offer u a complete et of non-dominant rank reult for a pecific earch query and it candidate ad, not only covering all the reult from previou work, but alo providing potentially balanced reult in different objective function. Thirdly, from the Pareto Set in MOO framework, we can either viually chooe a proper contraint for one objective function or look up in the table of tatitical reult to find an optimized acrifice for one objective to improve other objective. Finally, we generate large cale of training data for parameter election from Pareto Set. Then we train a model to predict the parameter configuration for a pecific target of proportion of different objective function. In our future work, we will focu more on the combination of thi framework and the real-time earch engine. Orthogonal experimental deign may be applied to find optimized olution for online earch engine. Alo, for the comparion of different method, we hall formalize a pecific tandard to tell which ranking function give out good rank reult and which doe not. 7. REFERENCES [1] Özgür Çetin, K. Achan, E. Cantu-Paz, R. Iyer. Miing Click Hitory in Sponored Search: A Generative Modeling Solution. In ADKDD 1. [2] O. Chapelle, Y. Zhang. A Dynamic Bayeian Network Click Model for Web Search Ranking. In WWW 9, pp. 1-10. [3] N. Crawell, O. Zoeter, M. Taylor and B. Ramey. An Experimental Comparion of Click Poition-Bia Model. In WSDM 8, pp. 87-94. [4] G. Dupret, C. Liao. A Model to Etimate Intrinic Document Relevance from the Clickthrough Log of a Web Search Engine. In WSDM 1, pp. 181-190. [5] Y. Engel, D. M. Chickering. Incorporating Uer Utility Into Sponored-Search Auction. In AAMAS 8, pp. 1565-1568. [6] D. C. Fain, J. O. Pederen. Sponored Search: a Brief Hitory. Bulletin of the American Society for Information Science and Technology. 32(2), October 2006, pp. 12-13. [7] S. Fox, K. Karnawat, M. Mydland, S. Dumai, and T. White. Evaluating implicit meaure to improve web earch. ACM Tranaction on Information Sytem, 23(2), April 2005, pp. 147 168. [8] J. H. Friedman. Greedy Function Approximation: A Gradient Booting Machine. The Annal of Statitic, 29(5), October 2001, pp. 1180-1232. [9] F. Guo, C. Liu, Y. Wang. Efficient Multiple-Click Model in Web Search. in WSDM 9, pp. 1 4-131. [10] D. Hillard, E. Manavoglu, H. Raghavan, C. Leggetter, E. Cantú-Paz, R. Iyer. The Sum of It Part: Reducing Sparity in Click Etimation with Query Segment. Information Retrieval, 14(3), June 2011, pp. 315-336. [11] D. Hillard, S. Schroedl, E. Manavoglu, H. Raghavan and C. Leggetter. Improving Ad Relevance in Sponored Search. In WSDM 1, pp. 361-369. [12] IAB Internet Advertiing Revenue Report - 2010 Full Year Reult. http://www.iab.net/adrevenuereport [13] K. Jarvelin, J. Kekalainen. Cumulated gain-baed evaluation of IR technique. In ACM Tranaction on Information Sytem, 20(4), October 2002, pp. 422 446. [14] T. Joachim. Optimizing Search Engine uing Clickthrough Data. In SIGKDD, pp. 1-142. [15] T. Joachim, L. Granka, B. Pan, H. Hembrooke, F. Radlinki, and G. Gay. Evaluating. Evaluating the accuracy of implicit feedback from click and query reformulation in web earch. ACM Tranaction on Information Sytem, 25(2):7, April 2007. [16] S. Kullback, R. A. Leibler. On Information and Sufficiency. In Annal of Mathematical Statitic, 22(1), 1951, pp. 79 86. [17] T. Liu. Learning to Rank for Information Retrieval. Foundation and Trend in Information Retrieval, 3(3), March 2009, pp. 225-331. [18] R. T. Marler, J. S. Arora. Survey of multi-objective optimization method for engineering. Structural and Multidiciplinary Optimization, 26(6), April 2004, pp. 369-395. [19] R. Meng,Y. Ye,N. Xie. Multi-Objective Optimization Deign Method Baed on Game Theory. In WCICA 1, pp. 2220-2227. [20] F. Radlinki, A. Broder, P. Ciccolo, E. Gabrilovich, V. Joifovki, L. Riedel. Optimizing Relevance and Revenue in Ad Search: A Query Subtitution Approach. In SIGIR 8, Singapore, pp. 403-410. [21] H. Raghavan, D. Hillard. A Relevance Model Baed Filter for Improving Ad Quality. In SIGIR 09, pp. 762-763. [22] M. Richardon, E. Dominowka, R. Ragno. Predicting Click: Etimating the Click-Through Rate for New Ad. In WWW 07, pp. 521-529. [23] Y. Sawaragi, H. Nakayama, T. Tanino. Theory of Multiobjective Optimization. Academic Pre in Orlando, 1985. [24] I. F. Sbalzariniy, S. M üllery, P. Koumoutakoyz. Multiobjective optimization uing evolutionary algorithm. In Center for Turbulence Reearch Proceeding of the Summer Program 2000, pp. 63-74 [25] S. Schroedl, A. Keari, A. Nair, L. Neumeyer, S. Rao. Generalized Utility in Web Search Advertiing. In ADKDD 10. [26] D. Sculley, R. Malkin, S. Bau, R. J. Bayardo. Predicting Bounce Rate in Sponored Search Advertiement. In KDD 09, pp. 1325-1333. [27] B. Shaparenko, Özgür Çetin, R. Iyer. Data-Driven Text Feature for Sponored Search Click Prediction. In ADKDD 09, pp. 46-54. [28] D. Wang, W. Chen, G. Wang, Y. Zhang, B. Hu. Explore Click Model for Search Ranking. In CIKM 10, pp. 1417-1420. [29] Y. Yue, R. Patel, H. Roehrig. Beyond Poition Bia: Examining Reult Attractivene a a Source of Preentation Bia in Clickthrough Data. In WWW 10, pp. 1011-1018. [30] Y. Zhu, G. Wang, J. Yang, D. Wang, J. Yan, Z. Chen. Revenue Optimization with Relevance Contraint in Sponored Search. In ADKDD 09, pp. 55-60. [31] Y. Zhu, G. Wang, J. Yang, D. Wang, J. Yan, J. Hu, Z. Chen. Optimizing Search Engine Revenue in Sponored Search. In SIGIR 09, pp. 588-595. [32] Z. A. Zhu, W. Chen, T. Minka, C. Zhu, Z. Chen. A Novel Click Model and It Application to Online Advertiing. In WSDM 1, pp. 321-330.