Real Time Bid Optimization with Smooth Budget Delivery in Online Advertising

Transcription

1 Real Time Bid Opimizaion wih Smooh Budge Delivery in Online Adverising Kuang-Chih Lee Ali Jalali Ali Dasdan Turn Inc. 835 Main Sree, Redwood Ciy, CA ABSTRACT Today, billions of display ad impressions are purchased on a daily basis hrough a public aucion hosed by real ime bidding (RTB) exchanges. A decision has o be made for adverisers o submi a bid for each seleced RTB ad reques in milliseconds. Resriced by he budge, he goal is o buy a se of ad impressions o reach as many argeed users as possible. A desired acion (conversion), adveriser specific, includes purchasing a produc, filling ou a form, signing up for s, ec. In addiion, adverisers ypically prefer o spend heir budge smoohly over he ime in order o reach a wider range of audience accessible hroughou a day and have a susainable impac. However, since he conversions occur rarely and he occurrence feedback is normally delayed, i is very challenging o achieve boh budge and performance goals a he same ime. In his paper, we presen an online approach o he smooh budge delivery while opimizing for he conversion performance. Our algorihm ries o selec high qualiy impressions and adjus he bid price based on he prior performance disribuion in an adapive manner by disribuing he budge opimally across ime. Our experimenal resuls from real adverising campaigns demonsrae he effeciveness of our proposed approach. Caegories and Subjec Descripors 9.B [Economy, markes]: Online adverising; 3.B [Big Daa]: Scalable mehods; 1.B [Adapive learning]: Adapive experimenaion General Terms Online Adverising, Smooh Budge Delivery, Scalable Algorihms Keywords Bid Opimizaion, Algorihmic Adverising, Budge Pacing, Smooh Budge Delivery, Compuaional Adverising Permission o make digial or hard copies of all or par of his work for personal or classroom use is graned wihou fee provided ha copies are no made or disribued for profi or commercial advanage and ha copies bear his noice and he full ciaion on he firs page. To copy oherwise, o republish, o pos on servers or o redisribue o liss, requires prior specific permission and/or a fee. ADKDD 13, Augus 11, Chicago, Illinois, U.S.A.. Copyrigh 2013 ACM $ INTRODUCTION In recen years, he amoun of ad impressions sold hrough real ime bidding (RTB) exchanges has had a remendous growh. RTB exchanges provide a echnology for adverisers o algorihmically place a bid on any individual impression hrough a public aucion. This funcionaliy enables adverisers o buy invenory in a cos effecive manner, and serve ads o he righ person in he righ conex a he righ ime. However, in order o realize such funcionaliy, adverisers need o inelligenly evaluae each impression in real ime. Demand-side plaforms (DSPs) offer such a soluion called real ime bid opimizaion [13, 16] o help adverisers find he opimal bid value for each ad reques in milliseconds close o a million imes per second. The process of real ime bid opimizaion ries o maximize he campaign performance goal under he delivery consrain wihin he budge schedule. The performance goals ypically can be specified by minimizing cos-per-click (CPC) or cos-per-acion (CPA), as well as by maximizing click-hrough-rae (CTR) or acion-rae (AR). Typically, a smooh budge delivery consrain, expressed as no buying more han a se fracion of he impressions of ineres before a se ime, is used o preven he campaign from finishing he budge premaurely or avoiding a bursy spending rae. This consrain generally helps he adverisers o have susainable influence wih heir ads, avoid pushing large amoun of ads in peak raffic (while performance may be degraded), and explore a broader range of audience. I is challenging o perform real ime bid opimizaion in a RTB environmen for many reasons, including he following. Firsly, he decision of placing a bid and evaluaion of he bid price needs o be performed per ad reques in few milliseconds. In addiion, op DSPs ypically receive as many as a million ad requess per second while hundreds of millions of users simulaneously explore he web around he globe. The shor laency and high hroughpu requiremens inroduce exreme ime sensiiviy on he process. Secondly, los of informaion is missing in he real ime evaluaion of he individual ad requess, e.g., he feedback on previous decisions has normally a long delay in pracice. More specifically, he collecion of click informaion is delayed because of he duplicaion removal during he logging process. On he oher hand, mos of he view-hrough acions ofen require up o seven days o be convered and aribued o he corresponding impressions. Finally, click and conversion evens are usually very rare for non-search adverisemen and herefore he variance will be large while esimaing he pas performance merics.

2 Budge Spend Budge Spend Budge Spend (a) Premaure Sop (b) Flucuaing Budge (c) Uniform Pacing Budge Spend Budge Spend (d) Traffic Based Pacing (e) Performance Based Pacing Figure 1: Illusraion of differen budge pacing schemes wih respec o he porion of he budge spen every ime inerval. Inegraing hese plos gives he porion of he budge spend from he beginning of he day. In his paper, we presen an online approach o opimize he performance merics while saisfying he smooh delivery consrain for each campaign. Our approach firs applies a conrol feedback loop o ieraively esimae he fuure spending rae in order o impose smooh delivery consrains. Then, he spending rae is used o selec high qualiy impressions and adjus he bid price based on he prior performance disribuion o maximize he performance goal. The res of he paper is organized as follows. In 2, we formulae our problem and deail previous relaed work. In 3, we describe our proposed approach of online bid opimizaion. Various pracical issues encounered during bid opimizaion and he proposed soluions are discussed in 4. Thorough experimenal resuls are presened in 5, and in 6 we conclude by a discussion of our approach and possible fuure work. 2. BACKGROUND AND RELATED WORK In his secion, we firs formulae he problem of bid opimizaion as an online linear programming, and hen discuss he previous relaed work in he lieraure, and explain why hose proposed soluions are no suiable for our online bid opimizaion problem in pracice. 2.1 Problem Seup Le us consider he online bid opimizaion in he following seings: There are n ad requess arriving sequenially ordered by an index i. An individual adveriser would like o make a decision represened by an indicaor variable x i {0, 1} for all i {1,..., n} wheher o place a bid on he i h ad reques or no. We consider a oal daily budge B R as he oal cos of acquiring ad invenory. Typically, adverisers would like o have smooh budge delivery consrain, expressed as no buying more han a se fracion of he impressions of ineres before a se ime, in place o ensure he following wo siuaions will never occur: Premaure Campaign Sop: Adverisers do no wan heir campaigns o run ou of he budge premaurely in he day so as no o miss opporuniies for he res of day. Such premaure budge spend is shown in Fig. 1(a) finishing he budge 6 hours early. Flucuaion in Spend: Adverisers would like o be able o analyze heir campaigns regularly and high flucuaions in he budge makes he consisency of he resuls quesionable. Tha is why a budge pacing scheme similar o wha is shown in Fig. 1(b) is no suiable. A simple, ye widely used, budge pacing scheme ha mees he smooh delivery consrains is uniform pacing or even pacing shown in Fig. 1(c). In his scheme he budge is uniformly spli across he day. There are wo main issues wih his simple scheme as follows: Traffic Issue: Depending on he arge audience, he volume of he online raffic varies a lo hroughou he day. I migh be he case ha during he firs half of he day, we receive more relevan raffic comparing o he second half of he day; however, uniform budge pacing scheme does no allocae he budge accordingly. As a resul, eiher we migh no be able o deliver he budge by he end of he day or we migh be forced o buy low qualiy impressions in he second half of he day. A uniform budge pacing wih respec o he raffic (as opposed o wih respec o he ime) migh resolve his issue o some exen. Such scheme is depiced in Fig. 1(d).

3 Performance Issue: The qualiy of he online raffic changes over he course of he day for differen groups of audience. Wheher his qualiy is being measured by CPC, CPA, CTR or AR, he budge pacing algorihm should allocae mos of he budge o ime periods of he day wih high qualiy. Such scheme is depiced in Fig. 1(e) and ofen has few picks for he periods wih high qualiy. This poenially can cause high flucuaions ha migh violae smooh delivery consrains. Balancing he raffic and performance under smooh delivery consrains is challenging. In his paper, we propose a scheme ha resolves boh of hese issues simulaneously. In order o enforce he smooh delivery consrains (explained furher in 1), he overall daily budge B can be broken down ino a sequence of ime slo schedules {b 1,..., b T }, where, b R represens he allocaed budge o he ime slo, and T =1 b = B. In he nex secion, we will inroduce how o impose differen pacing sraegies o assign b s in order o selec higher qualiy impressions. Each ad reques i is associaed wih a value v i R and a cos c i R. The value v i represens he rue value for he adveriser if he given ad reques i has been seen by an audience. The cos c i represens he acual adveriser cos for he ad reques i paid o he publisher serving he corresponding impression. In summary, he bid opimizaion problem wih smooh budge delivery consrain can be formulaed as n maximize v ix i subjec o i=1 j I c jx j b {1,..., T }, (1) where, I represens he index se of all ad requess coming in he ime slo. Obviously his opimizaion problem is an offline formulaion due o he fac ha he cos and value of fuure ad requess are no clear a he ime of decision on x i. More precisely, afer he (curren) incoming ad reques i is received, he online algorihm of bid opimizaion mus make he decision x i wihou observing furher daa. For dynamic bidding campaigns, he opimizaion process also needs o esimae ĉ i as he bid price a he same ime. Please noe ha he bid price is no equivalen o he cos c i for he incoming ad reques i, because he cos is deermined by a second price aucion in he RTB exchange. More clearly, one should bid ĉ i = c i + ɛ i o be able o win he second price aucion and acually pay c i. The value of ɛ i is deermined based on he aucion properies and is unknown o he bidder a he bidding ime. 2.2 Relaed Work Eq. (1) is ypically called online linear programming, and many pracical problems, such as online bidding [8, 19], online keyword maching [21], online packing [11], and online resource allocaion [7], can be formulaed in he similar form. However, we do no aemp o provide a comprehensive survey of all he relaed mehods as his has been in a number of papers [3, 6]. Insead we summarize couple of represenaive mehods in he following. Zhou e al. [21] modeled he budge consrained bidding opimizaion problem as an online knapsack problem. They proposed a simple sraegy o selec high qualiy ad requess based on an exponenial funcion wih respec o he budge period. As ime goes by, he proposed algorihm will selec higher and higher qualiy of ad requess. However, his approach has an underlying assumpion of unlimied supply; i.e., here are infinie amoun of ad requess in he RTB environmen. This assumpion is impracical especially for hose campaigns wih sric audience argeing consrains. Babaioff e al. [5] formulaed he problem of dynamic bidding price using muli-armed bandi framework, and hen applied he sraegy of upper confidence bound o explore he opimal price of online ransacions. This approach does no require any informaion abou he prior disribuion. However, muli-armed bandi framework ypically needs o collec feedback quickly from he environmen in order o updae he uiliy funcion. Unforunaely, he collecion of bidding and performance informaion has longer delay for display adverising in RTB environmen. Agrawal e al. [3] proposed an general online linear programming algorihm o solve many pracical online problems. Firs hey applied he sandard linear programming solver o compue he opimal dual soluion for he daa which have been seen in he sysem. Then, he soluion for he new insance can be decided by checking if he dual soluion wih he new insance saisfies he consrain. The problem is ha he rue value v i and cos c i for he incoming ad reques is unknown when i arrives o he sysem. If v i and c i is esimaed by some saisical models or oher alernaive soluions, he dual soluion needs o be re-compued more frequenly for each campaign in order o impose budge consrains accuraely. This inroduces high compuaional cos in he real ime bidding sysem. 3. ONLINE BID OPTIMIZATION In his secion, we deail our mehod of online bid opimizaion. We firs revisi he smooh delivery consrain and explain how we conrol he spending rae adapively when each ad reques comes sequenially. Aferwards, we discuss how we can ieraively apply he spending informaion o selec ad requess and adjus heir bid price o opimize he objecive funcion. One should recognize wo differen classes of campaigns: (i) Fla CPM campaigns, and, (ii) Dynamic CPM (dcpm) campaigns. The main difference beween he wo ypes is ha he firs one submis a fla bid price whereas he second one opimized he bid price. Boh also need o decide wheher o bid on an ad reques. The meric for he goodness of he decision wih fla CPM campaigns is ypically eiher CTR or AR; while he meric for dcpm is ypically effecive CPC (ecpc) or effecive CPA (ecpa). The difference beween CPA and ecpa is ha CPA is he goal o reach while ecpa is wha is acually realized. Regardless of he ype of he campaign, we ry o opimize he following goal: min -CTR, -AR, ecpc or ecpa T s.. s() B ɛ s() b δ {1,..., T } ecp M M where he firs consrain is he oal daily budge consrain (where s() is he budge spen a ime slo ), he second consrain enforces smooh delivery according o he schedule b and he hird consrain requires ha ecpm does no (2)

4 exceed he cap M. The las consrain makes a dcpm campaign appear like a CPM campaign in average over ime, hence, he use of CPM in dcpm. In his formulaion, he opimizaion parameer is b, since he oal budge B and average impression cos cap M are se by he adveriser. We deail our budge pacing scheme in he res of his secion and show how we improve his opimizaion by smar budge pacing. 3.1 Smooh Delivery of Budge The original idea of budge pacing conrol is o ake he daily budge as inpu and calculae a delivery schedule in real-ime for each campaign. Based on he delivery schedule, he DSP will ry o spread ou he acions of acquiring impressions for each campaign hroughou he day. We break down a day ino T ime slos and in each ime slo, we assign a budge o be spen by each campaign. In he ime slo, he spend of acquiring invenory is considered o be proporional o he number of impressions served a ha ime slo; assuming he price of individual impressions for a paricular campaign remains approximaely consan during ha ime slo. In realiy, he lengh of he ime slo should be chosen such ha he variance of individual impression price for each campaign is small. Our analysis shows ha his assumpion holds on our real daa if he lengh of he ime slo is properly chosen. To each campaign, we assign a pacing rae for each ime slo. The pacing rae is defined o be he porion of incoming ad requess ha his campaign would like o bid on. To see he relaionship of he pacing rae wih oher parameers of our proposed bid opimizaion sysem, consider he following equaion derived from consrains in Eq. 1: s() = j I c jx j imps() reqs() bids() imps() reqs() bids() reqs() pacing rae() win rae() Here, s() is he dollar amoun of money spen, reqs() is he number of incoming ad requess ha saisfy he audience argeing consrains of he campaign, bids() is he number of ad requess ha his campaign has bid on, and finally imps() is he oal number of impressions of he campaign, i.e., he bids ha are won in he public aucion, all during he ime slo. Wih hese definiions, we naurally define he raio of bids o ad requess as pacing rae and he raio of impressions o bids as win rae. Noice ha if we assume hose ad requess ha saisfy he audience argeing of a cerain campaign are uniformly disribued across all of he incoming ad requess, hen, one can replace reqs() wih some consan imes he oal number of incoming ad requess. Tha consan can be absorbed by he proporion in (3). To make progress, we wan o ake a dynamic sequenial approach in which, we ge a feedback from he previous ime slo spend and adjus our pacing rae for he nex ime slo. By working on he proporion (3), he pacing rae for he nex ime slo + 1 can be obained by a simple recursive (3) equaion as follows: pacing rae(+1) (4) = pacing rae() s(+1) reqs() win rae() s() reqs(+1) win rae(+1) = pacing rae() b+1 reqs() win rae() s() reqs(+1) win rae(+1) where, reqs(+1) and win rae(+1) represen he prediced number of ad requess and he prediced winning rae for he bids in he nex ime slo + 1. One can do such predicions using hisorical daa keeping in mind ha we are only ineresed in he raio of hese parameers o heir previous values and no necessarily heir absolue values. This recursive definiion inroduces a simple adapive feedback conrol for smooh budge pacing. The fuure spend, i.e., s(+1), in (4) is se o be equivalen o he ideal desired spend b +1 a ime slo + 1 in order o impose he budge consrain. Differen choices of b +1 inroduces differen sraegies for he budge pacing. For example, one simple sraegy, called uniform pacing, ries o spend he budge evenly for he given campaign hroughou he day. This sraegy can be easily implemened by defining fuure spend b u +1 (u o denoe uniform) as ( b u +1 = B ) L( + 1) s(m) T m=+1 L(m) (5) m=1 where, he firs facor represens he remaining budge of he day and he second facor is he raio of he lengh of he ime slo + 1 o he remaining ime in he day; i.e., L() is he lengh of he ime slo. If ime slos have equal lengh, one can simplify (5) o ge b u +1 = ( B m=1 ) 1 s(m) T. (6) Uniform pacing is no necessarily he bes sraegy as discussed in 2.1. We propose a sraegy o spend more money on he ime slos where a paricular campaign has more chance o ge evens of ineres (clicks or conversions). To do so, we look a he campaign hisory daa and measure he performance of he campaign during each ime slo. Based on his measuremen, we build a discree probabiliy densiy funcion described by a lis of click or conversion probabiliies: p 0,..., p T assuming T ime-slos per day, and T p = 1. Now a each ime slo, we compue he ideal spending b p +1 (p o denoe probabilisic) for he nex ime slo as ( ) b p p +1 L(+1) +1 = B s(m) T. (7) m=+1 pm L(m) m=1 Similar o he uniform pacing case, if he ime slos have equal lenghs, his can be simplified o ( b p +1 = B ) p +1 s(m) T. (8) m=+1 pm m=1 In pracice, i is imporan o noice ha if p j = 0 for some j, hen ha campaign will never spend money during ha ime slo and hence, i will never explore he opporuniies coming up during ime slo j. To preven his siuaion, one can spli he budge and use a combinaion of he above

5 Number of Ad Requess q (x) τ() r*() CTR or AR In pracice, since he CTR or AR disribuion q (x) is no compued frequenly, i inroduces some oscillaions for he hreshold τ() in differen ime slos if q (x) is no close o he curren ad reques disribuion. Noe ha his mismach is very probable since q (x) is generaed from hisorical daa ha migh no perfecly correlae wih he curren realiy. In order o preven his siuaion, we evaluae a confidence inerval of he hreshold parameer τ(). Firs, we incremenally updae he mean µ τ () and variance σ τ () of he hreshold τ() using he online adapaion as follows Figure 2: Illusraion of our performance based budge pacing for fla CPM campaigns. wo sraegies. This way, here will be always a chance o explore all possible opporuniies. Afer he pacing rae is calculaed, each campaign can apply his informaion o adapively selec cerain porions of high qualiy impressions, as well as adjus he bid price in order o maximize he objecive funcion. We explain hose deails in he nex subsecions. 3.2 Selecion of High Qualiy Ad Requess - Fla CPM Campaigns We consider wo cases of fla CPM and dynamic CPM separaely as he former, unlike he laer, does no need a bid price calculaion. For fla CPM campaigns ha always submi a fixed bid price c o RTB exchanges, he goal is o simply selec a se of ad requess o bid on considering he curren ime slo pacing rae. Since we do no know if he curren incoming ad reques will evenually cause a click or conversion even during he ime of bid opimizaion, he rue value of he ad reques is esimaed by he predicion of is CTR or AR using a saisical model. The deails of he offline raining process of CTR or AR predicion is described in 3.4. Noice ha o fulfill he smooh delivery consrain, we require a minimum number of impressions given by imps () = s() c. This number of impressions can be reached only if we have bids () = imps () win rae(). Similarly, o ge hese many bidding opporuniies, we expec o have reqs () = bids () pacing rae(). Now, we are going o selec hese ad requess from he se of incoming ad requess whose chance of a click or a conversion is high. To do so, we consruc an empirical hisogram of CTR or AR disribuion q (x) based on he hisorical daa for each campaign, where q (x) represens he number of ad requess in ime slo ha are believed o have CTR or AR of x, e.g., see Fig. 2. Our online algorihm finds a hreshold τ() in he ime slo o filer ad requess in he region of q (x) ha has low CTR or AR rae such ha he smooh delivery is fulfilled. Such hreshold can be formulaed as 1 τ() = arg min x q (s)ds reqs () (9) x µ τ () = µ τ ( 1) + 1 (τ() µτ ( 1)) (10) στ 2 () = 1 στ 2 ( 1)+ 1 (τ() µτ ( 1)) (τ() µτ ()) Second, assuming ha τ() comes from a Gaussian disribuion, we bound τ(). The upper and lower bounds of he hreshold τ() can be saed as µ τ () + γ and µ τ () στ () d γ ino he hisory of he daa o make he saisics. The criical value γ = 1.96 provides 95% confidence inerval. The upper bound and lower bound of he CTR or AR hreshold are updaed in each ime slo. Puing all ogeher, when an ad reques comes o he sysem, is CTR or AR value is firs esimaed by he saisical model. If he prediced value is larger han he upper bound of he hreshold, his ad reques will be kep and he fixed bid price v will be submied o he RTB exchange. If he prediced value is smaller han he lower bound of he hreshold, his ad reques will be simply dropped wihou furher processing. If he prediced value is in beween he upper and lower bounds, he ad reques will be seleced a random wih probabiliy equal o pacing rae(). This scheme, alhough approximae, ensures ha he smooh delivery consrain is me while he opporuniy exploraion coninues on he boundary of high and low qualiy ad requess. στ () d, respecively, where d is he number of days we looked 3.3 Selecion of High Qualiy Ad Requess - Dynamic CPM Campaigns For dcpm campaigns, which are free o change he bid price c i dynamically for each incoming ad reques, he goal is o win enough number of high qualiy impressions for less cos. We firs consruc bidding hisogram, wih c being he hisorical average of c i in ime slo, o represen he saisics of good and bad impressions as discussed in he previous subsecion. Then, saring from a base bid price, we scale he bid price up or down properly considering he curren pacing rae() o mee he budge obligaion. We explain he second sep in his subsecion. For simpliciy and wihou loss of generaliy, we will focus on CPA campaigns. Noice ha pacing rae() conrols he frequency of bidding; however, if he submied bid price is no high enough, he campaign migh no win he impression in he public aucion. On he oher hand, if he bid price is very high, hen cos per acion migh rise (even in second price aucion as he oher bidders will increase heir bid price oo). To adjus he bid price, defining hresholds 0 β 1 β 2 1, we consider hree regions for he pacing rae: (a) safe region: when pacing rae() β 1 and here is no under delivery issue due o audience argeing, (b) criical region: when β 1 pacing rae() β 2 and he delivery is normal, and (c) danger region: when β 2 pacing rae() and he cam-

6 paign has a hard ime o find and win enough impressions. We rea each of hese cases separaely. Typically dcpm campaigns work owards meeing or beaing a CPA goal G (compared wih ecpa). We use his goal value o define a base bid price u i = AR G, where AR is he prediced AR for he curren ad reques. We discuss he esimaion of he AR in he nex subsecion. If our campaign is in he criical region, we submi ĉ i = u i as our bid price since he campaign is doing jus fine in meeing all he obligaions. For he case where our campaign is in he safe region, we sar learning he bes bid price from he second price aucion. In paricular, we consider he difference beween our submied bid price and he second price we acually pay for boh good and bad impressions. If he AR esimaion algorihm is a high qualiy classifier, one expecs o see bigger differences for high qualiy impressions compared o low qualiy ones. The reason is ha a good classifier ypically generaes high AR for high qualiy impressions resuling in high values of u i and hence a bigger difference from he second price unless all he bidders use he same or similar algorihm. Suppose in he pas we have submied ĉ i as our bid price and we acually paid c i. For hose impressions ha led o an acion, we can build he hisogram of θ = c i ĉ i and find he θ o be he boom 1 or 2 percenile on ha hisogram. We propose o submi ĉ i = θ u i as he bid price in his case. Obviously, his scheme hurs he spending while improving he performance; however, his is no a problem because he campaign is in he safe region. Finally, for campaigns in he danger region, we need o undersand why hose campaigns are in his region. There are wo main reasons for underdelivery in his design: (i) The audience argeing consrains are oo igh and hence, here are no enough incoming ad requess seleced for a bid, and, (ii) he bid price is no high enough o win he public aucion even if we bid very frequenly. There is nohing we can do abou he firs issue, however, we can fix he second issue by boosing he bid price. Consider a bid price cap C which in realiy is being se by each RTB exchange. We would like o boos he bid price wih parameer ρ 1 so ha if pacing rae is greaer han β 2, he parameer ρ increases as pacing rae increases. One suggesion can be a linear increase as ρ = 1 + C/c 1 1 β 2 (pacing rae() β 2). (11) A he end we submi ĉ i = ρ u i as our bid price. Noice ha c as defined in he beginning of his subsecion is he average hisorical value of c i and i dynamically (and slowly) changes as he marke value of he impressions change. 3.4 Esimaion of CTR and AR Thus far in he paper, we based our algorihm on he op of he assumpion ha we have a good sysem o accuraely esimae click hrough rae (CTR) and acion or conversion rae (AR). In his secion, we will describe how o do his esimaion. Again for simpliciy and wihou loss of generaliy, we will focus on AR. This esimaion plays a crucial role in bid opimizaion sysem for many reasons including he followings. Firsly, he esimaed AR provides a qualiy assessmen for each ad reques helping o decide on aking acion on ha paricular ad reques. Secondly, he base bid price is se o be he esimaed AR muliplied by he CPA goal, which direcly affecs he cos of adverising. There are many proposals for esimaing AR in he lieraure. Since conversions are rare evens, hierarchical srucure of feaures for each riple combinaion of (user, publisher, adveriser) have been commonly used o smooh and impue he AR for he leaf nodes ha do no have enough conversion evens [1, 2, 13, 14, 20]. On he oher hand, here are also some sudies ha ry o cluser users based on heir behaviors and ineress and hen esimae AR in each user cluser, e.g., see [4, 9, 16]. In addiion, many sandard machine learning mehods, e.g., logisic regressions [13, 17] and collaboraive filering [15], are used o combine muliple AR esimaes from differen levels in he hierarchy or user clusers o produce a final boosed esimae. We use he mehodology inroduced in [13] and make some improvemens on he op of ha. In his mehod, we would like o find he AR for each riple combinaion of (user, publisher, adveriser) by leveraging he hierarchy srucure of heir feaures. Each acual creaive (he graphic ad o be shown on he page) is a leaf in he adveriser ree hierarchy. The hierarchy sars wih he roo and coninues layer afer layer by adveriser caegory, adveriser, inserion order, package, line iem, ad and finally creaive. Using he hisorical daa, one can assign an AR o each node in his ree by aggregaing oal number of impressions and acions of heir children as a raw esimae. Same hierarchy and raw esimaions can be done for publisher and user dimensions. Afer consrucing all hree hierarchies wih iniial raw esimaes, we employ a smoohing algorihm similar o he one discussed in [1, 2] o adjus he raw esimaes on differen levels based on he similariy and closeness of (user, publisher, adveriser) riple on he hierarchy rees. Then, for each riple on he leaves of he rees, we run a logisic regression over he pah from ha leaf o he roo. This scheme resuls in a fairly accurae esimaion of AR. 4. PRACTICAL ISSUES In his secion, we discuss several pracical issues encounered during he implemenaion of our proposed bid opimizaion mehod and presen our curren soluions. 4.1 Cold Sar Problem When a new campaign sared, click or conversion evens require some ime o feedback o he sysem and herefore here is no sufficien informaion o perform CTR or AR esimaion as well as bid opimizaion. This is known as cold sar problem and has been well sudied in he lieraure, e.g., see [10, 12, 18]. We follow similar ideas o apply conen feaures such as user and publisher aribues o recommend a lis of high qualiy websies and audience groups by inferring similariies among exising campaigns. In addiion, a conexual-epsilon-greedy based sraegy is performed during he online bid opimizaion. If he incoming ad reques is inside one of hose recommended publisher or audience groups, a higher bid is placed. Oherwise, he ad reques will be randomly seleced wih a defaul bid price o explore unseen sies and users. As he campaign ges older and accrues more daa, he aciviy of online exploraion will be decreased and he regular predicion model will jump o play he major role of bid opimizaion.

7 Receive Adrequess <uid,url,ad> Esimae CTR/AR Compue bid price based on pacing rae Submi bid o exchange server Hierarchical esimaes and model parameers Evaluae pacing rae Log impression daa Receive winning noificaion Train CTR/AR predicion model Generae hierarchical esimaes Figure 3: Illusraion of ad reques workflow in our proposed bid opimizaion framework. 4.2 Prevenion of Overspending Since budge spending is conrolled by he pacing rae, if here is a huge amoun of ad requess coming all of a sudden, he overall daily spend migh exceed he allocaed daily budge B. In order o overcome his problem, several monioring processes have been implemened o frequenly check he overall daily budge spend as well as he inerval spend in each ime slo. If he overall spending exceeds he daily budge B, he campaign will be compleely sopped. If he inerval spend exceeds b + δ, he bidding aciviy will be emporarily paused unil he nex ime slo Disribued Archiecure Fig. 3 illusraes he simplified algorihmic flow char for each individual ad reques. Please noe ha his workflow o submi a bid needs o happen wihin less han 50 milliseconds, and close o a million ad requess need o be processed in a second. Therefore, he enire bid opimizaion is implemened and parallelized on many disribued compuing clusers cross differen daa ceners. The offline raining process uilizes R, Pig, and Hadoop o generae hierarchical CTR and AR esimaes as well as rain campaign-specific predicion models over a large number of campaigns. The online process sreams incoming ad requess o many servers and evaluaes he bid price via a real-ime message bus. Our proposed algorihm works very well in his disribued environmen, and deailed experimenal resuls are presened in he nex secion. 5. EXPERIMENTAL RESULTS Our proposed framework of bid opimizaion has been implemened, esed, and deployed in Turn, a leading DSP in he Inerne adverising indusry. In his secion, we presen simulaion resuls from our saging environmen o compare differen sraegies of budge pacing. In addiion, we also Ideal Uniform Spend Acual Spend Figure 4: Simulaion resul of uniform budge spend. The x-axis represens he ime slo, and he y-axis represens he oal budge spend. The blue line depics he ideal uniform spend, and he red line depics he acual spend in our uniform budge pacing. show resuls from real campaigns ha serve large amouns of daily impressions in order o demonsrae he overall performance improvemen in erms of CPC or CPA performance merics. 5.1 Comparison of Pacing Sraegies In his secion, we would like o show he simulaion resuls of he budge pacing in our saging environmen o verify ha our proposed bid opimizaion framework does no violae he budge consrain specified in Eq. 1, i.e., do no overpace or underpace. For he simulaion experimen,

8 Mehod FC1 FC2 FC3 FC4 FC5 FC6 FC7 Our proposal Baseline Improvemen (%) 286% 59% 63% 155% 132% 47% 124% Table 1: CTR improvemen for seven seleced fla CPM campaigns Rela.ve Performance Figure 5: Relaive performance across differen ime slo. The x-axis represens he ime slo, and he y- axis represens he relaive performance in erms of 1 ecp A Ideal Spend Acual Spend Figure 6: Simulaion resul of budge spend based on performance disribuion. The x-axis represens he ime slo, and he y-axis represens he oal budge spend. The blue line depics he ideal spend based on he performance disribuion, and he red line depics he acual spend in our performancebased budge pacing. we launch a fla CPM pseudo-campaign and assign a fixed amoun of daily budge. A se of ad requess are randomly generaed in each ime slo. The simulaion server hen generaes he bids based on he pacing rae and logs he winning impressions ino he daabase. Fig. 4 shows he ideal uniform spend and he acual spend using our uniform pacing sraegy in Eq. 6. We can noice ha he wo lines are prey close o each oher, and he average difference across all ime slos is 0.56 dollars, which is less han 1% error compared o he daily budge. Fig. 6 shows he ideal spend and he acual spend using performance based pacing sraegy in Eq. 8 based on he relaive performance shown in Fig. 5. We can noice ha he acual spending curve indeed follows he ideal spending curve, and he average difference is 0.17 dollars, which is abou 2.3% error of he daily budge. 5.2 Evaluaion of Real Campaign Performance In his secion, we evaluae he enire bid opimizaion framework wih respec o wo major classes of campaigns in our sysem: fla CPM campaigns and dynamic CPM campaigns. For he evaluaion of fla CPM campaigns, he CTR meric is used and he higher rae represens beer performance. For he evaluaion dynamic CPM campaigns, CPC and CPA merics are used based because hese merics ake boh oal cos of impressions and he oal number of clicks and conversions ino accoun. The lower values for CPC and CPA merics represen beer performance. We firs repor he performance improvemen in seven acive fla CPM campaigns randomly seleced across differen adveriser caegories. Those seven campaigns were se o run based on our proposed approach and he exising baseline mehod. Each mehod was run for one week and finally wo weeks of daa were colleced for performance comparison. Our baseline mehod is a simple adapive feedback conrol algorihm ha muliplies a consan facor o he curren hreshold of CTR τ() based on he pacing rae in he ime slo. The firs wo rows shown in Table 1 represen he CTR performance in our proposed approach and he baseline mehod respecively. The hird row shows he percenage of improvemen for each individual campaign. The average performance lif achieved by our proposed approach is 123%. Nex we would like o repor he performance improvemen for dynamic CPM campaigns. In his evaluaion, we ry o compare our proposed approach wih he exising baseline mehod ha only applies he pacing rae o uniformly selec incoming ad requess wihou furher adjusmen of bid price. Two differen ses of campaigns based on he goal ype (CPC and CPA) were randomly seleced across differen adveriser caegories. The CPC and CPA values and he percenage of performance lif for each individual campaign are shown In Table 2 and Table 3. We can observe ha all weny seleced campaigns running on our proposed framework perform much beer in erms of CPC and CPA merics and he average performance lif is 19.02% for CPC campaigns, and 30.87% for CPA campaigns. 6. CONCLUSIONS We have presened a general and sraighforward approach o perform budge and smooh delivery consrained bid opimizaion for adverising campaigns in real ime. Due o he simpliciy of our algorihm, our curren implemenaion can handle up o a million of ad requess per second and we hink i can scale o even more. Our experimenal evaluaion wih simulaed and real campaigns shows ha our proposed algorihm provides consisen improvemens in sandard performance merics of CPC and CPA wihou underpacing or overpacing. In he fuure, we would like o inegrae he capabiliy of real ime analyics o perform online opimizaion across more user, publisher, and adveriser s aribues.

9 Mehod DC1a DC2a DC3a DC4a DC5a DC6a DC7a DC8a DC9a DC10a Our proposal $1.32 $1.29 $7.92 $1.30 $1.98 $0.22 $2.77 $0.58 $3.23 $1.18 Baseline $1.21 $1.12 $5.61 $0.8 $1.8 $0.21 $2.15 $0.53 $3.07 $1.16 Improvemen (%) 9.4% 14.73% 41.14% 62.89% 9.95% 6.31% 28.37% 10.82% 5.41% 1.14% Table 2: CPC improvemen for en seleced dynamic CPM campaigns. Mehod DC1 DC2 DC3 DC4 DC5 DC6 DC7 DC8 DC9 DC10 Our proposal $ $3.14 $1.37 $59.32 $68.70 $ $16.64 $3.76 $ $ Baseline $ $3.27 $1.34 $65.44 $72.95 $ $22.93 $4.31 $ $ Improvemen (%) 39.43% 4.45% -2.23% 10.31% 6.2% 85.84% 37.78% 14.6% 76.82% 35.51% Table 3: CPA improvemen for en seleced dynamic CPM campaigns. Acknowledgmens We would like o hank Xi Yang and Changgull Song for esing he enire bid opimizaion framework in he saging environmen. 7. REFERENCES [1] D. Agarwal, R. Agrawal, and R. Khanna. Esimaing raes of rare evens wih muliple hierarchies hrough scalable log-linear models. ACM SIGKDD Conf. on Knowledge Discovery and Daa Mining, [2] D. Agarwal, A. Broder, D. Chakrabari, D. Diklic, V. Josifovski, and M. Sayyadian. Esimaing raes of rare evens a muliple resoluions. ACM SIGKDD Conf. on Knowledge Discovery and Daa Mining, [3] S. Agrawal, Z. Wang, and Y. Ye. A dynamic near-opimal algorihm for online linear programming. arxiv preprin arxiv: , [4] A. Ahmed, Y. Low, M. Aly, V. Josifovski, and A. J. Smola. Scalable disribued inference of dynamic user ineress for behavioral argeing. ACM SIGKDD Conf. on Knowledge Discovery and Daa Mining, [5] M. Babaioff, S. Dughmi, R. Kleinberg, and A. Slivkins. Dynamic pricing wih limied supply. The 13h ACM Conference on Elecronic Commerce, [6] M. Babaioff, N. Immorlica, D. Kempe, and R. Kleinberg. Online aucions and generalized secreary problems. ACM SIGecom Exchanges, 7(2):7, [7] A. Bhalga, J. Feldman, and V. Mirrokni. Online allocaion of display ads wih smooh delivery. ACM SIGKDD Conf. on Knowledge Discovery and Daa Mining, [8] C. Borgs, J. Chayes, O. Eesami, N. Immorlica, K. Jain, and M. Mahdian. Dynamics of bid opimizaion in online adverisemen aucions. Proceeding of he 16h inernaional conference on World Wide Web, [9] D. Cerrao, R. Jones, and A. Gupa. Classificaion of proxy labeled examples for markeing segmen generaion. ACM SIGKDD Conf. on Knowledge Discovery and Daa Mining, [10] H. Cheng, R. Zwol, J. Azimi, E. Manavoglu, R. Zhang, Y. Zhou, and V. Navalpakkam. Mulimedia feaures for click predicion of new ads in display adverising. ACM SIGKDD Conf. on Knowledge Discovery and Daa Mining, [11] J. Feldman, M. Hezinger, N. Korula, and V. S. Mirrokni. Online sochasic packing applied o display ad allocaion. ESA 10 Proceedings of he 18h annual European conference on Algorihms: Par I, [12] B. Kanagal, A. Ahmed, S. Pandey, V. Josifovski, L. Garcia-Pueyo, and J. Yuan. Focused marix facorizaion for audience selecion in display adverising. 29h IEEE Inernaional Conference on Daa Engineering, [13] K.-C. Lee, B. Oren, A. Dasdan, and W. Li. Esimaing conversion rae in display adverising from pas performance daa. ACM SIGKDD Conf. on Knowledge Discovery and Daa Mining, [14] B. M. Marlin and R. S. Zemel. Collaboraive predicion and ranking wih non-random missing daa. pages 5 12, [15] A. Menon, K. Chirapura, S. Garg, D. Agarwal, and N. Koa. Response predicion using collaboraive filering wih hierarchies and side-informaion. ACM SIGKDD Conf. on Knowledge Discovery and Daa Mining, [16] C. Perlich and B. Dalessandro. Bid opimizing and invenory scoring in argeed online adverising. ACM SIGKDD Conf. on Knowledge Discovery and Daa Mining, [17] M. Richardson, E. Dominowska, and R. Ragno. Predicing clicks: esimaing he click-hrough rae for new ads. pages , [18] A. I. Schein, A. Popescul, L. H. Ungar, and D. Pennock. Mehods and merics for cold-sar recommendaions. ACM SIGIR Conf. on Informaion Rerieval, [19] W. Zhanbg, Y. Zhang, B. Gao, Y. Yu, X. Yuan, and T.-Y. Liu. Join opimizaion of bid and budge allocaion in sponsored search. ACM SIGKDD Conf. on Knowledge Discovery and Daa Mining, [20] L. Zhang and D. Agarwal. Fas compuaion of poserior mode in muli-level hierarchical models. Neural Informaion Processing Sysems Foundaion, [21] Y. Zhou, D. Chakrabary, and R. Lukose. Budge consrained bidding in keyword aucions and online knapsack problems. Proceeding of he 17h inernaional conference on World Wide Web, 2008.