An Empirical Study of Search Engine Advertising Effectiveness

Transcription

1 An Emprcal Study of Search Engne Advertsng Effectveness Sanjog Msra, Smon School of Busness Unversty of Rochester Edeal Pnker, Smon School of Busness Unversty of Rochester Alan Rmm-Kaufman, Rmm-Kaufman Group LLC

2 1. Introducton: Jupter Research projects that onlne search advertsng 1 expendture for 006 wll be approxmately $6.5bllon. Ths s about 40% of all onlne advertsng and s expected to reach $11.1 bllon, by 011. In recent years there has also been a rapd move away from onlne dsplay advertsng (a.k.a. banner ads) and towards search related advertsng. Ths method of advertsng has become attractve for several reasons. Search engnes lke Google and Yahoo! attract very heavy traffc makng them useful marketng channels. The search engnes are also attractve because, through the search phrases, they elct nformaton about what potental consumers are nterested n allowng ads to be targeted to these nterests. In general Internet marketng has attracted nterest because of the ablty to more closely measure ts mpact than tradtonal marketng efforts such as TV, prnt and drect advertsng. The same advantage holds for search engne marketng. Ironcally, despte the vast amount of data beng contnuously collected electroncally about consumer responses to search engne advertsng, many advertsers fal to use these effectvely, and some fal to use these data at all. Ths reduces the economc beneft advertsers can derve from ths marketng channel. Search marketng advertsers create ads that are then mapped to search phrases. These ads have lnks that assocate them to a set of webpages. The advertsers place maxmum bd amounts for each search phrase sgnfyng ther wllngness to pay for a clck on the ad assocated wth that phrase. The bds themselves (n the case of Yahoo!) and the product of bds and clcks (n the case of Google ) determne the poston of the ad on the search page. Some of the clcks that occur lead drectly to sales. Ultmately advertsers would lke to know the value of each clck and calbrate ther maxmum bd amounts accordngly. Along these lnes t would be useful to understand how poston mpacts clck through rates (CTR) and ultmately sales. Advertsers would also lke to know how ad characterstcs mght nfluence CTR and sales as well. Unfortunately despte the enormous detaled data avalable consderable challenges reman to answerng these questons. In ths presentaton we wll emprcally explore some of these questons and dscuss some of the related statstcal and data challenges. As work n progress our am s to gve some answers to the man problems faced by search engne advertsers and to dentfy problems of nterest to researchers. We wll present an analyss of a rch data set of advertsng done by a sngle frm on Yahoo! and Google over the course of several months.. Descrpton of Data Set: We obtaned a data set comprsed of pad search mpressons, clcks, and orders for an onlne specalty retaler sellng automotve parts and accessores. For the advertser n queston we had 1 Ths form of advertsng s also known as keyword advertsng, pay per clck advertsng and search engne marketng. Essentally, these ads are place by the search engne based on a match between the search phrase entered but the consumer and the keywords assocated wth the ad. Google s rankng mechansm s actually less transparent than depcted here. 1

3 the followng data: the mappng of search phrases and ads that they ran on Yahoo and Google for three months n 006. For each search phrase we had the number of daly mpressons, number of clcks, number of orders, average dsplay poston, and cost per clck on both Yahoo! and Google. Summary statstcs appear n Table 1 and gve an ndcaton of the sze and complexty of the management problem faced by the advertser. Table 1: Summary Statstcs Varable Yahoo Google Impressons/Ad/Day Clcks/Ad/Day Clck Through Rate 7.80% 7.45% Orders/Ad/Day Order Rate 0.97% 0.89% Cost per Clck ($) Sales Rev per Clck ($) Sales Rev per Order ($) Average Poston of Ads Average # of words.76.6 Unque Ads Total Costs ($) Total Sales ($) Sales to Cost Rato The Emprcal Investgaton: In what follows, we dscuss the mpact of advertsement characterstcs on the clck through rate and the order rate. 3.1 Clcks: The clck-through-rate s defned as the rato of clcks to mpressons (number of searches where the ad was dsplayed) and s a standard measure used n ndustry parlance. Borrowng from ths construct, we model clcks usng a bnomal regresson condtoned on the number of mpressons. In other words, ( ) C ~ Bn p, I. t t t C t and I t are the number of clcks and mpressons (respectvely) of advertsement on day t. Note that the success probablty, p t, s the clck-through-rate and we model that usng a logstc transform, p t ( β X + ϕ( t ω) ) ( β X ϕ( t ω) ) exp pos ; = 1+ exp + pos ;. In the above equaton, X s a vector of advertsement specfc characterstcs and s the poston of advertsement on day t. Note that we allow poston to have a heterogeneous and nonlnear mpact by pos t

4 ϕ z; ω = ω z+ ω z. Ths heterogenety s crtcal because each ad may demonstrate a specfyng: ( ) 1 dfferent senstvty to ts poston. We assume that the ω s are dstrbuted normally (ndependently) wth ω1 ω and varances v 1, v Consequently, the lkelhood for the clcks data can be wrtten as means [, ] ( β X + ϕ( t ω) ) X ( ) C N T exp pos ; 1 L = = 1 t= 1 1+ exp( β + ϕ pos t; ω ) 1+ exp( β X + ϕ( pos t; ω) ) wth F ( ) ( v ) ( 1; 1, 1 ;, ) t It Ct df ( ω ) ω =Φ ω ω Φ ω ω v, Φ beng the normal dstrbuton functon. We use Monte- Carlo methods to approxmate the ntegral n the lkelhood and then maxmze (Smulated Maxmum Lkelhood) to obtan the parameters of nterest. Snce we have two data sets (Yahoo and Google) we repeat the analyss for each case. We dscuss each n turn. Yahoo: The ad characterstcs( X ) we used ncluded product category dummes and dummes for the number of words n the advertsed phrase. (Note that due to match type algorthms at the engne, the numbers of words n the user s search query may exceed the number of words n the advertsed phrase. For example, an advertser buyng clcks on the search phrase blue wdget who opts for one of the looser non-exact match algorthms could see ther ad dsplayed on searches for blue wdget, nexpensve blue wdget, and blue green wdgets.) Snce the category effects were ncluded smply for control purposes we wll not dscuss them here. The more nterestng results pertan to the effect of the number of words n the advertsed phrase and poston on the clck through rate. In the Yahoo data, the mpact of the number of words was monotoncally ncreasng. In other words more specfc ads dsplayed n more complex search envronments had a hgher probablty of beng clcked. Ths s not surprsng, snce, there s a typcally a strong matchng of user ntent when the number of words n the search phrase s large. In a sense the number of words serves a sa proxy for the specfcty of the user s search. The effect of poston of clcks was also n accordance wth ntuton. We found that [, ] [ 0.09,.013] ω ω = mplyng that lower postons (remember that 1 s a hgher poston than 1 ) have lower clck through rates but the effect s dmnshng. There was also sgnfcant heterogenety n ths lnear effect, evdenced by the estmates, [, ] [ 0.15, 0.001] v v =. Together, these results suggest that poston has a sgnfcant mpact on clcks but ths effect vares sgnfcantly over ads. 1 Google: The results for the Google clck through data were qualtatvely smlar. The category effects were all of the same sgn as n the Yahoo results, as were the effects of words and poston. Ths lends credblty to the 3

5 model and our analyss snce the data come from two very dfferent sources. Agan, as wth Yahoo, we found that ads dsplayed n the context of more complex searches tend to have hgher clck-through rates. However, the effect seemed to be stronger n the Google results. A smlar pattern was observed wth the poston effect. ω ω = 5 suggestng that the effect of poston on clcks was twce as large n We found that [, ] [ 0.173,.0 ] 1 Google as n Yahoo. In contrast, we found that, [, ] [ 0.079, ] v v =, mplyng that the level of 1 heterogenety n these effects was substantally lower than n Yahoo. Note that the drecton of the effects s the same n both data. What s dfferent n the average magntude of the effect and the spread across ads. We note that whle our coeffcents for CTR were decreasng wth poston, the frst few postons (between zero and three) are often rendered at the top of the page, whle the followng postons are rendered on the rght sde of the page. Thus, whle traffc and CTR are typcally monotoncally decreasng wth poston, t s mportant to recognze that some users on certan renderngs of search results pages fnd poston three more promnent than poston two. Ths non-monotoncty n dsplay locaton may explan some of the varance n our coeffcent estmates. Gven that Google vares the locaton of the varous ad postons more often than Yahoo, we fnd the lower level of heterogenety noteworthy. Ths could suggest that Google has a more precse understandng of the economc value of each page locaton than Yahoo, and allocates ad nventory accordngly. There could be a number of explanatons for our results. Frst, t s well known that the Google and Yahoo busness models are very dfferent. Ths s partcularly true when t comes to the defnton of the poston rank construct. Whle Yahoo sets the poston of the ads based solely on the bds of the advertsers, Google uses a more complex combnaton of bds and the number of clcks (.e. total revenue). Ths of-course calls nto queston the Google analyss and requres that we explore alternatve mechansms (possbly smultaneous equaton methods) to model the Google phenomenon. Ths s one object of our ongong research. Nevertheless, the smlartes (and dfferences) provde some new nsghts nto the nature of clck through rates and ts antecedents. 3. Orders: The model of Orders was very smlar to the earler model descrbed for clcks. In ths case Orders were modeled usng a Bnomal regresson condtoned on clcks. We wll skp detals for brevty. Yahoo: The effect of the number words on order exhbted some nterestng results. We found that searches wth a few words (one or two) and those wth many (fve) were more lkely to result n orders than searches wth ntermedate number of words (three or four). We conjecture that ths s a result of the nature of the match between the consumer s search and the ad. To elaborate, magne a consumer who searches for a product and uses fve words to descrbe what she needs. If a gven ad were to show up n that search context t 4

6 s lkely that ths ad s partcularly good match. In such a case, condtonal on clckng that ad, the consumer s also lkely to buy. Now magne the other extreme. The consumer uses a sngle word to descrbe the search and consequently the natural search results wll be poor matches. Now, under the scenaro that the consumer fnds a relevant ad and clcks on t, an order s lkely smply because the outsde alternatve s poor. The -3 word case represents a stuaton where the natural search results are stronger and the consumer envsons a hgher payoff from contnung the search and decdes not to buy. Whle, we beleve ths to be a plausble story further behavoral research s requred to valdate our conjecture. The Yahoo results also reveal that there s no sgnfcant mpact of poston on the order rate. Ths mples that once the effect of poston on clcks s accounted for there s no resdual mpact on the lkelhood of an order. We note that ths advertser ncluded SKU desgnatons n ther ad portfolo, such as ACME134. These hghly precse sngle word phrases may have sgnfcantly hgher converson than generc sngle word phrases, such as WIDGET. The effectveness of usng number of words as a proxy for the precson of an ad s therefore dependent upon the context. Google: The Google results from the order model are n stark contrast to the Yahoo results. Frst, there s no dscernable effect of the number of words. Second, there seems to be a poston effect. Contnung our dscusson from the Clcks results, t s dffcult for us to dsentangle the true results from spurous ones because of the nature of the Google poston construct. We hope to do so n our ongong research. 4. Lookng forward: The second and thrd largest search engnes, Yahoo and MSN, respectvely, have joned Google n rankng ads by a mx of maxmum bd, clck through rate, and subjectve factors, or soon wll do so. Ths makes poston-based bd approaches much more dffcult to control. Advertsers must rely on nstantaneous and average poston data to manage bds n ther attempt to hold a certan poston on the page. Ths study, conducted n the wanng months of the older Overture (now Yahoo) platform, offers a rare glmpse nto the tradeoff between poston and bd, and hopefully nforms the newer more complcated envronment. 5