Comparng erformance Metrcs n Organc Search wth Sponsored Search Advertsng Anndya Ghose Stern School of Busness ew York Unversty ew York, Y-1001 aghose@stern.nyu.edu Sha Yang Stern School of Busness ew York Unversty ew York, Y-1001 syang0@stern.nyu.edu ABSTRACT Wth the rapd growth of search advertsng, there has been an ncreased nterest amongst both practtoners and academcs n enhancng our understandng of how consumers respond to contextual and sponsored search advertsng on the Internet. An emergng stream of work has begun to explore these ssues. In ths paper, we focus on a prevously unexplored queston: How does sponsored search advertsng compare to organc lstngs wth respect to predctng converson rates, order values and profts from a keyword ad? We use a Herarchcal Bayesan modelng framework and estmate the model usng Markov Chan Monte Carlo (MCMC methods. Our analyss suggests that on an average whle the converson rates, order values and profts from pad search advertsements were much hgher than those from natural search, most of the keyword-level characterstcs have a statstcally sgnfcant and stronger mpact on these three performance metrcs for organc search than pad search. Ths could shed lght on understandng what the most attractve keywords are from advertsers perspectve, and how advertsers should nvest n search engne advertsng campagns relatve to search engne optmzaton. Categores and Subject Descrptors J.4 [Socal and Behavoral Scences]: Economcs General Terms erformance, Measurement, Economcs. Keywords Organc Search, Herarchcal Bayesan modelng, ad search advertsng, Electronc commerce, Internet Economcs ermsson to make dgtal or hard copes of all or part of ths work for personal or classroom use s granted wthout fee provded that copes are not made or dstrbuted for proft or commercal advantage and that copes bear ths notce and the full ctaton on the frst page. To copy otherwse, or republsh, to post on servers or to redstrbute to lsts, requres pror specfc permsson and/or a fee. ADKDD 08, August 4, 008, Las Vegas, evada, USA. Copyrght 008 ACM. 1. ITRODUCTIO The advertsng world has changed dramatcally n the past decade. In the pre-internet era, frms reled heavly upon tradtonal meda advertsng lke televson, magaznes, drect mal, and even rado. But today, marketers have embraced the Internet wth search engne marketng, socal meda networks, nteractve webstes, etc. In fact, n the past year alone, onlne advertsng expendtures grew 6% to total $1.4 bllon. Though there are many nnovatve ways frms can advertse onlne, the bulk of onlne advertsng conssts of two man forms: dsplay ad (banner advertsng and pad search advertsng (sponsored ads that appear on the search results pages of search engnes. Snce consumers perceve dsplay ads as annoyng and obtrusve, they represent a small proporton of onlne advertsng. Conversely, pad search advertsng represents 40% of onlne advertsng expendtures, and has grown % n the past year alone. What has fueled ths growth? Sponsored search has gradually evolved to satsfy consumers penchant for relevant search results and advertsers' desre for nvtng hgh qualty traffc to ther webstes. These keyword advertsements are based on customers own queres and are thus consdered far less ntrusve than onlne banner advertsements or pop-ups. In many ways, one could magne that ths enabled a shft n advertsng from mass advertsng to more targeted advertsng. By allottng a specfc value to each keyword, an advertser only pays the assgned prce for the people who clck on ther lstng to vst ts webste. Because lstngs appear when a keyword s searched for, an advertser can reach a more targeted audence on a much lower budget. Hence, t s now consdered to be among the most effectve marketng vehcles avalable n the onlne world.. As companes are showng more wllngness to advertse on the nternet, a recent survey conducted by McKnsey ndcates that marketng executves stll worry over the lack of metrcs. 1 In the past, marketers sought to ncrease the number of page vews for ther webste. ow, these executves want more concrete metrcs whch relate more drectly to proftablty. Currently, search engnes offer the most measurable form of advertsng metrcs; they can provde estmates on clck-through rates and average bd prces for every possble keyword. Whle managers recognze the mportance of pad advertsng, many companes 1 Green, H. Stumblng Blocks for Onlne Advertsng. BusnessWeekOnlne, September 007.
have also begun nvestng heavly n search engne optmzaton (SEO to mprove ther organc search results, ether n addton or n leu of search engne marketng (SEM. SEO refers to the process of talorng a web ste to optmze ts unpad (or organc rankng for a gven set of keywords or phrases. SEM refers to nvestments n pad (or sponsored rankngs. In 007, search engne optmzaton accounted for 18% of all search engne marketng expendtures and s expected to grow as SEO s generally less expensve than pad search. A survey conducted by the emarketer revealed that 46% of onlne retalers found that SEO performed best, compared to 7% of retalers who preferred pad-per-clck advertsng. evertheless, a queston that nterests many frms s whch keywords wll gve them the best return-on-nvestment (ROI. For pad search, managers seek to fnd keywords that wll result n hgh clck-through rates and more mportantly, hgher converson rates. In organc search, ths s the same, but snce search engne optmzaton depends on keyword type, frms marketng dollars could also be talored to focus on searches wth a hgh rate of converson. Despte the growth of search advertsng, our understandng of how consumers respond to sponsored search advertsng on the Internet s stll nascent. In ths paper, we focus on a prevously unexplored queston: How does the content of a keyword mpact sponsored search versus natural search lstngs wth respect to predctng converson rates, order values and profts? Whle an emergng stream of lterature n sponsored search has looked at ssues such as the mpact of keyword attrbutes on sponsored search and spllover effects from keyword campagns, no pror work has emprcally analyzed ths queston. Gven the shft n advertsng from tradtonal banner advertsng to search engne advertsng, an understandng of the determnants of converson rates and clck-through rates n search advertsng can be useful for both tradtonal and Internet retalers. Ths s partcularly true for companes tryng to decde between makng nvestments n SEO versus nvestments n SEM. There s a growng debate on whch of these two search mechansms s more effectve. On the one hand, because an advertser can control the message of a pad search, one would expect hgher conversons. On the other sde, because people value the edtoral ntegrty of organc searches, one would expect hgher conversons from them. Snce frms are now tryng to grapple wth the trade-offs n each of these two forms of referrals, emprcal research based on actual data from an advertser can shed some lght on these ssues. Usng a unque panel dataset of several hundred keywords collected from a large natonwde retaler that advertses on Google, we study the effect of sponsored search advertsng at a keyword level on consumer search, clck and purchase behavor n electronc markets. We propose a Herarchcal Bayesan modelng framework n whch we model consumers behavor jontly wth the advertser s decson. We emprcally estmate the mpact of keyword attrbutes (such as the presence of retaler nformaton, brand nformaton and the length of the keyword on consumer purchase propenstes. Ths classfcaton s motvated by pror work on the goals for users web search such as [5, 19]. We fnd that whle the mean converson rate, mean order value and mean proft from pad search advertsements was much hgher than that from a correspondng set of natural search lstngs avalable durng the same tme perod, the varous keyword level covarates have a stronger mpact on natural search than on pad search. In partcular, the presence of retaler-specfc nformaton ncreases the Converson rate, the Value and the roft n both forms of search advertsng pad and natural. In contrast, whle the presence of a brand name ncreases Converson rates, Value and roft n natural search, t does not affect any of these performance metrcs n pad search. Fnally, the length of a keyword negatvely mpacts the performance on all three metrcs for natural search lstngs but only affects the Value n pad search.. DATA.1 Data Descrpton We frst descrbe the data generaton process for pad keyword advertsement snce t dffers on many dmensons from tradtonal offlne advertsement. In sponsored search, advertsers who wsh to market ther product or servces on the Internet submt ther webste nformaton n the form of keyword lstngs to search engnes. A keyword may consst of one or more words. Bd values are assgned to each ndvdual keyword to determne the placement of each lstng among search results when a user performs a search. Bascally, search engnes pt advertsers aganst each other n aucton-style bddng for the hghest ad placement postons on search result pages. Once the advertser gets a rank allotted for ts keyword ad, these sponsored ads are dsplayed on the top left, and rght of the computer screen n response to a query that a consumer types on the search engne. The ad typcally conssts of headlne, a word or a lmted number of words descrbng the product or servce, and a hyperlnk that refers the consumer to the advertser s webste after a clck. Ths sponsored ad shows up next to the organc search results that would otherwse be returned usng a separate crtera employed by the search engne. The servng of the ad n response to a query for a certan keyword s an mpresson. If the consumer clcks on the ad, he s led to the landng page of the advertser s webste. Ths s recorded as a clck, and advertsers usually pay on a per clck bass. In the event that the consumer ends up purchasng a product from the advertser, ths s recorded as a converson. The data used n ths study s smlar to that used n ([11]. It contans weekly nformaton on pad search advertsng from a large natonwde retal chan, whch advertses on Google. The data span all keyword advertsements by the company durng a perod of three months n the frst quarter of 007, specfcally for the 1 calendar weeks from January 1 to March 1. Unlke most datasets used to nvestgate on-lne envronments whch usually comprse of browsng behavor only, our data are unque n that we have ndvdual level stmulus (advertsng and response (purchase ncdence. Hallerman, D. Search Engne Marketng: User and Spendng Trends. emarketer. January 008. Value refers to the prce of the product that was sold durng the transacton.
Each keyword n our data has a unque advertsement ID. The data conssts of the number of mpressons, number of clcks, the average cost per clck (CC, the rank of the keyword, the number of conversons, the total revenues from a clck (revenues from converson and the average order value for a gven keyword for a gven week. Whle a search can lead to an mpresson, and often to a clck, t may not lead to an actual purchase (defned as a converson. The product of CC and number of clcks gves the total costs to the frm for sponsorng a partcular advertsement. Thus the dfference n total revenues and total costs gves the total profts accrung to the retaler from advertsng a gven keyword n a gven week. Our data s aggregated at a weekly level. Smlar to the data on pad search results, our dataset has nformaton that conssts of conversons, order value and total revenues accrung from natural searches for the same retaler durng the same tme perod. We compare the set of keyword advertsements across the 1-week perod that appears n both the pad and natural lstngs. There are 776 unque keyword lstngs n the dataset gven to us by the advertser. However, not all keywords are sponsored by ths advertser n all the weeks n our sample. Smlarly, there are certan weeks where the advertser s lnk dd not show up n the natural lstngs of Google n response to the user-generated query. Hence, we have a dfferent number of observatons for clcks and conversons from pad ads n comparson to the number of observatons for clcks and conversons from the natural lstngs for the same product sold by the advertser. Our mappng yelded a total of 065 observatons from the pad searches, and a total of 18 observatons from the natural searches. Table 1 reports the summary statstcs. Interestngly, we note that the mean converson rate was 5.4% and.76% from pad and natural searches, respectvely. Smlarly, the mean order value and proft from pad search advertsements was much hgher than that from natural search lstngs. There are three mportant keyword specfc characterstcs for a frm (the advertser when t advertses on a search engne ([11]. Ths ncludes whether the keyword should have ( frmspecfc nformaton, ( brand-specfc nformaton, ( and the length (n words of the keyword. A consumer seekng to purchase a dgtal camera s as lkely to search for a popular brand name such as IKO, CAO or KODAK on a search engne as searchng for the generc phrase dgtal camera on the same search engne. Smlarly, the same consumer may search for a retaler such as BEST BUY or CIRCUIT CITY on the search engne. In recognton of these electronc marketplace realtes, search engnes do not merely sell generc dentfers such as dgtal cameras as keywords, but also wellknown brand names that can be purchased by any thrd-party advertser n order to attract consumers to ts Web ste. The length of the keyword s also an mportant determnant of search and purchase behavor but anecdotal evdence on ths vares across trade press reports. Some studes have shown that the percentage of searchers who use a combnaton of keywords s 1.6 tmes the percentage of those who use sngle-keyword queres [19]. In contrast, n 005 Oneupweb conducted a study to determne f the number of keywords n a search query was related to converson rates. They focused ther study on data generated by natural or organc search engne results lstngs and found that sngle-keywords have on average the hghest number of unque vstors. To nvestgate the mpact of the length of a keyword, we constructed a varable that ndcates the number of words n a keyword that a user quered for on the search engne (and n response to whch the pad advertsement was dsplayed to the user. We enhanced the dataset by ntroducng some keyword-specfc characterstcs such as Brand, Retaler and Length. For each keyword, we constructed two dummy varables, based on whether they were ( branded or unbranded keywords and ( retaler-specfc or non-retaler specfc keywords. To be precse, for creatng the varable n ( we looked for the presence of a brand name (ether a product-specfc or a company specfc n the keyword, and labeled the dummy as 1 or 0, wth 1 ndcatng the presence of a brand name. For (, we looked for the presence of the advertsng retaler s name n the keyword, and then labeled the dummy as 1 or 0, wth 1 ndcatng the presence of the retaler s name. There were no keywords that contaned both retaler name and brand name nformaton. Ths enabled a clean classfcaton. Table 1: Summary Statstcs of the ad and atural Matched Data (_ad=065; _atural=18 Varable Mean Std. Dev. Mn Max ad_retaler 0.11 0.7 0 1 ad_brand 0.599 0.49 0 1 ad_length.4 0.81 1 5 ad_impressons 919.91 4. 1 9744 ad_clcks 79.1 818.15 0 0 ad_converson 0.054 0.1 0 1 Rate Log(ad_ 1.176 1.945 0 7.1 Value Log(ad_Revenue 1.7. 0 10.7 Log(ad_roft 0.667.77-4.9 10.71 atural_retaler 0.94 0.49 0 1 atural_brand 0.60 0.465 0 1 atural_length.16 1.0 1 5 atural_clcks 51.58 776.04 1 608 Log(atural_ 0.7 1.. 0 0.675 Value atural_converson 0.076 0.15 0 1 Rate Log(atural_roft 0.4 1.48 0 10.5
. EMIRICAL MODEL: COMARIG ERFORMACE METRICS I AID AD ORGAIC SEARCH An mportant determnant of the effectveness of sponsored search advertsng s the extent to whch the same keyword also appears n the natural or organc lstngs of the search engne. Organc rankngs are determned by the content of the webste and the webste's relatve mportance. In organc search there s no guarantee as to specfc rankng postons or the tmng for rankngs to appear/change. In order to mprove rankngs a frm almost always requres changes to webste content and/or structure. [17] conducted a survey wth 45 respondents, wheren more than 77% of partcpants favored non-sponsored lnks more than the sponsored lnks, as offerng sources of trusted, unbased nformaton. Based on a survey of 1,649 Web users, [15] found that 60% of Google users reported non-sponsored results to be more relevant than sponsored. Ths was even hgher for predomnantly Google users (70%. [1] nvestgated the relevance of sponsored and non-sponsored lnks for e-commerce queres on the major search engnes, and found that average relevance ratngs for sponsored and non-sponsored lnks are practcally the same, although the relevance ratngs for sponsored lnks are statstcally hgher. These studes then beget the queston that f natural searches lead to more purchases than sponsored ads, then to what extent should frms nvest n sponsored search advertsements. Revenues from Sponsored Search 0 500 1,000 1,500,000 Weeks 1 4 5 6 7 8 9 10 11 1 1 Fgure 1a: Dstrbuton of Revenues from ad Search Across Weeks Revenues from atural Search 0 50 100 150 00 Weeks 1 4 5 6 7 8 9 10 11 1 1 Fgure 1b: Dstrbuton of Revenues from atural Search Across Weeks Fgures 1a and 1b show that there are consderable dfferences n the revenues accrung from pad and natural search over tme. In ths secton, we ntend to compare the mpact of the three keyword covarates on the performance of pad vs. natural searches. More specfcally, we compare the mpact of the three covarates on converson rates, order value and proft accrung from pad (sponsored search to those from natural (organc searches. The study of these three metrcs enables us to get better nsghts nto the factors that drve product sales and proftablty for retalers n the search engne advertsng ndustry..1 Modelng Converson We cast our model n a Herarchcal Bayesan (HB framework and estmate t usng Markov chan Monte Carlo methods (see [8] for a detaled revew of such models. In HB models, probablty dstrbutons are used to quantfy pror belefs about the parameters whch are updated wth the nformaton from the data to yeld a posteror dstrbuton. The HB model s called "herarchcal" because t has two levels. At the hgher level, we assume that ndvduals parameters (betas are descrbed by a multvarate normal dstrbuton. Such a dstrbuton s characterzed by a vector of means and a matrx of covarances. At the lower level we assume that, gven an ndvdual s betas, hs/her probabltes of achevng some outcome (choosng products, or ratng brands n a certan way s governed by a partcular model, such as multnomal logt or lnear regresson [8]. Recent advances n computatonal technques such as MCMC methods have proven to be very useful n estmatng such models. Rather than dervng the analytc form of the posteror dstrbuton, MCMC methods substtute a set of repettve calculatons that, n effect, smulate draws from ths dstrbuton. These Monte Carlo draws are then used to calculate statstcs of nterest such as parameter estmates and confdence ntervals. The dea behnd the MCMC engne that drves the HB revoluton s to set up a Markov chan that generates draws from posteror dstrbuton of the model parameters [8]. An advantage of estmatng herarchcal Bayes (HB models wth Markov chan Monte Carlo (MCMC methods s that t yelds estmates of all model parameters, ncludng estmates of model parameters assocated wth specfc respondents (whch n our case translates nto keywords. We use the Metropols-Hastngs algorthm wth a random walk chan to generate such draws ([6]. We postulate that the decson of whether to clck and purchase n a gven week wll be affected by the probablty of advertsng exposure (for example, through the rank of the keyword and ndvdual dfferences (both observed and unobserved. Among the n clck-throughs of pad searches, there are m clck-throughs that lead to purchases for keywords at week j. Let us further assume that the probablty of makng a purchase s q. Then, the lkelhood of the number of purchases s specfed as: n m f m =! ( q (1 q m!( n m! n m (1.1
ote that the superscrpt stands for pad searches, and the superscrpt stands for natural searches. Smlarly, for natural searches, the lkelhood of the number of purchases s specfed as: We assume there s a latent spendng ntenton ( z, of a consumer that determnes how much to spend for keyword at order j through a pad advertsement. Hence, we have n m n m f m =! ( q (1 q m!( n m! (1. y z = f z, > 0 (1.8 ext we derve the converson probabltes n pad and organc searches. Dfferent keywords are assocated wth dfferent products. Snce product-specfc characterstcs can nfluence consumer converson rates, t s mportant to control for unobserved product characterstcs that may nfluence converson rates once the consumer s on the webste of the advertser. Hence, we nclude the three keyword characterstcs to proxy for the unobserved keyword heterogenety stemmng from the dfferent products sold by the advertser. Ths leads us to model the converson probabltes as follows:, y = 0 f Smlarly, for natural searches, we have,, y z = f, y = 0 f z, 0 (1.9 z, > 0 (1.10 z, 0 (1.11 q exp( α 1Retaler Brand Length + ε = 1+ exp( α Retaler Brand Length + ε 1 (1. We model the latent buyng ntenton of consumers from pad advertsements and natural lstngs, respectvely, as follows: q exp( γ + δ1retaler + δ Brand + δ Length + η = 1+ exp( γ + δ Retaler + δ Brand + δ Length + η 1 (1.4 z β = α Length + ε 1 Retaler Brand + (1.1 To complete the specfcaton, we have ε ~ (0, C (1.5 η ~ (0,,C (1.6 C C θ = ( α, γ ' ~ MV ( θ, Σ (1.7 When specfyng the dstrbuton of the ntercept and the error terms, we use to denote a normal dstrbuton and MV to denote a multvarate normal dstrbuton.. Modelng Value ote that the order value (the prce of the product s always postve. Ths mples that the data wll be left censored. In censored data, t s well known that the use of smple OLS regressons leads to nconsstent estmates []. Hence, we use a Tobt specfcaton to model the order values. 4 4 The Tobt model s an econometrc method that descrbes the relatonshp between a non-negatve dependent varable y and an ndependent varable (or vector x. The model supposes that there s a latent (.e. unobservable varable y. Ths varable lnearly depends on x va a parameter (vector β whch determnes the relatonshp between the ndependent varable (or vector x and the latent varable y (just as n a lnear model. In addton, there s a normally dstrbuted error term u to capture random nfluences on ths relatonshp. The observable varable y s defned to be equal to the latent varable whenever the latent varable s above zero and zero otherwse. If the relatonshp parameter β s estmated by regressng the observed y on x, the resultng ordnary z, β, = α, Length + ε, 1, Retaler, To complete the model specfcaton, we have Brand + (1.1, ε ~ (0, (1.14, ε ~ (0,, (1.15, ( α, α ' ~ MV( α, Σ (1.16. Modelng roft ote that roft can have both negatve and postve values because the total revenues from an advertsement may be less than the total costs ncurred for dsplayng that pad advertsement. Hence, we can use an ordnary least squares (OLS regresson to model the pad proft. We model the proft of the pad searches n the form of the followng regressons: least squares estmator s nconsstent. [] has proven that the lkelhood estmator for ths model s consstent.
y roft roft roft roftr = α 1 Retaler Brand + (1.17 β roft Length + ε roft ote that the proft n natural searches s always postve. Ths s because there are no drect advertsng costs nvolved for the retaler for sellng through natural lstngs, and hence profts are smply equal to revenues n ths case. Ths mples that the data on profts from natural searches wll be left censored. Hence, we use a Tobt specfcaton to model the proft of the natural searches as follows:,r oft,r oft y z = f,r oft y = 0 f z z, r oft > 0 (1.18, r oft 0 (1.19 Table a: Coeffcent Estmates for redctng Converson 5 Intercept Retaler Brand Length C Σ ad -5.14.465 0.8-0.074 4.01 1.45 (0.01 (0.59 (0.4 (0.105 (0.14 (0.1 atural -7.70 0.56 0.488-0.19 11.14 0.57 (0.57 (0.179 (0.1 (0.09 (1.86 (0.8 Table b: Coeffcent Estmates for redctng Value As before, we model the latent buyng ntenton from natural lstngs as follows: Intercept Retaler Brand Length Σ z,roft,roftr,roft,roft = α 1 taler Brand + (1.0 β,roft Length + ε,roft Re To complete the model specfcaton, we have the followng: r oft ε ~ (0, r oft (1.1 ad -.997.80 0.416-0.59 15. 1.079 (0.78 (0.6 (0.47 (0.7 (1.1 (1.91 atural -1.08.681 1.775-1.154 48.16 11.60 (0.97 (0.69 (0.6 (0.9 (.7 (.14,r oft ε ~ (0,,r oft (1., roft, roft roft roft ( α, α '~ MV( α, Σ (1..4 Results We now examne the effect of keyword covarates at the mean level (see Tables a, b and c. The overall pattern of the results ndcates that the presence of retaler name, brand name and the length of the keyword are assocated wth the decson to purchase, the amount of purchase and the frm s overall proft n any gven week. Specfcally, the presence of retaler-specfc nformaton s assocated wth an ncrease n the Converson rate, the Value and the roft n both forms of search advertsng pad and natural. In contrast, whle the presence of a brand name s assocated wth an ncrease n Converson rates, Value and roft n natural search, t mpact on any of these performance metrcs n pad search s not statstcally sgnfcant. Fnally, the length of a keyword s negatvely assocated wth an ncrease n the performance on all three metrcs for natural search lstngs. In the case of pad search, the mpact of the keyword length has a statstcally sgnfcant and negatve mpact only on the Value. Thus, we see that longer keywords generally tend to have a detrmental affect on keyword performance such as converson rates and profts. Table c: Coeffcent Estmates for redctng roft Intercept Retaler Brand Length roft Σ ad 0.505.144 0.07-0.177 5.19 1.559 (0.76 (0.8 (0.167 (0.101 (0.176 (0.167 atural -15.5 4.5.04-1.44 66.049 15.059 (1.1 (0.799 (0.77 (0.9 (.675 (.81 How do these estmates translate nto actual percentage changes? In ad search, the presence of retaler nformaton n the keyword ncreases converson rates by 11 %, an ncrease n length of the keyword by one word decreases order value by 7.7 % whle the presence of retaler nformaton n the keyword ncreases proft by 5. %. In atural search, the presence of 5 osteror means and posteror standard devatons (n the parenthess are reported, and estmates that are sgnfcant at 95% are bolded n Tables a -c.
retaler nformaton n the keyword ncreases converson rates by 9.74 %, the presence of brand nformaton n the keyword ncreases converson rates by 4.9 %, and an ncrease n length of the keyword by 1 word decreases converson rate by 5.41 %. In atural search, the presence of retaler nformaton n the keyword ncreases order value by 67.61 %, the presence of brand nformaton n the keyword ncreases order value by 45.19 % and an ncrease n length of the keyword by 1 word decreases order value by 0.01%. In atural search, the presence of retaler nformaton n the keyword ncreases proft by 68. %, the presence of brand nformaton n the keyword ncreases proft by 44.6 % and an ncrease n length of the keyword by 1 word decreases proft by 10.71 %. These results are summarzed n Table below. Table : Summary of ercentage Effects of Keyword Covarates Based on Estmates from Tables a-c. 6 Retaler Brand Length ad_converson Rate 11% A A ad_ Value 70.4% A -7.7% ad_roft 5% A A atural_converson Rate 9.74% 4.9% -5.45% atural_ Value 67.61% 45.19% -0.01% atural_roft 68.% 44.6% -10.71% To analyze whether the dfferences n the mpact of dfferent covarates on the performance metrcs between ad and atural searches were statstcally sgnfcant, we conducted parwse t-tests. The analyses reveals that the presence of retaler nformaton s assocated wth a bgger mpact on pad search advertsements than natural search lstngs n predctng converson rates. However, we cannot say anythng conclusvely about ether the dfferental mpact of retaler nformaton or the mpact of keyword length n predctng average order values between pad and natural searches. 4. RELATED WORK Our paper s related to several streams of research. Frst, t contrbutes to recent research n onlne advertsng n economcs and marketng by provdng the frst known emprcal analyss of sponsored search keyword advertsng. Much of the exstng academc (e.g., [7] on advertsng n onlne world has focused on measurng changes n brand awareness, brand atttudes, and purchase ntentons as a functon of exposure. Ths s usually done va feld surveys or laboratory experments usng ndvdual (or cooke level data. In contrast to other studes whch measure (ndvdual exposure to advertsng va 6 ercentage effects for statstcally nsgnfcant estmates n Tables a- c are not computed and lsted as A. aggregate advertsng dollars ([18], we use data on ndvdual search keyword advertsng exposure. [4] looks at onlne banner advertsng. Because banner ads have been perceved by many consumers as beng annoyng, tradtonally they have had a negatve connotaton assocated wth t. Moreover, t was argued that snce there s consderably evdence that only a small proporton of vsts translate nto fnal purchase ([7], clck-through rates may be too mprecse for measurng the effectveness of banners served to the mass market. Interestngly however, [4] found that banner advertsng actually ncreases purchasng behavor, n contrast to conventonal wsdom. A large lterature n economcs sees advertsng as necessary to sgnal some form of qualty ([16], [6]. There s also an emergng theoretcal stream of lterature exemplfed by ([] [8], [], and [] that examnes aucton prce and mechansm desgn n sponsored keyword auctons. Despte the emergng theory work, very lttle emprcal work exsts n onlne search advertsng that looks at conversons and profts. Ths s prmarly because of dffculty for researchers to obtan such advertser-level data. [5, 19] classfes queres as nformatonal, navgatonal, and transactonal based on the expected type of content destnaton desred and analyze clck through patterns of each. They fnd that about 80% of Web queres are nformatonal n nature, approxmately 10% each beng transactonal, and navgatonal. [0, 1] nvestgate the relevance of sponsored and non-sponsored lnks for e-commerce queres on the major search engnes. Other emprcal work has so far focused on search engne performance ([4], [1]. Moreover, the handful of studes that exst n search engne marketng have typcally analyzed publcly avalable data from search engnes. [1] look at the presence of qualty uncertanty and adverse selecton n pad search advertsng on search engnes. [14] examne the factors that drve varaton n prces for advertsng legal servces on Google. [0] studed the converson rates of hotel marketng keywords to analyze the proftablty of dfferent campagn management strateges. Our prevous work ([11], [1] has analyzed the mpact of dfferent keyword covarates on sponsored search, and estmated the cross-sellng potental from a keyword. In a related paper, [1] we estmate the nter-dependence between natural search lstngs and pad search advertsements and vce-versa, and conduct polcy smulatons to nvestgate f these two processes have a complementary or substtutve effect on each other s clckthrough rates. However, none of these studes compared the performance of sponsored search wth natural search by examnng the mpact of keyword content on performance metrcs lke converson rates, order value and proft. To summarze, our research s dstnct from extant onlne advertsng research as t has largely been lmted to the nfluence of banner advertsements on atttudes and behavor, and to studyng the performance of sponsored search advertsements. The doman of natural search lstngs has largely been gnored. We contrbute to the lterature by emprcally comparng varous performance metrcs n sponsored search wth natural search lstngs by estmatng the mpact of dfferent keyword characterstcs on pad and natural search lstngs.
5. COCLUSIOS AD FUTURE WORK In most search-based advertsng servces, a company sets a daly budget, selects a set of keywords, determnes a bd prce for each keyword, and desgnates an ad assocated wth each selected keyword. If the company s spendng has exceeded ts daly budget, however, ts ads wll not be dsplayed. Wth mllons of avalable keywords and a hghly uncertan clckthrough rate assocated wth the ad for each keyword, dentfyng the most proftable set of keywords gven the daly budget constrant becomes challengng for companes wshng to promote ther goods and servces va search-based advertsng ([9]. In ths regard, our analyss reveals that whle retalerspecfc nformaton s more mportant than brand nformaton n predctng converson rates n both pad and organc search. Ths result can have useful mplcatons for a frm s Internet pad search advertsng strategy. Our results can have mplcatons on the ssues related to search engne optmzaton (SEO vs. search engne marketng (SEM n partcular because many advertsers engage n both knds of actvty. Our analyss suggests that most of the keyword-level characterstcs have a stronger mpact on the performance of natural search than pad search. Ths could shed lght on understandng what the most attractve keywords from a frm s perspectve are, and how t should nvest n search engne advertsng campagns relatve to search engne optmzaton. We are cognzant of the lmtatons of our paper. These lmtatons arse prmarly from the lack of nformaton n our data. For example, we do not have precse data on competton snce our data s lmted to one frm and one ndustry. That s, we do not know the keyword ranks or other performance metrcs of keyword advertsements of the compettors of the frm whose data we have used n ths paper. Future research can use data on competton and hghlght some more nsghts on how frms should manage a pad search campagn by runnng more detaled polcy smulatons that ncorporate compettve bd prces. Further, we do not have any knowledge of the other marketng varables such as any promotons durng consumers search and purchase vsts. We also do not have data on the textual content n the copy of the ad and detaled content n the landng pages correspondng to the dfferent keywords, although some evdence suggests that the presence of the keyword n the ttle of the ad s more mportant than that n the ad copy n nfluencng clck-through rates ([5]. Future researchers can conduct varous sorts of experments to examne how the content of the ad copy nteracts wth keyword attrbutes to determne both consumer and frm behavor. We hope that ths study wll generate further nterest n explorng ths mportant emergng area n web search. REFERECES [1] Anmesh A., V. Ramachandran, S. Vswanathan. Qualty uncertanty and adverse selecton n sponsored search markets, Workng aper, Unversty of Maryland, College ark, 006. [] Amemya, T. Regresson analyss when the dependent varable s truncated normal. Econometrca 41 (6, 997 1016, 197. [] Aggarwal, G., J. Feldman, and S. Muthukrshnan. Bddng to the Top: VCG and Equlbra of oston-based Aucton, arxv: cs.gt/0607117 v1, 006. [4] Bradlow, E.T., and Schmttlen, D.C. The lttle engnes that could: Modelng the performance of world wde web search engnes, Marketng Scence, 19(1, 4-6, 000. [5] Broder, A. Taxonomy of web search, SIGIR Forum, vol. 6, pp. -10, 00. [6] Chb, S., and Greenberg. E. Understandng the Metropols- Hastngs algorthm. The Amercan Statstcan, 49, 1995, 7-5. [7] Cho, C., Lee, J., and Tharp, M. Dfferent forced-exposure levels to banner advertsements, Journal of Advertsng Research, 41(4, 001, 45-56. [8] Edelman, B., Ostrovsky, M., and Schwarz, M. Internet advertsng and the generalzed second-prce aucton: sellng bllons of dollars worth of keywords. Amercan Economc Revew, 97(1, 007, 4-59. [9] Feng, J, Bhargava, H., and ennock, D. Implementng sponsored search n web search engnes: Computatonal evaluaton of alternatve mechansms. Informs Journal on Computng, 19(1, 007, 17-148. [10] Fnkelsten, L., Gabrlovch, E., Matas, Y., Rvln, E., Solan, Z., Wolfman G., and Ruppn, E. lacng search n context: the concept revsted. roceedngs of WWW 001. [11] Ghose, A., and Yang, S. An Emprcal Analyss of Sponsored Search erformance n Search Engne Advertsng. roceedngs of the ACM WSDM 008. [1] Ghose, A., and Yang, S. Analyzng Search Engne Advertsng: Frm Behavor and Cross-Sellng n Electronc Markets. roceedngs of WWW 008. [1] Ghose, A., and Yang, S. Impact of Organc Lstngs on ad Search Advertsng: Complements, Substtutes or ether? Workng aper, ew York Unversty. [14] Goldfarb, A., and C. Tucker. 007. What makes search engne advertsng valuable? Understandng search term varaton n legal servces, Workng aper, SSR. [15] Greenspan, R. Searchng for balance. vol. 004: ClckZ stats. [16] Grossman, G., and Shapro, C. Informatve advertsng wth dfferentated products. Revew of Economc Studes. 51(1, 1984, 6-81. [17] Hotchkss, G., Garrson, M., and Jensen, S. Search engne usage n orth Amerca. Enquro, 005. [18] Ilfeld, J., and Wner, R. Generatng webste traffc. Journal of Advertsng Research, 4, 00, 49-61. [19] rospect Inc. Search engne user atttudes, 005. [0] Jansen, B., and Spnk, A. The effect on clck-through of combnng sponsored and non-sponsored search engne results n a sngle lstng, roceedngs of the 007
Workshop on Sponsored Search Auctons, WWW Conference, 007. [1] Jansen, B. J. The comparatve effectveness of sponsored and non-sponsored results for web ecommerce queres. ACM Transactons on the Web. 1(1,, 007. [] Katona, Z., and M. Sarvary. The race for sponsored lnks: A model of competton for search advertsng, Workng aper, ISEAD, 007. [] Klpatrck, D. Keyword advertsng on a cost-per-clck model,www.stepronews.com/archves/00/feb/14prt.html [4] Manchanda,., Dubé, J., Goh, K. and Chntagunta,. The effect of banner advertsng on nternet purchasng. Journal of Marketng Research, 4(1, 006, 98-108. [5] MarketngExperments.com. C ad copy tested. Marketng Experments Journal, 005. [6] Mlgrom,., and Roberts, J. rce and advertsng sgnals of product qualty, Journal of oltcal Economy, 94, 1986, 796-81. [7] Moe, W., and Fader,. Dynamc converson behavor at e- commerce stes. Management Scence, 50(, 00, 6-5. [8] Ross,. E., and Allenby, G. Bayesan statstcs and marketng. Marketng Scence,, 00, 04-9. [9] Rusmevchentong,., and Wllamson, D. An adaptve algorthm for selectng proftable keywords for searchbased advertsng servces, Workng aper, 006. [0] Rutz, O., and R. Buckln. A model of ndvdual keyword performance n pad search advertsng, Workng aper, 007. [1] Telang, R.,. Boatwrght, and T. Mukhopadhyay. A mxture model for Internet search engne vsts, Journal of Marketng Research, Vol. XLI (May, 06-14, 004. [] Varan, H. oston auctons. Internatonal Journal of Industral Organzaton, 5 (6 116-1178, 006. [] Wooldrdge, J. Econometrc analyss of cross secton and panel data. Cambrdge, MA: MIT ress, 00