A Quality-Based Auction for Search Ad Markets with Aggregators

Transcription

1 A Quality-Based Auction for Search Ad Markets with Aggregators Asela Gunawardana Microsoft Research One Microsoft Way Redmond, WA 98052, U.S.A. + (425) aselag@microsoft.com Christopher Meek Microsoft Research One Microsoft Way Redmond, WA 98052, U.S.A. + (425) meek@microsoft.com Jody Biggs Microsoft Corporation One Microsoft Way Redmond, WA 98052, U.S.A. + (425) jbiggs@microsoft.com ABSTRACT We explore the role of aggregators in search ad markets and their effects on that marketplace. Aggregators can have positive effects similar to those of an arbitrageur. We argue and provide empirical evidence that, unlike arbitrageurs, aggregators can continue to profit even in the absence of price imbalances. Furthermore, we argue that the standard generalized second price (GSP) auction mechanisms create incentives for aggregators to design their sites to negatively affect the user experience and that the existence of aggregators in the marketplace can also negatively affect nonaggregator merchants. We propose a specific quality-based GSP mechanism that reduces the negative impacts of aggregators on the user and marketplace while allowing some of their positive benefits for users. Categories and Subject Descriptors H.4.0 [Information Systems Applications]: General; J.4 [Computer Applications]: Social and Behavioral Sciences Economics; G.0 [Mathematics of Computing]: General General Terms Algorithms, Economics, Human Factors Keywords Search Advertising, Ad Aggregation, Arbitrage, Allocation, Pricing.. INTRODUCTION Online advertising is a massive industry Google alone reports revenue of over US$0 billion for the first three quarters of 2007 [4]. A large fraction of this revenue is due to pay-per-click search advertising and pay-per-click syndicated advertising. In search advertising, advertisers bid to have short text ads shown with search engine results and clicked on by search engine users. The ads that are displayed are chosen through a Generalized Second Price (GSP) auction [2,4,] which also determines the price each advertiser is charged when their ad is clicked. Advertisers are charged only when their ad is clicked, not when it is displayed. Per-click prices vary across different search engines due in part to differences in the auction mechanisms and differences between the set of advertisers and their bids on each engine. In syndicated advertising, the advertising platform syndicates ads, which then appear on third-party publisher websites rather than on the search results page. Google s popular adsense program is an example of such a contextual ad syndication service. These ads are also sold on a pay-per-click basis, but the payment is shared between the search engine and the third-party publisher. In this paper we consider the role of aggregators in search ad markets. The most common type of aggregation is ad aggregation which is the result of combining search advertising and syndicated advertising. Ad aggregators place syndicated ads on their web pages, and then attract traffic to these web pages by placing search ads. They are profitable when they pay less for incoming clicks on their search ads than they receive for outgoing clicks on the syndicated ads that they host. Another type of aggregation is shopping aggregation in which users can compare products. In general we define an aggregator as someone who aggregates content and referring links and makes money when people click on referring links. Because aggregation involves buying clicks in one market and selling them in another for a profit, it is often called click arbitrage. Unlike arbitrage in a typical commodities market, an aggregator purchase of one click on a search engine can lead to multiple referrals. This peculiarity is due to the fact that the advertiser is buying the search user s attention. One implication of this is that an aggregator s opportunity for profit does not necessarily disappear as prices converge. This stands in stark contrasts with arbitrage, which can only be profitable in the presence of a price imbalance. Unfortunately, while aggregators can provide some positive user benefits, aggregators have a clear incentive to design their pages to capture users attention and induce them to click on more referring links, irrespective of the effect that this has on the users utility. In contrast, merchants have a different incentive to sell users goods and services, which presumably provides users with positive utility. We argue that the standard GSP mechanism is inappropriate in this context, because it attempts to maximize social welfare of advertisers including aggregators, without respect to the effect on the user. Since aggregators value is determined by the clicks they get on referring links, the standard GSP mechanism reinforces their incentive to increase these clicks, even at the expense of user utility. In addition, we show that at equilibrium, aggregators that induce users to click on more than one referring link can always defeat merchants in the auction. This means that users can be faced with lower utility in the presence of aggregators the advertisers that win the auction will be aggregators who offer a poor user experience. This could also reduce the ability of the ad market to attract and retain quality merchants. We also provide evidence from a real ad market that aggregators do design their websites to increase referring clicks at the expense of user utility. We present empirical data from a large scale study that shows that many aggregators do obtain enough referring clicks to displace merchants at equilibrium. Thus, we argue that

2 aggregators need to be regulated so that merchants are able to compete. In fact, Google, the largest such ad market has already taken steps to limit the presence of aggregators in their sponsored results. We propose a simple modified GSP auction that uses a multiplicative quality factor. We note that Google and Yahoo purportedly use a quality factor in their auction mechanism but that the details of the mechanism are not published. Our proposed quality factor attempts to alleviate the problems introduced above by analyzing user behavior on the advertiser s site and then adjusting for their rate of referring clicks. This guarantees that merchants can compete with aggregators at equilibrium. In contrast to simply limiting or disallowing aggregators, this approach allows them to profit from arbitrage opportunities and drive price convergence, at which point the arbitrage opportunity disappears. Thus, the contributions of this paper are: ) characterizing the economics of ad aggregators and how they affect the market, 2) presenting large scale empirical data from a real ad market that real aggregators do achieve a high rate of referring clicks and that they are able to displace merchants, along with an illustration of how they sacrifice user utility to do so, and 3) proposing a modified GSP auction incorporating a multiplicative quality factor designed specifically to reduce the incentive for aggregators to harm user utility while still allowing aggregators to drive price convergence through exploiting arbitrage opportunities. We begin by analyzing the economics of aggregation in Section 2. We contrast aggregation to arbitrage, and show how their resulting price equilibria differ. In Section 3, we then analyze the impact of these differences on users as well as other advertisers. We show a real-life example which shows how an aggregator is able to displace merchants, and how that aggregator optimizes their website to increase their redirect rate at the expense of user utility. We also provide the results of a large-scale quantitative study of the ad clicks from about 2700 users, which provides evidence that aggregators do indeed achieve high redirection rates, and that they do dominate search ad auctions to the detriment of merchants. In Section 4, we then propose a modified GSP auction that uses a multiplicative quality factor in an attempt to alleviate these effects. In Section 5, we discuss some connections between this paper and other relevant work, and suggest directions for further exploration. Finally, we conclude in Section THE ECONOMICS OF AGGREGATION In this section, we describe the economics of ad aggregation, and how it differs from arbitrage. In particular, we examine the case of a standard GSP auction where the ranking function is a perimpression bid obtained by multiplying each advertiser s perclick bid by their click-through rate. We describe the price equilibria in the case of arbitrage, and discuss how arbitrage leads to price convergence. We then show how these equilibria change in the presence of aggregators who capture user attention and monetize it multiple times by causing users to click on multiple syndicated ads. We show that in this case, aggregators are profitable even when the same merchants bid their true value on both the search and syndicated ad markets, in contrast to arbitrageurs who would no longer be profitable. 2. Goods and Asset Arbitrage Arbitrage is usually defined as the practice of taking advantage of a difference in the price of a good or asset in two different markets by buying the good in the cheaper one and selling it in the more expensive one. Much of the literature deals with perfect arbitrage, where the sale and the purchase are accomplished simultaneously, realizing a profit with no commitment of capital, and no risk a free lunch [0]. It is somewhat of a folk theorem that arbitrage results in the prices in the two markets converging, resulting in the Law of One Price (LOP) [7]. The idea is that arbitrage leads to increased supply in the expensive market, and increased demand in the cheaper one, driving the prices to converge. In less idealized settings, the transaction may involve some costs (such as transportation), and risks (such as spoilage), so that a profit can be made only if the price difference is large enough. Thus, small price differences may persist, but large differences are quickly arbitraged away [8]. A key feature is that arbitrage opportunities are temporary arbitrage brings about price convergence, which eliminates the arbitrage opportunity, so that arbitrageurs effectively put themselves out of business. Arbitrage in asymmetric cases such as when one market has import barriers (such as high tariffs, regulations, etc) or when the other market has export barriers will be relevant in this paper. In these cases, one market can only be a source in cross-market transactions, while the other can only be a sink. Arbitrage can only operate in one direction from the source to the sink. The trade barrier removes the arbitrage opportunity when the price in the source market is higher than the price in the sink market so that the price difference can persist. However, if the price in the source market is lower than the price in the sink market, an arbitrage opportunity exists, and the difference will be arbitraged away. Thus, when an asymmetric barrier to trade exists, the full LOP cannot operate. Still, the temporary nature of arbitrage is preserved under this half LOP. 2.2 Search Ad Arbitrage In pay-per-click search ad markets, advertisers bid to have their ads displayed on the results page corresponding to particular queries. Clicks on the ads lead users to landing pages controlled by the advertisers, and result in a payment from the advertiser to the search engine. Thus, advertisers are often described as buying clicks, while search engines are described as selling clicks. The choice of ads to be displayed and the pricing of clicks are usually determined through a GSP auction. For each ad that is a candidate for being displayed for a particular query, its expected maximum cost per impression is computed by multiplying the corresponding bid by an estimate of the probability that the ad will be clicked. The ads are then sorted in order of decreasing expected maximum cost per impression, and the top ads displayed so that their relative prominence mirrors the sort order. When an ad is clicked, the advertiser is charged the minimum they would have had to bid to still retain the position in which the ad was displayed. Since each search engine that offers pay-per-click search advertising has its own independent auction, there are separate markets for clicks. The price charged for an ad click on a particular search query can vary between these markets, which could present an arbitrage opportunity. However, this can only be an arbitrage opportunity if a mechanism existed to transport clicks between these markets. We describe a hypothetical mechanism which would enable arbitrage between ad markets, which we term transparent syndication. This will be useful in understanding when ad aggregation is indeed arbitrage. In transparent syndication, a syndicating engine provides a syndication feed consisting of the ads and their current per-click prices. Arbitrageurs can then enter the ads into the auction on a publishing engine. The arbitrageur makes no change to the ad, except for directing clicks on the ad to themselves instead of the

3 original advertiser. The arbitrager then redirects these clicks to the syndicating engine, who in turn redirects them to the corresponding advertiser. The advertiser pays the syndicating engine the per-click price determined on the syndicating engine, and the syndicating engine passes this payment on to the arbitrageur. The arbitrageur pays the publishing engine the perclick price determined on the publishing engine. The mechanism is transparent to the users, in that they cannot distinguish between ads entered into the publishing engine by the arbitrageur from those that are entered into the engine by the original advertisers, and it is transparent to advertisers in that they cannot distinguish between clicks from users on the syndicating and publishing engines. When the per-click price of an ad is higher on one engine than on the other, an arbitrageur would be able to use the transparent syndication mechanism to buy clicks in the inexpensive market and sell them in the expensive one. Since this option is open to multiple arbitrageurs, and the price they can afford to pay per click is set by the per-click price on the expensive market, they would compete with each other, driving up the per-click price on the inexpensive market. Thus, transparent syndication would allow arbitrage, which in turn would cause price convergence. Any costs associated with this trade (including any fee retained by the syndicating engine) are analogous to transportation costs and risks in goods arbitrage. If the expensive market does not provide a transparent syndication feed, then arbitrage cannot take place, whereas the inexpensive market not providing a feed has no effect on arbitrage, just as in the case of goods arbitrage in the presence of a trade barrier. 2.3 Ad Aggregation While the transparent syndication mechanism described in the previous section does not exist, a different mechanism for syndication does. In real-world ad syndication a syndicating engine provides ads which third-party websites display. These websites and the syndicating engine share the revenue resulting from clicks on these ads. While the publishing engine doesn t directly publish the syndicated ads, the third-party websites can then advertise on the publishing engine, and pay the publishing engine for clicks on this ad. If a user arrives on a third-party website through this ad, and then clicks on syndicated ads on this website, this website earns revenue. Such third-party websites are referred to as ad aggregators. An ad aggregator is profitable if they earn more from clicks on syndicated ads on their site than they have to pay the publishing engine for incoming clicks. For this reason, ad aggregators are often thought of as engaging in arbitrage, and are often described as search engine arbitrageurs. Ad aggregation differs from the idealization of transparent syndication in a number of significant ways. An aggregator does not insert a separate ad into the publishing engine s auction corresponding to each syndicated ad. Instead, the aggregator submits a single ad to the publishing engine auction. When a user clicks on this ad, the aggregator is billed for the click, and the user is taken to the aggregator s landing page, which displays the syndicated ads. Thus, real-world ad syndication is not transparent to the user. When the user clicks on an ad on the aggregator s page, they are redirected through the syndicating engine to the advertiser landing page, and the syndicating engine charges the advertiser and pays the aggregator. Syndicating engines usually For simplicity, we do not specifically address keyword arbitrageurs in this paper, although similar considerations carry over to this case as well. inform the advertiser that the click was on a syndicated ad rather than a search ad, and often give their advertisers a discount (compared to the price charged for clicks on search ads). In addition, the syndicating engine only passes on a portion of this payment to the aggregator, retaining the balance as a syndication fee. Thus, the economics of aggregation are subtly different from the economics of ad arbitrage. We analyze these differences in the remainder of this section. 2.4 Aggregator Price Equilibria We will consider an asymmetric world with two search engines, S and P, where S syndicates its ads, but P does not. We choose to model this case because it is an idealization of the real world, where most aggregators carry ads from Google s highly successful adsense syndication program, but where Google itself has taken explicit steps to prevent aggregators from appearing in its sponsored results (we will provide evidence for the need for taking such steps). For simplicity of presentation, we will consider only a single search query, and will assume that the market for clicks is highly competitive, so that all advertisers on S have the same bid B S and therefore pay B S per click. We will also assume that all ads on P (including ads from aggregators) will have the same clickthrough-rate (CTR) if shown in the same position, so that perclick prices are only a function of the bids. The argument carries through when these assumptions are relaxed. Let α be a discount factor so that the syndicating engine S charges its advertiser αb S rather than B S per syndicated click, and let the syndication fee be α( β)b S so that an aggregator receives αβb S per syndicated click. The factor β is purely under the control of the syndicating engine S. Finally, let N be the number of ads displayed by each engine Ideal Ad Aggregation Suppose an ideal aggregator is one where a user clicking on the aggregator s ad on P will click on exactly one syndicated ad on the aggregator s page. Of course, this is an idealization real users may get distracted or frustrated and leave the aggregator s page before clicking on an ad, or click on multiple ads. However, it is instructive to see that an ideal aggregator could afford to pay up to αβb S per click on P (if we ignore overhead). We then have the following result: Theorem The equilibrium price of a click on the i th ad on P is no lower than αβb S as long as i < N and at least i+ ideal aggregators exist. Proof: We assume that the equilibrium price of a click on the i th slot on P is B P < αβb S to prove the theorem by contradiction. This means that the (i + ) st bid is B P. Thus, i + ideal aggregators could bid B A on P, where αβb S > B A > B P. Each aggregator would then win a slot and make a profit of at least αβb S B A > 0 on each click. The (i + ) st bid is now B A > B P, so that the price of a click on the i th slot is now B A > B P. This contradicts the assumption that that the equilibrium price of a click on the i th slot on P is B P. Thus, when α = and β =, ideal aggregation behaves like arbitrage. The discount rate α and the syndication fee factor β both introduce friction, and are analogous to transportation costs in goods arbitrage. In fact, ideal aggregation is entirely equivalent to goods arbitrage in the presence of an asymmetric trade barrier. If both search engines were to syndicate ads, and to allow ads

4 from aggregators, arbitrage in both directions would lead to a Law of One Price User Attention and Ad Aggregation We now examine how the equilibrium price on P is affected when aggregation is imperfect. That is we now remove the assumption that a user clicking on an aggregator s ad clicks on exactly one of the syndicated ads displayed by the aggregator. We use the term redirection rate to denote the average number of syndicated ads a user clicks on each time he or she arrives at the aggregator page from P. The redirection rate r depends on a number of factors, including whether or not the user is attracted to the aggregator page, whether or not they return to P, and whether or not they terminate their search or click on an ad. In short, it depends on what the user chooses to pay attention to. An aggregator that is more successful at capturing the users attention will induce users to remain on the aggregator s site (as opposed to returning to the publishing engine) or to keep returning to it, and perhaps to click on more syndicated ads. Such an aggregator can thereby attain a higher redirection rate. This is important because the aggregator earns αβb S per click on a syndicated ad, and therefore earns rαβb S per click on their ad on P. Following the same argument as above, we can now prove Theorem The equilibrium price of a click on the i th ad on P is no lower than rαβb S as long as i < N and at least i+ aggregators with redirect rates of at least r exist. Thus, in this case, the prices of the slots are differentiated by the aggregators redirection rates. Assuming their bids exceed the other advertisers bids (see below), aggregators can compete and earn profits based on their ability to capture users attention, which is reflected in their different redirection rates. For example, if the top two aggregators have redirection rates r > r 2, the top aggregator earns a profit of r r 2 αβb S. This profit is stable as long as the redirection rates are greater than one and differ from each other. This is in contrast to ideal aggregation and goods arbitrage, where arbitrageurs cause price convergence, which in turn causes the arbitrage opportunity to disappear The Role of the Syndicating Engine The syndicating engine S receives a profit of α β B S per syndicated ad click, which means it receives rα β B S per click on an aggregator s ad on P. Thus, the syndicating engine s profit is increasing in α and decreasing in β. However, S is constrained in how much it can increase α and decrease β. Increasing α reduces the incentive of advertisers to participate in the syndication program. However, this may be alleviated by varying the discount factor α depending on the source of the syndicated traffic, so that α for traffic from a particular aggregator can be discounted depending on its value to the advertisers. Decreasing β is even more constrained. If we denote the minimum bid necessary to have an ad displayed on P by B P, S must ensure that β rα B P Otherwise, the aggregator will be unable to bid high enough to be displayed, and neither the aggregator nor the syndicating engine will profit. It is evident that within these limits, the syndicating engine can vary α and β for each aggregator and each advertiser to optimize its profits. It is also possible to vary α and β competitively, to impact the publishing engine s profits and the incentives of the advertisers to participate in the publishing engine s auction. For example, since the most an aggregator can afford to bid on the publishing engine is rαβb S, and since the syndicating engine can set β on a per-aggregator basis, the syndicating engine may, for strategic reasons, hold β low enough to ensure that rαβb S is lower than B S, delaying or perhaps even preventing price convergence. On the other hand, doing so increases the incentive for merchants to advertise on the publishing engine directly. While we note these interesting possibilities in passing, we do not study these in detail in this paper. 3. THE IMPACT OF AD AGGREGATORS In light of the differences between arbitrage and the economics of aggregators described above, we now examine how ad aggregators affect the search ad market. Even though they are not arbitrageurs, they bring some of the same benefits that arbitrageurs do they allow users on a search engine and advertisers on a different syndication engine to reach each other, and drive price convergence. In addition, they could have other benefits. They could allow small advertisers to advertise on multiple markets without incurring the fixed costs of managing a separate campaign on each of them. A special class of aggregators that are termed shopping aggregators provide users with the value added service of providing price comparisons between the different merchants whose ads they aggregate. It is interesting to note that nearly all shopping aggregators are also ad aggregators. However, the differences highlighted in the previous section also lead to some negative impacts on merchants and users. We examine these below. 3. Impact on Merchants We now examine the implications of the above results on online merchants (i.e. online advertisers that earn revenue by selling to the consumer, rather than through aggregation). Suppose a merchant that advertises on S also desires to advertise on P. Such a merchant would already have some click volume from users of P through aggregators that advertise on P and display the merchant s aggregated ad, although the merchant may not be aware of this. However, the volume of clicks received through aggregators would usually be lower than the volume that would be received by winning a top slot on P. B S

5 If the merchant values a click on P at αb S (what they are currently paying for this traffic), they are only able to outbid aggregators with r < β. If they are willing to pay up to B S, they are able to outbid aggregators with r < αβ. Since the price of the top slots are set by aggregators with higher redirection rates, the merchant would only be able to win lower slots in the auction, if they are able to win a slot at all. As a result the merchant would get a lower level of traffic than they would have got in the absence of aggregators. Thus, aggregators have an adverse impact on online merchants, and provide them with a disincentive to advertise on P. It is interesting to note that if all the aggregators were ideal aggregators (i.e., if the situation were analogous to arbitrage), the merchants could compete with the aggregators and the aggregators would have no profit, since the aggregators would have to pay αb S or B S respectively per click, and would receive only αβb S per click. Alternatively, merchants could decide to display syndicated ads themselves, causing their value of a click to increase to at most α + rαβ B S or + rαβ B S respectively. This is an upper bound because it assumes that syndicated ads have no effect on sales. In reality, merchants would have an incentive to trade off their sales against revenue from aggregation, and move into the spectrum between pure merchant and pure aggregator. The cost to the total user experience (discussed below) is not factored into this trade-off. Thus, there is an incentive for merchants to become aggregators themselves, and thereby compete better. Moreover, merchants will be unable to compete with aggregators without doing so. 3.2 Impact on Users Aggregators always have some negative impact on users of the publishing engine since users require two clicks rather than one in order to get to a merchant. In the best case users get access from the publishing engine to merchants that advertise on only the syndicating engine in return for this increased effort. However, the effects we describe above make the net impact of aggregators on users much worse than in this best case. First, aggregators with high redirection rates reduce the ability of merchants to win higher slots on the publishing engine directly, or even to advertise on the publishing engine at all. Thus, as fewer and fewer merchants advertise on the publishing engine, or as the existing merchants on the publishing engine win lower and lower slots in the auction, users are forced to go through aggregators more and more often. Thus, the existence of aggregator ads on the publishing engine s auction causes users to have fewer and fewer options for reaching merchant ads other than using an aggregator. Second, as described above, since aggregators profits depend on their redirection rate, they need to attract and retain users attention if a user reaching an aggregator s site returns to the search results page without clicking on an ad, the aggregator does not recover their payment for the ad click that brought the user to the page. If they only use the aggregator s site to click on a single ad, the aggregator can only benefit from a temporary arbitrage opportunity. Thus, it is in the aggregator s interest to influence the user to not return to the search results page, but rather to return to the aggregator s page and click on other ads. Aggregators typically accomplish this through a variety of methods that result in a poor user experience. These include: Popups clicking on a syndicated ad on the aggregator page causes the corresponding advertiser s landing page to open in a popup window on top of the aggregator Figure. The results page on Live Search for the query red quilts. Notice that aggregators occupy the top circled slots while merchants occupy lower circled slots. page. Closing this window causes the user to see the syndicated ads again, perhaps leading to another click. Redirect traps the aggregators landing page silently redirects to another aggregator page, usually via an HTML Meta Refresh tag. The timing of the redirect makes it difficult for the user to use the browser s back button to return to the search results page. Home page traps scripts on the aggregator s page change the user s home page to the aggregator, without the user s permission. This makes it more likely that the aggregator will get more syndicated clicks from the user at a later session. Deceptive UI. E.g.: o Making the aggregator page look like the results page of a search engine. o Making the aggregator site look like a merchant site. Thus, while aggregators could in theory provide users with the service of making syndicated ads available through the publishing engine, the economic incentives cause the aggregators to use techniques that arguably damage the user experience. Note that these incentives are a result of the economic advantage conferred by having a higher redirection rate. Absent this, aggregators would not have an incentive to damage the user experience as described above. 3.3 A Qualitative Example In this section, we describe a real-life example of ad aggregation that illustrates how a real aggregator is able to displace merchants in the sponsored search results of a search engine, and how this aggregator optimizers their landing page to increase redirect rate. We examine the results of the query red quilts on Live Search, shown in Figure. Note that the top three slots are occupied by aggregators who defeat the two merchants shown. Clicking on one of these aggregator ads leads to a page where syndicated ads

6 Figure 2. The landing page reached by clicking one of the aggregator ads shown in Figure. Note that a syndicated ad for a merchant that appeared in a lower slot as circled in Figure appears prominently on the aggregator s landing page (circled here). from two merchants are prominently displayed (see Figure 2). Following the associated landing page URLs reveal that both advertisements have been syndicated by Google. In fact, entering the query red quilts on a separate Google search confirms that these two merchants are the top advertisers on Google for this query. Further examination of the results page on Live Search reveals that one of the two merchants appears there as well, but that numerous aggregators have pushed the merchant to the last slot on the page. Thus, while that merchant can win a top slot on Google, an aggregator is able to outbid them on Live Search, even though the merchant is one of that aggregator s top two syndicated advertisers for this keyword. While it is possible that merchants are bidding more on the syndicating engine (Google) than the publishing engine (Live Search), or that the aggregators achieve higher click-through probabilities, this suggests that the aggregator achieves a high enough redirection rate to outbid the merchant. The user study described below confirmed that this particular aggregator had a redirection rate of., which is high enough to defeat the merchant even if they bid the same amount on Google and Live Search, and had as high a click-through probability as the aggregator. Because the aggregators have displaced the users on the Live Search results page, a user who would have reached the merchant in one click in the absence of aggregators now needs two clicks. In fact, clicking on a syndicated ad on the aggregator page shown in Figure 2 causes the ad landing page to open in a popup browser window, which arguable further damages user utility and increases the probability of further clicks on syndicated ads, as discussed in the previous section. Note that it could be argued that sites like the aggregator of Figure 2 provide the user with additional utility from price comparisons. However data collected for our user study below showed that Figure 3. The results page on the aggregator of Figure 2 when the user navigates to the site using a URL and enters the query red quilts. Note that in contrast to Figure 2 the prominent syndicated ads do not appear. when users enter the aggregator site by clicking ads from Live Search, over 90% of the out going clicks were in fact on syndicated ad clicks rather than on price comparisons. In fact, users who enter the aggregator site directly (instead of through an ad on Live Search) and search for red quilts are not presented with the prominent syndicated ads (see Figure 3). Thus, it would seem that the aggregator optimizes the prominence of syndicated ads and price comparisons based on their estimate of the likelihood of a user clicking on a syndicated ad or a price comparison and the resulting expected revenue, at the expense of user utility. 3.4 Empirical Study We performed a large scale empirical study to determine how often situations such as the one described above occur in practice, and whether aggregators are able to achieve significant redirection rates. This was done by using data recorded by Windows Live Toolbar, which is a browser add-in which records aspects of users web browsing history with their permission. We collected search queries along with the resulting ad clicks and clicks on syndicated ads on the advertisers sites during a fixed time interval, from the large population of toolbar users. In this paper, we report on user behavior observed in approximately 4,500 search ad clicks and the resulting aggregated syndicated ad clicks from a representative subset of about 2,700 toolbar users. Figure 4 shows the redirection rates of the ten advertisers with the highest number of clicks in our study. Notice that all ten advertisers have redirection rates higher than zero. Most of the advertisers on the list are aggregators, and many achieve redirection rates higher than unity. That is, they will be able to win the auction even if price convergence between Live Search and the syndicating engines (usually Google) takes place, assuming the syndicating engines attempt to maximize revenue as discussed above. Visiting the ad landing pages shows that these

7 Redirect Rate Number of Merchants 3 2 Agg Merch Advertiser Rank Figure 4. The redirect rates of the ten advertisers with the highest click volume. Aggregators and merchants are differentiated. Note that even the merchants have a positive redirect rate Advertiser Rank Figure 5. The cumulative number of pure merchants (i.e. merchants without syndicated ads) against advertiser rank (in terms of click volume). Only 2 of the top 50 advertisers are pure merchants. high redirection rates are achieved through deceptive UIs, popups, and redirection and homepage traps. In fact, even the three merchants that appear in the top ten advertisers display syndicated ads and earn revenue from them, in line with the incentives we describe in the previous section. We will refer to such merchants as aggregating merchants. Examining aggregating merchant web sites shows that they typically aggregate syndicated ads to back fill their inventory. In cases where these merchants have inventory that matches the user s query, they tend not to display syndicated ads, or at least make them less prominent. In cases where they do not have inventory that matches the user s query, these merchants aggregate, prominently displaying syndicated ads. In these cases users clicking on a reputable merchant s ads are presented with syndicated ads rather than items available for purchase from that merchant, arguably violating the users expectations. Figure 5 shows that few advertisers with high click volume refrain from using syndicated ads to enhance revenue only 2 advertisers in the top 50 advertisers by click volume do so. The empirical data demonstrates that the problems with aggregators in search ad markets discussed above are not only abstract possibilities. Aggregators do attain redirection rates higher than unity and thus can continue to be profitable after prices increase and can displace pure merchants at equilibrium. In addition, even the top merchants engage in aggregation, in line with their incentives. Without intervention, pure merchants are unable to compete effectively with aggregators. 4. A QUALITY-BASED AUCTION We propose a modified GSP auction for allocating and pricing search ads in order to guard against the negative impact that aggregators can otherwise have. This modified GSP auction incorporates a multiplicative quality factor Q in ranking and pricing. Note that in contrast to what Varian [] terms a quality factor, the factor we propose is a property of the advertiser s site, as opposed to a factor which accounts for a particular ad or advertiser s click-through probability. Just as in the usual GSP auction, our modified auction proceeds in two steps. First, ads are sorted in decreasing order of the product of their quality factor, click-through probability, and bid, and the top ads are assigned the corresponding ad slots. Second, the i th selected ads in the sorted list is assigned the price P i = Q i+ C i+ Q i C i B i+ where Q j, C j, and B j denote the quality factor, click-through probability, and bid of the j th item in the sorted list. The clickthrough probabilities above are typically adjusted to be position independent, by factorizing and discounting the effect of position [9]. Note that this mechanism differs from the usual GSP auction only through the presence of the quality factors, which can be thought of as adjustments applied to the click-through probabilities. Thus, just as in the usual GSP mechanism, the price paid per click by an advertiser is upper bounded by their bid, and it is possible for the advertiser to be charged their full bid (i.e. the bound is tight). It is evident that varying the choice of the quality factors Q yields a family of generalizations of the usual GSP auction. We determine conditions on the quality factors Q such that the following principles are satisfied: ) The publishing engine should ensure that pure merchants (i.e. advertisers with no revenue from syndicated ads) can compete fairly without being penalized for not aggregating. 2) The auction should allow aggregation to drive price convergence. Note from the description of the modified auction above that the quality scores of all advertisers can be scaled by an arbitrary positive constant with no effect on the auction. Thus, if we assume that all advertisers that do not display syndicated ads have the same quality score, we can assume that this score is unity without any further loss of generality. We make this assumption in the rest of the paper. 4. Fair Competition between Merchants and Aggregators To ensure fair competition, we will first insist that a merchant that is willing to pay B S per click (the price on the syndicating engine) should be able to outbid an aggregator on the publishing engine. That is, aggregators should not compete unfairly with merchant ads. Recall that the aggregator can afford to pay at most rαβb S per click. We will assume that the aggregator bids this true value bidding more than this could cause the aggregator to lose money. We then want to ensure that the merchant can win the

8 auction as long as the merchant s click-through probability is at least as good as the aggregator s. Thus, we need as long as C M B S Q A C A rαβb S C M C A where we have used the subscript M to denote the merchant and the subscript A to denote the aggregator, and we have assumed that Q M =, as mentioned above. This implies Q A rαβ 4.2 Fair Competition between Merchants While we have analyzed competition between merchants and aggregators, this setting is somewhat idealized. As we saw above, in reality, aggregating merchants also display syndicated ads in order to generate supplemental revenue it is possible to add syndicated ads to a merchant site so that any resulting loss in sales revenue is more than offset by revenue from the syndicated ads displayed. Thus, it is desirable that a non-aggregating (pure) merchant who does not show syndicated ads can compete fairly with similar aggregating merchants. Suppose the non-aggregating merchant has a value per click of B S. The aggregating merchant has a value of at most B S from sales, and a further value of rαβb S from aggregated syndicated ads. We will assume that this bound is tight. That is, we assume that it is possible for a merchant to achieve some positive redirection rate without negatively impacting sales. Such an aggregating merchant can bid up to + rαβ B S. For the non-aggregating merchant to compete, we need which requires that B S Q + rαβ B S Q + rαβ Note that ensuring that non-aggregating merchants can compete fairly with aggregating merchants also ensures that they can compete fairly with pure aggregators. Since α and β are both determined by the syndicating engine, and since it is possible for the engine to adjust these factors to manipulate the auction on the publishing engine, the conservative choice is to use Q + r If the maximum discount factor α that the syndicating engine can use is fixed by the advertisers value of syndicated clicks to be less than one, this choice can be relaxed to Q + rα 4.3 Driving Price Convergence Subject to the fair competition constraints above, it is desirable to allow aggregators to drive price convergence, as they would if they were ideal aggregators (arbitrageurs) acting in an ad market with an unmodified GSP auction. Suppose the highest pure merchant bid B P on the publishing engine P is less than the price per click B S on the syndicating engine S. Either the same merchants are bidding B P < B S on P while bidding B S on S, or merchants willing to pay B S > B P are not participating in the auction on P. Assuming the value of clicks are the same on P and S, it is desirable for aggregators to have an incentive to bid B P > B P, since this either forces merchants to bid their true value (to avoid being displaced by aggregators bidding B P ), or it provides the users of P and the merchants on S access to each other. In short, it is desirable for aggregators to have the incentive to perform the positive functions of ideal aggregators or arbitrageurs. Thus, aggregators can profitably drive price convergence as long as their revenue per click rαβb S exceeds their cost per click B P Q, if the aggregator and merchant had the same click-through probability. Thus, aggregators can profitably drive price convergence as long as Q > B P rαβ B S A similar argument shows that aggregating merchants can only drive price convergence as long as B Q > P + rαβ 4.4 The Tension between Fair Competition and Driving Price Convergence Recall that guaranteeing fair competition between merchants with and without aggregation requires B S Q + rαβ This means that if fair competition is guaranteed, aggregators can only drive price convergence when B P < rαβ + rαβ B S Once B P reaches rαβ B +rαβ S, aggregators with redirection rate r are no longer profitable. Driving B P all the way to B S would require a pure aggregator with infinite redirection rate, even if α = and β =. However, by analyzing the aggregation opportunities of a merchant that can achieve a small redirection rate by carefully displaying aggregated ads in a manner that has no negative influence on sales, we can prove the following: Theorem Suppose that the publishing engine P uses an advertiser quality score of Q = +rα. Then, assuming that advertisers on S value syndicated clicks at αb S, (a) Fair competition is guaranteed irrespective of the values of α and β, (b) Suppose that the maximal pure merchant bid on P is B P, that this bid represents a merchant s value per click from sales, and that such a merchant could display syndicated ads to achieve some small redirection rate with no adverse effect on sales. Such an aggregating merchant can defeat a pure merchant in the auction as long as B P B S < α α. (c) Any other choice of Q that guarantees (a) will not allow syndicated ads to be profitable for a larger range of B P B S. Proof: Result (a) results directly from the bound

9 Q > + rαβ for fair competition between merchants with and without aggregation, and the assumptions α α (by definition of α ) and β < ( S demands positive revenue from syndication). To show result (b): A merchant competing with the maximal pure merchant bid could display syndicated ads, earning B P + rαβb S and would need to bid + rα B P to defeat them. This is profitable if rαβb S rα B P > 0. The syndicating engine gains revenue of rα β B S from this traffic at no additional cost if > β > α B P. Since B P < α, α B S B S α such a value of β exists, and the aggregating merchant can safely bid + rα B P and defeat the pure merchant. To show result (c): We note that an aggregating merchant competing with a pure merchant with maximal bid could earn at most B P + rαβb S and would need to bid B P to defeat Q the pure merchant. This is profitable only if rαβb S > Q B P. The highest value of B P for which this can occur B S is rαβ Since β < by assumption, and Q. +rα for Q fairness, we have that displaying syndicated ads is only profitable when B P < α B S α if the modified GSP auction allows fair competition between aggregating and non-aggregating merchants. Thus, our modified GSP auction uses Q = +rα. Note that this reduces to the usual GSP when r = 0 for all advertisers. Under this rule, pure aggregators and aggregating merchants will only participate in the auction until the per-click price increased. In particular, pure aggregators would only be profitable until the price reached +rα B S, and aggregating merchants could only profit from aggregating until the price +rα β reached +rα B S. When the price reaches this point, the syndicating engine ceases to earn revenue through the traffic on the publishing engine. Therefore, the syndicating engine has an incentive to increase α to α and β as close to as profitable in order to retain some share of this revenue for as long as possible. Even then, merchants would have no incentive to aggregate after the price reached B S, and at this point, the syndicating engine would lose the associated revenue. In practice, although the exact redirect rate r for an advertiser is not available to the publishing engine, it is possible to use data sources such as toolbar logs or proxy logs from third party data providers (e.g. comscore or Nielsen//NetRatings) to estimate the redirect rate for at least the advertisers with many clicks, as we have done above. 5. RELATED AND FUTURE WORK The equilibria of GSP ad auctions and the resulting search engine revenue have received considerable attention in the literature [2,4,]. The allocation and pricing rule we advocate in this paper is an instance of a GSP ad auction utilizing an advertiser specific multiplicative quality score which has been discussed in this literature. In particular, results regarding equilibria of such auctions apply here. rαβ However, these analyses address only advertiser value, and do not address the externalities imposed by advertisers on each other and on users, or the utility derived by users. The more recent work of Abrams and Schwarz [] shows how externalities imposed by the advertisers can be taken into account in a modified GSP auction, while that of Engel and Chickering [5] shows how publisher and user utilities can be incorporated into the auction. However, it is not obvious how to explicitly quantify these externalities or utilities in practice. Our proposed quality factor avoids the need for such an explicit quantification by simply preventing aggregators from having the incentive to impose a particular type of externality or disutility on the user after price convergence takes place. Interestingly, when viewed in the framework of Engel and Chickering [5], our choice of quality factor implies a particular quantification of utility, and the results shown there then apply. In particular, their results on Symmetric Nash Equilibria (SNE) apply to our proposed solution. The related work of Athey and Ellison [3] models user attention, user surplus, and the resulting externalities of advertisers on each other. They study a model where users incur a cost every time they click on an ad, gain a constant utility when their need is met, and decide to stop browsing further ads when the cost of another click exceeds the expected benefit of continued browsing. This model illustrates some of the excess negative externality imposed on other advertisers by advertisers that have high click-through probability and high bid, but low probability of meeting the user s need such an advertiser would reduce the user s expectation of the utility of continued browsing. The paper then analyses a novel mechanism that takes into account the advertisers probabilities of meeting a user s need, and describes how such a mechanism maximizes user surplus. However, while the paper describes how search-diverting sites can lead to merchants dropping out of the publishing engine s auction, the model described therein assumes counterfactually that all advertisers derive an expected per-click payoff given by the advertiser s probability of meeting the user s click times a constant. In other words, an advertiser s payoff is contingent on meeting a user s need, and meeting a user s need results in the same payoff for all advertisers. In contrast, the idea of a redirection rate that is a property of each advertiser is central to our work. A user s attention has an inherent value to advertisers, and so the value of a click to an advertiser may have little to do with whether a user s need was met. Thus, we explicitly allow for the possibility that advertisers can achieve a per-click payoff that is many times that of a pure merchant even if they do not meet the user s need, and explicitly correct for this through our choice of quality factor. While the redirection rate is the only measure of an advertiser s excess per-click payoff we have used, there are other measures that could be considered. For example, how often an advertiser sets or reads a cookie on the user s browser, whether the advertiser installs browser plugins or other software on the user s machine, whether the user returns immediately to the search results page, etc. are measures related to how much utility the advertiser and the user gain from the click. Such measures can identify asymmetric exchanges where the advertiser gains a payoff from a user s click while the user suffers a cost without gaining any utility. A more sophisticated approach would be to make use of such measures in the auction. In contrast to our proposed mechanism, such a nuanced approach may also be able to distinguish the so called shopping aggregators that provide some utility to users in the form of price comparisons and to the aggregators that may provide very little utility to users.

10 Although this paper has only described aggregation of pay -perclick ads across search engines, the effects described can arise in more general settings. Similar arguments can be made for pay - per-impression pricing. Even the use of pay-per-action pricing does not alleviate the difficulty, since the syndicating engine may still use a pay-per-click model, so that aggregators are still able to command a higher expected value per impression that any single pure merchant. The effects described here can also be extended to aggregators who round-trip, using the same engine as both the syndicating and publishing engines. Similarly, they can also be extended to aggregators who engage in keyword arbitrage. These and similar extensions are left for future work. In Section we mentioned the possibility that through manipulating the syndication fees and discounts, the syndicating engine may have some control over the extent to which aggregators drive price convergence on the publishing engine. The long-term incentives of the publishing and syndicating engines in the light of this possibility also make up a fascinating area of study which has been left for future work. Such considerations would probably be part of a more complete view of fair pricing than the one we give here. It is easy to imagine many other considerations that should also be part of formulating a truly fair pricing rule. Finally, although it is easy to estimate redirect rates for dominant advertisers through means such as toolbars or third party data providers as discussed above, estimating redirect rates for tail advertisers is an interesting problem in its own right. 6. CONCLUSION We have described how the phenomenon of ad aggregation differs from arbitrage, and how this difference affects the economics of search advertising. In particular, aggregators can extract more value per click than any non-aggregating merchant if they induce users to click on multiple syndicated ads. Such aggregators can defeat non-aggregating merchants in ad auctions, and dominate the market. The standard GSP auction reinforces their incentive to increase clicks on syndicated ads even at the expense of the users utility. We showed empirical evidence that aggregators actually do extract more value from clicks, and that they do dominate ad auctions. In extreme cases, this effect could lead to no ads from merchants being displayed, which is arguably suboptimal for both users and merchants. We also discussed how aggregators may allow one engine to manipulate prices on another. As a first step in counteracting the negative effects aggregators may have on search ad markets, we presented a modified GSP auction that uses an advertiser specific multiplicative quality score based on how often users at a given advertiser s click on syndicated ads. Since all major search engines have access to user behavior data, this solution is easily implemented. The proposed solution allows merchants who do not aggregate to compete with aggregators. At the same time, the proposed rule allows aggregators to take advantage of pricing imbalances, facilitating price convergence. 7. REFERENCES [] Abrams, Z., & Schwarz, M. Ad Auction Design and User Experience. WINE (2007), [2] Aggarwal, G., Goel, A., and Motwani, R. Truthful auctions for pricing search keywords, in ACM Conf. on Electronic Commerce (2006), -7. [3] Athey, S. and Ellison., G. Position auctions with consumer search (2007), mimeo. [4] Edelman, B., Ostrovsky, M., and Schwartz, M. Internet advertising and the generalized second price auction: Selling billions of dollars worth of keywords. American Economic Review 97, (2007), [5] Engel, Y., & Chickering, D. M. Incorporating User Utility Into Sponsored-Search Auctions. Proc. 7th Int. Conf. on Autonomous Agents and Multiagent Systems (2008), to appear. [6] Google, Inc. Google Announces Third Quarter 2007 Results. Mountain View, CA, [7] Isard, P. How far can we push the "law of one price?" American Economic Review 67, 5 (977), [8] O'Connell, P. G., and Wei, S.-J. "The bigger they are, the harder they fall": Retail price differences across U.S. cities. Journal of International Economics 56 (2002), [9] Richardson, M., Dominskowa, E., and Ragno, R. Predicting clicks: Estimating the click-through rate for new ads. WWW (2007), [0] Varian, H. R. The arbitrage principle in financial economics. Economic Perspectives, 2 (987), [] Varian, H. R. Position auctions. International Journal of Industrial Organization, to appear.