Algorithms and optimization for search engine marketing

Transcription

1 Algorithms and optimization for search engine marketing Using portfolio optimization to achieve optimal performance of a search campaign and better forecast ROI Contents 1: The portfolio approach 3: Why did the rulesbased method fail? 3: The SEM portfolio 3: The efficient frontier 4: Measuring lift with the EF 5: The need for algorithms 5: Models 8: Automation 8: Models, optimization, and automation: The synergy for performance at Adobe 9: The power of f orecasting 10: Conclusion 10: About the Adobe Digital Marketing Suite 10: About Adobe Systems Incorporated Given the complexity and volatility in the marketplace, marketers need a proven method to manage their search campaigns. The portfolio theory of bid optimization lies in building good keyword models that capture behavior from both the cost side and the revenue side. Backed by good models and bid automation, portfolio optimization increases search campaign performance. These models and optimization also give marketers the power to forecast performance for different business constraints and goals. The synergy of models, automation, and optimization enable advertisers to run campaigns predictably without compromising performance. Volatile markets are a difficult environment for online advertisers. If advertising spend is cut, sales volume is lost. If spend is increased, the company exposes itself to significant risk on the margin. Achieving success and daily business goals in a volatile market can be accomplished with the right balance of the complex interplay of science and technology. This guide describes the mathematics and technology behind Adobe s methods that ensure success for search advertisers during both good and bad economic times. The portfolio approach The core of Adobe s method of solving the search engine marketing (SEM) challenges lies in its portfolio approach to bid management. Let s look at a common business problem. An investor has $50,000 to build three plants. For each plant, the company can invest at four different levels. The following table shows the estimates for each project and the expected revenues. Project 1 Project 2 Project 3 Expenditure Level Expense Revenue ROI Expense Revenue ROI Expense Revenue ROI % 0 0 0% 0 0 0% 1 $30,000 $23,000 77% $25,000 $35, % $26,000 $19,000 73% 2 $25,000 $22,500 90% $15,000 $30, % $20,000 $16,000 80% 3 $20,000 $20, % $10,000 $25, % $15,000 $14,000 93% 4 $8,000 $15, % $6,000 $18, % $10,000 $12, %

2 The investor needs to use the money to maximize revenue. The solution is not readily apparent because there are 125 different ways (5 3 ). Of these, the following are choices that are within the $50,000 budget. Expense Level Project 1 Project 2 Project 3 Total Investment Total Revenue ROI $50,000 39,000 78% ,000 60, % ,000 57, % ,000 64, % ,000 62, % ,000 64, % ,000 61, % In these seven choices, the fourth is the best because it produces the highest revenue. Note that a simple rule of thumb such as pick the investment outlay with ROI over 100% would not have worked because the ROI of the individual projects for the optimal choice are varied (90%, 200%, and 120%, respectively). This methodology of considering all possible choices across the range of possibilities is called the portfolio-based approach. This example is analogous to type of problem that search marketers face. Instead of projects or businesses, marketers need to choose search keywords, and the outlay levels are analogous to keyword positions. When a keyword is bid to a position, we spend on paid clicks the number of clicks times the cost per click (CPC) and we expect revenue from some of the clicks converting on the website. The goal is to decide the right bids for each keyword to maximize the revenue from the set of all available keywords. The choice of bids is not obvious and often counterintuitive. This time, let s look at a marketing example. A search marketer is advertising two DVDs for sale. The search engine offers only two positions for bidding. The marketer has developed accurate estimates of CPC, clicks, and orders on a given day. DVD A Position CPC Clicks Cost Orders $/Order 1 $ $ $ $ DVD B Position CPC Clicks Cost Orders $/Order 1 $ $ $ $ The goal of the advertiser is to get the maximum number of orders possible subject to the constraint that the overall cost per order does not exceed $20. A simple rule of thumb approach (also called a rule-based method) is to bid each keyword to a position that is closest to but not exceeding the targeted cost per acquisition (CPA). The results of this approach are shown in Table 1. Table 1. Choices of the rules-based approach to attain a CPO of $20. Each keyword is bid to the position where the CPO is closest to but lower than the target CPO. A total of 25 orders are expected across both keywords. Keyword Position Cost Orders CPO DVD A 1 $ $20 DVD B 2 $ $20 Total $ $20 2

3 An advertiser using this approach would be satisfied that the CPO has been met. However, consider the exact opposite choice, as shown in Table 2. The total number of orders is higher than the rules-based method, despite the fact that the CPO for keyword DVD B has been exceeded. Table 2. The optimal solution obtained after considering all the possible choices across both keywords (portfolio-based approach). Keyword Position Cost Orders Cost/Order (CPO) DVD A 2 $ $10 DVD B 1 $ $25 Total $ $20 Why did the rules-based method fail? The rules-based method applies the CPO constraint and looks at every keyword individually rather than at the entire set of keywords. As a result, all possible solutions are not considered. Using the portfolio-based approach, we had four possibilities to consider, resulting in two solutions at the CPO goal of $20. And the rules-based approach picked the wrong one. DVD B Position DVD A Position Total Cost Total Orders CPO $ $20 d Optimal choice $20 d Rules-based choice $15 By bidding DVD A to position 2 with a CPO of $15, we use the CPO efficiency gained to bid DVD B above the acceptable CPO of $25 to get more volume. Because we are concerned only about the overall CPO, we are still on target. The rules-based approach precluded the optimal solution because it did not allow a keyword to exceed the overall CPO goal. The SEM portfolio A set of keywords that is optimized for a common business goal is called a portfolio. The business goal can be a common metric, such as revenue maximization or CPO reduction, or a more intricate one that involves maximizing multiple objectives such as registration and revenue. Clustering keywords for purposes of reporting does not constitute a portfolio; the keywords must be optimized using the portfolio approach. An average organization has multiple objectives and multiple metrics for each objective. As a result, the organization has multiple portfolios, each one trying to achieve a unique business goal. The efficient frontier The above examples demonstrate that finding the optimal solution requires considering all possible combinations of bids. When all combinations for the first example are charted, we obtain Figure 1. Each dot represents a possible portfolio with its unique position combination. Because there are three projects in five expense levels, there are 125 (5 3 =125) combinations. The blue line is the efficient frontier and represents the set of position combinations of the three projects that generate the highest revenue at every spend level. At a given spend level, the efficient frontier represents the maximum revenue that an advertiser can generate. While it is apparent that every advertiser would like to be at the efficient frontier, it is only possible to do so when all position combinations are considered i.e. a portfolio approach is used. 3

4 Figure 1. The efficient frontier for Example 1. Each pink dot represents a position combination of the three projects. The horizontal value of the dot represents the amount of money you expect to spend with this combination, and the vertical value represents the revenue you expect to obtain. At each spend level, there is a combination that will generate the most revenue. When these points are traced, you obtain the efficient frontier. Measuring lift with the efficient frontier Figure 2. Measuring lift with the efficient frontier. (1) Increase in volume of the revenue metric at the same spend level. (2) Reduction in the spend level at the same volume of the revenue metric. (3) Increase in the spend and revenue metric at the same CPO. The strong dependence of the efficient frontier with the spend level demonstrates that SEM obeys the law of diminishing returns, that is, you must spend increasing amounts to obtain an incremental gain in the revenue metric. In Figure 2, the efficient frontier first rises sharply, indicating a sharp increase in orders for a small increase in spend. At higher spend levels, the frontier flattens out, indicating very little increase in the orders for a large increase in spend. Hence, when measuring lift in pre- and post- periods, you must control one of the following parameters. Spend. When spend is kept the same, lift is measured by the increase in the volume of revenue metric. Revenue metric. When the revenue volume (for example, orders) is kept the same, the lift is measured by the reduction in spend. CPO. When the CPO is kept the same, lift is measured by the increase in both spend and revenue. In Figure 2, an advertiser was initially operating at a point below the efficient frontier. If the advertiser uses portfolio optimization, lift could be measured by either controlling for spend, revenue, or CPO. 4

5 The trouble with heuristics One way to reduce computational difficulties is to develop rule-of-thumb methods, known as heuristics in computer science. Heuristics differ from algorithms in that they do not guarantee the optimal solution. In SEM, common heuristics include bidding each keyword to a target CPA goal or target ROI, bidding a keyword up or down by a certain percentage based on the previous week s performance, and constraining certain keywords to high or low positions based on impression volume or revenue expectations. While these methods significantly reduce computational complexity, they oversimplify the problem and almost always give a suboptimal set of bids. For instance, the second example showed that the rules based approach failed for even a two keyword problem. The bid-high/bid-low heuristic fails on two counts: It doesn t use a portfolio approach, so many potential combinations are not considered and by considering keywords individually, you lose sight of the overall budget target for the SEM campaign. The need for algorithms With an increasing number of keywords, the possible combinations to be considered grow exponentially, as shown in Figure 3. With two keywords in eight positions, 256 positions (2 8 ) must be considered. With 10 keywords, the number becomes 10 million, and with 100 keywords, the total is 10,000 trillion. In addition, the optimal solution is guaranteed only when all possible combinations are considered. Therefore, a brute-force technique in which every combination is explicitly calculated will fail because the method is computationally intractable. Algorithms are mathematical methods that guarantee a solution when a series of steps is followed. The brute-force method is an algorithm, but a very slow and computationally intensive one. To solve the bidding problem for the search marketer, we need smart and fast algorithms that calculate the correct bids for a set of keywords using the portfolio optimization approach. Adobe s patented portfolio optimization algorithms are fast enough to ensure that over 60 million keywords in thousands of portfolios are optimized every day. Figure 3: The graph shows the exponential relationship between the number of possible portfolios and the number of keywords when eight positions are considered. At 100 keywords 10,000 trillion combinations must be considered. Models Estimates of key performance metrics, such as CPC, number of clicks, and revenue of every keyword in the campaign, are the fundamental kernels of information on which any algorithm or heuristic operates. If these estimates are inaccurate, any method to optimize a SEM campaign for performance is unsuccessful. Given the variability in the marketplace, it is no surprise that this is a challenging task. Keyword models simplify this task by mathematically capturing the essential characteristics of the search market place and predicting the performance metrics based on this information. Keyword models can further be divided into click models that predict the bid required, the CPC, and the clicks obtained at every position for a keyword, or revenue models that predict the revenue obtained at those positions. Figure 4 shows an example of a click model, Figure 5 shows a revenue model. Figure 4. Click models predict the clicks and CPC that a keyword gets when bid to a certain position. Figure 5. Revenue models predict the revenue generated by the keyword when bid to a certain position. 5

6 It is imperative to build accurate models because the prediction of a keyword s performance is dependent on the model s accuracy. This in turn determines the success of the entire SEM campaign. Three main factors determine the quality of a model: the availability of historical data, the capability to properly model tail terms, and the adaptability of the model. Availability of historical data When you attempt to predict performance from only a day s worth of data, the predictions are very inaccurate due to day-to-day market fluctuations. A good model incorporates the trends seen in a week to a month s worth of data. Long-term historical data is also needed to predict performance of keywords that might perform well during a season and then decline. Figure 6 shows the seasonality pattern of the keyword cheap flights. In the months of January, June, and July, when air traffic is up, the keyword surges in impression volume. A good model anticipates this traffic trend and bids the affected keywords appropriately. Adobe has over a decade s worth of data across many verticals, which enables us to build models accurately. Further, based on our research of aggregated vertical data, model accuracy has been improved further. Figure 6. Impression trends for the keyword cheap flights. Source: Google Trends. Despite year-to-year trends in traffic volume, there is surge in traffic in January and the summer months. Ability to model tail terms The majority of terms in a search campaign (over 80%) are tail terms that typically get less than one click per day. Due to the lack of historical data, sophisticated data aggregation techniques are required to build keyword models. Adobe uses finite mixture models for these keywords. In this method, click and revenue data from several keywords are clustered to build a common model. Figure 7: A maximum likelihood finite mixture model based on the data of three keywords. Keyword 1 has only had revenue from positions 4 and 5, keyword 2 from positions 2 and 3, and keyword 3 from positions 7 and 8. This combined model predicts the expected revenue at every position for these keywords. 6

7 Adaptability of the model Search engine marketplaces are influenced by several factors: news events, competitors, user behavior changes, and search engine algorithm changes. Keyword models must adapt to marketplace shifts to maintain their accuracy. At Adobe, adaptability is built into the click and revenue models in two ways. Click and revenue models are built on a daily basis for every keyword and every match type for every client. In other words, over 60 million individual models are built on a daily basis. This ensures that keyword trends are picked up and are responded to immediately. Both models use historical data. Very responsive models emphasize the previous 2 days performance when predicting future performance, average response models weigh 7 14 days performance, and sluggish response models weigh the previous 30 days performance. When recent data is weighed significantly, the model becomes very responsive to sudden permanent marketplace shifts. Sluggish models are slow to respond to marketplace shifts and are suited to short, one-time events. Model response study Example 1. Sudden and permanent marketplace shift. A keyword was simulated to get five times the expected number of clicks from day 5 onward. The prediction accuracy of the model was then measured on a daily basis until day 25. In this scenario, the rapid response model does best here because it catches up to the trend in 2 3 days. Example 2. Sudden and short-term marketplace shift. A keyword s click volume was simulated to increase by a factor of three on day 5 and then return to normal on day 6. The rapid response model reacts quickly but overcompensates for a few days. In this scenario, the sluggish response model does best because it considers more historical data to make click predictions. 7

8 Automation The ability to place a large number of bids is the most fundamental requirement of a sophisticated SEM campaign. Even a simple campaign containing 20 words when broken out to two search engines in 20 geotargeted regions requires 800 bids a day. If it took one minute on average to manually set a bid, it would require over 13 hours to set all the bids. Thus, it is fairly evident that even for a moderately sized SEM campaign of 10,000 keywords, automation is necessary. The need to change many bids everyday is often debated. The argument states that if 50 to 100 keywords constitute 80% of the spend, you need to change bids on 100 or so keywords. Figure 8 shows that this is not the case. In this example, the performance impact of the number of bids set was investigated for a large SEM campaign. If only 100 bids were set daily, you could miss the revenue targets by as much as 36%. At 1,000 bids, you would be off by 7%. At 10,000 bids, you would still be off by 3%. To be right on target, 30,000 bids are needed. Although 7% appears to be a small figure, if you were $50,000 a day, you would have a revenue loss of $1.27 million. Number of bids vs. expected revenue Figure 8. Effect of the number of bids on the expected revenue. To be on the efficient frontier, 30,000 optimal bids must be placed. When the number of bids is lowered to 1,000, the advertiser is 7% off revenue targets. At a 100 bids, 36% of potential revenue could be lost. It must be noted that in this study the bids were assumed to be set in the decreasing order of significance i.e., the first 100 bids set were the optimal bids for the 100 most significant head terms, the first 1000 bids were assumed to be set for the most significant 1000 terms and so on. Models, optimization, and automation: The synergy for performance at Adobe A closed loop formed by modeling, optimization, and automation technologies ensure that SEM campaigns run effectively. As seen in Figure 9, CPC, click, impression, and position data, collectively called cost data, are obtained for every keyword from the search engine. The corresponding revenue information is obtained from the advertiser. This data is combined at Adobe s data warehouses and used to build click and revenue models. Based on the advertiser s budget, business goals, and the keyword models, the optimizer calculates the optimal bids needed to be made in the search engine to ensure that the campaign is at its efficient frontier. This system auto-corrects itself every day. When bids are placed on a daily basis, cost and revenue data are received in the data warehouse. The discrepancy between the expected and actual keyword performance is corrected by the modeling mechanisms. These models are then fed to the optimizer, which places the appropriate bids the next day. 8

9 Automation vs. optimization While automation and optimization are both needed to manage a search campaign effectively, many advertisers confuse the two. The mere ability to set a large number of bids does not let an advertiser manage campaigns effectively. Indeed, the potential to do harm is enhanced as a larger number of suboptimal bids can be set. Portfolio optimization algorithms based on good keyword models ensure that the bids set by the automation mechanism are the correct ones. Figure 9. Modeling, automation, and optimization closed loop The power of forecasting The synergy of modeling, optimization, and automation also enables the advertiser to predict the performance of an SEM campaign. At Adobe, forecasting is done in a hierarchical manner to enable the search marketer to spend more time with strategic decision-making on areas such as budgeting and campaign growth, and less time on tactical issues such as keyword bidding decisions. The forecasting hierarchy is as follows: Keyword level The most fundamental forecast is the keyword model. It predicts the number of bids, number of clicks, CPC, and revenue at every position for each keyword in the portfolio. Figure 4 is an example of a keyword model. Portfolio-level forecast Portfolio-level forecasts predict the trade-off between spend and the business objective. Because the trade-off between volume and ROI or CPO is critical for most businesses, portfoliolevel forecasts enable the advertiser to pick the sweet spot on the efficient frontier. When an advertiser has several business objectives (that is, several optimization portfolios), portfolio-level forecasts can be combined to calculate the combined trade-off between spend and the business goal. Figure 10. Portfolio-level forecasts show the trade-off between revenue and ROI across different spends. Although the portfolio performance has actually not been tested across these spends, the forecast is within 20% accuracy. 9

10 Long-term forecasts Portfolio-level forecasts can be combined with historical seasonality data to provide long-term forecasts to the advertiser. These forecasts aid the advertisers to adjust their budgets. Figure 11. Example of a long-term forecast. The expected revenue is shown for a chosen spend level for 6 months ahead. For every month, each portfolio s net contribution to overall revenue is shown, along with the overall ROI. Conclusion A portfolio management approach, backed by robust keyword models, is the key to maximizing SEM campaign performance. In the current uncertain marketplace, the model-based portfolio management approach has the additional advantage of forecasting campaign performance accurately. The success of this approach hinges on three key factors: automation to help scale and respond to marketplace uncertainties, models that accurately predict the performance of a keyword at every position, and algorithms that use the models to calculate the precise bid for every keyword to maximize overall campaign performance. It is the models and algorithms that distinguish the players in this space. Good models are always backed with sufficient data to account for seasonality and other marketplace uncertainties, as well as be capable of handling sparse datasets, such as for tail terms. Additionally, good models must be robust enough to adapt quickly to marketplace uncertainties. Finally, algorithms must be backed with rigorous mathematics and not heuristics that have no performance guarantee. Adobe is pushing the frontiers of search marketing. With over a decade of proven success, Adobe manages $2.5 billion in spend across 250 global clients. Our platform operates at massive scale, setting more than 5 million bids a day across more than 120 million keywords. About the Adobe Digital Marketing Suite The Adobe Digital Marketing Suite offers an integrated and open platform for online business optimization, a strategy for using customer insight to drive innovation throughout the business and enhance marketing efficiency. The Suite consists of integrated applications to collect and unleash the power of customer insight to optimize customer acquisition, conversion and retention efforts as well as the creation and distribution of content. For example, using the Suite, marketers can identify the most effective marketing strategies and ad placements as well as create relevant, personalized and consistent customer experiences across digital marketing channels, such as onsite, display, , social, video and mobile. The Suite enables marketers to make quick adjustments, automate certain customer interactions and better maximize marketing ROI, which, ultimately, can positively impact the bottom line. About Adobe Systems Incorporated Adobe is changing the world through digital experiences. For more information, visit Adobe Systems Incorporated 345 Park Avenue San Jose, CA USA Adobe and the Adobe logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States and/or other countries. All other trademarks are the property of their respective owners Adobe Systems Incorporated. All rights reserved. Printed in the USA /12