1 The CMO s Guide to Making Big Data a Competitive Advantage by Martin Smith
2 Contents Introduction...4 Beware of the Panacea Why Big Data Isn t Enough...6 Welcome to the Newer World of Big Measurement...11 Big Measurement Principles Big Measurement: Bringing Together the Measurement Models...23 Conclusion...27
3 Over the last three decades, I have been privileged to work with clients and companies to develop new measurement models and apply marketing data to solve large, complex problems. This has also involved developing new methods and ways of working across multiple models and channels. I have been fortunate to work on innovations in customer loyalty, the creation of real-time redemption systems, business intelligence solutions, and the development of advertising solutions on the Web. Over the last 16 years I have been living in the cross hairs of Big Data and analytics. Martin Smith
4 4 The CMO s Guide to Making Big Data a Competitive Advantage 1 Introduction There is an interesting shift in advertising taking place, with ad spending increasing nearly 5 percent, but not in traditional media. In fact, according to Advertising Age, measured spending for the top 100 advertisers actually fell by 0.2 percent. 1 Meanwhile, Internet display advertising, for example, has experienced double-digit measured-media gains. What s happening is that marketers are now putting more money into disciplines that directly connect them with targeted consumers, which validates the emerging paid, earned, owned, and shared media model. With it comes the need for actionable customer data and a growing requirement for effective measurement across all media. The desire to directly connect with consumers is reshaping the media pie. Exactly how that pie shapes out will depend largely on advertisers ability to mobilize their customer data and to accurately measure performance at the point of delivery. The advent of Big Data has made unprecedented levels of granularity available with the promise of immediate actionability. Just a few years ago this involved very expensive solutions using data warehouse appliance technologies that largely relied on proprietary hardware. However, an open source model that is gaining rapid adoption and support across the technology space is quickly replacing such systems.
5 5 The CMO s Guide to Making Big Data a Competitive Advantage Scores of companies have taken in venture capital dollars to create new models that can leverage Big Data, making The Digital Lumascape 2 even more complex. Today we find a gap between the technology of Big Data and the ability to prove its impact on the advertiser s business. That is what we are terming Big Measurement: the process of applying the right customer relationship data with the right measurement metrics, and implementing a systematic model that manages both, to produce a marketing outcome that is performance and price performance positive. Big Data Measurement Big Data Gap Big Data Technology Big Data Business Impact 1 Advertising Age, June 24, Display LUMAscape, Luma Partners LLC, 2012
6 6 The CMO s Guide to Making Big Data a Competitive Advantage 2 Beware of the Panacea Why Big Data Isn t Enough Every decade we are assailed by a new panacea for managing data in the marketing business. In the eighties it was out-sourced processing, in the nineties it was database marketing, while the millennium heralded CRM. Big Data is the latest in the series of data management models that are set to dramatically change the way we manage and associate marketing data. What is interesting about this latest model is the driver is not hardware, nor is it enterprise software. The big data revolution is based on a model that relies upon open-source software running on standards-based hardware. This overcomes one of the largest limiting factors to Big Data: the big price tag to build, manage, and maintain a data environment that can handle very large volumes of transactional data. Tech, Tech and more Tech In striving to solve their own data problems, Google, Yahoo! and Facebook built and then leveraged the open source community to aid in the development of data management tools to meet the exponential volumes and scale that are the routine byproduct of their businesses. Intriguingly named tools such as Hadoop, Value added Data scale
7 7 The CMO s Guide to Making Big Data a Competitive Advantage Cassandra and the whole array of eco-system tools including Sqoop, Hive, Pig allow IT groups to deploy solutions that can scale with simple investments in standards-based hardware. One of the major applications Big Data specialists see for this new model is the creation of high-availability systems for digital and multi-channel marketers. This is validated by the development and deployment of products by marketing services behemoths such as Experian. The teams at Experian have led a number of large scale data solutions and continue to migrate key elements of business to the newly emerging model. What this means for marketers is the open source model will become more accessible at lower price points, which will allow their big data to become accessible data. Organizations will be able to keep more transactional data at their fingertips and make more insightful associations at least in theory. However, solving data scale in and of itself, while interesting, only makes sense when you are able to do new (and better) things with the data. Understanding how to manage data with increasing granularity only has value when it is applied to solving problems in new ways. In the Big Data management world, new analytics and targeting models are quickly emerging and being refined. We are seeing the emergence of large-scale data analysis in both the attribution space and the ad targeting space, where access to high-availability solutions can allow advertisers to use hundreds or more data points to make decisions in tenths of a second. This creates huge opportunities for marketers and data scientists to develop models that allow for continual refinement of advertising performance while managing the cost of performance. However, technology on its own does not deliver the full power of the opportunity, nor does data analysis guarantee a performance lift. Calculating 1+1 in two milliseconds is no different than at ten seconds. Similarly, incorrect calculations made with greater speed and repetition can cause much worse impacts than not ever entertaining the idea in the first place.
8 8 The CMO s Guide to Making Big Data a Competitive Advantage Measure, Measure, Cut Once Big Data without a corresponding focus on Big Measurement provides no value. Marketers need to spend as much thought on what to measure as on how they plan to store the data. Therein lies the competitive advantage. As Ogilvy once said, Marketers use research like drunks use lampposts for support, not illumination. In their study, From Stretched to Strengthened, 3 IBM identified that 71 percent of CMOs believed they were not prepared to manage the impact of the explosion of data. It is also surprising that 56 percent of the 1,734 CMOs interviewed believed they were unprepared for ROI accountability, and 65 percent believed they were unprepared for the growth of channel and device choices. The latter two statistics point to the need to understand not just the explosion of data, but how to leverage it. If Big Data levels the playing field, then Big Measurement is what separates the winners from all other participants. Right Measurement The best example of this was when I was working in direct mail for one of the UK s banks. We were being asked to take over advertising for a mobile banking product that had been launched with some fanfare. However, adoption and initial conversion was weak. As usual, we asked for all the data at its most granular level, as well as all the summaries. The summarized data told an interesting story. For example, the average age of customers was 32, while the average income 40,000. Based on the summary data, it was easy to see why all the direct mail pieces were featuring images of people who matched that profile. However, the phone 3 ibm.com/cmostudy2011
9 9 The CMO s Guide to Making Big Data a Competitive Advantage product was actually gaining adoption with two different groups the very young as well as more mature and sophisticated users. By averaging the data, the real story was being missed. The young users were making use of the product to check balances to ensure they had adequate funds to make purchases and pay bills. The sophisticated users with multiple accounts, meanwhile, used the service to move money between accounts for investment and money management. Not surprisingly, the ensuing direct mail pieces were correctly oriented to the exact needs of the target users. Response rates improved 20-fold thanks to relevant communications driven by the data. This experience taught me a valuable lesson of never managing to the average and always breaking down measurement to granular levels. Even today we regularly see poor media conclusions being derived from overly summarized data. Sadly, this is more the norm than the exception. Right Key Performance Indicators Why is Brad Pitt like measurement? The association is Pitt s movie, Moneyball. The film is the story of how the Oakland A s consistently performed at or above the levels of teams with significantly better capital and power to buy the best players. The film dramatizes Michael Lewis s book of the same name. The A s simple strategy was to implement a model based on the identification of Key Performance Indicators that drove repeatable performance. Armed with this data and the guts to go against perceived wisdom, the A s were able to consistently out-punch their weight. David had understood how to master Goliath. Interestingly, this is not an exception, but a very manageable model. In his New Yorker article Malcolm Gladwell artfully shows how, by changing the rules, teams with significantly fewer resources can consistently out-match better resourced and equipped combatants. When David plays by Goliath s rules he will lose 73.5 percent 4
10 10 The CMO s Guide to Making Big Data a Competitive Advantage of the time. However, when David changes the rules and plays by them, he will win 63 percent of the time 3. Big Data only has value when it is accompanied by Big Measurement. Big Measurement is about challenging organizations to look at, not just how to tame data, but how to really apply it in ways that make a positive impact on Key Performance Indicators. The lesson is pretty clear whether you are Brad Pitt or a general in battle. You need to understand what you are measuring, why it is important, and how impactful it is to you. Build your system around what measurement drives your business.
11 11 The CMO s Guide to Making Big Data a Competitive Advantage 3 Welcome to the Newer World of Big Measurment It s not just about seeing the world with new eyes, but about using tools and technologies to measure things we were never able to measure We can then make connection between people and things. As marketers, that s what we re paid to do. John Harrobin, VP Marketing Communications & Customer Relationship Management, Verizon In the Moneyball example, the competitive advantage that the A s created was not sustainable because the metrics being used were available in the public domain. As a result, their methodology could be recreated by using data points that were widely and freely available. Only when the data that is being used is proprietary can advantage be sustained. This creates a very interesting starting position for advertisers who have access to valuable proprietary data: their relationships with their customers. If Big Data and Big Measurement do nothing else, please let it help advertisers be more relevant to their customers like me. Each month I pay my credit card bill online. Now, because I work in the industry, I know that it is extremely simple for the credit card company to create a simple variable that allows them to decision an ad message based
12 12 The CMO s Guide to Making Big Data a Competitive Advantage on any criteria. So I find it annoyingly amusing that each month I am targeted with banner ads asking me to sign up for a product that I have owned for over 15 years. There are plenty of things that this company can sell me. They can get me to use more features of my card I ll bet I don t know half the things I could use it for. Or they could upsell me to the next card, or simply say thank you. All would be more relevant than their current model. Advertiser, Meet your Customer (Digitally) Early in the Internet advertising space, Karen Ritchie s media team at General Motors, Cyberworks, had clued into the customer relationship concept and developed a series of landmark models through their technology providers. The GM team understood early in the game the pivotal opportunity that the emerging digital display media channel posed for major advertisers. The advent of ad technologies created a breakthrough that had not been seen in the history of media. At no point prior to digital display media was the advertiser present at the moment of delivery and the point of measurement. Unlike all other advertisers at that time, GM had the foresight to secure all media to their domain. This seems trivial, but every ad GM served was delivered through a designated sub-domain through their own ad server. This allowed GM to secure their data and manage it in their own data warehouse, which at the time was the fifth largest Oracle database. Larry Lozon, the head of GM Cyberworks, referred to this as the creation of their second asset. It drove key learnings on media analysis and measurement and would eventually drive targeting and optimization a full 13 years before Data Management Platforms (DMP) began to make news. What makes the GM story remarkable is the sheer breadth of vision that the team sponsored. Not only did they understand the power of the second asset, they also sponsored significant development of new measurement standards, and understood the strategic impact of developing and securing measurement standards for an advertiser.
13 13 The CMO s Guide to Making Big Data a Competitive Advantage Their innovation was borne out of frustration at the lack of accurate and comprehensive measurement standards available in display media. Publishers would provide lackluster reporting because of over 25 measurement standards in use by mainstream publishers and portals. Additionally, measurement was being distorted by caching of images by proxy servers that created huge measurement discrepancies. Ironically, what is now the industry s current measurement standard is actually patented technology the 302 redirect. In the land-grab of the internet and subsequent downturn of 2001, some important lessons got skipped over. Measurement became standardized and third parties provided operational rather than strategic services. The measurement and creation of second assets became less important because Search had risen as the new channel-of-choice and networks simply aggregated high quantities of low value inventory. The great thing about history is it so often repeats itself and areas that became old become new again. Big Measurement is about to become the new competitive advantage, and offers advertisers the ability to create large payoffs by understanding how the new measurement model will help them change their media strategies. The reason I am so confident of this is that we are already seeing companies that pay attention to Big Measurement reap significant reward. Oreck and Big Measurement Doug Cahill s team at Oreck has been eliminating sacred media cows for the last two years. Doug s view of the world is, simply, You have to get to the source of data or you ll be stupider than the people you work for... Doug hired Tarik Dekkar, a direct marketer with interactive media experience.* Tarik s background is interesting because it speaks to a key area that of investing in the right human capital, not just technology. New opportunities are being created by practitioners who understand different measurement models and how to take key aspects of them to drive new media models that make significant competitive difference. If the Oreck team had simply
14 14 The CMO s Guide to Making Big Data a Competitive Advantage applied the status quo technologies and standards, they would have had the same results as everyone else and made the same media investments as everyone else and then lost the competitive advantage they now enjoy. Instead, the Oreck team has developed their model to the point where they understand their measurement and performance and can exert significant control with a high degree of granularity. By following the principles of Big Measurement, Oreck was able to prove digital display was driving sales across all channels, identify a 55 percent improvement in demand creation, quantify a 24 percent reduction in cost of acquisition, and demonstrate a doubling of the ROI of their display advertising. They know how to balance media investments to the point where they confidently migrate media dollars from television to interactive. The net result is that they have grown their investment from a $50,000 digital media test to seven-figure monthly investments. They continually test and revise all the sacred truths to find that there is significant competitive advantage in taking the road less traveled. The performance and cost of performance battle is being won by focusing on applying the right data with the right measurement and implementing a systematic model that rigorously manages both. When you get the model right, the payoff is huge. It is exponential, not incremental. * Oreck is a client of TruEffect
15 15 The CMO s Guide to Making Big Data a Competitive Advantage 4 Big Measurement Principles Marketers must be much more hands-on in the media planning, buying, and optimization process in this world where the old model is in danger of losing relevancy and effectiveness when it comes to performance and audience-based campaigns. Joanna O Connell s Future of Digital Media Buying 5 To go beyond the merely obvious requires that the organization be more involved in the media process, or at least in understanding the process. To do this successfully we have tested a number of models in the Big Data/ Big Measurement space. From these we have developed a set of principles that offer CMOs a framework to build a model that delivers an effective ROI on Big Data investments in the media space. These principles include: 1. Accuracy 2. Getting the Data Hierarchy Right 3. Align Your KPIs with Your Measurement 4. Action/Communication 5. Integration 6. Stewardship 7. Systems Right-Sizing 8. Human Capital Alignment
16 16 The CMO s Guide to Making Big Data a Competitive Advantage Accuracy Data accuracy is oxygen to the organization. Often you can t see polluted data until it is too late and you have made poor decisions. Like pollution, you don t know how bad it is until you step into more rarified climates. Whenever I m starting in a new subject area, the first thing I look at is data accuracy. As Doug Cahill will tell you, you don t find the answers by sitting in the office or toying with the data. Instead, you have to do a little spade work. (Cahill s lecture to Vanderbilt is a must-watch: The Answers Come from Your $12.00 an Hour Team). You find answers in the call centers and rolling with the junior analytics folks or challenging the ad ops person. To ensure the quality of your data you need to investigate it all the way to the source. A classic example was when I was working in multi-channel consumer electronics. While reviewing source data at the transaction level, we started to see interesting associations such as which call center reps use which catalog source codes (the way of tying it back to mailings) and how often. I say interesting because rather than learning new source codes or asking the caller for their catalog code, the reps simply added codes that they knew worked. Armed with this knowledge, I started hanging out on the sales floor to understand the behavior. Beyond simply using codes they knew worked to expedite the calls, other reps went even further by using codes that carried 10 percent discounts which they used as deal-closers. The problem was they were being used on every call. We found that 40 percent of our codes were being entered in error. Without this knowledge, our measurement and subsequent decisions would have been skewed. In the digital media space we see equally interesting behaviors around media measurement. Over the last few years different elements have impacted measurement efficacy for ad serving. Security software now deletes 5 Forrester 2011
17 17 The CMO s Guide to Making Big Data a Competitive Advantage designated (tracking) cookies almost daily, and Apple s OS X and ios do not allow them as a standard setting. This has led to significant inaccuracies in media measurement. The alignment of data and its integration is a critical component in both Big Data and Big Measurement. There is no point on-boarding great big volumes of transactional data if it is not accurate, nor will ever be accurate. For example, the prevailing premise that third party technology miscounts reach by 3x means 66 percent of all the data being written to a transaction Big Data solution will have no value except to provide impression counts, which can be gleaned from existing summarized reports. The result is Big Data with no value. We have seen huge variance between advertisers in simple areas such as measurement accuracy, where traditional methodology error rates are between.5x and 12x. In purchase normalization, we see a range of 5 percent to 35 percent for misattributed purchase transactions. Given these very large variances, it makes significant sense to ask the tough questions and not take the numbers at face value. Getting the Data Hierarchy Right Big Measurement associates new types of data to inform and provide a framework for action. This data framework challenges the organization to understand how the data should be organized to facilitate access to the most powerful variables in the right sequence. Reach extension Act-alikes (audience augmentation) Visitors Customers When dealing with audience data, that data is typically organized in a hierarchical structure against the key drivers of response: Recency, Frequency, and Monetary Value. This hierarchical structure can then be used across a range of planning and measurement models. Frequency, for example, can have a direct
18 18 The CMO s Guide to Making Big Data a Competitive Advantage relationship with loyalty as three-time buyers (and beyond) are invariably and exponentially more loyal. In other markets different elements may be used, such as feature utilization. In the mobile phone business, for example, customers that use three or more features are six times more loyal in terms of recommendation and repeat purchase. In the financial market, it is more about the customer s product usage and potential usage. The important thread to point out is that we are starting with the data that has the strongest ability to inform us about the relationship between the advertiser and the consumer. We then work out from the center in decreasing degrees of association. Align Your KPIs with Your Measurement A big part of measurement rigor and getting results is to ensure your Key Performance Indicators (KPIs) are closely aligned to what you are measuring. While this sounds straightforward, it is surprising how often there are moments of silence in meetings when, after seeing 40 slides of data, the question is asked, Did it impact our KPIs? Interactive marketers are often guilty of this when debating whether a fractional improvement in the click-through rate was important, while missing the fact that the overall return on the investment was poor. The debate should instead be about outcomes. For example, did it generate increased sales or did it attract new customers? Did it improve brand awareness or did it reach a net-new audience? Action/Communication It is one thing to know something, but quite another to do something with what you know. Big Data and Big Measurement create an opportunity to make learning actionable in real-time. As one of my hunter friends noted, It is one thing to know a deer Relationships Sales 20% 60%
20 20 The CMO s Guide to Making Big Data a Competitive Advantage The reason integration is critical to Big Measurement is the need to make sure that various metrics are properly organized and represented. This seems particularly burdensome as we mentioned earlier in digital display media where there are very large discrepancies in traditional third-party media measurement. This discrepancy has been highlighted by groups such as Comscore, Google, and MediaMind 6 and it also happens in site analytics and media analysis. Stitching together data, but not integrating it, is a significant issue for organizations pursuing effective Big Data/Big Measurement solutions. The further distanced you are from your data, the more prone to measurement errors your solution becomes. In addition, the further distanced you are from the point of interaction, the less impactful you can be translating insights into timely and relevant actions. These can be as tactical as targeting or more macro such as media optimization, but in any case effective integration of your data is critical. It is not surprising that most channels use First-Party data approaches to most effectively manage these elements. Stewardship Big Data and Big Measurement also require that you manage stewardship of your data. A friend of mine at a very large CPG company once quipped that, Everyone uses our data except us. In a world of increasing data and access, this becomes increasingly important. It s not just what you do with data, but who has access to transactional information in the display advertising vendor eco-system. Stewardship is not merely a nice-to-have, but an imperative as advertisers need to increasingly demonstrate leadership in the privacy area. The increase in mobile usage, as well as the desire to measure multi-channel, multi-device usage, and increased requests by third parties to use data, creates increased opportunities for data proliferation, which is not necessarily a good thing. 6
21 21 The CMO s Guide to Making Big Data a Competitive Advantage There is an interesting paradox that studies have shown regarding consumers. While they like, and to some extent expect, relevant advertising, that is counter-balanced by an increased emphasis on privacy management. This was demonstrated in the uptick in Do Not Track opt-outs seen after the Google/Apple Safari issue. 7 Do Not Track Opt-outs 4.0% 3.5% 3.0% 2.5% 2.0% 1.5% 1.0% 0.5% 0.0% Q Q Q Data has shown a doubling of Do Not Track Opt-outs to 2.1 percent, with some network-heavy media reporting 4+ percent. Systems Right-Sizing The final principle of Big Measurement is systems right-sizing. Big Data/ Big Measurement operates best from within an advertiser-oriented environment. That does not mean data has to be directly managed, but the most effective ways to implement a cogent Big Data/Big Measurement approach is to orientate the systems at their most basic level to advertising. This removes ambiguity from stewardship, simplifies integration, improves the ability to take action, improves accuracy, and eliminates structures not tailored to the specific advertiser dynamics. Big Data/Big Measurement is not a one-size-fits-all model. Rather, it requires an understanding of specific advertising needs and strategic planning to create a framework that can deliver results. The rise of agile development in the systems space can also be applied to Big Data/Big Measurement