SOCIAL MEDIA MEASUREMENT: IT'S NOT IMPOSSIBLE Chris Murdough A lot of excitement and optimism surround the potential of social media for marketers-after all, that is where attractive audience segments are starting to amass-but just like any marketing medium, measurement is a critical component for success. Social media measurement is quite immature, just as Web analytics was back in the mid-1990s. It will evolve quickly as marketers attempt different approaches and hold enterprise measurement firms accountable to help make sense of all the activity data generated by social media interactions. In turn, social media can seem very challenging, and at times even impossible, to measure with regard to its effects. At its core, digital measurement is contingent on controlling the customer experience and laying tracking hooks along the way to understand response and subsequent behavior. However, social media's foundation challenges this very tenet: The consumer is now in control after all! In my experience, social media has nuances, just as any media platform does, but overall, similar principles should be planned for when embarking on any social media measurement program. I follow a continuous, iterative process that ensures quantitative rigor is applied to all phases of marketing program development, as well as to post-launch optimization. Social media efforts should follow a similar approach. Figure 1 illustrates the main phases of the social media measurement process. The Concept phase focuses on defining what brands want to accomplish through advancing their relationships with their customers and/or prospects. Social media is merely a means of getting to the goal. The types of activities that take place during the Concept phase include: Map measurement objectives to project or business goals. Identify specific key performance indicators (KPI) that align with objectives. Establish performance benchmarks or targets to gauge success. Figure 1. Social media measurement process Journal of Interactive Advertising, Vol 10 No 1 (Fall 2009), pp. 94 99. 2010 American Academy of Advertising, All rights reserved ISSN 1525 2019
95 Journal of Interactive Advertising Fall 2009 The output of the Concept phase might look like Figure 2, which is a sample metrics framework developed for a planned social media program. The important rule is to focus on just a few metrics for each objective so that program evaluation remains simple and one does not end up in "analysis paralysis." The process should force the prioritization of only key measures. Another attribute to note is the mix of both qualitative measures (discussion topics and sentiment) and quantitative measures (counts of reach, chatter mentions, site engagement events). Social media is unique in bringing both types of insight together to characterize performance and the value derived from social media efforts. Figure 2. Sample social media metrics framework The second phase in the social media process is Definition; it involves creating an outline of how social platforms could be leveraged to reach and interact with a brand's target audience to support the objectives. The types of measurement activities that take place during the Definition phase are: Itemize insight questions critical to understand program performance; which social media levers might have key influence over KPIs? Illustrate the analysis approach best suited to inform optimization actions. Determine the frequency of performance evaluation and timing of program enhancements. One of the measurement planning outputs of the Definition phase might look like Figure 3, which depicts an analysis construct used for many social media programs. Social media monitoring and measurement quantifies the impact of consumer dialogue. Social activities can influence all points of the consumer relationship funnel, but there are three pillars to organize around to navigate the complexity of social media performance and guide program enhancement decisions: 1. Reach. Monitor the quantity of mentions and quality of authors. Not all authors talking about a brand are created equal; some simply have more clout than others, based on the scale and activity (visit and comment frequency) of the fan base they potentially reach with their comments. It is valuable to keep tabs on the influence profile of the social audience. Also, it is important to understand where on the social Web discussions about a brand occur- Twitter, Facebook, MySpace, discussion forums, blogs, and so on-to understand how key messages propagate across social venues, as well as determine if and how a brand should develop an active presence at each of these venues. 2. Discussions. Understand chatter topics and sentiment composition. What are authors on the social Web talking
96 Journal of Interactive Advertising Fall 2009 about in terms of a brand or its product attributes? How do the topics of conversation differ from what the client would like authors to talk about? First listen to what issues or topics social discussions center on, then consider the tone or sentiment (positive or negative) before allowing the brand to jump in prematurely and broadcast its perspective. However, over time, by adding value to the audience and cultivating relationships through conversation, brands can influence the topics and perceptions being discussed. 3. Outcomes. Diagnose behavioral engagement indicators of intent (e.g., leads, purchases). Let's not forget that brands are in the business of making money. Even if that is not the primary goal, monitoring outcome behaviors is how to gauge any economic value resulting from social media efforts (either directly or indirectly). Outcome measures can take the form of the most relevant aspect to the brand, such as leads, purchases, or even a custom site engagement measure that acts as a proxy for purchase intentions. To provide context to social performance, it is imperative to evaluate this measure using the same lens employed by other primary referral sources to the brand Web site (e.g., SEO, SEM, OLA, direct/ bookmarked, e- mail). Figure 3. Sample social media analysis construct Synthesizing all three of these performance categories helps clarify comprehensively what is going on in the social Web for brands and best advise them how they can leverage the audience in this space. The third step in the process is Design, which lays out the specific tactics and venues most appropriate for a brand's active social media presence and also entails the following measurement activities: Establish performance data sources and/or methodologies. Enumerate the specific technical tracking hooks and manual interventions needed to ensure proper and complete data collection. Set up, configure, or customize performance reporting tools to ensure desired performance insight is readily available. Figure 4 illustrates the type of performance data ecosystem available to social media marketers. As mentioned previously, holistic social media measurement includes both qualitative and quantitative performance sources. The providers indicated in each of the four categories are intended to be a sampling of available tools, not an endorsement of one provider over another.
97 Journal of Interactive Advertising Fall 2009 Figure 4. Sampling of social media insight tools and data sources Let's examine each data source to provide some context for the unique insight each brings to the evaluation of social media programs: 1. Enterprise listening platforms (ELP). Most of these tools provide some work flow management capabilities, but primarily, they centralize all publicly accessible discussions across the social Web, made available for reporting and synthesis. With regard to the analysis construct, these platforms provide the data necessary to inform a large portion of the reach and discussions performance insight components. 2. Text mining partners. Manually reviewing every social comment that includes a brand is not only physically impractical, but it would also introduce human interpretation bias. So these particular solutions help social practitioners semi-automate the process of deciphering the meaning of social comments, specifically by pulling out topic categories and sentiment. Natural language processing is really challenging because of the large portfolio of exceptions inherent to the English language and people's usage of it. Therefore, nothing automated will be 100% accurate; however, it helps tremendously to gauge the primary themes of discussions efficiently, especially when this automatic effort gets supplemented with ongoing sampling and manual review of comments to ensure updates to any keyword tuning. 3. Platform API (application programming interface) tools. This portfolio of niche social measurement tools gives access to key social insight activities not readily available in the mainstream ELPs (i.e., Facebook Insights for private-access profiles; Visible Measures for video clip distribution, usage measurement, and propagation accounting). They are used on a case-by-case basis for social programs. 4. Site analytics solutions. These tools provide insight into a brand's Web site visitor behavior and, in the case of social media measurement, act as the primary source of a brand's outcome activities that are part of the analysis construct. Referrals from social media venues, such as blogs, Facebook, Twitter, and discussion forums, demonstrate what sort of high-value behaviors occur once social audiences reach the brand's site, including leads,
98 Journal of Interactive Advertising Fall 2009 transactions, and priority content consumption. With the right set up of referral domain categorization and campaign tracking code utilization, tying site activities back to direct social media efforts is possible. Thus, there are many social insight data sources available, and some even overlap in the capabilities they provide. It is important to establish data sources of record for each KPI identified in the metrics framework (during the planning stage) to ensure all parties involved agree on a single version of the truth about program performance, as well as determine the necessary data infrastructure to plan for and establish in time for the launch of any social media effort. With the social plan, strategy, and tactics ready, it becomes time to move into the fourth phase of program development: Deployment. In this phase, the focus is on ensuring that what is being seen and transpiring in social venues is what is expected, in terms of content, editorial calendars, work flow management, brand voice consistency, and so forth. The types of measurement activities that take place during the Deployment phase include the following: Conduct QA (quality assurance) of data collection methods. Validate performance reports are complete and accurate with expectations. Build data infrastructure needed by aggregating multiple data sources to populate the social media performance dashboard. After being in market with a social media program and collecting data, the final phase of the process entails Optimization. The Optimization phase is where all the work done in the previous four steps comes together in the form of reporting and insight to evaluate the social program performance against the KPIs determined in the Concept phase (metrics framework), diagnose drivers of performance to date according to the Definition phase (analysis construct), and identify opportunities for social program enhancement by developing a holistic picture of social activities and their impact (Design and Deployment phases). Figure 5 depicts a possible version of a social media performance dashboard. The social dashboard is an insight tool, built to navigate the complexity of social media performance and provide guidance for program enhancement and optimization. An executive summary area houses all program observations, performance diagnostics, and optimization opportunities centrally. Accompanying the executive summary is a U.S. map, overlaid with social mention volume by state or designated market area (DMA), which provides a macro view of the geographies in which brand discussions are concentrated. The three performance pillars from the analysis construct use specific performance visualizations to answer the following key questions and aid in decision making: 1. Reach. What is the volume of brand mentions across the social Web? How many authors are generating these mentions? Where are these mentions occurring (e.g., Twitter, social networks, blogs, discussion forums)? What's the social influence profile of the authors talking about the brand (high, medium, or low), and how does this author composition change over time as the brand discovers and cultivates social relationships with influencers? 2. Discussions. Which topics or themes are the main foci of discussions? How does this focus compare with what the brand wants consumers to talk about? What's the composition of mentions: positive or negative? Are there differential words of prominence used in positive mentions versus negative mentions? How does net sentiment change over time as a gauge of brand health? 3. Outcomes. What is the level of site engagement resulting from social Web efforts? Which referral sources drive purchase intent: blogs, Facebook, Twitter, direct, SEO, e- mail, OLA, or SEM?
99 Journal of Interactive Advertising Fall 2009 Figure 5. Sample social media performance dashboard A disciplined development and post-launch process very similar to the one we would use to plan and execute a marketing campaign or Web site redesign is used, but it is critical to understand that social media cannot succeed as a short-term execution, like a flight or a project. Social media demands a commitment. Social media never ends, and brands that consider embarking on active participation in the social venues where their audiences congregate (which they should) must be mindful of that commitment. Their resources and effort should be thoughtfully planned for and deployed to manage the social dialogue with customers and prospects actively. The nascent social space is rapidly expanding. That means it is experimentation time for marketers. This article recommends a social media measurement process: Decide on how a brand might want to change their relationship with customers, put a thoughtful plan together, accept the tracking imprecision of consumer control and calibrate expectations, and then get out there and see what happens. It is a great time to be a digital marketing measurement practitioner. ABOUT THE AUTHOR Chris Murdough is Vice President/Associate Director of Digital Analytics at Mullen. He is an experienced digital analytics advisor, providing measurement strategy guidance and creating actionable insight solutions for clients across many industries. Chris has a Bachelor's degree in mathematics and a Master's degree in statistics from the University of Vermont. E-mail: chris.murdough@mullen.com. ABOUT MULLEN Headquartered in Boston, Massachusetts, Mullen is a fullservice modern communications agency, integrating disciplines from creative to direct response, public relations, full-service media, social influence, digital production, and analytics. Beyond what it creates, Mullen is also a brand, defined by a strong and enduring culture of collective entrepreneurialism. It likes to build and grow businesses.