Behavioral Data as a Complement to Mobile Survey Data in Measuring Effectiveness of Mobile Ad Campaign

Transcription

1 Behavioral Data as a Complement to Mobile Survey Data in Measuring Effectiveness of Mobile Ad Campaign CASRO 2014 Thao Duong Steven Millman comscore, Inc. Proprietary.

2 Agenda 1. Research Objective 2. Data Collection (Survey & Behavioral) 3. Analysis and Results comscore, Inc. Proprietary. 2

3 Background & Research Objective Background Interactions are emerging as a new success metric for evaluating effective digital campaign performance, replacing the outdated click-through-rate. Previous research from the IAB demonstrated significantly higher interaction results from desktop display IAB Rising Star ads compared to standard banner ads. This goal of this research is to bring new learning to the industry and foster growth of mobile channel advertising by studying brand lift and interaction metrics of rich media in relation to banner ads on mobile devices. Research Objective To test the impact of different mobile ad formats compared to standard mobile banner ads, examining the effect of interactions on branding, the user experience and ad receptivity. To date, the interplay of these factors had not been studied. 3

4 Study Design Methodology Three leading U.S. brands participated in the study. The ad formats were developed in-house by Vibrant s Creative team from assets provided by each brand. Design A standard experimental design using control/exposed groups was utilized. A sample of 2,227 respondents from a mobile panel, ages took the survey on their ios smartphone or tablet. The three-phase sequential fieldwork took place from March-October Ads were developed for each of three brands as a standard mobile banner ad, IAB Mobile Rising Star Ads, and Vibrant Mobile Rising Star ad with in-text branded keyword. Each ad used the same creative assets for consistency and to eliminate bias during testing: Ads Tested: 1- Standard mobile banner ad 2- Adhesion Banner Ads 3- in-text branded keyword Metrics Collected Brand Lift from an online survey User Interactions based on pixeling ads, tracked by comscore Confidence Level Statistically significant at 90% 4

5 USERS SAW A MOCK WEB PAGE WITH: Either a Mobile Rising Star Filmstrip Ad or a Vibrant Rising Star Ad or Standard Mobile Banner Ad or No Ad (Control) Banner Ad Mock Page with Brand Flex Text 5

6 Survey Data Collection comscore has an online panel whom we and invite them to participate in the study. They have to take the survey from a mobile device. We sniff the device user agent present in the browser to make sure that the respondents are using their mobile device. They will be screened out after 2 nd attempt from non-mobile device. The survey includes a short set of brand impact measures and attitudinal questions related to the specific ad unit. For a balanced sample, a minimum of 200 completes in each group is needed. Experiment is set up for 3 different brands in different industries to avoid bias created by popularity of the brand 6

7 Behavioral data collection comscore implement a number of pixel calls on every interaction point for each mobile ad unit: Multiple pixel calls are made at every user s movement pertaining to the ad (eg., when the user swipes the ad, when he/she moves to the next level of content, when he/she leaves the ad) A special pixel to capture a randomly generated ID that was also captured in the survey records Time stamps associated with each interaction were captured in the pixel call => provide time length a user interacts with an ad However, in this paper, we treat interaction as a binary incidence (interacted vs. not) 7

8 Analyses Method: Test & Control Users exposed to each ad type (standard banner or adhesion banner or in-text branded keyword) represent a test group Users not exposed to any ad type is the control group Different combinations of pair wise groups are analyzed using survey data only no significant lift was found. With interaction data, we find that most users only looked at standard banners (< 5%) while higher proportions interacted with Adhesion Ads and In-text branded keyword Ads (~ 10%) interaction drives more lifts. 8

9 Preliminary Lifts Without Interaction Data Table 1: Results for users not interacting with any type of ads Top 2 Boxes Standard Banner Adhesive and in-text branded keyword Ad Point Lift Impression of the brand 20% 24% +4 Recall the brand 54% 55% +1 Resonate emotionally with positive brand memories 33% 32% -1 Recall the ad message 53% 44% -9 Recommend the brand 64% 62% -2 Improved impression of content 10% 12% +2 Ads are more engaging 21% 23% +2 Ads are more attention grabbing 38% 41% +3 Complimented the content 31% 32% +1 Ads are more enjoyable 32% 30% -2 Positive lift significant at 90% Negative lift significant at 90% 9

10 Lifts after combing survey and interaction data Table 2: Results for users interacted with MRS ads vs. users viewed standard banners Top 2 Boxes Standard Banner Interaction with Adhesive and in-text branded keyword Ad Point Lift Impression of the brand 20% 38% +18 Recall the brand 54% 90% +36 Resonate emotionally with positive brand memories 33% 44% +11 Recall the ad message 53% 63% +10 Recommend the brand 64% 72% +8 Improved impression of content 10% 21% +11 Ads are more engaging 21% 37% +16 Ads are more attention grabbing 38% 56% +18 Complimented the content 31% 43% +12 Ads are more enjoyable 32% 42% +10 Positive lift significant at 90% Negative lift significant at 90% 10

11 Conclusion The ability to add interaction points using pixel calls on different types of ads on mobile devices provides additional information to the survey data. Adding interaction data, we had a clearer picture and come closer to the truth in measuring the effect of ad types on mobile devices The interaction with certain ad types created lift compared to standard banner ads that are normally having lower interaction rates. Future research effort may include finding the relationship between length of user interaction period with the effectiveness of the ad. 11