The Economic and Cognitive Costs of Annoying Ads

Transcription

1 The Economic and Cognitive Costs of Annoying Ads Daniel G. Goldstein Siddharth Suri Fernando Diaz Microsoft Research, New York City R. Preston McAfee Microsoft Corporation Matthew Ekstrand-Abueg Northeastern University

2 Questions of the day How can online labor markets be used for field studies? What are the economic and cognitive costs of annoying ads?

3 Field studies in online labor markets Mechanical Turk Widely used as subject pool (much like the early ones of Reips, Birnbaum, Baron, Göritz, Goldstein, etc.) However, MTurk was created as a bona fide labour market Work as DV. Dropout as a good thing. Compensating differential Amount needed to pay someone to compensate for risk or unpleasantness Innovative method of Toomim et al for testing user interfaces

4 Publisher: New York Times Citibank paid New York Times Display Ad Advertiser: Citibank

5 Display advertising Online advertising about as large as broadcast TV Display: $13 billion USD in % of online advertising revenue Display expected to overtake sponsored search ads on mobile devices

6 Good Ads

7 Bad Ads

8 Mobile annoying ads

9 Tension " Advertisers pay publishers to display annoying ads " Annoying ads cost publishers page views through user abandonment

10 Impacted parties " Publishers " Lost traffic " Apparent desperation: Warning of service quality " Users " Interfere with content consumption. 200 million AdBlock downloads on Firefox alone. " Advertisers " Appear disreputable " Undermines money-to-burn signal

11 Research questions " What makes ads annoying? " What are the economic costs of annoying ads? " What are the cognitive costs of annoying ads?

12 Related Work " Yoo & Kim 2005, Burke et al 2005 " Moderate animation increases brand recognition and attitude " Heavy animation decreases brand recognition and attitude " Goldfarb & Tucker 2011 " Ads that are intrusive or match content increase intent to purchase " Ads that are both decrease attention decrease intent to purchase " Calculating compensating differentials, Toomim et al 2011

13 Preparatory study: Identifying annoying ads

14 Ranking Ads by Annoyingness " 72 animated ads, 72 static ads generated from their final frame " Users randomly shown either the static or animated version " First users previewed all ads

15 Ranking Ads by Annoyingness " 72 animated ads, 72 static ads generated from their final frame " Users randomly shown either the static or animated version " First users previewed all ads " Then rated each on a 1-5 scale of annoyingness

16 Ranking Ads by Annoyingness " 72 animated ads, 72 static ads generated from their final frame " Users randomly shown either the static or animated version " First users previewed all ads " Then rated each on a 1-5 scale of annoyingness " Finally, commented on why

17 Preparatory study participants " Conducted on Amazon Mechanical Turk " $0.25 flat rate, $0.02 per ad rated " 141 participants in 18 hours " Restricted to US workers with 95% approval rating

18 Identifying Good & Bad Ads bad ads good ads " Animated ads are most annoying, static are least " Used 10 most/least annoying ads in study 2

19 The Impact of Animation " Animation caused ads to be rated as substantially more annoying

20 What Makes Ads Annoying?

21 Study 1: The economic cost of annoying ads" "

22 Measuring the " Compensating Differential " Task: users classify , can stop any time " Randomly assigned into " 3 pay conditions:.2,.4,.6 cents per " 3 ad conditions: no ads, good ads, bad ads " Treatment not revealed until acceptance " Circumvented ad blockers

23 " Ads were randomly selected from the set of good or bad ads " New ads with each " s from the Enron data set

24 Study 1 participants " Conducted on Amazon Mechanical Turk " $0.25 flat rate, 0.2, 0.4, 0.6 cents/ " 1223 workers in two weeks " Restricted to workers with 90% approval rating

25 Compensating Differential: " Bad Ads to No Ads " Bad ads caused 12.7 fewer impressions compared to no ads ".2 cents per impression yields an additional ~16.6 impressions " To get an additional 12.7 impressions need to pay 0.2 * 12.7/16.6 =

26 Calculation If you get 16.6 additional impressions for every.2 cents you pay, how much does it cost to make up the 12.7 impressions lost to annoying ads? X ( 16.6 impressions/.2 cents )= 12.7 impressions X =12.7 impressions(.2 cents/16.6 impressions ) X =.153 cents/ impression X = $1.53 / 1k impressions =$1.53 CPM

27 Compensating Differential: " Bad Ads to Good Ads " Bad ads caused 9.5 fewer impressions compared to good ads ".2 cents per impression yields an additional ~16.6 impressions " To get an additional 9.5 impressions need to pay 0.2 * 9.5/16.6 =.115 " $1.15 CPM

28 Robustness " Data are skewed: " Many people categorized a few s " Few people categorized a lot " Means drop as observations are excluded reflecting skew, annoyingness effect remains " Good ads and no ads yield same number of impressions

29 Accuracy " Enron data set has s categorized as spam or ham Treatment Bad Ads Good Ads No Ads Accuracy 90% ±1 92% ±1 93% ±1 " Statistically significant difference between bad ads and good ads/no ads

30 Study 2: The cognitive cost of annoying ads" "

31 Mousetracking Mousetracking correlates with eye tracking Chen, Anderson, & Sohn, 2001; Guo & Agichtein, 2010; Huang, White & Buscher, 2012 Mousetracking proxies for user interest Steiger & Reips, 2010; Willemsen & Johnson, 2010; Navalpakkam, & Churchill, 2012

32 Study 2 participants " Conducted on Amazon Mechanical Turk " $0.50 flat rate, $0.10 per question answered " 2,840 participants

33 Task Good or Bad Ad

34 Manipulation check 42% listed ad as annoying in bad condition 5% did so in good condition

35

36

37 Good ad fixations

38 Bad Ad Fixations

39 Mousetracking metrics Fixations of the mouse in an area Distance mouse travels in an area Number of entrances of the mouse into an area Time mouse spends in an area

40 Mousetracking metrics:" Ad area Measure Bad Ads Good Ads P-Val (log OLS) Fixations 4.45 (0.67) 1.57 (0.26) <.001 Distance (7.9) (8.0).003 Entrances 1.31 (0.05) 1.13 (0.05).004 Time (ms) 1873 (321) 1101 (186).001

41 Mousetracking metrics:" Text area Measure Bad Ads Good Ads P-Val (log OLS) Fixations Distance Entrances Time (ms) (9.07) (9.05).047 1,492 (55.20) 1,570 (66.60) (.06) 1.50 (.07) ,268 (1,207.00) 36,367 (1,085.00).596

42 Mouse reading Rated by 3 judges About 7% incidence of mouse reading in good and bad conditions, no significant difference

43 Accuracy Accuracy was significantly lower in bad condition compared to the good ad condition (2.5 percentage point delta, 6 percentage points for answer at bottom of page)

44 Summary of mousetracking studies Annoying ads do capture user attention Annoying ads have negative affect on accuracy However, absence of evidence that people read differently when annoying ads present. Why? Could be depletion, however, in Experiment 1, the accuracy difference between the bad and good ad conditions held constant. Our hypothesis is that annoying ads make people care less about what they re reading or the questions they are answering

45 " Conclusions & Future Work"

46 Conclusions " Animation has a causal effect on annoyance " Annoying ads hamper accuracy in cognitive tasks " Annoying ads cause dropout " Compensating differential of bad ads to no ads/good ads is at least $1 CPM, which would price some ads out of the market " Good ads lead to about the same number of impressions as no ads " We would lose money by running annoying ads

47 Conclusions " Mousetracking shows that people attend to annoying ads and they the impact accuracy " However, they don t appear to affect reading behavior and may impact accuracy for motivational reasons

48 Research Agenda " Towards a psychologically informed display ad pricing scheme " Display ads should be priced in terms of time in view instead of impressions (Economics and Computation, 2011) " Two short ads are better than one long ad (Economics and Computation, 2012) " What is the cost of annoying ads? (Journal of Marketing Research, 2014)

49 " Thank you!"

50

51

52