Analyzing the Customer Journey: Attribution Modeling for Online Marketing Exposures in a Multi- Channel Setting

Transcription

1 Analyzing the Customer Journey: Attribution Modeling for Marketing Exposures in a Multi- Channel Setting Conference Innovative Approaches to Advertising Effectiveness Eva Anderl, Ingo Becker, Florian v. Wangenheim, Jan H. Schumann, & Franz Graf May 16, 2013 Sponsored by the Wharton Customer Analytics Initiative and the German Federal Ministry of Education and Research (FKZ 01FL10038)

2 Attribution Modeling 1 How can advertisers evaluate online channel contribution? Customer journey Conversion Marketers employ various online channels such as SEA or Display in their promotional mix Little is known on how to attribute credit to exposures along the "customer journey" Today, marketers often rely on simple heuristics like "last click wins" (SEO) Display Newsletter (SEA) Social Networks Blogs Forums shop 1 Group Buying Portals Price comparison Sites Blogs Micro media (SEA) Video Portals Review auctions (SEO) shop 2 (SEO) Affiliate Price comparison Sites Social Networks Display Newsletter shop 1 Micro media Advertisers questions What are the criteria for a good attribution system? Which methodology meets these criteria and ascertains the correct value contribution? How should marketing budgets be optimally allocated? Our contribution Comprehensive attribution framework based on first- and higher-order Markov graphs Catalogue of evaluation criteria for attribution modeling Implementation and practical impact in a real life system 1) The measurement of the partial value of each interactive marketing contact that contributed to a desired outcome (Osur, 2012) 1

3 1 Approach Graph-based Markov framework and "removal effect" to calculate attribution Basic idea I Markov graphs aggregate user journeys Journeys as Markov walks in directed graphs 1 States correspond to channels Edge weights represent transition probabilities II Ad factors capture structural properties of nodes Removal effect used to calculate attribution III Model extensions Higher-order Markov models (implemented up to fourth-order) Inclusion of position in journey in state definitions START 66% C1 NULL START 66% C1 66% NULL Evaluation of 16 model types 33% C2 C3 CONV 100% 33% C2 C3 CONV 100% A (first-order) Markov model M= S,W is defined by A set of states S= {s 1,, s n } and A transition matrix W with edge weights W ij = P (X t = S j X t-1 = S i ), 0 W ij 1, Σ W ij = 1 i N j = 1 A 1) Idea based on an approach by Archak et al. (2010) used in Engine Marketing 2

4 2 Evaluation We evaluate our attribution framework along a catalogue of five criteria and compare it to existing models A Fairness Description Credit attribution in accordance with factual channel contribution Evaluation methodology Evaluation of 4 real-life clickstream datasets, each containing at least 400,000 journeys Area under the ROC 1 curve as measure of predictive performance B Accuracy Correct prediction of conversion events 10-fold cross-validation Within- and out-of-sample evaluation Example: Within-sample validation C Stability Reproducibility of results True positive rate D Algorithmic efficiency Efficient handling of extensive data sets and real-time applications st order, simple 2 nd order, simple E Interpretability Understandability and transferability of results into managerial decisions rd order, simple 4 th order, simple False positive rate 1) Receiver operating characteristic 3

5 3 Impact Simple heuristics are replaced by a data-driven Markov model improving measurement of the real ability of channels to affect conversions Value contribution by channel 1 Percent Simple heuristic last click vs. novel Markov framework Last click wins Markov model (1st order, simple) retail apparel 1 retail luggage / equipment 2 Relative change, percent High contribution Type In % % SEA % % SEO % % Newsletter % Affiliate % % Display % Referrer Price 0.1 Comparison % 65% % 43% Low contribution Retargeting % 1) 206,519 click journeys; min. journey length: 2; avg. journey length: 4.48 (7.73); journey conversion rate: ) 105,031 click journeys; min. journey length: 2; avg. journey length: 3.00 (8.85); journey conversion rate:

6 4 Contribution Significant impact on theory and practice Markov attribution modeling framework Real output Markov graph of simple model Start States/channels Comprehensive set of evaluation criteria for good attribution Novel scientifically evaluated Markovian graph-based attribution framework Implemented and applied in practice at intelliad Applicable across industries and customizable in accordance with individual requirements Data-driven guidance for substantiated managerial and budgetary decisions Conversion Enabling additional applications such as real-time bidding (RTB) Null 5

7 Thank you for your attention! 6