Definitions Optimization of online diect maketing effots Xavie Dèze Andé Bonfe Lucid Easily undestood; intelligible. Mentally sound; sane o ational. Tanslucent o tanspaent. Limpid Chaacteized by tanspaent cleaness; pellucid. Easily intelligible; clea: wites in a limpid style. Calm and untoubled; seene. Test : Two Email campaigns Raw Results Taget: Email : Males who had egisteed to sweepstakes. Email2: Males and Females who had egisteed to the sweepstakes 2. Content: Email : Sex and Mayhem Email 2: Powe and elationship Male Female Impession Email Email 2 42% 4% 28% Click Impession Email Email 2 20% 4% 8%
Test 2 Test 2: Raw Results Taget: Registants fom othe Video Releases. Registants fom the Movie website. Content: Video Video 2 Men Women Video Video 2 20%* 4% % 8% * Click-though ates on tageted emails Taditional DM Testing Guess a esponse ate Set a confidence level Set an acceptable magin of eo Use pobability tables to set test size Send test Wait, wait wait Use pobability tables to evaluate test Do the same Do bette Constained testing Dynamic testing Online DM testing 2
Constained testing Weekly newslettes, limited time fo: Witing Testing Sending Numbe of opens 400 200 000 800 600 Open time distibution 400 200 0 2 3 4 Days A model of constained testing M emails to send T peiods to send the whole campaign emails pe hou 2 possible emails with click pobabilities of p, p2 How many emails should be used fo testing puposes, how long should the test last? How long should the test last? If we use N emails fo testing puposes then we can use T peiods fo testing: M T = T N 3
Test size? How many opens (S) can we expect given N and T? What is the powe of a test with size S? How do we balance etun fom testing and poduction? Expected opens: S Simple case: Send is instantaneous Open time is Expo(λ) λx f( x) = λe λx F( x) = e S = E[ open N, x] = N e λx ( ) Send is not instantaneous Send ate is, fo the i th email we have: ( xi, / ) ( xi, / ) i λ ( x ) λ ( xi, / ) f( x,, i) = e I I =, x i/ I = 0, x< i/ i λ ( x ) F( x,, i) = e, if x> i/ Expected open: S S = E[ open N, x, ] = F( x,, i) N = e i= λ( x i/ ) i= e e Ne + Ne = e e λ / λ/ + λn/ λx λ/ + λx λn N λx λ/ ( ) ( e ) = N λ λ ( x ) e e 4
Powe of test of size S α p p 4 P( P) S 2 = φ Expected evenue fo the test (,,,, λ,, ) = { + ( )( α + ( α) ) ER M N T p p NP M N p p 4444244443 2 2 Testing Poduction We can then integate ove the distibution of p and p 2 : whee P is the aveage of p, p 2 and Φ is the CDF of the standad nomal distibution. ER( M, N, T,, λ) = ( M N) ( αp+ ( α) p2) + NP P( p = P) P( p2 = P2) dpdp 2 PP 2 Optimal Size: N * Optimal test: Numeical Solution All we need to do then is to find N * such that: N (,,,, λ ) Max ER M N T st..: M N 0 M T,000,000 800,000 600,000 400,000 200,000 - Poduction Size N* Slack 2 3 4 5 6 7 8 9 0 2 3 4 5 6 7 8 9 20 2 22 23 24 25 Hous Test Poduction 5
Dynamic testing Now that we have a solution to the constained testing situation, we can implement a dynamic one. As the sending occus, one can update pios on p, p 2, and λ. As p, p 2, and λ ae updated, one can update N and T. Implementation issues () We have ignoed the time it takes fo people to ead the email and click on the links. T o >> T c When solving the constained poblem, we can ignoe T c because T is in hous and T c in tens of seconds, but if we update as we go, seconds matte! Click Time: T c T c is Expo(γ) Ignoe send time We can model the time to open and click as BOXMOD: t t E[ Clicks at t, ] N. [ e λ γ λγ = λ γ+ γ λe ] λ γ P(open) P(click) Expected numbes of click t t E[ Click, ] N. [ e λ γ λγ = λ γ+ γ λe ]. P( Open). P( Click) λ γ 6
5 9 Implementation issues (2) The open times and click times look exponential fom afa, but not fom up close: Numbe of opens 80 60 40 20 00 80 60 Titus Time to open/click is Log-Nomal In the shot un, log-nomal and expo ae vey diffeent. Thee is no closed-fom expession of the CDF fo the log-nomal. We have to solve numeically o maybe use a log-logistic appoximation. 40 20 0 3 7 2 25 29 33 37 4 45 49 53 57 6 Hous Log-Nomal Solution Still to do,000,000 800,000 600,000 400,000 Test Poduction Poduction Size N* Slack N*LN Slack Solve case fo log-logistic Solve updating fomulas fo P(open), P(Click), λ, and γ Simulate test using past data Simulate live test Live test 200,000-2 3 4 5 6 7 8 9 0 2 3 4 5 6 7 8 9 20 2 22 23 24 25 Hous 7
Enhancement Stopping ule fo failed tests 8