Additional Case Studies: The Strategic Formula for Success

Transcription

1 Track: Marketing #CNX14 #CNX14 Additional Case Studies: The Strategic Formula for Success Jon Powell, Sr. Manager, Executive Research and Development

2 Track: Marketing #CNX14 Relevance (rv)

3 Case Study Relevance Track: Marketing #CNX14 eme = rv(of+i) (f+a) Sent to: General house promo list From This Looking for a lower mortgage rate? You re not alone. To This Own your home sooner with free biweekly payments 16% Increase in opens 11.3% Increase in clicks

4 Track: Marketing #CNX14 Experiment ID: (Protected) Location: MarketingExperiments Research Library Test Protocol Number: TP2121 eme = rv(of+i) (f+a) Background: National B2C, B2B financial company with an online and retail presence Goal: To increase account activity, engagement Primary research question: Which subject line will generate the most opens? Approach: A/B/C/D split test (variable cluster)

5 Track: Marketing #CNX14 Version A [Name], Your Account Information is Ready To View Version B It's easy to access your [Bank Name] Accounts Online. Sign On Now Version C Your Accounts are Protected with [Bank Name] Online. Sign On Now eme = rv(of+i) (f+a) Which of these clearly connect to a recent event in the story of the customer? Version D Did you forget your [Bank Name] Online username or password?

6 Track: Marketing #CNX % Relative Increase in Open Rate Subject lines A and D significantly outperformed those that did not connect to an event # Subject Line Open Rate Rel. Diff. A [Name], Your Account Information is Ready To View 30.67% 92.2% D Did you forget your [Bank Name] Online username or password? 24.14% 51.3% C Your Accounts are Protected with [Bank Name] Online. Sign On Now 20.33% 27.4% B It's easy to access your [Bank Name] Accounts Online. Sign On Now 15.96% -

7 Experiment eme = rv(of+i) (f+a) Experiment ID: Pier 1 Imports Location: MarketingSherpa Research Library Test Protocol Number: A-TP1002 Background: B2C home products company with a significant online and retail presence Goal: To increase the click-through rate from the to the landing page Primary research question: Which design will generate the most clicks? Approach: A/B/C/D split test (variable cluster)

8 Experiment Sample eme = rv(of+i) (f+a)

9 Team A

10 Team B

11 Team C

12 Experiment eme = rv(of+i) (f+a) Team A Team B Team C Sample

13 18-52% Decrease in Click-through Each design achieved a statistically significant click-rate decrease Designs Click Rate Rel. Diff. Sample (client) 36.70% - Team A Design 17.68% % Team B Design 29.91% % Team C Design 24.07% %

14 Experiment Click-through rate comparison eme = rv(of+i) (f+a) Team A Team C Team B Client 17.68% 24.07% 29.91% 36.70% See a progression?

15 Track: Marketing #CNX14 Offer (of)

16 Experiment: Background eme = rv(of+i) (f+a) Experiment ID: (protected) Location: MarketingExperiments Research Library Test Protocol Number: TP2063 Background: A financial organization offering a quarterly newsletter for business leaders. These s often feature a single whitepaper download. Goal: To generate more whitepaper downloads Primary research question: Which design will produce the most downloads? Approach: A/B split test (variable cluster)

17 Experiment: Control LOGO eme = rv(of+i) (f+a) The original version of the provided an overview of the whitepaper download similar to what might be best practices for a landing page offer (strong headline, intro paragraph, bullet point, single CTA, etc.).

18 Experiment: Treatment LOGO eme = rv(of+i) (f+a) Our researchers wanted to test peaking the recipients interest by showing part of the article (title and first three paragraphs) that they could download. The hypothesis was that this would increase the value of the click.

19 Experiment: Results eme = rv(of+i) (f+a) 85% Increase in Downloads Treatment 1 increased the rate of clickthrough rate by 29% Test Designs CTR Relative Difference Download Rate Relative Difference Control 8.4% % - Treatment % 28.6% 33.6% 84.6% What you need to understand: By focusing on piquing interest just enough to get a click in the , not only did we generate more response from recipients, but we also generated more white paper downloads on the landing page.

20 Experiment eme = rv(of+i) (f+a) Experiment ID: (Protected) Location: MarketingExperiments Research Library Test Protocol Number: TP2801 Background: A physician-only social network that allows medical product companies to conduct first-hand research on potential purchasers. Goal: To increase the number of leads from a rented trade publication list. Primary research question: Which design will generate the most opens, clicks? Approach: Multi-factorial sequential test on a trade publication list

21 Experiment PROTECTED Control eme = rv(of+i) (f+a) 30-minute demo Get started with a free This assumes that readers understand the platform s audience and thus focuses on platform uses Get started with a free 30-minute demo It also assumes intent to purchase with multiple Get Started calls to action, one as early as the second sentence

22 Comparatively, this focuses less on platform and more on who s using it to answer questions of feasibility in the mind of the reader The primary call-to-action was written to synchronize with their state of interest PROTECTED Treatment

23 Experiment Control Treatment eme = rv(of+i) (f+a)

24 104% Increase in clicks Version B achieved a statistically significant increase in click-rate Click Rate Rel. Diff. Control 1.55% -- Treatment 3.16% 104%

25 Offer Test Protocol ID: TP2076 eme = rv(of+i) (f+a) Not this Target MDs with Surveys and Panels But this Reach 120,000 Physicians through Social Media 27.6% Increase in clicks

26 Track: Marketing #CNX14 Friction (f)

27 Friction For Length. eme = rv(of+i) (f+a) Not this But this

28 Friction For Difficulty. Not this eme = rv(of+i) (f+a) But this

29 Friction Test Protocol ID: TP2089 eme = rv(of+i) (f+a) But This Not this 21.2% Increase in clicks

30 Friction Test Protocol ID: TP2081 Not this eme = rv(of+i) (f+a) But This 41.4% Increase in clicks 52.5% Increase in app starts

31 Friction Test Protocol ID: TP2093 Not this But This eme = rv(of+i) (f+a) 10% Increase in clicks

32 Case Study From This To This Track: Marketing #CNX14 eme = rv(of+i) (f+a) 53% In Clicki rate

33 Case Study Difficulty Track: Marketing #CNX14 eme = rv(of+i) (f+a) From This

34 Case Study Difficulty Track: Marketing #CNX14 eme = rv(of+i) (f+a) To This 18% In Unsubscribe rate

35 Track: Marketing #CNX14 Anxiety (a)

36 Case Study Track: Marketing #CNX14 eme = rv(of+i) (f+a) From This

37 Case Study Track: Marketing #CNX14 eme = rv(of+i) (f+a) To This 45% In Unsubscribe Rate

38 Experiment: Background eme = rv(of+i) (f+a) Experiment ID: (protected) Location: MarketingExperiments Research Library Test Protocol Number: TP2002 Background: A MarketingExperiments sent out two s promoting their Landing Page Optimization course. Goal: To generate more traffic to our course product page Primary research question: Which copy approach result the most clicks? Approach: A/B split test (variable cluster)

39 Experiment: Treatment 1 Subject line: MarketingExperiments An increase of 541% Take the MarketingExperiments Landing Page Optimization Certification Course Dear MarketingExperiments Subscriber, Our Landing Page Optimization Certification Course for subscription sites begins on April 26, Consistent academic undertone Subject line Headline eme = rv(of+i) (f+a) Body Copy and bullets Passing this test will impact your business and personal career in two important ways: 1. The study and application of all that you learn, including our unique Conversion Index, will help you increase revenues for your business almost immediately. 2. When you pass the course, you will receive a Certification Certificate which you can add to your résumé and use to advance

40 Experiment: Treatment 2 Subject line: MarketingExperiments A Porsche or a Corolla? Take the MarketingExperiments Landing Page Optimization Certification Course... and change gear for ever. Dear MarketingExperiments Subscriber, Ask yourself a question. More sales-like, descriptive undertone Subject line Headline eme = rv(of+i) (f+a) Body Copy Think of one of your company s key Web site landing pages and ask yourself whether it s more like a Porsche 911 or a Toyota Corolla. Is it a high-performance page, fully optimized to maximize conversion rates? Or is it a Corolla... to which you have been adding cup holders and new upholstery in the hope that these little tweaks will enhance its

41 Experiment: Results eme = rv(of+i) (f+a) 29% Increase in CTR Resulting in twice the number of subscribers Messages CTR (%) 1 Academic 1.75% 2 Sales 1.35% Relative Difference 29.3% What you need to understand: By focusing on a more familiar academic tone, we were able to achieve twice the number of subscribers

42 Experiment eme = rv(of+i) (f+a) Experiment ID: (Protected) Location: MarketingExperiments Research Library Test Protocol Number: TP2083 Background: A large event management online software provider. Goal: To increase total lead inquiries (phone calls + form sign-ups) from visitors who abandoned the free trial sign-up process. Primary research question: Which tone will result in a higher rate of lead inquiries? Approach: A/B single factorial split

43 Experiment eme = rv(of+i) (f+a) Subject Line: Your Free RegOnline Access Dear [Name], Control You re just one step away from getting FREE access to RegOnline, our award winning Event Registration and Management Software. Quickly make an event website, try our event marketing tools, build a registration form template or even generate custom name badges. Click here to finish your profile and get started. Your personal profile is kept secure and we promise NEVER to sell or misuse your information. The control uses typical sales tone of a directresponse . Have questions? Call us direct at XXX-XXXX -- we can even set up your free access over the phone. Sincerely, Jon Powell Customer Service Representative RegOnline Direct: XXX-XXXX ext. XXXX

44 [Name of Representative] Customer Service Representative Experiment The treatment directly addresses the anxiety of completing the process. It takes on a categorical shift in tone from a directresponse tone to a customer service tone. Subject Line: Your Free RegOnline Access Hi [First], eme = rv(of+i) (f+a) I noticed that you started the process of getting free access to RegOnline but weren t able to finish. Are you concerned about giving out your phone number? Are you worried about high pressure sales tactics or mandatory contracts? We believe our product sells itself, so we re just here to provide you with whatever assistance you need in getting your event up and running - in whatever way works best for you. We promise NEVER to sell or misuse your information. Call me direct at XXX-XXXX and I can help get you rolling. If you d rather just try again online, use this link instead. Thank you in advance for your trust! Sincerely, Treatment

45 Experiment Control eme = rv(of+i) (f+a) Treatment Direct Response Tone Customer Service Tone

46 349% Increase in lead inquiry rate The service tone achieved a statistically significant increase in inquiries Inquiry Rate Rel. Diff. Control 1.58% -- Treatment 7.08% %

47 Track: Marketing #CNX14 Multiple Elements

48 Experiment eme = rv(of+i) (f+a) Partner ID: The New York Times Location: MarketingExperiments Research Library Test Protocol Number: TP2802 Background: Internationally recognized news service known for their journalism. Our Goal: To significantly increase subscriptions. Primary research question: Which will generate the greatest subscription rate? Approach: A/B split test (variable cluster)

49 Experiment eme = rv(of+i) (f+a) The Subscription Product Offering A subscription to the NY Times Home delivery of the Print edition Unlimited, unrestricted access to articles online, in their smartphone and tablet apps Free digital access for one family member or friend Subscriber only special offers, discounts, invitations Incentive 50% off for the first 12 weeks

50 The List

51 THE CONTROL Subject Line: Open this now for Special Savings

52

53

54

55

56

57

58

59

60

61

62

63

64

65 Experiment BEFORE AFTER eme = rv(of+i) (f+a)

66 Experiment BEFORE AFTER 62% Increase in Conversions The treatment increased the subscription conversion rate by 62.1% eme = rv(of+i) (f+a) + Subject Line Open Rate % Rel. Change Click Rate % Rel. Change Conv. Rate % Rel. Change Control 11.90% % % -- Treatment 13.70% 15.5% 0.52% 50.7%.008% 62.1% 99% 99% 96%

67 Experiment: Background Experiment ID: (protected) Location: MarketingExperiments Research Library Test Protocol Number: TP2061 Background: An established financial institution offering online savings accounts Goal: To increase click-through from an message to the landing page Primary research question: Which design will produce the most clickthrough? Approach: A/B split test (variable cluster)

68 Experiment: Control Note the following things about the control: Utilizes graphical imagery throughout Provides multiple calls-to-action Follows a typical clean landing page format Copy is difficult to read in some places

69 Experiment: Treatment 1 Note the following things about Treatment 1: Not many graphical elements Relies on copy-rich messaging Has a personal letter-style feel Links are embedded within the copy

70 Experiment: Treatment 2 Note the following things about Treatment 2: Utilizes graphical header to catch visitors eyes Organizes the content in a clean, easy-toscan format Displays security guarantee seal Single clear call-to-action

71 Experiment: Treatment 3 Note the following things about Treatment 3: Similar to Treatment 2 However, does not utilize graphical header Organizes the content in a clean, easy-toscan format Displays security guarantee seal Single clear call-to-action

72 Experiment: Side by Side Control Treatment 1 Treatment 2 Treatment 3

73 Experiment: Results 42% Increase in Clickthrough Treatment 1 (copy rich) outperformed the control by 42.34% CTR Rel. Diff. C Control 0.51% T1 Copy-rich 0.73% 42.34% T2 Single CTA (w/ header) 0.63% 24.12% T3 Single CA (w/o header) 0.65% 26.27%

74 Track: Marketing #CNX14