Track: Email Marketing #CNX14 #CNX14 Additional Case Studies: The Strategic Formula for Email Success Jon Powell, Sr. Manager, Executive Research and Development MECLABS @JonPowell31
Track: Email Marketing #CNX14 Relevance (rv)
Case Study Relevance Track: Email Marketing #CNX14 eme = rv(of+i) (f+a) Sent to: General house promo list From This Looking for a lower mortgage rate? You re not alone. To This Own your home sooner with free biweekly payments 16% Increase in opens 11.3% Increase in clicks
Track: Email Marketing #CNX14 Experiment ID: (Protected) Location: MarketingExperiments Research Library Test Protocol Number: TP2121 eme = rv(of+i) (f+a) Background: National B2C, B2B financial company with an online and retail presence Goal: To increase account activity, engagement Primary research question: Which subject line will generate the most opens? Approach: A/B/C/D split test (variable cluster)
Track: Email Marketing #CNX14 Version A [Name], Your Account Information is Ready To View Version B It's easy to access your [Bank Name] Accounts Online. Sign On Now Version C Your Accounts are Protected with [Bank Name] Online. Sign On Now eme = rv(of+i) (f+a) Which of these clearly connect to a recent event in the story of the customer? Version D Did you forget your [Bank Name] Online username or password?
Track: Email Marketing #CNX14 92.2% Relative Increase in Open Rate Subject lines A and D significantly outperformed those that did not connect to an event # Subject Line Open Rate Rel. Diff. A [Name], Your Account Information is Ready To View 30.67% 92.2% D Did you forget your [Bank Name] Online username or password? 24.14% 51.3% C Your Accounts are Protected with [Bank Name] Online. Sign On Now 20.33% 27.4% B It's easy to access your [Bank Name] Accounts Online. Sign On Now 15.96% -
Experiment eme = rv(of+i) (f+a) Experiment ID: Pier 1 Imports Location: MarketingSherpa Research Library Test Protocol Number: A-TP1002 Background: B2C home products company with a significant online and retail presence Goal: To increase the click-through rate from the email to the landing page Primary research question: Which email design will generate the most clicks? Approach: A/B/C/D split test (variable cluster)
Experiment Sample eme = rv(of+i) (f+a)
Team A
Team B
Team C
Experiment eme = rv(of+i) (f+a) Team A Team B Team C Sample
18-52% Decrease in Click-through Each design achieved a statistically significant click-rate decrease Email Designs Click Rate Rel. Diff. Sample (client) 36.70% - Team A Design 17.68% -51.83% Team B Design 29.91% -18.50% Team C Design 24.07% -34.41%
Experiment Click-through rate comparison eme = rv(of+i) (f+a) Team A Team C Team B Client 17.68% 24.07% 29.91% 36.70% See a progression?
Track: Email Marketing #CNX14 Offer (of)
Experiment: Background eme = rv(of+i) (f+a) Experiment ID: (protected) Location: MarketingExperiments Research Library Test Protocol Number: TP2063 Background: A financial organization offering a quarterly newsletter for business leaders. These emails often feature a single whitepaper download. Goal: To generate more whitepaper downloads Primary research question: Which email design will produce the most downloads? Approach: A/B email split test (variable cluster)
Experiment: Control LOGO eme = rv(of+i) (f+a) The original version of the email provided an overview of the whitepaper download similar to what might be best practices for a landing page offer (strong headline, intro paragraph, bullet point, single CTA, etc.).
Experiment: Treatment LOGO eme = rv(of+i) (f+a) Our researchers wanted to test peaking the recipients interest by showing part of the article (title and first three paragraphs) that they could download. The hypothesis was that this would increase the value of the click.
Experiment: Results eme = rv(of+i) (f+a) 85% Increase in Downloads Treatment 1 increased the rate of clickthrough rate by 29% Test Designs Email CTR Relative Difference Download Rate Relative Difference Control 8.4% - 18.2% - Treatment 1 10.8% 28.6% 33.6% 84.6% What you need to understand: By focusing on piquing interest just enough to get a click in the email, not only did we generate more response from email recipients, but we also generated more white paper downloads on the landing page.
Experiment eme = rv(of+i) (f+a) Experiment ID: (Protected) Location: MarketingExperiments Research Library Test Protocol Number: TP2801 Background: A physician-only social network that allows medical product companies to conduct first-hand research on potential purchasers. Goal: To increase the number of leads from a rented trade publication list. Primary research question: Which email design will generate the most opens, clicks? Approach: Multi-factorial sequential test on a trade publication list
Experiment PROTECTED Control eme = rv(of+i) (f+a) 30-minute demo Get started with a free This email assumes that readers understand the platform s audience and thus focuses on platform uses Get started with a free 30-minute demo It also assumes intent to purchase with multiple Get Started calls to action, one as early as the second sentence
Comparatively, this email focuses less on platform and more on who s using it to answer questions of feasibility in the mind of the reader The primary call-to-action was written to synchronize with their state of interest PROTECTED Treatment
Experiment Control Treatment eme = rv(of+i) (f+a)
104% Increase in clicks Version B achieved a statistically significant increase in click-rate Email Click Rate Rel. Diff. Control 1.55% -- Treatment 3.16% 104%
Offer Test Protocol ID: TP2076 eme = rv(of+i) (f+a) Not this Target MDs with Surveys and Panels But this Reach 120,000 Physicians through Social Media 27.6% Increase in clicks
Track: Email Marketing #CNX14 Friction (f)
Friction For Length. eme = rv(of+i) (f+a) Not this But this
Friction For Difficulty. Not this eme = rv(of+i) (f+a) But this
Friction Test Protocol ID: TP2089 eme = rv(of+i) (f+a) But This Not this 21.2% Increase in clicks
Friction Test Protocol ID: TP2081 Not this eme = rv(of+i) (f+a) But This 41.4% Increase in clicks 52.5% Increase in app starts
Friction Test Protocol ID: TP2093 Not this But This eme = rv(of+i) (f+a) 10% Increase in clicks
Case Study From This To This Track: Email Marketing #CNX14 eme = rv(of+i) (f+a) 53% In Clicki rate
Case Study Difficulty Track: Email Marketing #CNX14 eme = rv(of+i) (f+a) From This
Case Study Difficulty Track: Email Marketing #CNX14 eme = rv(of+i) (f+a) To This 18% In Unsubscribe rate
Track: Email Marketing #CNX14 Anxiety (a)
Case Study Track: Email Marketing #CNX14 eme = rv(of+i) (f+a) From This
Case Study Track: Email Marketing #CNX14 eme = rv(of+i) (f+a) To This 45% In Unsubscribe Rate
Experiment: Background eme = rv(of+i) (f+a) Experiment ID: (protected) Location: MarketingExperiments Research Library Test Protocol Number: TP2002 Background: A MarketingExperiments sent out two emails promoting their Landing Page Optimization course. Goal: To generate more traffic to our course product page Primary research question: Which email copy approach result the most clicks? Approach: A/B email split test (variable cluster)
Experiment: Treatment 1 Subject line: MarketingExperiments An increase of 541% ------------------------------------------------------ Take the MarketingExperiments Landing Page Optimization Certification Course Dear MarketingExperiments Subscriber, Our Landing Page Optimization Certification Course for subscription sites begins on April 26, 2007. Consistent academic undertone Subject line Headline eme = rv(of+i) (f+a) Body Copy and bullets Passing this test will impact your business and personal career in two important ways: 1. The study and application of all that you learn, including our unique Conversion Index, will help you increase revenues for your business almost immediately. 2. When you pass the course, you will receive a Certification Certificate which you can add to your résumé and use to advance
Experiment: Treatment 2 Subject line: MarketingExperiments A Porsche or a Corolla? ------------------------------------------------------ Take the MarketingExperiments Landing Page Optimization Certification Course... and change gear for ever. Dear MarketingExperiments Subscriber, Ask yourself a question. More sales-like, descriptive undertone Subject line Headline eme = rv(of+i) (f+a) Body Copy Think of one of your company s key Web site landing pages and ask yourself whether it s more like a Porsche 911 or a Toyota Corolla. Is it a high-performance page, fully optimized to maximize conversion rates? Or is it a Corolla... to which you have been adding cup holders and new upholstery in the hope that these little tweaks will enhance its
Experiment: Results eme = rv(of+i) (f+a) 29% Increase in CTR Resulting in twice the number of subscribers Email Messages CTR (%) Email 1 Academic 1.75% Email 2 Sales 1.35% Relative Difference 29.3% What you need to understand: By focusing on a more familiar academic tone, we were able to achieve twice the number of subscribers
Experiment eme = rv(of+i) (f+a) Experiment ID: (Protected) Location: MarketingExperiments Research Library Test Protocol Number: TP2083 Background: A large event management online software provider. Goal: To increase total lead inquiries (phone calls + form sign-ups) from visitors who abandoned the free trial sign-up process. Primary research question: Which email tone will result in a higher rate of lead inquiries? Approach: A/B single factorial split
Experiment eme = rv(of+i) (f+a) Subject Line: Your Free RegOnline Access Dear [Name], Control You re just one step away from getting FREE access to RegOnline, our award winning Event Registration and Management Software. Quickly make an event website, try our event marketing tools, build a registration form template or even generate custom name badges. Click here to finish your profile and get started. Your personal profile is kept secure and we promise NEVER to sell or misuse your information. The control email uses typical sales tone of a directresponse email. Have questions? Call us direct at 1-800-XXX-XXXX -- we can even set up your free access over the phone. Sincerely, Jon Powell Customer Service Representative RegOnline Direct: 1-800-XXX-XXXX ext. XXXX
[Name of Representative] Customer Service Representative Experiment The treatment directly addresses the anxiety of completing the process. It takes on a categorical shift in tone from a directresponse tone to a customer service tone. Subject Line: Your Free RegOnline Access Hi [First], eme = rv(of+i) (f+a) I noticed that you started the process of getting free access to RegOnline but weren t able to finish. Are you concerned about giving out your phone number? Are you worried about high pressure sales tactics or mandatory contracts? We believe our product sells itself, so we re just here to provide you with whatever assistance you need in getting your event up and running - in whatever way works best for you. We promise NEVER to sell or misuse your information. Call me direct at 1-800-XXX-XXXX and I can help get you rolling. If you d rather just try again online, use this link instead. Thank you in advance for your trust! Sincerely, Treatment
Experiment Control eme = rv(of+i) (f+a) Treatment Direct Response Tone Customer Service Tone
349% Increase in lead inquiry rate The service tone achieved a statistically significant increase in inquiries Email Inquiry Rate Rel. Diff. Control 1.58% -- Treatment 7.08% 349.41%
Track: Email Marketing #CNX14 Multiple Elements
Experiment eme = rv(of+i) (f+a) Partner ID: The New York Times Location: MarketingExperiments Research Library Test Protocol Number: TP2802 Background: Internationally recognized news service known for their journalism. Our Goal: To significantly increase subscriptions. Primary research question: Which email will generate the greatest subscription rate? Approach: A/B split test (variable cluster)
Experiment eme = rv(of+i) (f+a) The Subscription Product Offering A subscription to the NY Times Home delivery of the Print edition Unlimited, unrestricted access to articles online, in their smartphone and tablet apps Free digital access for one family member or friend Subscriber only special offers, discounts, invitations Incentive 50% off for the first 12 weeks
The List
THE CONTROL Subject Line: Open this now for Special Savings
Experiment BEFORE AFTER eme = rv(of+i) (f+a)
Experiment BEFORE AFTER 62% Increase in Conversions The treatment increased the subscription conversion rate by 62.1% eme = rv(of+i) (f+a) Email + Subject Line Open Rate % Rel. Change Click Rate % Rel. Change Conv. Rate % Rel. Change Control 11.90% -- 0.35% --.005% -- Treatment 13.70% 15.5% 0.52% 50.7%.008% 62.1% 99% 99% 96%
Experiment: Background Experiment ID: (protected) Location: MarketingExperiments Research Library Test Protocol Number: TP2061 Background: An established financial institution offering online savings accounts Goal: To increase click-through from an email message to the landing page Primary research question: Which email design will produce the most clickthrough? Approach: A/B email split test (variable cluster)
Experiment: Control Note the following things about the control: Utilizes graphical imagery throughout email Provides multiple calls-to-action Follows a typical clean landing page format Copy is difficult to read in some places
Experiment: Treatment 1 Note the following things about Treatment 1: Not many graphical elements Relies on copy-rich messaging Has a personal letter-style feel Links are embedded within the copy
Experiment: Treatment 2 Note the following things about Treatment 2: Utilizes graphical header to catch visitors eyes Organizes the content in a clean, easy-toscan format Displays security guarantee seal Single clear call-to-action
Experiment: Treatment 3 Note the following things about Treatment 3: Similar to Treatment 2 However, does not utilize graphical header Organizes the content in a clean, easy-toscan format Displays security guarantee seal Single clear call-to-action
Experiment: Side by Side Control Treatment 1 Treatment 2 Treatment 3
Experiment: Results 42% Increase in Clickthrough Treatment 1 (copy rich) outperformed the control by 42.34% Email Email CTR Rel. Diff. C Control email 0.51% T1 Copy-rich email 0.73% 42.34% T2 Single CTA (w/ header) 0.63% 24.12% T3 Single CA (w/o header) 0.65% 26.27%
Track: Email Marketing #CNX14