Read Me! Click Me! Innovations in Email Invitation Design for Today s Digital World Ted Saunders Manager - Digital Solutions MaritzCX Alexa Kessler Analyst - Quality, Loyalty & Brand Research Market Intelligence Nissan North America, Inc. February 11, 2015
Agenda 1 Background 2 Responsive Email Design 3 In-Invite Question Design 4 Study Design 5 Results and Conclusions 2
Background
Mobile Phone Usage Continues to Grow Across Web Studies Trend of MaritzCX Web Survey Starts by Device Type 4
Many Companies Have Established Mobile Survey Design Best Practices Programming Question Presentation Survey Length 5
There Is A Gap Between Our Mobile Survey Start Metrics & Email Industry Metrics 53% of total email opens occurred on a mobile phone or tablet in Q3 2014. This is an increase from the 48% percent seen in Q2 2014. 30% of consumers now read their email exclusively on mobile devices. -Experian Quarterly email benchmark report (Q3 2014) -Yesmail Email Compass: The mobile Effect (2013) By 2018 time, 80% of worldwide email users are expected to access their email accounts via a mobile device. -The Radicati Group Email Statistics Report, 2014-2018 Source: Litmus Email Analytics 6
Ignoring Mobile Email Users Can Be a Costly Mistake Source: BlueHornet 2013 Consumer Views of Email Marketing Report 7 7
Email Design Has Evolved Desktop-centric Mobile Aware Responsive 8
Responsive Email Design
Responsive Email Design This Some approach things that uses may media be different queries in to the determine mobile template screen size versus of device the desktop viewing template email are: If media query is supported by email application and a small screen (max width ~600px) is detected a mobile-specific format is rendered Font size, colors Amount of text and placement of call to action Images used / use of images Single column of text (versus two or more columns in some PC designs) 10 10
Marketers Have Tested Responsive Email Design Career Builder Tested responsive email design on a marketing message; changes included Streamlined design from thee columns to one Increased font Moved call to action up in the email These changes resulted in a 15%-17% increase in open rate and a 21%- 25% increase in click rates 11 11
Quick Start In-Invite Question
New Invitation Options: In-Invitation Question Currently referred to as Quick Start Involves inserting a simple question into the body of the email invite Important that the question require little cognitive effort and have few response options (ideally two) This approach is already being used on several studies with positive results 13
Original Email Invitation New Quick Start Invitation 14
Study Design
Study Design 2x2 design to test for effectiveness of responsive email design and inquestion surveys starts Test used over 10,000 email invitations randomly assigned to each of the four cells (over 2,500 invitations per cell) Cell One (control) Standard Invitation Format Standard Survey Start Cell Three Standard Invitation Format In-Invite Question Cell Two Responsive Invitation Design Standard Survey Start Cell Four Responsive Invitation Design In-Invite Question Analysis to compare click rates, unsubscribes, undeliverables, data comparability and data quality measures across cells 16
Survey Background Nissan defector study Mobile-aware email invitation design Mobile-optimized survey Quick Start Option required the first survey question to be re-written 17
Cell One (control) Standard Invitation Format Standard Survey Start 18
Cell Two Responsive Invitation Design Standard Survey Start 19
Outlook Threw Us a Curveball 20
Work-Around for Outlook 21
Cell Three Standard Invitation Format In-Invite Question 22
Cell Four Responsive Invitation Design In-Invite Question 23
Test Results
Click Rates (Unique Clicks / Initial Invitations Sent) * 30% 25% 20% Click Performance Relative To Control Standard Start / Responsive Design In- Question/Standard In-Question / Format Responsive Design Desktop -18% +37% -4% Mobile +29% +163% +138% OVERALL +6% +100% +67% 17.2% 15% 15.6% 10% 6.5% 8.4% 5% 6.6% 5.5% 9.1% 6.4% 0% Control Standard Start / Responsive Design In-Question/Standard Format In-Question / Responsive Design Mobile Click Rate Desktop Click Rate * Through three weeks of field; study is still open 25
Overall Survey Response Rate (Completed Surveys / Initial Invitations Sent) * 12% 10% 8% 6% Performance Relative To Control Standard Start / Responsive Design In- Question/Standard In-Question / Format Responsive Design Desktop -12% +25% -1% Mobile +64% +125% +135% OVERALL +19% +65% +54% 5.4% 5.6% 4% 2.4% 3.9% 2% 3.5% 3.1% 4.4% 3.5% 0% Control Standard Start / Responsive Design Survey Response Rate - Mobile Completes In-Question/Standard Format In-Question / Responsive Design Survey Response Rate - Desktop Completes * Through three weeks of field; study is still open 26
Unsubscribe / Undeliverable Summary 5% Unsubscribes By Blast 4% 3% 2% 1% 0% Control Standard Start / Responsive Design In-Question/Standard Format Initial Invitation Reminder #1 Reminder #2 In-Question / Responsive Design We also looked at undeliverables and saw no significant differences across the test cells 27
Data Comparability Only one question consistent across all survey versions No significant differences at 95% confidence level when compared mean scores to the overall satisfaction questions across all four tests Looked at consistency of answers from the invitation to the first question in the survey Only one response (less than 0.2% of completes) differed from the invitation to the survey 28
Data Quality Metrics Open End Response Rate and Mean Survey Length 94% 92% 90% 88% 86% 84% 82% 80% 8.4 8.2 8 7.8 7.6 7.4 7.2 7 6.8 78% Control Standard Start / Responsive Design In-Question/Standard Format In-Question / Responsive Design 6.6 Open-end Response Rate Mean Survey Length (minutes) 29
Conclusion / Next Steps
Responsive Design Conclusions Next Steps We tested limited aspects of responsive email design (font and text order) and experienced significant improvement in click rates and survey completes Much of the bump was tied to less text being presented in the message We continue to refine our approach and plan to continue testing Mobile respondents who started the survey through a responsive invitation were significantly more likely to complete the survey Standard Start / Responsive Design In- Question/Standard Format In-Question / Responsive Design Control Mobile Completes / Clicks 36.8% 46.7% 31.4% 36.2% Desktop Completes / Clicks 53.2% 57.0% 48.5% 54.9% Responsive email design involves significant labor (at least initially) and significant testing is required Budgeting for additional cost and time is recommended when implementing a responsive email design 31
In-Invite Question Conclusions Next Steps In-Invite questions leading to double the number of clicks and 65% more survey completes that the control group Much larger impact on mobile but improvements to mobile and desktop Mobile respondents who started the survey through the in-invite question were less likely to complete the survey, responded to open ends at a lower rate and spent less time taking that survey This leads us to believe that there may be a curiosity factor around this email design and we question if the increases we are seeing are sustainable Plan to monitor click and survey response rates on programs using this technique Currently recommending this approach for projects that need a nearterm bump in completes but caution against baking initial survey response rates from this technique into future analysis plans 32
APPENDIX
Useful Information for Designing & Testing Invitations 35
Email Client Categorizations Mobile Webmail Desktop Andriod 4.x (Native email client) AOL Mail (Chrome) Apple Mail AOL Mail (Android browser) AOL Mail (Firefox) Lotus Notes 8.5 AOL Mail (iphone browser) AOL Mail (IE) Outlook 2000-2003 BlackBerry OS7 Gmail (Chrome) Outlook 2007/2010/2013 BlackBerry Z10 Gmail (Firefox) Outlook 2011 (Outlook for Mac) Gmail (Andriod browser) Gmail (IE) Thunderbird Gmail (iphone browser) Outlok.com (Chrome) Gmail app (Android) Outlook.com (Firefox) Gmail app (ios) Outlook.com (IE) ios 6.x (Native email client) Yahoo! Mail (Chrome) Mailbox (ios) Yahoo! Mail (Firefox) Outlook.com (Android browser) Yahoo! Mail (IE) Outlook.com (iphone browser) Windows Mobile 7.5 Windows Phone 8 Yahoo! App (Andriod) Yahoo! App (ios) Yahoo! Mail (Andriod browser) Yahoo! Mail (iphone browser) 36
Thank You! Questions and Answers How to Contact Me: Ted Saunders Digital Solutions Manager 636.827.1492 Ted.Saunders@maritzcx.com Learn more here: www.maritzcx.com Copyright 2015, MaritzCX LLC