Winning Marketing Offers A case study on four direct mail pieces
Summary What you ll learn in this guide We ll show you how to use surveys to determine which of your marketing mailers will win before you go to print. Our guide includes: Tips for getting key performance metrics. Best practices on writing marketing offer testing surveys. A real-world example of how our direct mail offer testing framework can be applied to your brand. Case study key findings We compared four direct mail pieces against each other and found: oo The target market for these mailers can be segmented into two distinct groups: early adopters and those who wait to purchase new products. oo The differences in attitude and behavior between early adopters and later-stage adopters are significant. oo The data suggest that later-stage adopters may need an incentive to act and a reminder that a deadline looms. oo Wells Fargo s offer with a clear deadline and It s time to switch messaging scored higher overall. Created by and for the use of SurveyMonkey Audience for marketing purposes only. This study was not commissioned by American Express, Citi, Discover, Wells Fargo, Visa, MasterCard, or any other companies mentioned in the study. 2
Table of contents 5 Marketing Offer Testing Overview 9 Real-World Direct Mail Offer Testing Examples 13 Market Segmentation 3
Methodology What? How? When? SurveyMonkey Audience fielded a Direct Mail Offer Test study on four direct mail pieces to demonstrate how to successfully test marketing offers. The survey was fielded using SurveyMonkey Audience. Respondents represented the general U.S. population. A total sample size of 441 was surveyed. We fielded the survey from 5/11/15 5/12/15. 4 4
Marketing Offer Testing Overview 5
5 benefits of testing offers with surveys Test various offers quickly and easily Filter out the ineffective offers Refine your good ideas Test with diverse audiences Save money A survey allows you to present a wide range of offers to your target market and get qualitative feedback fast. Investigate which offers work and which fall flat by asking your target market directly. If you test a bunch of offers through a survey, you can quickly see which ones are rising to the top. Use surveys, rather than focus groups, for a scalable way to get responses from different segments in your target market. Traditional testing methods like focus groups cost a lot of time and money. Surveys are a faster, easier way to choose a winning offer. 6
How to determine marketing offer success The quickest way to determine a winning offer is to use a standard scoring model. This is an example scorecard that incorporates several key success metrics depending on the type of offer you re testing and how you define a successful offer. Top 2 Box: The percentage of respondents who answered the two most positive answer options A B Attributes Tested Define your key success metrics. Set the number of metrics depending on the offer you re testing and how you define success. 1 2 Purchase Intent Statistical Significance Appeal Statistical Significance Uniqueness Statistical Significance Relevance Statistical Significance % % % % (A) % % % % Average Score % % *Statistical Significance at 95% confidence interval 5 Created by SurveyMonkey Audience Determine which offers performed better by using stat testing, a calculation that shows if two numbers or scores have a significant, or big enough, difference. This shows if one offer truly resonates with consumers more than the others. An (A) in the (B) column here signifies that Concept B is both statistically significant and scored higher than Concept A. 3 4 Use the top 2 answer choices for each question to more easily compare how each offer performed. Compare the average scores of each offer to determine which one performed best based on the success metrics you chose. 7
Key success metrics Here are several key metrics you can incorporate into an offer test: Willingness to Open Whether your offer motivates respondents to open the envelope Appeal The overall appeal of your offer is a strong indicator of Willingness to Open Uniqueness How different one offer is from the others Perception of Quality If your offer is perceived to represent a high quality brand/product Message Clarity How clearly your offer messaging is understood by respondents Believability How trustworthy your offer is perceived to be 8
Direct Mail Offer Test 9
Marketing offer test summary Background We used SurveyMonkey Audience to test the effectiveness of four direct mail offerings. The direct mail pieces appear on the following page. Goal Put four current direct mail pieces head to head to see which one performs better. Key insights We found that Wells Fargo s offer with a clear deadline and It s time to switch messaging scored higher overall. When we segmented the market into two distinct groups to see which offer resonated best with the target market, we found that later-stage adopters may need more of an incentive to act and a reminder that a deadline looms. 10
The 4 offers Offer 1: American Express Invitation with membership points and $0 annual fee Offer 2: Citi Pre-selected 0% APR and savings Front Back Front Back Offer 3: Discover Offer 4: Wells Fargo No annual fee, cash back, and 0% APR It s time to act, 0% APR, and $0 annual fee Front Back Front Back 11
Direct mail offer testing scorecard We found that the Wells Fargo offer with a combination of It s time to act messaging, 0% APR, and $0 annual fee focus scored the highest in most key metrics: Offer Test Scorecard (Top 2 Box Scores) Survey Results as of: June 12, 2015 American Express (A) Citi (B) Discover (C) Wells Fargo (D) Attributes Tested n=441 n=441 n=441 n=441 Willingness to Open Statistical Significance Top 2 Box = Definitely would open + Probably would open 17% 14% 22% (B) 24% (ABD) Quality Statistical Significance Top 2 Box = Extremely high quality + High quality 12% 10% 14% 15% (B) Uniqueness Statistical Significance Clarity Statistical Significance Top 2 Box = Extremely unique + Very unique 7% 5% 14% (ABD) Top 2 Box = Extremely clearly + Clearly 31% 31% 38% (B) 10% (B) 47% (ABC) Believability Statistical Significance Top 2 Box = Extremely believable + Very believable 26% (B) 17% 29% (B) 29% (B) Average Score 19% 15% 23% 25% *Z-test scores at 0.95 confidence interval Wells Fargo received the highest overall average score and scored significantly higher than the three other offers in Clarity and better than the Citi offer in Quality, Uniqueness, and Believability. For Willingness to Open, it got the highest score, clearly beating out both American Express and Citi. 12
Market Segmentation 13
Identifying what resonates with the target market Visa is the brand name that appears on Wells Fargo cards. We wanted to see if there were any differences of opinions in the Wells Fargo offer for Visa and MasterCard cardholders. The differences for current Visa cardholders were significant: Among the attributes tested, they found Wells Fargo s offers less convincing, valuable, and memorable. 40% 39% Getting MasterCard holders to act and switch over to Visa could be the key to Wells Fargo s success as they found the offers more convincing, valuable, and memorable. 35% 31% 34% 32% 36% 36% 30% 25% 20% 15% 10% 10% 0% Convincing Valuable Memorable Visa MasterCard 14
Segmenting responses reveals additional data Taking a further look, we segmented the market into two distinct groups: early adopters and those who wait to purchase new products. I usually try new products before other people do. Identifying what resonates with the target market allows for more targeted advertising and higher likeability and purchase intent. Strongly agree/ Agree Early adopters Strongly disagree/ Disagree Later-stage adopters 15 15
Identifying what resonates with the target market The differences in offer likeability for later-stage adopters (Wells Fargo s likely target market because of their It s time to switch messaging) were significant: Later-stage adopters liked the offers more than (38% Early vs 63% Later) their early stage counterparts: Taking into account that all four offers included some sort of $0 annual fee or 0% APR, later-stage adopters may need an incentive to act and a reminder that a deadline looms. 80% 70% 63% 60% 50% 40% 30% 38% Early Later 20% 10% 0% Offer likeability 16
Taking It to the Next Level 17
Going beyond your first test Offer testing goes beyond asking respondents to rank and rate offers against one another. Take it a step further: Ask respondents open-ended questions to get the "Why?" behind their answers. Track your performance over time by setting up recurring surveys. Need some help selecting metrics? We ve got you covered. SurveyMonkey provides you with a list of all the available key metrics you can test. The possibilities are endless: Package testing, logo testing, name testing, claims testing...the list goes on. Our concept testing framework spans a variety of categories all you have to do is provide us the materials to test, and we can make that concept assessment happen. Contact us. 18
Bring Your Best Ideas to Market with Offer Testing Learn more about testing your marketing content and messaging by contacting SurveyMonkey Audience today: Contact us to get started 2015 SurveyMonkey. SurveyMonkey, the SurveyMonkey logo, SurveyMonkey Audience and the SurveyMonkey Audience logo are trademarks of SurveyMonkey Inc. and its affiliates. Other company and product names may be trademarks of their respective owners.