The ABCs of A/B Testing: Optimize Your s and Landing Pages for Better Conversions

Transcription

1 The ABCs of A/B Testing: Optimize Your s and Landing Pages for Better Conversions FPO (in act-on colors, or in white with dark background) The ABCs of A/B Testing 1

2 Market conditions are constantly changing, and so are your buyers, so you can t always count on what has worked in the past. Marketers need to proactively adapt to these shifts, but with constant pressures to increase demand and revenue, they can t afford to send a lackluster that drives to an underperforming landing page. Testing is essential for optimizing digital marketing campaigns, because it provides a proven method to find the combination of design and content that does the best job of enticing your audience and spurring them to action. Brilliant concepts and executions are obviously important, but they re only the beginning when it comes to developing successful marketing campaigns. An overwhelming 68% of digital marketers rated the ability to test new campaigns as having the greatest impact on their marketing efforts (Direct Marketing Association, 2015). And that s not surprising, considering and landing page testing takes some of the risk out of trying new optimization tactics. Testing gives you the statistical data you need to back up your decisions, so you can develop and evolve your campaigns with more confidence. A/B testing is often the best way to uncover the perfect combination of factors to boost the performance of your campaigns and improve your ROI. The ABCs of A/B Testing 2

3 Types of Testing: A/B Before jumping into the world of testing, let s examine the pros and cons of the two primary types of digital testing most commonly used today. A B A/B Testing This is the process of analyzing the effect of one change, known as the variant, against a current design, known as the control. By keeping your tests identical (aside from one variation), you can evaluate the results to see which variant impacts your or landing page performance, and in what way. For example, you can quickly discover whether the call to action Buy Now or Shop for Deals will work better with your target segment. Pros Because you are changing only one element of your design, A/B testing is quick and easy to implement. This makes it simple to optimize your designs for specific audience segments, such as participants in a nurture program. Cons The results might not be what you expect. When making small adjustments to copy or design, there is a possibility that you might not have an overwhelming majority leader. Sometimes it s a draw. The ABCs of A/B Testing 3

4 Types of Testing: Multivariate This is the process of testing many different versions of an or landing page, with many different combinations of variables you have changed. It is essentially the everything-and-the-kitchen-sink method Pros Multivariate testing is ideal if you need to test multiple variants but you don t have the time to conduct smaller ongoing tests. Additionally, because the tests themselves are larger, a larger test group is required, which could provide more data for analysis. Cons Because you are testing many different elements, this process requires a much longer planning cycle for design and execution. And while you will be able to determine which version performs better, you will not be able to pinpoint which variant had the largest impact on your performance. Although each system has its good points, in the end we favor A/B testing. It can provide a more precise determination of a variant s effect; and when you know for certain which variable is responsible for success, you can scale that knowledge to your subsequent efforts. If you have an automated testing system, it takes less time and effort than manually testing every element. In this ebook, we ll focus on the ABCs of A/B testing. You ll discover the different types of testing variations, get ideas for what you should be testing, and see real examples of testing successes. The ABCs of A/B Testing 4

5 Getting a Statistically Significant Result Statistical significance is important, as it removes the possibility that the effect you caused in your test happened just by chance, and makes it more certain that your results were indicative of reality. A VS B Version A is sent to 10% of audience Version B is sent to 10% of audience 46% RESULTS 32% A Winning version is delivered to the remaining 80% of your audience after a predetermined test period The ABCs of A/B Testing 5

6 How A/B Testing Works A/B testing can have a significant effect on campaign results. Knowing that the best version will go out to the majority can equal instant optimization. If you believe the open rate on an you re going to test will be low, you ll need to increase the total number of members in your sample pool to achieve that significance. There are plenty of calculators out there that can help you make sure you re getting a statistically significant result. For now, we ll keep it simple. Just keep in mind that the lower the number of conversions expected, the larger the sample size will need to be. As a rule of thumb, aim for at least 200 responses to each variation. Define Your Test Audience One of the most common questions in A/B testing is about creating segments. How do you decide which audience members receive version A or B of the test? The short answer is you don t have to decide technology can do that for you. When using A/B testing technology, all you have to do is define the audience you would like to expose your tests to, and the testing solution does the heavy lifting by splitting your audience into two test segments. Depending on your setup, you can choose to do a 50/50 split with your tests delivered at one time to get your results. Or as in the as in the example of the previous page, you can set it up so that only 10% of your audience receives version A, and 10% receives version B. After the specified test period has ended and the results have been determined, the winning version would be delivered to the remaining 80% immediately. Again, technology can automate this process, so you won t have to worry about manually splitting lists or scheduling the launch of your winning versions. The ABCs of A/B Testing 6

7 Test Yourself (Before You Wreck Yourself) It s important to have a testing system in place so that you and your team can become deliberate testers. As a general rule of thumb, you should always work towards improving your conversions. There are three important metrics which could impact your conversion performance: 1. Open rates 2. Click rates 3. Form submissions There are many different factors which could impact each of these metrics, so how do you decide what you should be testing? According to a study conducted by Experian Marketing Services (The State of Cross-Channel Marketing Report, 2014), the most popular items to test in marketing are: Types of Testing Conducted by US Marketers, July 2014 (% of respondents) Subject line Creative 95% 98% That s a lot of tests, and it can be easy to get overwhelmed. But the good news is that while you will never run out of new theories to try out, you don t have to try everything. But before we take a deeper dive into the different items to test, here are some additional tips to help you get the most out of your testing strategy. Set goals: Create a hypothesis for the test and define which metrics you will monitor to identify the winning version. Trust the data: Don t let your personal preference be your guide trust the test outcomes. Synch your tests: Tests should run at the same time for exactly the same amount of time. Marketing automation and A/B testing technology make this easy. Call to action Time of day Frequency 64% 77% 76% Make it ongoing: You ll want to keep testing on a regular basis, since the effectiveness of any element in your or landing page can change over time. Day of week Pre-header text Product placement Number of products 61% 55% 54% 46% Share the results: Some of the results might impact campaigns other members of your team are working on. Share the results so the department can continue to optimize across campaigns and channels. Mobile optimization Friendly form Multivariate 45% 37% 34% Now that we ve covered how to conduct your testing, let s explore what you should be testing. 8% Don t currently test Source: Experian Marketing Services, The State of Cross-Channel Marketing Report, Dec 23, 2014 The ABCs of A/B Testing 7

8 Inbox Testing: How to Get Your s Opened There are some prime suspects that influence the rate at which your s will get opened. The major variables to test include: From Addresses Do you send from a real person s name or your corporate brand identity? The number one question driving whether we open an is usually, Do I know you? People are more inclined to respond if they know a real person is reaching out to them, so test sending your messages from different aliases. Subject Lines A recent study released by Howling Mad (The State of Split Testing, 2014) showed subject lines were ranked among the top factors for impacting response rates. However, 22% of marketers have not tried split testing for subject lines. When testing subject lines, you should be thinking about personalization. Try adding a first name, company name, or the name of the prospect s website in order to increase engagement. And try to keep the subject line short. Between 35 and 40 characters works best for testing. The thought here is that the longer it takes for a person to read through the subject line, the less likely they are to read the . Opening Copy This is the top copy, headline, or value proposition. Many people rely on the preview or the first few lines of the to decide whether to open the message. You should consider how this combines with your subject lines. A great test to perform is swapping the headline and the subject line to see which version receives more opens. Nicola Tesla Don t be a Turkey! Serve up Content Your Aud... Click here to view this message in a browser window... Act-On Software Thanks for Attending: The Idea Book We re so glad you were able to join us for our webinar, The Idea Book... The ABCs of A/B Testing 8

9 Inbox Testing: How to Get Your s Opened, Cont d. Day and Time of Send Whether you re a B2B or B2C marketer, you are directly connecting with people. And according to a recent study by Moveable Ink, 49.5% of marketing s were opened on a mobile device in What does that mean for you? For one thing, it means that your s can and will be opened outside of the normal Monday through Friday business hours. You can try experimenting with sending on a weekend or after business hours to see which days and hours will get open rates for your s. Case Study When the marketing team at Act-On began testing send times, we were shocked to find holidays and weekends received higher-than-average open rates. In this test example you can see that s sent on Saturday received a higher open rate. Test Name Sent Open Click Open % CTR Weekday Send % 3.0% Saturday Send % 4.3% And in a separate test for holidays, an that was in an automated program scheduled to deliver on Mondays was sent out on Labor Day. The results were astounding with a 40% open rate, which was much higher than the average. This is a perfect example of why it pays to test your theories, because your gut instinct might not always be correct. The ABCs of A/B Testing 9

10 Your Has Been Opened Now How Do You Get a Click? The goal of most s is simple: Produce a click! And the best way to do that is send an that delivers on the promise of the subject line. It s not just a good idea it s a key tenet of the US CAN-SPAM Law. So if you ve done that, what s next? Here are a few tests to help you get more engagement with your messages: Copy The first thing to test is which type of copy captures the interest of your audience more. Experiment with shortening the copy in your s; save the longer, detailed-oriented copy for your landing page. Personalization Go beyond Hello First Name in the opening of an . Think of more compelling information to personalize. For example, we ve seen some success with including a prospect s website, dynamically populated in the copy. Format Balance your copy with value. Consider testing a bolded or underlined call to action against your control, so you can draw the attention of your reader to the most important parts of your message. Some other tests to consider are changing the color and size of your font. Images We ve all heard it; a picture is worth a thousand words. Is it true for your audience? Try shortening your copy to provide more charts and graphs to convey the same message, while adding more visual appeal. Caveat: Make sure your reader can still understand the opportunity if the images don t render. Animations can also have an impact on your conversion rates. Adding some movement to your images can help grab the attention of your audience and drive them closer to the click. Case Study Act-On marketers tested static images vs. animated gifs and achieved nearly double the clickthrough rates. You can t see the animation in this ebook, but in the image on the right the toolbox slowly increases size within the magnifying glass. For us, it s clear that animated images can significantly increase audience engagement. VS static image animated image 10

11 The Key to Getting The Click Buttons and CTAs Getting that click is so important and the call to action is the key to the click. Explore different colors and copy, and then A/B test to see which one gets more responses. For example, you can test Submit vs. Enter to win or Sign up vs. Participate. Pro Tip: Avoid using Submit on a button it sounds pushy. Use friendly, action-oriented phrases like Sign up now or Take part instead. Case Study The Act-On team conducted a head-to-head test using a static green button versus a gray button with multicolored, animated arrows. In this test, the open rates were identical. However, since we were focused on clicks for this test, the animated button proved to be the winner. static button animated button Test Name Sent Open Click Open % CTR Static Button 25,854 2, % 3.10% Animated Button 25,889 2, % 4.26% 11

12 Responsive Design Gets Results Responsive Design Using responsive design can be critical to the success of your campaign. As we previously mentioned, basically half of all marketing s are opened on a mobile device. So if your doesn t look good on different screen sizes, your message probably won t get read. Case Study At Act-On, we tested reformatting an existing for responsive design, and here s what we found: These examples show how each version is viewed on a desktop. The design is different but the message is still the same. The biggest difference is how the s look when opened on a mobile device. The mobile version on the right reformats the text and images so all the reader has to do is scroll down to read the message. The version on the left cuts off the image and the text, forcing the reader to either zoom, or scroll left, or swipe down to view the whole message. The result? The proof we needed was in the conversion rate. The responsive design yielded a whopping 93.3% increase in overall engagement! If you re not using responsive design, this could be the single biggest improvement you could make to your campaigns. 12

13 Stick the Landing: Getting that Click to Convert You ve tested messages to get a click, but now what? Depending on your objectives, you still need a little more action from your audience action that takes place on the landing page. Your landing page should be a stand-alone page that supports your message, and works towards getting the coveted form submission. The tests you can conduct on landing pages are similar to the variables we have discussed for s. Here s a summary of tests the Act-On team has conducted to increase conversions on landing pages. Form Testing Length: One of the easiest tests to conduct is experimenting with the number of fields you are requesting. As a general rule, the shorter the form is, the more likely someone will be to provide their information. According to Ascend2 research, 61% of marketers worldwide use shorter forms with between two and four fields ( List Growth Survey Summary Report, 2015). Pre-Filled Fields: Filling out forms can be a pain, especially when you know the company already has your information. Try selecting a pre-fill form option, so all your audience has to do is hit the button if you already have their information. This also reinforces for them that they already have a relationship with you, and that this isn t the first time they have said yes to one of your offers. Case Study In order to optimize landing page conversions, Act-On marketers experimented with using personabased fields. In the persona testing, a marketing professional received different set questions than someone with a sales-related job title. In each test, persona testing outperformed the questions that were geared toward the general public. What is your biggest marketing challenge?* What is your biggest sales challenge?* What are your company s biggest challenges?* Persona-Based Fields: Tailoring your form fields to your target personas is another test that can help your conversion rates. You can get better information about your audience if you are tailoring the forms to address their previous behaviors, needs, or interests. Adding Trust Seals Trust seals and privacy policies help ease the natural fear of filling out a form. After adding the TrustE and Safe Harbor logos to Act-On landing pages, we noticed a 15% increase in conversions. The ABCs of A/B Testing 13

14 And the Winner is The Act-On team conducts a monthly survey campaign that has become our go-to guinea pig for testing. Using the winning results from all of our A/B tests, we implemented our own multivariate test to create a complete redesign of the campaign. The image on left was the control version of the survey. On the right is the multivariate test version, which implemented responsive design, a shorter form with drop-down options in place of checkboxes, and an animated GIF. (You can t see it here but in the example on the right, the bars in the graph on the laptop move up and down.) 60% After testing over many months, the results were astounding! We ve been able to continually learn from our tests, and ultimately we saw a huge conversion increase of 57% in the campaign. These results are a perfect example of how you can implement small changes over time to create a large impact on your conversions. A/B testing is a tried-and-true method for uncovering what works and what doesn t. It doesn t always have to be complicated, but it should be an integral part of your development processes. 50% 40% 30% 20% 10% 17% 18% 28% 57% 0% SEPT OCT NOV DEC 14

15 Acclaim for Connect with us to learn more See all of Act-On s awards and accolades About Act-On Software Act-On Software delivers cloud-based integrated marketing automation software. Marketers can manage all their online marketing efforts from a single dashboard that can be seamlessly integrated with CRM, giving sales access into various marketing functions. Act-On s fresh approach to marketing automation gives its users full functionality without the complexity other systems impose, and makes campaign creation and program execution easier and faster. #ActOnSW 2015 Act-On Software, Inc. Trademarks belong to their respective owners. All rights reserved. The ABCs of A/B Testing 15