Advanced Email Marketing: A/B Testing

Email campaigns and newsletters can be a great way to get repeat business. A/B testing is a great way to refine and optimize your emails to help drive not only higher open and click rates, but ideally translate into higher conversion rates and more revenues.

With A/B testing, you want to adhere to following guidelines for optimal results:

• Always test simultaneously to reduce the chance your results will be skewed by time-based factors
• Test as large a sample as you can for accurate and statistically significant results
• Test early and test often for the best results
• Only test one variable at a time for best results

Things to A/B Test It’s important to only test one thing at a time to get accurate results. Things you might consider testing include:

• Call to action button or text
• Subject lines
• Testimonials to include (or whether to include them at all)
• Email Template and Design
• Personalization
• Body text
• Headline
• Closing text
• Images
• Discount offer

Each of these things is likely to have an effect on different parts of the conversion process. For example, your call to action is obviously going to have a direct affect on how many people buy your product or click through to your website. Your subject line, on the other hand, will directly affect how many people open your email in the first place.

Think about this when you’re deciding which things to test first. If not many people are opening your emails, then you’ll likely want to start with your subject line. You’ll likely want to test the more important parts first. Your headline and call to action will likely have a great impact on conversions than the images you use or your body text. Test those things first, and then test the others in greatest to least importance.

Test Your Whole List, Or Just Part? In the vast majority of cases, you’ll want to test your entire list to get an accurate picture of how your email list responds to your new email campaign. The larger your test sample, the more accurate your results will be. Make sure that the split is done randomly. Hand-picking recipients (or even using two lists from different sources) can lead to skewed results.

There are a few instances, though, where you might not want to test your entire list:

• If you’re trying something really extreme, you might want to limit how many people potentially see it.
• If you’re running a limited-time offer, and want to get as many conversions as possible, you might want to run a small test batch first, and then send out the winner to your entire list.

Establish a Baseline Before you A/B test emails, it’s important to decide what you’ll be testing for and what you define as success. First, look at your previous results and establish a baseline. For example, if you’ve been using the same email campaign style for months or years, then you’ll have a good pool of data to pull from for click rates. Or, if your historic conversion rate is 2%, then you might want to increase that to 4% by testing promotional offers.

Technique to Test You can setup an A/B test manually by simply splitting your current list into two separate lists, and then send one version of your email campaign to one list and the other to the other list.

Analyze the Results Once you’ve run your email campaign with the two different email versions, it’s time to take a look at the results. There are a few different categories of results you’ll want to look at:

• Open Rate
• Click Rate
• Conversion Rate

The reasons behind tracking the first two are pretty obvious. But a lot of people might wonder why we’d want to track the conversion rate outside of the email. Wouldn’t that be beyond the control of the email itself?

It’s important that the message you give in your email is consistent with the message on your website. If you’re promising your visitors a special deal, and that deal isn’t perfectly apparent on your website, then you’re going to lose customers. The same can happen if your email doesn’t echo the look and feel of your website. Visitors might get confused, and wonder if they’ve landed on the correct page.

The end goal here is conversions, not just click-throughs. You may find that one email gets more click-throughs than the other, but that it doesn’t result in as many conversions. In that case, you’ll probably want to do more testing to see if you can get an email that not only results in higher clicks, but also higher conversions.