A good email marketing automation tool is a complex beast. We want lots of features from something so essential for running our business, but it's also easy to get lost in all the complexity.
For today's review, I want to look at one of the most pivotal features in email marketing tools: A/B testing.
A/B testing can make a huge difference to your business, especially if you do it consistently, over time. So let's take a look at what split testing features are available in different email marketing tools and how they work.
Let's start with an overview of the tools reviewed and the A/B testing broken down into sub-features:
A/B Testing for Broadcast Emails
A/B Test Emails in Automation Flows
A/B Test Entire Automation Flows
Optimize for Product Purchase
Note: every solution that has A/B testing features allows optimizing for opens or clicks, so I didn't include that in the comparison table.
As you'll see further below, the cons of ActiveCampaign are really first world problems, compared to how lacking some of the other A/B testing feature sets are. As far as I'm concerned, all ActiveCampaign needs to be a perfect solution is revenue tracking and more integrations. As it is, you'll have to do some custom coding or integrate via Zapier, to be able to optimize for product purchase related goals.
Having said that: ActiveCampaign clearly boasts the most extensive and most powerful A/B testing features of any of the tools I've tested so far.
Not using ActiveCampaign yet? Click here to get it.
Autopilot's A/B testing feature is the perfect example of "so close and yet so far". Having a split feature as a step to insert in an automation flow is great, but the feature's potential is ruined by a lack of conversion goals. Without goals, you can't automatically determine a winner and that makes using the feature slightly impractical for broadcast emails and very impractical for ongoing automations.
Convertkit takes the extremely minimal approach to A/B testing. There's only one feature: testing two subject lines against each other. And it's only available in broadcast emails. The upside of this is that it's very quick and easy to use. And it is a feature you'll benefit from: make a habit of always testing subject lines and over time, the benefits really accumulate. It's too bad that there are literally no other ways to incorporate A/B testing in your Convertkit marketing campaigns.
Drip really broke my heart on this one. In my testing so far, I'd really warmed up to it, because it's wonderfully geeky and power-user oriented. But on A/B testing, it really drops the ball. The feature that exists involves many steps to set up and is limited to emails within follow-up sequences. You can get some benefit out of the feature as it is, but it just pales in comparison to the ease of use offered in other tools or the power offered in ActiveCampaign.
GetResponse is a solid 2nd rank solution, regarding A/B testing. The testing feature is well designed and easy to use (once you found the first step) and you can insert emails that are automatically optimized for conversions in your automation flows. You can't, however, test entirely separate flows against each other.
iContact receives just about receives a passing grade, in this comparison. The A/B testing feature isn't so bad or so limited that it's a total deal breaker, but it's also just not very good. There's nothing exciting about it and it's certainly not a reason to switch to iContact.
MailChimp was a very pleasant surprise in this roundup review. The standout feature is that you can optimize your emails for revenue! OMG, MailChimp, what is this? I'm starting to like you!
This is the one feature I wish other solutions would implement as well. In many cases, it's perfectly reasonable to optimize for email opens or link clicks, but for direct response type promotional emails, optimizing for revenue is awesome.
MailerLite offers another mid-tier entry in this roundup. There's a good enough testing feature available for broadcast emails, but that's where it ends. Better than iContact or the tools that don't offer any testing at all, but a far cry from something like ActiveCampaign.
Mautic is another candidate that misses the mark. It is possible to run split tests with this tool, but setting them up and taking action on the results is a bit of a convoluted mess. User friendliness is something I take seriously because of how us humans behave. We may have good intentions and say to ourselves "testing is important and I'll do it even if the feature is complicated" but in practice, in the real world, the more complicated a feature is, the less likely we are to actually use it.
I selected the tools to test here based on what's popular and what I get asked about a lot. Among the candidates, Aweber, SendinBlue and Sendy are basically instantly disqualified because they don't have any split testing features at all.
In terms of choosing a tool to use, a lack of A/B testing is a deal breaker for me. It's a reason to not start using any of these tools (even if they're cheaper or have other good features) and it's a reason to switch from these tools to something else, if you're currently using them.
I've been recommending ActiveCampaign for a long time and as you can see, it comes off as a clear winner, here. Plus, it's the first one in the list!
This may make you suspicious. Am I just fluffing ActiveCampaign, here?
I was actually surprised at the results from this test. I'd been warming up to Drip over the course of my testing (it has some features I really, really love) and I was expecting it to have a much stronger showing here. I was also surprised that Autopilot didn't fair better and looking at the comparison table at the top: going into the review, I would not have guessed that there'd be so many red crosses and so few green checkmarks in the end.
So, no: this review wasn't set up to make ActiveCampaign look good. And it's first in the list because I sorted the tools alphabetically.
Needless to say, now that I've investigated all these tools more closely, it further reinforces my recommendation of ActiveCampaign. I do hope that other tools will catch up, though.
I'd like to know from you: are you interested in learning some advanced marketing tricks that can be done with ActiveCampaign (or a different tool)? Would you be willing to pay for a course on the topic?
Also, if you're using a different solution, do you need help switching?
Let me know your thoughts on this by leaving a comment below.
And of course, if there are other tools you think I should include in a future update to this review, please let me know!
I'm the founder of ActiveGrowth and Thrive Themes and over the last years, I've created and marketed a dozen different software, information and SaaS products. Apart from running my business, I spend most of my time reading, learning, developing skills and helping other people develop theirs. On ActiveGrowth, I want to help you become a better entrepreneur and product creator. Read more about my story here.
Treat Every New Subscriber like a VIP Using This Email Automation Template
Livestream Replay: Your Questions Answered, My Favorite Underrated Marketing Skill & More
Email Marketing Automation: The Best Tools Compared
How To Steal Top Email Marketing Secrets From the World’s Best Internet Marketers
Please log in again. The login page will open in a new tab. After logging in you can close it and return to this page.