Things To Keep In Mind For A/B Testing

By Jessy Troy

Jessy is an active blogger and social media user. She loves blogging about writing, creativity and getting things done. You can see her tweet as @JessyTroy
J. Troy

A/B testing…it’s a common practice. Yet some people never heard of it, much less taken part in it. It’s the unsung hero of marketing, with many missing out on the benefits of this simple process. What makes it so unique isn’t just the data that can be gleaned. It’s the fact that anyone, even those with no real experience marketing a business or website, can do it. You just need to keep a few things in mind.

What Is A/B Testing?

Email Marketing

Let’s say you were looking to create a landing page that lead to people subscribing to your mailing list. You would create several versions of this page using different design, language or other changes. Then send different groups to one of the variations to see how they respond.

Using the data gathered from these experiments, you will be able to craft a final version that has the greatest impact. By engaging on a level you know your audience responds to, you improve your chances of getting those mailing list subscriptions.

This same idea can be applied to any process, as long as you are testing two or more competing versions of the same thing.

How To Begin A/B Testing

A/B Testing

Your first step should always be clearly defining your goals. A/B testing can be used on any element of a marketing, or even design, strategy.

Are you looking to figure out what wording best works in your sale announcements? Get more likes on your business’s Facebook page? Redesign a more efficient, user friendly website? Develop a more intuitive UI for your company app?

Once you have established what you are trying to do, you should

decide on how. New A/B adopters may want to keep things simple and start out with a two variant, split testing format. That is taking two versions and testing them, rather than a multivariate that uses three or more mockups.

Finally, set your groups and schedule. A proper A/B test needs to last at least a couple of days. But it can go on for weeks, or even months, depending on the extent of data you are looking to gather. The good news about this form of data collection is that it is passive. So you don’t have to continuously monitor, and can go about business as usual while the test is running.

Simple matters like the wording of newsletters can usually benefit from a shorter test. Larger scale changes like full site redesigns are better suited to longer tests that go on for at least a few weeks.

Things To Keep In Mind When A/B Testing

Optimizely Testing

Now that you are ready to begin, there are some things you should keep in mind:

  • Split testing that uses two variations dictate that you only change one component at a time and stick with that change until the end of the test. Doing otherwise will confuse the results. Those changes should be consistent in every bit of content, including web pages where design tweaks have been made. Multivariate testing allows you to do more than one change at a time, but it becomes more complicated the more versions you create.
  • You will need to repeat your tests. One instance of A/B testing is not going to be enough. To get the best results you will need to have plenty of data to compare them to. Failing to do so will make the information you gather unreliable, and so the changes to your campaign you make might not be as beneficial.
  • Don’t cross test your groups. Some claim that having one test group look at another variation during a different test is fine. They might be right, but they also concede that it could have a negative impact or confuse the user. It is better to not risk it and just get a new group for each variation and test. Or at least cycle them out with a wide berth of time between each one.
  • There are tools to help you out. A/B testing software is all over the place these days. They offer an easy, low cost and generally efficient way to use both split and multivariate formats. Some examples are Optimizely, Visual Website Optimizer and Verster Multivariate and A/B Testing Software.

Conclusion

At the heart of it, A/B testing is just split or multivariate testing. You are taking two (or more) similar components and pitting them against one another. By finding which variation bring in the greatest results, you’re able to make fundamental changes to campaigns which in turn improves search visibility, traffic, profits or other goal.

Be patient throughout. A/B testing is effective, but exhausting. It takes time, energy, and sometimes the results aren’t what you would have liked. More than once a user had to change their entire marketing campaign to reflect the needs and desires of their target audience. It can feel like a cursed blessing, to put in so much work, only to find out more work will be involved in putting it to use.

But in the end, A/B testing is worth it – It’s a crucial tool that everyone should be aware of.

Featured images:
  •  License: Creative Commons image source 
  •  License: Image author owned 
  •  License: Image author owned 

2 thoughts on “Things To Keep In Mind For A/B Testing

  1. Fantastic article learn yet another great trick from Alex’s blog and credit to the author Jessy she puts across a great argument for the pro’s of A/B testing. So time to check out some of the recommended tools.

Leave a comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s