Why A/B tests may not be the best for you

Publicado por Andres Aguerrevere en

Why A/B tests may not be the best for you

Many sites are using A/B or split testing these days as the best way to measure their conversion rates, it is easy, simple and cheap. They do help you keep track of the conversions on your site with just a few minimal changes to your page. While this is true, and you can read many testimonials of how the conversion rate increased the number of conversion in a site; A/B testing might not be the best idea for you.

So, if it’s proven that using A/B testing can help you get more conversions, why do I say it is a waste of time? Whereas it can be a good idea, it doesn’t necessarily mean it is THE BEST idea for you.

Let’s take a look at 6 reasons why A/B testing might not be what you need.


  • It represents only the 20% of the conversion process:

Many people believe that applying an A/B test to their sites is the best way to start a new page, so they can measure their conversion rates from the beginning. But you need to set a few steps before heading this direction.

Yes, it is easy and fun to see how your rates increase but there is more to it than just measuring conversions through a button color test. You have to set your project and analyze the possible problems on your page and how visitors react to it. Then, only after you understand how the interaction works between visitors and your page it is possible and how they feel with it, you can apply an A/B test.


  • It fails in understanding your market

It is crucial for your brand’s success to know your customers, and understand their behavior. There are many elements you need to ask in order to fully grasp why they come to you.

Why you and not your competitors?

Why do your new customers leave?

What makes your loyal customers coming back?

What is that made you stand from the rest?

These are fundamental questions you need to ask, in order to improve your services and increase not only the conversion rate, but also the traffic to your site. The more you understand your audience the easier it will be to reach out to them. And sadly A/B testing does not allow you to collect all this information.


  • A/B tests only focus on conversion

Why isn’t this a positive thing? Well, while looking up at the numbers and seeing how your conversion rates increase it can distract you from more important aspects that you really need to consider. The quality of the converts, are they long term prospects? What can you do to keep these visitors coming back to you?

On the other hand, while you are focusing on the conversion rates on the visitors your already have, you may leave aside strategies to increase your web traffic. Which for new business is definitely more important, you need people to know about you, to know your website, right?

Let’s talk about numbers here, imagine that your regular traffic is 100 visitors, and you have a 20% conversion rate. After your A/B test the rate increases to 30%, it means that you go from 20 conversions to 30.  But, if you work in your website traffic and start getting 1000 visitors, the same 20% would mean 200 conversions, a way better number.


  • Defective testing tools.

Since A/B testing is becoming more popular these day users can find a huge range of software designed for this purpose. They are low cost and easy to use, but sadly most of them are not as accurate as you may hope, not only because they are faulty, but because the one that you chose might not be the best option for your website.

Moreover, these tools can slow down your site resulting in less page views and therefore a decrease in your conversion rates. Lastly, we can say that using the wrong tool for your site can give you inaccurate results, which at the end will be harmful for your site.


  • Seeing early results leads you to stop testing too soon.

Many testing tools advise you to stop your test as soon as you see relevant results. It is only logical, right? You want results and you get them fast, so why keep trying?

It is a common mistake that comes with A/B testing, it is so easy to measure up quickly and see some results that you may just stop testing and assume you’ve got it all covered. Yet, these first results may be false positives. A/B testing requires you set a realistic time frame based on your website traffic and that you stick to it. There is more work and organization than you may think, and the results you get are not as complete as you may wish.


  • You only test small incremental changes.

Refreshing your website’s image is important; it can lead you to increase dramatically the traffic and the conversion number. Of course, all these radical changes must be thoughtfully planned and you need to invest a lot of time to its organization, but sometimes in order to win big, you need to bet big.

For applying A/B tests you need to keep your entire page layout the same, making really small changes that do not affect its general appearance. The “color button test” has limited in many ways the possible modifications and improvements you can apply to your site. This can draw you back from making significant changes that can benefit you greatly.


When it comes to measure and optimize your traffic and conversion rates, there is varied number of options to make it possible, and its crucial to get the results that you need to find the tool that better meets your needs. Do not settle with A/B tests because “they are easy”, find the tool for you and increase your numbers!


Previous Article Next Article