A/B split testing for you and me

A/B split testingA/B split testing is the scientific way to see if a change is helping or hurting your site. Or if the change is just a change. It is founded in the core belief that trial and error economics works far better than HiPPO marketing.

How does A/B testing work?

The idea is to change some part of your website and see if the new version does better than the old.  A good place to start is just changing words – maybe a headline. You call the existing version “A” and the new version “B“.

You already know that the page is converting at 2% so you could just try the new headline for a few days and check the new conversion rate.

Trouble with this method is that conversion rates change from day to day. You can guess at some of the reasons: weather, day-of-week, competitor activity.  So you may be trying a bad headline on a good day or a great headline on a lousy day – and make the wrong decision about the headline.

Focus on what has changed

We can remove these other factors from the test. We show A and B at the same time. Software randomly decides if a new visitor to the site is going to see A or see B, then tracks visitors through to conversion and measures if A or B is doing better.

  • If it’s raining – people will see both A and B.
  • If it’s sunny – people will see both A and B.
  • If a rival starts a price war – people will see both A and B.

You have removed the other factors.

Is it significant?

We use a statistical test to check that results are significant – and not just random effects. Sometimes improvements are small, but still worth having. Several 2-3% increases start to add up.

Is it just Connected doing split testing?

Amazon, Zappos, Google, eBay, EasyJet – all the big names use A/B split testing. Join the big boys – with our help.