Most organisations, especially marketing-driven ones, are not familiar with one of the greatest web weapons available, namely continual testing. Traditional departments hum their day away thinking of great new “marketing” initiatives and wish they had invented the Meerkat or the two annoying blokes on the Safestyle Windows advert.

Most of what they produce is low-grade, recycled, tat. And that’s a shame.

Some of the brightest people I have met worked in marketing departments, but mostly they weren’t thinking about marketing, they were thinking about operational needs, budgets and getting out the latest offer in the post, by Friday. Ring any bells?

Most senior management and executive boards are not really that keen to sign-off on big changes, especially when “it’s the brand you’re playing with, don’t you know”. So here is a quick one paragraph trick to get them listening and, more importantly, keen to try something new.

Tell them “every company that tried online testing with their website saw at least a 20% improvement in conversion rates, in a month” and “that equates to £100k per month” – but obviously use a figure that represents 1/5th of your online turnover or if you are in the data acquisition world 1/5th of the total number of data items the online world generates for you. That should get their attention! Next tell them to half your salary and pay you a bonus based on performance with the condition that when the online conversion rate has doubled they have to pay you double your old salary. It that doesn’t catch them, which it bloody well should, contact me and I’ll organize a free test for you.

The results speak for themselves and as almost everyone is now getting into feverish testing you’d better start testing…or update your CV.

Thankfully, this is not the case. Much like the focus on “bounce rate” it all ended up being a huge Red Herring. It took a number of scientifically accurate tests to prove that the fold is not quite as critical as you might imagine. Don’t get me wrong, it can make a difference but, for gods sake, test it first as we found out this year with two big tests concerning what is and what isn’t “above the fold”. In both cases it made no difference so we can welcome back “long copy” that until recently has been consigned to the bin.

We all know what really matters is the customer journey and overall conversion and not whether all the content and actions are above the fold on the home page. Or, as an aside, does the bounce rate have special meaning.

Here is a brief outline of what you can expect in the next release of the worlds first, commercially available, personalisation and customer journey platform

• Faster core platform, requiring less server computing power and faster serving of content
• In-built AB testing functions, faster, easier testing of content, pages and call-to-actions
• Server load balancing, giving higher system availability, improved fault tolerance and improved performance
• Off the shelf CMS support, de-skilling and speeding up changes to content
• Faster profile management, faster and easier creation of new customer journeys
• Reporting API, simplifying the export of business-critical data giving easier and faster access to real knowledge
• New User Group to support discussions, bug-tracking, feature request and cross-learning between clients

Initially released in 2006, VITES was designed to dramatically improve on and off site conversion rates by providing a scaleable platform that offered proper customer journey management (ala Amazon, Ebay etc) combined with a suite of testing tools that allowed accurate testing of new content, CTAs and traffic streams

Since it’s release, every client using the platform has seen at least a doubling of conversion rates and huge reductions in cost per enquiry/sale

The latest release is a ground-up rethink of what our clients and marketplace needed and part of this was a massive simplification in deployment of changes, testing and profiles

All current clients are on a migration plan to complete the porting to the new version by the end of 2010 and all new clients will automatically get the latest version of the platform

License charges remain unchanged, starting at just £500 per month for the basic 10k users per month version

Contact Liam, Martin or Nick now to find out more about how VITES 3.0 (Rangoon) can supercharge your web strategy.

VITES remains the only commercially available off the shelf journey profile and testing platform

Contact our licensing team (Liam, Martin or Nick) for further information.

System integrators and agencies should contact our CEO (Martin Dower) to discuss how VITES can help your clients

We’ve all heard the “lies, damn lies and statistics” quote and still, all too frequently, statistics have been made “to fit” a required outcome

Most people know that this is wrong but still blindly accept the results, probably from a lack of their own self-confidence to question the results

The point still remains the same and the goal is unchanged; we want to better understand and predict the world around us and mathematics offers us the clearest path to that understanding What’s lacking, quite often, is the right information required to correctly analyse a situation and come to a correct answer

Probability 101

Q. I have two children and one is a boy, what is the probability that I have boys?

Common sense tells me that the other child has (for the purposes of this experiment) a 50/50 chance of being either gender so the the common sense answer would be 1/2. Except that this is not true as is has a precedent (I already have one boy). The possible combinations of children are BG, GB, BB or GG and since I already have one boy this removes GG from the equation leaving the probability as 1/3 of having two boys.

Probability 102

Q. I have two children and one is boy born on Tuesday. What is the probability that I have two boys?

Again, common sense suggests that it will be the same as above, why would the day of birth make any difference to the statistical outcome. But it does.

Let’s, using the above naming convention, call a boy born on a Tuesday a BTu. This gives the following scenarios.

* When the first child is a BTu and the second is girl born on any day of the week there are SEVEN possibilities.

* When the first child is a girl born on any day of the week and the second is a BTu there is an additional SEVEN possibilities.

* When the first child is a BTu and the second is a boy born on any day of the week then, again, there are SEVEN possibilities.

* Finally, there is a situation where the first child is a boy born on any day of the week and the second child is a BTu. Again there are seven possibilities but, and here it gets interesting, one of them has been counted before so there are only SIX possibilities.

Counting likely outcomes we then have a total of 7+7+7+6=27 different combinations and 13 of them include two boys the answer is 13/27, wildly different to the 18/27 (1/3) answer to the first question. This is surprisingly odd and (entertainingly) illustrates that seemingly unconnected pieces of information can make a huge and statistically very important difference to outcomes.Whilst this post is folly of sorts it does have a serious side. When you are trying to measure information to produce meaningful outcomes you really must be very careful to decide what to include and what to exclude. And, you must have a grasp of how to use the information correctly, even if the mathematics required were learned when you were 13.

Too bloody many! All of them purport to offer the answer to the holy grail of advertising landing page design.

Most of the advice is re-regurgitated, old, unproven and bloody obvious if you have more than an ounce of common sense. What would be more useful would be a real guide to what has worked, why and how well.

The reality is that landing pages are very often the real “heavy lifters” on a web-site in terms of generating enquiries, sales, data and actions. That is a good thing as at least we know where to start when in comes to optimising performance as PART OF AN OVERALL STRATEGY.

Let’s not get bogged down in the fine detail of what to try, where and why as this does vary from site to site, person to person and market to market. Let’s start where we should do, at the beginning and from the visitor’s point of view.

1. The visitor arrives at the landing page after consciously clicking on a sponsored link. They want something, you know what they searched for. Give them what they want. If they search for a brand term then give brand options, if they search for a product then show a product, if they search for a place then give then location-based information.

Focussed landing pages have performed 150% better than generic landing pages. In real tests, with real visitors, this year.

2. Don’t make ‘em wait, impatient or frustrated people don’t convert as well. That means a fast loading page (ever wondered why Google’s home page is just 16k in size?). Speed of loading is dependent on small pages, fast servers, good connectivity and few outside “includes”.

Frequent testing with our clients show a direct correlation between logical size of page and performance. A recent test saw a 25k page perform 50% better than a 100k page and 75% better than a bloaty 150k page. The pages were identical aside from logical size.

3. Visitors want to do something, such as buy, apply, find, request or contact. Make those options clear, visible, easy and fast to do.

Above the fold can make a difference but only if the page is uncluttered. A recent test on a cluttered page showed no difference in above and below the fold for a conversion point.

4. No clutter. Less is more. Reduce choices to keep the visitors decision process simple. No need for complex navigation (or any real navigation at all?). Big text, small words, white space, compelling reasons for an action and no bloody clutter. Everything on the landing page is there to drive the conversion, everything else is dead-weight that WILL hamper conversion rates.

Testing a series of landing pages recently saw a dramatic increase in on-page, in-session and intra-journey conversion when the navigation system was removed as the page was de-cluttered.

5. Everyone makes mistakes. Most of the time. And that includes your carefully crafted (call to action) form so make sure the error handling is world class. It’s a trick called “soft-erroring” and works by never actually producing a traditional error. When an error occurs the visitor is gently guided to a “thanks but” page which gently tells the visitor we need a little more information and the reasons why and maybe makes a suggestion or two. With testing you’ll produce better forms in the first place.

This IS a killer. You can easily see a doubling in first-time conversion if the information you require is difficult to acquire (telephone number, for example) or the question is complex.

In fact, we recently ran a test on a client web-site that had, over time, morphed into a rambling page with various (ahem) marketing items starting to push (important) calls to action down the page. This has reached a point where the [Submit] button on the main CTA form actually fell under the fold. Shock, horror, “burn them” I can hear you crying. However, when tested with the button above and below the fold is made no statistical difference.

This didn’t worry us – we believe the results, not the opinions of experts – and it seems we are not alone in terms of myth-busting. What should we learn from this? Always test your ideas and then re-test them again frequently.

The copy approach could certainly work, it would be certainly cheap and (sort of) embraces the “crowdsourcing” philosophy that is gaining traction. However, if you are only as good as the next guy then you are simply “keeping up with the neighbours”. It’s not going to give you a competitive advantage unless your competitors are blind; which many probably still are.

True innovation, tested properly, can give a company competitive advantage if properly implemented – so real thought-leading organisations need to plough their own development and testing furrow whilst keeping an eye out for what others (not just competitors) are doing.

Not testing yet? You’re screwed, probably, even if you’re not screwed you’re certainly handicapping your marketing efforts.

How does A/B testing work?

The idea is to change some part of your website and see if the new version does better than the old.  A good place to start is just changing words – maybe a headline. You call the existing version “A” and the new version “B“.

You already know that the page is converting at 2% so you could just try the new headline for a few days and check the new conversion rate.

Trouble with this method is that conversion rates change from day to day. You can guess at some of the reasons: weather, day-of-week, competitor activity.  So you may be trying a bad headline on a good day or a great headline on a lousy day – and make the wrong decision about the headline.

Focus on what has changed

We can remove these other factors from the test. We show A and B at the same time. Software randomly decides if a new visitor to the site is going to see A or see B, then tracks visitors through to conversion and measures if A or B is doing better.

• If it’s raining – people will see both A and B.
• If it’s sunny – people will see both A and B.
• If a rival starts a price war – people will see both A and B.

You have removed the other factors.

Is it significant?

We use a statistical test to check that results are significant – and not just random effects. Sometimes improvements are small, but still worth having. Several 2-3% increases start to add up.

Is it just Connected doing split testing?

Amazon, Zappos, Google, eBay, EasyJet – all the big names use A/B split testing. Join the big boys – with our help.

So in many ways it’s been a rear-guard action as we’ve had to fight against the client’s, co-supplier’s and consultant’s wishes to drive home the philosophy of continual testing and letting the masses decide which page, landing zone, micro-site, call-to-action and design is better.

I’d be lying if I said it wasn’t a hard slog and at a number of points we were on the point of giving up but after 2 months showing the results of real numbers and metrics they’re sold and quite rightly they should after seeing a dramatic upswing in conversion rate.

We’re in the final throes of putting together a case study on this client and I hope it will be ready sometime in April so if you want to reserve a PDF version of the case study now then drop me an email and I’ll make sure you get one of the first batch.

This will be the first time we’ve openly shared so much information about how we work and we’ll be showing actual test results with annotated screenshots. The case study is part of our 64Monkeys projects that was kicked-off in 2007 as an internal knowledge base holding the planning and results behind every test we’ve carried out (around 4,500 so far). We’re planning an alpha roll-out of 64Monkeys later this year to openly share and collaborate with our clients and encourage a great deal of cross-fertilisation.

These are all questions typically running around marketing departments at the moment and the stock answer is to “get a new one designed” on the basis that it must be better than the old one as it’s newer and we’ve learned things (have we?) about the current versions of the landing pages.

Trial and error economics

Don’t guess, don’t risk the return on the heavy-lifters. Your current series of pages act as a ‘banker’ – put simply the new stuff has to race the best of the best you have already. When testing the new pages against the current bankers use a reliable testing method (such as the null hypothesis) that gives results you can be confident in. If you can be confident in the results then you can queue up hundreds of ideas to be tested and leave the testing harness to do the hard work or evaluating the changes.

The risk is low because:

• Pages that perform a lot worse than the banker will show up as failing very quickly and can be removed from the test quickly.
• Pages that perform a lot better than the banker will show up as succeeding very quickly and can replace the banker quickly.
• Pages that perform similarly will take a lot longer to determine their value but as they are not hurting (or helping) the conversion rate there is no loss associated with leaving them in test except the loss of the opportunity to run another test.
• Testing small changes can help with specific learning. For example, the data-entry form might perform better with a solid blue background versus white. This is real learning and can be applied (after testing) across other landing pages.

In fact, taken as a continuous process (Kaizen) optimising the landing pages can be the easiest and fastest way to continually improve a site, although not very glamorous for a marketing department.

About VITES™ as a testing tool

Split testing using the null hypothesis is built-into the core of VITES™ and offers a fast, reliable and repeatable test harness. Testing can be done via profile, traffic type, campaign or any other superset of visitor data (postcode range, for example) and is not limited to A/B testing with support for 26 concurrent tests running in each profile.  Traffic can be split in any range, typically the fastest results are achieved using a 50:50 ratio in an A/B test but 80:20 tests are commonly used when clients are nervous about radical changes to high net-worth landing pages.