We’ve just finished our financial year and, feck me, it’s been a busy old year. I’ll do a fuller post highlighting how the year went in more detail but the big thing we’ve seen this year is the explosive growth and (almost) 100% adoption of web site testing. From just 35% of our clients embracing testing in our last FY.

The growth has been so dramatic that we actually completed over 250 individual tests in May, June and July versus just 100 in the same period last year. So what is being tested and what’s working?

1. Page load speed is still an important factor. Landing pages, especially, benefit from slimming classes and whilst the need to have skinny 16k versions seems to be less critical than last year the target is still under 40k as you can expect to achieve twice the conversion versus a 120k page.

2. Relevancy is now a prerequisite. Dedicated landing pages for advertising terms out perform generic home pages by a factor of 2 and edge in front of deep-links by 25%. It’s worth double checking your traffic supplier or Adwords campaign to ensure you don’t have any old orphan campaigns that go to non-dedicated pages, far too many still slip through the net and this costs your conversion rate heavily.

3. Video is starting to make in-roads into certain areas and, critically, at certain times of the day and to certain audiences. In a group of personalised tests we found that certain groups of visitors (eg those that have generic hotmail, MSN or yahoo email addresses) respond far better to video-based pages than standard landing versions. Its not just video, the time to start using profiling pro-actively has arrived and start serving differing landing and content pages to distinct groups. Personalized targeting has shown a typical 10% lift in conversion over non-personalized content.

4. Numbers rock. Testing quantitative versus qualitative in copy is starting to show that using hard numbers work much better than general (soft) marketing messaging. Be mindful that the numbers you use have to be impressive in the context of the visitors needs.

The biggest lesson is the big picture here, continual testing shows continual improvement without the expected plateau that you would think occur. In fact the reverse is true, testing a little can be dangerous as you run the risk of testing items that are too small or too insignificant – think big, be brave and keep a close eye on the numbers so you can cut failed tests and rev-up the successful ones.

I expect our testing load to double in the next year. Bring it on

Most organisations, especially marketing-driven ones, are not familiar with one of the greatest web weapons available, namely continual testing. Traditional departments hum their day away thinking of great new “marketing” initiatives and wish they had invented the Meerkat or the two annoying blokes on the Safestyle Windows advert.

Most of what they produce is low-grade, recycled, tat. And that’s a shame.

Some of the brightest people I have met worked in marketing departments, but mostly they weren’t thinking about marketing, they were thinking about operational needs, budgets and getting out the latest offer in the post, by Friday. Ring any bells?

Most senior management and executive boards are not really that keen to sign-off on big changes, especially when “it’s the brand you’re playing with, don’t you know”. So here is a quick one paragraph trick to get them listening and, more importantly, keen to try something new.

Tell them “every company that tried online testing with their website saw at least a 20% improvement in conversion rates, in a month” and “that equates to £100k per month” – but obviously use a figure that represents 1/5th of your online turnover or if you are in the data acquisition world 1/5th of the total number of data items the online world generates for you. That should get their attention! Next tell them to half your salary and pay you a bonus based on performance with the condition that when the online conversion rate has doubled they have to pay you double your old salary. It that doesn’t catch them, which it bloody well should, contact me and I’ll organize a free test for you.

The results speak for themselves and as almost everyone is now getting into feverish testing you’d better start testing…or update your CV.

The absence of many of the senses (touch, smell, taste and largely, hear) when interacting with t’interweb is a challenge that at some point needs addressing if we are to replicate the real world into the virtual world.

Lots of studies have shown that these senses greatly influence our decisions and if virtual environments seek to achieve comparable success to the real world it won’t be overlooked for long.

Sound

On the face of it, sounds easy to deal with – simply play a backing track on a page and you have sound except that’s not the whole story. We use sound in a far more complicated manner and how we perceive or react to sound also varies depending on what we are doing.

We’re going to start testing using sound as an aid to navigation, initially it will be as simple as small “click and ding” cues to assure site visitors that actions they are trying to do, have, in fact been achieved. Sound is, potentially, the easiest sense to deal with as most browsing devices have a speaker, very few have “smell-o-vision” and I’m not aware of any device that produces taste (urgh!).

Touch

Is an interesting one. The growth of touch-enabled devices such as the iPad and other touch screen tablets shows some promise. Joshua Ackerman of MIT suggests that our use of tactile concepts in metaphors that relate to behaviour might influence our judgement which provides an interesting basis to start thinking about how we might use tactile feedback to “convince” a web site visitor. There is not yet a way for a site to directly feedback touch but there most certainly is a way to read touch via devices that have accelerometers installed. In fact simple examples of this exist already on, for example, the iPhone that can carry out actions when shaken, turned or moved.

Early work has started on meta-materials that can impart different touch sensations such as rough, smooth, rippled and whilst this work has an early home working with vision-impaired people the simple idea that a site can “talk” to you through the feeling of the site as your finger or palm wanders over the page is appealing, if not a little ‘Star Trek’.

This is the practice of hiring a group of mainly poor kids to ply their way through the myriad of online games (Everquake, World of Warcraft et al) collecting things of value to other gamers such as gold, potions, weapons etc. These items are then traded, via a broker, for real money to players that really can’t be arsed to go collecting or alternatively want to short-cut the tedious lower levers of these games.

The online gaming community really don’t like these “gold farmers” and tend to hound them pretty hard and even kill them (virtually) if the game allows it.

It’s quite a step to think that an online game has such a black economy, especially as most games are not actively policed that hard and rely on crowd-policing to deal with problems.

So what’s this got to do with the commercial world?

At the moment very little except to say that this is an example of entrepreneurship in terms of brokering items of value. As we start to see the rise of proper interconnected social networks who’s to say that “information farmers” cannot carve out their place in the information economy.

The more we live our life in public the more this information is freely available but time consuming to acquire.

Here is a brief outline of what you can expect in the next release of the worlds first, commercially available, personalisation and customer journey platform

• Faster core platform, requiring less server computing power and faster serving of content
• In-built AB testing functions, faster, easier testing of content, pages and call-to-actions
• Server load balancing, giving higher system availability, improved fault tolerance and improved performance
• Off the shelf CMS support, de-skilling and speeding up changes to content
• Faster profile management, faster and easier creation of new customer journeys
• Reporting API, simplifying the export of business-critical data giving easier and faster access to real knowledge
• New User Group to support discussions, bug-tracking, feature request and cross-learning between clients

Initially released in 2006, VITES was designed to dramatically improve on and off site conversion rates by providing a scaleable platform that offered proper customer journey management (ala Amazon, Ebay etc) combined with a suite of testing tools that allowed accurate testing of new content, CTAs and traffic streams

Since it’s release, every client using the platform has seen at least a doubling of conversion rates and huge reductions in cost per enquiry/sale

The latest release is a ground-up rethink of what our clients and marketplace needed and part of this was a massive simplification in deployment of changes, testing and profiles

All current clients are on a migration plan to complete the porting to the new version by the end of 2010 and all new clients will automatically get the latest version of the platform

License charges remain unchanged, starting at just £500 per month for the basic 10k users per month version

Contact Liam, Martin or Nick now to find out more about how VITES 3.0 (Rangoon) can supercharge your web strategy.

VITES remains the only commercially available off the shelf journey profile and testing platform

Contact our licensing team (Liam, Martin or Nick) for further information.

System integrators and agencies should contact our CEO (Martin Dower) to discuss how VITES can help your clients

We’ve all heard the “lies, damn lies and statistics” quote and still, all too frequently, statistics have been made “to fit” a required outcome

Most people know that this is wrong but still blindly accept the results, probably from a lack of their own self-confidence to question the results

The point still remains the same and the goal is unchanged; we want to better understand and predict the world around us and mathematics offers us the clearest path to that understanding What’s lacking, quite often, is the right information required to correctly analyse a situation and come to a correct answer

Probability 101

Q. I have two children and one is a boy, what is the probability that I have boys?

Common sense tells me that the other child has (for the purposes of this experiment) a 50/50 chance of being either gender so the the common sense answer would be 1/2. Except that this is not true as is has a precedent (I already have one boy). The possible combinations of children are BG, GB, BB or GG and since I already have one boy this removes GG from the equation leaving the probability as 1/3 of having two boys.

Probability 102

Q. I have two children and one is boy born on Tuesday. What is the probability that I have two boys?

Again, common sense suggests that it will be the same as above, why would the day of birth make any difference to the statistical outcome. But it does.

Let’s, using the above naming convention, call a boy born on a Tuesday a BTu. This gives the following scenarios.

* When the first child is a BTu and the second is girl born on any day of the week there are SEVEN possibilities.

* When the first child is a girl born on any day of the week and the second is a BTu there is an additional SEVEN possibilities.

* When the first child is a BTu and the second is a boy born on any day of the week then, again, there are SEVEN possibilities.

* Finally, there is a situation where the first child is a boy born on any day of the week and the second child is a BTu. Again there are seven possibilities but, and here it gets interesting, one of them has been counted before so there are only SIX possibilities.

Counting likely outcomes we then have a total of 7+7+7+6=27 different combinations and 13 of them include two boys the answer is 13/27, wildly different to the 18/27 (1/3) answer to the first question. This is surprisingly odd and (entertainingly) illustrates that seemingly unconnected pieces of information can make a huge and statistically very important difference to outcomes.Whilst this post is folly of sorts it does have a serious side. When you are trying to measure information to produce meaningful outcomes you really must be very careful to decide what to include and what to exclude. And, you must have a grasp of how to use the information correctly, even if the mathematics required were learned when you were 13.

The oft misused phrase that “information is power” generates some pretty big headaches for organisations. Just by gathering information on, for example, web-site activity suddenly turns ageing IT and obsolete marketing departments into “great houses of power”. Funny, eh?

One of the great dilemmas facing organisations embracing the web today is not information poverty, as it was in the late 90s but information overload as we trudge, neck-deep in data. It is quite surprising that organisations have not recognised this more and look to block this ever-increasing problem.

The simplest answer is to hire a Chief Information Officer (CIO) who has great background in statistics, data mining and a focus on the commercial aspects of the organisations. There is a skill in converting information into knowledge and, sadly, even the early generation of CIOs are not quite there yet. They need more power, more budget and more say in the strategy of the organisations.

So as most organisations are forced to live in the information age the really bright ones are embracing the knowledge age.

In a word, screwed. The great downside of client-side analytics such as GA is simply that, the tracking is done directly on the visitors’ machine and not at the heart of the web-site and whilst this makes it easy to deploy it also allows it to be easily circumvented. The ideal solution is server-side recording but that’s more involved, more tricky to deploy and requires wider skills and (usually) a, paid-for, commercial solution.

These are all questions typically running around marketing departments at the moment and the stock answer is to “get a new one designed” on the basis that it must be better than the old one as it’s newer and we’ve learned things (have we?) about the current versions of the landing pages.

Trial and error economics

Don’t guess, don’t risk the return on the heavy-lifters. Your current series of pages act as a ‘banker’ – put simply the new stuff has to race the best of the best you have already. When testing the new pages against the current bankers use a reliable testing method (such as the null hypothesis) that gives results you can be confident in. If you can be confident in the results then you can queue up hundreds of ideas to be tested and leave the testing harness to do the hard work or evaluating the changes.

The risk is low because:

• Pages that perform a lot worse than the banker will show up as failing very quickly and can be removed from the test quickly.
• Pages that perform a lot better than the banker will show up as succeeding very quickly and can replace the banker quickly.
• Pages that perform similarly will take a lot longer to determine their value but as they are not hurting (or helping) the conversion rate there is no loss associated with leaving them in test except the loss of the opportunity to run another test.
• Testing small changes can help with specific learning. For example, the data-entry form might perform better with a solid blue background versus white. This is real learning and can be applied (after testing) across other landing pages.

In fact, taken as a continuous process (Kaizen) optimising the landing pages can be the easiest and fastest way to continually improve a site, although not very glamorous for a marketing department.

About VITES™ as a testing tool

Split testing using the null hypothesis is built-into the core of VITES™ and offers a fast, reliable and repeatable test harness. Testing can be done via profile, traffic type, campaign or any other superset of visitor data (postcode range, for example) and is not limited to A/B testing with support for 26 concurrent tests running in each profile.  Traffic can be split in any range, typically the fastest results are achieved using a 50:50 ratio in an A/B test but 80:20 tests are commonly used when clients are nervous about radical changes to high net-worth landing pages.

The art of improvement is simple; make a change and then test it. If it’s better then use the latest version as the best and then cycle round again. The compounding effect of making lots of small improvements gives a number of key benefits:

• Continual improvement drives the performance up, relentlessly
• A frequent change culture encourages the testing of lots of ideas, sometimes the daftest or smallest of ideas have a major impact on performance
• Reduction of risk, a poorer performing idea is quickly and efficiently scrapped
• Much wider pool of thinking, more ideas from more people adheres closely to the concept of “the wisdom of the masses”
• Avoids the “Highest paid persons opinion” (HiPPO) quandary where decisions are made at the wrong level by the wrong people
• Compounding small improvement generates a huge improvement over time
• Real learning is possible and this helps the feedback loop for the next test
• No idea is right or wrong – it simply generates learning

The downside of kaizen in the digital world is that most web-site environments are simply not built to allow the kind of rapid change, test function that is required to get the most from this approach. VITES™ at it’s core is built around this philosophy.

Trial and error culture

Connected adopt a Kaizen approach to ongoing development and conversion rates where staying agile and making lots and lots of small changes – each time testing them to measure success – and then moving on again. Connected offer a number of services to help make this happen:-

• Fixed Kaizen contracts. A fixed number of tests or hours in a given period with a planned target to reach.
• A/B testing suites. Utilising the VITES™ platform to provide accurate, reliable A/B testing.
• Profile/persona-based testing. VITES™ supports unlimited profiles and profiles types, each can be tested independently or raced against a “banker”.
• Micro-site build & development. Avoiding the need to completely re-develop a client web-site and provide much of the improvement via a micro or landing-page site.
• Business Intelligence reporting. VITES™ Passport ™ provides a huge mine of data that can be accessed to provide unique learning. Provided as a service or as an API.
• Landing-page heavy lifting projects. The key first-wins for most commercial traffic sites, provided as a service, turnkey or consultancy.
• Performance contracts. Fixed target-based contract rewarded around specific goals and milestones.

Connected provide these services either standalone or linked with existing build-and-manage contracts.

What results can you expect?

Depending on the start point most clients can expect to achieve a doubling of conversion rates during the first year. The law of diminishing returns can apply and often the major advances are made in the early part of the project. Pushing down the “long tail” still has great value and the ongoing refining of the solution frequently uncovers surprises and big jumps on online conversion rates.

Keen to find out more?

Thinking of entering this world? Talk to Liam, Nick or Martin on 0845 051 4228 and he’ll sit down with you and go through the various options available and how to best approach improving your digital offering.