Do You Have Too Many Variates in Your MultiVariate Tests?
Update 11/2017:
Google Website Optimizer was shuttered in 2012 by Google. Other than that, this article is still spot on! There are plenty of other testing platforms out there. One of the best to use as an all-around awesome tool to promote and grow your business is Kartra. Kartra has all the tools and training you need to succeed, including split testing.
Do you use Google Website Optimizer or some other multivariate testing tool?
Do you know what Multivariate testing is?
Do you know what A/B or Split testing is?
Testing is the only way to know what is working and what is not working, you can’t guess, you have to track it.
Let’s start with A/B Split testing. If you have a sales page or a squeeze page or a signup page (anything really) on your web site, you can make two versions of the page and using a split testing tool such, display one version of the page to half the visitors and the other version of the page to the other half of the visitors. The split testing tool will track which version of the page is more successful at making sales, or getting signups, or whatever the purpose of the page (you have to have a “goal”, some target action such as a signup, or a sale, or even a click that can be measured). You can then take the winner and test it against something else.
So now that you understand A/B Split test, let’s dive into multivariate testing. Google Web Site Optimizer also supported Multivariate testing (not all tools do and that’s OK). Multivariate testing allows you to test multiple versions of various portions of your page at the same time. For example you can test several headlines, a few versions of a product photo, different font sizes or colors, different copy, and the list goes on and on. This is important because you can maximize results much faster than performing a sequence of A/B split tests. But this requires more traffic than a simple A/B test.
Multivariate is also important because a certain combination of items may perform much better together than alone. For example a certain headline may perform well, and a certain photo may perform well, but when used together they may perform many times better. Finding that combination with split testing may take so long that it’s practically impossible. In an ideal environment, multivariate testing allows that combination to bubble up much faster.
But what are the inherent dangers of multivariate testing? For me, my nature is to try too many variations. I tend to put too many variables in my multivariate testing! Why is this a problem? Because if I have 6 versions of a headline, 2 versions of a logo, 2 versions of body copy, and 6 background colors and 2 border styles to test, that equals 288 combinations! It takes a lot of traffic to thoroughly test 288 combinations.
Test the major things first: I find it much better to get a winning combination of headline, copy, and photo or video (if you have a product photo or video) first. Once you have gotten a few combinations that do well, remove the losers and add in some of the more mundane tests such as background color and font color.
I know that some people will tell me that font color and background color will and can make a huge difference in conversion so I should not call them mundane. Yes, they can make a big difference, that’s why we need to test them, but unless you have a lot of traffic to your site, I think it’s better to “stage” or sequence your testing. Unless you’re testing something “normal” for a background, such as white, against pea green, then you’re much better off to test the major things like headlines first.
I hope this helps a few people to have more success.
Until next time,
Fred
[addtoany]