Do You Have Too Many Variates in Your MultiVariate Tests?

Do You Have Too Many Variates in Your MultiVariate Tests?

Do you use Google Website Optimizer or some other multivariate testing tool?

Do you know what Multivariate testing is?

Do you know what A/B or Split testing is?

Testing is the only way to know what is working and what is not working, you can't guess, you have to track it.

Let's start with A/B Split testing. If you have a sales page or a squeeze page or a signup page on your web site, you can make two versions of the page and using a split testing tool such as Google Web Site Optimizer, display one version of the page to half the visitors and the other version of the page to the other half of the visitors. The split testing tool will track which version of the page is more successful at making sales, or getting signups, or whatever the purpose of the page. You can then take the winner and test it against something else.

So now that you understand A/B Split test, let's dive into multivariate testing. Google Web Site Optimizer also supports Multivariate testing. Multivariate testing allows you to test multiple versions of various portions of your page at the same time. For example you can test several headlines, a few versions of a product photo, different font sizes or colors, different copy, and the list goes on and on. This is important because you can maximize results much faster than performing a sequence of A/B split tests.

Multivariate is also important because a certain combination of items may perform much better together than alone. For example a certain headline may perform well, and a certain photo may perform well, but when used together they may perform many times better. Finding that combination with split testing may take so long that it's practically impossible. Multivariate testing allows that combination to bubble up much faster.

But what are the inherent dangers of multivariate testing? For me, my nature is to try too many variations. I tend to put too many variates in my multivariate testing! Why is this a problem? Because if I have 6 versions of a headline, 2 versions of a logo, 2 versions of body copy, and 6 background colors and 2 border styles to test, that equals 288 combinations! It takes a lot of traffic to thoroughly test 288 combinations.

Test the major things first.
I find it much better to get a winning combination of headline, copy, and photo or video (if you have a product photo or video) first. Once you have gotten a few combinations that do well, remove the losers and add in some of the more mundane tests such as background color and font color.

I know that some people will tell me that font color and background color will and can make a huge difference in conversion so I should not call them mundane. Yes, they can make a big difference, that's why we need to test them, but unless you have a lot of traffic to your site, I think it's better to "stage" your testing. Unless you're testing something "normal" for a background such as white against pea green, then you're much better off to test the major things like headlines first.

I hope this helps a few people to have more success.

If you need help understanding how to set up multivariate tests in Google Web Site Optimizer and think I should create a product on how to do that, leave a comment saying so. Or, if you've had a really good or really bad experience with a particular tool, leave a comment about that as well.

Until next time,
Fred

Get Free Updates! Enter your name and e-mail address to receive a short notice each time I make a new post. First Name: Last Name: E-Mail Address: E-Mail again: NOTE: You will receive a confirmation email. You must click the link in the email to activate your free updates. Please check your spam folder(s) if you don't receive the email.