Four Step Formula to Be Successful at Multivariate Testing
Multivariate Testing - MVT
Posted by Adam Hayes
Muiltvariate testing we help quickly increase conversions. Here are the four simple steps to implement multivariate testing.
We recently wrote a from the ground up, multivariate testing package to use with all of our upcoming websites. We've always believed in testing, testing, testing, and the iterative design process. However, normal split AB testing has become too slow at adapting to the speed of change necessary to keep margins high and conversions up. So we've jumped on the multivariate testing bandwagon and developed a quick way to test multiple changes at once.
Multivariate testing is nothing more than testing more than one (multi) change (variable) at the same time. Normally the scientific process tells us to only change one thing, test, and then evaluate results. Unfortunately this method is much too slow to be effective online. Plus we can see how multiple elements add or detract from one another in a quicker fashion.
The Multivariate Testing Process
The multivariate testing process is quite simple to understand.
- First you want to keep the experience consistent for the individual user, so you need a way to identify specific users and provide them with the same variables every time. Setting cookies or session varaibles for the user allows you to track and consistently display the same information to User X.
- Next you need a way to track which variables were shown to User X. We rewrote our tracking software to add the different variables to the stats of User X so we know that they were shown Heading A, Copy C, and Call to Action B while User Y was shown Heading B Copy C and Call to Action A.
- Third you need a way to identify success. Whether that is making it to a specific page, or signing up for a newsletter, or staying on the site for X amount of time. And each success needs to be linked to the specific test you are running.
- Last you need a way to compile successes and adapt to the findings. As you view your findings you can immediately remove ineffective variables from the test and replace them with modifications of the effective variables and rerun the test.
After building our testing package we immediately put it to the test. The amount of data you can collect is amazing. We've started with some simple changes and have loved the results so far. We will be heavily using multivariate testing in our new sites to identify more precisely how users react to specific varaibles.
Getting Off to a Good Start
Now, I'm not going to jump on the "we don't know anything until we've tested" bandwagon. I think it is a cop out. Good designers know good design when they see it. Good programmers know good programming when they see it. There are still basic "rules" that can be followed to improve conversions. The trick is to start testing good versus better scenarios instead of bad versus bad versus bad.
Leave Comment