Look Inside a 1,024 Recipe Multivariate Experiment

Do small changes make a big difference? In the world of landing page testing, they certainly can. A few weeks ago, we ran one of the largest multivariate experiments ever: a 1,024 recipe experiment on 100% of our US-English homepage. Utilizing Google Website Optimizer, we made small changes to three sections on our homepage (see below), with the goal of increasing the number of people who signed up for an account. The results were impressive: the new page performed 15.7% better than the original, resulting in thousands more sign-ups and personalized views to the homepage every day.

Below is a screenshot of the homepage before the test and the three sections we focused on:

And here are examples of various fonts, messages, and colors we tested:

Section 1: (The words "Sign Up" were put in all caps)

Section 2: (We placed these bubble messages to the left of the word "Sign Up")

Section 3: (We experimented with these different banners in that area)

Can you guess which combination of elements performed the best?

(humming "Jeopardy" music)

If you guessed the Original "Sign Up" Font (section 1) + Red Bubble "Sign Up" (section 2) + White "Don't Just Watch, Participate" box (section 3), you win!

Winning Page:

While we could have hypothesized which elements result in greater conversions (for example, the color red is more eye-catching), multivariate testing reveals and proves the combinatorial impact of different configurations. Running tests like this also help guide our design process: instead of relying on our own ideas and intuition, you have a big part in steering us in the right direction. In fact, we plan on incorporating many of these elements in future evolutions of our homepage.

P.S. If you need assistance running tests on your website, there are many Google Website Optimizer authorized consultants who can help. Special thanks to VKI Studios who assisted on ours.