UPC gets Emma award using multivariate testing in email campaign

UPC, a market leading Dutch provider of Internet, television and telephony sent an email to their most loyal clients with a personalized offer.
22Times, the agency of UPC created 40 different buttons to be tested: 4 different colors were tested, in combination with 10 different button texts.

4 different colors


10 different button texts

Activate now
Order now order now
More info
More information MoreInformation
My offers
My subscription scan
switch over now
Personal subscription scan
My personal subscription scan
Upgrade now

The Process:
22Times used the real time optimization service of 8Seconds to optimize this emailcampaign. When the entire emailcampaign has been sent out, 8Seconds showed these different buttons to the customers opening their email and measured which button worked best. The best converting button was found very early in the campaign thanks to the very consistent conversion behavior of the recipients.

The Results:
The 2 buttons with the lowest CTR are: and  , both with a CTR of 0%.  The readers of UPC apparently cannot be motivated to click by means of the pushy ‘order now’ text.
The button with the highest CTR (61.90% !)  is .

The Averages:

Let’s now have a look at some averages:

Looking at the chart above, it’s very clear that the purple button works better than the orange button, although many other campaigns indicated that orange is supposed to be a better colour for buttons.

But if we look at the performance of the texts on the buttons, then the differences are even sharper.

What can we learn from the chart above?
‘order now’, ‘upgrade now’, ‘switch now’,… are commands, and the customers of UPC do not want to get commands! They want to be addressed personally and they clearly expressed their wish to be invited to have a look at the personal offer that has been selected for them.  Giving the impression that UPC has done a personal scan to select the best offer without being too pushy increased conversion enormously.

This campaign was repeated several times, with very similar and consistent results. No wonder this email campaign is nominatid for the prestigious EMMA-award! Congrats to 22Times !!


9 Responses to UPC gets Emma award using multivariate testing in email campaign

  1. Great, straightforward test. To re-assure the statisticians about this test's validity–what was the sample size?

  2. The sample size was well over the 20K and we tested it several times, just to make sure.

  3. dicke Titten says:

    very intresting results, color makes the difference ;-)

  4. Pingback: Multivariate Testing 101: A Scientific Method Of Optimizing Design - Smashing Magazine

  5. Pingback: 8Seconds tests social and adds new features | the Email Optimization Company

  6. Veryi nteresting results. I find the difference in the usage of the text astonishing. Remarkably ‘my personal subscription scan’ scores significantly higher than ‘personal subscription scan’. Once again this shows that you really have to direct your mesaage at your customer.
    Thanks for sharing this information!

  7. Pingback: V12 Group Email Resources – When is it time to change your design?

  8. Pingback: Call to Action Series Case Study Boomerang Rentals

  9. Pingback: Call to Action Series: Wording

Leave a Reply

Your email address will not be published. Required fields are marked *