Which Test Won and Testing Email Templates

At Corelio Publishing, a leading online and offline news publisher in Belgium, we’re testing our newsletter templates endlessly. With several 100k emails a day even the smallest change could mean a huge difference in the Click Through Rate and traffic to our websites.

Although A/B testing with email templates is not that hard, it demands effort, time, the right tools and a decent preparation. After a period of trial and error we made progress, improved our templates and finally achieved a considerably higher CTR. All this hard work didn’t go unnoticed. That is why …

I’m very happy and proud to announce that we have won this year’s Email Testing Award with Which Test Won for our A/B testing methods and results.

“This test received a gold award because it did a phenomenal job decluttering the newsletter while providing the same number of stories via text links”
— Anne Holland of Which Test Won

For those who don’t know Which Test Won yet, it’s a team of independent journalists (supported by many industry-leading sponsors) and the #1 weekly evangelizing in conversion optimization — in particular through A/B and multivariate testing. Every week they publish the results of a real A/B test without mentioning the winning version. It’s up to the visitor and his or her gut feeling to pick out the winner of the test. Believe you me, it ain’t easy.

Our case study at Which Test Won

Do you want to give it a try as well? Then read our case study at whichtestwon.com and find out which newsletter version improved the CTR with 36%.

Because this case study is pretty limited in description I will write a more detailed article later on. I will explain our testing methods, the test setup we’re using, which metrics really matter and how you can start improving your emails.