Testing: Who, What, Where, When, Why, How?!

When it comes to increasing your email marketing results, testing is the king!(Check out my blog post here all about testing and my 5 steps to get started here >)But what exactly is testing when it comes to email marketing?Well we have two main options:A/B Split testingThis is the most common form of testing in email marketing and is the process of taking 2 variations of your test element and seeing which variation ‘wins’. You choose a winner based on your hypothesis and analysing your statistics.Let’s take a look at this example to illustrate how these types of tests work (these are real life statistics from a test that I recently ran on one of Shine a Light Media’s emails). In this case I was testing the subject line element of my emails to see which produced the highest open and click rates:Results:

Subject Line	Open %	Click %	Complaint %	Unsubscribes
Hi *|FNAME|* I need your help	31.5%	5.6%	0.0%	0.5%	WINNER!
*|FNAME|* I want to get to know you better!	19.7%	4.2%	0.0%	0%

 It’s really important to make sure you look at all of your statistics together after running a test and before deciding a winner, and not just one metric in isolation, in order to have a true view of how the test performed. For example, I’ve seen subject line tests run where the open rate decreased but conversions actually increased. Multivariate testingThis is the process of testing more than one change to an individual element at the same time. With this type of testing you have a base version of your email that follows your usual strategy to benchmark an uplift or decrease against, and multiple different versions.Let’s look at another example (these statistics are not real and just for illustration purposes), again based on a subject line test:

Variation	Open %	Click %	Complaint %	Unsubscribes
Base	35%	5.4%	0.01%	1.1%	This is the normal version of your email that we are benchmarking the alternate variations against
1	42%	6.3%	0.02%	0.8%
2	41.5%	6.1%	0.01%	0.4%	WINNER!
3	31%	4.2%	0.06%	0.9%
4	26.2%	4.6%	0.01%	1.3%
5	43.6%	3.4%	0.01%	1.2%

 So now you know the ways in which you can test your emails, here are some ideas of elements to test in your emails:

• Subject Line
• From Address
• Send day/time
• Design
• Call to action; placement, colour, wording
• Type of content served
• Personalisation
• Imagery

And don’t forget, you should also be testing your sign up forms and landing pages too; if you want to increase conversions for example, you need to look at the whole process of how a subscriber makes the journey from your email content through to the purchase process.Now I want to hear from you… what was the last test you ran? Did you experience an uplift in your results? What was your key takeaway from testing? 

---

Testing tells you what happened

Whether your team can act on it consistently is a different question, and it is the one that decides whether results compound. The free CRM Maturity Assessment scores that capability across six areas in about 15 minutes.