How to test if adding video to email improves performance
Does adding video to email improve performance?
Given the significant additional cost incurred when adding video to your email campaigns, it is paramount that the investment is delivering value. As Video Email gains traction, claims of performance improvement, from adding video to email are prevalent. Here at Video Email we are advocates of testing, analysing and re-testing to understand the impact of adding video to email.
To assess the value of video email we need to compare its performance to campaigns that do not feature video content and feature the same message, or offer.
All the Email Service Providers have A/B split functionality which is perfect for testing the performance of video email campaigns.
Adding video to email significantly increases the cost of email campaigns so it is critical to understand if this additional spend is delivering incremental sales, and a positive return on investment.
From our extensive testing of video email campaigns we have identified four tests which provide all the data you’ll need to assess the performance of video email marketing and ascertain whether it delivers sufficient value to become an integral part of your email marketing activity.
A/B Testing Framework
When we conduct these tests we use the following A/B testing framework.
To ensure the test campaigns delivers a statistically significant result ensure the A and B test audiences are:
- Randomised, to avoid any bias.
- Of sufficient size, mindful that the audience you are measuring are the openers of the emails who will see the video or still image.
- Sent at the same time.
When assessing the test campaigns performance the key performance indicators to appraise will be:
- Unique Click Through Rate
- Total Click through Rate
- Conversion Rate (be it purchase or a sign up to a form).
- Sales.
The four tests we run are:
Test 1. A/B split test of video email versus Still image
This test is designed to assess the performance of a campaign featuring a video in substitution for a static image.
All other content within the email should remain the same, so the only difference between the 2 emails (A and B) is that one has video content as opposed to a static image promoting the Offer/Promotion or service.
Test 2. Video email versus GIF
This test is designed to assess the performance of an email campaign featuring a video versus an email campaign featuring a GIF.
All other content within the email should remain the same, so the only difference between the 2 emails (A and B) is that one uses video to display content at the head of the email template and the second uses a GIF.
The content within both the video clip and the GIF are identical, so this test is looking to assess the performance of the format used to publish the content video versus GIF.
Test 3. GIF versus static image
This test is designed to assess any performance uplifts of replacing a Static image in an email template with a GIF. The hypothesis here is that a GIF drives more response than the static image as within the GIF we are able to include more content and also showcase this in a more engaging way.
Test 4. Product focussed video email versus offer focussed video email
Here we look to test the most responsive content for a video email.
Product focussed video emails feature video content that showcase the product and often include a demonstration of how it works or the benefit it delivers. This approach is favoured by cosmetic brands when showing before and after looks.
Offer focussed video emails feature video content that communicates promotional messages, which might include price related content or a free delivery period. This approach is favoured by fashion retailers, particularly during sale campaigns
Tests that we have conducted that you should not repeat.
Tests we have found to be marginal in terms of assisting with the assessment of how effective video email campaigns are:
- Including multiple videos in a campaign versus a single video – Single video always wins.
- Publishing videos at different aspect ratios within the email campaigns – delivers no data of value.
- Publishing the video in different positions within the email. We have conclusively found that publishing the video content at the head of the email, at full width of the email is the most effective position and format.
- Testing the hypothesis that recipients become more responsive to video email over time, following a period where they become familiar with seeing video in their emails and “learn” how to respond to them. We have emphatically proven this not to be the case, response rates have no relationship to the number of times a recipient has been exposed to video email campaigns.
