Conduct A/B-tests

 

This article describes the old Selzy email builder. You can find information about key features of the new email builder in the overview article available in English.

If you need help with the functionality described in this article but in the new builder and can’t find it in the overview article, please contact our Support Team.

Selzy Support Team is available for all users 24/7.

Running tests is the rule of thumb for a marketer.  Tests are the only tool that can let you know if you are getting the most out of your mailings or if there's room for improvement.

What Can You Test?

• Email Subject Line. What is the subject line that works best: personalized or not, a longer or a shorter one? Do you need a question mark at the end of the subject line? Should you add a call to action?
• Preheader (the initial text that in some services is displayed next to the subject line). Which version of the preheader works best?
• From Name. You can test how people respond to different e-mails and senders. For example, which from name will have a higher open rate: Ivan Rozhdestvensky, Ivan Anatolyevich, or Ivan (Selzy).
• Send Time. What time is the best to send a newsletter?
• Headers and Subheaders: do you need them and how do they influence redirections?
• Text Length. Would a longer or a shorter text work better?
• Design. Even if you believe it is better to use green color on a red background to communicate with your clients, you can also test this:). But seriously, there is a lot of room for experimenting.
• Images. Do you need them? Which are the options that increase your conversion rate?
• Do links or buttons work better?
• How many links do you need in an email?
• Where is it better to place the links?

Use the Split Test function to check those ideas and many others.  You can use it to send different versions of emails to a certain percentage of your subscribers and see which one will work better and should be sent to your main database.

How to Conduct the Test

Let's say we want to find out which button in the email will bring more clicks:

• Option 1: blue
• Option 2: red

This is what the button in the email looks like:

To test the color of the button, copy the message. To do this, go to Campaigns — Emails.

Click Create a Copy next to the message you need, and Go to Copy in the pop-up window.

In the copied email, click the New Email tab.

Change the color of the button in the editor, then click Save.

Now go to the Enter Subject and Address tab.

For clarity, we changed the subject of the message, now it is "Red Button Letter". Don't do it for a real test, for the subject line also affects the result. Follow the rule of one test per hypothesis.

Click Next and than Save and quit.

Then create the split test.  To do this, select Campaigns — A/B-test in the menu.

Click Create A/B test.

This will open a page with the split test parameters.

On this page:

1. Enter the split test  name.
2. Select the list to send the split test to.
3. Select the email options. In Selzy, you can test up to 10 options simultaneously.

4. Select the split test type:

• Without final part. In this case, all recipients will be distributed evenly among the emails that participate in the split test.
• With final part. An A/B-test is performed on a subset of your subscribers. The best performing option will go to the rest of your list.

5. Specify the winner criteria (if you chose  Waiting for the Final Part):

• biggest number of clicks (testing button colors/link location);
• highest open rate (testing the send time/the subject);
• select the winner manually.

In the example, we are testing which button users will click the most, so we will choose  the biggest number of clicks.

6. Specify the audience share for the test. In our example, we will select 20% of the total audience: 10% will get Option 1, another 10%, Option 1. Recipients are selected randomly. After the split test is completed, the winning letter will be automatically determined, and the remaining 80% of the audience will receive the best version of the letter. With the Select the Winner Manually option, you will need to determine the winner yourself.
7. Set the test duration. We recommend making it at least 3 hours. This is the optimal period for people to open their emails. If the audience is inactive (rarely checks email), make it 1 or 2 days.
8. Choose if you want to send your emails right now, or schedule them for a different time and date. For example, let's select Send Now.
9. Click Launch.

You're all set! The split test has been created.

You can see the results of the split test in the report. To view the report, select Campaigns — A/B-test.

Click on the split test you need to see the report and statistics for each email.

Congratulations, now you know how to do split testing!