Test what content generates greater engagement among your Profiles. What color is better on the button? Where should I place the link? What image generates more clicks?

A/B testing compares different aspects of your Email on smaller groups. The winning version can then be sent to a wider audience. Set up A/B test after selecting Subscription to send to.

For Profile to be a part of an A/B test, the Profile must exist in the account before the Email is scheduled. Otherwise the Profile will not be included in the A/B test.

You can do an A/B test on either Email design or Send time.

You prepare Email content versions in the Email editor. In the A/B test step, you set up the test: define the splits and add Email content versions.

The All Profiles in the sending will be randomly divided into groups, and receive the different content versions. When testing sender time, all Profiles receive the same content but at different times.

By studying the result of which group performed higher engagement (opens and clicks), you better understand your Profiles preferences.

Example:

Test which call to action (CTA) button color generates higher engagement: pink or cyan.

The different Emails versions for an A/B test is created in the same Email. The specific versions are then defined similarly to how you would segment an Email.

To create a version, you select Row(s) and connect them to a version.

Rows with no version connected to them, will be displayed in all Email versions.

1. Create an Email in the Email editor, as you would for a regular sending. Include all content you are planning to send (for all versions).

2. Time to define, what Row belongs to what version.
​Define "version 1" by selecting a Row in the structure panel to the left.

3. Open the A/B Test Versions option, in the Row settings on the right.

4. Define the selected Row as part of a version.

Add the version name (for example “version1”), and select it.

5. Define version 2 by selecting a Row you want to display in version 2.

In Row settings, create another version name (for example “version 2”) and select it.

6. Continue to connect a version to all Rows that belong to a certain version.

7. Use Preview to visualise what the different versions look like.

Rows with no version = displayed to all

Keep in mind that Rows that are not connected to a certain version, will be part of the general email and displayed to all recipients.

Adjust Email settings. If you have selected a bulk Email to use for the A/B test, these settings will no longer be editable once leaving the editor. Sender name, subject line and preheader can then be edited on the A/B test page.

8. When definition of versions is completed, go to the Next step.

9. Select Subscription and proceed to the A/B testing step.

Select Split type in the dropdown.

Select number of splits, number of groups you want in you test.

Click Calculate now to see how many Profiles match the setting.

Use the percentage sliders to define sizes of the splits. To lock a value, click the padlock.

Set Email details for the different splits: Subject line, preheader and sender name.

Select Content version for each split.

In a A/B test without winner, click Next to proceed to scheduling sendout.

Schedule the sending of the A/B test. We recommend a minimum of 3 hours from the moment of the scheduling.

If performing a A/B test with winner, click the Define winner criteria button.

Clicking the Cancel split-sending button, will cancel the entire A/B test and the Email will be sent as a regular sending.

Select winner criteria in the dropdown.

Define when to consider it a tie. How small should the difference be in order to consider the versions to have performed equally?

Setting the majority condition to 5% would mean that its a tie if, the difference between the top two splits is less than 5% from each other.

If you prefer to use a simple majority select 0%.
​

Select version to send to the remaining Profiles, in case of a tie.

Decide number of hours to pass before sending out the winning version, after conditions are met.

To edit the split settings, click the Back to splits button. To proceed, click Next step.

Schedule send time.

On the Overview page, you can view send time and split sizes.
​

Click Activate to activate the A/B test.

When performing an A/B test without winner, you study the results in the Email Report.

In the Report Overview for an A/B test with winner, the winning version is marked in the top graphs (based on the criteria you defined). This section presents the delivered and bounces Emails, in numbers and percentage.

In the Email breakdown you can study the recipients interaction with your Email: Open, clicks and unsubscribes. Unique opens and unique clicks counts the number of individual recipients interacting. A recipient opening the Email is registered as "Opens (unique) =1, and remains1 regardless of how many the recipient opens the Email.

Click the information icon next to the title read explanations.

Below the Email details and time for the different splits are presented.

Under this tab you view the Email and toggle between desktop, mobile and text view.

Under the Clicks tab you can see how many and which Profiles interacted with what links. Here you also find the link IDs.

Under the Bounces tab you can study bounces. Click the View Profiles button to view what Profiles did not receive the Email and for what reason (bounce type).

1. Select Split type: Sender time, in the dropdown.

2. Select number of groups you want to split the Profiles into.

3. Set the distribution of Profiles between the different groups with the sliders.

4. Schedule sending time for each group (expand with the green arrow on the right).

4. Expand the section with the arrow and schedule time for each Split 1.

5. For the remaining Split(s), schedule send time in relation to Split 1, number of minutes, hours, days or weeks after first sending.

6. Choose also whether to include new Profiles.

7. Click Next to proceed to the Details page, where you look over Subject line, Preheader and sender information.

Click Skip if you prefer to skip the splitting and proceed regularly with your sending.

8. On the Details page, enter Email details for split version 2.

9. Click Next.

10. Control the Split test set on the Overview page.

As this type of sending doesn't have a winning version, you have to study the results in the Email Report.

To clone the A/B test, select the test on the Email start page and click Clone in the bottom bar. The whole A/B test activity is cloned. Adjust send time.

When cancelling an A/B test the sending will revert to a regular Email sending. Control the Email content as Rows with A/B test version added to them, will not be displayed to anyone.

Create and send Email - Step by step

Email Report