NOTE: This article covers a method of A/B Testing that predates automation; it has since been replaced by a preferred method. The preferred method of A/B Testing is documented in Create an A/B Testing Campaign.

A/B Testing allows you to send different versions of the same message (e.g., sample A and sample B – hence the name) to a small sample of Contacts to see which is opened more and/or results in a link being clicked. After the test messages are sent, you’ll have access to tracking reports to compare their performance.

The information provided by A/B Testing can be important to fine-tune your emails. For example, is one subject line more effective than another? Does sending at a specific time of day lead to better success?

NOTE: A/B Testing is a feature that must be enabled on your account. If it's not, create a case.

Permissions

When enabling A/B Testing for your account, your account’s administrator may determine that only certain users should be allowed to use this functionality. In order to send an A/B Test, a login must have the following permissions.

• Send Message to Group
• Create Groups

See User Permissions to learn how to set User Login permissions.

Create an A/B Test

1. Access the Admin interface.
2. Navigate to the Messages > A/B Testing > Send.
3. In the text field, give your A/B test a descriptive name, and then click GO.

Step 1 - Create the test & select Contacts

The first step is to create the test and select Contacts for the test messages.

1. Select the type of testing you would like to complete:

1. Same Message Different Subject Line - Select this if you're testing different subject lines.
2. Same Message Different "From Field" - Select this if you're testing different From addresses.
3. Different Messages - Select this if contacts are receiving two different messages.

2. Specify the number of Groups to test. For example, if you're testing two subject lines, specify 2, if you're testing three messages, specify 3, and so on.
3. Click Submit.

Step 2 - Segment the Groups

Now, you need to set up the group segmentation.

NOTE: At this point, don't make any changes to the groups you're testing and don't click the Back button in your browser. If you do either of these, you must delete your A/B Test and begin the configuration phase again.

If sending the same message with different "From Field" or Subject Line:

1. In the Source area:

1. Select your Source Group. This is the main group where all of your contacts reside, and the group from which the system will extract your random sample.
2. Select the message you want to test.

2. For each Target Group:

1. Enter your sample size, using either a percentage or actual number.
2. Select to add the random sampling to an existing group OR new group (if new, enter a descriptive name).

3. Click Submit.

If sending different messages:

1. In the Source area, select your Source Group. This is the main group where all of your contacts reside, and the group from which the system will extract your random sample.
2. For each Target Group:

1. Enter your sample size, using either a percentage or actual number.
2. Select to add the random sampling to an existing group OR new group (if new, enter a descriptive name).
3. Select the message you want to test for each group.

3. Click Submit.

Step 3 - Configure the message details for each Target Group

The final step is to configure the message details you want to test. If you're testing multiple groups, you'll do this for each group.

NOTE: Do not select any groups on this page. Your groups have already been selected.

Once you have configured the message's details, click Submit at the bottom of the page.

You'll then need to confirm your message details on the resulting page. Click Submit again to send your message. If you're testing multiple groups, you'll repeat these steps for each one.

Step 4 - Compare the results

After the messages have been sent, tracking data is collected and stored, and available for you to view in an A/B test report.

1. Navigate to Messages > A/B Testing > Reports.
2. Select your A/B test from the menu and click GO.

The results are displayed on the following page, broken down by message. At the top of the results for each message, you will see basic information to identify the message, including message name, subject line, from field, and group name. The element that you designated for testing (e.g., Subject Line) is displayed in red.

With these results, you can determine which elements performed best and use them to send the final message to the rest of the group contacts.