NOTE: This article is for customers with the New Admin Experience interface (vertical left-navigation bar).
If your interface is different, view the "classic" version of this article.
A/B Testing campaigns allow you to test the effectiveness of your messages. You do this by sending two or more messages, that are not identical, to multiple groups to determine which message results in a better response.
In this article, you'll learn how to access and understand the Tracking page for A/B Testing campaigns, which shows you live results while the campaign is running, and then final results after the evaluation time has passed.
Access the Tracking page
- Access the Admin interface.
- Navigate to Campaigns > List.
- Select the A/B Testing campaign you want to track. The row is highlighted.
- Click Track.
Tracking page overview
The top of the Tracking page displays high-level information about your A/B Testing campaign: its start date, how many members are in the campaign, the number of days the campaign has run, and its current status (e.g., stopped, still running).
Below are four sections that dive into the campaign metrics:
- Winner by Open/Click Rate
- Samples Comparison
- Winner Funnel
- Winner Daily Engagement
Let's look at each section in more detail.
1 - Winner by Open/Click Rate
The left side of this report shows you which test message won according to the criteria you set for the campaign.
- If it's still before the date and time you set for a winner to be determined, the report will show a current winner.
- If no test message won, or if there is no current winner, the report will say “no winner.”
The right side of this report shows you a bar graph with the open or click rate for each test message, depending on whether you chose open rate or click rate as the winning condition for the test.
2 - Samples Comparison
This report compares the delivery statistics for the test messages sent to each sample. Using the menu on the right:
- Choose by number to see the total number for each column.
- Choose by percentage to see the total number of sent messages, with the other columns as a percentage of that value.
Statistics breakdown
Each data column in the comparison table is described below:
- Sent - The number of test messages sent.
- Delivered - The number of messages successfully delivered to the recipient.
- Opened - The number of messages opened by the recipient.
NOTE: This number can be higher than the number of deliveries because a recipient can forward the message to another recipient.
- Clicked - The number of links clicked by the recipient.
NOTE: This number can be higher than the number of opens because the same recipient can click a link multiple times, or they can forward the message to another recipient who can also click the link.
- Bounced - The number of attempted deliveries which failed.
NOTE: If the message bounced because the system did not receive a message receipt, it can take a few hours for the bounce to be counted because the system waits to see if the recipient's mailbox was just slow in accepting the message before counting that message as bounced.
3 - Winner Funnel
This report shows where each campaign member is as part of your classic marketing funnel for the winning message. It tracks the results for the winning message after it's sent to the remainder of the recipients.
- It does not include messages sent before the evaluation date and time.
- If the winning message has not been sent to the remainder yet, this report will be blank.
Statistics breakdown
Each row of the funnel is described below:
- Sent - The number of test messages sent.
- Delivered - The number of messages successfully delivered to the recipient.
- Opened - The number of messages opened by the recipient.
NOTE: This number can be higher than the number of deliveries because a recipient can forward the message to another recipient.
- Clicked - The number of links clicked by the recipient.
NOTE: This number can be higher than the number of opens because the same recipient can click a link multiple times, or they can forward the message to another recipient who can also click the link.
4 - Winner Daily Engagement
This chart displays the “rolling” engagement rate for both opens and clicks for the winning message of the A/B test, when the winning message is sent to the remaining campaign members.
- It does not include messages sent before the evaluation date and time.
- If the winning message has not been sent to the remainder yet, this report will be blank.
The data is calculated as follows:
- Total number of opens/total number of delivered
- Total number of clicks/total number of delivered
EXAMPLE:
1. On day one, you send out 50 messages to sample A and 50 messages to sample B.
2. In sample A, 7 recipients open the message, and in sample B, 3 recipients open the message.
3. The total number of messages sent is 100, and the total number of opens is 10, so the open rate is 10%.
4. The next day you do not send any additional messages, but another 2 recipients in sample B open the message. The open rate increases to 12%.
Each campaign member gets one opportunity to engage with each message (multiple opens and multiple clicks of the same message by the same campaign member are only counted once.)
Print the tracking results
If you'd like to print your tracking results or save them as a PDF, click the gear icon at the top right of the page and select Print Report.
