NOTE: This article is for customers with the "classic" Admin interface (horizontal top-navigation bar).
If your interface is different, view the New Admin Experience version of this article.
The Tracking page for A/B Testing campaigns will show you live results while the campaign is running, and then final results after the evaluation time has passed.
How to access the Tracking page
To access the Tracking page:
- Navigate to the Automation tab.
- Click Campaign Management.
- Select the A/B Testing campaign you want to track.
- Click Track.
How to use the Tracking page
This page comprises four main sections:
- Winner by Open/Click Rate
- Samples Comparison
- Winner Funnel
- Winner Daily Engagement
Let's look at each section in more detail.
1 - Winner by Open/Click Rate
The left side of this report shows you which test message won according to the criteria you set for the campaign.
- If it's still before the date and time you set for a winner to be determined, the report will show a current winner.
- If no test message won, or if there is no current winner, the report will say “no winner.”
The right side of this report shows you a bar graph with the open or click rate for each test message, depending on whether you chose open rate or click rate as the winning condition for the test.
2 - Samples Comparison
This report compares the delivery statistics for the test messages sent to each sample. Using the menu on the right:
- Choose by number to see the total number for each column.
- Choose by percentage to see the total number of sent messages, with the other columns as a percentage of that value.
Statistics Breakdown
Each column in the comparison table is described below:
- Sent - The number of test messages sent.
- Delivered - The number of messages successfully delivered to the recipient.
- Opened - The number of messages opened by the recipient.
NOTE: This number can be higher than the number of deliveries because a recipient can forward the message to another recipient.
- Clicked - The number of links clicked by the recipient.
NOTE: This number can be higher than the number of opens because the same recipient can click a link multiple times, or they can forward the message to another recipient who can also click the link.
- Bounced - The number of attempted deliveries which failed.
NOTE: If the message bounced because Higher Logic Thrive Marketing Enterprise (Thrive Marketing Enterprise) did not receive a message receipt, it can take a few hours for the bounce to be counted. This is because Thrive Marketing Enterprise will wait to see if the recipient’s mailbox was just slow in accepting the message before counting that message as bounced.
3 - Winner Funnel
This report shows where each campaign member is as part of your classic marketing funnel for the winning message. It tracks the results for the winning message after it is sent to the remainder of the recipients.
- It does NOT include messages sent before the evaluation date and time.
- If the winning message has not been sent to the remainder yet, this report will be blank.
Statistics Breakdown
Each row of the funnel is described below:
- Sent - The number of test messages sent.
- Delivered - The number of messages successfully delivered to the recipient.
- Opened - The number of messages opened by the recipient.
NOTE: This number can be higher than the number of deliveries because a recipient can forward the message to another recipient.
- Clicked - The number of links clicked by the recipient.
NOTE: This number can be higher than the number of opens because the same recipient can click a link multiple times, or they can forward the message to another recipient who can also click the link.
4 - Winner Daily Engagement
This chart displays the “rolling” engagement rate for both opens and clicks for the winning message of the A/B test, when the winning message is sent to the remaining campaign members.
- It does NOT include messages sent before the evaluation date and time.
- If the winning message has not been sent to the remainder yet, this report will be blank.
The data is calculated as follows:
- Total number of opens/total number of delivered
- Total number of clicks/total number of delivered
EXAMPLE:
1. On day one, you send out 50 messages to sample A and 50 messages to sample B.
2. In sample A, 7 recipients open the message, and in sample B, 3 recipients open the message.
3. The total number of messages sent is 100, and the total number of opens is 10, so the open rate would be 10%.
4. The next day you do NOT send any additional messages, but another 2 recipients in sample B open the message. The open rate would then increase to 12%.
Each campaign member gets one opportunity to engage with each message (multiple opens and clicks of the same message by the same campaign member are only counted once.)