CTRwow

Please find attached below the topics explained on this page:

What is the Split Test section about?
- Split Test Feature vs Funnel Builder
Where to find the Split Tests section
What’s in the Split Tests section
How to create a new Split Test
The amount of traffic to include in the test
Reset Statistics
The Split Test Tracking Script
Editing the Split test once the Test started
The Split Test Report

What is the Split Test section about?

In the Split Test section is available the feature that enables users to create Split Tests by using any URLs they need to test.

Split Test Feature vs Funnel Builder

The difference between this new feature and the Funnel builder is how we control and split the traffic.

In the Funnel Builder, the user can control the traffic from the funnel URL by setting up the test with the funnel URL from the beginning.
The Split Tests feature allows users to set any page to control the traffic. Thus, they can inject the test into any page which is already live with traffic.

Where to find the Split Tests section

The Split Test section is located under the BUILD section (see image below)

What’s in the Split Tests section

In the top right corner of the page, there is the + New Test button, by clicking on it, the user is redirected to the new Split test page, where he/she can start building the new (A/B) test
When users open the Split Tests page, they can filter their Split Tests by status:

Not Ready
Ready to Start
Running
Paused

3. On each Split Test, users can have a Control button: Start | Pause.

4. URL is the control URL that is used to preview the page under test.

5. On the right side of each Split Test, by clicking on the Actions (three dots) button a drop-down menu, will pop up giving the user the below options:

Edit
Delete
View Split Test Report

How to create a new Split Test

Once users click on the + New Test button

they are redirected to the new Split Test page (see below example):

The Settings

From the Settings page, the user can easily customize the new Split Test by filling:

The Name of the new Split Test
Include a Description

The URL Configuration

After the Settings, the user needs to set up the URL Configuration as per the below instructions.

By default, there are two options available: Control URL | Variation 1 URL

Users can click on Add URL and the Add New URL tab will pop to select:

An internal page, from the drop-down menu a list of all the sites in CTRwow and expand them to see the pages on each site.

Please note that a specific page URL needs to be chosen because some pages may have been published in different domains.

Or enter an external page on the textbox

It’s important to highlight that if the test page(s) is built outside CTRwow, the Split Test Tracking needs to be added into the header of the external page.

Please check the section below, called: How to add the Split Test Tracking

2. Users can add more variations with the Variation URL 1 where, by clicking on Add URL button, the Add New URL tab will be displayed, and the user can select an internal page or enter an external page:

3. Distribution percentages are divided equally unless users update them, dividing the 100% Traffic into unequal parts (i.e.: 30% in the Control URL and 70% in Variation 1)

4. Additional Variation can be included by the user, just selecting the +Add new variation button

and then repeat the above points 3 and 4.

The Add conditions for your test option

The Add conditions for your test is an optional choice. If this option is enabled, the user can add some conditions to the test so only the traffic which matches those conditions is redirected to the test.

The Type of Conditions available to be included are:

Location
Device
URL Parameter

Please note that these configurations are similar to the set condition popup for the Condition Splitter node in funnel builder. For more information about it, please click on the following helpdesk article: The Condition Splitter.

However, the main difference between the Add conditions for your test option and the Condition Splitter is that in the Split Test the customers for which the conditions are not applied will not go through the test, but they will go through the normal funnel.

Therefore, this will be reflected on the Page statistics because:

On the Website, under ANALYTICS, it shows the total number of customers that visited the page
On the Split Test Report are displayed only the customers for which the condition(s) have been applied.

For example, the user sets as a condition: Location = United States on his/her Split Test

The page, on the Split Test, receives:

100 visitors from the US
100 visitors from the rest of the world

On the page Statistics will be reported:

On the Website, under ANALYTICS, the total number of customers, which visited the page, is 200 (100 US + 100 from the rest of the world)
On the Split Test Report, the number of customers, which went through the page and the test, is just 100 (the 100 US customers).

To start adding the condition(s), users should click on the +ADD RULE or/and the +ADD GROUP button(s)

By selecting + Add a Rule: means that just one condition will be set

and by clicking on + Add group: means that a group of rules will be included

Users can add multiple conditions and combine them with AND or OR
Additionally, users can select one of the three types of conditions supported:
1. By Location (country): This is helpful for the traffic team when they already run traffic on a URL, and they want to add more GEOs. Each GEO may have a separate localized URL, so they can just add to this test without having to switch multiple links for their ads.
Once they select the condition Locations, next it needs to be set the country, choosing from the drop-down list (see the below example)

2. By Custom Query (URL Parameter): This condition is used by split test teams when they want to perform a test with a specific parameter (i.e. a particular AffId).
Once they select the condition Custom Query, users need to enter in the text field the Parameter Value

3. By device: This condition is also used by split test teams when they want to do a test for specific devices, including desktop, tablet, and mobile.
Once they select the condition Device, users must select a device from the list given on the drop-down menu (see the below example)

Next, they can set the comparison such as:
1. Equal to: ==
2. Not equal to: !=
3. In any (for multiple values)

The Add tracking metrics by URL pattern to your test option

The Add tracking metrics by URL pattern to your test is another optional choice. When enabled, users can add some extra metrics to the test, based on URL patterns.

Users can define a URL pattern to match a part of the URL customers land on throughout the flow.

For example, when customers buy the purchase popup by clicking on the add button

this parameter clickid=btnadd* is added to the URL when they navigate to the next page.

*btnadd refers to the button to add the product (as it is shown in the above image example).

Each added URL pattern will be treated as a new metric in the Split Test Report.

Saving the Test

Once all configurations have been made, the user has to click on the Save button, at the bottom left of the page.

Once the test is saved, it is not automatically running but the user needs to click on the Start button either:

On the Split Test main page
Or on the Split Test page itself

By selecting after selecting the Start button, a tab will pop up asking the user to confirm the action:

because if the page is already online and receiving traffic, the user needs to consider that the Split Test Script will be added to the Control page which will be re-published to allow this test to start; to confirm this action, just click on the Run Split Test button.

Upon saving the test, the user can go back to the page and also configure the below options, such as:

The amount of traffic to include in the test
Reset Statistics
View Split Test Report
The Split Test Tracking Script

It has to be noted that if in the Control URL, the same page is used in two different Split Test, when the user click on the button Start the test, a warning message will pop us informing the user that a page can be used in one test at the time

The amount of traffic to include in the test

By default, it is set at 100%, which means that users can change the amount of traffic they want to test with.

However, once the user has finished setting up the Test and saved it, the user can go back to editing the Split Test, by clicking on the Edit option (see below image)

and on the Split test page, on the right side, it can be adjusted the amount of traffic to be redirected to it

By default, the traffic is set to 100%, but users can change it at any point.

Moreover, the amount of traffic that is redirected to the Test will be also split into the pages according to what was previously set on URL Configuration:

For example, when running a test for a top-selling product, users don’t want to get all the traffic which might affect the sale.

Therefore, they can start with only 50% of the traffic and this means the remaining 50% being used for the test will be split into 25% for control and variation.

As a result, in case the test might not go well, the user can decrease the amount to minimize the cost.

Reset Statistics

There are cases in which users need to reset the statistics of the test. For instance, when users want to clean the report after they’ve done some manual test, and they are about to start the test; or when there is a bug, and they want to reset the test after the bug has been fixed.

In order to reset these data, they can just go back to the Split test page, by clicking on the Edit option (see below image)

from there, on the right side, just click on the Reset Statistics

This will reset all metrics of the test to 0.

The Split Test Tracking Script

If on the test are included any external (not built-in CTRwow) page, the user needs to add on the header of each external page of the test the Split Test Tracking

The Split Test tracking script is automatically added and available on all internal pages.

To obtain the Split test Tracking Script, once the Split test has been saved, the user needs to click on the Edit option, on the right side of the page:

and be redirected on the Split test page. On the right side of the page, there is a button Click here

that once clicked will display a small tab from which the user can copy the Split Test Tracking Script

This Tracking Script needs to be pasted on the header of all external pages, even the ones that follow the external page URL added into the Split Test.

Editing the Split test once the Test started

Once the test started, users can still make changes. To edit a Split Test, from the Split Test main page, by clicking on the Actions button, on the right side of each Split Test and then select Edit

On the Split Test page itself, at the top of the page, it shows a message that warns users to Pause the test prior to saving any changes:

The Split Test Report

To check the result of the test, there is a specific Report called, the Split Test Report.

This report is available in the Split Test section, and can be open either:

1. From the Split Test main page, by clicking on the Actions button, on the right side of each Split Test and then selecting Split Test Report

2. Inside each Split Test page, on the right side, there is the View Split test Report button that once clicked will open the Split Test Report