A/B Testing

Perform A/B tests on your audience to figure out which push and email configurations work better for you

A/B test helps you decide which push and email configurations work better for your audience. Create and send different versions of push or email messages to understand which one is more relevant and leads to more conversions. Specify different content, actions performed after the push is opened, segmentation or scheduling rules, and track the results!


Before start, please make sure you've created Presets and Filters to use in your A/B Tests.

To start with A/B Testing, go to Messaging -> Push notifications -> A/B Experiments section of your Control Panel.

Here, you'll see the list of already launched experiments.

To set up a new A/B Test, press +Create Experiment.

Step 1. Enter the name of your experiment.

Step 2. Select an app to carry out an experiment on.

Step 3. Define the audience to which you'd like to send messages in the course of the experiment.

  • First, select a Filter to apply. The experiment will be held on only for those users who meet Filter's conditions.

  • Then, specify a segment size — the percentage of users meeting the Filter's conditions who will take part in your experiment.

To learn more about Filters and audience segmentation, please refer to the guide.

If you use a precompiled Filter, you’ll see the number of users complying with its conditions and the number of users in a segment specified.

Please be aware that a Filter saved in a preset does not apply to A/B Test segmentation. The experiments are only carried out on the audience you specify on step 3 of the experiment setting.

For example, if you set up the Purchased something Filter to be applied and the 50% segment size, half of the users who have bought something in your app or on your website will receive your A/B Testing messages, no matter what Filters are saved in the presets you send.

Step 4. Split the segment into groups. Each group will receive its own message.

Please note the A/B Tests are intended to compare the messages' content only. Scheduling and segmentation rules from the preset will not apply.

You can create up to 10 groups here.

Step 5. Specify content that will be sent to each of the groups.

  • First, select a content typePush preset or Email preset. The selected content type will be used for both groups.

  • To learn more about creating Presets, please refer to the guide.

  • Then, select a preset that will be sent to each of the groups.

The experiment will be held for all platforms enabled in all presets accumulatively. For example, if a preset for a Group A has the iOS platform enabled, and the preset for a Group B has the Android and Chrome platforms, the A/B experiment will include users with iOS, Android, or Chrome.

All that users will be split into groups according to the rate you set on step 4. In case a user's platform does not match the platform of preset specified for the user's testing group, that user will receive the preset with no platform-specific parameters.

When everything's set up, press Send. The experiment will start immediately, that means all messages will be sent to all users taking part in the experiment with no delay.


To explore the results of your A/B Test, go to A/B Tests section and press Statistics button next to the experiment.

Statistics for the experiment contains the summary of conditions applied and the number of users involved. For each of the groups, here you'll see the number of messages sent and opened, as well as the CTR. To see the detailed stats for each message, press the Statistics button placed in the corresponding table row.