Understanding control groups

A control group excludes a fixed share of your users from marketing messages on Push, Email, In-App, SMS, and WhatsApp. Pushwoosh compares those users to everyone else in the group’s audience who keeps receiving campaigns, so you can measure whether your messaging drives results.

An app can run several control groups at once, for example one per market. Each group holds out its own share of users within its own countries and tag.

Control group users still receive transactional messages on those channels when you set the message type accordingly.

Important

• Control groups are configured per application. Each group has its own on/off state, size, scope, and membership. Creating or changing a group in one app doesn’t affect any other app.
• Only admins can create, edit, or delete control groups. Roles are assigned in My Account → Users and groups. Learn how to manage roles and permissions.

How control group membership works

Pushwoosh doesn’t keep a stored list of control group members. It calculates membership with a formula on every check:

1. Pushwoosh combines your app, the current group version number, the group, and the user ID.
2. A hash formula turns them into a number.
3. If the number falls within the percentage you set (for example, 5%), the user is in the control group. If not, they aren’t.

What this means for you:

• All existing users are covered at once. Turning the group on applies to your entire user base right away, not only to people who sign up afterward.
• Membership changes only at set moments. A user stays in or out of the group until one of these happens: you reshuffle the group or change its size, the user’s ID changes, or an Auto-refresh cycle ends or an Experiment reaches its end date and Pushwoosh selects a new group.
• Re-enabling brings back the same group. If you disable the group and turn it on again without reshuffling, you get exactly the same users, because the formula hasn’t changed.

Link every contact method to one User ID

Pushwoosh assigns users by User ID, not by device:

• Contact methods under different User IDs, such as a mobile device, an email address, or a phone number, count as separate users. One of them can be excluded from marketing messages while another for the same person still receives them.
• Contact methods without a User ID are never excluded from marketing messages (Push, Email, In-App, SMS, WhatsApp). Pushwoosh needs a User ID to run the membership formula.

Register every contact method for the same person under one User ID, so the person has the same control group status on every channel. Learn how to manage User IDs for omnichannel campaigns

How several control groups work together

Every enabled control group of the app applies to every marketing message the app sends. You don’t pick a group per message.

• Each group selects its own users. Two groups of the same size hold out different users, because the group is part of the membership formula.
• One group is enough to hold a user out. If a device falls into the audience of several groups and any of them selects its user, the device doesn’t receive the marketing message.
• Groups can overlap. Pushwoosh lets you create groups whose countries and tag values match the same users, and warns you about it. Users who match both groups are held out by each of them, so the statistics of both groups are skewed. To keep results clean, give each group its own countries or its own values of the same tag.
• Disabled groups don’t hold anyone out. They keep their settings and history, and they don’t count toward overlaps.

How the control group affects message delivery

Control group membership blocks or allows messages based on the message type you set for each send.

• Marketing message: blocked for control group users on Push, Email, In-App, SMS, and WhatsApp.

• Transactional message: still delivered on those channels. Select Transactional message when creating a customer journey or sending a one-time message.

• Silent push: always delivered, because it carries data for the app, not a message for the user.

Choose Marketing message or Transactional message for every send on those channels. See where to set the message type.

Note

• Viber: control group members never receive Viber messages, even transactional ones.
• Missing message type in API sends: the message_type parameter is optional. If you omit it, Pushwoosh treats the send like Marketing message, not transactional, so control group users are excluded. Set Transactional message in the campaign or message_type to transactional in the API to reach everyone, including the control group.

Learn more about marketing and transactional messages.

Set up and measure your control group

Create and manage a control group

Name the group, set its size and mode, scope it by country or tag, then change, rename, reshuffle, export, disable, or delete it later.

Learn more

Analyze control group impact

Compare events between users who get your marketing messages and the control group, and read the Impact report.

Learn more

Control groups FAQ

Answers to common questions about membership, modes, scope, and results.

Learn more

Related

To measure a single campaign or journey against its own holdout instead of the application’s control groups, see the uplift report.

Uplift report