Analyze control group impact

Each control group’s page shows how your messaging performs against that group. Open it from Settings > Control groups by clicking the group’s row.

Check the group’s mode and dates

Below the group’s name, the page shows the Group size, the Mode, and a date that depends on the mode:

Mode	Date below the name	Label of the date-range selector
Permanent	Running since: the date the group started	No label
Auto-refresh	Range of the current cycle	Current cycle
Experiment with an end date	Range of the current cycle	Current experiment

To see the results of an earlier cycle, open the date-range selector and pick it under Previous. Ended cycles are listed there by number, as Cycle or Experiment. A Permanent group can have earlier cycles too, because resizing, rescoping, or changing the mode starts a new one.

What the page compares

The page has two blocks: an events comparison table at the top and an Impact report below it. Both compare two groups of users on any event you choose:

• Treatment group: all users who are not in the control group. They keep receiving your marketing messages (Push, Email, In-App, SMS, WhatsApp) as usual.
• Control group: the holdout you configured above. These users are excluded from marketing messages, so any difference in their behavior isn’t influenced by your campaigns.

Both groups are drawn from your entire app user base. Pushwoosh doesn’t filter either group down to users with a valid push token, verified email, or other reachable channel.

Treatment and control come from the same set of users, with no separate filtering between them.

Results from before August 25–26, 2026

Before August 25–26, 2026, the two groups were counted differently. If your time window includes those dates, Conv. Control and Uplift may show a sudden step that isn’t caused by anything in your app.

Compare events

1. Select a time window: Last 3 days, Last 7 days, or Last 30 days. To compare an earlier cycle, pick it under Previous.

   The measurement window restarts, and earlier data is no longer included, when:

   • you resize the group or change its countries;
   • you reshuffle the group;
   • you change the mode;
   • an Auto-refresh cycle ends or an Experiment reaches its end date.

   Data from before that point no longer matches the current members or audience. Ended cycles are listed under Previous in the date-range selector.

2. Review the events comparison table. It only shows events automatically if you’ve set Conversion Goals in Customer Journey. Otherwise, the table starts empty.

3. Click Add events to compare to add any event you want to measure. You can add up to 10 events in total.

   Message-engagement events can't be compared

   Push or email opens, clicks, sends, and bounces aren’t available here, even if you set one as a Conversion Goal. Control group users never receive marketing messages, so these events would only show that a message was sent, not what it achieved.

The table shows these metrics for each event:

Metric	What it shows
Conv. Control	The share of unique users held out from marketing messages who triggered the event at least once anyway, without any messaging influence.
Conv. Treatment	The share of unique users who received your marketing messages (Push, Email, In-App, SMS, WhatsApp) and triggered the event at least once.
Uplift	The percentage difference between the treatment and control conversion rates, relative to the control rate. A positive value means the treatment group converted better. Calculated as (conv. treatment − conv. control) / conv. control × 100%.
Significance	Whether the difference is statistically reliable. See the statuses in the next table.

Significance status	What it means
Significant	The uplift is unlikely to be due to chance.
Not significant	The difference isn’t statistically reliable yet.
Not enough data	Either group has fewer than 10 conversions or fewer than 10 non-conversions for this event, so significance can’t be calculated.

View the Impact report

Use the View event dropdown to select an event from the table. The Impact report below shows a deeper breakdown for that event.

Metric	What it shows
Uplift (change from control)	How much the treatment conversion rate differs from the control conversion rate. The same value as in the events table.
Conversions driven by messaging	The estimated number of extra events your messages generated beyond what’s expected without messaging. The percentage shows them as a share of all treatment events. Calculated as (events per user in treatment − events per user in control) × treatment group size.
Significance	Whether the difference is statistically reliable: Significant, Not significant, or Not enough data. Based on Confidence and P-value.
Confidence	How sure Pushwoosh is that the uplift reflects a real difference rather than chance, as a percentage. The higher it is, the stronger the evidence.
P-value	How likely this difference would appear by chance if messaging had no real effect. The lower it is, the stronger the evidence.
Z-score	How far apart the two groups performed. Close to 0, the difference could be random. The further from 0, the more confident you can be that one group is better.
Group breakdown	Treatment and Control side by side: Estimated users, Total number of events, Events per user, and Conv. rate (unique).

How to read the results

• Significant, positive Uplift: your messaging is driving this event. Keep sending it.
• Significant, negative Uplift: the control group converts better than the treatment group on this event. Before concluding messaging is working against you, check whether this event is one messaging is actually meant to drive.
• Not significant: the difference isn’t reliable yet, not that there’s no effect. Let the event accumulate more conversions before judging it.
• Not enough data: either group has fewer than 10 conversions or fewer than 10 non-conversions for this event. Compare an event with more volume, or wait for more of it to happen.

Two more things to keep in mind:

• Compare within one measurement window. Reshuffling, resizing, rescoping, changing the mode, an Auto-refresh rollover, or an Experiment end restarts the window. Results from before and after one of these aren’t comparable.
• Pick events your campaigns are meant to drive, such as a purchase or a sign-up, rather than events messaging has no direct path to.