Holdout Split

Use the Holdout Split element to hold a share of users back from part of a journey, then compare their goal completion against everyone who continued through it. Unlike the account-wide Control group, which is fixed per application, a Holdout Split is scoped to one journey and one point in its flow.

When to use Holdout Split

• Measure one journey’s impact without an app-wide control group. Hold out a share of the users reaching this point and see whether the rest of the journey moves your Conversion Goals more than doing nothing would.
• Test a specific segment or flow. Combine Holdout Split with an Audience-based or Trigger-based entry to measure impact only for the users who matter for this campaign, instead of holding out your whole app.
• Get a per-goal uplift, not just an event count. If the journey has several Conversion Goals, Holdout Split reports uplift and statistical significance for each one separately.

Prerequisites

Define at least one Conversion Goal for the journey. Without one, the report described below shows a prompt to add a goal instead of any numbers.

Add and configure Holdout Split

1. Place the Holdout Split element on the canvas after any point, and double-click it to configure.
2. Set Share held out: the percentage of users reaching this point who are routed to the Holdout branch. The rest continue on the In journey branch.
3. Leave the Holdout branch unconnected. It’s meant to stay that way: users routed there exit the journey immediately and receive nothing else from it. Connect the In journey branch onward to the rest of your flow.
4. Click Save.

नोट

Held-out users are excluded from this journey only. They can still receive messages from your other journeys and one-time sends. That’s different from the account-wide Control group, which blocks marketing messages everywhere.

Read the uplift report

Open a running journey’s Holdout Split point to see its current split and a Goal uplift vs holdout report below it. The same report also appears in the journey’s Campaign health dashboard, once for each Holdout Split point in the journey.

Pushwoosh recomputes the report once a day, so the first numbers appear the day after the journey starts. Until then, the report shows a placeholder message instead of any numbers.

The report’s header line shows how many users were on each branch as of the last computation, the attribution window (how many days after passing the point a goal completion still counts toward the report), and when it was last computed.

For each Conversion Goal in the journey, the report shows:

• Conversion, in journey: the conversion rate and count for users on the In journey branch.
• Conversion, holdout: the conversion rate and count for users on the Holdout branch.
• Uplift: the relative difference between the two conversion rates. Shown as — when the holdout branch had zero conversions for this goal, since there’s no base to calculate a relative difference against.
• A significance badge: Significant, Not significant, or Not enough data. Significance follows the same confidence, P-value, and Z-score calculation as control group impact reports.