Launch and measure

Test every journey on your own phone before real shoppers get its messages, then check whether it works.

Test each journey before you launch it

A journey has no test mode, so once it is running, real shoppers who match its entry get its messages. Testing on your own phone first catches a missing event, an empty segment, or a wrong message before shoppers see it.

Run through this list for each journey:

1. Install the version of your app that includes the Pushwoosh SDK on your phone, and log in with a test account. If you do not have one, ask your developers for a test version. The phone must register in the application where the journey runs. To send test messages from presets, also register it as a test device.

2. Use the app and do what the journey needs, for example add a product to the cart. Then open Audience → User explorer, find the test device, and check Events history. The events must be there with the expected attributes, or the segment and the journey do not see them.

3. If the journey starts from a segment, open it in Audience → Segments and refresh its size to see whether it counts any shoppers.

4. Launch the journey for your test shoppers only. Build a test segment from your test User IDs with the User ID filter. If the journey starts from a segment, use the test segment as its entry. See How to test a journey with a test segment.

   Journeys that start on an event

   If the journey starts on an event (a Trigger-based entry), every shopper who triggers the event enters it, including real shoppers, and the entry has no segment filter. To keep real shoppers out of the messages, use one of these:

   • Right after the entry, add a Condition split on your test segment and leave its All others branch empty. Real shoppers enter but leave the journey at once without messages, and they count in its entries. Remove the split before the real launch. If the journey has a Campaign Entry Limit such as a lifetime, real shoppers can use up their entry during the test, so clone the journey instead.
   • Clone the journey to a test application in the same account and run it there with a phone that is registered in that application. Events the test application lacks are created there, but check the segments and the content of the clone.

5. On the test phone, do the action that the entry waits for.

6. Check that the message arrives, and follow the test shopper with Find user path.

7. After a day, check how many shoppers entered the journey. See Read the journey statistics.

Read the journey statistics

Journey statistics show whether shoppers enter a journey, get its messages, and reach its goal. They count goal conversions, not revenue. To compare revenue, use Conversion events.

To see the journey statistics:

1. Open the journey on the canvas. The statistics panel on the left shows the whole journey.
2. For numbers on each element, open Canvas settings (the gear icon in the bottom-right corner) and turn on Show metrics. See Journey element statistics.

For what each metric means, see Overall journey statistics.

Check the first-week signals

In the first week after launch, look for these signals:

If you see	It usually means	What to do
No entrances	The entry segment is empty or the entry event does not arrive	Repeat steps 2 and 3 of Test each journey before you launch it
Far fewer messages sent than entrances	Many shoppers can’t get push, or frequency capping and the silence period hold messages back	Open Communications on the statistics panel and check Suppressions in the Frequency capping card, add an in-app or email branch after a Reachability check, and check your delivery rules
Many drop-offs	Shoppers left the journey because of a problem, such as a device that was not found or an invalid push preset	Expand the Drop-offs block on the element to see the reasons
Messages are opened, but few shoppers reach the goal	The message works, but the offer or the moment does not	Test another text or offer with an A/B/n split

Journey does not behave as expected?

If a journey does not behave as expected, follow one shopper through it with Find user path, and look up why shoppers leave a journey or skip a message in Drop-off and skip reasons.

Measure results

To measure whether a journey changes shopper behavior, use the three tools below, from simplest to strongest.

• Conversion goals: every marketing journey in this playbook has one. Order status updates and Review request after delivery do not. The goal can be reached within the period you set, up to 30 days after the shopper leaves the journey. See Conversion goals.

• Holdout Split: hold back 1% to 20% of the shoppers who reach a Holdout Split element and compare their goal rate with everyone else. The Goal uplift vs holdout report updates once a day.

  Where to place a Holdout Split

  • Put it right after the entry, before any message. Connect only the In journey branch and leave Holdout empty, so held-out shoppers leave the journey there and get nothing.
  • The journey needs a goal, because the report compares goal rates between the two groups.
  • To measure one part of a journey, put another Holdout Split in front of that part.

• Control group: by default, holds out a share of the shoppers from all marketing messages in the application, so you can measure the combined effect of your messaging. Set it up before you launch. See Set up a control group before you launch. To read the results, see Analyze control group impact.

Cases to check before launch

These cases decide whether your segments fill up and your journeys send messages. Check them before you launch a journey.

General cases

These cases are the same in every store app and in other industries.

Case	What happens	What to do
iOS push permission prompt	iOS shows it only once, so a shopper who declines on the first launch can’t be asked again	Ask for push permission at the right moment
Repeating events such as Product added to cart	They fire many times a day, and each one after the previous run of the journey ended starts it again	Set a campaign entry limit. See Best practices for every journey
Two paths to revenue	If the app sends PW_Conversion and Order placed also carries __amount, Pushwoosh records each order twice	Use one path per order. See Track revenue from orders
Shoppers without a push token	An Audience-based entry and segment filters with never take in only devices with a push token, so shoppers whose app has not asked for push permission yet are left out	Ask for permission early enough for your journeys. See Build the segments

Store cases

These cases come from how a store tracks shoppers and orders.

Case	What happens	What to do
A guest who uses two devices	Until the shopper logs in, each device has its own User ID, so one shopper counts as two, and a cart left on one device is not matched with the order placed on the other	Identify shoppers by User ID
A cart left before login	The cart event is recorded under the guest’s HWID and the order under the customer ID, so the cart reminder can still arrive after the order	Ask your developers to call setUserId before the shopper can add a product, for example by asking guests to log in when they add the first product. See Identify shoppers by User ID
Order, delivery, and restock events sent from the app	The app does not know when an order is delivered or a product is back in stock, so these events never arrive	Send them from your server with the shopper’s userId. See Send the events from the app
Inactivity segments right after launch	At-risk and Lapsed customers are empty until Order placed has been recorded for 60 and 180 days	Use PW_ApplicationOpen in the same filters at first. See Build the segments