You heard how effective push notifications could be, especially compared to email campaigns, so you dived in with both feet. You came up with some content ideas for tactical messaging, adjusted delivery timing to account for past user behavior, and even factored in target device type. Now, thanks to your careful planning and strategic use of push, your users are re-engaging with your app more than ever. Congratulations!

Now what?

You invested a lot of time and effort to bring your users back; how do you know if the post-push experience is satisfying? What happens after the push, and does the result justify all the work required to achieve it? It’s easy to measure click-through, but does a single number tell the whole story?

At a high level, two key indicators are especially important as we attempt to answer these questions. It’s interesting to consider, after our push notification,

1. How many users took a desired action, but also
2. How many users enjoyed an enhanced app experience.

Some consider these to be equally important factors when evaluating the success of a push notification campaign.

What metrics should you track?

The variety of metrics available to you is limited by the type and number of your goals; that is, there is no limit. How you measure value when it comes to a push notification—as with anything—depends entirely on what you hope to get out of it. Click Rate (or Open Rate) is certainly the most visible and readily accessible statistic associated with push, but in most cases, you probably care more about conversion, or engagement, or revenue.

That is, your goals decide what you measure. For example, Conversion Rate is a general way to describe the percentage of users who take a specific action after some event. It represents our first key indicator, above. Say you push a notification exhorting users to play the new anti-gravity level you just added to your mobile game. In this case, “conversion” refers to a player trying the new level. You can measure the impact of the message by comparing the conversion rate before and after the push notification.

Similarly, Engagement Rate describes the average number or duration of user sessions. You can evaluate the efficacy of your push messaging by looking for changes in how often or how long users interact with your app, once they’ve seen your notification. For many, Revenue may be the bottom line (literally and figuratively). You may want to measure new revenue generated by users who have seen your message, compared to those who haven’t. Both Engagement and Revenue are good at quantifying user satisfaction with the app experience, which makes them good proxies for our second key indicator, above.

Control, control, control

To fairly evaluate the impact of your push notification campaign, you must be able to compare data before and after it’s launched. Of course, many factors may contribute to the average revenue per user, or average daily visits, or average session length, or any of the other metrics you choose to measure. Accounting for myriad variables like time of day, holiday sales, competitor campaigns, positive press, negative press, influencer attention, etc. results in some gnarly number crunching—when it’s even possible.

That’s why it’s so important to maintain controls during push notification campaigns. A control is a small group of customers who are excluded from the campaign. All other things being equal, the control allows you to effectively isolate the impact of your campaign for measurement purposes.

For example, say you run a post-Christmas push campaign to boost revenue. Comparing January sales to those in December would obviously be problematic, and wouldn’t necessarily reveal how effective your campaign was. Instead, you would compare average January revenue from your control group against revenue among customers who saw your push notification. That would give you a more accurate estimate of your campaign’s effect.

New eBook available for download

Push notifications are one of the most effective forms of marketing today, and require the same level of planning and measurement as traditional online marketing. Set the right goals—and exercise the same rigor—and your reporting can reveal true, actionable results.

Want more information about developing a push notification strategy for your app? Check out our latest eBook, Best Practices for Push Notifications. It highlights some of the most effective ways to develop personalized, timely, and relevant push notifications. The eBook also covers success measurement and discusses tools you can use to get started with push notifications.

-peter (@peterdotgames)