Every loyalty app dashboard I have looked at makes the program look like a success. Members spend more, buy more often and have higher LTV than non-members. All of that is usually true, and almost none of it proves the program did anything. Your best customers join loyalty programs because they are already your best customers. The question worth answering is whether the program changed how people buy, and that takes a different set of numbers than the ones Smile.io, LoyaltyLion or Yotpo Loyalty put on the home screen.
What the loyalty dashboard is counting
Loyalty apps tend to report revenue from members and revenue from orders where points or rewards were redeemed. Both overlap heavily with what your email flows, SMS and ad platforms already claim. A member who gets a points reminder email, clicks it and buys shows up as loyalty revenue, email revenue and possibly retargeting revenue, all for one order. Add the totals across your tools and you will often find more revenue than your store actually did. I walked through the email side of this in email revenue overcounting, and loyalty has the same shape.
The other problem is selection. People who sign up for points are more engaged before they ever earn a point. Comparing members to non-members compares engaged customers to everyone else. The gap mostly reflects who joined, not what the program did.
Compare before and after, not members and non-members
If you have a clean launch date, the simplest honest check is a cohort comparison across it. Take customers who made their first purchase in the three months before launch and the three months after. Compare their 90-day repeat rate, their time to second order and their average order value on repeat purchases. If the program works, the post-launch cohorts should repeat sooner or more often. If the curves look the same, the program is mostly rewarding behavior that was already happening.
Seasonality muddies this, so line up comparable months where you can, and don’t launch a loyalty program the same quarter you change pricing or your email program. The cohort method is covered in more detail in reading LTV cohort curves on Shopify.
Run a holdout if you can
A cleaner test is to hold a random slice of new customers out of the program’s marketing. You may not be able to stop them from joining, but you can leave them out of the points reminders, tier emails and reward nudges for a few months. Compare their repeat behavior to the group that got the full program. This tells you what the program’s communication is worth, which is usually where most of the effect lives anyway.
Count the cost, not just the revenue
Points are a discount you have promised but not paid yet. Track the value of rewards redeemed each month alongside the app fee, and subtract both from whatever lift you measure. A program that adds a few points of repeat rate but gives away 8 percent of member revenue in rewards can easily lose money. If you already use a profit tracker, make sure redeemed rewards show up as discounts so your margin numbers reflect them.
Keep loyalty out of your acquisition numbers
Loyalty drives returning orders. If your attribution reports blend new and returning customers, a successful loyalty program can make your paid channels look better than they are, because the returning orders it encourages often pass through an ad click on the way. Split every channel report into new and returning customers, and judge acquisition spend on new customers only. The channel side of this is in measuring LTV by acquisition channel on Shopify.
Where ThoughtMetric fits
ThoughtMetric reports new and returning customer orders by channel, which makes it easier to see what your acquisition channels are doing without loyalty-driven repeat orders inflating them. (Disclosure: ThoughtMetric sponsors this site.) It uses multi-touch attribution from its own pixel with a lookback window up to 90 days, and it includes server-side tracking and post-purchase surveys. Pricing is based on monthly pageviews, starting at $99 a month for 50,000 pageviews, with every feature on every plan. It will not measure the lift from your loyalty program by itself. The cohort comparison or holdout above is still the test for that.
What a working program looks like
In stores where I have seen loyalty pay off, the signal shows up in time to second order. Customers who joined came back a few weeks sooner than comparable customers before launch, and the share of first-time buyers who made a second purchase within 90 days moved up and stayed up. Where it didn’t pay off, member revenue grew while overall repeat rate stayed flat, which means the program was relabeling orders rather than creating them.
Picking an approach
- Have a clean launch date? Compare first-purchase cohorts before and after it on 90-day repeat rate.
- Can control program emails? Hold a random slice of new customers out of them and compare.
- Only looking at the loyalty dashboard? Expect it to overstate the program and overlap with email and SMS.
- Reporting profit? Subtract redeemed rewards and app fees from any lift.
- Want acquisition reported cleanly next to loyalty? ThoughtMetric (our sponsor) splits new and returning orders by channel.
Leave a Reply