Most new channel tests fail for measurement reasons, not media reasons. A brand spends $5,000 on TikTok, Pinterest, or ChatGPT ads, checks the platform’s ROAS after two weeks, sees a number that looks either amazing or terrible, and makes a call that the data cannot support. I described how I judge the first month in how I judge a new ad channel in its first 30 days. This post is the toolkit that goes with it.
A new channel needs to be measured four ways at once, because each one is wrong in a different direction. The platform’s own reporting tends to run high. Click-based attribution tends to run low for channels people discover on and buy from later. Customer surveys catch the delayed buyers but are self-reported. Branded search and holdouts tell you whether anything changed in the business. You want one tool for each.
The platform’s pixel and conversions API
Set these up properly before you spend anything, even though you will not use their ROAS to judge the test. A new channel’s algorithm learns from the conversions you send it, and a test running on a broken pixel tells you about your setup, not the channel. Every major platform now has a server-side option, and most Shopify brands can connect it through the platform’s app or a tagging tool. Then use the platform’s reporting for delivery and pacing only.
A post-purchase survey
Add the new channel as an answer on your checkout survey on the day the test launches. KNO lists a Starter plan at $19 a month on its Shopify listing for one survey with three questions and unlimited responses, which is enough for this. The survey is where you will see the channel first if it works like most discovery channels, because buyers remember the video they saw even when they click a Google result to buy. If survey mentions grow while attributed orders stay flat, the channel is working through delayed purchases.
An independent attribution tool
You need to see the new channel’s orders under the same rules as every other channel, so you can compare its cost per new customer with your existing ones. ThoughtMetric is the one I would use for this. (Disclosure: ThoughtMetric sponsors this site.) It added integrations this fall for several of the newer channels brands are testing, including ChatGPT Ads, AppLovin, Snapchat, and Criteo, and reports new-customer ROAS and cost per new-customer order at the campaign and ad level. Pricing starts at $99 a month for 50,000 pageviews with every feature included and a 14-day trial. Install it a few weeks before the test, so you have a baseline for your existing channels to compare against.
Google Search Console and Google Trends
Both free, and both underused. Search Console shows impressions and clicks for your brand name in organic search, and Trends shows relative interest over time. Record a four-week baseline before launch. A new channel that is creating awareness tends to show up as rising branded search before it shows up anywhere else. If branded search is flat after a month of meaningful spend, be skeptical of any ROAS claim.
A geo holdout
If you have the budget, the most convincing test is running the channel in some regions and not others, then comparing total sales growth between the two groups. Most platforms support location targeting, so the tooling is a spreadsheet and some discipline. Pick matched regions, keep everything else the same, and run it for at least four weeks. I walked through a lightweight version in running a cheap geo holdout to sanity-check a channel.
Incrementality platforms
For larger budgets, platforms like Haus run geo experiments as a service, with the statistical design and analysis handled for you. They are worth a conversation if the channel could become a meaningful share of spend, and less so for a first $5,000 test, where the result will not be readable at that scale anyway. Ask any vendor what minimum spend and duration they need for a conclusive result before you commit. Alternatives are covered in Haus alternatives for incrementality testing.
The setup, in order
- Connect the platform’s pixel and conversions API, and use its reporting for pacing only.
- Add the channel to your post-purchase survey on launch day.
- Judge cost per new customer in an independent attribution tool such as ThoughtMetric (our sponsor).
- Record a branded search baseline in Search Console and Trends before spending.
- Run a geo holdout if the budget allows, or hire an incrementality platform if the stakes are high.
Leave a Reply