Most new ad channels get judged in one of two bad ways. Either someone looks at platform ROAS after ten days and kills it, or someone keeps it running for six months because the platform keeps reporting good numbers and nobody wants to be the one who turns it off. Both come from the same mistake, which is starting a test without deciding in advance what result would end it.
This is the process I use now, whether the new channel is Reddit, AppLovin, CTV, or a podcast. Thirty days is not enough to know a channel’s long-term value. It is enough to know whether it deserves another sixty.
Before launch
I write three things down before a dollar goes out. The test budget, sized so the channel can produce enough orders to say something. As a rough floor, I want the budget to buy at least 30 to 50 orders at a CAC I would be happy with, because below that the numbers are mostly noise. The kill criteria, meaning the result at day 30 that ends the test. And the scale criteria, meaning the result that earns a bigger budget for the next 60 days. If I cannot write those down, I am not ready to run the test.
I also set up measurement before launch, not after. UTMs that match the channel’s campaign names, the channel added as an option in the post-purchase survey, and the channel connected or added as a custom channel in my attribution tool so spend sits next to everything else. Then I save a two-week baseline of survey answers, branded search, and new customer orders, so I have something to compare against.
Week one: is it working mechanically?
The first week is not about performance. It is about whether tracking works. I click my own ads, confirm the landing URLs carry the right parameters, and check that orders from the channel are showing up in my attribution tool under the right name. I look at whether the platform is spending, on what placements, and at what CPMs. I make no performance judgment yet, because new campaigns on most platforms go through a learning period and the first week’s numbers rarely hold.
Week two: who is it reaching?
In week two I look at the share of the channel’s attributed orders that come from new customers. A new channel that mostly reaches people who already buy from you is not adding much, however good its ROAS looks. Retargeting-heavy channels are especially prone to this. I also look at first-order AOV against my site average, because a channel that brings in buyers at much lower order values changes the CAC math.
Week three: does anything outside the channel move?
By week three, survey responses start to say something. If customers are picking the new channel in the survey at a rate well above what the pixel attributes to it, the channel is probably influencing purchases through search and direct. I also check branded search impressions and direct traffic against the baseline. For channels that rarely get clicked, such as CTV or podcasts, this is the main evidence I have.
Week four: the decision
At day 30, I put three numbers side by side. Click-based new customer CAC from my attribution tool, which is usually the most conservative. The platform’s own reported CPA, which is usually the most generous. And survey share among new customers. I then compare that picture to the criteria I wrote down before launch. If it clears scale criteria, the budget grows for the next 60 days with the same measurement. If it falls between kill and scale, it gets one more month at the same budget with a specific change, such as new creative or a different audience. If it hits kill criteria, it stops, even if the platform report looks fine.
For the attribution layer, I use ThoughtMetric, which sponsors this blog, because it puts the new channel’s spend, new customer orders, and survey responses in the same place as the channels I already trust. Any tool that can do that will work. What matters is that you are not judging the channel only on the number the channel reports about itself. If the channel becomes a big budget line, that is when a geo holdout is worth the cost. I covered a lightweight version in running a cheap geo holdout.
What I have learned to ignore
Day-by-day ROAS swings, which are mostly noise on small budgets. View-through conversions on a platform’s default settings, which will almost always make a new channel look better than it is. And blended MER changes in the first month, unless the test budget is large relative to total spend, because a small test rarely moves the blended number in a way you can read. I explained why in why ROAS drops when you increase ad spend, which also matters for what happens when you scale a winner.
The 30-day framework
- Before launch, write down the budget, the kill criteria, and the scale criteria.
- Set up UTMs, a survey option, and a two-week baseline before spending.
- Week one, check tracking, not performance.
- Week two, check the new customer share and first-order AOV.
- Week three, check survey share, branded search, and direct traffic.
- Day 30, compare click-based CAC, platform CPA, and survey share against your criteria, then scale, extend once, or stop.
Leave a comment