The Ecomm Analyst

Something I do with every new account, usually in the first week, is take a single month of spend and run it through three attribution models without changing anything else. Same orders, same spend, same window. The only variable is the rule for dividing credit. It is a cheap exercise and it tends to end an argument before the argument starts.

The point is not to find the right model. The point is to see how much of your channel level picture is a property of the business and how much is a property of the accounting.

What the exercise looks like

Take a full month, ideally one without a big promotion in it. Pull channel level attributed revenue under last click, then under first click, then under a position based model. Put the three side by side in a spreadsheet with spend held constant and calculate ROAS for each channel under each model.

Total attributed revenue should be roughly the same in all three columns, because you are dividing the same pie differently. If the totals move a lot, that is its own finding and usually means the tool is applying different lookback windows alongside the different models. Worth knowing before you go further.

What tends to move

Branded search moves the most, and it moves the most predictably. Under last click it is usually one of the best performing lines in the account. Under first click it collapses, because almost nobody discovers a brand by searching a brand name they had never heard. That collapse is the clearest signal in the whole exercise. Branded search is capturing demand, and the interesting question is who created it.

Email moves the same direction for the same reason. It is a closing channel that looks like an acquisition channel under last click.

Meta prospecting moves the opposite way. Weak on last click, considerably stronger on first click, somewhere in between on position based. On several accounts I have looked at, Meta’s attributed revenue roughly doubles going from last click to first click while branded search drops by more than half. Neither number is the truth. The distance between them is the finding.

What barely moves is worth attention too. A channel that looks similar under all three models is either genuinely doing complete journeys on its own, or it is not getting connected to the rest of the journey at all because of a tracking gap. Those two situations look identical in a report and are completely different problems. When something sits suspiciously still, I go check the UTM setup for that channel before concluding anything.

The mistake I see people make with this

They run the comparison, find the model that makes the current strategy look best, and adopt it. Usually this happens right after a bad quarter, and usually the chosen model is whichever one flatters the channel somebody has been defending in meetings.

If you switch reporting models, you have to accept that everything before the switch is no longer comparable to everything after. Your trend lines break. Six months later somebody will ask why performance jumped in March and the honest answer is that you changed the ruler, which is not an answer anyone finds satisfying.

So I pick one model for reporting, keep it, and use the others strictly as diagnostics. Last click for the number in the weekly, because it is conservative and it does not let anyone claim revenue that did not clearly land nearby. The others get looked at when I want to understand a specific channel, not when I want a better headline.

What it does not tell you

None of this is incrementality. Every model in the comparison is dividing observed credit among touchpoints that were recorded. All three could rate a channel highly and that channel could still be entirely unnecessary, because none of them ever ask what happens when you turn it off.

The model comparison narrows down where to spend testing budget. If branded search looks great on last click and terrible on first click, that is the channel to hold out first. Model comparison generates the hypothesis and a holdout tests it. Skipping to the test without the comparison means you are guessing about what to test.

There is also a limit on the tooling side. If a tool only supports one model, the exercise is not available to you, which is one of the reasons I care about model flexibility when evaluating attribution software. I use ThoughtMetric, which sponsors this blog, because switching models across the same order data is a couple of clicks rather than a rebuild, and it holds the underlying order records constant while it does it. Northbeam supports this kind of comparison as well at a higher price point. GA4 will show you a version of it for free, with the caveat that its order data will not reconcile against Shopify and you will spend the afternoon explaining why.

Run it once a quarter. It takes an hour and the channel level shifts are usually stable, which is itself useful, because when the shape suddenly changes it means something in the tracking setup changed and nobody told you. For the underlying difference between the blended and channel level views, I covered that in MER, blended ROAS, and platform ROAS, and what each one is actually for.