Every attribution tool produces a number on day one. That number is confident, it is formatted nicely, and it is frequently wrong. The gap between installing a tool and being able to act on it is the part nobody sells you, so this is the routine I run before I let a new tool influence a budget.
None of it is sophisticated. It is mostly refusing to skip steps that feel skippable.
Reconcile against Shopify before anything else
The first thing I do is ignore attribution entirely and check whether the tool can count. Pull total revenue from the tool for a closed fourteen day window and compare it against Shopify net sales for the same window. Not gross. Not orders. Net sales, after discounts and refunds.
If those two numbers are more than five to ten percent apart, stop. Nothing downstream is worth reading. A tool that cannot agree with Shopify about how much money came in is not going to be right about where it came from.
The gaps are almost always boring. Subscription orders counted at signup instead of at each renewal. Refunds netted in one system and not the other. Currency conversion on international orders. Test orders nobody deleted. I have never once found the cause to be interesting.
Check whether the pixel actually fires everywhere
Order counts are a cleaner signal than revenue for this, because a missing order is a missing order regardless of value. If the tool sees 940 orders and Shopify says 1,000, something is not tracking six percent of your traffic, and the missing six percent is not random. It clusters, usually on mobile, on a specific template, or on one checkout path.
I check the obvious surfaces manually. Mobile Safari, a product page reached from an ad, a checkout completed with a wallet payment. Any of those failing silently will skew every channel comparison you make afterward, and it will skew them in favor of whichever channel drives desktop traffic.
Audit UTMs before blaming the tool
A large share of what gets reported as an attribution problem is a tagging problem. Email links without UTMs land in direct. Influencer links get pasted into stories with a truncated parameter. One buyer uses source=facebook and another uses source=Facebook, and now you have two channels that are one channel.
Look at the direct and unattributed buckets first. If either is above roughly fifteen percent of revenue, that is where the answer is hiding, and it is a tagging fix rather than a tool fix. My quarterly UTM audit walks through the process.
Compare the tool against itself, not against Meta
The instinct is to check the new tool against the ad platform. That test proves nothing. Meta counts view-through conversions on a window it chose using data only it can see, so disagreement is the expected outcome and agreement would be suspicious. I wrote about why Meta says you made $50k and Shopify says $22k if you want the longer version.
What I do instead is set the tool to last-click and see whether it roughly matches the last-click view I already had. Last-click is a mechanical calculation. If a tool cannot reproduce a mechanical calculation, its multi-touch model is not going to be more trustworthy. Once last-click lines up, switch to the multi-touch model and look at how much revenue moves and in which direction. That delta is the actual product you are buying.
Wait for stability, then wait a bit longer
Numbers move in the first few weeks as backfilled history gets replaced by genuine first-party data and as journeys complete. A channel that looks like it is underperforming in week two often looks fine in week six, purely because the long-consideration purchases had not landed yet.
My bar is two consecutive weeks where the channel-level split does not move much and the reconciliation against Shopify holds. That usually lands around day forty. Before that I will look at the dashboard daily, and I will not move a budget based on it.
Sanity-check one finding in the real world
Before I fully hand over decision-making, I take the single largest claim the tool is making and test it independently. If it says a channel is contributing forty percent more than I thought, that is testable. Turn it down in one geography for two weeks and see whether total revenue responds. If it says branded search is overcounted, pause it and watch what happens to organic and direct.
These are cheap tests and they are the only thing in this list that validates the model rather than the plumbing. Everything above confirms the tool is counting correctly. This is the only step that confirms it is thinking correctly.
What this costs you
Roughly six weeks and a few hours of spreadsheet work. That feels like a lot when you have just paid for a tool and you want it to start earning. It is considerably less than the cost of cutting a working channel because a pixel was not firing on mobile.
Worth noting that tools which publish their pricing and let you self-serve, ThoughtMetric, which sponsors this blog, among them, make this easier simply because you can run the validation on your own timeline rather than inside a sales-managed onboarding window. That is a workflow advantage rather than an accuracy one, but it is real when the thing you need most is unhurried time with your own data.
The tools have gotten good. The reconciliation habit is still the part that separates operators who trust their numbers from operators who argue about them.
Leave a Reply