The Ecomm Analyst

Growing stores, one honest take at a time.

How to understand the impact of ChatGPT ads before BFCM

Black Friday is November 27 this year. That leaves about seven weeks, which is just enough time to find out whether ChatGPT ads deserve a place in your holiday budget. Not enough time to find out perfectly, and not enough time if you start in November. The brands that will know something by Thanksgiving are the ones that launch a disciplined test in the next week.

Testing during BFCM itself is the mistake I would avoid. Competition for attention is at its peak, every conversion rate is inflated by discounts, and returning customers flood in regardless of what you spend. A channel’s numbers during sale week tell you very little about how it performs the rest of the year, a problem I covered in how to read ROAS during a sale or promotion. You want the read before the noise, so you can make a calm budget decision for the sale.

Week one: set up before you spend

Four things go in place before the first dollar. Install OpenAI’s pixel or set up its Conversions API, because conversion-optimized campaigns depend on it. Put tracking parameters on every ad so your own attribution tool can match orders to campaigns. Add ChatGPT as an answer on your post-purchase survey. And write down your baseline for branded search volume and direct traffic over the last four weeks, since that is where delayed ChatGPT buyers tend to show up.

Then decide your success criteria in writing. How many orders do you need before you judge it? What new-customer CAC would make it worth scaling? If you set these after the results come in, you will find a way to like whatever you see.

Weeks two through five: hold still

Run a fixed daily budget for four weeks and do not touch it. That is the hardest part for most teams, because the instinct is to cut what looks weak after five days. Four weeks gives you a full cycle of people who saw an ad, kept researching, and came back later. Change bids or budgets halfway through and the result becomes impossible to read.

Size the budget so it produces enough orders to mean something. If your average order value is $80 and you would accept a $60 new-customer CAC, a few hundred dollars a week will not produce enough orders to tell signal from noise. Better to run a meaningful test on one product category than a token test across the catalog.

Keep in mind the platform is still moving. OpenAI announced on October 5 that it will start testing a new visual ad format later this month with an initial group of US advertisers. If you are not in that group, your test runs on the existing formats, and the result may not transfer cleanly to whatever ships next.

Week six: read it three ways

Look at attributed orders in your own tool, not the ROAS OpenAI reports. Expect your number to come in lower than the platform’s, the same way it does on Meta. Then compare survey mentions with attributed orders. If far more buyers say they found you through ChatGPT than your attribution credits, the channel is influencing purchases that click tracking misses. Finally, check branded search and direct traffic against your baseline. A clear lift there during the test, with nothing else changed, is supporting evidence.

Pay close attention to the new-customer share. In OpenAI’s announcement, Triple Whale reported that 93 percent of one brand’s ChatGPT Ads visitors were new. If your test shows mostly returning customers, something about your targeting or creative is off, and that matters more than ROAS at this stage.

Week seven: decide the BFCM budget

You will end up in one of three places. Strong new-customer CAC and survey support means you increase spend for BFCM, modestly, because sale-week costs run higher. Weak attributed numbers but strong survey and branded search signals means you hold budget flat through the sale and extend the test into December. Weak on every measure means you pause and revisit in the new year, when the platform has matured.

For the measurement itself, any attribution tool with a ChatGPT Ads integration will do the job, and I compared them in tools that measure ChatGPT ads. ThoughtMetric, which sponsors this blog, reports ChatGPT Ads down to the ad level with new and returning customers split out, and it is the setup I would use for this kind of test.

The short version

  • Launch this month, not in November.
  • Set up the pixel, tracking parameters, survey answer, and branded search baseline before spending.
  • Run a fixed budget for four weeks without touching it.
  • Judge on new-customer CAC in your own attribution, survey mentions, and branded search lift.
  • Use the result to set a modest BFCM budget, or to decide to wait.

Leave a Reply

Navigation

About

Six years in e-commerce. Three Shopify stores across different niches, one scaled past seven figures. I’ve tested hundreds of ad creatives, obsessed over email flows, and learned more from my failures than my wins.

Now I focus on conversion optimization, retention marketing, and the analytics behind it all. This blog is where I share what actually works, backed by real numbers. No fluff, no guru energy.