Can't I Just Have AI Build My Marketing Mix Model? Why Garbage In Still Means Garbage Out

Linda Orr

5 days ago

7 min read

Updated: 4 hours ago

Short answer: AI can help with parts of a marketing mix model, like cleaning data, writing code and summarizing results. It can't decide what the model should test, which variables belong in it, or whether the answer it produces is true. Those judgment calls are the model. When they're skipped, by an AI tool or by an automated MMM platform, you get an answer that looks precise and may not be right.

Can AI build your marketing mix model? 4 in 10 cause-and-effect questions wrong, 12% of hidden errors found, 48% ROAS swing

When I was in my PhD program, we had a young statistics professor whose teaching style drove the older faculty a little crazy. They joked that he taught the "cookbook approach": load the data into SPSS, click run, read the output. That wasn't really fair. He covered the theory and he told us what should be done and why. He just didn't make us spend days working through proofs before we were allowed to touch real data, and he gave us fun problems to solve instead. For the record, I liked the way he taught.

But the joke has stuck with me, because I'm watching the cookbook approach take over marketing measurement. AI tools make it easy to drop a spreadsheet in and get a confident answer back. And for the last ten years, well before ChatGPT, I've seen many of the big MMM companies, MMM software platforms, and multi-touch attribution (MTA) companies do something similar at scale. Data goes in, a model comes out, and very little happens in between. This post explains why that's a problem, what should happen instead, and what the research says about AI's ability to do this work.

1. Can ChatGPT or Claude build my marketing mix model?

They can run one. Give an AI tool a few years of weekly sales and spend by channel and it will write the code, fit a regression or a Bayesian model, and hand you a chart of channel ROI. What it can't do is tell you whether that ROI is real. A marketing mix model is a causal claim: this channel caused this much revenue. Getting there depends on decisions that sit outside the data, and AI makes those decisions by default, silently, or not at all.

2. What does "garbage in, garbage out" mean for an MMM?

Every statistician learns this phrase early. If the data going in is wrong, incomplete or unrepresentative, the model coming out will be wrong too, no matter how sophisticated the math. In marketing data, the garbage is rarely obvious. It's a promotion nobody coded, a tracking change that cut reported conversions in half for three months, a store opening, a competitor's price cut, or a spike from one viral post that the model credits to whatever channel happened to be spending that week.

AI takes the data it's handed at face value. In SpreadsheetBench 2, a June 2026 benchmark built from real business spreadsheets, the best AI model found and fixed hidden errors only 12% of the time. In DSBench, a 2025 benchmark of real data analysis tasks, the best AI agent solved about 34% of them. Finding the problem nobody told you about is the hardest part of the job, and it's the part AI does worst.

3. How can the same data tell completely different stories?

Here's something anyone who has built statistical models knows and most buyers of marketing measurement don't. I can make a model say almost anything I want. Cut the sample one way instead of another. Drop a variable or add one. Make a relationship curved instead of straight. Add more outside factors or fewer. Throw in a few interactions. Each choice is defensible on its own, and together they can push the result in whatever direction someone would like. That's why you learn to read the fine print on studies paid for by someone with a stake in the answer.

Google's own researchers have made the same point about MMM. Their 2017 paper on the challenges of media mix modeling showed that models built on the same data can report very different ROI with the same level of statistical confidence. Google has also written that "the same data can produce very different results depending on how it's incorporated into the model," and cited Nielsen research where ROAS for individual video platforms varied by as much as 48% depending only on how the data was grouped. The modeling choices are the work. Hand those choices to software or AI without a strategy behind them, and you are letting chance, or a vendor's defaults, decide your budget.

4. What's wrong with the "throw the data in" approach?

Many automated MMM and MTA platforms, and now AI tools, follow the same basic recipe: collect every data feed, run variable selection, fit the model, and report whatever comes out. There may be some strategy layered on top, but there's very little of the work that should come first. Nobody asks how the weather affected sales that spring, whether last year's seasonality was normal, what competitors were doing, or whether the brand campaign that ran in the fall was actually any good before deciding how to represent it in the model.

The result is a model that fits the past well and explains it poorly. And when a model's story isn't grounded in what actually happened in the business, there's a strong temptation to sell it anyway. I think that's the biggest thing missing from a lot of big-company measurement, and it's missing almost completely from AI.

5. Why can't AI do the cause-and-effect part?

Because language models aren't built for causal inference, and the research shows it. On CLadder, a NeurIPS 2023 benchmark of formal cause-and-effect questions, GPT-4 answered roughly 62% correctly, meaning about 4 in 10 were wrong. A NeurIPS 2024 study asked whether language models actually help with time-series forecasting and found that removing the language model from those methods, or replacing it with a simple attention layer, didn't hurt accuracy and usually improved it. And when researchers asked the same questions five times with settings locked, model accuracy still swung by up to 10% between runs.

Michael Kaminsky, co-CEO of the MMM company Recast, put it plainly in AdExchanger in 2025: language models produce "confident-sounding but often wrong statistical analyses" and can't perform causal inference effectively. That matches what I see. The output reads well. Whether it's true is a separate question the tool can't answer for you.

6. What should happen before anyone runs the model?

Real statistics starts with a hypothesis. Before I build anything, I want to know what the data should say if our understanding of the business is right. What are we looking for? Which channels do we expect to matter, with what delay, and why? That comes from qualitative and strategy work: talking with the team, reviewing past campaigns, looking at the brand, the competition and the market, and understanding what changed and when.

Then come the variable decisions. Should weather be in the model, and measured how? Is seasonality a simple calendar pattern or something that shifted this year? Do competitor actions need their own variables? Was the branding work strong enough to matter, and if so, how should it carry over from week to week? How long does each channel's effect last, and at what point does more spend stop paying off? Each of those answers changes how a variable is built, and each one should be decided on purpose, before the model sees the data.

7. What tests make sure the data is clean and representative?

Before modeling, I test the data itself. I check for missing weeks, outliers and reporting changes that would distort the trend. I look at how closely channels move together, because when two channels always rise and fall at the same time, no model can cleanly separate their effects. I check for trends and breaks that need to be handled rather than ignored, and I make sure the period and the sample actually represent the business decision we're trying to make. Only then does the model get built.

8. How do you know a model is telling the truth?

A good model has to pass more than a fit statistic. It should forecast periods it hasn't seen, keep stable estimates when the data shifts a little, line up with any experiments or lift tests you've run, and tell a story that matches the qualitative research. If the model says a channel drove a huge share of revenue but nobody in the business can explain how, that's a reason to investigate before it goes anywhere near the board deck. I never try to sell a company a story its own data and market research don't support.

9. Where does AI actually help with MMM?

Quite a bit, as long as it's doing the right jobs. AI is useful for writing and checking data-cleaning code, documenting assumptions, flagging anomalies worth a closer look, summarizing model output for different audiences and drafting scenario comparisons. I use it for all of that. It speeds up the work around the model so more time goes into the decisions that make the model right.

Think of it the way my old professor taught: the software can run the analysis, but someone still has to know what should be done and why. That part hasn't changed, and it's the part you're paying for.

If you're comparing an AI tool or an MMM vendor with hiring someone, here is how I approach marketing mix modeling consulting, and what a marketing mix model costs.

10. What does a properly built MMM cost?

My custom marketing mix models typically run $8,000 to $40,000, depending on how many channels and markets are involved, how much history you have, how clean the data is, and whether you want ongoing refreshes. I break down what drives the price in How Much Does Marketing Mix Modeling Cost?, and you can get a rough estimate with the calculator on my marketing mix modeling page. I built Upwork's first enterprise marketing mix model across an $88 million annual media portfolio, and every model I build starts with a hypothesis long before it touches the software.

If you're not sure your data is ready, start with the free Marketing Measurement Scorecard, or book a free 30-minute call and I'll tell you what it would take.

Keep reading: Is Being Data-Driven a Good Thing? | Marketing Mix Modeling vs. Attribution | Orr Consulting: PhD-Led Fractional CMO Services