Every quarter, we sit with a CMO and walk through a marketing dashboard that says paid social drove 38% of attributed revenue. And every quarter, we run an incrementality test and discover that turning paid social off changes revenue by 4%. Attribution is broken. Here’s how we work around it.

Why multi-touch attribution lies

MTA models try to allocate credit across touchpoints. The math is elegant. The problem is that the inputs are wrong:

  • Cross-device journeys aren’t connected because third-party cookies are dead.
  • iOS attribution is delayed, aggregated, and modeled — not measured.
  • Last-touch dominates because that’s the only data the dashboard actually has.

The result: every model over-credits the channel closest to conversion (usually paid search or retargeting) and dramatically under-credits the channel that created the demand in the first place (usually content, email, or word of mouth).

What we do instead: incrementality testing

Incrementality measures what changes when you turn a channel off (or up, or down). It’s the most honest measurement we have, because it doesn’t rely on attribution at all — just causal observation.

The 3-step framework

  1. Pick a channel you suspect is over-credited. Usually paid social or branded search.
  2. Run a geo holdout for 2–4 weeks. Turn the channel off in matched-pair geos and watch what happens to total revenue (not channel revenue — total).
  3. Compare the holdout group’s revenue to the control’s. The gap is the channel’s true incremental contribution.

What we typically find

Across the 18 incrementality tests we ran last year:

  • Branded paid search was over-credited by ~70% on average. Most of those clicks would have come through organic search anyway.
  • Email was under-credited by ~40%. The last-click model gave it almost no credit; the holdout showed it carried huge weight.
  • Content/SEO was the most under-credited — often appearing as “direct traffic” but actually doing the heaviest lifting in the demand stack.

How to start without a data team

You don’t need a full MMM build to do this. Three things will get you 80% of the way there:

  1. Pick one channel you suspect is over-credited. Test it.
  2. Use matched-market design (DMA-level holdouts) — there are free tools for this.
  3. Run the test long enough to clear noise. Two weeks minimum, four is better.

If you want help designing your first incrementality test, that’s literally what our data practice does. See how we work or drop us a note.