I paused all paid media in one state for 21 days. The incremental lift was 38%, not the 92% the attribution report claimed.

Quick Take

Attribution reports do not measure incrementality. They measure correlation between the click and the order. The board reads “Meta drove 38% of revenue” and assumes that without Meta, 38% of revenue goes away. That assumption is wrong by a wide margin in almost every home-brand account I have audited. The cheapest way to find the actual number is a 21-day geo holdout: pick a small market with a stable trend, pause all paid media in that geography for 21 days, measure organic revenue against the same market in the prior comparable period. The delta is the true incremental lift. Across the last four holdouts I have run on home brands the lift has landed between 30% and 60% of the reported attribution number. The gap funds the next 18 months of budget decisions.

The receipt

Most recent holdout: a home-furnishings brand on Shopify spending $48K/month across Google Ads and Meta. Reported blended ROAS was 4.1x. Reported attribution said 92% of revenue was paid-influenced.

I picked Oregon as the holdout market. 4.3% of revenue, stable 90-day trend, no concentration of physical retail to skew the result. Paused every Meta and Google campaign that targeted or expanded into Oregon for 21 days. Left all email, organic social, and SEO running as normal. Held the rest of the country at the existing spend.

The Oregon result over the 21 days:

• Total revenue from Oregon: down 38% vs the same 21-day window in Q1
• The other 49 states: down 2% vs Q1 (within seasonal noise)
• Organic search traffic from Oregon: down 6% (small halo from paid social driving branded search)
• Direct traffic from Oregon: down 11%. The bigger halo. Paid had been building brand recall.

True incremental lift on Oregon paid spend: 38%. The reported attribution number had been 92%. Smart Bidding had been claiming credit for the other 54% that would have happened anyway through email, organic, retention loops, and a customer base that was already going to buy.

That is the gap the board needs to see. The budget conversation that comes out of “92% incremental” is a different conversation than “38% incremental.” Different number, different decision.

The geo holdout, defined

The cleanest definition: a controlled experiment where you remove paid-media spend from one geographic market for a fixed period, hold the rest of the account constant, and measure what happens to revenue in the test market vs the control markets.

Geo splits matter because most other holdout types (audience splits, dayparting splits, channel splits) cannot fully prevent cross-contamination. Audiences see ads in both arms. Dayparting still gets brand recall from yesterday. Channel splits still see the same buyer through other channels. Geography is the only dimension where you can confidently say “this person was not exposed to the paid media we removed.”

Pick a market that is:

• Small enough to absorb the revenue loss. 3-7% of total revenue is the sweet spot. Bigger and the holdout costs real money. Smaller and the result has too much noise to trust.
• Stable. A market that was 5% of revenue in Q4, 8% in Q1, and 4% in Q2 has too much variance for a 21-day signal to land cleanly. Pick a market with a 90-day flat trend.
• Not concentrated in a physical channel. If you have a flagship store in Brooklyn, do not pick New York. The store traffic confounds the result.
• Not on the receiving end of a recent press hit or seasonal event. Same logic. A confound you cannot subtract out.

Most home brands have one or two states that fit these criteria. Often Oregon, Colorado, Tennessee, Minnesota, or Arizona work well. Cycle the holdout state between quarterly tests so the same market does not get repeatedly starved.

The 21-day protocol

The full setup runs over 5-6 weeks calendar time. 1-2 weeks of baseline, 3 weeks of holdout, 1-2 weeks of analysis.

Days 1-7: Baseline.

Pull the prior 90 days of daily revenue and order count for the candidate holdout market and at least 3 control markets of similar size. Confirm the holdout market’s 90-day trend is flat or matches the rest of the account. If a confound shows up (seasonal spike, store opening, PR mention), pick a different market.

Document the existing paid-media spend pattern in the holdout market: daily spend across Google Ads + Meta + any other paid channel, broken down by campaign type. This becomes the “what we removed” reference for the result write-up.

Days 8-28: The holdout.

Day 8, exclude the holdout state from every paid campaign across every channel. The cleanest way:

• Google Ads: location exclusion on every campaign (campaign settings → locations → exclude → add the state). Performance Max requires the exclusion to be set on the campaign before it propagates to all asset groups.
• Meta: location exclusion on every ad set targeting expansion that includes the state. Also exclude from any lookalike audience that defaults to broad geography.
• Microsoft Ads (if running): location exclusion on every campaign.
• Programmatic / display: exclude via the DSP’s geo controls.
• Affiliate / influencer: typically excluded by default (they target audience, not geography), but confirm with the partner.

What stays running: organic social posts that may surface in the state (do not pull these, they are part of the control), email to in-state subscribers (also part of the control), SEO content the state may rank into.

Watch for spillover. If a campaign somewhere else is targeting “Pacific Northwest,” that includes Oregon and needs to be tightened. The audit pass on day 8 should verify zero paid impressions are firing in the holdout market.

Days 29-35: Restart + analysis.

Day 29, restore every exclusion. Paid media resumes in the holdout market at the prior spend level. Most accounts see paid revenue in the holdout market return to baseline within 7-10 days as the bid models relearn the geography.

Pull the 21-day revenue numbers from the holdout market and compare to the 21-day window from the prior comparable period (Q1 if you ran the test in Q2, etc.). Adjust for any baseline differences across the control markets. If the rest of the country was up 5% YoY, the holdout market’s “expected” revenue should also be up 5% YoY before you measure the delta.

The delta after that adjustment is the incremental lift the paid spend was producing.

How to read the number

Three common result patterns I see:

Result A: incremental lift is 20-40% of reported attribution. Most accounts I have run this on. The paid is working, but a lot of the credit is going to a buyer who would have converted through retention, email, or organic. The action: keep the paid running, but stop using reported ROAS as the budget input. Plan against the incremental number.

Result B: incremental lift is 50-70% of reported attribution. Less common. The account is acquiring efficiently and the attribution is closer to reality. The action: lean in. The reported number is closer to the truth than I’d usually assume.

Result C: incremental lift is under 20% of reported attribution. Rare but it does happen, almost always on accounts with a mature email list or a strong organic presence. The action: budget conversation gets uncomfortable. The paid is mostly harvesting demand that would convert anyway. Reduce spend, redirect to upper-funnel demand creation, retest.

Result A is the typical one for a $10M+ home brand. The reported ROAS makes the account look like a 4x machine. The incremental ROAS shows the account is closer to a 1.5x machine. Same revenue, very different decision-making frame.

What this changes about the budget conversation

The board-deck memo flagged incrementality lift as one of the four numbers a board should see in a quarterly paid-media update. A geo holdout is how you generate that number.

Without one, the board is making capital-allocation decisions on the platform’s most-generous attribution model. With one, the board has a real read on what fraction of the next budget increment is buying new revenue vs harvesting revenue that was already coming.

For a $10M+ home brand, the gap between “92% incremental” and “38% incremental” on a $500K annual paid budget is the difference between budgeting for growth and budgeting against an inflated number. One holdout per quarter is enough to keep the budget honest.

Common failure modes

Skipping the baseline week. If you start the holdout without 90 days of clean baseline data, you have no comparison set. The result will be noisy and unconvincing to a skeptical board member. Spend the first week on the baseline.

Picking a market with a confound. A holdout in the same month as a regional PR hit, a store opening, a state-specific holiday, or a major weather event is unreadable. Pick a different state or a different quarter.

Not adjusting for control-market trend. If revenue across the rest of the country dropped 8% during the holdout period (because the whole vertical softened), the holdout market’s expected revenue would have dropped 8% too. The incremental-lift calculation has to subtract that baseline before measuring the delta.

Cutting paid mid-test. Three weeks feels long when the dashboard shows revenue dropping in the test market. The board may push to end the test early. The test ending early invalidates the result. Hold the line.

Running once and stopping. A single holdout in a single market in a single quarter gives you one data point. The pattern only becomes trustworthy after 3-4 holdouts across different quarters and different markets. Cycle the holdout state quarterly. Build a 12-month picture.

Keep going

If this hit, the next two pieces in the same universe:

• The 4 numbers I put in a $20M home brand’s next board deck on paid. The board-prep memo that named incrementality lift as one of the four numbers. This post is the methodology behind the slide.
• Why Performance Max Gets Credit for Shopping Conversions. The other attribution misread that leads boards to cut the wrong budget.

Free PDF: The 25-page Google Ads Setup Audit. The audit walkthrough behind the tracking-side of the budget conversation.

If your quarterly board deck still uses reported ROAS as the headline number and you have not run a holdout to verify it, the setup call is the audit conversation. One holdout in the next 90 days resets the entire budget frame.