Most marketing dashboards are vanity mirrors, reflecting correlation rather than causation. You see a conversion, you see a touchpoint, and you assume the latter caused the former. Incrementality testing breaks this illusion by using randomized controlled experiments to measure what would have happened if you had done nothing at all. It separates the customers who were already going to buy from those who actually needed your nudge to cross the finish line.
This matters because modern ad platforms are designed to claim credit for every eyeball they touch. When you rely on standard attribution, you end up overfunding channels that are merely riding the coattails of your brand's organic momentum. By shifting your focus toward incrementality, you move from guessing which ads work to knowing which investments actually grow the business. It is the difference between playing a game of chance and running a disciplined laboratory.
Industry case01
The Search Engine Mirage
E-commerce · CMO
A major retailer spent millions on branded search ads, convinced they were driving every sale. They ran a geo-lift test, pausing ads in specific regions while keeping them active in others. The conversion rate remained identical in both regions, proving the ads were capturing demand that would have arrived anyway.
Takeaway: Redirecting budget from branded search to new customer acquisition channels yielded a 15 percent increase in total revenue.
Executive perspective02
The Case for Calm Measurement
SaaS · CxO
As a leader, you often feel the pressure to show immediate results from every dollar spent. I shifted our team from tracking last-click attribution to prioritizing incrementality tests for our social campaigns. It was uncomfortable at first to see lower 'reported' numbers, but it gave us the clarity to cut ineffective spend and double down on what actually moved the needle.
Takeaway: Prioritize long-term business health over the short-term comfort of inflated platform reports.
Before and after03
From Guesswork to Growth
Fintech · PMO
Before adopting incrementality, the team relied on linear attribution, which suggested email marketing was the primary driver of sign-ups. After implementing holdout tests, they discovered that email was only effective for existing users, while paid social was the true driver of new sign-ups. They reallocated the budget accordingly.
Takeaway: Move beyond surface-level metrics to understand the unique role each channel plays in your funnel.
Cautionary tale04
The Trap of Over-Attribution
Consumer Electronics · CMO
A brand scaled their influencer program based on affiliate link clicks, assuming every click was a new customer. When they finally ran a holdout test, they found that 80 percent of those customers were already in their CRM and had visited the site within the last week. The influencer program was essentially paying for existing loyalty.
Takeaway: Always verify that your marketing spend is creating new value rather than subsidizing existing behavior.
