Promotion Holdouts: Measure Incremental Contribution, Not Coupon Use

In this article

A redeemed coupon does not establish that the coupon caused the order. Some customers would have purchased at the normal price; others simply bought earlier. Evaluating a promotion requires an estimate of what would have happened without it.

Google’s Conversion Lift explanation uses exposed and control groups to assess incremental conversions. The workflow here is a proposed store-promotion experiment, not a claim that a particular advertising feature is available to every account.

Choose the outcome before sending the offer

Define eligibility and measure contribution per eligible customer after relevant goods, discounts, fulfilment, payment and return costs. Coupon redemption alone is a weak commercial outcome. Set the measurement period in advance rather than selecting the most flattering day after the results arrive.

Build a credible comparison

Randomly assign eligible customers where practical and limit offer leakage into the control group. Check balance in prior purchasing, location and previous order value. Comparing an offer sent to loyal buyers with first-time visitors creates a misleading baseline.

In a hypothetical example, each group contains two hundred customers. Net contribution is QAR 1,800 in the offer group and QAR 1,600 in the control group. The arithmetic difference is QAR 1 per eligible customer, or QAR 200 for the exposed sample, before any additional setup costs omitted from those figures.

Report uncertainty alongside the estimate

That difference does not establish a durable effect. One unusually large order or ordinary random variation could drive it. Review the distribution, sample size and uncertainty, and use suitable statistical analysis before expanding. With sparse sales, “inconclusive” is more useful than declaring an unsupported winner.

Look beyond the discount window

Observe returns and subsequent purchasing over a defined follow-up period. A promotion can move an order forward rather than create new demand. Measure across the full window and record simultaneous changes in advertising, pricing or stock availability.

Begin with one offer and one customer segment. Write down the decision for a positive, negative or inconclusive result before launch. The experiment should tell the team what to change, rather than merely produce a larger coupon-redemption count.