# The Consent Banner Test That Measures the Wrong Thing

Source: https://cookiebastion.com/blog/2026-11-08-banner-test-wrong-metric.html

[Home](../index.md) / [Blog](index.md) / Consent Records: How Long to Keep Proof, and What to Keep / The Consent Banner Test That Measures the Wrong Thing


Published 2026-11-08

The test ran for two weeks and variant B won by a wide margin. The team shipped it, and the consent records got worse. Variant B had moved the reject button behind a settings link, and the higher accept rate was measuring how many people gave up looking for it.

Accept rate is the number every banner test reports first, and it is the wrong number to optimize. It measures how many people said yes, not whether the yes means anything. A banner that hides refusal will always win on accept rate, because refusal was never really on offer. The CNIL fined Google 150 million euros in 2022 for exactly that asymmetry: accepting took one click, refusing took several. The test that copies that design is not an optimization. It is a fine with a dashboard.

## What to measure instead

Measure consent quality first. Did the winning variant produce consent that matches what the site actually does? A variant that lifts acceptances while hiding the reject path has bought a higher number and a weaker record. Regulators read the banner, not the dashboard, and a record built on a banner that made refusal hard is a record that does not prove what it claims.

Measure the variant, not the pile. Each variant needs a version number and a frozen copy: the exact text, the button labels and order, the colours, the vendor list shown. Every consent record from the test has to carry the variant ID alongside the timestamp and the string. Without it, the records from the test are a single undifferentiated pile, and nobody can reconstruct which banner text produced which yes.

## The test worth running

Change one thing at a time: the button wording, or the reject placement, or the vendor list order. A test that changes three things at once tells you the winner moved the number and nothing about which change did it. Keep the test narrow, run it a full week so weekday and weekend visitors both appear, and decide in advance what a win means: not a higher accept rate, but the same accept rate with a cleaner record, or a higher rate with refusal just as easy as acceptance.

When the test ends, write the decision down: which variant won, what the numbers were, and the date the losing variant stopped serving. File it with the consent records. Six months from now, when someone asks why the accept rate jumped in November, that note is the only thing standing between a story and a shrug.

Keep reading: [CMP Pricing: Per Domain or Per Session, and What Each One Hides](https://cookiebastion.com/blog/2026-11-07-cmp-pricing-per-domain-vs-session.md), [Consent Records: How Long to Keep Proof, and What Counts](https://cookiebastion.com/blog/2026-11-06-consent-records-retention-proof.md), and [TCF Vendor Lists: Who Is Liable When a Vendor Misbehaves](https://cookiebastion.com/blog/2026-11-06-tcf-vendor-list-liability.md) cover the nearby cases in this series.

## Frequently asked questions

### Why is accept rate the wrong metric for a banner test?

It measures how many people said yes, not whether the yes means anything. A banner that hides refusal wins on accept rate because refusal was never really on offer, and the record it produces does not prove what it claims.

### What did the CNIL fine Google for in 2022?

150 million euros, in part for a banner where accepting took one click and refusing took several. The asymmetry itself was the violation, not the consent rate it produced.

### What should each banner variant record?

A version number, a frozen copy of the banner text, button labels and order, colours, and vendor list shown, plus the variant ID in every consent record collected while it ran.

### How long should a banner test run?

A full week at minimum, so weekday and weekend visitors both appear, and long enough that the result is not noise from a few hundred visitors.