The problem
A score is a sample, not a measurement.
Two hundred questions a week stand in for millions of phrasings. Whoever picks the two hundred decides what the number says.
The same question answers differently by session, by account, by region, and differently again after a model update nobody announced.
A competitor's press release, a journal paper, a registry update. Your number changes and none of it was work anyone did for you.
Each of these makes a visibility score drift on its own. A number that rose is not evidence until you can say what it rose against.
The method
How do you prove AI visibility work caused the improvement?
You hold a fifth of the questions back and work on nobody’s behalf there. If the worked questions rise and the held-back ones do not, the difference is yours. If both rise, the market moved — and the report has to say so.
50–200 questions a buyer would actually type, approved by you before anything runs. Each one is tagged with the buying stage it belongs to.
Twenty percent is selected at random, stratified across stages so the control is not accidentally all easy questions. The draw is recorded with the panel version.
Held-out questions are excluded from every pipeline — no content, no pages, no outreach is built from them. Tests in the codebase fail if one leaks through.
Both sets are measured on the same schedule against the same engines. What we claim is the gap between them, not the level of either.
Limits
What this still cannot tell you.
A control group fixes attribution. It does not turn a sampled metric into a census, and anyone who says otherwise is selling something.
Tell you what real users saw
We query engines through official APIs. That is what the engine will say, not a log of what buyers were shown. No third party has the latter.
Survive a tiny panel
Below roughly fifty questions the holdout is too small for the difference to mean much. We say so rather than reporting a number with a straight face.
Separate spillover
Fixing a registry entry can lift held-out questions too. That makes our measured lift conservative — understated, never flattering.
Start here
See the version of this
that is about your company.
The free diagnostic does not need a holdout — it is a snapshot of where public sources already disagree about you, with every line linked to the page we read it from. The measurement above starts when a program does.