Measuring it
That is not a bug and it does not settle down, which is why a single check tells you almost nothing. Here is what a tracking setup actually needs, and the trap that makes a number go up when nothing has happened.
The short answer
Every part of that sentence is doing work. Drop the repeats and you are reporting noise. Drop the fixed question set and you are reporting your own choices. Drop the held-back fifth and you can never say the change was yours.
The thing that breaks naive tracking
This is not a bug and it does not settle down. These models sample, and a live web search adds variance on top. Any number built from one run per question is a draw, not a measurement.
| Requirement | Why | Without it |
|---|---|---|
| Repeats per question | The answer is sampled. Several runs give you a rate instead of a coin flip. | Your number moves week to week for no reason and nobody trusts it. |
| A fixed question set | Whoever writes the questions decides the score. | Every report is subtly re-scoped and the trend line is meaningless. |
| Confidence intervals | A rate from a small sample has a wide range, and the range is the honest part. | You celebrate 33% rising to 38% when both readings say “somewhere between a quarter and a half”. |
| Citations, not just mentions | Being named matters; being named because of your own page is the thing you can act on. | You know you appeared and have no idea what to change. |
| Official APIs | Scraping a consumer app gives you a different product, against its terms, and it breaks silently. | Data you cannot defend, from a pipeline that dies on a UI change. |
| A holdout set | Questions nobody works on are the only way to separate your effect from the market's. | Every improvement is a story. The method is here. |
A trap we walked into
Engines restate the question before answering it — “Here are the companies relevant to…” If your company name is in the question, that restatement gets counted as a mention, and your rate goes up without anything happening.
We found this because one question came back at 100%. In a run where the underlying probability was known to be 0.3, the measured rate was 1.0. Stripping the restatement — the full repeat of the question, and any sentence that is mostly the question's own words — brought it to 25%, which is what it should have been. A tracker that does not do this reports a number that rises when you mention yourself.
What a reading looks like
########............ 40% licensing opportunities
########............ 40% selective versus broad: trade-offs
######.............. 30% which companies are developing…
####................ 20% assets available for out-licensing
questions where you appear at all 8 of 8
mention rate across all runs 33% (95% CI 23–43%)
answers citing your own pages no citations
80 runs · 0 failures · holdout questions untouched
The interval is the honest part. 33% is not a number, 23–43% is. If your next reading is 38%, nothing has been demonstrated — and a tracker that reports the point estimate alone will let you believe otherwise.
Before you buy anything
Write down ten questions a buyer would actually type — describing the problem, not naming you. Ask them. Write down whether you appeared and what was cited. It costs an afternoon and nothing else, and it will tell you whether there is a problem worth spending on. We publish what the paid version costs us to run, which is cents per question, so that you can tell what you are being charged for.
What the free version cannot give you is a number that moves. For that you need the repeats, the fixed set, the schedule and the holdout — and at that point it has stopped being something anyone does by hand.
Questions
Because these models sample rather than look up. The same question, same engine, same minute can name a different set of companies, and a live web search adds a second source of variance on top. This is the single most misunderstood thing about measuring AI visibility: one run is a draw from a distribution, not a reading.
For a first look, yes, and we recommend it — it costs nothing and it is usually enough to tell you whether you have a problem. It stops being enough the moment you want a number that moves, because you need the same questions asked the same way on a schedule, with repeats, and nobody does that by hand for long.
Weekly is the useful default for a programme that is actively changing things, because the corrections you make take days to weeks to show up. Daily measurement mostly buys you noise at a higher bill. The thing that matters more than frequency is that the question set stays fixed.
Enough to cover the ways a buyer describes the problem, which is usually fifty to two hundred rather than ten. Ten questions can be chosen, consciously or not, to produce a flattering number — and a small set gives a confidence interval so wide that no change you make will ever clear it.
Related reading
The holdout in full: how it is drawn, how it is fenced off, and what it still cannot tell you.
Nine questions that separate a measurement from a dashboard. No vendor is named.
Our own measured figures for what a question actually costs to ask properly.
What to do once the measurement tells you the record is wrong.
Start here
The free diagnostic: how public sources describe your company and your main product today, where they disagree, and which disagreement to fix first. Every line links to the page we read it from. Nothing is published anywhere without your written approval.