An AI Visibility Score Without Error Bars Is a Horoscope

An AI visibility score without error bars is a single number with no way to check it. This piece works through one illustrative, fully specified example: a run of 40 evaluated responses (n=40), allocated as 14 GPT responses, 13 Claude responses, and 13 Gemini responses on 2025-03-01 compared against a second run of 40 evaluated responses with the same 14/13/13 allocation on 2025-06-01, to show why a 10 percentage point (pp) shift in the observed mention rate (35% to 45%) cannot be interpreted from point estimates alone when the two 95% confidence intervals overlap. No live client data is used anywhere in this example; every figure below is constructed to demonstrate the statistical method, not pulled from a production account. The remainder of the piece specifies the fields — sample size, model list with version identifiers, run date, and interval — that any AI visibility report needs before its headline number carries meaning, and it states up front what this method does not cover: search rankings, referral traffic, or revenue attributable to a mention.

Mention rate: the share of evaluated model responses in a stated run that contain a brand or domain mention under a rule fixed before the run. Wilson score interval: a confidence interval for a binomial proportion. NIST's handbook on confidence intervals for a proportion (page last modified 2025-09-12 02:15 GMT; retrieved 2026-09-06, UTC) presents the Wilson formulas and notes that the method does not strongly depend on the values of n and p, and that unlike the common normal-approximation interval its lower limit cannot become an impossible negative value.

TrustGrowth has argued elsewhere that composite scores need an uncertainty label attached; see how the TrustGrowth score is kept honest for that broader discussion. This piece narrows the argument to one metric: the LLM mention rate, its sampling error, and one appropriate statistical method, the Wilson score interval, used here to bound it, a level of formula-level detail the composite-score discussion does not cover.

What an AI visibility score without error bars actually measures

The typical construction sends a fixed set of prompts to one or more large language models, checks each response for a brand or domain mention, and aggregates the hits into a single mention rate. That rate is a sampled proportion — a binomial rate — not a census of every query a real user could type into a chat interface. A rate built from 30 prompts describes those 30 prompts, not the space of category intent behind them. The specific fields a run should log are described in how TrustGrowth measures AI visibility, and the rationale for tracking this metric at all is covered in why visibility belongs in the TrustGrowth score.

AI visibility has an additional, visible source of run-to-run variance: a generative model can return different text for the same input. Rank observations can vary too because of location, personalization, index changes, and test conditions, so this article does not treat rank tracking as deterministic. The narrower point is that a sampled mention proportion needs its denominator and uncertainty reported.

Method: how a mention rate is produced

An AI visibility measurement is built from a fixed procedure, and that procedure has to be disclosed before the resulting number carries weight.

• Data source: responses returned by LLM APIs — Anthropic's Claude, OpenAI's GPT models, and Google's Gemini — to a defined prompt set.
• Sample size (n): the count of evaluated responses in the run, with the allocation across prompts and models stated. There is no universal minimum in this article: report the actual denominator and let the interval show the sampling precision it supports.
• Date range: a single run date, or a defined window if runs are batched, logged in ISO 8601 format with the timezone stated (UTC unless otherwise noted).
• Filters: what counts as a "mention" — a brand-name string match, a domain citation, or a linked source — defined before the run, not after.
• Exclusions: prompts that return errors, refusals, or empty completions, removed from the denominator and reported as a separate count.
• Tooling: the API client and model version identifiers used, since a vendor-side model update changes the instrument mid-measurement.

Why an AI visibility score without error bars is unfalsifiable

A change from 41 to 44 is unreadable without an interval, because there is no way to tell whether it reflects a real shift in the underlying mention rate or ordinary sampling noise from a 40-response draw. The statistical reason is direct: a sampled proportion is an estimate of an unknown population proportion. NIST's confidence-interval guidance for proportions (page last modified 2025-09-12 02:15 GMT; retrieved 2026-09-06, UTC) gives the Wilson limits and explains why the common normal-approximation interval can produce impossible bounds. AI visibility reporting that omits the denominator and interval hides the precision its sample actually supports, the same category problem described in estimated scores that misrepresent certainty.

The three sources of variance in an AI visibility score

Three distinct mechanisms push a measured mention rate away from the underlying true rate, and each needs a separate fix rather than a single blanket disclaimer.

Prompt-sampling variance

A small prompt set produces a wide interval by definition, because a proportion estimated from few trials is imprecise regardless of how carefully the trials were chosen. Using the Wilson score interval formula published in NIST's confidence-interval guidance for proportions (page last modified 2025-09-12 02:15 GMT; retrieved 2026-09-06, UTC), a 95% interval around an observed 50% mention rate spans roughly 40pp wide at n=20, narrowing to roughly 14pp at n=200. The fix is not a cleverer prompt list; it is more prompts, or an honest acknowledgment that the interval stays wide at low n.

Model non-determinism

Generative output can vary across repeated calls, so the identical prompt sent to the same model can return a mention in one response and not another. OpenAI's evals documentation (page last modified 2026-09-06 05:34 GMT; retrieved 2026-09-06, UTC) treats evaluations as essential when upgrading or trying new models, because performance against a fixed prompt is not assumed to stay constant. This is a property of the generation process, not measurement error in the traditional sense, but it still adds variance a single-run score hides.

Model-version and date drift

Model behavior can change between snapshots, so a score from 2025-03-01 and a score from 2025-06-01 is not directly comparable unless the exact provider-returned model identifier is logged with each run. OpenAI's deprecations page (retrieved 2026-09-06, UTC) states that software relying on OpenAI models may need occasional updates as older models are retired, and it publishes minimum notice periods before retirement. For other providers, preserve the exact identifier returned by that provider rather than substituting a family name. An unlogged historical score may therefore represent a different measurement instrument.

What an error bar on an AI visibility score should show

The Wilson score interval is an appropriate method for a binomial proportion. NIST recommends it (page last modified 2025-09-12 02:15 GMT; retrieved 2026-09-06, UTC) broadly across combinations of n and p, and notes that unlike the common normal approximation its lower limit cannot become an impossible negative value. The interval is computed as follows, where p is the observed mention rate, n is the number of evaluated responses, and z is 1.96 for a 95% confidence level:

def wilson_interval(p, n, z=1.96):
    denom = 1 + z**2 / n
    center = (p + z**2 / (2 * n)) / denom
    margin = z * ((p * (1 - p) / n + z**2 / (4 * n**2)) ** 0.5) / denom
    return center - margin, center + margin

Five fields belong on every published figure:

Field What it must state Sample size n, the count of evaluated responses in the run, with prompt/model allocation Confidence level typically 95% Interval lower and upper bound in percentage points Model list vendor and version identifier for each model queried Run date ISO 8601 date, timezone stated

Source: field list compiled for this article from the method described in how TrustGrowth measures AI visibility. The rationale for still reducing this to one headline figure, carried with its range, is covered in how the TrustGrowth score works, why it changes, and where to see the proof.

State the limit here, not in a footnote: a confidence interval describes sampling uncertainty in the mention rate. It does not certify that the prompt set represents what real users ask a chatbot, and it does not certify that a mention causes any downstream business outcome such as a click, a signup, or a sale.

A worked example: reading an AI visibility score with its confidence interval

No live client data is used in this example; every number below is constructed to illustrate the method, not pulled from a production run. Consider a constructed run with 40 evaluated responses drawn from purchase-intent and comparison prompts across three illustrative model slots, with a mention defined as an exact domain-name string match anywhere in the response text. This is a teaching design, not a description of fields verified on TrustGrowth's current public proof-page surface. In the first run, 14 of 40 responses contained a mention, for an observed rate of 35%. In a second run three months later, 18 of 40 contained a mention, for an observed rate of 45%. Running the wilson_interval function above on both runs, and checking its output against NIST's published Wilson formula (page last modified 2025-09-12 02:15 GMT; retrieved 2026-09-06, UTC), gives 22% to 50% for the first run and 31% to 60% for the second.

Field Run 1 Run 2 Date 2025-03-01 2025-06-01 n (evaluated responses) 40 (14 GPT, 13 Claude, 13 Gemini) 40 (same allocation) Mentions observed 14 18 Observed rate 35% 45% Δ point estimate — +10pp 95% Wilson CI 22% to 50% 31% to 60%

Units: percentage points except where noted. Method: Wilson score interval, z=1.96, computed per the function above and checked against NIST's published formula (page last modified 2025-09-12 02:15 GMT; retrieved 2026-09-06, UTC). Source: illustrative example constructed for this article using the prompt-and-mention design described above, not a live proof-page pull.

The point estimate moved 10pp between these constructed runs, while the two 95% Wilson intervals overlap between 31% and 50%. That overlap is a warning against reading the point estimates alone, but overlap of two separate intervals is not itself a formal test of the difference. To claim a change, predefine and report an interval or statistical test for the between-run difference; this illustrative pair does not do that. The logging convention for comparing dated runs on a live proof page is described in how the TrustGrowth score works, why it changes, and where to see the proof. If a content change shipped between the two dates and the mention rate happened to rise, that correlation does not establish causation: a concurrent model update, seasonal query patterns, or the interval overlap shown above could explain the movement instead.

What this method still can't tell you

Two limitations apply to this entire method, stated here rather than as a closing disclaimer.

Prompt sets sample a much larger query space. A fixed prompt list covers only the prompts and model runs included in the design, and no amount of statistical rigor around the sample fixes an unrepresentative prompt list. A wide, carefully constructed prompt set reduces this risk; it does not eliminate it.

Models update without a uniform public-notice rule across vendors. A score's comparability window is only as long as the vendor's release cadence. OpenAI's deprecations page (retrieved 2026-09-06, UTC) documents notice periods for OpenAI model retirement; public vendor documentation reviewed for this article does not establish a uniform advance-notice commitment for every production model-weight update across every provider. Treat that as a documentation boundary, not proof that no such commitment exists. A single run also captures a snapshot, not a trend, so one data point cannot support a claim about direction over time.

This piece has described the mention rate and its sampling uncertainty only. It has not measured search rankings, referral traffic, or revenue impact from AI visibility mentions, and no claim about those outcomes is made here. For a related failure mode — tools that report a score after failing to reach the site at all — see whether a tool measured your site or just failed to reach it.

Assumptions used in this article

The Wilson interval treats each evaluated response as an independent Bernoulli trial. That is a modeling choice, not a proof: prompts can cluster by topic, and the same model can correlate across nearby calls. The constructed n=40 example does not adjust for clustering, does not treat prompt representativeness as solved, and does not treat model drift between 2025-03-01 and 2025-06-01 as measured. Those limits sit next to the figures rather than in a closing disclaimer.

How TrustGrowth reports an AI visibility score

For any TrustGrowth AI-visibility figure, the publication requirement is to disclose the evaluated-response count, prompt/model allocation, model identifiers actually returned by the providers, run date and timezone, mention rule, exclusions, point estimate, and 95% Wilson interval. At the time of this revision, this article does not claim that the live public proof-page surface already exposes all of those fields; it states the reporting standard without claiming the current product already displays it. The public method page is how TrustGrowth measures AI visibility. A related step-by-step checker is how to check your AI visibility.

FAQ

What is an AI visibility score?

An AI visibility score is typically a mention rate: the percentage of prompts sent to one or more LLMs, like GPT, Claude, or Gemini, whose responses contain a mention or citation of a given brand or domain, calculated from a fixed prompt sample and a stated run date. See how TrustGrowth measures AI visibility.

Why does a margin of error matter for an AI visibility score?

Because the score is a proportion estimated from a sample, not a census. Without an interval, a change in the reported number cannot be interpreted against the sampling uncertainty supported by that denominator; NIST's proportion guidance (page last modified 2025-09-12 02:15 GMT; retrieved 2026-09-06, UTC) provides the formula used in this article.

What is the Wilson score interval and when should it be used?

The Wilson score interval is a method for computing a confidence interval on a binomial proportion that remains usable at small sample sizes and at rates near 0% or 100% — the property NIST documents for it (page last modified 2025-09-12 02:15 GMT; retrieved 2026-09-06, UTC) — unlike the naive normal-approximation interval, which can produce impossible bounds under those same conditions.

How many prompts does a reliable AI visibility score need?

There is no fixed threshold that guarantees reliability, but the interval narrows as n grows: a 95% Wilson interval around a 50% observed rate is roughly 40pp wide at n=20 and roughly 14pp wide at n=200, per the NIST Wilson formulas (page last modified 2025-09-12 02:15 GMT; retrieved 2026-09-06, UTC), so any reported score should state its n alongside the interval it produced.

Can AI visibility scores from GPT, Claude, and Gemini be compared directly?

Only if each model's version identifier and the run date are logged, because model weights and system prompts change between vendor releases, and an unlogged comparison may be measuring two different instruments rather than one changing mention rate. OpenAI's deprecations page (retrieved 2026-09-06, UTC) is the dated source for OpenAI retirement and notice-period language used here.

Key takeaways

• An AI visibility score is a sampled binomial proportion — a mention rate — not a census of user queries.
• A point estimate without a confidence interval cannot show whether a change between two runs is real or sampling noise, the core reason an AI visibility score without error bars behaves like a horoscope.
• The Wilson score interval is an appropriate method for small-sample proportions; a 95% interval around a 50% rate is roughly 40pp wide at n=20 and roughly 14pp wide at n=200 (NIST, retrieved 2026-09-06, UTC).
• Every published figure needs five fields: sample size, confidence level, interval, model list with versions, and run date.
• A confidence interval bounds sampling uncertainty only; it does not validate prompt representativeness, and it does not prove that a mention drives any business outcome.