Evidence standards — what is measured, what is modelled, what stays unknown.

This page is written for the person who has to defend a number they took from a brief. It states how each class of figure is treated, what confidence it carries, and the cases where a verdict should not be trusted.

Measured: Observed directly during the analysis, with a timestamp: what your site publishes, what is winning in the market today, and the demand and cost figures returned for the market in question.
Modelled: Computed from measured inputs — gradings, coverage, exposure and economics. Labelled as modelled at the point of use, never presented as an observation.
Unknown: A value that could not be measured renders as UNKNOWN and stays UNKNOWN. Nothing is interpolated, back-filled from a similar market, or replaced with a category average.
Never a zero: An unmeasured figure is never shown as zero. A zero in a brief means a measured zero.

Confidence

Every brief carries a coverage statement: which checks completed, which did not, and what was missing. A conclusion drawn from partial evidence says so, and a section with nothing behind it states its absence rather than quietly disappearing.

Demand and cost figures are estimates with wide error bars in low-volume, non-English and long-tail markets. A verdict that turns on a small difference is the least reliable kind we produce, and the brief will say when that is the case.

Bounded evidence

A brief is deliberately finite: a fixed number of rivals, a fixed number of entry rows, a capped set of examples for anything killed. The full counts are always preserved — you are told how many were judged and how many were killed — but you are never handed an unsorted mass of rows to sort through.

The arithmetic behind the economics is inverted rather than predicted: you are told the conversion rate required to break even at your price, not a forecast of what you will earn. Where a benchmark was not measured, none is implied.

Known limitations

Search results are personalised and volatile. An analysis is a snapshot from one location at one moment; re-running it later can change the picture without anything in your market changing.
Sites behind aggressive bot protection, heavy client-side rendering or login walls are read incompletely, and the brief will understate them.
Classification and pattern judgements are model judgements. They are right often enough to be useful and wrong often enough that you should read the evidence rows, not only the verdict word.
Demand that never appears in search — offline brand, sales-team influence, partner channels — is not measured. A business can be healthy in a market graded weak.
No brief predicts revenue. It can say a channel is arithmetically unviable at a price point. It cannot say what you will earn if you build.

Corrections

If a figure in your brief is wrong, write to hello@searchdiligence.com with the section and the value. Where an error is confirmed we correct the standard, re-run affected reports and say what changed.