Reading a result

Read the sample before the number. How the output is laid out, and how to recognise an honest null.

Page kind
Reference
Applies to
Every result panel, all 7 suites
Describes
Intended behaviour — nothing has shipped
Instrument version
NOT YET PUBLISHED

Not on this pageThis page covers the order a panel is read in. It does not cover the refusal case in detail, or the limits that apply across the whole catalogue. When an instrument refusesWhat a result does not establish

Most analytics are read top-down: find the number, form a view, move on. A Hadal result is intended to be read in a different order, because the number is the least informative part of it. This page describes that order and the vocabulary the panels use. It describes intended output: three instrument pages now cite a reading from my own research book, and no instrument has yet published a customer result.

Read the sample block first

Every result opens with what it was computed from: symbol, venue, date range, session coverage, observation count, and a hash over the input bytes. Read this before the estimate, every time.

The reason is blunt. A staleness distribution over four hours of one quiet afternoon and the same distribution over three months are different objects wearing the same label. The sample block is what stops you from confusing them, and it is why the block sits above the finding rather than in a footnote.

Then the estimate — usually a distribution

Where an instrument can report a shape, it reports a shape. Percentiles rather than means. Regimes with start and end times rather than “it widened sometimes”. An age distribution rather than an average age. This is not statistical decoration: the interesting behaviour in feed and execution data lives in the tails, and an average is precisely the operation that removes it.

Single figures do appear, and where they do they carry their uncertainty with them. A probability without an interval is an opinion in a numeral costume.

Then what it does not establish

Every panel closes with an explicit statement of what the finding cannot support. It is styled as part of the result because it is part of the result. A measured cadence floor describes the feed; it does not establish that anyone chose to impose it. A high overfitting probability describes a search; it does not establish that a strategy will lose money. See what a result does not establish for the full treatment.

Recognising an honest null

Sometimes the output is that there is nothing to report. That is a result, and it is printed as one. There are three distinct cases and they are not interchangeable:

  • Nothing measurable happened. The battery ran, the sample cleared its bar, and the effect being looked for was not present at a level the estimator can distinguish from noise. This is a real finding and it should be recorded as one — we looked and found nothing is information about your feed.
  • The sample cannot support the question. The battery declined to estimate, because the observations available fall below the completeness bar declared in the method. This is not a finding about the market at all. It is a finding about your data. See when an instrument refuses to answer.
  • The question was outside scope. The instrument does not measure the thing asked of it, and says so rather than returning the nearest available proxy.

Reporting the first as though it were the second inflates confidence. Reporting the second as though it were the first invents evidence of absence. Keeping them apart is most of what honest measurement is.

The four things a panel is not allowed to omit

Whatever the dimension, an intended panel carries the same four parts, in the same order, and a panel missing any of them is defective rather than concise:

  • The sample. What was read, over what window, with what coverage, hashed.
  • The estimate or the refusal. One or the other, never a softened version of the refusal.
  • The uncertainty. An interval, a distribution, or the statement that the quantity is reported as an exact count rather than an estimate.
  • The limits. What this specific finding does not establish, written for this dimension rather than copied across the catalogue.

If you are building a habit for reading these, read those four in that order and stop before forming a view until you have all four.

The vocabulary

Terms in these outputs are used in their strict sense, not their conversational one. Where a word has a looser everyday meaning — provenance, staleness, coordination, regime, censoring — the glossary holds the working definition the instruments actually use, including what each term is not. If a result seems to say something surprising, check the definition before checking the market.

Keeping the result

An output is intended to be archived, not screenshotted. Keep the artifact file: it carries the inputs, the method version and the hash, which together are what allow a number to be re-derived months later by someone who does not trust you. That standard is the whole subject of the methodology, and it applies to your own runs exactly as it applies to anything published here.


Documentation — page 4 of 9, in reading order