The floor that never fired

A guard in my own codebase was set at 0.90 and had never once fired. I took that as evidence the thing it guarded was healthy. It was evidence of nothing, and finding out why produced a better artifact than the guard ever was.

I call the shape a Silent Guard: a check that has never failed, whose silence gets read as validation, when the threshold may simply sit where nothing real could cross it.

What the guard does

Two time series are only comparable where they share a clock. The check measures retention — each series’ share of observations that survive the intersection with another — and refuses to compute a relationship when too few survive.

One detail matters more than it looks. Retention is measured inside the overlapping span, so a series with a shorter history is not punished for starting late. The artifact carries the receipt for why: across the seven CFTC contract books, two begin sixteen years after the other five, and a naive len(common) / len(all) reads 0.191977 on the long legs. Span-restricted, the same comparison reads 1.0. The naive number is not a finding about the data; it is an artefact of the question.

What the measurement showed

Across 378 combinations of 28 FX series:

The tightest pair in the entire population is EURJPY~GBPCAD, and it retains 99.96% of its observations. The artifact’s own note: one clock, one corpus — the residual is genuine per-pair gaps, not grid disagreement.

So the guard at 0.90 sat more than ten percentage points below the worst thing that has ever happened. Its silence was structural.

And the consequence is sharper than “the threshold was loose”. A guard at 0.90 can catch a catastrophic mismatch. It cannot catch a partial one — a half-shared grid at 0.92, a badly gapped leg — which is the failure actually worth catching, because a catastrophic mismatch announces itself in a dozen other ways first.

Where the new floor came from

0.99, and it was placed between two measured populations rather than chosen.

The legitimate side is the table above: minimum 0.999606.

The banned side had to be computed too, from deliberately broken constructions — the other population a floor must sit above:

The third is the near-miss and the one that does the work: it is the least broken thing the guard must still reject, and it retains a third of its observations. A floor has to sit above that and below 0.999606, and 0.99 does, with room for a series whose gap rate is an order of magnitude worse than anything currently on disk.

The artifact also carries the rule for moving it again: lowering the floor requires a new measurement showing a legitimate population beneath it, recorded in the file. In its own words — it is not a tuning knob.

The exclusion, published rather than assumed

Three instruments sit on the same disk and are not governed by this floor: XAGUSD, XAUUSD, XTIUSD. Metals and oil keep different sessions from spot FX, so pooling them into an FX grid measures the session difference and reports it as a grid mismatch.

Excluding them is defensible. Excluding them silently would not be, so the artifact measures what the exclusion is worth: pooled, the population becomes 465 combinations with a minimum of 0.944352 — beneath the floor itself.

That number exists for a specific reason, stated in the file: so a future series landing on disk cannot be absorbed without the count changing. Which is exactly what nearly happened. An earlier version of this calibration recorded n_pairs: 28 and never named them, so a re-derivation read whatever was on disk — and by then the metals had landed. Unpinned, the floor would have quietly begun governing a population it was never derived for.

The cure is the population pin: every one of the 28 series named in the artifact with its bar count and its content hash. Not the pair list, which matched while the retentions moved in the sixth decimal, and not the date span, which was stable across every cutoff. The corpus interior had changed, and only hashing the inputs catches that.

What this does not establish

The result is a calibration, not a registered hypothesis — and that difference is the reason this page reads the way it does. Elsewhere on this site, measurements are pre-registered before the first tick. A floor is not a hypothesis and was never a candidate for pre-registration: it is derived from measurement, and what gets published is the derivation. That is a category difference, not a weaker standard, and the derivation is on disk where a frozen declaration would not be.

The floor governs one corpus on one clock. It says nothing about whether either series is a faithful record of the market, and retention is a statement about shared timestamps — never about economic relationship. Two series can share every observation and be unrelated.

And the honest limit on the guard itself: because it was never approached, it has never caught anything. Raising it to 0.99 makes it capable of catching a partial mismatch. It has not yet done so, and a guard that has not fired is exactly what this page is about.

Check it

The file is downloadable and its hash is published beside it. --verify re-runs the derivation from the pinned inputs and compares; it exits zero today. If your copy hashes differently, or your recomputation disagrees, send it — a correction with a receipt gets published as a correction, with the original left legible beside it.