Back to Instruments
P18Backtest Honesty

Reproducible Verdict Kernel

Same inputs, same verdict, byte for byte — or it is not a verdict.

Anticipated pricing£249/mo — enterprise from £15,000/yrIndicative while pre-launch — final pricing publishes at launch. Nothing is on sale yet.

An event study that cannot be recomputed is an anecdote with error bars. The Reproducible Verdict Kernel is the engine we built to make event-study claims adjudicable: feed it the events and the series, and it returns a verdict that anyone holding the same inputs can regenerate byte for byte.

The kernel exists — 5,037 lines of Rust, 77 test functions, zero dependencies. Zero-dependency is not an aesthetic choice: every library between your data and your conclusion is a version number that can silently change your answer. Here the entire inferential chain is in one auditable codebase, and determinism is a tested property, not a hope.

What it measures

  • Effect estimates with honest uncertainty. Event effects with confidence intervals from the stationary block bootstrap — resampling that respects the serial dependence in financial series instead of assuming it away — and HAC standard errors that survive autocorrelation and heteroskedasticity rather than pretending the residuals are polite.
  • Multiplicity accounting. When you test a family of events or hypotheses, some will look significant by volume alone. The kernel applies Benjamini-Hochberg false-discovery-rate masking across the family, so the verdict sheet distinguishes effects that survive multiplicity from effects that merely got lucky in a crowd.
  • The can-fail proof, built in. The kernel’s keystone self-test plants a known synthetic effect and demands the engine recover it — while simultaneously killing forty placebo effects it must reject. An engine that cannot demonstrate both powers, detection and refusal, has no business issuing verdicts. This test runs before we trust any output, including our own.
  • Byte-reproducible verdicts. Same inputs, same binary, same verdict — down to the byte. A verdict you receive can be recomputed by you, by a reviewer, by a counterparty, and the recomputation either matches exactly or the discrepancy is itself a finding.

The verdicts on our published event families — which effects survived, at what estimated magnitude, under which FDR threshold: NOT YET PUBLISHED. Each will ship with the content-hashed inputs that make its recomputation possible, because a verdict published without its inputs is asking for trust the whole design exists to make unnecessary.

What it does not do

It does not find you an edge. The kernel adjudicates evidence for effects you claim; it does not generate candidates, and it happily — mechanically — returns “no effect survives” on a family you were fond of. That answer is not a malfunction. It is the most common true answer in this business.

Who it is for

Teams whose event-study claims must survive hostile recomputation: quant researchers publishing internally to a risk committee, desks evaluating third-party research, and anyone who has watched a “significant” effect evaporate the moment someone else ran the numbers.

Where this sits

Reproducible Verdict Kernel is one of the instruments in the Backtest Honesty suite. How that suite measures — the per-instrument battery, and the receipts each measurement will carry — is set out in the Backtest Honesty methodology, part of the site-wide measurement methodology.

Also in the Backtest Honesty suite