How to test the things traders argue about, written so the test can be run without me — plus the studies and the incidents that came out of my own house.
Three kinds of thing publish here. Method guides take a question people actually search, answer it directly, and hand over the procedure — including the cases where the honest answer is that the question as posed has no answer. Measurement studies state their sample before they state their finding and name what the measurement does not establish. Post-mortems are my own failures, published as first-party testimony — with the forensic packs behind them marked NOT YET PUBLISHED where they have not been released.
There are no performance claims here and there never will be. Nothing on this page tells you what to trade. Where an article ships with a dataset, the artifact is content-hashed and the hash is printed beside the download, so the finding can be recomputed rather than believed. Broker feed measurements from the Observatory battery: NOT YET PUBLISHED — the methodology pre-registers before broker #1 is measured, and the articles arrive with the artifacts that prove them.
Method guides introduce a named term as the answer's name. A name is an editorial act and carries no measurement, so it needs no artifact; every figure still does. The coined terms are collected, one line each, in the lexicon; the claims each article examines are indexed at the refutation index; definitions for the standing vocabulary live in the glossary.
37 articles published25 method guides12 studies and post-mortems
One searched question per article, answered in the searcher’s own words. Each teaches the procedure a reader can run on their own records — what a rigorous answer requires, what the common answers get wrong, and where the measured version will publish. No figures appear in any of them: an article that graduates to a measurement moves to the studies shelf with its artifact.
Answers“does my hmm regime filter use future data in a backtest”
Usually not through a coding bug. The standard workflow decodes the state with the whole series in hand, so each label was computed using the days after it.
Answers“why did my broker charge me so much in swaps”
Because a swap is two costs wearing one name: an interest-rate differential anyone can compute, and a residual on top of it that almost nobody checks.
Answers“do round numbers matter in forex trading”
A 13-month window gave z = minus 2.59. Run the same test across twelve windows and the mean is plus 0.08, none significant, some pointing the other way.
Answers“does a wick rejection actually mean anything”
An 8% against 46% split at z = 198, and a volatility-matched surrogate reproduces it at +39% against +38%. The effect is real and means nothing.
Answers“does my backtest have survivorship bias”
You cannot tell from inside the dataset. The audit is of your selections — the symbols, data sources and strategies that never reached the record.
Answers“why do my forex orders get rejected”
A rejection is not an error. It is a decision taken inside a window you cannot see — and it makes the fills you did get a sample somebody else selected.
Answers“what does a funded account breach mean”
Not a data breach. In funded trading it means a rule threshold was crossed and the account was ended or suspended — and the two kinds are not the same.
Answers“what can a trading statement prove”
Its own arithmetic and the facts inside the file — the closed ledger — but never the market around your fills, a quote's honesty, or anyone's intent.
Answers“where does a prop firm breach dispute go”
The routes a UK trader can take after a funded account is closed against a rule — and the gate on each one, which is where most of them actually end.
Answers“why was my account breached while in profit”
An evaluation can end on a day that closed above its limit. The floor and the test run on different clocks, and that gap is where the account went.
Answers“how many trades to prove a trading edge”
There is no universal number: the trades required scale with the square of your edge's dispersion-to-size ratio. How to compute your own, from your own log.
Answers“why do correlated pairs decouple on low timeframes”
Below a measurable interval, two instruments barely share prints — most fine-grain decoupling is the measurement dissolving, not the relationship breaking.
Answers“why do two dashboards show different numbers”
Usually both are right by their own hidden definitions — the real defect is that neither number's derivation can be produced on demand. A ten-minute test.
Answers“how do you verify a trading track record”
What a verified quant track record actually requires: the trade list, the accounting rules and the provenance that let a third party recompute it.
Answers“is my drawdown limit calculated on balance or equity”
The breach that surprises traders is computed on equity while they watched balance. How the definitions differ, why trailing limits bite, how to check yours.
Answers“why does my backtest give different results with different data feeds”
In spot FX there is no single tape: every feed is one venue's filtered, aggregated history. Where feeds diverge, why results move, and how to diff yours.
Answers“why does my ea work on demo but not on a live account”
Demo fills are simulated: no spread paid, no slippage, no requotes. What changes at a live venue, and how to measure the gap on your own account.
Answers“how do you tell if a ci gate can actually fail”
You plant the defect it exists to catch and watch it go red. A gate that has only ever passed proves nothing, because passing is also what a broken one does.
Answers“how to test a backtest for overfitting”
You cannot test the backtest. Overfitting is a property of the search that produced it, so the test needs four artifacts most research processes never record.
Answers“is my broker trading against me”
As asked it is unfalsifiable. The answerable version is narrower: does your execution quality change with your own behaviour? Here is how to record that.
Answers“what causes a prop firm drawdown breach”
Rarely a losing streak. A breach happens when a trader watches one termination rule while a different one is closer, and the rules move on separate clocks.
Answers“what is slippage really costing me”
Not what the spread figure says, and not what your average slippage says either. The cost lives in the asymmetry, and your own statements already contain it.
Answers“what stops an ai coding agent from faking its results”
Nothing you say to it. What stops it is arithmetic: making the honest path to done cheaper than the fake one, and making the fake one leave a shape.
Answers“why do backtests fail in live trading”
Because a backtest is a reconstruction, and every reconstruction borrows from reality in five places. Live trading is where the borrowing is called in.
Answers“why does my backtest use data that did not exist yet”
Not a bug in your code. Your dataset was assembled in hindsight, so the simulation reads a past that was corrected, curated and completed after the fact.
First-party work: measurement studies that state their sample before their finding, and incidents in my own house published in full — including the July 2026 case in which an AI agent working on my core engine forged operator sign-offs and fabricated results.
Answers“can a trading edge be statistically significant but not profitable”
My engine's first end-to-end edge hunt found statistically real structure — leans of half to two percent from a coin flip — and none of it is tradable.
Answers“do brokers hunt stop losses by widening the spread”
Up-side spread tags run 67% at round numbers against a modelled floor of 77%. The claim that stops are hit more often than chance does not survive it.
Answers“do order blocks actually predict anything”
Controlled for recent momentum, the order-block measure added nothing out of sample. It was redundant, not underpowered — and the distinction is the method.
Answers“does cot positioning predict reversals”
Crowded positioning extremes precede fewer liquidations, not more. The interval excludes zero, p = 0.0066, and the sign is backwards.
Answers“is my volatility indicator just measuring time of day”
I removed the time-of-day pattern from one FX pair and a standard volatility classifier stopped finding compression at all. Six features, all six moved.
Answers“how do you know a detected regime change is real”
A joint change-point detector flagged one break across twelve market channels, then tested it against two independent bars and cleared neither.
Answers“how do you know a check that never fails is working”
A guard set at 0.90 never fired in years of use. Measuring the population showed why: every real observation sat ten points above it.
Answers“do price levels remember what happened there before”
My first real finding said price levels remember their own history. At sixteen years and 183,689 revisits, the memory was recency wearing a costume.
Answers“is my broker's feed honest”
Honest is a comparison, and retail traders have nothing to compare against. Here is the narrower question that is answerable, and how to record the evidence.
Answers“why is my live spread wider than my backtest”
Usually because the backtest used a spread that never existed. Here is how to find which of the four causes is yours, and what each one costs you.
An agent on my own engine fabricated results and forged sign-offs. What it was like to find out, how I caught it, and what is still not published.
What 1,820 scheduled-release windows across 28 pairs were measured for — and why the range-expansion factors stay unpublished until their artifact ships.