# Why do backtests fail in live trading? > Because a backtest is a reconstruction, and every reconstruction borrows from reality in five places. Live trading is where the borrowing is called in. - Canonical: https://hadalinstruments.com/research/why-do-backtests-fail-in-live-trading/ - Published: 2026-08-01 - Author: Hadal Research - Answers the question: "why do backtests fail in live trading" - Coins the term: **Reconstruction Debt** — The accumulated borrowings a backtest takes against reality — each simplification reasonable alone — which live trading calls in all at once, in whichever place the model borrowed most. --- ## The short answer A backtest is not a rehearsal of live trading. It is a **reconstruction** of a market that no longer exists, assembled from a record that is incomplete by construction, and every reconstruction takes shortcuts in five specific places: selection, data vintage, fill assumptions, cost model, and capacity. Each shortcut is a borrowing against reality. Individually they are reasonable — you cannot simulate a market without simplifying it. Collectively they accumulate into what I call **reconstruction debt**, and live trading is where the debt is called in, all at once, in whichever place you borrowed most. The productive question is therefore not "why did it fail?" but "which of the five did I borrow from, and how much?" All five are auditable against records you already hold, and four of them are cheaper to test than the one everybody reaches for first. **WHAT THIS IS — AND WHAT IS NOT PUBLISHED.** This article is method. **I have published no study of backtest-to-live degradation** — no attribution percentages, no survival rates, no comparison of causes — and no such figure appears here or should be inferred from the fact that I describe the categories. The instruments named at the end are pre-launch and their outputs are `NOT YET PUBLISHED`. ## The Observable Mechanism The mechanism is substitution. At every point where the historical record could not answer a question the simulation needed answered, something was substituted: an assumed fill where no fill existed, a price series revised after the fact standing in for the one available at the time, a universe of instruments that survived standing in for the one that was tradeable then, a constant cost standing in for a distribution. The simulation runs on the substitutions. Live trading runs on the mechanism. The difference between them is the entire failure. ## The five borrowings ### 1. Selection — you kept the winner The first and largest borrowing is that the strategy you are trading was chosen from among alternatives, and the choosing used up the evidence. Under a null of zero skill, the best of many trials still looks good, because you selected on the noise. This is [backtest overfitting](/glossary/backtest-overfitting/), and it is the only one of the five that is a property of your *process* rather than your data. It is also the only one with a well-developed statistical literature and a clean test procedure, which I treat separately in [how to test a backtest for overfitting](/research/how-to-test-a-backtest-for-overfitting/). The short version: the equity curve carries no information about how many alternatives were discarded, so the test has to be applied to the search, and the search has to have been recorded. **Cheapest diagnostic:** count your trials honestly, including the ones you deleted. If the honest count is much larger than the one you would have quoted, stop reading and go there first. ### 2. Data vintage — the history was revised Your backtest ran on a dataset as it exists *today*. The strategy would have traded on the dataset as it existed *then*. Those are different objects, and the difference is not noise — it is systematically favourable, because revisions correct errors, restatements refine estimates, and both arrive after the fact. Three named cases: - **[Look-ahead bias](/glossary/look-ahead-bias/):** any input available in the simulation before it was available in reality. Publication lags on economic series are the classic; so is a corporate action applied at the announcement date rather than the effective date. - **[Survivorship bias](/glossary/survivorship-bias/):** a universe assembled from instruments that still exist. The delisted, the merged and the collapsed are absent, and they are absent for a reason correlated with returns. - **Silent joins.** A merge that carries a value forward to fill a gap manufactures a price that was never quoted. A [carry-forward artifact](/glossary/carry-forward-artifact/) looks like a flat market to every statistic you run over it. **Cheapest diagnostic:** ask your data vendor whether the series is [point-in-time](/glossary/point-in-time-data/), and whether you can retrieve the file as of the date you are simulating. If the answer is no, the vintage borrowing is not bounded — you cannot say how large it is, only that it exists. ### 3. Fill assumptions — the simulator was generous A backtest fills your order because you told it to. The market fills your order if someone was there. The generosity hides in defaults. An order filled at the mid when you would have crossed the spread; a limit filled because the price *touched* the level, with no model of queue position; a stop filled at the stop level in a market that gapped past it; a market order filled at the last printed price rather than at the top of book you would actually have taken. The gap cases matter most, because they are exactly the moments a risk plan depends on. A simulator that fills your stop at your stop will call a day survivable that emptied the account. When the book is empty there is no level, only the next print — sometimes a long way away. **Cheapest diagnostic:** rerun the backtest with the fill model deliberately pessimistic — cross the spread every time, fill stops at the next print rather than the level, assume zero queue priority. If the result inverts, the strategy was never a strategy; it was a description of your fill model. ### 4. Cost model — a constant stood in for a distribution Almost every backtest models [execution cost](/glossary/execution-cost/) as a number. Real cost is a distribution with structure: it varies by session, by [spread regime](/glossary/spread-regime/), by event proximity, and by order size relative to available depth. The error is not random, and this is the part that surprises people. A strategy trades when its conditions occur, and its conditions are frequently the same conditions under which cost is elevated — volatility, news, thin hours, the open. So the strategy's *realised* cost is drawn disproportionately from the expensive tail while the backtest charged it the average. The shortfall then scales with turnover, which means the strategies most damaged are the ones whose backtests looked most reliable, because high trade counts produce smooth curves. **Cheapest diagnostic:** you probably already hold the evidence. Take your own broker statements, measure signed slippage per fill against the quote you actually received, and compare the distribution to the constant you assumed. That procedure is the subject of [what slippage is really costing you](/research/what-is-slippage-really-costing-me/). ### 5. Capacity — the size was never in the book The final borrowing is that the backtest traded a size the market never had to absorb. Historical volume tells you what traded; it does not tell you what would have traded had you also been there. At sufficient size the strategy's own orders move the price it is trying to capture, and the simulation has no mechanism for that because your orders were never in the book. **Cheapest diagnostic:** rerun at a fraction of intended size and at a multiple. If the result scales linearly, your simulator has no market-impact model at all — which is information about the simulator, not evidence about capacity. ## Reconstruction debt Naming it as debt is not a metaphor for its own sake. It does three things a list of biases does not. **It makes the borrowings additive.** A strategy can be clean on selection and still fail, because the debt was taken out in the cost model. Auditing one category and declaring the strategy sound is the most common form of the mistake. **It makes the size estimable.** Every borrowing has a pessimistic bound you can compute: charge the worst plausible fill, the worst plausible cost, the smallest plausible universe. A strategy that survives the pessimistic reconstruction has small debt. A strategy that only survives the optimistic one has told you nothing about markets and a great deal about your assumptions. **It puts regime change last.** "The regime changed" is the explanation that needs no evidence and blocks every other diagnosis. It is sometimes true. It should be the residual after the other five have been bounded — never the first reach, because it is unfalsifiable from where you are standing and the other five are not. The practical protocol is one page long: bound each borrowing pessimistically, rerun, and see what survives. What survives is your result. What only appears under favourable assumptions was always a statement about the assumptions. ## What This Does Not Establish (The Limits) This article is a taxonomy and a set of procedures, not a finding. I have measured nothing about how often each cause dominates, and this page deliberately assigns no weights, percentages or rankings to the five — anyone who does so without an artifact is guessing in public. The list is not exhaustive: strategies also fail through broker-side changes, corporate events, funding and borrow costs, tax and regulatory effects, and plain implementation bugs between the research code and the live code, which is its own large category. Bounding every borrowing pessimistically does not make a strategy profitable and does not predict live results; it establishes what the historical record can support, which is a smaller and more defensible claim. Nothing here is trading advice, no outcome is promised, and a backtest that survives every audit on this page can still lose money for reasons no reconstruction could have contained. ## Where the measured version publishes None of the five diagnostics requires a vendor. Each is a rerun with a harsher assumption, and the harshest honest assumption you can state is worth more than any tool applied to a generous one. Where instrumentation helps is in the substrate and the fills. [Data Forensics](/instruments/data-forensics/) is being built to audit the dataset rather than the strategy — carry-forward artifacts, gap structure, per-side staleness, bar provenance — on the principle that immaculate statistics over a lying substrate produce a confident wrong answer. The [Execution Cost Auditor](/instruments/execution-cost-auditor/) is intended to replace the assumed cost constant with your account's own measured distribution, and the [Stress Harness](/instruments/stress-harness/) to replay recorded shocks against your current book with gap-through honesty rather than obliging fills. All are pre-launch; all have outputs marked `NOT YET PUBLISHED`. **If you suspect the substrate rather than the strategy:** read what Data Forensics measures, and the block stating what it does not establish. It is pre-launch and nothing is for sale. --- ## Claims examined ### Claim 01 — canonical: https://hadalinstruments.com/refutations/#claim-6c1e01d7 > "The strategy stopped working because the market regime changed." — our reading: Unproven Regime change is the explanation that requires no evidence and forecloses every other diagnosis. It is sometimes correct and it is always available, which is what makes it dangerous. The four cheaper hypotheses — selection, data vintage, fill assumptions and cost model — are all testable from records you already hold, and each should be eliminated before a claim about the market is accepted. **What is true:** Before the market gets the blame, the reconstruction debt has to be paid down: execution, costs, data and timing differences between the simulation and the venue are measurable one by one, and a regime claim is testable only after the mundane explanations are quantified and found insufficient. Evidence: https://hadalinstruments.com/research/why-do-backtests-fail-in-live-trading/#claim-6c1e01d7 ### Claim 02 — canonical: https://hadalinstruments.com/refutations/#claim-10d04424 > "I added a generous fixed cost per trade, so execution is accounted for." — our reading: Misleading A constant deduction models the mean and erases the structure. Real cost concentrates: by session, by regime, by event window, and disproportionately in the conditions that trigger the strategy. A flat charge that matches the average will understate cost exactly where the strategy trades most, and the error grows with turnover rather than staying fixed. **What is true:** Execution cost is state-dependent — widest and most hostile exactly when many systems act — so a flat haircut misprices the trades that decide the result; honest accounting prices each simulated fill under the conditions that trade would actually have met. Evidence: https://hadalinstruments.com/research/why-do-backtests-fail-in-live-trading/#claim-10d04424 ## Cite This Article APA BibTeX HTML Hadal Research. (2026). Why do backtests fail in live trading?. Hadal Research. https://hadalinstruments.com/research/why-do-backtests-fail-in-live-trading/ Version 861aa78, 2026-09-14. @misc{hadal_2026_why-do-backtests-fail-in-live-trading, author = {Hadal Research}, title = {Why do backtests fail in live trading?}, year = {2026}, url = {https://hadalinstruments.com/research/why-do-backtests-fail-in-live-trading/}, howpublished = {Hadal Research}, version = {861aa78}, note = {Published: 2026-08-01; version dated 2026-09-14} } Source: Hadal Research, Why do backtests fail in live trading?. Original Research Copy Citation **Version 861aa78** identifies the commit that last changed this page in Hadal's content repository. That repository is not public, so the identifier does not resolve externally — it is published so a citation pins one specific state rather than a moving page. To obtain the exact version cited, use the [press and research route](https://hadalinstruments.com/press/). ## Explore further ### Instruments - [Data Forensics](https://hadalinstruments.com/instruments/data-forensics/) - [Execution Cost Auditor](https://hadalinstruments.com/instruments/execution-cost-auditor/) - [Stress Harness](https://hadalinstruments.com/instruments/stress-harness/) ### Concepts - [Backtest Overfitting](https://hadalinstruments.com/glossary/backtest-overfitting/) - [Execution Cost](https://hadalinstruments.com/glossary/execution-cost/) - [Look-Ahead Bias](https://hadalinstruments.com/glossary/look-ahead-bias/) - [Point-in-Time Data](https://hadalinstruments.com/glossary/point-in-time-data/) - [Survivorship Bias](https://hadalinstruments.com/glossary/survivorship-bias/) ### Research - [Why did my broker charge me so much in swaps?](https://hadalinstruments.com/research/why-did-my-broker-charge-me-so-much-in-swaps/) Asked as: why did my broker charge me so much in swaps - [Why does my backtest use data that did not exist?](https://hadalinstruments.com/research/why-does-my-backtest-use-data-that-did-not-exist/) Asked as: why does my backtest use data that did not exist yet - [What is slippage really costing me?](https://hadalinstruments.com/research/what-is-slippage-really-costing-me/) Asked as: what is slippage really costing me [All Hadal research](https://hadalinstruments.com/research/)[This article as plain markdown](https://hadalinstruments.com/research/why-do-backtests-fail-in-live-trading.md) --- ## Raw artifact — NOT PUBLISHED FOR THIS PAGE No downloadable artifact ships with this page. Eight published measurements do, each content-hashed so a reader can verify the figures independently. Where a measurement is published here without one, that is a gap rather than a policy, and it is stated rather than left to be noticed. [Measurements that ship their data](https://hadalinstruments.com/research/)