89 / 97
Survivorship Bias
Working definition
A distortion introduced when a dataset includes only the entities that survived to the present, so that failures vanish from history and everything measurable looks better than it was.
Survivorship bias is history rewritten by attrition. Build a backtest on today’s index constituents and you have implicitly conditioned every historical trade on knowledge unavailable at the time: that these particular companies would still exist. The stocks that were delisted, went bankrupt, or were absorbed in distressed takeovers — precisely the positions that would have hurt — are absent from the universe, and their absence flatters every long strategy tested on it.
The bias generalises well beyond equities. Fund databases skew performant because closed funds stop reporting and some vendors backfill only the track records of funds successful enough to join. In FX and CFD data, symbols are renamed or discontinued, brokers shut down and their feed histories disappear, and vendors retire instruments with awkward histories. Even a strategy universe curated by a researcher — we test on the majors because the exotics were too messy
— imports a mild form of the same conditioning.
The defining property of survivorship bias is that it is invisible in the data you have. Nothing in a survivor-only dataset flags its own incompleteness; the missing rows do not announce themselves. Detection therefore requires comparing against point-in-time records: universe membership as it stood on each historical date, including the members that later died. The audit of your own backtest — where the bias actually enters a retail test, door by door, ending in a written status for each — is does my backtest have survivorship bias.
The defence is point-in-time data discipline — universes, prices, and fundamentals as they were known on the date being simulated — which is the same discipline that defeats look-ahead bias, and which depends on knowing what a dataset actually contains and how it has been altered since capture: the domain of tick data provenance. A backtest built without it does not measure a strategy against history. It measures a strategy against a version of history in which the losers have been quietly buried, which is one of the surest roads into backtest overfitting.
Commonly confused with
Neighbouring concepts that get used interchangeably, and the distinction that actually separates them.
- Look-ahead bias
Both are failures of point-in-time discipline, and they are not the same failure. Look-ahead uses information that existed later; survivorship uses a universe assembled later. One leaks a value, the other leaks a membership list — and the same discipline defeats both.
- Selection bias
Survivorship is the special case where the selection was performed by failure itself rather than by a researcher. That is what makes it insidious: nobody chose to exclude the losers, so nobody remembers doing it.
- Backtest overfitting
Overfitting is fitting to noise that is present in the data. Survivorship is noise that is absent from it. A perfectly disciplined search over a survivor-only universe will still produce an inflated result, because the problem is upstream of the search.
- Censoring
Censoring is an observation truncated — you know it existed and not how it ended. Survivorship is an observation missing entirely, with no row to mark its absence. Censoring leaves a gap you can account for; survivorship leaves no trace at all.
How to measure it in your own data
A definition you cannot test is a definition you have to take on trust. This is the shortest honest route from the concept to a number you computed yourself.
- Records you need
Point-in-time universe membership: which instruments, funds or constituents were in the universe on each historical date, including the ones that later delisted, closed, were absorbed or were retired by the vendor. Today's membership list is precisely the thing that cannot answer this.
- What you compute
Reconstruct the universe as of each simulated date and compare it against the universe your backtest actually traded. Count what is present now but absent then, and — the number that matters — what was present then and is absent now.
- What the answer tells you
Any positive count in that second category establishes that the bias is present. How large it is depends on how those departed members performed, which is a separate computation — a count is evidence of the distortion and not a measurement of it. The defining property is that nothing in a survivor-only dataset flags its own incompleteness: the missing rows do not announce themselves, so an internal consistency check will pass on a dataset that has quietly buried every loser. Only an external point-in-time record can settle it.
If this has already cost you
If a backtest ran on a universe assembled today, whether the losers are missing is a question about the file rather than about the strategy.
- Dataset Forensics Assay“Can I trust the data file my research stands on?”Will not establish: That the data is correct. Structure can prove a file defective; it cannot prove it faithful to the market — only a second independent source can, and the report names that limit.
Intake is not open yet, so none of these can be commissioned today. They are listed here so you know the measurement exists and what it would and would not settle — the launch list hears first.
Questions and answers
Does survivorship bias only affect equity backtests?
No. Fund databases skew performant because closed funds stop reporting and some vendors backfill only the track records successful enough to join. In FX and CFD data, symbols get renamed or discontinued, brokers shut down and take their feed histories with them, and vendors retire instruments with awkward histories. Even a decision to test only the majors because the exotics looked messy is a mild version of the same conditioning.
How can I tell whether my dataset has survivorship bias?
Not from the dataset itself, which is the whole difficulty — the absent rows leave no marker and no consistency check will find them. It requires an external point-in-time record of what the universe contained on each historical date. If no such record is available for your data, the honest position is that the bias is unquantified rather than absent.
Is survivorship bias the same as cherry-picking?
No, and the difference matters when apportioning blame. Cherry-picking is a person choosing the flattering subset. Survivorship is failure performing that selection on its own, years before anyone opened the file. The result looks the same and nobody had to act in bad faith to produce it.
What actually fixes it?
Point-in-time data discipline — universes, prices and fundamentals as they were known on the date being simulated, including the members that later died. That requires knowing what a dataset contains and how it has been altered since capture, which is why provenance and survivorship are the same investigation wearing different names.
Related terms
Derived from the links this entry makes and the entries that link back to it.
In the research
Survivorship Bias comes up in seven research notes on this site, and this entry lists three of them.