06 / 110
Backtesting
Working definition
Backtesting is the replay of a trading rule over historical data to see what it would have done, under the assumptions the test declares.
Backtesting replays a trading rule over past market data and records the trades it would have taken, so the rule can be judged before real money is at stake. The replay answers a conditional question. Its result holds under the assumptions it declares, the data it read, the costs it charged and what it treated as known at each moment, and it is only as honest as those assumptions.
The measurement is what a backtest borrows from reality. Two feeds can hold different histories, a backtest can charge a spread that never existed, and its data can be a past corrected after the fact. Each borrowing has its own note on this site.
Why do two data feeds give different backtests? starts from the fact that spot FX has no single tape: every feed is one venue’s filtered, aggregated history. Feed differences cluster at news, rollover and thin sessions, which is where many strategies act, so a verdict that flips between feeds has found its edge inside recording noise. The belief that a broker’s historical data is the real market sits on the refutation index as false.
Why is my live spread wider than my backtest? finds the usual cause in the test itself: a constant spread narrower than any the market quoted, when real spreads widen exactly when most strategies trade. The belief that data from your broker carries your own spreads is on the refutation index as unproven. Why does my backtest use data that did not exist? traces the third borrowing, a dataset assembled in hindsight, and the belief that clean code rules out look-ahead is on the refutation index as misleading.
What a backtest cannot carry is a forecast. It is a statement about one history under stated assumptions. How the rule behaves on data it has not seen is the question walk-forward validation exists to ask, and backtest overfitting is what a replay reports when the rule was fitted to its own history. This entry recommends no software and states no result for any strategy.
Commonly confused with
Neighbouring concepts that get used interchangeably, and the distinction that actually separates them.
- Backtest overfitting
Backtesting is the replay. Overfitting is one way the replay misleads: the rule has been fitted to noise in the particular history it was replayed on, so the result describes that history rather than a persistent effect.
- Walk-forward validation
A single backtest judges a rule on history it may have been chosen on. Walk-forward validation fits on one window and judges on the next, unseen one, rolling the process forward through time.
- Look-ahead bias
Look-ahead is information a simulated decision could not have had at its moment. It can enter through the data as well as the code, which is why the vintage of a backtest's data is one of its assumptions.
How to measure it in your own data
A definition you cannot test is a definition you have to take on trust. This is the shortest honest route from the concept to a number you computed yourself.
- Records you need
The backtest's trades, the data file it ran on with its source stated, and a second history of the same market from another feed.
- What you compute
Re-run the rule on the second feed, re-price every trade at the spread that prevailed at its own moment, and check that each simulated instant used only data that was knowable then.
- What the answer tells you
A verdict that flips between feeds has located its edge inside recording noise. A result that holds only at a constant spread was priced at a spread that never existed. Data revised after the dates it describes gives the simulation a past no trader possessed.
Questions and answers
What is backtesting a trading strategy?
Running the strategy's rules over past market data as if trading then, and reading the trades it would have taken. The result is conditional on everything the test assumed: the data, the costs and what was knowable at each simulated moment.
Does backtesting software give the real result?
Backtesting software gives the result of its inputs. Platform defaults often ask for a single spread number and make a constant spread the assumption, and the history is one feed's record of the market. Checking those inputs is the trader's part of the work.
How do I backtest a trading strategy honestly?
Declare the assumptions before the run, price each trade at the spread of its own moment, use data as it was knowable at each simulated instant, and judge the rule on history it was not chosen on. Each step removes one way the replay flatters the rule.
What is the difference between forward testing and backtesting?
A backtest judges a rule on history, some of which the rule may have been chosen on. Forward testing judges it on data that arrives after the rule was fixed, so the rule cannot have been fitted to it. Walk-forward validation applies the same separation inside the historical record.
Related terms
Derived from the links this entry makes and the entries that link back to it.