# Combinatorial Purged Cross-Validation (CPCV) > A backtest protocol that partitions a history into groups, holds out every combination of them in turn with purging and an embargo, and so produces many out-of-sample paths instead of a single one. - Canonical: https://hadalinstruments.com/glossary/combinatorial-purged-cross-validation/ - Term set: https://hadalinstruments.com/glossary/ --- A conventional backtest produces exactly one out-of-sample path: this strategy, over this history, in this order. Any statistic computed from it — a [Sharpe ratio](/glossary/sharpe-ratio/), a maximum [drawdown](/glossary/drawdown/) — is a single draw from a distribution nobody has seen, and one draw supports no statement about variance. Combinatorial purged cross-validation, proposed by López de Prado, generates the distribution. The history is partitioned into N contiguous groups; every combination of k groups is held out as a test set in turn, with [purging](/glossary/purged-cross-validation/) and an [embargo](/glossary/embargo-period/) applied at each boundary; the held-out segments are then stitched into whole out-of-sample paths. Choosing N and k fixes how many paths result — a count that grows quickly, which is the method's cost and its point. What a distribution supports differs in kind from what one path supports. The spread of Sharpe ratios across paths says how much of a headline figure is sampling variation. The share of paths in which the configuration selected as best in-sample fails to beat the median of its rivals out-of-sample is the [probability of backtest overfitting](/glossary/probability-of-backtest-overfitting/). Neither is available from a single pass, which is why [walk-forward validation](/glossary/walk-forward-validation/) and CPCV answer different questions rather than competing. The compute cost is genuine: every path is a full fit and evaluation. The subtler cost is that the paths are not independent — they are drawn from one finite history — so the distribution is narrower than it looks, a point [effective sample size](/glossary/effective-sample-size/) makes precise. ## Why it matters A backtest is usually asked *did this work?* The question worth asking is *how often would something like this have worked, and how far apart are the outcomes?* CPCV is the machinery that makes the second question answerable — and, like every multiple-testing correction, it is honest only if the trial count fed into it is the true one. ## Commonly confused with Neighbouring concepts that get used interchangeably, and the distinction that actually separates them. - **[Purged cross-validation](https://hadalinstruments.com/glossary/purged-cross-validation/)** — Purging is the leak repair applied at every fold boundary. CPCV is the protocol that applies it across every combination of held-out groups. The first makes one split honest; the second turns many honest splits into a distribution. - **[Walk-forward validation](https://hadalinstruments.com/glossary/walk-forward-validation/)** — They answer different questions rather than competing. Walk-forward mirrors deployment — always trading on parameters chosen without the future. CPCV abandons that resemblance to extract many out-of-sample paths from the same history, which is what makes variance estimable. - **A conventional backtest** — A conventional backtest produces exactly one out-of-sample path: this strategy, this history, this order. Any statistic from it is a single draw from a distribution nobody has seen, and one draw supports no statement about variance. - **[Probability of backtest overfitting](https://hadalinstruments.com/glossary/probability-of-backtest-overfitting/)** — PBO is one of the things a CPCV distribution can be used to compute — the share of paths where the in-sample winner fails to beat the median out of sample. The protocol produces the paths; PBO is a reading taken from them. ## How to measure it in your own data A definition you cannot test is a definition you have to take on trust. This is the shortest honest route from the concept to a number you computed yourself. - **Records you need** — The full history, the information interval of every observation so purging can be applied at each boundary, and the performance of every configuration if PBO is the goal rather than a single strategy's variance. - **What you compute** — Partition the history into N contiguous groups, hold out every combination of k in turn with purging and an embargo at each boundary, then stitch the held-out segments into whole out-of-sample paths. Choosing N and k fixes how many paths result, and the count grows quickly. - **What the answer tells you** — Read the spread, not the mean. The dispersion of Sharpe ratios across paths says how much of a headline figure is sampling variation — which is the whole reason to run it. Two costs are real: every path is a full fit and evaluation, and the paths are not independent, since they are drawn from one finite history. The distribution is therefore narrower than it looks. ## Questions and answers ### What does CPCV give me that a normal backtest does not? A distribution instead of a single draw. A conventional backtest produces one out-of-sample path, so any Sharpe ratio or maximum drawdown computed from it is one sample from a distribution you have never seen. CPCV generates many paths from the same history, which is what makes it possible to say how much of the headline number is sampling variation. ### How do I choose N and k? Together they fix the number of paths, and the count grows quickly with both. That growth is simultaneously the method's cost — every path is a full fit and evaluation — and its purpose, since a distribution needs enough paths to have a shape. The practical constraint is usually compute. ### Is CPCV better than walk-forward validation? Neither replaces the other. Walk-forward resembles deployment, always trading on parameters chosen without knowledge of the future, which is the property you want when asking whether a strategy could have been run. CPCV gives up some of that resemblance to buy variance estimates from the same history. ### Are the paths independent? No, and this is the subtle cost. Every path is drawn from one finite history, so they share data and the resulting distribution is narrower than its shape suggests. Treating the spread as if it came from independent samples overstates how much you have learned — the same dependence that effective sample size makes precise. ## Related terms Derived from the links this entry makes and the entries that link back to it. - [Drawdown](https://hadalinstruments.com/glossary/drawdown/) The decline of an equity curve from its running peak, measured until a new peak is made; the maximum drawdown is the deepest such decline anywhere in the record. - [Effective Sample Size](https://hadalinstruments.com/glossary/effective-sample-size/) The number of independent observations a dependent sample is actually worth — the count that governs a statistic's standard error once autocorrelation and overlapping windows are accounted for. - [Embargo Period](https://hadalinstruments.com/glossary/embargo-period/) A span of observations discarded immediately after a test window, so that serial correlation cannot carry information from the tested period into the data used to train. - [Probability of Backtest Overfitting (PBO)](https://hadalinstruments.com/glossary/probability-of-backtest-overfitting/) A statistic estimating the probability that the strategy configuration selected as best in-sample will underperform the median of its rivals out-of-sample, typically computed via combinatorially symmetric cross-validation. - [Purged Cross-Validation](https://hadalinstruments.com/glossary/purged-cross-validation/) A cross-validation scheme for time-dependent data in which any training observation whose information horizon overlaps the test window is removed, so that no label can leak across the split. - [Sharpe Ratio](https://hadalinstruments.com/glossary/sharpe-ratio/) The ratio of a return series' mean excess return to its standard deviation — how much return was earned per unit of volatility borne, expressed as a single scalar. - [Walk-Forward Validation](https://hadalinstruments.com/glossary/walk-forward-validation/) An out-of-sample testing protocol in which a strategy is fitted on one window of historical data and evaluated on the subsequent, unseen window, with the process rolled forward through time. ## In the research Combinatorial Purged Cross-Validation (CPCV) comes up in one research note on this site. - [Do order blocks predict anything?](https://hadalinstruments.com/research/do-order-blocks-predict-anything/) Controlled for recent momentum, the order-block measure added nothing out of sample. It was redundant, not underpowered — and the distinction is the method. ## Cite This Definition APA BibTeX HTML Hadal Instruments. (2026). Combinatorial Purged Cross-Validation (CPCV). Hadal Glossary. https://hadalinstruments.com/glossary/combinatorial-purged-cross-validation/ Version a73ac1e, 2026-08-04. @misc{hadal_2026_combinatorial-purged-cross-validation, author = {Hadal Instruments}, title = {Combinatorial Purged Cross-Validation (CPCV)}, year = {2026}, url = {https://hadalinstruments.com/glossary/combinatorial-purged-cross-validation/}, howpublished = {Hadal Glossary}, version = {a73ac1e}, note = {Pre-launch publication; version dated 2026-08-04} } Source: Hadal Instruments, Combinatorial Purged Cross-Validation (CPCV). Original Research Copy Citation **Version a73ac1e** identifies the commit that last changed this page in Hadal's content repository. That repository is not public, so the identifier does not resolve externally — it is published so a citation pins one specific state rather than a moving page. To obtain the exact version cited, use the [press and research route](https://hadalinstruments.com/press/).