# False Discovery Rate (FDR) > The expected proportion of rejected null hypotheses that are in fact true — the share of your discoveries that are false, as distinct from the probability of making any false discovery at all. - Canonical: https://hadalinstruments.com/glossary/false-discovery-rate/ - Term set: https://hadalinstruments.com/glossary/ --- When one hypothesis is tested, a p-value below the threshold means what it says. When a thousand are tested, some will clear the threshold by construction, and the question changes from *is this result significant?* to *what fraction of my significant results are noise?* Two answers exist. Family-wise error control — the Bonferroni correction and its relatives — bounds the probability of making even one false rejection. It is the conservative choice, and across a search over thousands of candidates it is so conservative that almost nothing survives, real effects included. The false discovery rate, introduced by Benjamini and Hochberg, bounds instead the expected *proportion* of rejections that are false. Accepting that a stated share of discoveries will be wrong buys back a great deal of power. The procedure is mechanical: rank the p-values, compare each against a threshold that rises with its rank, and reject everything up to the largest index that clears. The result is often reported as a q-value — the smallest FDR at which a given finding would be called a discovery. Under general dependence, which is the realistic case for related strategies tested on one history, the Benjamini–Yekutieli variant applies a stricter penalty. The arithmetic is unremarkable. The input is not. Every FDR procedure conditions on the total number of tests, and in strategy research that total includes the specifications abandoned quietly, the parameters nudged and re-run, and the ideas tried on the same data months earlier. An FDR computed over the tests one remembers is not a correction; it is [data snooping](/glossary/data-snooping/) with a citation. The honest count comes from a [kill ledger](/glossary/kill-ledger/) kept while the searching happens. ## Why it matters FDR converts a vague worry into a stated budget: this share of these findings is expected to be wrong, and the threshold was fixed before anyone looked. That is a claim a reader can audit — provided the denominator is the true number of trials, which is why the number belongs in the [pre-registration](/glossary/pre-registration/) rather than in the write-up. ## Commonly confused with Neighbouring concepts that get used interchangeably, and the distinction that actually separates them. - **Family-wise error rate** — Family-wise control bounds the probability of making even one false rejection; FDR bounds the expected proportion of rejections that are false. Across a search over thousands of candidates the first is so conservative that almost nothing survives, real effects included. They answer different questions and only one of them suits strategy search. - **The p-value** — A p-value belongs to a single test. FDR is a property of a collection of them. Reporting a p-value from the best of a thousand configurations as though it were the p-value of one hypothesis is the error the whole procedure exists to correct. - **The q-value** — The q-value is how an FDR result is usually reported — the smallest false discovery rate at which a given finding would still be called a discovery. It is the output of the procedure rather than an alternative to it. - **[Data snooping](https://hadalinstruments.com/glossary/data-snooping/)** — Snooping is the failure; FDR is one of the corrections. The correction only works if the count of tests is honest, which means an FDR computed over the tests you happen to remember is not a correction at all — it is snooping with a citation attached. ## How to measure it in your own data A definition you cannot test is a definition you have to take on trust. This is the shortest honest route from the concept to a number you computed yourself. - **Records you need** — Every p-value from the search, and — the input that actually decides the answer — the true total number of tests. In strategy research that total includes the specifications abandoned quietly, the parameters nudged and re-run, and the ideas tried on the same history months earlier. - **What you compute** — Rank the p-values, compare each against a threshold that rises with its rank, and reject everything up to the largest index that clears — the Benjamini-Hochberg procedure. Under general dependence, which is the realistic case for related strategies tested on one history, apply the Benjamini-Yekutieli variant and its stricter penalty. - **What the answer tells you** — The arithmetic is unremarkable and the denominator is everything. FDR converts a vague worry into a stated budget — this share of these findings is expected to be wrong, and the threshold was fixed before anyone looked — which is a claim a reader can audit only if the trial count is true. The honest count comes from a ledger kept while the searching happens, not reconstructed afterwards. ## Questions and answers ### How is FDR different from a Bonferroni correction? Bonferroni and its relatives control the probability of making any false rejection at all, which is the right goal when a single false positive is costly. FDR controls the expected share of your rejections that are false. Accepting that a stated proportion of discoveries will be wrong buys back a great deal of statistical power, which is what makes it workable over a large search. ### What is a q-value? The smallest false discovery rate at which a given finding would still be called a discovery — the usual way an FDR result is reported. Reading it as "the probability this particular result is false" is close enough to be tempting and wrong enough to matter: it is a property of the threshold, not of the individual finding. ### Does FDR control work when my strategies are related? Not with the standard procedure, which assumes a form of independence that related strategies tested on one history do not satisfy. The Benjamini-Yekutieli variant handles general dependence with a stricter penalty. Using the standard version on a correlated family understates the correction. ### Can I compute FDR after the search is finished? The arithmetic, yes. The meaning, only if you know the true number of tests — and that is the number most searches fail to record. Every FDR procedure conditions on the total, so a correction computed over remembered trials produces a number that looks like rigour and is not. ## Related terms Derived from the links this entry makes and the entries that link back to it. - [Data Snooping](https://hadalinstruments.com/glossary/data-snooping/) The reuse of one dataset to evaluate many hypotheses — sequentially, collectively, or unconsciously — until something fits, at which point the fit is reported as if it were the only hypothesis ever tried. - [Kill Ledger](https://hadalinstruments.com/glossary/kill-ledger/) An append-only research record of every strategy configuration that was tried and abandoned, kept so that multiple-testing corrections can be computed from the true number of trials rather than the remembered one. - [Pre-Registration](https://hadalinstruments.com/glossary/pre-registration/) The practice of recording a hypothesis, its test design, and its success criteria — timestamped — before examining the data that will judge it, so that a confirmation cannot have been shaped by the answer. ## Where the term is used Instrument pages whose published copy uses this term. Each page states what it measures and what it does not establish. - [Reproducible Verdict Kernel](https://hadalinstruments.com/instruments/reproducible-verdict-kernel/) Same inputs, same verdict, byte for byte — or it is not a verdict. Backtest Honesty ## In the research False Discovery Rate (FDR) comes up in two research notes on this site. - [Do round numbers matter in forex?](https://hadalinstruments.com/research/do-round-numbers-matter-in-forex/) A 13-month window gave z = minus 2.59. Run the same test across twelve windows and the mean is plus 0.08, none significant, some pointing the other way. - [The edge that was real and worth half a pip](https://hadalinstruments.com/research/the-edge-that-was-real-and-worth-half-a-pip/) My engine's first end-to-end edge hunt found statistically real structure — leans of half to two percent from a coin flip — and none of it is tradable. ## Cite This Definition APA BibTeX HTML Hadal Instruments. (2026). False Discovery Rate (FDR). Hadal Glossary. https://hadalinstruments.com/glossary/false-discovery-rate/ Version 3330c70, 2026-08-04. @misc{hadal_2026_false-discovery-rate, author = {Hadal Instruments}, title = {False Discovery Rate (FDR)}, year = {2026}, url = {https://hadalinstruments.com/glossary/false-discovery-rate/}, howpublished = {Hadal Glossary}, version = {3330c70}, note = {Pre-launch publication; version dated 2026-08-04} } Source: Hadal Instruments, False Discovery Rate (FDR). Original Research Copy Citation **Version 3330c70** identifies the commit that last changed this page in Hadal's content repository. That repository is not public, so the identifier does not resolve externally — it is published so a citation pins one specific state rather than a moving page. To obtain the exact version cited, use the [press and research route](https://hadalinstruments.com/press/).