AlphaAssayStatistical power: could your backtest even detect a real edge? — with a calculator
Statistical power answers the question that a green equity curve quietly skips: if your edge were exactly as large as you claim, what is the probability your data sample would detect it at all? A test with 12% power misses a real edge 88% of the time — so both of its possible outcomes are close to meaningless: a pass proves little, and a fail proves little. Power is a property of the sample size and the effect size (Cohen's d, the per-period mean return divided by its volatility), computed here as a one-sided one-sample test at α = 0.05, with 80% as the conventional adequacy line (Cohen, 1988). The calculator runs entirely in your browser; nothing is uploaded. Every AlphaAssay verdict now carries this number: an underpowered test is stamped underpowered instead of being allowed to read as an acquittal.
Can your data even show the edge you claim?
Units cancel in Cohen's d, so use any consistent pair — percent per day with percent per day, decimals with decimals. The defaults (d = 0.05 over 3 years of daily data) reproduce the 39% row in the table below — same formula, same defaults.
How much history does a small edge need?
| effect size d (per period) | ≈ annualised Sharpe | observations n | achieved power | n for 80% power |
|---|---|---|---|---|
| 0.05 | 0.8 | 756 (3y daily) | ≈ 39% | 2,473 |
| 0.10 | 1.6 | 756 (3y daily) | ≈ 87% | 619 |
| 0.10 | 1.6 | 34 | ≈ 14% | 619 |
| 0.20 | 3.2 | 252 (1y daily) | ≈ 94% | 155 |
The first row is the one that stings: a daily edge worth an annualised Sharpe of about 0.8 — a real, tradeable edge — gives three years of daily data only a 39% chance of detecting it. Most retail backtests are shorter than that, on smaller edges. The arithmetic does not care how the equity curve looks.
The formula, step by step
Step 1 — the effect size. d = mean return per period / volatility per period
(Cohen's d; multiply by √periods-per-year and you have the annualised Sharpe).
Step 2 — achieved power. For a one-sided one-sample test at level α:
power = Φ( d·√n − z(1−α) ) — the probability the test statistic clears the critical value
when the edge is real.
Step 3 — the sample you actually need.
n₈₀ = ⌈ ( (z(0.80) + z(1−α)) / d )² ⌉ observations for 80% power. Halve the effect size and
the requirement quadruples — power is quadratic in 1/d, which is why small edges are so expensive to
prove.
Why is a pass on thin data not an acquittal?
Because an underpowered test cannot convict, its failure to convict carries almost no information — and treating that silence as evidence is how weak signals get promoted. This is the asymmetry the power honesty check inside every AlphaAssay verdict exists to expose: when the sample cannot support the claim, the verdict says so, in the open, instead of letting „no fail detected" masquerade as „validated". The same honesty runs in the other direction as backtest-length accounting — see MinBTL — and both are one battery with deflation: how big is the edge, how long was the look, how many things were tried.