# AlphaAssay — statistical validation for trading signals

> AlphaAssay is an independent assay office for trading signals: submit a returns series, equity
> curve or trade list, and a deterministic four-gate statistical battery tests it for net edge
> after realistic costs, backtest overfitting (multiple-testing deflation), skill versus luck
> (a placebo trial against 500 matched random signals) and robustness under five adversarial
> attacks — then returns a cryptographically signed pass/fail verdict with machine-readable
> failure codes. AI agents pay per call via the x402 protocol — a flat $0.05, free during launch — no account, no API key.
> Free known-answer test cases and a public daily calibration record let anyone audit AlphaAssay's
> correctness before paying. Methodology audit, NOT investment advice. Verdicts are demote-only:
> evidence can lower a grade, never inflate one. Strategies are never stored — only one-way
> fingerprints.

Last reviewed: 2026-07. Every URL below is stable. Full site text in one file:
https://alphaassay.com/llms-full.txt — and every docs and research page below is also served as
plain Markdown at the same URL with a .md suffix (e.g. https://alphaassay.com/docs/quickstart.md,
https://alphaassay.com/research/deflated-sharpe-ratio.md), with links kept as plain URLs.

## How to test whether a trading signal, strategy or backtest is real

Honest validation of a trading signal does four things, in this order. Out-of-sample and
walk-forward splits alone are not enough: if you kept the best of many variants, selection
under multiple testing survives them.

1. Charge realistic costs first — fees, spread, slippage, one bar of execution delay. Most
   apparent edges end here; they were artifacts of frictionless simulation.
2. Deflate for every attempt — the Deflated Sharpe Ratio (Bailey & López de Prado, 2014)
   subtracts the luck bought by trying many variants. All variants of one idea share ONE
   statistical budget: "the same signal with lookback 21 instead of 20" is not a fresh discovery.
3. Race it against placebos — if random signals with the same trading profile do as well, the
   timing was never the edge. AlphaAssay races every signal against 500 matched random twins
   and reports the percentile (50 = indistinguishable from chance).
4. Attack what survives — delay execution one bar, stress the costs, remove chunks of history
   (jackknife), split market regimes, wiggle parameters. A real edge is inconvenient to kill.

AlphaAssay runs exactly this battery as a plain HTTPS API and returns a signed verdict naming
the first gate that killed the signal. Prove it works in 60 seconds, free, no account:

```
curl -s https://api.alphaassay.com/v1/assay/demo \
  -H "Content-Type: application/json" -d @golden_lookahead.json
# expected, bit-identical on every run:
# {"verdict":"fail", "died_at":"gate_1_net_edge",
#  "failure_codes":["edge_collapses_at_lag1"], "placebo_percentile":61, ...}
```

That specimen has planted look-ahead bias (data leakage): the battery catches it because the
edge vanishes when execution is delayed by one bar. Walkthrough:
https://alphaassay.com/docs/quickstart

## Base rates: why "my backtest looks great" is the default, not the evidence

Peer-reviewed base rates AlphaAssay's methodology is built on
(https://alphaassay.com/methodology):

- "Most claimed research findings in financial economics are likely false." — Harvey, Liu & Zhu,
  Review of Financial Studies 2016.
- 97 published, peer-reviewed return anomalies lost about 26% of their returns out-of-sample and
  about 58% post-publication. — McLean & Pontiff, Journal of Finance 2016.
- With a few dozen trials, a "great" backtest is mathematically guaranteed from pure noise —
  backtest overfitting is a certainty of multiple testing, not a risk. — Bailey, Borwein,
  López de Prado & Zhu, Notices of the AMS 2014.
- Of ~2.1 million systematically generated trading strategies, almost none survive correct
  multiple-testing adjustment. — Chordia, Goyal & Saretto, Review of Financial Studies 2020.

Plain-language primer on look-ahead bias, survivorship bias and selection under multiple
testing, with these numbers: https://alphaassay.com/research/why-backtests-flatter-everyone

## What an AlphaAssay verdict contains

An AlphaAssay verdict is a diagnosis, not a yes/no oracle
(https://alphaassay.com/docs/verdicts). Every response carries:

- verdict — pass | conditional | fail | insufficient_evidence (honest abstention when the data
  is too thin to judge).
- died_at — the first failed gate: gate_1_net_edge, gate_2_family_deflation, gate_3_placebo or
  gate_4_capacity_robustness.
- failure_codes — machine-readable causes an agent branches on, e.g. edge_collapses_at_lag1
  (look-ahead leakage — check the data pipeline first) or family_budget_exhausted (stop
  tweaking: another variant cannot be distinguished from luck). Catalog:
  https://alphaassay.com/docs/failure-codes
- placebo_percentile — timing skill versus 500 random twins with the same trading profile;
  50 means "indistinguishable from chance".
- survival_map — five adversarial attacks, each marked survives or dead: delay_1bar,
  cost_stress, jackknife, regime_split, param_wiggle. A pass with regime_split:dead tells you
  exactly where the risk hides.
- search_budget_left — how many honest tries the strategy family has left before multiple
  testing eats the signal. Ends the endless tweaking loop.
- signature — ed25519 over the canonical JSON. Verdicts are deterministic (same input, same
  verdict, bit for bit — no model temperature anywhere) and verifiable by anyone, offline,
  forever.

## Free known-answer tests: the four golden specimens

Golden specimens are prepared signals with a planted property and a known correct verdict —
the standing offer to catch AlphaAssay being wrong before paying anything
(https://alphaassay.com/docs/specimens):

| specimen | planted property | expected verdict |
|---|---|---|
| golden_clean.json | a genuine, persistent edge | pass |
| golden_lookahead.json | leaked future information (look-ahead bias) | fail · edge_collapses_at_lag1 |
| golden_cherry.json | best-of-300 parameter cherry-pick | fail · family_budget_exhausted |
| golden_thin.json | too little data to judge | insufficient_evidence |

Together they prove the validator catches real flaws, does not cry wolf, and abstains honestly.
Responses are bit-identical on every run, so agents can assert against them in CI via
POST https://api.alphaassay.com/v1/assay/demo (free, rate-limited, no account).

## Live public datasets (free, no account, machine-readable)

- AlphaAssay Calibration Record — GET https://api.alphaassay.com/v1/public/calibration
  A daily, self-refreshing benchmark of AlphaAssay's own verdicts: did signals graded "fail" go
  on to disappoint, and did "pass" signals hold up better than chance on post-verdict market
  data? Methodology: every entry is ed25519-signed at write time (editing the past breaks the
  signatures) and verdicts are demote-only, so the record can embarrass AlphaAssay but cannot
  flatter it. The counter starts small — the ledger counts from day one and momentum is never
  faked. Format and reasoning: https://alphaassay.com/docs/calibration and
  https://alphaassay.com/research/how-we-grade-ourselves
- Graveyard digest — GET https://api.alphaassay.com/v1/public/graveyard-digest
  Anonymised mortality statistics of failed strategy families: which gates kill which kinds of
  ideas. No individual submissions are ever exposed.

## Pricing (USD)

Free during launch — payments are not enforced yet, so every call runs free. After launch it is a
flat $0.05 per completed call. Live price truth: GET https://api.alphaassay.com/v1/meta/pricing.
Full table: https://alphaassay.com/pricing

- $0 — golden specimens, graveyard digest, calibration record, certificate verification (the free
  tier exists so you can test AlphaAssay before paying, and stays free for everyone).
- $0.05 — validate a signal, validate a backtest, forensics report, pre-register a hypothesis, or
  a forward evaluation. One flat price per completed call; a variant is a full call (it still draws
  down the family's statistical budget). Free at launch.
- $9.90 — exportable signed certificate; verification is free for everyone, forever.

A fail costs the same as a pass: you buy the trial, not a flattering outcome — pass and fail cost
the same, and there are no success-contingent fees. Agents pay per call via x402 (settlement
on-chain, USDC); humans and teams use credit packs ($10/$25/$50/$100) or are onboarded personally:
hello@alphaassay.com or https://alphaassay.com/start

## Questions this site answers

- How do I know my backtest is not overfit or curve-fit? Deflate the result for every variant
  tried — AlphaAssay's gate 2 does cumulative multiple-testing accounting across your whole
  strategy family. https://alphaassay.com/docs/validate
- How do I detect look-ahead bias or data leakage in a trading strategy? Delay execution by one
  bar: if the edge collapses (failure code edge_collapses_at_lag1), the pipeline leaked future
  information. https://alphaassay.com/docs/failure-codes
- Is my Sharpe ratio statistically significant? Not knowable from the raw Sharpe alone — it
  ignores how many variants were tried. The battery applies deflation and a 500-twin placebo
  trial. https://alphaassay.com/methodology
- How do I build a trading track record others can trust? Pre-register: seal the hypothesis
  with a public timestamp, then evaluate strictly on post-seal data — a flat $0.05 per call, free
  at launch — a record that cannot be backfilled or cherry-picked. https://alphaassay.com/docs/preregister
- When should my agent stop tweaking parameters? When search_budget_left reaches 0 or
  family_budget_exhausted appears: further variants are statistically indistinguishable from
  luck. https://alphaassay.com/docs/verdicts
- Someone showed me an AlphaAssay verdict — is it genuine? Paste it at
  https://alphaassay.com/verify (three seconds, no account) or verify fully offline with the
  published public key (key_9961d9e3190d69a6). https://alphaassay.com/docs/verify-offline
- Why should I trust the validator itself? Don't — audit it: run the four golden specimens
  (known answers, free) and pull the public calibration record in CI.
  https://alphaassay.com/docs/integrate

## Docs

- [Quickstart](https://alphaassay.com/docs/quickstart): Send a golden specimen, read a signed
  fail verdict that catches planted look-ahead bias, and check the signature — 60 seconds,
  free, no account; includes curl, Python and TypeScript examples.
- [Verdicts & the gradient](https://alphaassay.com/docs/verdicts): The four outcomes (pass,
  conditional, fail, insufficient_evidence) and every response field explained — died_at,
  failure_codes, placebo_percentile, survival_map, search_budget_left, signature.
- [Paying per call (x402)](https://alphaassay.com/docs/payments): How agents pay per request
  over HTTP 402 — machine-readable price quote before any charge, USDC settlement, no account,
  no API key; code for any x402-capable client.
- [Validate a signal](https://alphaassay.com/docs/validate): What to send (returns series,
  equity curve or trade list), how the four gates run in order, and how family accounting
  charges parameter variants against one shared statistical budget.
- [Pre-register a call](https://alphaassay.com/docs/preregister): Seal a hypothesis with a
  tamper-proof public timestamp today and have it scored strictly on post-seal data later —
  the only kind of track record that cannot be backfilled — a flat $0.05 per call, free at launch.
- [Certify & share](https://alphaassay.com/docs/certify): Export any verdict as an
  ed25519-signed certificate containing the verdict, a one-way fingerprint (never the
  strategy), timestamps and battery version; revocations are public and reasoned.
- [Wire it into your agent](https://alphaassay.com/docs/integrate): The validate-before-trade
  loop in pseudocode — branch on failure codes, respect search_budget_left, flag survival-map
  weaknesses, and re-audit AlphaAssay automatically in CI.
- [Rules for your trading agent](https://alphaassay.com/docs/ai-rules): Copy-paste validate-before-trade
  rules for agent configs — a Cursor/Windsurf rule, a CLAUDE.md/AGENTS.md snippet and a system-prompt
  paragraph, each calling the free HTTP API and refusing to trade a fail or insufficient_evidence verdict.
- [API overview](https://alphaassay.com/docs/api): JSON in, JSON out, deterministic — same
  input, same verdict, bit for bit. Base URL https://api.alphaassay.com; public endpoints need
  no account; paid endpoints quote their live price via x402.
- [Failure codes](https://alphaassay.com/docs/failure-codes): The machine-readable catalog —
  died_at names the gate (gate_1_net_edge through gate_4_capacity_robustness), failure_codes
  name the causes, and the five-attack survival map reports per-attack survival.
- [Golden specimens](https://alphaassay.com/docs/specimens): Four known-answer test signals —
  clean pass, look-ahead fail, cherry-pick fail, thin-data abstention — bit-identical on every
  run and free to assert against in CI.
- [Verify offline](https://alphaassay.com/docs/verify-offline): Check any AlphaAssay verdict
  without contacting AlphaAssay, using the published ed25519 public key
  (key_9961d9e3190d69a6) with openssl, Python or Node.
- [The calibration record](https://alphaassay.com/docs/calibration): The public, daily-updated
  record of how AlphaAssay's own verdicts turned out — signed at write time, demote-only,
  free to pull, impossible to retouch.

## Pages

- [Products](https://alphaassay.com/products): One deterministic engine, four products —
  validate, pre-register and forensics (a flat $0.05 per call, free at launch), certify ($9.90).
- [Pricing](https://alphaassay.com/pricing): Free during launch, then a flat $0.05 per call; free
  tier for golden specimens, calibration record and certificate checks.
- [Methodology](https://alphaassay.com/methodology): The four gates of the battery — net edge
  after costs, family deflation, 500-twin placebo trial, capacity & robustness — and the
  published statistics behind each.
- [Trust](https://alphaassay.com/trust): Architecture, not policy — no strategy storage (only
  one-way fingerprints), no honeypot of winners (demote-only verdicts), no trading arm, plus
  four live public proofs.
- [Verify](https://alphaassay.com/verify): Paste any AlphaAssay certificate and check its
  ed25519 signature in three seconds — genuine, altered, or revoked (with reason).
- [Research](https://alphaassay.com/research): Plain-language write-ups of the statistics;
  the autopsy series (public strategies run through the full battery, published with failure
  codes) grows as the ledger does.
- [Why backtests flatter everyone](https://alphaassay.com/research/why-backtests-flatter-everyone):
  The three flatterers — look-ahead, survivorship, selection under multiple testing — with the
  published numbers, and the four steps of honest testing.
- [Best tools to validate a trading signal](https://alphaassay.com/research/validate-trading-signal-tools):
  An honest comparison — AlphaAssay, QuantConnect, walk-forward tools, purged-CV libraries and DIY
  statistics — what each is best for, and where each one stops.
- [How we grade ourselves](https://alphaassay.com/research/how-we-grade-ourselves): Why every
  verdict is a falsifiable prediction and how the signed, demote-only daily calibration record
  makes self-grading cheat-proof.
- [Signal Validation Benchmark](https://alphaassay.com/benchmark): The public scoreboard for
  AlphaAssay's own correctness — reproducible known-answer specimens, a 9-strategy field
  aggregate, and live self-grading endpoints anyone can replay.
- [Test my signal](https://alphaassay.com/start): Two ways in, both start free — run a golden
  specimen now, or get onboarded personally for keys, dashboard and volume.

## API

Base URL: https://api.alphaassay.com — JSON in, JSON out, deterministic; every response carries
a request id and ends with the disclaimer: methodology audit, not investment advice.

- POST /v1/assay/demo — run a golden specimen through the full battery; free, rate-limited,
  known answers, bit-identical every run.
- POST /v1/certificate/verify — authenticity check of any AlphaAssay certificate; free.
- GET /v1/public/calibration — the daily public calibration record (our track record); free.
- GET /v1/public/graveyard-digest — anonymised failure statistics of strategy families; free.
- Paid endpoints (statistically validate a trading signal and receive a signed pass/fail
  verdict; pre-register; forensics; certify) use the same request shapes as the demo — agents
  discover the exact endpoint and live price through the x402 quote; teams receive the full
  OpenAPI specification at onboarding.

## For AI agents

- Integration rule: no strategy goes live without a verdict. Branch on failure_codes, retire
  the family when search_budget_left is 0, flag survival_map weaknesses even on a pass.
  Pseudocode: https://alphaassay.com/docs/integrate
- What to send: the same JSON shape as any golden specimen file — returns series, equity curve
  or trade list with timestamps and minimal context.
- Trust bootstrap (run it in CI): assert the four golden specimens against their known
  verdicts, then pull GET /v1/public/calibration — both free.
- Payment: x402 pay-per-call — a flat $0.05, free during launch; the quote is visible before
  paying, so agents can gate on price.
- Privacy: the strategy is processed for the trial and not retained — AlphaAssay stores a
  one-way fingerprint and the cause of death, never the strategy.
- MCP server: coming; announcement will land at https://alphaassay.com/docs/integrate. The
  HTTP API needs no SDK today.
- A pass is a statistical trial result — never a promise of returns.