Skip to content
QEV

QEV · Validation method

Trust is a procedure, not a promise.

These rules govern every examination, so the same findings always produce the same verdict — whoever runs the test.

§01

Principles

  1. 01

    Freeze first, run second

    Rules, data window, cost assumptions, and failure criteria are written down before any code executes. The specification cannot drift toward a hoped-for answer.

  2. 02

    Code calculates, models do not

    Metrics come from versioned deterministic code against recorded snapshots. An AI-generated number is never evidence.

  3. 03

    Every run leaves a record

    Source, snapshot hash, code version, parameters, outputs, warnings, and reviewer identity are retained per run.

  4. 04

    Negative results stay published

    Rejections and failed replications remain visible. Hiding failures would poison everything else.

§02

Data and evidence policy

What the data lane owes every examination.

  • Approved sources only

    Licensed providers under recorded rights. Public reports display only what those rights permit.

  • Point-in-time discipline

    Data as it existed at the time, not restated history. Estimates and reported facts never mix silently.

  • Frozen snapshots

    Each run pins an immutable snapshot with a checksum. Anyone can verify which bytes produced which numbers.

  • Corporate actions named

    Splits, dividends, delistings, and survivorship handling appear in the report itself.

  • Costs recorded, not idealized

    Fee, spread, slippage, and turnover assumptions stated per test, with break-even cost levels.

§03

What QEV tests

Gates run cheapest-first and stop when a claim fails. Passing one gate never repairs failing another.

  1. 01

    Data integrity

    Vendor reconciliation, timestamp and stale-price audits, corporate-action and survivorship handling. The input must represent the real market.

  2. 02

    Implementation integrity

    The code implements the written rules without leakage. Deterministic re-runs match recorded outputs; cash and buy-and-hold baselines reconcile.

  3. 03

    Controls

    Benchmark comparison against buy-and-hold, plus rule-matched randomized negative controls. A claim must clear noise, not just the index.

  4. 04

    Generalization

    Untouched out-of-sample windows and walk-forward tests on data the construction never saw.

  5. 05

    Selection bias

    How many variants were searched? Multiple-testing corrections applied where they matter, variant counts disclosed.

  6. 06

    Stability

    Parameter neighborhoods, subperiods, universe perturbations, regime analysis. One tuned value proves nothing.

  7. 07

    Economic reality

    Fees, spread, slippage, latency, turnover, capacity, and break-even cost levels. Statistical edge must survive trading reality.

  8. 08

    Tail robustness

    Historical and hypothetical stress paths, bootstrap and simulation families where appropriate — with randomized assumptions stated.

  9. 09

    Independent checker

    A reviewer with a falsification mandate probes leakage, alignment, benchmark error, and overstated conclusions before anything ships.

§04

Maker–checker separation

Whoever produces a result never approves it. A separate reviewer receives one task: break the finding. Their findings ship with the report, and a human signs every public verdict.

Chain of custody

  1. 01maker writes frozen specification
  2. 02runner executes on hashed snapshot
  3. 03artifacts and metrics recorded
  4. 04checker attempts falsification
  5. 05findings reconciled
  6. 06human signs the verdict

§05

How verdicts are derived

Verdicts derive from mandatory findings, never assigned by feel. Reading the findings tells the story before the verdict line does.

Replicated
Reproduction matches within stated tolerance · controls cleared · costs disclosed · out-of-sample holds within scope.
Partially replicated
The exact sub-claims that held and the ones that failed, each with where it was measured.
Could not replicate
The broken step named: reproduction gap, leakage finding, failed control, or cost reality.
Insufficient evidence
The missing input identified, so a future resubmission knows exactly what to add.

One more rule: statistical significance cannot repair corrupted data, and simulation cannot repair leakage. See the sample report for how findings read in practice.