# Historical Evidence Publication Gate

## Purpose

A backtest artifact is not automatically public evidence just because it exists.

The repository contains evaluation machinery that can support stronger historical analysis, including matched baselines, costs, drawdowns, timing sensitivity, higher-cost sensitivity, and optional walk-forward plans. Some of those safeguards are optional at runtime. This gate makes the stronger disclosure contract explicit and machine-checkable before a historical simulation is promoted as decision-useful evidence.

For structural validation:

```bash
python tools/validate_public_evidence.py path/to/evidence.json
```

For an actual investment-performance publication gate:

```bash
python tools/validate_public_evidence.py \
  --require-publication-eligible \
  path/to/evidence.json
```

The second form returns non-zero unless the artifact is both structurally valid and explicitly eligible for performance publication. This prevents a valid methodology demonstration or synthetic test fixture from being mistaken for publishable investment evidence.

## What passing means

Passing structural validation means the submitted JSON contains the minimum context required for its declared evidence class.

Passing with `--require-publication-eligible` means the historical-simulation artifact also satisfies the current performance-publication gate.

Neither result means:

- the strategy is good;
- the strategy will outperform;
- the assumptions are necessarily correct;
- the evidence has been independently verified;
- the strategy is suitable for a particular investor;
- historical returns predict future returns.

Those questions require separate evidence and judgment.

## Evidence classes

### `methodology_validation`

Used for calculation tests, reproducibility demonstrations, and synthetic examples.

It must explicitly state that it is not performance evidence and cannot be marked decision-useful. It can be structurally valid but will never be performance-publication eligible.

### `historical_simulation`

Used for historical backtests or replays.

A historical simulation may be marked `decision_useful: true` only when the strong gate passes. The gate requires evidence identity, methodology identity, evaluation period, timing/return assumptions, costs, a fair comparison, adverse outcomes, walk-forward evidence, timing sensitivity, higher-cost sensitivity, limitations, and a claim boundary.

A historical simulation may instead be marked `decision_useful: false`, but it must explain why it is not decision-useful.

### `paper_or_shadow_result`

The validator recognizes this as a separate evidence class but intentionally does not yet approve it for publication. A dedicated contemporaneous-decision evidence contract must be implemented first.

### `live_broker_result`

The validator recognizes this as a separate evidence class but intentionally does not yet approve it for publication. Live evidence needs a separate sanitized verification contract and must not inherit simulation requirements as a substitute for live verification.

## Historical simulation requirements

For `historical_simulation` with `decision_useful: true`, the current validator requires:

### Evidence identity

- immutable evidence ID;
- ISO publication date.

### Methodology identity

- strategy version;
- policy version;
- feature version;
- decisioning version;
- source commit.

### Evaluation period

- ISO start date;
- ISO end date;
- an end date that is not earlier than the start date;
- explicit statement of whether the period was predeclared.

A non-predeclared period can still be structurally valid, but the validator emits a selection-bias warning.

### Assumptions

- assumption mode;
- transaction costs enabled;
- costs reported;
- return definition;
- cash-flow treatment;
- dividend treatment.

### Comparison

- named baseline;
- reason the baseline is appropriate;
- confirmation that timing is matched;
- confirmation that costs are matched.

### Results and adverse evidence

- total return;
- maximum drawdown;
- cost drag;
- worst period;
- benchmark-underperformance periods, including an empty list when there were none;
- failed or skipped runs, including an empty list when there were none.

### Walk-forward discipline

- walk-forward plan identifier;
- at least one out-of-sample window;
- explicit embargo trading days;
- explicit holdout status.

### Sensitivity

- minus one trading-day timing result;
- plus one trading-day timing result;
- 2x higher-cost result;
- 3x higher-cost result.

### Interpretation

- at least one material limitation;
- a claim-boundary sentence stating what the evidence does and does not establish.

## Privacy gate

The validator rejects fields whose keys indicate credential or private runtime material, including access/refresh tokens, API keys, account numbers, broker account IDs, customer/recipient emails, Webull usernames, passwords, tax lots, credentials, and secrets.

This is a backstop, not a substitute for the broader publication/security review in `../publication-boundary.md`.

## Test fixtures

A complete artifact may set `test_fixture: true` to prove the contract is satisfiable in tests. Such an artifact can be structurally valid but is always `publication_eligible: false`.

This lets the repository exercise the full gate without creating fake investment-performance evidence.

## Fail-closed principle

When the validator cannot establish a required field, it returns an error instead of guessing or silently downgrading the requirement.

This is deliberate. The public evidence standard should be easier to weaken through an explicit reviewed methodology change than through an accidental missing field.