# Public Evidence Standard

## Purpose

Performance evidence should help a skeptical reader understand what happened, under what assumptions, and how much uncertainty remains. It should not be optimized to produce the most impressive number.

This standard applies to any future public statement about historical returns, benchmark comparison, drawdown, risk, or strategy effectiveness.

## Evidence classes

Every published result must be labeled as exactly one of the following:

### 1. Live broker result

An outcome actually experienced in a real brokerage account under the production system.

A live result must identify the measurement period and methodology without publishing private account information.

### 2. Paper or shadow result

A contemporaneous decision produced without executing the corresponding trades in a live account.

This must not be described as money earned, customer return, or realized performance.

### 3. Historical simulation / backtest

A replay of a strategy against historical information using declared timing, cost, and data-availability assumptions.

A backtest must never be presented as a track record.

## Required context for a published performance result

A result is not publication-ready unless the accompanying artifact states:

- evidence class;
- strategy/methodology version;
- start and end date;
- decision and execution timing convention;
- initial portfolio or benchmark definition;
- treatment of deposits and withdrawals when applicable;
- dividend/total-return treatment;
- fee, commission, and slippage assumptions;
- benchmark definition;
- total return and, where meaningful, annualized return;
- volatility and maximum drawdown;
- turnover and estimated trading-cost drag;
- material known data limitations;
- whether the period or strategy was selected after results were observed.

If any required item cannot be established, the result should be withheld or clearly marked incomplete.

## Baseline fairness

The primary strategy and comparison portfolios must use the same:

- decision calendar;
- execution timing convention;
- market-data availability rules;
- return window;
- transaction-cost model where trading occurs;
- cash-flow assumptions;
- starting date.

The repository already defines comparison implementations for a static 60/40 allocation and a diversified static portfolio, plus a simple trend baseline. A public comparison should not quietly substitute an easier benchmark after seeing outcomes.

For owner-level evidence, a frozen-start portfolio and a risk-appropriate passive benchmark should be preferred when those comparisons can be calculated defensibly.

## Data leakage and historical knowledge

Historical evaluation must exclude information unavailable at the historical decision timestamp.

The current pipeline explicitly slices historical observations by `asof_ts <= decision_ts`. Rows without a usable availability timestamp are excluded rather than assumed to have been known.

Public backtests should describe this rule in plain language and retain enough version information to reproduce the data boundary.

## Costs and stress

Published backtests must include trading costs rather than showing frictionless results only.

The current policy models commissions and bucket-level slippage and increases assumed costs during volatility stress. The canonical protocol also calls for higher-cost sensitivity tests such as 2x and 3x stressed slippage.

A result that changes materially under reasonable cost assumptions should say so.

## Unfavorable evidence

Credibility requires retaining bad outcomes.

A public evidence ledger should preserve:

- negative periods;
- maximum drawdowns;
- periods of underperformance versus a benchmark;
- failed or skipped runs caused by integrity controls;
- methodology versions later abandoned;
- material changes to assumptions and the reason for each change.

Do not delete or hide an earlier evidence release because a later version performs poorly against it.

## Walk-forward discipline

A historical result should be considered decision-useful only when evaluation follows a declared walk-forward discipline.

At minimum:

- declare development and evaluation windows;
- prevent future label information from leaking into training or parameter selection;
- do not tune parameters inside the final holdout;
- record how many strategy variants were explored;
- keep evaluation rules stable across comparisons.

If a result comes from exploratory research that does not meet this standard, label it exploratory.

## Uncertainty

Point estimates are not enough. Where technically appropriate, publish uncertainty or sensitivity information alongside performance, including some combination of:

- bootstrap confidence intervals;
- timing-shift sensitivity;
- higher-cost sensitivity;
- rolling-window performance;
- scenario/stress-window performance;
- number of tested variants.

The objective is not to create a statistical badge. It is to show how fragile or robust a conclusion is.

## Claims language

Prefer:

- `The strategy returned X during this backtest period under these assumptions.`
- `The live portfolio outperformed/underperformed the declared benchmark by Y percentage points during this period.`
- `The evidence is currently too short or noisy to support a conclusion.`

Avoid:

- `The AI made X dollars.`
- `The strategy beats the market.`
- `This proves the model works.`
- annualizing a very short favorable period without equally prominent context;
- showing gross results when relevant costs materially change the comparison.

## Publication gate

Before a performance artifact becomes public, verify:

1. calculations reproduce from retained inputs or redistribution-safe equivalents;
2. evidence class is unambiguous;
3. unfavorable outcomes for the same declared period are included;
4. benchmark and methodology versions are frozen and named;
5. no customer, broker, credential, or private account data are present;
6. the public wording does not make a stronger claim than the evidence supports.