Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
# ADX Ablation + Random-Entry Control Arm — 2026-07-20

> **Cost note (added 2026-09-02).** The figures below are priced at the flat 5bp
> slippage floor. [the per-product restatement](2026-09-01-per-product-slippage-restatement.md) later measured that **no
> asset in keel's universe reaches that floor** — the range is 1.1× to 36.8× — so every
> profit factor here is optimistic by roughly 0.09 at the median. **The verdict is
> unaffected:** the correction only ever moves a number *down*, and every result here was
> already negative. Nothing on this page has been rewritten; records are appended to, not
> revised.

**Status:** in-sample measurement on full cached history. **NOT out-of-sample. NOT a promotion decision.**

Motivated by two findings in KB `source-58` (Katz & McCormick, *The Encyclopedia of Trading Strategies*):
Expand Down
8 changes: 8 additions & 0 deletions docs/experiments/2026-07-20-exit-lookback-ratio.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
# `exit_lookback` ratio test — §79.6's monotone prediction does NOT replicate

> **Cost note (added 2026-09-02).** The figures below are priced at the flat 5bp
> slippage floor. [the per-product restatement](2026-09-01-per-product-slippage-restatement.md) later measured that **no
> asset in keel's universe reaches that floor** — the range is 1.1× to 36.8× — so every
> profit factor here is optimistic by roughly 0.09 at the median. **The verdict is
> unaffected:** the correction only ever moves a number *down*, and every result here was
> already negative. Nothing on this page has been rewritten; records are appended to, not
> revised.

**Date:** 2026-07-20
**Status:** in-sample sensitivity characterisation on cached daily history. **Not a promotion decision.**
**Ledger session:** `exit-lookback-2026-07-20` (5 diagnostic columns + 1 decision row)
Expand Down
8 changes: 8 additions & 0 deletions docs/experiments/2026-08-08-between-family-independence.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
# Between-family independence — the §80.16 harness exists, and it is calibrated

> **Cost note (added 2026-09-02).** The figures below are priced at the flat 5bp
> slippage floor. [the per-product restatement](2026-09-01-per-product-slippage-restatement.md) later measured that **no
> asset in keel's universe reaches that floor** — the range is 1.1× to 36.8× — so every
> profit factor here is optimistic by roughly 0.09 at the median. **The verdict is
> unaffected:** the correction only ever moves a number *down*, and every result here was
> already negative. Nothing on this page has been rewritten; records are appended to, not
> revised.

**Date:** 2026-08-08
**KB basis:** §80.16 (the measurement), §73.5 (why it is non-optional), §74.3/§58.10a (why arm B
is refuted), §80.10/§80.14 (who the real candidate is)
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
# `turtle_breakout` clears `min_trades=100` on hourly bars — and loses on all 19 of them

> **Cost note (added 2026-09-02).** The figures below are priced at the flat 5bp
> slippage floor. [the per-product restatement](2026-09-01-per-product-slippage-restatement.md) later measured that **no
> asset in keel's universe reaches that floor** — the range is 1.1× to 36.8× — so every
> profit factor here is optimistic by roughly 0.09 at the median. **The verdict is
> unaffected:** the correction only ever moves a number *down*, and every result here was
> already negative. Nothing on this page has been rewritten; records are appended to, not
> revised.

**Date:** 2026-08-11
**Issue:** none — nobody asked for this. It fell out of the 2026-08-11 scout run
(`~/keel/proposals/2026-08-11-shortlist.json`), which carried DOGE forward on a daily
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
# Re-tuning `turtle_breakout` to the hourly clock buys 51% and still loses on everything — 0 of 144 parameter sets

> **Cost note (added 2026-09-02).** The figures below are priced at the flat 5bp
> slippage floor. [the per-product restatement](2026-09-01-per-product-slippage-restatement.md) later measured that **no
> asset in keel's universe reaches that floor** — the range is 1.1× to 36.8× — so every
> profit factor here is optimistic by roughly 0.09 at the median. **The verdict is
> unaffected:** the correction only ever moves a number *down*, and every result here was
> already negative. Nothing on this page has been rewritten; records are appended to, not
> revised.

**Date:** 2026-08-11
**Issue:** none. This is the experiment `docs/experiments/2026-08-11-hourly-backtest-turtle-breakout.md`
§9.4 recommended someone file, run on the same day instead of filed. That document identified its
Expand Down
8 changes: 8 additions & 0 deletions docs/experiments/2026-08-12-fee-curve-and-rsi-meanrev.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
# At zero fee `turtle_breakout` makes money and `rsi_meanrev` does not — two failure modes that were one inference away from being pooled

> **Cost note (added 2026-09-02).** The figures below are priced at the flat 5bp
> slippage floor. [the per-product restatement](2026-09-01-per-product-slippage-restatement.md) later measured that **no
> asset in keel's universe reaches that floor** — the range is 1.1× to 36.8× — so every
> profit factor here is optimistic by roughly 0.09 at the median. **The verdict is
> unaffected:** the correction only ever moves a number *down*, and every result here was
> already negative. Nothing on this page has been rewritten; records are appended to, not
> revised.

**Date:** 2026-08-12
**Issue:** none. Two threads meet here. One is
`docs/experiments/2026-08-11-hourly-param-sweep-turtle-breakout.md` §10.3(b) — *the execution-cost
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
# `rsi_meanrev`'s edge is selectivity, not alpha — and the search for it found a simulator defect

> **Cost note (added 2026-09-02).** The figures below are priced at the flat 5bp
> slippage floor. [the per-product restatement](2026-09-01-per-product-slippage-restatement.md) later measured that **no
> asset in keel's universe reaches that floor** — the range is 1.1× to 36.8× — so every
> profit factor here is optimistic by roughly 0.09 at the median. **The verdict is
> unaffected:** the correction only ever moves a number *down*, and every result here was
> already negative. Nothing on this page has been rewritten; records are appended to, not
> revised.

> **⚠️ AMENDED 2026-08-13 — read alongside
> [`2026-08-13-restated-under-a-production-faithful-engine.md`](2026-08-13-restated-under-a-production-faithful-engine.md).**
>
Expand Down
8 changes: 8 additions & 0 deletions docs/experiments/2026-08-12-shipped-defaults-intersection.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
# Three rules, 24 assets, zero free parameters — the viable intersection is empty, and one of the three fails for the opposite reason we recorded

> **Cost note (added 2026-09-02).** The figures below are priced at the flat 5bp
> slippage floor. [the per-product restatement](2026-09-01-per-product-slippage-restatement.md) later measured that **no
> asset in keel's universe reaches that floor** — the range is 1.1× to 36.8× — so every
> profit factor here is optimistic by roughly 0.09 at the median. **The verdict is
> unaffected:** the correction only ever moves a number *down*, and every result here was
> already negative. Nothing on this page has been rewritten; records are appended to, not
> revised.

> **⚠️ AMENDED 2026-08-13 — read alongside
> [`2026-08-13-restated-under-a-production-faithful-engine.md`](2026-08-13-restated-under-a-production-faithful-engine.md).**
>
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
# Two engine defects, both invisible to 2,712 passing tests — and what the conclusions become without them

> **Cost note (added 2026-09-02).** The figures below are priced at the flat 5bp
> slippage floor. [the per-product restatement](2026-09-01-per-product-slippage-restatement.md) later measured that **no
> asset in keel's universe reaches that floor** — the range is 1.1× to 36.8× — so every
> profit factor here is optimistic by roughly 0.09 at the median. **The verdict is
> unaffected:** the correction only ever moves a number *down*, and every result here was
> already negative. Nothing on this page has been rewritten; records are appended to, not
> revised.

**Date:** 2026-08-13
**Amends:** `2026-08-12-shipped-defaults-intersection.md` (#252) and
`2026-08-12-rsi-meanrev-scale-vs-selectivity.md` (#255)
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
# 2026-08-17: The honest-cost simulate re-run, and the DCA dip-bonus ablation

> **Cost note (added 2026-09-02).** The figures below are priced at the flat 5bp
> slippage floor. [the per-product restatement](2026-09-01-per-product-slippage-restatement.md) later measured that **no
> asset in keel's universe reaches that floor** — the range is 1.1× to 36.8× — so every
> profit factor here is optimistic by roughly 0.09 at the median. **The verdict is
> unaffected:** the correction only ever moves a number *down*, and every result here was
> already negative. Nothing on this page has been rewritten; records are appended to, not
> revised.

Two measurements, one date, both enabled by Phase 9: the first `keel simulate` run under
per-product slippage (#334, v0.9.0), and the first-ever measurement of the DCA family in a
fee-explicit harness. Recorded together because they answer the two questions Phase 10's
Expand Down
8 changes: 8 additions & 0 deletions docs/experiments/2026-08-22-trailing-vs-static-exits.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
# Trailing vs static exits: does the ratchet-only exit policy help, at the fee actually paid?

> **Cost note (added 2026-09-02).** The figures below are priced at the flat 5bp
> slippage floor. [the per-product restatement](2026-09-01-per-product-slippage-restatement.md) later measured that **no
> asset in keel's universe reaches that floor** — the range is 1.1× to 36.8× — so every
> profit factor here is optimistic by roughly 0.09 at the median. **The verdict is
> unaffected:** the correction only ever moves a number *down*, and every result here was
> already negative. Nothing on this page has been rewritten; records are appended to, not
> revised.

**Issue #442.** Driver: [`2026-08-22-trailing-vs-static-exits.py`](2026-08-22-trailing-vs-static-exits.py)
(pre-registered in its docstring before the run). Artifact: every number below is a row of
[`2026-08-22-trailing-vs-static-exits.jsonl`](2026-08-22-trailing-vs-static-exits.jsonl) — 240
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
# The fill-model hazard in externally-sourced strategies: what to establish before a foreign number means anything here

> **Cost note (added 2026-09-02).** The figures below are priced at the flat 5bp
> slippage floor. [the per-product restatement](2026-09-01-per-product-slippage-restatement.md) later measured that **no
> asset in keel's universe reaches that floor** — the range is 1.1× to 36.8× — so every
> profit factor here is optimistic by roughly 0.09 at the median. **The verdict is
> unaffected:** the correction only ever moves a number *down*, and every result here was
> already negative. Nothing on this page has been rewritten; records are appended to, not
> revised.

**Date:** 2026-08-27 · **Issue:** #529 · **Origin:**
[`docs/superpowers/specs/2026-08-23-strategy-api-expressiveness-prd.md`](../superpowers/specs/2026-08-23-strategy-api-expressiveness-prd.md)
§4.2 ("a warning worth encoding, not a feature").
Expand Down
8 changes: 8 additions & 0 deletions docs/experiments/2026-08-30-slippage-cap-options.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
# The 50 bp slippage cap: all three of #523's options, measured on one cohort by one method

> **Cost note (added 2026-09-02).** The figures below are priced at the flat 5bp
> slippage floor. [the per-product restatement](2026-09-01-per-product-slippage-restatement.md) later measured that **no
> asset in keel's universe reaches that floor** — the range is 1.1× to 36.8× — so every
> profit factor here is optimistic by roughly 0.09 at the median. **The verdict is
> unaffected:** the correction only ever moves a number *down*, and every result here was
> already negative. Nothing on this page has been rewritten; records are appended to, not
> revised.

**Issue #523.** Driver: [`2026-08-30-slippage-cap-options.py`](2026-08-30-slippage-cap-options.py)
(pre-registered in its docstring before the run). Artifact: every number below is a row of
[`2026-08-30-slippage-cap-options.jsonl`](2026-08-30-slippage-cap-options.jsonl) — 210 cells
Expand Down
8 changes: 8 additions & 0 deletions docs/experiments/2026-09-01-cusum-event-first-measurement.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
# CUSUM event gating, first measurement — feasible, and without a gross edge

> **Cost note (added 2026-09-02).** The figures below are priced at the flat 5bp
> slippage floor. [the per-product restatement](2026-09-01-per-product-slippage-restatement.md) later measured that **no
> asset in keel's universe reaches that floor** — the range is 1.1× to 36.8× — so every
> profit factor here is optimistic by roughly 0.09 at the median. **The verdict is
> unaffected:** the correction only ever moves a number *down*, and every result here was
> already negative. Nothing on this page has been rewritten; records are appended to, not
> revised.

**Date:** 2026-09-01 · **Issue:** [#341](https://github.com/CodeGateSoftware/keel/issues/341) ·
**Rule:** `cusum_event` · **Driver:** `2026-09-01-cusum-event-first-measurement.py` ·
**Ledger row:** `cusum-event-first-measurement-2026-09-01` (168 trials disclosed)
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
# Triple-barrier exits: a real gross improvement, worth exactly nothing after friction

> **Cost note (added 2026-09-02).** The figures below are priced at the flat 5bp
> slippage floor. [the per-product restatement](2026-09-01-per-product-slippage-restatement.md) later measured that **no
> asset in keel's universe reaches that floor** — the range is 1.1× to 36.8× — so every
> profit factor here is optimistic by roughly 0.09 at the median. **The verdict is
> unaffected:** the correction only ever moves a number *down*, and every result here was
> already negative. Nothing on this page has been rewritten; records are appended to, not
> revised.

**Date:** 2026-09-01 · **Issue:** [#342](https://github.com/CodeGateSoftware/keel/issues/342) ·
**Rule:** `triple_barrier` · **Control:** `cusum_event`
([measured the same day](2026-09-01-cusum-event-first-measurement.md)) ·
Expand Down
7 changes: 7 additions & 0 deletions docs/experiments/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,12 @@
# Experiment records — index

> **Cost note (added 2026-09-02).** Every record here dated before 2026-09-01 prices fills
> at the flat 5bp slippage floor. [the per-product restatement](2026-09-01-per-product-slippage-restatement.md)
> measured that **no asset in keel's universe reaches that floor** — 1.1× to 36.8× — so
> those profit factors are optimistic by roughly 0.09 at the median. **No verdict in this
> directory changes:** the correction only ever moves a number *down*, and they were
> already null. Records are appended to, never revised.

This directory is keel's experiment log: **pre-registered, reproducible records** of
measurements, feasibility studies, and engine-defect findings. Every document states its own
scope and status up front (in-sample vs out-of-sample, "not a promotion decision", "amends
Expand Down
80 changes: 80 additions & 0 deletions tests/test_experiment_cost_pointers.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
"""An experiment record priced at the flat slippage floor must say where the correction is.

`2026-09-01-per-product-slippage-restatement.md` measured what that floor costs: the model
reaches it only at a $500M/day anchor and **no asset in keel's universe reaches it**, so every
figure produced under it overstates the profit factor — by a median of 0.090 across 120 cells.

That does not change any document's VERDICT. The correction is conservative-only: real cost is
higher, so a corrected profit factor can only fall, and every one of the 120 deltas was negative
or zero. Re-running thirteen documents to move numbers that were already null would be spending
compute to reach the same conclusion.

What it does change is what a reader may conclude by comparing ACROSS documents. A profit factor
of 0.9 in one and 1.06 in another are not on the same scale if both were priced at a rate neither
asset trades at, and a reader has no way to know that from the page. So each such record carries
one line saying so.

**Records are never rewritten** — `docs/experiments` is an append-only account of what was run,
and editing a measured number would falsify it. A pointer is an addition, not a revision.
"""

from __future__ import annotations

import re
from pathlib import Path

_ROOT = Path(__file__).resolve().parent.parent
_EXPERIMENTS = _ROOT / "docs/experiments"

#: The correction every flat-priced record must point at.
RESTATEMENT = "2026-09-01-per-product-slippage-restatement.md"

#: Records that report a profit factor AND state a slippage assumption are in scope. A document
#: that reports neither has nothing to qualify.
_REPORTS_PF = re.compile(r"profit factor|\bPF\b", re.I)
_STATES_SLIPPAGE = re.compile(r"slippage", re.I)


def flat_priced_records() -> list[Path]:
"""Every record whose figures rest on a slippage assumption, excluding the correction."""
return [
path
for path in sorted(_EXPERIMENTS.glob("*.md"))
if path.name != RESTATEMENT
and _REPORTS_PF.search(text := path.read_text(encoding="utf-8"))
and _STATES_SLIPPAGE.search(text)
]


def test_there_are_records_in_scope() -> None:
"""A guard on the guard: if the discovery ever matches nothing, every assertion below
passes vacuously and this file stops meaning anything."""
assert len(flat_priced_records()) >= 10


def test_every_flat_priced_record_points_at_the_correction() -> None:
"""The pin. A reader comparing profit factors across records has no way to know from the
page that both were priced at a rate neither asset trades at."""
missing = [
path.name
for path in flat_priced_records()
if RESTATEMENT not in path.read_text(encoding="utf-8")
]
assert not missing, (
f"{len(missing)} experiment record(s) report a profit factor under a slippage "
f"assumption without pointing at {RESTATEMENT}:\n " + "\n ".join(missing)
)


def test_the_pointer_says_the_verdict_is_unchanged() -> None:
"""A bare link would read as "this result is wrong". It is not — the correction is
conservative-only and every verdict here survives it. A pointer that let a reader conclude
otherwise would be a worse error than the one it corrects.
"""
for path in flat_priced_records():
text = path.read_text(encoding="utf-8")
window = text[max(0, text.find(RESTATEMENT) - 600) : text.find(RESTATEMENT) + 600]
assert "verdict" in window.lower(), (
f"{path.name}: the pointer does not say the verdict is unaffected, so it reads as "
"a retraction of a result that still stands"
)
Loading