docs(experiments): cusum_event measured — feasible, and without a gross edge (#341)

docs/experiments/2026-09-01-cusum-event-first-measurement.md

@@ -0,0 +1,120 @@
+# CUSUM event gating, first measurement — feasible, and without a gross edge
+
+**Date:** 2026-09-01 · **Issue:** [#341](https://github.com/CodeGateSoftware/keel/issues/341) ·
+**Rule:** `cusum_event` · **Driver:** `2026-09-01-cusum-event-first-measurement.py` ·
+**Ledger row:** `cusum-event-first-measurement-2026-09-01` (168 trials disclosed)
+
+## Declared before the run
+
+* **Primary metric: `n_trades`.** An event filter trades less by construction, and the ρ=−0.77
+  bind between edge and sample size means a rule that fires rarely cannot be admitted whatever
+  its profit factor. Declared primary exactly as the `rsi_meanrev` diagnostic grid declared it.
+* **Secondary: profit factor at fee 0 / 0.006 / 0.012.** Zero bounds from above everything an
+  execution fix could ever buy; 0.006 is the maker rate this account cannot reach; 0.012 is what
+  it pays.
+* **Arm A is ONE configuration** — the shipped default. One config means no argmax, so nothing
+  in arm A is a maximum-of-N.
+* **Arm B sweeps the rule's own headline knob**, `threshold_friction_mult` ∈ {1,2,3,4} at the
+  taker rate. Disclosed as a sweep; its per-asset best is a maximum of four draws.
+
+24 assets, `ONE_HOUR`, ~5 years of cached candles — the same universe as the restated
+intersection, so this sits beside the null it is compared against rather than beside a different
+population.
+
+## 1. Feasibility: YES, and it contradicts the worry filed with the issue
+
+#341 was filed with the concern that *"gating cuts n on rules already below the admission
+floor"*. **It does not.**
+
+| | |
+| :-- | --: |
+| assets with n ≥ 100 (the admission floor) | **21 of 24** |
+| median n | **553** |
+| range | 32 (PAXG-USD) – 884 (FET-USD) |
+
+A 5% cumulative move in an hourly crypto series is a common event, not a rare one. The
+feasibility objection to this rule is answered and should not be repeated. The three assets below
+the floor are PAXG-USD (32), PAXG-USDT (51) and TON-USD (69) — gold and the thinnest name in the
+universe, which is the expected shape.
+
+## 2. Profitability: no, and not marginally
+
+Arm A, all 24 assets:
+
+| fee | PF median | PF max | above 1.0 |
+| :-- | --: | --: | --: |
+| 0% | 0.925 | 1.238 | 8 / 24 |
+| 0.6% (maker, unreachable) | 0.553 | 0.750 | **0 / 24** |
+| 1.2% (taker, actual) | 0.343 | 0.479 | **0 / 24** |
+
+Best cell at the rate actually paid: AVAX-USD, PF 0.479, n=601.
+
+**The intersection of n ≥ 100 and PF > 1.0 is empty across all 168 trials, in both arms.**
+
+## 3. The diagnostic: this is the `rsi_meanrev` disease, not the `turtle_breakout` one
+
+The number that matters is the zero-fee column. **At zero cost only 8 of 24 clear 1.0, and the
+median is 0.925.**
+
+`turtle_breakout` at zero fee was profitable on 4 of 4 with a maximum of 2.713 — a real gross
+edge that cost destroyed, which is why cheaper execution was a coherent thing to want. This rule
+has **essentially no gross edge for cost to destroy**: it loses money at a median before a single
+fee is charged.
+
+Same symptom as turtle at 1.2%, different disease — and the one that cannot be fixed by
+execution, because zero fee bounds from above everything an execution improvement could ever buy
+and zero fee is already a median loss.
+
+## 4. The knob works exactly as designed, and converges to nothing
+
+Arm B, `threshold_friction_mult` at the taker rate:
+
+| mult | threshold | median n | PF median | PF max | above 1.0 | n ≥ 100 |
+| --: | --: | --: | --: | --: | --: | --: |
+| 1 | 2.5% | 1252 | 0.226 | 0.388 | 0 | 23 |
+| 2 | 5.0% | 553 | 0.343 | 0.479 | 0 | 21 |
+| 3 | 7.5% | 330 | 0.361 | 0.533 | 0 | 21 |
+| 4 | 10.0% | 224 | 0.379 | 0.593 | 0 | 20 |
+
+Raising the multiple monotonically raises median PF and cuts median n. **The mechanism is real**
+— trading less does lose less per toll. It is also converging to a ceiling well below 1.0 while n
+falls toward the admission floor, so there is no multiple at which both conditions hold. A
+fourfold increase in the threshold buys 0.15 of profit factor and costs 82% of the sample.
+
+## 5. The source's own setting is the worst cell
+
+`mult=1` is the paper's 2.0–2.5% threshold, and on this venue it is *exactly one round trip*.
+It is the worst arm-B cell on every axis: the lowest median PF (0.226) with the highest median n
+(1252) — **the most trading at the least edge.**
+
+That is #341's friction-scaling argument measured rather than asserted. A threshold spelled as a
+percentage looks conservative; the same threshold spelled as `threshold_friction_mult=1` says
+what it is.
+
+## Honesty
+
+**Selection bias.** Arm A is one pre-declared configuration and carries none. Arm B's per-asset
+best is a maximum of four draws and must never be quoted as an edge estimate — the best cell
+overall is CRV-USD at mult=4, PF 0.593, n=322, still 0.4 short of 1.0. The bias does not change
+the verdict in either direction.
+
+**Validation.** Screening result only. No walk-forward, no out-of-sample split, no CSCV/PBO and
+no deflated Sharpe: `backtest` emits aggregates and no per-trade series (`series_missing`). Same
+cached candles and the same ~5-year window as every other document here. `slippage_pct` held at
+0.0005 in every cell, so the zero column is zero *fee*, not zero cost — round-trip friction there
+is still 0.10% of notional.
+
+**Changed nothing.** A document, a driver script and a ledger row. No rule row was added, nothing
+was promoted, no config was touched, no allowlist changed. `cusum_event` remains registered and
+untraded.
+
+## Recommended next
+
+1. **Not further `cusum_event` tuning.** It is negative at zero cost on 16 of 24 assets, and its
+   own headline knob moves median PF by 0.15 across a fourfold range. A different event
+   definition is a new rule with a new pre-registration, not a continuation of this one.
+2. **#342 (triple-barrier exits) is now the only untested half of the source's claim** — but its
+   prior tightened here rather than loosening: the entry half has no gross edge for a better exit
+   to harvest.
+3. **The null grows rather than breaks.** The measured intersection goes from 0 of 90 to **0 of
+   114**. That is the result, and publishing it is the point.

docs/experiments/2026-09-01-cusum-event-first-measurement.py

@@ -0,0 +1,148 @@
+"""First measurement of `cusum_event` (#341). Pre-declared, then run.
+
+WHAT IS DECLARED BEFORE THE RUN, so the reader can hold this document to it:
+
+* **Primary metric: `n_trades`.** The question this rule's mechanism actually raises is
+  FEASIBILITY -- an event filter trades less by construction, and the ρ=-0.77 bind between edge
+  and sample size (2026-08-12-fee-curve-and-rsi-meanrev.md) says a rule that fires rarely cannot
+  be admitted no matter what its profit factor looks like. Declared primary exactly as the
+  rsi_meanrev diagnostic grid declared it, and for the same reason.
+* **Secondary: profit factor at fee 0 / 0.006 / 0.012.** Zero bounds from above everything an
+  execution fix could ever buy; 0.006 is the maker rate this account cannot currently reach;
+  0.012 is what it pays.
+* **Arm A is ONE configuration -- the shipped default.** `threshold_friction_mult=2`,
+  `lookback=168`, `atr_period=20`, `atr_stop_mult=2`, `target_rr=3`. One config means no argmax,
+  so nothing in arm A is a maximum-of-N and the selection-bias caveat that governs the fee-curve
+  document does not apply to it.
+* **Arm B is the rule's own headline knob**, `threshold_friction_mult` over {1, 2, 3, 4} at the
+  taker fee only. Disclosed as a SWEEP: 4 x 24 = 96 additional trials. Its per-asset best is a
+  maximum of four draws and must never be quoted as an edge estimate.
+
+`1` in arm B is the source's own setting and is exactly one round trip on this venue -- included
+so the paper's configuration is measured rather than argued about.
+
+Trials disclosed: 24 x 3 = 72 (arm A) + 96 (arm B) = 168.
+
+Re-run:
+    KEEL_EXPERIMENT_DB=~/keel/keel.db \
+      python docs/experiments/2026-09-01-cusum-event-first-measurement.py
+"""
+
+import json
+import os
+import time
+from concurrent.futures import ProcessPoolExecutor, as_completed
+from decimal import Decimal
+from pathlib import Path
+
+DB = os.environ.get("KEEL_EXPERIMENT_DB") or str(Path.home() / "keel" / "keel.db")
+OUT_DIR = os.environ.get("KEEL_EXPERIMENT_OUT") or str(Path(__file__).resolve().parent / "_out")
+Path(OUT_DIR).mkdir(parents=True, exist_ok=True)
+JSONL_PATH = f"{OUT_DIR}/cusum_first.jsonl"
+
+#: The same 24-asset universe the restated intersection used, so this result sits directly
+#: beside the null it is being compared against rather than beside a different population.
+UNIVERSE = [
+    "BTC-USD", "ETH-USD", "ADA-USD", "LINK-USD", "LTC-USD", "SOL-USD",
+    "XLM-USD", "PAXG-USDT", "BCH-USD", "AAVE-USD", "DOGE-USD", "DOT-USD",
+    "UNI-USD", "ZEC-USD", "ALGO-USD", "FET-USD", "CRV-USD", "ICP-USD",
+    "AVAX-USD", "NEAR-USD", "XRP-USD", "PAXG-USD", "WLD-USD", "TON-USD",
+]
+
+FEES = ["0", "0.006", "0.012"]
+SLIPPAGE = Decimal("0.0005")
+TAKER = "0.012"
+MULTIPLES = ["1", "2", "3", "4"]
+
+#: The admission floors this is measured against (#337/#338).
+MIN_TRADES = 100
+
+
+def build_jobs():
+    return [("A", asset, "2") for asset in UNIVERSE] + [
+        ("B", asset, mult) for asset in UNIVERSE for mult in MULTIPLES
+    ]
+
+
+def run_job(job):
+    arm, asset, mult = job
+    from keel_core.types import Granularity
+
+    from keel.data.db import connect
+    from keel.data.repository import Repository
+    from keel.strategy import backtest as bt
+    from keel.strategy.rules.cusum_event import CusumEvent
+
+    fees = FEES if arm == "A" else [TAKER]
+    rows = []
+    try:
+        repo = Repository(connect(DB))
+        candles = repo.get_candles(asset, Granularity.ONE_HOUR)
+    except Exception as exc:
+        return [{
+            "arm": arm, "product": asset, "mult": mult,
+            "error": f"{type(exc).__name__}: {exc}",
+        }]
+    if not candles:
+        return [{"arm": arm, "product": asset, "mult": mult, "error": "no hourly candles"}]
+
+    for fee in fees:
+        try:
+            rule = CusumEvent(product_id=asset, threshold_friction_mult=Decimal(mult))
+            result = bt.backtest(
+                rule, candles, fee_pct=Decimal(fee), slippage_pct=SLIPPAGE
+            )
+            rows.append({
+                "arm": arm,
+                "product": asset,
+                "mult": mult,
+                "fee": fee,
+                "bars": len(candles),
+                "n_trades": int(result.n_trades),
+                "win_rate": float(result.win_rate),
+                "profit_factor": float(result.profit_factor),
+                "expectancy": float(result.expectancy),
+                "max_drawdown": float(result.max_drawdown),
+            })
+        except Exception as exc:
+            rows.append({
+                "arm": arm, "product": asset, "mult": mult, "fee": fee,
+                "error": f"{type(exc).__name__}: {exc}",
+            })
+    return rows
+
+
+def done_combos():
+    """Combos already written. Append-only and each row names its own combo, so a killed run
+    resumes rather than redoing -- the same discipline as the restated-intersection driver."""
+    if not os.path.exists(JSONL_PATH):
+        return set()
+    done = set()
+    for line in open(JSONL_PATH):
+        line = line.strip()
+        if not line:
+            continue
+        try:
+            row = json.loads(line)
+        except json.JSONDecodeError:
+            continue  # torn final line from a kill mid-write
+        done.add((row["arm"], row["product"], row["mult"]))
+    return done
+
+
+def main():
+    jobs = [job for job in build_jobs() if (job[0], job[1], job[2]) not in done_combos()]
+    print(f"{len(jobs)} combos to run -> {JSONL_PATH}", flush=True)
+    started = time.time()
+    with open(JSONL_PATH, "a") as sink, ProcessPoolExecutor() as pool:
+        futures = {pool.submit(run_job, job): job for job in jobs}
+        for index, future in enumerate(as_completed(futures), start=1):
+            for row in future.result():
+                sink.write(json.dumps(row) + "\n")
+            sink.flush()
+            print(f"  {index}/{len(jobs)} {futures[future]}", flush=True)
+    print(f"done in {time.time() - started:.0f}s", flush=True)
+
+
+if __name__ == "__main__":
+    main()