docs(experiments): cusum_event measured — feasible, and without a gross edge (#341) - #684
Merged
Merged
Conversation
…oss edge (#341) The measurement #341 asked for, pre-declared and then run. 168 trials across 24 assets on ~5 years of hourly candles, at fees 0 / 0.006 / 0.012. FEASIBILITY: YES, AND IT CONTRADICTS THE WORRY FILED WITH THE ISSUE. #341 was filed with the concern that gating cuts n on rules already below the admission floor. It does not. At the shipped default n>=100 on 21 of 24 assets, median 553, max 884 -- a 5% cumulative move in an hourly crypto series is a common event, not a rare one. The three below the floor are the two PAXG pairs and TON, which is the expected shape. That objection is answered and should not be repeated. PROFITABILITY: NO, AND NOT MARGINALLY. At the 1.2% taker rate ZERO of 24 clear PF 1.0 -- median 0.343, best 0.479 (AVAX-USD, n=601). At the 0.6% maker rate this account cannot reach, still zero of 24. The intersection of n>=100 and PF>1.0 is EMPTY across all 168 trials in both arms. THE DIAGNOSTIC, WHICH IS THE POINT OF THE DOCUMENT. At ZERO fee only 8 of 24 clear 1.0 and the median is 0.925. turtle_breakout at zero fee was profitable on 4 of 4 with a max of 2.713 -- a real gross edge that cost destroyed, which is why cheaper execution was a coherent thing to want. cusum_event has essentially NO GROSS EDGE for cost to destroy: it loses at a median before a single fee is charged. Same symptom as turtle at 1.2%, different disease -- the rsi_meanrev one, and the one execution cannot fix, because zero fee bounds from above everything an execution improvement could buy and zero fee is already a loss. THE KNOB WORKS AND CONVERGES TO NOTHING. threshold_friction_mult 1 -> 4 raises median PF monotonically 0.226/0.343/0.361/0.379 while cutting median n 1252/553/330/224. The mechanism is real -- trading less loses less per toll -- and it converges to a ceiling well below 1.0 while n falls toward the floor. A fourfold threshold buys 0.15 of profit factor and costs 82% of the sample. THE SOURCE'S OWN SETTING IS THE WORST CELL. mult=1 is the paper's 2.0-2.5% and is exactly one round trip here: lowest median PF (0.226) with the highest median n (1252), the most trading at the least edge. #341's friction-scaling argument, measured rather than asserted. Arm A is ONE pre-declared configuration, so it carries no selection bias. Arm B's per-asset best is a maximum of four draws and is stated as such; its best cell is CRV-USD at mult=4, PF 0.593 -- still 0.4 short, so the bias changes nothing in either direction. THE LEDGER ROW WAS WRITTEN TWICE, and the first one is not in the file. Every non-int summary value must be a STRING: `_decode_summary` turns anything that is not an int into a `Decimal` on read, so a JSON float round-trips to a different value and the row stops hashing to its own `row_hash`. `verify_chain` caught it immediately, the corrupt trailing row was removed -- restoring the chain to exactly its prior state, verified clean at 90 rows -- and it was re-appended with the encoding every existing row already uses. The chain verifies at 91 rows. Changed nothing else: a document, a driver, a ledger row. No rule row added, nothing promoted, no config or allowlist touched. cusum_event stays registered and untraded, and the measured intersection goes from 0 of 90 to 0 of 114. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NzuKAe2RVrPt9acVAWjRyL
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The measurement #341 asked for: pre-declared, then run. 168 trials, 24 assets, ~5 years of hourly candles, fees 0 / 0.006 / 0.012.
1. Feasibility: yes — and it contradicts the worry filed with the issue
#341 was filed with the concern that "gating cuts n on rules already below the admission floor." It does not.
A 5% cumulative move in an hourly crypto series is a common event, not a rare one. The three below the floor are the two PAXG pairs and TON — gold and the thinnest name in the universe, which is the expected shape. That objection is answered and shouldn't be repeated.
2. Profitability: no, and not marginally
The intersection of n ≥ 100 and PF > 1.0 is empty across all 168 trials.
3. The diagnostic — this is the
rsi_meanrevdisease, not theturtle_breakoutoneThe number that matters is the zero-fee column: only 8 of 24 clear 1.0, median 0.925.
turtle_breakoutat zero fee was profitable on 4 of 4 with a max of 2.713 — a real gross edge that cost destroyed, which is why cheaper execution was a coherent thing to want. This rule loses at a median before a single fee is charged. Same symptom as turtle at 1.2%, different disease — and the one execution cannot fix, because zero fee bounds from above everything an execution improvement could buy, and zero fee is already a loss.4. The knob works exactly as designed, and converges to nothing
Monotonic in the right direction — trading less does lose less per toll, so the mechanism is real. It is also converging to a ceiling well below 1.0 while n falls toward the floor. A fourfold threshold buys 0.15 of profit factor and costs 82% of the sample.
5. The source's own setting is the worst cell
mult=1is the paper's 2.0–2.5%, which on this venue is exactly one round trip: lowest median PF (0.226) with the highest median n (1252) — the most trading at the least edge. That's #341's friction-scaling argument measured rather than asserted.Honesty
Arm A is one pre-declared configuration, so it carries no selection bias. Arm B's per-asset best is a maximum of four draws and is stated as such — best cell CRV-USD at mult=4, PF 0.593, still 0.4 short, so the bias changes nothing either way. Screening result only: no walk-forward, no PBO (
series_missing). Slippage held at 0.0005 throughout, so the zero column is zero fee, not zero cost.The ledger row was written twice
Worth recording. Every non-int summary value must be a string:
_decode_summaryturns anything that isn't an int into aDecimalon read, so a JSON float round-trips to a different value and the row stops hashing to its ownrow_hash.verify_chaincaught it immediately. The corrupt trailing row was removed — restoring the chain to exactly its prior state, verified clean at 90 rows — and re-appended with the encoding every existing row already uses. The chain verifies at 91 rows.Changed nothing else
A document, a driver script, a ledger row. No rule row added, nothing promoted, no config or allowlist touched.
cusum_eventstays registered and untraded.The measured intersection goes from 0 of 90 to 0 of 114. That is the result, and publishing it is the point.