docs(readme): lead with the fee-reality benchmark, rendered from the ledger (#646, partial) - #679
Merged
Merged
Conversation
…ledger (#646, partial) The README already stated the honest result in prose -- "no shipped rule family is net-positive at the taker fee actually paid". #646 asks for it as NUMBERS: the same rule priced twice, fee-blind and then honestly, before the feature list. RENDERED, NEVER TYPED, and that is the issue's binding constraint rather than a nicety. Every figure is read out of `docs/experiments/trials-ledger.jsonl` -- the hash-chained record of what was actually run -- by `scripts/render_fee_reality.py` and reformatted. A hand-typed benchmark would be a marketing claim wearing a measurement's clothes, in the one block whose entire argument is the difference between those two. The result, from the fee-curve row of `fee-curve-and-rsi-meanrev-2026-08-12`: four of four assets profitable with the fee removed, ZERO of four surviving the 1.2% taker actually charged, and break-even fees spanning 0.068% (BTC) to 1.741% (ZEC) -- a factor of ~26 across four assets running the same rule on the same clock over the same window. THE CAVEATS TRAVEL WITH THE NUMBERS, and a test pins that they do. Every configuration is the argmax of its asset's 144-cell slice, selected on the same data it is re-priced on -- maxima of 144 draws, not edge estimates, and the ledger's `selection_bias` field says so at full strength. Quoting them as an asset's expected profit factor is exactly what the experiment record forbids. The bias runs AGAINST the finding, which is why the comparison survives it: it inflates the arm that wins with the fee removed, and that arm still dies when the fee is charged. Slippage is held at 0.0005 in every cell, so the zero column is zero FEE and not zero cost. PARSING PROSE IS DELIBERATE. The ledger stores the curve as a recorded sentence, because that row was written by the experiment that produced it and rewriting history to suit a renderer would be the tail wagging the dog. So the parser is strict and every recognition failure is fatal -- a parse that silently produced three of four assets would be the quiet half-truth this block exists to refuse. Two bugs the tests found in the renderer's own output, both worth recording: `[\d.]+` is greedy enough to swallow the SENTENCE'S full stop into the last asset's profit factor, and "1.303." is still readable, still wrong, and would have shipped inside a table. And the fee columns rendered as raw fractions ("PF at 0.012") -- making the reader perform the conversion that produced #247's costing error in the first place. 9 mutants, 9 killed, and the last took three attempts to write. The column-count guard sat behind the missing-break-even guard, so every input meant to reach it tripped the earlier one instead and the mutation survived while looking pinned. NOT DONE: the terminal capture. #646 also asks for a ~10-second GIF of a real run with committed generation tooling; that is unbuilt and the issue stays open for it. What is here is the numbers half, and the half a GIF would have to agree with. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NzuKAe2RVrPt9acVAWjRyL
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The README already stated the honest result in prose. #646 asks for it as numbers: the same rule priced twice, before the feature list.
Four of four profitable with the fee removed. Zero of four survive the fee actually charged. Break-even spans 0.068% to 1.741% — a factor of ~26 across four assets running the same rule on the same clock over the same window.
Rendered, never typed
That is the issue's binding constraint, not a nicety. Every figure is read out of
docs/experiments/trials-ledger.jsonl— the hash-chained record of what was actually run — byscripts/render_fee_reality.pyand reformatted. A hand-typed benchmark would be a marketing claim wearing a measurement's clothes, in the one block whose entire argument is the difference between those two.tests/test_fee_reality_block.pyfails if the README and the renderer disagree, and separately asserts that every number in the block appears in the ledger row — stronger than "the renderer produced it", because a renderer with a literal baked in would satisfy the drift pin with both sides carrying the same invention.The caveats travel with the numbers
Pinned by a test, because dropping them would make the block a stronger claim than the measurement supports:
selection_biasfield says so at full strength.Parsing prose is deliberate
The ledger stores the curve as a recorded sentence, because that row was written by the experiment that produced it, and rewriting history to suit a renderer would be the tail wagging the dog. So the parser is strict and every recognition failure is fatal — a parse that silently produced three of four assets would be the quiet half-truth this block exists to refuse.
Two bugs the tests found in the renderer's own output:
[\d.]+is greedy enough to swallow the sentence's full stop into the last asset's profit factor.1.303.is still readable, still wrong, and would have shipped inside a table.PF at 0.012), making the reader perform the conversion that produced fix(strategy): price fills at the taker rate, and wire the dormant PBO gate into promotion #247's costing error in the first place.Verification
9 mutants, 9 killed — and the last took three attempts to write. The column-count guard sat behind the missing-break-even guard, so every input meant to reach it tripped the earlier one instead, and the mutation survived while the test looked like it pinned something.
Full suite 5135 passed / 3 skipped; ruff and mypy clean.