docs(experiments): every flat-priced record points at the cost correction - #690
Merged
Conversation
…tion Recommended by #335's own restatement and not done at the time. Sixteen records report profit factors priced at the flat 5bp slippage floor; one document -- the restatement itself -- knows that no asset in keel's universe reaches that floor. WHY A POINTER AND NOT A RE-RUN. The correction is conservative-only: real cost is higher, so a corrected profit factor can only fall, and every one of the 120 measured deltas was negative or zero. Re-running thirteen documents to move numbers that were already null spends real compute to reach the same conclusion. What changes is not any document's VERDICT but what a reader may conclude by comparing ACROSS documents -- a 0.9 in one and a 1.06 in another are not on the same scale when both were priced at a rate neither asset trades at, and nothing on the page says so. NOTHING IS REWRITTEN. `docs/experiments` is an append-only account of what was run; editing a measured number would falsify it. Every change here is an addition, and `git diff` against main contains no removed line that is not part of a note being reflowed. THE INDEX GOT ITS OWN NOTE, because the blanket one was FALSE there. Applied uniformly, the note told `docs/experiments/README.md` that "the figures below are priced at the flat floor" -- and the index has no figures. It now says what is true of it: that every record dated before 2026-09-01 is flat-priced. A note that misdescribes the page it sits on is worse than no note. TEST FIRST, and the third assertion is the one worth having. A bare link would read as "this result is retracted", which would be a worse error than the one being corrected -- so the pin requires the word "verdict" within 600 characters of the link, and a mutation replacing the qualification with "These numbers are wrong." fails it. Tests: `tests/test_experiment_cost_pointers.py` (3 tests). 3 mutants, 3 killed: a record losing its pointer, a pointer that reads as a retraction, and a discovery regex matching nothing -- the last because every assertion here passes vacuously if the scan finds no records. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NzuKAe2RVrPt9acVAWjRyL
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Recommended by #335's own restatement, not done at the time.
Sixteen records report profit factors priced at the flat 5bp slippage floor. One document — the restatement itself — knows that no asset in keel's universe reaches that floor (1.1× to 36.8×, median ~10×).
Why a pointer and not a re-run
The correction is conservative-only: real cost is higher, so a corrected profit factor can only fall, and every one of the 120 measured deltas was negative or zero. Re-running thirteen documents to move numbers that were already null spends real compute to reach the same conclusion.
What changes isn't any document's verdict — it's what a reader may conclude by comparing across documents. A 0.9 in one and a 1.06 in another aren't on the same scale when both were priced at a rate neither asset trades at, and nothing on the page said so.
Nothing is rewritten
docs/experimentsis an append-only account of what was run; editing a measured number would falsify it. Every change here is an addition —git diff origin/main -- docs/experiments/contains no removed line that isn't part of a note being reflowed.The index needed its own note, because the blanket one was false there
Applied uniformly, the note told
docs/experiments/README.mdthat "the figures below are priced at the flat floor" — and the index has no figures. It now says what's true of it: that every record dated before 2026-09-01 is flat-priced.A note that misdescribes the page it sits on is worse than no note. I caught this by reading the rendered output rather than trusting that a uniform edit was uniformly correct.
Test first
The third assertion is the one worth having: a bare link would read as a retraction, which is a worse error than the one being corrected. The pin requires the word "verdict" within 600 characters of the link, and a mutation replacing the qualification with "These numbers are wrong." fails it.
Tests:
tests/test_experiment_cost_pointers.py— 3 tests.3 mutants, 3 killed: a record losing its pointer; a pointer that reads as a retraction; and the discovery regex matching nothing — that last one because every assertion here passes vacuously if the scan finds no records.
Full suite 5258 passed / 3 skipped; ruff clean.