fix(execution): a SELL is clamped to what the venue holds, and refused when it holds nothing (#667) - #670
Merged
Merged
Conversation
…d when it holds nothing (#667) `_sell_base_size` quantized the exit down to the venue's increment and did nothing else. No SELL path read a base balance -- rail 13's own comment recorded the asymmetry: "SELL intents never fetch a balance (the rail exempts them)". So keel sold the quantity its ledger believed it held, and the ledger runs high in three ways it cannot see from its own records: a venue that takes its taker fee out of the received base leaves 0.9985 where the order said 1.0000; a partial fill leaves less still, and `orders.filled_quantity` is written only when the venue's post-fill status was observable; an operator who moves coins out tells keel nothing at all. On a cash account each is a rejected exit -- a position that wanted out, still in, unprotected. On a margin-enabled account (#666) the venue fills the difference by opening a short. That is *bay' ma la yamlik*, arrived at by arithmetic rather than by anyone's intent, and reachable without a single line of the strategy layer proposing one. TWO MECHANISMS, DELIBERATELY SPLIT. `_clamp_to_held` shrinks an order that is too big for a position that really exists -- down only, never up, never to zero, and unchanged when the holding is unreadable. Rail 21 refuses the one case the clamp will not touch: a venue affirmatively reporting nothing held while the ledger expects something. `_build_intent` already returns None when the LEDGER says zero, so reaching rail 21 at all means the two disagree -- and refusing sells nothing and traps nothing, because there is nothing there to trap. `Balance.total`, NEVER `Balance.available`, and this is the load-bearing line. `available` excludes base committed to resting orders, and keel's own protective bracket commits the ENTIRE position -- so for exactly the products keel is protecting, `available` reads ~0. A clamp on that number would shrink every exit to nothing and rail 21 would then veto it, refusing every stop roll and every exit keel has ever placed. `total` is available + hold, and ownership rather than encumbrance is what the prohibition asks about. It also makes `_roll_stop`'s cancel-then-place ordering irrelevant to the number, which `available` would not. RAIL 21 FAILS OPEN ON UNKNOWN -- the deliberate inverse of rails 12/13/17, and the one place in this engine where "unknown is a rejection" would do more harm than the hole it closes. A refused BUY costs nothing; a refused SELL strands a position that wanted out. `_sell_base_size`'s asymmetry table, applied to a balance instead of an increment. Do not fix this into consistency. This is NOT the auto-resize `_record_observed_fill_quantity` declines to do, and that refusal is well argued and still stands unchanged. What it refuses is CANCELLING A PROTECTIVE ORDER on the strength of a settling snapshot. Nothing here cancels anything; it only declines to ask for more than is there, which no snapshot ambiguity can make wrong. Also: `filled_quantity` and `average_filled_price` were coupled by one guard that belongs to the price alone, so a venue answering with a filled size and no average price lost the size too -- and the size is the half that measures the drift. The quantity observation now runs first. Drift is recorded under `balance_drift:<product>` and surfaced by `keel doctor` as `balance.drift` (WARN, not FAIL -- every cause is legitimate). A log line alone requires an operator to know to grep for it, and the drift outlives the order that discovered it. 21 mutants, 21 killed. One survivor was worth the exercise: every rail-21 test built its intent by hand, so dropping `available_base=` from the intent the executor actually builds left all of them green. The end-to-end tests through `execute`, `place_bracket`, `scale_out` and `roll_to_break_even` exist because of that survivor. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NzuKAe2RVrPt9acVAWjRyL
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #667.
_sell_base_sizequantized the exit down to the venue's increment and did nothing else. No SELL path read a base balance — rail 13's own comment recorded the asymmetry: "SELL intents never fetch a balance (the rail exempts them)."So keel sold the quantity its ledger believed it held, and the ledger runs high in three ways it cannot see from its own records:
orders.filled_quantityis written only when the venue's post-fill status was observable;On a cash account each is a rejected exit — a position that wanted out, still in, unprotected. On a margin-enabled account (#666) the venue fills the difference by opening a short. That is bay' ma la yamlik arrived at by arithmetic, reachable without a single line of the strategy layer proposing one.
Two mechanisms, deliberately split
executor._clamp_to_heldbase_balance)_build_intentalready returnsNonewhen the ledger says zero, so reaching rail 21 at all means the two disagree. Refusing there sells nothing and traps nothing, because there is nothing to trap.Neither cancels a protective order — which is why
_record_observed_fill_quantity's refusal to auto-resize stands beside them unchanged. What that refusal declines is cancelling a resting bracket on the strength of a settling snapshot. Nothing here cancels anything; it only declines to ask for more than is there, which no snapshot ambiguity can make wrong.Balance.total, neverBalance.availableThe load-bearing line of the change.
availableexcludes base committed to resting orders, and keel's own bracket commits the entire position — so for exactly the products keel is protecting,availablereads ~0. A clamp on that number would shrink every exit to nothing and rail 21 would then veto it, refusing every stop roll and every exit keel has ever placed.totalisavailable + hold; ownership, not encumbrance, is what the prohibition asks about.It also makes
_roll_stop's cancel-then-place ordering irrelevant to the number. Onavailablethe holding would jump from ~0 to the full position across the cancel, and a roll's fate would depend on which side of that line it was read from. There is a test for this specifically.Rail 21 fails OPEN on unknown
The deliberate inverse of rails 12/13/17, and the one place in this engine where "unknown is a rejection" would do more harm than the hole it closes. A refused BUY costs nothing; a refused SELL strands a position that wanted out. It is
_sell_base_size's asymmetry table applied to a balance instead of an increment, and the comment says not to fix it into consistency.Rail 21 joins
LIVE_STATE_RAILS, so paper skips it and reports the skip — a paper track record that silently skipped a live-state rail would score a strategy on evidence live trading would have refused.Two more things
filled_quantitywas coupled to the price. Onefill <= 0guard returned before the quantity was recorded, so a venue answering with a filled size and no average price lost the size too — and the size is the half that measures the drift. The quantity observation now runs first. The guard itself is kept, because a zero overwriting a usable price estimate was always the right thing to decline.Drift is surfaced, not absorbed. Recorded under
balance_drift:<product>and reported bykeel doctorasbalance.drift— WARN, not FAIL, because every cause is legitimate. A log line alone requires an operator to know to grep for it, and the drift outlives the order that discovered it: the next exit is clamped by the same amount.Costs and behaviour changes
get_balancesper SELL-ish order. An entry now costs two (rail 13's quote read, then the bracket's base read).test_exit_signal_never_fetches_a_balanceassertedget_balances_calls == 0and was right for as long as the only balance question was rail 13's. A SELL is never clamped to the venue's available base — fee dust, partial fills and out-of-band drift all oversell #667 asks a different question of the same endpoint, and it is the exit that must ask it.Verification
21 mutants, 21 killed. One survivor was worth the exercise: every rail-21 test built its
OrderIntentby hand, so droppingavailable_base=from the intent the executor actually builds left all of them green. The end-to-end tests throughexecute,place_bracket,scale_outandroll_to_break_evenexist because of that survivor.Full suite: 5081 passed, 3 skipped. ruff and mypy clean across 120 source files.
Note on ordering with #669
The rail count moved from nineteen to twenty in
guards.pyand five docs, anddocs/fiqh-basis.mdgains a "rail 21 is the second fiqh rail" section. #669 (the bay' ma la yamlik amendment) edits the same file on a branch cut from the same base, so whichever merges second needs a rebase — I'll reconcile them rather than let either land stale.