Repository navigation
Conversation
karknu
added
tx-submission
karknu
force-pushed
the
karknu/txsub-bounded-sweep
branch
6 times, most recently
from
October 24, 2026 00:20
063846c to
a91a62b
Compare
karknu
force-pushed
the
karknu/txsub-bounded-sweep
branch
5 times, most recently
from
November 2, 2026 07:25
7b41d68 to
77e3e7f
Compare
karknu
force-pushed
the
karknu/txsub-bounded-sweep
branch
8 times, most recently
from
November 11, 2026 07:29
dee0432 to
75d6384
Compare
V2 pipelined a txid request whenever the window had room, so a peer whose txids were all unacked and unresolved was polled with ack=0, req=N messages, and under load that polling cost a relay most of its CPU. A pipelined txid request now both acks and requests: with nothing to ack the peer waits for its unacked txids to resolve. This is stricter than V1, which pipelines such a poll after a body request while the window has room. The disablePipelinedTxIdRequests flag goes, and with it the guard that held a pipelined request after an empty reply, which the rule leaves nothing to fire on. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The sim-benchmarks never built a large retention queue or swept one with expired entries waiting, so nothing measured the sweep a relay runs when it falls behind. Bench.TxSubmissionV2Sweep builds queues of 10k and 100k retained transactions and times inserting them, and one sweep of 100k with none, 1k, 10k and all of them past their deadline. At this commit one sweep of 100k with all expired takes about 20 ms, against 0.7 ms with none: a single STM transaction that long rarely commits beside peers writing the shared state. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The counters thread swept the shared tx state in one STM transaction that expired every retained transaction past its deadline. Under constant load on a testnet relay that transaction ran longer than the gap between peers' writes, so it never committed, nothing expired, and every retry had more to do: the thread spun at 2-3 GB/s of allocation and stopped emitting counters. Each sweep now does at most 1000 units of work: it expires up to half of them, earliest first, and reports whether expired entries remain; a backlog drains in further transactions of that size, 10 ms apart so that peers' transactions commit between them, for at most an interval before the next sweep. The sweep no longer scans the tx table: it carries a cursor and examines the rest of its budget as entries from there on, round the table, and dooms those claimable for a grace of two sweep intervals with nobody attempting or submitting them. The next tick reads each live peer's advertised set, one peer per transaction, and drops the doomed entries no peer holds in the sweep's transaction, its budget included, sparing the rest. An advertisement or a claim of a doomed entry clears the doom, so advertising an existing entry writes nothing shared, as before, and the sweep reads nothing but the state. A lookup entry lives with its residence: expiry drops it, a peer still holding the key acknowledges it as resolved, and a body arriving for it counts as late. The shared-state trace carries sharedRevision, which every write advances. Tests compare the bounded forms with the unbounded sweep and check the cursor's rotation, the doom, and, in IOSim, the thread's cadence, drain cap, review and budget. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The retained-tx queue was an IntPSQ keyed on Time, whose comparisons go through Integer picoseconds. Profiling the sweep stalled under load put a fifth of its CPU in bigNatCompare, integerCompare and __gmpn_cmp beside the IntPSQ merges themselves. The queue now keys on Word64 nanoseconds of monotonic time, 584 years of range, and the RetainedTxs API takes and returns Word64. retainDeadline converts a deadline rounding up and retainNow the current time rounding down, so an entry never expires before its deadline and at most a nanosecond after it; a negative time, as a non-positive lifetime gives, is due at once, and a time past the range at its end, rather than wrapping. The shared-state traces render deadlines as seconds, numbers where they were strings. Tests: deadlines over arbitrary picosecond times, through the end of the range, never expire early and expire within a nanosecond; the existing retention tests compare against the stored nanosecond deadline. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The counters thread read the retired accumulator, the registry and every live peer's counters cell in one STM transaction. Each peer writes its cell on every protocol event, so with a hundred peers that transaction is restarted by every write and can starve, as the sweep did. The snapshot now reads like the mux counters: one transaction for the retired accumulator and the list of live cells, then each cell in its own. A peer retiring in between is counted once: retirePeerCounters moves its counts into the accumulator in the transaction that unregisters it, and the cell keeps its final counts. The registry is keyed by the peer's address and a registration number, so a release removes its own entry even when a reconnect has registered the address again. snapshotCountersWith takes the transaction runner, so a test can run peers between a snapshot's transactions and check that every count is taken exactly once; registerPeerCounters is the one place a registration is made. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The counters thread traces the shared tx state every seven seconds when it changed, and since the bounded sweep that trace carries the numbers that say how the sweep is doing: the revision, whose growth is the rate of writes to the shared state, and the residences. They were readable only with the trace at Debug, because the TraceTxLogic instance had no metrics, where TxSubmissionCounters has them and so reaches Prometheus with no configuration. TraceSharedTxState now carries metrics: txSubmission.sharedState.revision and .generation, and one gauge per count of sharedTxStateCounts, named by its field. They are emitted at the trace's cadence whatever its severity, and documented with the namespace. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A relay polls every inbound peer for txids at about one exchange per round trip, and under load that polling is most of its CPU while nearly every txid it carries is a duplicate. The cost is per exchange, not per txid, and it grows with every inbound peer. The server now keeps a bounded set of peers polled at full speed without judging peers on what they deliver: local roots by standing, tx sources while they produce, and the first txFastPeers of the inbound peers in the salted order of their addresses, the salt drawn afresh every rotation, so a departure moves the next peer up at once. Everyone else waits a jittered cadence between txid requests, parked meanwhile on a counter that moves only when an entry becomes claimable, a lease is released or a stuck cap raised, unless it holds a body it could not submit yet. The knobs live on the decision policy, which the counters thread checks at start; that thread runs the rotation and asks the governor once a sweep which peers are local roots, so that no decision reads its state. The txid request cap is now the window, since a peer on a cadence accumulates more than six ids between requests, and the counters tell slow-tier requests apart and give a census of the peers by standing. Properties check the spacing, the source rule, the placement against the sorted order and the census; the simulation runs with every peer slow. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The mean txids per request could not tell empty replies, which are wasted round trips, from full windows, which say the window size binds. The inbound tx-submission counters gain five reply-size counts, none, one, two to four, five to nine and ten or more, a partition of the replies to requests for txids, published as metrics. A registry property checks each reply lands in its bucket for every size and boundary, and that a reply to a request for acknowledgements alone lands in none. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A decision's keys were resolved to txids in a later read of the shared state. A key leased to this peer whose tx another peer had submitted meanwhile lost its lookup entry when the retention expired, and the resolution then failed the peer. The registry now resolves the action in the deciding transaction: the run functions return the action with the txids and sizes to request or the bodies to submit, and the two resolve calls leave the API. A park is resolved there too: the decision names the counter to wait on and the registry reads its value, so the pure layer carries no counter values of its own. A slow peer whose gate holds, with no body buffered and none of its advertised keys in the table, parks on its timer alone: nothing the claimable counter announces is its to claim before the gate opens.
The check that every body answers a requested txid went through a map, so a reply could carry one body twice. The second copy answered no pending key and passed for a late body, which cancelled the omission of a key the peer had left out of the same reply. A reply with more bodies than distinct txids is now a protocol error, traced like the other body errors. An impairment in the simulation sends such replies, and a property checks that the inbound side rejects them.
A peer's score, which delays its claims, netted a batch's rejections against the accepted txs of the same batch, so a batch mixing invalid bodies with valid ones cost the peer nothing up to seven invalid beside three valid at the default decrement. The accepts now come off the score the peer had and the batch's rejections are added after. A unit test covers a mixed batch. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A peer whose decision found nothing to do ran the scheduler once more when it had just come from a request or a submission, in case the idle peer could now claim what the busy one could not. Nothing in a decision reads that phase, so the second run could only repeat the first: the counter a park waits on is read in the deciding transaction, and the park's delay is the earliest timed wake. Every episode paid a decision for nothing. The server parks at once. The phase, written and never read, goes with the re-run. A property drives the server to a park after a txid reply and counts its decisions: one. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Every wake of a park registered the gate's delay again, and a registered delay could not be cancelled: each claimable bump left one dead timer per parked slow peer until its gate passed, all of them in the timer manager. The idle loop keeps the timer it holds while the park's deadline, the gate's time or the next timed wake, is unchanged, and cancels it when the deadline changes or the park ends. RegisteredDelay is built on the cancellable delay the timer library offers and gains cancel; a non-positive delay is fired at birth. A property counts the timers a gated peer registers across claimable bumps from the simulation's trace: one, one more for the lease's claim time once the gate has passed, one swapped for an earlier deadline, and the one it leaves cancelled. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
karknu
force-pushed
the
karknu/txsub-bounded-sweep
branch
from
November 13, 2026 10:00
75d6384 to
ade2c7e
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
TX-submission improvement based on leios testnet experience.
Checklist
Quality
Maintenance
ouroboros-networkproject.