Repository navigation
raid-walk: a busy gear slot is retried, not an exit 18 - #457
Merged
Merged
Conversation
…g job out Walk 98b restarted 11 times with exit 18 on ACTION_ERR|gear|busy|A gear job is already running (7 of them in its first 7 min, 7 raids launched at once). The module's gear queue is one slot for the whole server (BotGearService::Queue refuses while m_running), and a job's TBM:GEAR verdict goes only to the character that queued it (SendLines -> the requester's chat handler). The previous retry waited for the blocking job's own TBM:GEAR line, which another raid's session never receives: it burned the whole 60 s budget and ended the walk 18. The refit now re-sends the same line after a jittered backoff (2 s, 4 s, 8 s, 16 s, then 20 s plus 0-3 s), each retry noted, until a send is accepted or the 120 s budget leaves no room for another. Each send is answered before the next goes out, so two gear requests never overlap, and only an accepted send's verdict counts. A budget spent on busy answers ends the walk on the module's own line, exit 18. Tests (fake clock, jitter stubbed): busy-busy-ok re-sends after 2 s and 4 s with both retries noted; a 30 s slow busy answer is waited out; the slot busy for the whole budget makes 9 sends over 110 s and ends 18.
…ckoff The busy retry slept its backoff and then re-sent without reading the chat. The module's gear slot can be held by this commander's own earlier job (a walk restarted while its previous process's refit still ran), and the verdict goes to the requester's chat (BotGearService::SendLines): that job's TBM:GEAR line, read after the re-send, was taken as the retry's success, so the walk went on with its refit still running. The line carries no job or requester id (BotGear::SummaryLine is status, counts, tier, cap, averages, capped, skipped, purged), so it cannot be correlated. Each backoff is now a drain: every TBM:GEAR line the module answers in the window is read, dropped and noted as an earlier job's, and only a verdict read after a send the module accepted counts. The module frees the slot only after sending the job's verdict (Finish sends, then clears m_running), so a line before the re-send is never the re-send's, and the retry still re-sends at the same 2/4/8/16/20 s times. Tests (fake clock, jitter stubbed): an earlier job's partial verdict lands 1.5 s into the backoff; the walk drops it, re-sends at 2 s and ends 0 on the accepted send's ok line (the old code ended 18 on the partial).
…d retry The busy retry drained its backoff and re-sent without reading the chat: the slot's job can be this commander's own earlier refit (a walk restarted while its previous process's job ran), and that job's TBM:GEAR line, read after the re-send, was taken as the retry's. The module gives no way to tell: one slot for the whole server, the verdict sent only to the requester's chat (BotGearService::SendLines), and neither TBM:GEAR (BotGear::SummaryLine) nor ACTION_ERR|gear|busy (SendActionError) carries a job or requester id. The walk now gives the slot's job the same 1m bound it gives a gear job of its own and waits for that job's verdict instead of re-sending. A TBM:GEAR line for the refit's tier inside the window is the slot's job answering this commander, so it is used as the refit's verdict (a partial ends the walk 18). A line for another tier answers another request and is dropped. Only a window with no verdict sends the line again: the slot's job is then another requester's, whose verdict never reaches this chat, so the accepted re-send's verdict is the only one that can answer the refit. The 2m budget is one window for the slot's job plus one for the re-send's own verdict. Remaining limit, in the code comment and docs/okf/raids/raid-walk-reference.md: the window's last poll and the re-send are separate calls, so a same-tier verdict written between them can still be read as the re-send's. With no job or requester id in the reply, no client can close that window. Tests (fake clock, jitter stubbed), each failing on the r2 drain: another requester's job holds the slot 5 s; the walk waits the 1m bound, re-sends once, and only the re-send's verdict ends it (r2 sent 3 times, at 2 s and 6 s). The slot's own job answers partial at 5 s: used, one send, exit 18 (r2 dropped it and re-sent). A tier-1 line lands in the window and is dropped (r2 ended 18 on it).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The bug
When several raids start on one server, the module's single gear job slot refuses all but one: "A gear job is already running". The walk then ended with exit 18, and the autopilot relaunched it. ZG walks 98b to 100 saw 6 to 11 of these restarts each, most in the first minutes.
The fix
BotGearService.cpp:138), so a verdict at another tier is dropped.Documented limit: a same-tier verdict that lands between the last poll and the re-send is not correlated (the module carries no job id). A retry can also run one reply window past the budget.
Tests
Resend timing, the own-job verdict used with one send, other-tier drop, a slow busy reply, and the cap.
Review
Codex adversarial: 3 rounds, now no high. Cold review: SHIP.