Skip to content

fix: hand out unique nonces when the counter catches up - #198

Open
pucedoteth wants to merge 1 commit into
hyperliquid-dex:masterfrom
pucedoteth:fix-duplicate-nonce-on-catch-up
Open

fix: hand out unique nonces when the counter catches up#198
pucedoteth wants to merge 1 commit into
hyperliquid-dex:masterfrom
pucedoteth:fix-duplicate-nonce-on-catch-up

Conversation

@pucedoteth

Copy link
Copy Markdown

The bug

next_nonce() can hand the same nonce to two concurrent callers, and the exchange rejects the duplicate.

#108 added the catch-up branch so that an idle client, whose counter has drifted behind the wall clock, stops signing with stale nonces. That PR's stated goal was that next_nonce "always returns a unique nonce". The branch advances the counter but returns a value it never claimed from it:

if nonce + 300000 < now_ms {
    CUR_NONCE.fetch_max(now_ms + 1, Ordering::Relaxed);
    return now_ms;   // <- every racer in this branch gets this same value
}

Every caller that reads a stale nonce before the first fetch_max lands takes this branch, and each returns the identical now_ms.

The window is narrow, but it opens precisely when a burst is most likely. The counter advances one per call while the clock advances 1000/s, so any client sending fewer than ~1000 actions per second drifts behind, and after ~300 s of quiet the next actions all land in this branch together. next_nonce backs every exchange action (order, cancel, modify, transfers, …), so the loser of the race gets a rejected action.

Observed on master — eight threads released from a barrier with the counter parked behind the clock:

duplicate nonces handed out:
[1787958744609, 1787958744612, 1787958744610, 1787958744611,
 1787958744613, 1787958744614, 1787958744615, 1787958744609]
                                              ^^^^^^^^^^^^^ same as the first

The fix

Catch the counter up to the wall clock, then claim a slot from it, so each caller in the branch takes its own value:

CUR_NONCE.fetch_max(now_ms, Ordering::Relaxed);
return CUR_NONCE.fetch_add(1, Ordering::Relaxed);

The first caller still returns now_ms and the counter still lands on now_ms + 1, so single-threaded behaviour is byte-for-byte what #108 intended; only the racing callers change, from a duplicate to the next value.

Testing

next_nonce_is_unique_after_catch_up parks the counter behind the clock and releases eight threads from a barrier, repeated 50 times so the race is caught reliably rather than flakily.

  • On current master with this test: fails 6 out of 6 runs.
  • With the fix: passes 6 out of 6 runs.

Full suite: cargo test --lib → 13 passed, 0 failed. cargo fmt --check and cargo clippy --all-targets are clean.

hyperliquid-dex#108 added a catch-up branch to next_nonce so an idle client whose
counter has fallen behind the wall clock does not keep signing with stale
nonces. Its goal was that next_nonce always returns a unique value, but
the branch returns now_ms to every caller that reaches it:

    if nonce + 300000 < now_ms {
        CUR_NONCE.fetch_max(now_ms + 1, Ordering::Relaxed);
        return now_ms;
    }

The counter is advanced past now_ms, but the value handed back is not
claimed from it. Concurrent callers that read a stale nonce before the
first fetch_max lands all take this branch and all return the same
now_ms, and the exchange rejects the duplicates.

The window is small but it opens exactly when a client is most likely to
send a burst: after an idle period, which is what puts the counter more
than 300 seconds behind in the first place.

Catch the counter up and then claim a slot from it, so each caller in the
branch gets its own value. The first caller still returns now_ms, so
single-threaded behaviour is unchanged.

Adds a regression test that fails reliably on the current code.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

@koriyoshi2041 koriyoshi2041 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed exact head 5417ba3. The fetch_max(now_ms) plus fetch_add(1) sequence gives each catch-up racer a distinct claimed slot while preserving the first caller result of now_ms; relaxed ordering is sufficient for uniqueness because the guarantee comes from atomic modification order. I ran the focused barrier regression, five consecutive cargo test --lib runs, cargo fmt --check, and cargo clippy --all-targets with warnings denied; all passed. The repeated full-suite runs also exercised the shared global counter test alongside the rest of the lib tests without interference.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants