fix(client): coordinate rate-limit backoff across concurrent requests

[pchuri]

Problem

#258 fixed retries that fired immediately when Retry-After is zero. It does not fix rate limiting during recursive page traversal: getAllDescendantPages runs up to 10 requests concurrently while each request retries on its own (3 retries, 1s/2s/4s). A burst that hits a low shared limit (Data Center can allow as little as 3 requests per second) is rejected together, retried in lockstep, rejected again, and runs out of retries even though a patient client would have finished.

Reproduction

tests/traversal-rate-limit.test.js adds a deterministic stand-in for such a server: a token bucket on jest's fake timers (virtual time, so it is exact and takes milliseconds). On main, a 40-page tree against a 3 requests/second bucket fails with Request failed with status code 429.

What did not work

• Capping concurrency (halving the allowed in-flight requests on each new throttle) failed 107 of 225 simulated scenarios: a server that answers immediately lets even one request at a time arrive faster than the limit. The fix has to pace request starts.
• A first version of the pacing regressed against main in two ways that review found: a persistent refusal made ten concurrent requests fail after 27 minutes instead of 7 seconds, and 429s unrelated to the request rate ratcheted the spacing up for good.
• Decaying the spacing with elapsed time (half-life 1-2 s) instead of per success made rate-independent 429s recover faster, but crossed a real limit so often that a 1 request/second server failed outright (and cost 5x the rejections at 2 s), so it was dropped.

Changes

New lib/request-gate.js, shared by every request one client makes (a request interceptor acquires a start, the response interceptor reports successes):

• Only 429s. A 503 keeps its independent per-request retries exactly as before and never paces other requests.
• Spacing. A new 429 doubles the minimum spacing between request starts (100 ms, capped at 2 s); each success shrinks it by 10%, with no memory of what caused the throttle. Starts are released one at a time in arrival order.
• Retry-After. Only a positive Retry-After pauses every request; without one (Data Center answers 0) there is nothing to wait for and just the spacing changes. The retrying request still waits its own delay, which for a single request is identical to before (the exponent never exceeds what one request would reach alone, so [1000, 2000] and [3000] are unchanged).
• Stale rejections and progress. A rejection from a burst that was already in flight when the spacing last changed is stale: it does not widen the spacing again. A rejection counts against a request's own retries only if it is new and no request has succeeded since this one first started; otherwise a single unlucky request could run out of retries while the server serves everyone else. Such free retries are capped at the request's own maxRetries, so a page that is always refused fails after the same four attempts as before.
• Circuit. After a run of new 429s with no success (maxRetries + 1 with a Retry-After, maxRetries + 4 without, since the spacing needs a few doublings to find the server's pace) the gate stops retrying and stops delaying until a request succeeds, so a server that keeps refusing fails about as fast as a single request would.
• No throttling, no cost. Without a throttle there is no pause and no spacing; starts are released back to back, so concurrency and throughput are unchanged (the gate adds a microtask per request).

Unchanged: the number of retries per request, the delay for a single request, which statuses and methods are retried, and the error surfaced when the retries run out. The gate waits with its own timer, so a caller that stubs ConfluenceClient.sleep to skip the per-request backoff is not affected.

Known limits

• A positive Retry-After is now honored by the whole client, as RFC 9110 describes, rather than only by the request that received it. If a server sends frequent short Retry-Afters unrelated to load, every one of them pauses all requests: with 5% such responses a 500-page traversal takes about 46 s against about 4 s on main at zero latency, and roughly 1.5-2x at 100-300 ms.
• 429s that do not depend on the request rate at all (a flaky proxy that answers 10% of requests with 429) leave the client spacing requests for a while after each one. 1000 pages at 10% take about 3 min against 13 s on main at zero latency (117 s against 24 s with 100 ms latency); at 5%, 44 s against 7 s (28 s against 18 s). At 20% main fails. A 503 flaky proxy is unaffected.
• Limits stricter than about 0.5 requests/second are not fully tamed: the spacing is capped at 2 s. main fails on those too.
• Retried requests re-enter the queue behind newer ones, so a throttled traversal is not strictly breadth-first.

Testing

• Gate unit tests: no waiting without throttling, shared pause only with a Retry-After, new vs stale, doubling and the cap, spacing between starts, expire, recovery and no ratchet from isolated throttles, the circuit (opens; stale rejections and successes keep it closed; bypasses waits while open; a pause requested by the opening throttle does not come back), and a sleep that returns early cannot loop.

• Traversal tests on the virtual-time server: the 40-page tree at 3 requests/second; a wider tree under 1 and 2 requests/second, a large burst with a low steady rate, a positive Retry-After, and none; no limit (nothing paced or delayed); a server that never lets a request through; fifty concurrent requests against a server that always refuses must fail within 20 s (with Retry-After: 60: within 4 minutes); one page that is always refused fails after the same four attempts while the rest is served; 503s are neither gated nor paced; isolated 429s do not leave the client throttled; concurrent requests share one pause.

• A sweep against main with a virtual clock and a custom axios adapter, 10 concurrent requests (the harness is in the review thread of this PR):

  Scenario	main	this PR
  3 req/s bucket, 1000 pages	fails	333 s (the limit alone needs 333 s)
  0.5 / 1 / 2 / 3 / 10 req/s, 100-1000 pages	fail except 10 req/s	all complete; 1.0x the limit's own time at 2 req/s and above, up to 1.55x at 1 req/s or less
  10 / 50 concurrent requests, server always refuses	7 s / 7 s	8.4 s / 7.1 s
  10 concurrent, always Retry-After: 60	180 s	180 s
  3% random 503 on reads, 2000 pages	8 s	8 s
  2 s outage, 500 pages	3 s	4.7 s
  single early Retry-After: 30	30 s	31 s
  no throttling, 1000 pages	0 s	0 s

• npm test (42 suites, 1766 tests) and eslint pass; the existing 20 rate-limit retry tests pass unmodified.

Not verified

The Data Center 9.2.9 numbers in #258 were not reproduced against a live server; everything here is checked against simulated servers. A real server's latency and bucket shape may differ, so this is Refs, not Fixes.

README gets one paragraph about the behavior next to the existing note on rate limits.

Refs #261

[github-actions]

🎉 This PR is included in version 2.27.4 🎉

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot 📦🚀