Skip to content

perf: mapLimit fail-fast orphans in-flight spine-part downloads — no cancellation, they race the next title and delay exit #32

Description

@JavaGT

Filed by the Account-2 read-only amplifier (2026-09-18, HEAD 246b710; coordinator + 3 read-only workers; static evidence only — no benchmark or script was run, per the read-only contract). Strong confidence; semantics verified by reading every file cited.

Finding

mapLimit is fail-fast: the first worker rejection sets failed = true so free workers stop taking items, but in-flight calls run to completion and their results are discarded — the contract is documented in the code itself (src/pool.mjs:13,22-35). downloadPart's worst case is 3 × 60 s idle-timeout attempts plus backoff ≈ ~180 s per part (src/download.mjs:24-26,101-108), and there is no AbortSignal / cancellation plumbing anywhere in src/ (rg over src/ + bin/: only per-request timeout req.destroy).

So when part k of a title fails terminally, the 1–2 sibling parts still in flight keep downloading — each for up to ~180 s of retry cycles — after the title's failure is already decided.

Impact

Decision question (implementation-ready shape, but adopt/decline first)

Thread a cancellation signal mapLimit → fn (for archive.mjs:98: downloadPart attempt loop → followRedirects), so fail-fast aborts siblings: mapLimit(items, limit, fn, { signal }) or a returned AbortController; downloadPart(part, dest, { signal }) checks between retries and destroys the in-flight request. Alternatively accept and document the bounded waste.

Evaluate whether the failure-path waste justifies the signal plumbing, or decline with reasons.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions