feat(transport)!: page large results in parallel by offset; remove parallel_chunks

[thodson-usgs]

Summary

A multi-page Water Data result used to cost one round trip per page, in strict
sequence. It no longer does. Where the API honors offset, every page's URL is
computable up front, so the pages are fetched concurrently — at a fixed
limit the request count is essentially unchanged, only the timing. Since the
USGS quota is volume-based (x-ratelimit-limit, default 1000/hr), overlapping
pages costs no extra quota. See the caveat below on limit: the speedup is real
but it is not available at the default page size.

The same change removes parallel_chunks(n), which bought parallelism the
other way — splitting a request that already fit into more sub-requests. That
approach spent extra quota, and did nothing at all for a single-site query,
which has no multi-value axis to split. Overlapping the pages a request was
going to fetch anyway subsumes it.

Measured, fully warm, against the live API (single site, full daily history,
limit=2000):

Config	Wall clock	Speedup
API_USGS_CONCURRENT=1 (sequential cursor walk)	7.35 s	—
=4	3.49 s	2.1×
=16	3.08 s	2.4×
=32	2.70 s	2.7×

Important caveat, and the main thing to review. Those numbers use
limit=2000, passed explicitly. They do not describe default behavior.

The offset stride must equal limit (otherwise pages gap or overlap), and the
API's ceiling is offset <= 40000 — a ceiling in rows, not pages. The default
limit is 50,000. So at defaults the only legal offset is 0: the walk fetches
one page and hands the entire rest of the query to the sequential cursor tail.

Measured on mocks, a 200,000-row pull — how many requests get overlapped versus
walked one at a time:

limit	pages	overlapped (offset)	serial (cursor tail)
50,000 (default)	4	2	3
20,000	10	4	7
10,000	20	6	15
2,000	100	22	79

So this feature helps materially only when limit is well below 40,000, and a
smaller limit means more requests — which the volume-based quota does charge
for. The "no extra quota" claim is true at a fixed limit; it is not a claim
that shrinking limit to buy parallelism is free. Reviewers should decide
whether the getters ought to pick a paging-friendly limit themselves (e.g.
20,000, trading 10 requests for 4 overlapped instead of 2) rather than leaving
that to the caller. I did not make that change here — it alters default request
counts for every user, which is a policy call, not an implementation detail.

The single-site case is still the one with no prior answer: ChunkPlan provably
returns 1 sub-request for a single site regardless of parallel_chunks(n)
(verified at n=1, 8, and 32), so a single-site deep history was sequential and
no knob changed it.

Why cursors are the slow part

OGC API - Features Part 1 defines paging as a next link relation. The
link's target is opaque — Water Data returns ?cursor=<token> — so page N+1's
URL literally does not exist until page N has come back and been parsed:

GET items?limit=2000               ──► page 1 ──► parse ──► next=?cursor=abc
GET items?cursor=abc               ──► page 2 ──► parse ──► next=?cursor=def
GET items?cursor=def               ──► page 3 ──► ...

Ten pages, ten serialized round trips. Latency dominates: each page is a small
payload behind a ~0.5 s round trip.

offset breaks the chain. It is not in the OGC spec (more on that below), but
Water Data supports it, and with it the whole URL set is a closed-form
computation — offset = i × limit:

GET items?limit=2000&offset=0     ─┐
GET items?limit=2000&offset=2000  ─┤ all in flight at once
GET items?limit=2000&offset=4000  ─┤
GET items?limit=2000&offset=6000  ─┘

Same four requests. One round trip of wall clock instead of four.

How it knows when to stop

Here is the wrinkle: the walk does not know how many pages there are. OGC
API - Features Part 1 makes numberMatched optional — "each page may
include information about the number of selected and returned features" — and
Water Data omits it. A page carries numberReturned (its own row count) but no
total. So the page count cannot be computed in advance; it has to be probed.

That is why the walk fetches in waves, and why the wave width ramps —
1 request, then 2, then 4, doubling up to the API_USGS_CONCURRENT cap. A flat
wave at the default width would make a one-page result cost 32 requests to
discover it was already finished; since the quota is volume-based, that is a
straight 32x tax on exactly the queries with nothing to gain from parallelism.

The ramp gives three properties:

• a single-page result costs exactly one request — identical to the
  sequential walk, so small queries pay nothing for this feature;
• total requests stay under 2x the pages that actually exist, because with
  doubling every prior wave summed is less than the current one;
• round trips are logarithmic in the page count — a 10-page result is 4
  waves, not 10 serialized round trips.

Measured at the default width (_CONCURRENCY_DEFAULT = 32):

Result	Pages needed	Requests sent
5 rows (1 page)	1	1
4,000 rows (2 pages)	2	3
19,000 rows (10 pages)	10	15

Worked example: 25 rows, limit=10, width=4

wave 1 (width 1) ───────────────────────────────────────────────────
  offset=0    ──►  10 rows   (== limit → full page, more may follow)

wave 2 (width 2) ── both issued concurrently ───────────────────────
  offset=10   ──►  10 rows   (== limit → full page, more may follow)
  offset=20   ──►   5 rows   (<  limit → SHORT PAGE, this is the end)
                             ▲
                             └─ stop_index = 1 (within this wave)

_stop_index walks the wave in offset order and returns the index of the
page that ends the walk. Pages after it are dropped, which is what makes a
speculative overshoot harmless — an overshooting request was already paid for in
parallel with the pages that mattered, and its rows are never concatenated.

Result: 25 rows from 3 requests in 2 round trips. The ramp happens to land
exactly on the three pages that exist, so parallel paging here costs the same
requests the sequential walk would have spent — it just spends them in 2 round
trips instead of 3. tests/waterdata_offset_paging_test.py::test_page_count_is_not_inflated_by_parallelism
pins that count.

Returning an index rather than a bool is what makes the earliest terminal page
win: if a wave contains both a short page and a later page that also looks
terminal, the earlier one ends the walk and no page between them can resurrect
it.

The four stop conditions, in precedence order

_stop_index is the single source of truth; the module docstring documents the
same order.

1. A short page — fewer than limit rows. The last page by construction:
   the server had no more rows to give. This is the normal exit. The page is
   kept, rows included.

2. An empty page — zero rows. The previous page ended exactly on a limit
   boundary and this offset is past the end. Everything before it is kept; the
   empty page is not. (With 20 rows at limit=10, no page is ever short — the
   empty page at offset=20 is the only signal available.)

3. The row cap — max_rows is reached, so further pages would be discarded
   anyway. A wave can overshoot the cap, so the cap is re-applied to the
   combined frame (result.head(row_cap)) rather than to a wave boundary;
   otherwise max_rows=25 would return whatever a wave happened to land on.

4. The offset ceiling — Water Data rejects offset > 40000 with HTTP 400
   InvalidQuery. This is not an end-of-data signal, so it must not end the
   walk, or a deep pull would silently truncate. Instead the offsets stop and a
   sequential cursor continuation takes over. Offsets have a ceiling; cursors
   don't. The hybrid is fast over the parallelizable prefix and complete over
   the rest.

Waves continue until one of these fires, widening as long as every page comes
back full.

The ceiling seam needs a rewind

Worth spelling out, because the obvious implementation is subtly wrong — and it
was wrong here until an integration test caught it.

At the ceiling, the next offset the walk would need is by definition past the
ceiling. So it can't seed the continuation either — that request would earn the
same HTTP 400 the offset walk just avoided (offset=40010 > 40000). The walk
therefore rewinds one page: it drops the last page it fetched and re-seeds
the cursor walk at offsets[-1], the largest offset the service still accepts,
following next links from there. One page is re-fetched per deep query, in
exchange for a seam with neither a gap (missing rows) nor an overlap
(duplicates).

Risks, and what they cost

offset is a non-standard extension. Part 1 defines only limit and the
next relation. Worse, an unrecognized query parameter is conventionally
ignored rather than rejected — so a server that drops offset support would
answer every offset with page 1, and a naive walk would concatenate the same
rows N times and report success. Silent duplication is the worst failure
mode available to this design, so it is checked for directly: _offset_ignored
compares the first two full-length pages of the first wave, and identical frames
raise OffsetUnsupported. That happens before any rows are returned, so the
fallback re-walk cannot double-count. The query then completes via standard
next-link paging — slower, correct, and needing no extensions. A false
positive (two genuinely identical pages) costs a fallback, not an error.

The design does not assume the offset ceiling a priori in a way that can
break silently. max_offset is declared per service on OgcDialect, default
None, and None means "don't use offsets at all" — so the conservative path
is what an undeclared service gets. If USGS lowers the ceiling, requests past
the new limit fail loudly with the existing typed HTTP error rather than
truncating. If they raise or remove it, the current value just leaves some
speed on the table.

What could still regress: a server that honors offset inconsistently
across pages (rather than not at all) would slip past a check that samples the
first wave. Nothing in the API's behavior suggests that, and the alternative —
validating every page against its neighbors — would mean holding the whole
result to compare it. Flagging it as the known gap rather than papering over it.

What was removed

Removed	Why
parallel_chunks(n) (public, dataretrieval and waterdata)	Split a fitting request into more sub-requests. Quota-positive, and a no-op when there's no multi-value axis.
ChunkPlan.max_chunks	The dial's entry point into planning. Now a TypeError.
ChunkPlan._refine()	~65 lines of fan-out-driven splitting, with its own cap-overshoot arithmetic.

Byte-driven chunking is untouched and remains a correctness requirement —
the OGC edge WAF caps request bytes at ~8200 and returns HTTP 414 above it. Only
the parallelism half of the chunker is gone. tests/waterdata_chunking_test.py
now pins that split explicitly: test_byte_driven_chunking_survives_the_removal
and test_unchunkable_still_raised_without_the_dial.

Migration

# before
with waterdata.parallel_chunks(32):
    df, md = waterdata.get_daily(monitoring_location_id=sites, parameter_code="00060")

# after — delete the wrapper; the pages inside it are now overlapped
df, md = waterdata.get_daily(monitoring_location_id=sites, parameter_code="00060")

API_USGS_CONCURRENT now bounds the page-fetch wave width as well as
sub-request fan-out — one env var for everything in flight. API_USGS_CONCURRENT=1
pages strictly sequentially, via standard cursors (not offsets with a wave of
one). unbounded is clamped to a finite width here, because a wave is
speculative and an unbounded one would issue arbitrarily many past-the-end
requests to find a single short page.

Three bugs found while reviewing this, and fixed here

Worth reading, because each one was invisible to the tests as originally written
and each was found by probing the shipped defaults instead of the pinned test
values.

1. A one-page query cost 21 requests. The wave width started at the full
API_USGS_CONCURRENT (32), clipped to the ceiling — so a small query fired 21
requests to discover it was already finished, a 21x quota tax on exactly the
queries with nothing to gain. Nothing caught it because the conftest pins
API_USGS_CONCURRENT=1 and the new integration tests pinned 4; no test
exercised the default. Fixed by the ramp described above (1 request → 1). The
regression test deliberately deletes the env var rather than setting a number.

2. Peak in-flight requests exceeded the connection pool. Sub-requests are
gated by a semaphore sized to API_USGS_CONCURRENT, and the pool was sized to
match — but each sub-request now fans its own pages out, and those pages are
deliberately ungated (a sub-request holds its permit for its whole attempt, so
its own pages would deadlock waiting on it). Peak in-flight is the product:
measured 16 against a pool of 8. The excess queued inside httpx against the 60 s
pool-acquire timeout — the exact spurious-timeout failure ChunkedCall._run's
own docstring warns about. It surfaced in the live sweep as a mid-walk
ReadError and a resumable ServiceInterrupted at width=8. The pool is now
sized to the product.

3. A no-data query raised instead of returning an empty frame. The single
empty page was correctly discarded as past-the-end, which left the walk with no
response to report, so it fell through to its "issued no requests" guard and
raised DataRetrievalError. The sequential walk returns an empty frame for the
same query, and "a no-data result is not an error" is a documented promise of
the modern getters — so this was a behavior regression reachable by any typo'd
site id or unmeasured parameter code.

Testing

ruff check clean, ruff format --check clean, mypy dataretrieval/ clean on
43 files, pytest tests/ 676 passed.

• 11 new unit tests (tests/transport_test.py) for the service-neutral
  walk: each stop condition, no gap or overlap across waves, the ceiling
  hand-off, warn-and-truncate without a continuation, refusing a server that
  ignores offset, accepting distinct equal-length pages, offset clipping, and
  page-failure wrapping.
• 11 new end-to-end tests (tests/waterdata_offset_paging_test.py) through
  get_daily, fully mocked. These exist because conftest.py pins
  API_USGS_CONCURRENT=1 suite-wide, so the existing tests never touched the
  parallel path — the feature had zero end-to-end coverage. The module earned
  its place four times over: it caught the ceiling-seam bug (tail walk seeded one
  page past the ceiling, which would have failed a real deep pull with HTTP 400)
  plus the three bugs above. Several of these tests assert at the default
  width specifically, since that was the blind spot.
• 1 new chunker test pinning that the connection pool covers
  sub-requests × pages.

Disclosed as incomplete: a head-to-head of offset paging against the removed
_refine fan-out on a large real multi-site pull was started and never
finished — it ran into the 1,000-request anonymous quota (HTTP 429,
retry-after: 290). The two measurements in the table above are complete and
repeated; that third comparison is not, and is not being claimed.

Docs

README.md and docs/source/userguide/errors.rst replace their
parallel_chunks sections with "large downloads are paged in parallel
automatically", covering quota neutrality, the API_USGS_CONCURRENT=1 escape
hatch, and both fallbacks. ADR 0006 gains the second page-walk strategy — why
both live in transport, and why the strategy choice is a dialect decision rather
than a transport one. NEWS.md carries the breaking-change entry.

🤖 Generated with Claude Code