feat(config)!: resolve settings through a layered chain

.gitignore

@@ -118,3 +118,11 @@ ENV/
 
 # pyscn analysis reports (rebuildable: `pyscn analyze dataretrieval`)
 .pyscn/
+
+# Working design note for the layered-configuration work; the durable
+# record is ADR 0009 + docs/source/userguide/configuration.rst.
+CONFIG-PLAN.md
+
+# Resolver lock for local dev; the package ships a range-based pyproject and
+# is not deployed from a pinned set.
+uv.lock

.importlinter

@@ -21,7 +21,11 @@ type = layers
 containers =
     dataretrieval
 layers =
-    ngwmn | nldi | nwis | streamstats | waterdata | wateruse | wqp
+; The deprecated ``wateruse`` alias re-exports ``nwdc``, so it sits above the
+; adapters rather than beside them. A compatibility facade may depend on the
+; adapter it forwards to; nothing may depend on the facade.
+    wateruse
+    ngwmn | nldi | nwdc | nwis | streamstats | waterdata | wqp
     ogc
     utils
     _querying
@@ -32,6 +36,7 @@ layers =
     _wqx
     _ambient | _response_metadata | codes | combining | interruptions | rdb
     credentials
+    configuration
     exceptions
 ; Every top-level module must be placed in the stack deliberately. A new
 ; top-level module fails this contract until someone decides where it sits.
@@ -114,6 +119,7 @@ source_modules =
     dataretrieval.streamstats
     dataretrieval.transport
     dataretrieval.utils
+    dataretrieval.nwdc
     dataretrieval.waterdata
     dataretrieval.wateruse
     dataretrieval.wqp

AGENTS.md

@@ -6,7 +6,7 @@
 - Exclude `.claude/worktrees/` from searches and edits; it contains stale worktrees that pollute results.
 
 ## Example Notebooks
-- `demos/*.ipynb` — top-level Water Data tour: `USGS_WaterData_Introduction_Examples.ipynb` is the entry point; `_ContinuousData_`, `_DailyStatistics_`, `_DiscreteSamples_`, `_ReferenceLists_` cover individual collections; `WaterData_demo.ipynb`, `peak_streamflow_trends.ipynb`, `USGS_WaterUse_Examples.ipynb` (NWDC water-use data via `wateruse.get_wateruse`), and `R Python Vignette equivalents.ipynb` are standalone walkthroughs.
+- `demos/*.ipynb` — top-level Water Data tour: `USGS_WaterData_Introduction_Examples.ipynb` is the entry point; `_ContinuousData_`, `_DailyStatistics_`, `_DiscreteSamples_`, `_ReferenceLists_` cover individual collections; `WaterData_demo.ipynb`, `peak_streamflow_trends.ipynb`, `USGS_WaterUse_Examples.ipynb` (NWDC water-use data via `nwdc.get_wateruse`), and `R Python Vignette equivalents.ipynb` are standalone walkthroughs.
 - `demos/hydroshare/*.ipynb` — per-service HydroShare examples (NLDI, NWIS WaterUse, and Water Data DailyValues / GroundwaterLevels / Measurements / ParameterCodes / Peaks / Ratings / Samples / SiteInfo / SiteInventory / Statistics / UnitValues). Mirror these when adding examples for a new collection.
 - `demos/nwqn_data_pull/` — non-notebook example: a lithops/Docker batch pipeline (`retrieve_nwqn_samples.py`, `retrieve_nwqn_streamflow.py`) with its own `README.md`.
 - Any `Untitled*.ipynb`, `*_test.ipynb`, or notebooks not listed here are untracked local scratch; ignore them.

CONTEXT.md

@@ -71,8 +71,12 @@ statistics. The package's primary target.
 **NGWMN** — The National Ground-Water Monitoring Network, a distinct OGC API
 covering sites, water levels, lithology, well construction, and providers.
 
-**NWDC** — The National Water Availability Assessment Data Companion, providing
-modeled national-scale water-use data.
+**NWDC** — The National Water Availability Assessment Data Companion. Serves
+ten modeled national-scale datasets, of which the water-use models are five;
+the rest are hydrologic, atmospheric-forcing, and assessment outputs. The
+package reaches it through the `nwdc` adapter, named for the service like every
+other adapter. Legacy: that module was `wateruse`, which named one subset of
+what the service offers.
 
 **WQP** — The Water Quality Portal, a multi-agency water-quality clearinghouse.
 
@@ -105,6 +109,70 @@ term. Legacy: the deprecated NWIS getters and the WQP profiles call this a
 **Metadata** — The second half of every getter's return: the request URL, the
 elapsed time, and the response headers. Describes the *retrieval*, not the data.
 
+## Configuration
+
+**Configuration profile** — A named set of settings for one adapter, stored in
+the configuration file or built in code. **Configuration** is the short form.
+A profile is an *input* to resolution, never its result.
+
+**Default profile** — The profile an adapter uses when no other is selected:
+the `[<adapter>]` table's own keys. Always in effect. A **named profile**
+(`[<adapter>.bulk]`) is in effect only when a caller selects it, so adding one
+to a file never changes an existing script.
+
+**Effective configuration** — The resolved set of settings a call will use:
+what the chain produces after every profile, variable and default has been
+applied. Distinct from a configuration profile, which is one contribution to
+it. **Configure** is the verb for applying one.
+
+**Setting** — One named tunable the caller may adjust: the API key, the
+concurrency cap, the retry count, the progress line, the fan-out baseline. A
+setting means the same thing wherever it applies, but it does not apply
+everywhere: `concurrency` and `parallel_chunks` are meaningless to an adapter
+that issues one request, and `ssl_check` is meaningful to only three. Which
+settings an adapter accepts is part of that adapter's vocabulary.
+
+**Package-wide setting** — A setting that applies to every adapter: the retry
+count, the progress line, the stall timeout. Set once, honored everywhere.
+
+**Adapter-scoped setting** — A setting named under one adapter, applying to
+that adapter and no other. It overrides the package-wide value for that adapter
+alone; it does not replace the package-wide tier. An adapter rejects a setting
+it has no use for, rather than accepting and ignoring it.
+
+The scope is the *adapter*, not the service and not the host, because the
+adapter is what owns the conventions being tuned. The API key is the
+counter-example that fixes the distinction: it belongs to the gateway fronting
+a host, so Water Data and NGWMN — two adapters, one host — necessarily share
+one key and one quota pool. Credentials are host-scoped; tunables are
+adapter-scoped.
+
+**Source** — Where a setting's value came from. Sources are ordered, and the
+order is resolved per setting rather than per source: a value supplied for one
+setting does not displace another setting's value from a lower source.
+
+**Selection** — Naming which profile an adapter should use. Done in code; a
+profile is never selected by the environment or implied by the file, so the
+set of profiles in a file is inert until something asks for one.
+
+**Built-in default** — The value a setting takes when no source supplies one.
+Package-wide.
+
+**Adapter default** — The value a *particular adapter* prefers when no source
+supplies one, because that adapter warrants a different figure — NWDC asks for
+4 concurrent requests where the OGC getters take 32. Supplied by the adapter in
+code, not by the user. It replaces the built-in default for calls through that
+adapter and nothing else. A value from any source outranks it: an adapter able
+to override an explicit setting would make that setting a lie.
+
+Distinct from an **adapter-scoped setting**, which is the *user* naming a value
+for one adapter. Both narrow to a single adapter; only one of them is something
+the caller wrote.
+
+All three are called "the default" in casual use, and they are not the same
+value. Where the distinction matters — reporting what a call will actually use
+— say which one is meant.
+
 ## Boundaries
 
 **Adapter** — A module owning one service's conventions: its URLs, parameters,

CONTRIBUTING.md

@@ -115,9 +115,10 @@ about the upstream service rather than about this package.
 
 **Before adding a small helper, check whether a leaf already generalizes it.**
 This package keeps its general mechanisms in dependency-free leaves --
-`_ambient.Ambient` for scoped context values, `transport.retry._read_env_number`
-for `API_USGS_*` settings, `transport.links.resolve_next_url` for pagination
-cursors. Each of those has been re-implemented at least once by someone who
+`_ambient.Ambient` for scoped context values, `config` for every setting
+(`API_USGS_*`, the config file, and `configure()` blocks all resolve through
+it, and it is the only module that reads the environment for one),
+`transport.links.resolve_next_url` for pagination cursors. Each of those has been re-implemented at least once by someone who
 did not know it was there, and the copies drift: the same question gets a
 different cycle guard, a different error message, a different edge case. None
 of the automated checks catch it, because two eight-line helpers are below the

NEWS.md

@@ -1,3 +1,7 @@
+**08/11/2026:** Settings resolve through a layered chain instead of the environment alone, and a *configuration profile* is a named set of settings for **one adapter**. The new `dataretrieval.configuration` module resolves every setting in one order, highest first: a configuration passed to an active `dataretrieval.configure(...)` block, a profile that block selected, the setting's `API_USGS_*` environment variable, the adapter's `[<adapter>]` table in `~/.dataretrieval/config.toml` (or `DATARETRIEVAL_CONFIG`), the file's top-level keys, the adapter's own built-in preference, then the package default. Precedence applies **per setting**, so a file that sets only `concurrency` leaves an environment `API_USGS_PAT` in effect, and a `[ngwmn]` table still inherits every top-level key it does not name. `configure()` takes configuration objects positionally, at most one per adapter and nothing else: `configure(Configuration(api_key=vault.read("usgs/pat")), WaterdataConfiguration.load("bulk"), NgwmnConfiguration(concurrency=4))`. The adapter a configuration targets is a property of its class, so a caller never restates it, and each adapter owns its class in the module that *reads* those settings (`waterdata.WaterdataConfiguration`, `ngwmn.NgwmnConfiguration`, `nwdc.NwdcConfiguration`, `wqp.WqpConfiguration`, `nldi.NldiConfiguration`, `streamstats.StreamstatsConfiguration`) — an adapter accepts only the settings it reads, so `[streamstats] parallel_chunks = 8` is an error rather than a line that quietly does nothing. The block is delivered through a `ContextVar`, so a credential set inside it cannot leak across threads or asyncio tasks, which is what makes it safe for a server or notebook handling several users' keys and is the thing assigning to `os.environ` could never do (issue #352). The file gains named profiles beside each adapter's default profile: `[waterdata]` is always in effect, `[waterdata.bulk]` only when a caller selects it with `WaterdataConfiguration.load("bulk")`, and a selected profile still inherits the default profile and the package-wide keys per setting. A profile named in code outranks the setting's environment variable — the one place the ladder inverts the environment-above-file rule, because losing a deliberate selection to a stale shell export is what a caller would file a bug about. An adapter's configuration may also carry a `base_url`, which redirects that adapter's requests for the duration of the block — a staging instance, a mirror, a recording proxy — and no other adapter's; for Water Data one value moves the OGC collections, the Samples database, the statistics service and the STAC catalog together. It is settable in a `configure()` block only: a `base_url` key in the file and an exported `API_USGS_BASE_URL` each raise rather than being read, since a redirect a config file or a shell profile can set is one no reader of the script can see. The API key does not follow a redirect — it is scoped to the single host that honors it — and is deliberately not per-adapter: it authenticates to the gateway fronting a host, and Water Data and NGWMN share that host, one key, and one hourly quota. `dataretrieval.show_configuration()` reports each setting's effective value and where it came from, naming the profile behind each value (`configure() block [waterdata.bulk]` rather than a bare block), listing the profiles a file defines whether or not this run selected any, and naming any adapter this process has not imported rather than omitting it — without ever printing the key. One parser per setting owns its grammar, so a value means the same thing whichever source wrote it. **Breaking change:** `RetryPolicy.from_env()` is now `RetryPolicy.from_configuration()` and resolves through the whole chain rather than the environment alone. **Behavior change:** a credential-shaped keyword passed to a getter's `**kwargs` query passthrough — Water Data's `**queryables` and every WQP getter's search filters — now raises `TypeError` naming `configure(Configuration(api_key=...))` instead of putting a secret in a URL that clients, proxies and logs retain. The names refused are `api_key=`, `token=`, `x_api_key=`, `password=`, `auth=`, `pat=` and similar spellings; a filter the server actually defines is unaffected. **Bug fix:** `API_USGS_STALL_TIMEOUT` was read straight from `os.environ`, so it could not be set by a `configure()` block or the config file and never appeared in `show_configuration()`; it now resolves through the chain like every other setting. Rationale in ADRs 0009, 0010 and 0011; terms in `CONTEXT.md`.
+
+**08/11/2026:** `dataretrieval.wateruse` is now `dataretrieval.nwdc`. Every other adapter is named for the service it retrieves from — `ngwmn`, `nldi`, `wqp`, `streamstats`, `nwis` — and this one was named for one subset of what its service offers. The National Water Availability Assessment Data Companion serves ten modeled datasets; the water-use models are five of them, the rest being hydrologic, atmospheric-forcing, and assessment outputs (`GET https://api.water.usgs.gov/nwaa-data/models`). **Deprecation:** `dataretrieval.wateruse` still works and re-exports `dataretrieval.nwdc` unchanged, emitting a `DeprecationWarning` on import; it will be removed on or after 2027-08-11. The alias forwards rather than copies, so `wateruse.get_wateruse is nwdc.get_wateruse` — monkeypatching or identity comparison through either spelling behaves the same. `import dataretrieval` stays silent: the package imports `nwdc` directly, so only code naming `wateruse` itself sees the warning. Function and constant names are unchanged (`get_wateruse`, `MODELS`, `WATERUSE_URL`, `DEFAULT_CONCURRENT_REQUESTS`). Terms are defined in `CONTEXT.md`.
+
 **08/09/2026:** `waterdata.get_cql` takes `collection` rather than `service`. OGC API - Features (17-069r4) normatively names this value the `collectionId`: Requirement 20 fixes the path template `/collections/{collectionId}/items`, and Requirement 18 defines `collectionId` as each `id` in the collections response -- which is literally how the package builds the URL, and what the live API returns. *Service* names the API itself (Water Data, NGWMN). **Deprecation:** `service=` still works and resolves to `collection`, with a `DeprecationWarning`; it will be removed on or after 2027-08-09. Positional callers (`get_cql("daily", cql)`) are unaffected. The `WATERDATA_SERVICES` type alias is now `WATERDATA_COLLECTIONS`, with `WATERDATA_SERVICES` retained as a permanent alias for the same object. Terms are defined in `CONTEXT.md`.
 
 **08/09/2026:** Every retrieval path now runs through one executor. `waterdata.get_cql` (via the OGC `fetch_ogc_request`) and `waterdata.get_stats_por` / `get_stats_date_range` (via the Statistics page walk) previously bypassed `dataretrieval.transport.fanout.FanOut` through a private sync bridge, which meant they were the only getters in the package with **no retry**: a mid-page-walk 429 or 503 failed the whole call while every typed getter and Water Use rode it out. Both now run as a one-item fan-out and the 25-line `transport/sync.py` is gone. **Behavior change:** those three getters now retry transient failures (`API_USGS_RETRIES`, default 4) and, when the retries are exhausted, raise the resumable `ServiceInterrupted` / `QuotaExhausted` rather than `ServiceUnavailable` / `RateLimited` / `NetworkError` — all remain `DataRetrievalError`, so broad handlers are unaffected, but narrow handlers around those calls must widen, and `.call.resume()` is now available on the interruption. A failure that retrying cannot fix (bad scheme, a hostname that does not resolve) still surfaces as `NetworkError` immediately. The progress line moved with it: `FanOut.resume()` opens the reporter it ticks into, so a driver can no longer run the shared executor and silently print nothing, and a `.call.resume()` fired long after the interruption now reports progress instead of running mute. Internal tidying with no public effect: the WQX3 / legacy-WQP CSV datetime shaping moved out of `dataretrieval.utils` (whose docstring reserves it for non-service-specific shaping) into the `dataretrieval._wqx` leaf; the five Water Data endpoint URLs are declared once in `dataretrieval.waterdata.endpoints` instead of being derived in three modules; the OGC queryables document is parsed by `dataretrieval.ogc.schema` so every OGC adapter can offer the table, with `waterdata.get_queryables` unchanged as its documented wrapper; and `ogc/engine.py` imports each symbol from the module that defines it.