Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
104 changes: 98 additions & 6 deletions docs/concurrency-queue.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,9 @@

[← Docs index](./README.md)

Limit how many requests are in flight at once (useful against rate limits):
A browser or server process firing dozens of requests at once can overwhelm a
backend or trip its rate limiter. The concurrency queue caps how many
requests are **in flight simultaneously**; the rest wait their turn.

```ts
http: {
Expand All @@ -14,19 +16,109 @@ http: {
}
```

Requests beyond the limit wait their turn. Aborting a queued (not-yet-started)
request removes it from the queue and rejects it.
## How it actually schedules (verified against `utilities/queue.ts`)

`createQueue({ concurrency, priority })` keeps a `waiting` array and a
`running` counter. Whenever a slot frees up, `dispatch()` pulls the next item:

```ts
const item = priority === 'lifo' ? waiting.pop() : waiting.shift();
```

- **`'fifo'` (default):** `waiting.shift()` — strict first-in-first-out; the
request that queued earliest runs next once a slot opens.
- **`'lifo'`:** `waiting.pop()` — the *most recently* queued request runs
next, ahead of ones that have been waiting longer. Useful when only the
newest request's result matters (e.g. a fast-typing search box) and older
queued ones are effectively stale.
- There is **no separate priority level or per-request priority field** —
`priority` is a single queue-wide scheduling mode, not a per-call priority
you can attach to individual requests. All items in the queue are peers;
ordering is purely FIFO or LIFO.
- Aborting a queued (not-yet-started) request via its `AbortSignal` removes it
from `waiting` and rejects it immediately with an `AbortError` — it never
occupies a concurrency slot.
- A per-call `queue: false` bypasses the queue entirely for that request
(`createClient.ts`'s `queueForThisCall = resolved.queue ?? queueEnabled`);
it runs immediately regardless of how many other requests are in flight.

## Per-module vs global queue

The queue is created **once** per client and shared across all modules and
even non-HTTP module calls (`createClient.ts`: "Shares the client's queue +
deduplicator so non-HTTP module work coordinates with HTTP requests") — there
is no separate queue instance per module. To give one module a different
effective concurrency, use the per-call/per-method escape hatch instead of
expecting a second global queue:

```ts
// Global queue: at most 6 concurrent requests across the WHOLE client.
const api = createClient({
baseURL,
http: { queue: { concurrency: 6 } },
openapi: { mode: 'runtime' },
})

// Opt a specific, latency-sensitive call OUT of the shared queue so it never
// waits behind bulk/background traffic:
await api.search.query({ q }, undefined, { queue: false })
```

## Worked example: burst throttling against a rate-limited backend

If a backend enforces, say, 5 concurrent connections per client, set
`concurrency` to match (or slightly under) that limit so requests queue
client-side instead of getting rejected server-side:

```ts
const api = createClient({
baseURL: 'https://api.example.com',
http: {
queue: { enabled: true, concurrency: 5, priority: 'fifo' },
retry: { attempts: 3, backoff: 'exponential', baseDelay: 500 }, // see retries.md
},
openapi: { mode: 'runtime' },
})

// Firing 50 calls only ever runs 5 at a time; the rest wait in FIFO order.
await Promise.all(items.map((item) => api.items.sync(item)))
```

Combine with [deduplication](./deduplication.md) so identical calls inside
that burst don't each consume a separate queue slot — dedup coalesces them
into one before the queue/network layer ever sees more than one request for
the same identity.

**See it live:** the example configures `http.queue.concurrency: 6` —
[`examples/react-vite/src/lib/api/api.config.ts`](../examples/react-vite/src/lib/api/api.config.ts).
Combined with [deduplication](./deduplication.md), the Feature Lab's "Deduplication
(6→1)" burst demonstrates how concurrent traffic is managed.
Combined with deduplication, the Feature Lab's "Deduplication (6→1)" burst
demonstrates how concurrent traffic is managed.

## Advanced: standalone queue utility

```ts
import { createQueue } from '@developerehsan/api-client'

const queue = createQueue({ concurrency: 3, priority: 'fifo' })
await queue.add(() => doSomeWork(), { signal: controller.signal })
queue.size() // tasks waiting, not yet started
queue.active() // tasks currently running
```

See the [API reference](./api-reference.md#standalone-utilities).
</content>

## Gotchas / troubleshooting

- **"Setting `priority: 'lifo'` didn't change which request finished first,
only which one started next."** Correct — LIFO changes *dispatch order out
of the waiting list*, not execution speed. Once running, requests still race
independently; LIFO only affects which queued item is pulled next when a
slot frees up.
- **"My per-module concurrency setting seems to affect other modules too."**
Expected — there's one shared queue for the whole client, not one per
module. Use per-call `{ queue: false }` for one-off exceptions instead.
- **"Queue never processes my request."** Check `http.queue.enabled` isn't
`false` at any layer and that you didn't pass an already-aborted `signal`.
- Related: [deduplication](./deduplication.md) for the layer checked right
after the queue in the pipeline, [retries](./retries.md) for what happens
when a queued-and-dispatched request fails.
86 changes: 77 additions & 9 deletions docs/deduplication.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,30 +2,98 @@

[← Docs index](./README.md)

Identical in-flight requests are collapsed into **one** network call; every
caller receives the same result (or the same error). On by default for `GET`.
When several callers ask for the same thing at the same time — two components
mounting and both requesting the current user, a burst of retries hitting the
same endpoint — deduplication collapses the identical **in-flight** requests
into **one** network call; every caller receives the same result (or the same
error). On by default for `GET`.

```ts
http: {
deduplication: true, // default
dedupeMethod: ['GET'], // add 'POST' etc. to dedupe those too
}
```

## Pipeline order: cache is checked first

The client's `run` closure resolves a response in this order (verified against
`packages/core/src/factory/createClient.ts`):

1. **Cache lookup** (`produce()`, `createClient.ts:989-991`) — for a `GET`
with cache enabled, the cache store is consulted *before* the queue or
dedup ever run. A cache hit (under `cache-first`, the default strategy)
returns immediately — no queueing, no dedup, no network call.
2. Only on a cache **miss** (or under `strategy: 'network-first'`) does the
request reach `fetchThrough()` (`createClient.ts:930-984`), which wraps the
network call as `withQueue(() => withDedup(async () => { ...runNetwork() })`
— i.e. **queue, then dedup**, then the actual dispatch/retry/validate cycle.
3. A successful network response is written back to the cache
(write-through) before being returned.

So the full order is **cache → queue → dedup → dispatch(retry) → validate →
cache write-through**. This resolves an apparent conflict between two
descriptions elsewhere: cache genuinely sits in front of queue/dedup for a
single request, but because concurrent *cache misses* all fall through to the
same `fetchThrough()` call, they still coalesce into one shared in-flight
network request via dedup — the cache is only checked once per call, not
re-checked while waiting on that shared promise.

Dedup keys include the **auth fingerprint** and **tenant**, so requests with
different credentials or tenants are never merged (see
[authentication](./authentication.md#auth--cachededup-safety)).

## Custom dedup key via `keyResolver` interaction

Dedup itself doesn't take a custom-key option — it derives its key from
method + identity URL + body + tenant + auth fingerprint
(`computeDedupeKey`). If you need requests with different query params that
are *conceptually* the same call to dedup together, normalize the URL/args
before calling (e.g. sort query params in the descriptor), rather than trying
to override the dedup key directly — there's no separate dedup `keyResolver`
the way `cache.keyResolver` exists for cache keys (see
[caching](./caching.md)).

## Per-call opt-out

// Opt out for a single call:
```ts
await api.users.get('42', undefined, { skipDedup: true })
```

Dedup keys include the **auth fingerprint** and **tenant**, so requests with
different credentials are never merged.
Use this when you deliberately want two calls that look identical to hit the
network independently (e.g. a manual "retry" button that shouldn't just
attach to the original in-flight promise).

## Opting additional methods in

```ts
http: {
deduplication: true,
dedupeMethod: ['GET', 'POST'], // now identical POSTs also coalesce
}
```

Deduplication happens *after* the [cache](./caching.md) lookup, so concurrent
cache misses still coalesce into a single network round-trip and then populate
the cache once.
Only opt a `POST`/`PATCH`/etc. into dedup when it's genuinely idempotent from
the caller's point of view — two logically-identical mutating calls sharing
one in-flight promise means the second caller never actually re-triggers the
side effect, it just observes the first one's result.

## See it live

The Feature Lab "Deduplication (6→1)" button fires **six** identical requests at
once and the live pipeline log shows only **one** `→ request` line — the other
five shared it:
[`examples/react-vite/src/features/FeatureLab.tsx`](../examples/react-vite/src/features/FeatureLab.tsx).
</content>

## Gotchas / troubleshooting

- **"My dedup isn't merging requests."** Check `dedupeMethod` includes your
HTTP method, and that neither call passed `skipDedup: true`. Also confirm
both calls resolve to the same auth fingerprint — see
[authentication](./authentication.md#auth--cachededup-safety).
- **"Two different users are seeing merged data."** Shouldn't happen — dedup
keys always include the auth fingerprint and tenant. If it does, check that
your `getToken`/tenant resolver isn't returning a shared/incorrect value.
- Related: [caching](./caching.md) for the layer checked before dedup,
[concurrency queue](./concurrency-queue.md) for the layer checked between
cache and dedup.
105 changes: 103 additions & 2 deletions docs/environments.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,13 @@

[← Docs index](./README.md)

An app typically talks to different hosts across dev/staging/prod, and the
same client code may run in different JS runtimes (browser, Node, edge) that
support different adapters. `environments` handles the first; runtime
detection (automatic, no config) handles the second.

## Configuring named environments

```ts
createClient({
environments: {
Expand All @@ -21,11 +28,105 @@ api.setEnvironment('staging')
time (fail fast).
- A module can target a different host with `config.baseURL` — see
[modules & methods](./modules-and-methods.md#module-level-configuration).
- Switching environments **clears the cache**, so stale data from one
- `setEnvironment(name)` throws a `ConfigurationError` if `name` isn't a key in
`environments` (`factory/createClient.ts`'s `setEnvironment`), and on success
it clears **the entire cache** (`cacheStore.clear()`) so stale data from one
environment never leaks into another.

## Runtime detection: `DetectedEnvironment` / `PlatformCapabilities`

Separately from the `environments` map, the client detects *what JS runtime
it's executing in* (`packages/core/src/environment/detect.ts`,
`detectEnvironment()`) and uses that to pick an HTTP adapter and decide
whether tenant-context propagation via `AsyncLocalStorage` is available. The
shapes, verified against `packages/core/src/types/environment.types.ts`:

```ts
type Environment = 'browser' | 'node' | 'edge' | 'nextjs-server' | 'nextjs-client';

interface PlatformCapabilities {
supportsAxios: boolean; // false on edge runtimes
supportsAsyncLocalStorage: boolean; // Node-like server runtimes only
hasDom: boolean; // window + document present
hasFetch: boolean; // global fetch available
hasVisibilityApi: boolean; // document present (visibilitychange)
}

interface DetectedEnvironment {
environment: Environment;
capabilities: PlatformCapabilities;
}
```

| Field | Type | Meaning |
| --- | --- | --- |
| `environment` | `Environment` | Which of the five runtimes was detected |
| `capabilities.supportsAxios` | `boolean` | Whether the axios adapter can be used |
| `capabilities.supportsAsyncLocalStorage` | `boolean` | Whether Node's `AsyncLocalStorage` is usable |
| `capabilities.hasDom` | `boolean` | Whether `window`/`document` are present |
| `capabilities.hasFetch` | `boolean` | Whether global `fetch` exists |
| `capabilities.hasVisibilityApi` | `boolean` | Whether `visibilitychange` events are available |

Detection order (from `detectEnvironment`'s own doc comment): **edge → browser
(DOM present) → node → nextjs-server**. It never throws, and the result is
memoized at the module level for the life of the process.

## Edge downgrade example

If you explicitly request the axios adapter but the detected runtime is edge
(`capabilities.supportsAxios === false`), the client silently downgrades to
`fetch` rather than failing (`environment/edgeSafe.ts`):

```ts
createClient({
baseURL: 'https://api.example.com',
http: { adapter: 'axios' }, // requested...
openapi: { mode: 'runtime' },
})
// ...but on Cloudflare Workers / Vercel Edge, this transparently becomes
// the fetch adapter — no error, no config change needed on your part.
```

This is why the CLAUDE.md architecture note describes adapter choice as
"`fetch` or `axios`, chosen by `environment/` detection (edge downgrades
axios→fetch)" — there's no separate edge-specific config; it's automatic
based on `PlatformCapabilities.supportsAxios`.

## Manual `setEnvironment` override

Useful for an admin/debug panel that lets an operator point the same running
app at a different backend without a reload:

```ts
const api = createClient({
environments: { dev: 'http://localhost:3000', prod: 'https://api.example.com' },
activeEnvironment: 'dev',
openapi: { mode: 'runtime' },
})

function switchToProd() {
api.setEnvironment('prod') // throws ConfigurationError if 'prod' isn't in the map
// any cached responses from 'dev' are now gone — the next call re-fetches
}
```

**See it live:** the Feature Lab "Environments" button calls
`api.setEnvironment(...)` to switch the active base URL at runtime and shows the
resolved config change —
[`examples/react-vite/src/features/FeatureLab.tsx`](../examples/react-vite/src/features/FeatureLab.tsx).
</content>

## Gotchas / troubleshooting

- **"`setEnvironment` throws immediately."** The name must already be a key in
the `environments` map passed to `createClient` — it can't introduce a new
host on the fly.
- **"All my cached data vanished after switching environments."** Expected —
`setEnvironment` clears the whole cache (`cache.clear()` semantics), same as
[logout](./authentication.md#logout-clearing-the-cache), to prevent
cross-environment data leaking.
- **"Axios options I set are being ignored on edge."** Check
`detectEnvironment().capabilities.supportsAxios` — on edge runtimes it's
always `false` and the fetch adapter is used regardless of `http.adapter`.
- Related: [modules & methods](./modules-and-methods.md#module-level-configuration)
for per-module `baseURL` overrides, [caching](./caching.md) for what "clears
the cache" actually clears.
Loading
Loading