Add scoped email login to the external gateway

.github/workflows/ci.yml

@@ -330,8 +330,6 @@ jobs:
         run: docker build -f services/gateway/Dockerfile -t gateway:ci .
       - name: Smoke-test gateway image (imports incl. contracts/ resolve at boot)
         run: docker run --rm gateway:ci python -c "import src.api.app"
-      - name: Verify gateway against the current directory over HTTP
-        run: python3 services/gateway/tests/smoke_stack.py
       - name: Build discord-bot image
         run: docker build -t discord-bot:ci ./discord-bot
       - name: Smoke-test discord-bot image (source parses)
@@ -340,6 +338,8 @@ jobs:
         run: docker build -f services/verification/Dockerfile -t verification:ci .
       - name: Smoke-test verification image (imports incl. contracts/ resolve at boot)
         run: docker run --rm verification:ci python -c "import src.api.app"
+      - name: Verify gateway against directory and verification over HTTP
+        run: python3 services/gateway/tests/smoke_stack.py
       - name: Build meeting image
         run: docker build -f services/meeting/Dockerfile -t meeting:ci .
       - name: Smoke-test meeting image (imports incl. PyAV/ffmpeg resolve at boot)

AGENTS.md

@@ -59,7 +59,7 @@ Violating any of these is a bug even if tests pass.
 - **`down_revision` points at the current head**, and `downgrade()` must actually reverse `upgrade()` except for the explicit retirement exceptions below. Verify the round trip: `upgrade head` → `downgrade -1` → `upgrade head`.
 - **Retirement exceptions:** documentation-system [009](services/documentation-system/docs/DEPLOYMENT.md#registry-cleanup-migration-009) and team-tracking [011](services/team-tracking/docs/DEPLOYMENT.md#registry-cleanup-migration-011) intentionally preserve their data cleanup on downgrade; only the registry entries are restored. Follow the linked backup/recovery guidance and verify the round trip with existing rows, checking that unrelated data survives. These exceptions do not waive reversibility for other migrations.
 - **New columns on existing tables are nullable or have a `server_default`**, mirrored in the Pydantic model and both adapters.
-- Current heads: team-tracking **011**, documentation-system **018**, verification **001**, gateway **001**. `llm`, `meeting`, `connectors` have no schema. (Verify before relying on these — `ls services/<service>/migrations/versions/` is authoritative, this line is not.)
+- Current heads: team-tracking **011**, documentation-system **018**, verification **001**, gateway **002**. `llm`, `meeting`, `connectors` have no schema. (Verify before relying on these — `ls services/<service>/migrations/versions/` is authoritative, this line is not.)
 
 ### Configuration
 
@@ -124,7 +124,9 @@ Run what CI runs for that service — `.github/workflows/ci.yml` is authoritativ
 - **Cross-worktree tests need source-path verification.** Pytest can prepend a historical tests checkout to `sys.path` and import its `contracts` instead of the edited code. Pin the target `src` and `contracts` packages before collection and verify imported modules' `__file__` paths; passing against the wrong checkout is not verification.
 - **`uv --project` is mandatory for the key CLIs.** Both `services/team-tracking` and `services/documentation-system` declare a top-level `src` package with a console script at `src.cli:main`. In the shared workspace venv they collide, so a bare `team-tracking-keys …` can resolve documentation-system's CLI and mint a `doc_`-envelope key that team-tracking rejects. Always `uv --project services/<service> run <service>-keys …`, and verify the token's prefix. The same applies to every other team-tracking console script (`team-tracking-seed`, `team-tracking-groups`).
 - **Gateway provisioning must pin the import path too.** After installing all workspace members, `uv --project` alone can still import another service's `src.cli`. From the repo root use `uv --directory services/gateway --project . run python -m src.cli ...` (and the team-tracking directory for its internal key). The isolated containers can use their installed key CLI directly.
-- **Gateway is the public entry point.** It accepts only issued `gw_` keys, has no env-bootstrap `API_KEY`, and owns only its key database. Compute-platform data access must use explicit scoped HTTP operations; see `services/gateway/docs/ARCHITECTURE.md`. Run one replica/worker for its process-local rate limits.
+- **Gateway is the public entry point.** It accepts only issued `gw_` keys, has no env-bootstrap `API_KEY`, and owns its key registry plus durable login challenges/abuse counters. Compute-platform data access must use explicit scoped HTTP operations; see `services/gateway/docs/ARCHITECTURE.md`. Run one replica/worker for its process-local rate limits. Size `LOGIN_STATUS_PER_MINUTE` and `EDGE_REQUESTS_PER_MINUTE` together. Only verification’s explicit pre-send cooldown permits a bounded delivery retry; never retry an ambiguous send.
+- **Gateway email login stays on Misty.** Verification owns code generation/send/check; gateway owns eligibility, consumer-bound challenges, durable quotas, and atomic single-use redemption. Preserve Discord’s existing correct-code replay. The compute backend attests `client_ip` from trusted ingress and must bind challenges to browser sessions; it never gets a verification key. See `services/gateway/docs/API.md#email-login`.
+- **Misty login issuers are environment-bound.** Successful auth responses carry `misty:<GATEWAY_ENV>`. Compute verifies and pins issuer plus gateway origin to its database; existing compute accounts need explicit UUID mappings. Do not restore email-only account claiming or a shared mailbox guess lockout; the gateway uses three guesses per independent challenge and five deliveries per mailbox window.
 - **Gateway proxy trust is specific to `X-Real-IP`.** Railway's public edge overwrites it; `X-Forwarded-For` may include a shared proxy hop and is ignored. Health probes skip the IP quota, so preserve the per-app readiness cache and nonblocking probe lock; see `services/gateway/docs/API.md#health`.
 - **documentation-system and verification both bind host port 5434.** They cannot run locally at the same time. Remap one to an unused port (gateway uses 5435 by default) and update its `DATABASE_URL`. If Alembic reports an unknown revision, you're almost certainly pointed at the other service's database.
 - **Discovery imports; refresh repairs.** `doc-discover` enumerates new Google file identities over authenticated HTTP, with a persistent checkpoint volume and `discover` connector scope. It never grants access from listing metadata, reactivates retired docs, or refetches existing docs. See [discovery operations](services/documentation-system/docs/DISCOVERY.md); keep #292 refresh separate.

README.md

@@ -44,10 +44,10 @@ Root [`VERSION`](VERSION) identifies the platform release candidate; immutable
 | [team-tracking](services/team-tracking/README.md) | People, teams, memberships, identities, channel scopes and managed Google Groups | Directory for the bot, catalog and gateway; Google Groups sync requires its own OAuth setup. |
 | [documentation-system](services/documentation-system/README.md) | Catalog, indexed full text, authorized search, source grants and durable meeting records | Used by `/doc`, helper document questions, discovery/refresh jobs and the meeting hand-off. |
 | [llm](services/llm/README.md) | Stateless chat, client-executed tool transport and embeddings | Bot helper, meeting minutes/Q&A and catalog embeddings; providers configured per operation. |
-| [verification](services/verification/README.md) | Email ownership codes | The bot uses it for account linking and additional verified emails. |
+| [verification](services/verification/README.md) | Email ownership codes | Bot account linking and additional verified emails; gateway login uses a separate scoped `vf_` key in `CONSUMER_KEYS`. |
 | [meeting](services/meeting/README.md) | Live transcription, minutes/PDF and live Q&A | `/record` and `/qna`; completed text can be handed to the catalog. One process owns live sessions. |
 | [connectors](services/connectors/README.md) | Google/GitHub fetch, Google discovery and meeting-Doc operations | Catalog consumer with separate read, discovery and writer scopes/credentials; no local database. |
-| [gateway](services/gateway/README.md) | Scoped public API backed by private services | `GET /v1/resolve/discord/{github_login}`; external consumers need an issued `gw_` key. |
+| [gateway](services/gateway/README.md) | Scoped public API backed by private services | Discord identity resolution and opt-in email login for active directory members; external consumers need an issued `gw_` key with the operation's scopes. |
 | [discord-bot](discord-bot/README.md) | Slash commands, mention/DM helper, voice recording and local web playground | Commands register globally. Live Q&A remains in progress; helper tools require explicit activation. |
 
 This is the implemented surface. Use the [deployment runbook](docs/RAILWAY-DEPLOYMENT.md)
@@ -58,7 +58,8 @@ tables. The catalog calls the directory for ownership and access context,
 connectors for source content/ACLs, and llm for embeddings. Meeting calls llm
 for minutes and hands saved text to the catalog, which uses the connector's
 Google writer. The bot calls the internal APIs; gateway exposes curated public
-operations backed by the directory. Ownership-label lookup can degrade during
+operations backed by the directory and email login backed by private verification.
+Ownership-label lookup can degrade during
 a directory outage, while authorized retrieval fails closed. Each of the four
 database-backed services owns its database; llm, meeting and connectors have none.
 
@@ -186,7 +187,7 @@ All seven services are built the same way on purpose — learning one gives you
 
 - **`contracts/` Protocol boundary.** Each service has a `contracts/` package of Pydantic domain types plus `Protocol` interfaces. Application code depends on the Protocols, never on a concrete implementation.
 - **Swappable storage adapters.** `InMemoryStorageAdapter` for fast tests, `PostgresStorageAdapter` for real runs — both satisfy the same Protocol. Tests use in-memory; a small integration test suite gates the Postgres adapter too.
-- **Scoped API-key auth.** Protected HTTP endpoints use `X-API-Key`; health and API schema/UI routes remain unauthenticated within the service’s network exposure. Keys are argon2-hashed with a set of per-resource scopes (`people:read`, `teams:write`, `chat`, `meetings`, `fetch`, etc.). This machinery is implemented once in the shared [`packages/auth`](packages/auth) (`platform_auth`) library and consumed by all seven services through a thin shim (`src/api/auth.py`, plus `hashing.py` for the services that mint their own keys) that binds its own key prefix and config. Anything the library exposes ready-to-use — `AuditLogMiddleware`, for one — is imported from `platform_auth` directly; a per-service file earns its place only by binding something. team-tracking, documentation-system, and gateway store consumer keys in their own `api_keys` tables; `llm`, `meeting`, and `connectors` seed them from a `CONSUMER_KEYS` JSON env var instead, so rotating one there is a redeploy.
+- **Scoped API-key auth.** Protected HTTP endpoints use `X-API-Key`; health and API schema/UI routes remain unauthenticated within the service’s network exposure. Keys are argon2-hashed with a set of per-resource scopes (`people:read`, `teams:write`, `chat`, `meetings`, `fetch`, etc.). This machinery is implemented once in the shared [`packages/auth`](packages/auth) (`platform_auth`) library and consumed by all seven services through a thin shim (`src/api/auth.py`, plus `hashing.py` for the services that mint their own keys) that binds its own key prefix and config. Anything the library exposes ready-to-use — `AuditLogMiddleware`, for one — is imported from `platform_auth` directly; a per-service file earns its place only by binding something. team-tracking, documentation-system, and gateway store consumer keys in their own `api_keys` tables; `llm`, `meeting`, `connectors`, and `verification` seed them from a `CONSUMER_KEYS` JSON env var instead, so rotating one there is a redeploy.
 - **Credentials are `SecretStr`, never `str`.** Every credential field in a service's `Settings` is `pydantic.SecretStr`, so it renders as `**********` in any repr, log line, traceback, or failing assertion diff — a plain `str` once printed a real Google private key into a transcript. Unwrap with `.get_secret_value()` at the boundary; `platform_auth` still takes plain `str`. See [`packages/auth/README.md` → Credential config convention](packages/auth/README.md#credential-config-convention) for the one way forgetting to unwrap fails _silently_ rather than loudly.
 - **Attested actor.** The `created_by`/`updated_by` on every audit field is the authenticated key's own name — a caller can't claim to be someone else.
 - **Per-request audit log.** Middleware emits one JSON line per request with the resolved actor, endpoint, status, and duration.

docs/ARCHITECTURE.md

@@ -219,19 +219,17 @@ describes how it applies them concretely.
   current heads rather than pinning a deployment to an old revision. For
   cleanup/rollback exceptions, follow the service deployment guide. `llm`,
   `meeting`, and `connectors` have no schema.
-- **Three auth storage models.** The convention is scoped, argon2-hashed keys
+- **Two auth storage models.** The convention is scoped, argon2-hashed keys
   behind `platform_auth`'s `ApiKeyStore` protocol — but where they *live* varies
   with how many consumers a service has:
   - **team-tracking, documentation-system, gateway** — separate `api_keys`
     tables, minted/revoked with each service’s CLI. Gateway accepts only issued
     `gw_` keys and has no inbound bootstrap env key.
-  - **`llm`, `meeting`, `connectors`** — no table. Keys are seeded at boot from
-    a `CONSUMER_KEYS` JSON array env var, so their "mint" CLIs only *print* a key
-    and its JSON entry. Adding or revoking one is a variable edit plus a
+  - **`llm`, `meeting`, `connectors`, `verification`** — no key table. Keys are seeded at boot from
+    a `CONSUMER_KEYS` JSON array env var, with argon2 hashes and scoped consumer identities. Services with "mint" CLIs
+    only print a key and its JSON entry; verification uses the shared generator. Adding or revoking one is a variable edit plus a
     redeploy, and a malformed `CONSUMER_KEYS` fails the deploy at boot by design.
-  - **`verification`** — a `NullApiKeyStore`: no per-consumer keys exist, and
-    only the bootstrap `API_KEY` env var authenticates. It has exactly one
-    consumer (the bot), so issuing keys would be ceremony without benefit.
+
 
   The protocol is what makes this vary cleanly — each service picks a store, and
   no route code changes.
@@ -258,8 +256,9 @@ callers go through a gateway at all.
   Callers outside the org's trust boundary (a GitHub Action, a future third-party
   integration) never get a team-tracking key. Instead they hold a key issued by the
   gateway's own external registry (`gateway-keys`, scoped e.g. `resolve:discord`), and
-  the gateway holds exactly **one** internal team-tracking key
-  (`identifiers:read`) for its own outbound calls. It composes and curates — it never
+  the gateway holds its own internal team-tracking key
+  (`identifiers:read`, plus `people:read` for login) and a separate private
+  verification key when email login is enabled. It composes and curates — it never
   passes an internal response straight through — and adds the protections an
   externally-facing surface needs that internal services don't: a public Railway
   domain, per-key rate limiting behind a per-IP flood guard, and audit logging of
@@ -275,7 +274,7 @@ callers go through a gateway at all.
   every caller must present an issued, scoped key. When adding an
   externally-reachable service, copy that, not the internal shim.
 
-The gateway's first (and so far only) endpoint is the resolver,
+The gateway's first endpoint is the resolver,
 `GET /v1/resolve/discord/{github_login}` — it turns a GitHub login into the Discord id
 linked to the same person in the directory, and nothing else. It is a working
 example for future gateway operations; no current integration consumes it.
@@ -284,10 +283,11 @@ but do not call the resolver or ping reviewers' Discord accounts. New external
 use cases get new narrow endpoints on the gateway, not broader access to
 team-tracking itself.
 
-The compute platform will use the same per-consumer gateway contract when its
-required operations are defined. The [gateway architecture](../services/gateway/docs/ARCHITECTURE.md#future-compute-platform-access)
-describes how to add access while preserving service-owned databases and access
-checks; the current permission grants only the Discord resolver.
+The compute platform can use the [email-login contract](../services/gateway/docs/API.md#email-login)
+with `auth:email` and `auth:status`. Misty handles eligibility, delivery, code
+verification, and abuse limits; compute owns its sessions and permissions. The
+[gateway architecture](../services/gateway/docs/ARCHITECTURE.md#future-compute-platform-access)
+describes adding further operations without sharing databases or internal keys.
 
 ## Why the directory is built first
 

docs/DEVELOPMENT.md

@@ -287,10 +287,10 @@ their code:
   seeded from a `CONSUMER_KEYS` JSON env var and parsed at boot into an
   in-memory store, not an `api_keys` table. Rotating a key means editing the
   variable and redeploying; there is no `revoke` command.
-- **`verification` has a database but no key table.** It uses a
-  `NullApiKeyStore`, so only the bootstrap `API_KEY` authenticates — there are no
-  per-consumer keys to mint. Its database retains verification request state
-  and code hashes; expiry does not delete rows.
+- **`verification` has a database but no key table.** Scoped `vf_` consumers
+  are configured in `CONSUMER_KEYS`; the bot bootstrap key remains supported.
+  Its database retains verification request state and code hashes; expiry does
+  not delete rows. Follow its [provisioning and retention guide](../services/verification/docs/DEPLOYMENT.md).
 - **`meeting` is stateful.** It holds one in-memory session per live meeting.
   It cannot be horizontally scaled without sticky routing on `session_id`, and
   a restart drops every in-flight meeting. See