Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -187,7 +187,7 @@ non-fork application matrix and remediation gates.
1. Matching precedence is exact `(identity_provider, subject)`, then verified
email, then explicit operator link.
2. Unverified email never authorizes linking or merge.
3. Merge and SCIM replacement share one user-operation lock.
3. Merge, SCIM replacement, and `PATCH active=false` deprovisioning share one user-operation lock.
4. Merged duplicates remain disabled tombstones with a survivor pointer.
5. Registration creates no password and rolls back if enrollment initialization
fails.
Expand Down
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,10 @@ Keep a Changelog, and releases use semantic versioning.
process-wide counter.
- Account merge and SCIM replacement now share the same user-operation lock
boundary.
- SCIM `PATCH active=false` and `DELETE` deprovisioning now share that lock
boundary with merge and replacement, return a root-level
`application/scim+json` error with retryable `503` on lock contention, and
have deterministic pre-mutation concurrency regressions.
- SQLite configuration and audit stores support safe multi-threaded access with
WAL mode and bounded busy timeouts.
- Application shutdown closes Keycloak, audit, and configuration resources and
Expand Down
12 changes: 11 additions & 1 deletion docs/OPERABILITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,17 @@ Mapper unit tests alone do not prove Naruon product authorization readiness.

## Account merge recovery

Merge and SCIM full replacement (`PUT`) must hold the shared operation lock. Protected-main `PATCH active=false` is not currently inside that shared-lock guarantee and must not be treated as transactionally serialized with merge. On failure, classify whether state changed in Keycloak, Keyverse audit, linked identities, or tombstone status. Re-observe before retry. Never infer a retry is safe solely from the previous HTTP response. Preserve survivor and duplicate lineage in audit.
The active PR implementation makes merge, SCIM full replacement (`PUT`),
`PATCH active=false`, and `DELETE` hold the shared operation lock. Only SCIM
lock contention is translated at this boundary into a retryable `503` before
entering the mutation sequence. Merge retains its existing service error
contract; SCIM endpoints use the root-level RFC 7644 error envelope for
`_scim_error` failures, while `PUT` and `DELETE` retain their existing
mutation/status semantics. The protected-main promotion remains
an active-PR gate in `docs/TRACEABILITY.md`. On failure, classify whether state
changed in Keycloak, Keyverse audit, linked identities, or tombstone status.
Re-observe before retry. Never infer a retry is safe solely from the previous
HTTP response. Preserve survivor and duplicate lineage in audit.

## Desired-state recovery

Expand Down
8 changes: 5 additions & 3 deletions docs/PRD.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,9 @@ Its job is to let CWL products consume stable standards-based identity without e
- 100% production statement/branch/docstring quality gates and protected review/security workflows.
- an explicit per-RP Keyverse token-validation and downstream ABAC/RBAC acceptance boundary; application login alone is not authorization readiness.

The current SCIM `PATCH active=false` deprovisioning path is not protected by the shared cross-process user-operation lock used by merge and full replacement. It must not be represented as transactionally serialized with merge until a source change and concurrency regression prove that boundary.
Active PR #113 extends that lock boundary to SCIM `PATCH active=false`
deprovisioning. Its exact-head implementation evidence is not a protected-main
capability until the PR passes independent review and protected Checks.

## 3. Integrated protected-main changes

Expand Down Expand Up @@ -70,11 +72,11 @@ Keyverse SHALL support deployment-owned external SAML/OIDC and LDAP/AD onboardin

### PRD-FR-003 Account unification

Account matching and merge SHALL follow exact subject → verified email → explicit operator link precedence. Merge SHALL preserve a canonical survivor, disable/tombstone duplicates, retain auditable lineage, and coordinate full-replacement SCIM writes through the shared user-operation lock. Any additional SCIM read-modify-write operation may claim the same serialization guarantee only after it uses that lock and has a concurrency regression covering merge/tombstone interaction.
Account matching and merge SHALL follow exact subject → verified email → explicit operator link precedence. Merge SHALL preserve a canonical survivor, disable/tombstone duplicates, retain auditable lineage, and coordinate SCIM full-replacement and `PATCH active=false` writes through the shared user-operation lock. Any additional SCIM read-modify-write operation may claim the same serialization guarantee only after it uses that lock and has a concurrency regression covering merge/tombstone interaction.

### PRD-FR-004 SCIM

Inbound SCIM SHALL map authoritative enterprise lifecycle operations into Keycloak while preserving Keyverse merge/tombstone invariants and failing closed on unsafe identity ambiguity. Protected-main currently serializes merge with full SCIM `PUT` replacement; the `PATCH active=false` path is a narrower deprovisioning path and is not yet part of that shared-lock guarantee.
Inbound SCIM SHALL map authoritative enterprise lifecycle operations into Keycloak while preserving Keyverse merge/tombstone invariants and failing closed on unsafe identity ambiguity. Protected-main serializes merge with full SCIM `PUT` replacement. Active PR #113 extends that boundary to the supported `PATCH active=false` deprovisioning path; its lock contention is surfaced as retryable SCIM `503` only after protected promotion.

### PRD-FR-005 Relying-party lifecycle

Expand Down
4 changes: 2 additions & 2 deletions docs/TEST_STRATEGY.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ Skipped, cancelled, absent, stale, predecessor-head, synthetic-only, rate-limite
- explicit link behavior;
- survivor/duplicate tombstone semantics;
- merge idempotency and rollback;
- merge/SCIM shared lock concurrency.
- merge/SCIM PUT/PATCH shared lock concurrency and lock-timeout errors.

## SCIM tests

Expand Down Expand Up @@ -80,7 +80,7 @@ repository supplies its own exact token-validation and ABAC/RBAC evidence.

- PostgreSQL/KV migrations and rollback for Keyverse-owned records;
- configuration bootstrap and runtime store behavior;
- user-operation locks across processes;
- user-operation locks across processes, including merge/SCIM PATCH linearization;
- desired-state idempotency/concurrency;
- secret scanning and redacted logs;
- Compose health/readiness;
Expand Down
3 changes: 1 addition & 2 deletions docs/TRACEABILITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,7 @@
| LDAPS directory profile | LDAP RFC 4511–4515 + Keycloak component docs | directory preflight/reconciliation tests | implemented-main |
| secret-free RP desired state | OAuth/OIDC/PKCE/Keycloak client docs | RP preflight/reconciliation/integrity tests | implemented-main |
| RP audience/role/org/workspace mapper profile | OIDC/JWT audience + Keycloak mapper docs | PR #72 protected-main source/tests; downstream RP acceptance remains required | implemented-main |
| merge/SCIM PUT shared operation lock | concurrency/data-integrity decision; ADR-0006 | merge + full-replacement lock/concurrency tests | implemented-main |
| SCIM PATCH active=false shared-lock parity | ADR-0006 boundary | current PATCH source has no shared-lock proof | gap-not-claimed |
| merge/SCIM PUT/PATCH shared operation lock | concurrency/data-integrity decision; ADR-0006 | merge + full-replacement + active=false PATCH lock/concurrency tests | active-PR |
| intent before mutation, receipt after re-observation | desired-state/recovery decision | federation/directory/RP reconciliation tests | implemented-main |
| receipt bound to exact desired-state version/hash | threat/recovery contract; ERD | persistence/migration/idempotency evidence required | accepted-contract |
| remote-first deletion | consistency/recovery decision | delete/reconciliation tests | implemented-main |
Expand Down
2 changes: 1 addition & 1 deletion docs/TRD.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ Identity matching precedence is exact `(identity_provider, subject)` → verifie

## 5. Concurrency and transactions

User merge and SCIM full replacement (`PUT`) share one cross-process operation-lock boundary. Protected-main `PATCH active=false` is currently outside that shared-lock guarantee and must not be represented as serialized with merge. Any PATCH or future SCIM read-modify-write operation that can affect tombstone/survivor invariants must join the same lock boundary and add a concurrency regression before the stronger guarantee is promoted. Desired-state records and apply receipts require deterministic keys, transaction-safe update semantics, exact desired-version binding, and reconciliation after crash/retry. Remote deletion precedes local desired-state removal when local-first deletion could falsely report success.
User merge, SCIM full replacement (`PUT`), and supported `PATCH active=false` deprovisioning share one cross-process operation-lock boundary. Lock contention returns retryable SCIM `503` before the mutation sequence. Any future SCIM read-modify-write operation that can affect tombstone/survivor invariants must join the same lock boundary and add a concurrency regression before the stronger guarantee is promoted. Desired-state records and apply receipts require deterministic keys, transaction-safe update semantics, exact desired-version binding, and reconciliation after crash/retry. Remote deletion precedes local desired-state removal when local-first deletion could falsely report success.

## 6. Federation requirements

Expand Down
9 changes: 6 additions & 3 deletions docs/UML.md
Original file line number Diff line number Diff line change
Expand Up @@ -133,7 +133,7 @@ Unverified email cannot enter `candidate_link` by itself.
```mermaid
flowchart LR
PUT[SCIM full replacement PUT]
PATCH[SCIM PATCH active=false — current narrower path]
PATCH[SCIM PATCH active=false]
MERGE[Merge/link mutation]
LOCK[user_operation_lock_state]
USER[Keycloak user state]
Expand All @@ -142,11 +142,14 @@ flowchart LR
PUT --> LOCK
MERGE --> LOCK
LOCK --> USER
PATCH -. not currently in shared-lock guarantee .-> USER
PATCH --> LOCK
USER --> AUDIT
```

Protected `main` guarantees the shared cross-process lock for merge/link and full SCIM replacement. The current `PATCH active=false` path is explicitly not represented as serialized with merge. Extending that guarantee is a source-and-concurrency-test change, not a documentation relabel.
Protected `main` guarantees the shared cross-process lock for merge/link and
full SCIM replacement. Active PR #113 extends the boundary to the supported
`PATCH active=false` deprovisioning path; its retryable SCIM `503` behavior is
not a protected-main guarantee until exact-head review and Checks pass.

## Automation authority

Expand Down
16 changes: 7 additions & 9 deletions docs/adr/0006-user-operation-lock.md
Original file line number Diff line number Diff line change
@@ -1,23 +1,21 @@
# ADR-0006: Share one user-operation lock across merge and SCIM full replacement
# ADR-0006: Share one user-operation lock across merge and SCIM mutations

**Status:** Accepted
**Date:** 2026-08-09

## Context

Account merge/link operations and a SCIM full user replacement can target the same Keycloak user. The full replacement path reads tombstone state and then writes the user representation, so it must not race a merge that creates the tombstone between those operations.

Protected `main` also supports the narrower `PATCH active=false` deprovisioning path. That PATCH path currently performs its read/deactivate/read sequence outside the shared cross-process lock. This ADR therefore must not imply that every SCIM mutation is serialized with merge.
Account merge/link operations and SCIM user mutations can target the same Keycloak user. The replacement and deprovisioning paths read user state and then write it, so they must not race a merge that creates the tombstone between those operations.

## Decision

Keyverse uses one cross-process user-operation lock boundary for account merge/link and SCIM `PUT /Users/{id}` full replacement. Those operations serialize consistently, preserve tombstone/survivor invariants, and can be retried or recovered from observed durable state.
Keyverse uses one cross-process user-operation lock boundary for account merge/link, SCIM `PUT /Users/{id}` full replacement, SCIM `PATCH /Users/{id}` with `active=false`, and SCIM `DELETE /Users/{id}`. These operations serialize consistently, preserve tombstone/survivor invariants, and can be retried or recovered from observed durable state. A lock timeout returns retryable SCIM `503` without entering the mutation sequence.

The current SCIM `PATCH active=false` path is explicitly outside this Accepted shared-lock guarantee. If PATCH or any future SCIM read-modify-write operation can affect tombstone, survivor, or reactivation invariants, it must join the same lock boundary and add a concurrency regression before documentation may claim equivalent serialization.
Future SCIM read-modify-write operations that can affect tombstone, survivor, or reactivation invariants must join the same lock boundary and add a concurrency regression before documentation may claim equivalent serialization.

## Consequences

- Merge and SCIM full replacement share one documented concurrency authority.
- The protected-main PATCH behavior remains usable but must not be described as transactionally serialized with merge.
- Merge, SCIM full replacement, and both supported deprovisioning paths share one documented concurrency authority.
- SCIM deprovisioning lock contention is a retryable service-unavailable outcome; it is not evidence that a partial mutation occurred.
- Expanding the lock guarantee requires a source/test change, not a documentation-only promotion.
- Clustered deployments must provide the same shared-lock semantics for every operation included in this boundary.
- Clustered deployments must provide the same shared-lock semantics for every operation included in this boundary.
2 changes: 1 addition & 1 deletion docs/adr/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
| [0003](0003-identity-matching.md) | Exact external subject → verified email → explicit link matching precedence | Accepted |
| [0004](0004-desired-state-reconciliation.md) | Side-effect-free preflight plus intent/reconcile/re-observe/receipt lifecycle | Accepted |
| [0005](0005-secret-ownership.md) | Deployment/KV owns secrets; portable desired state remains secret-minimized | Accepted |
| [0006](0006-user-operation-lock.md) | Merge and SCIM full replacement share one user-operation lock boundary | Accepted |
| [0006](0006-user-operation-lock.md) | Merge/link, SCIM full replacement, and supported `PATCH active=false` share one user-operation lock boundary | Accepted |
| [0007](0007-automation-authority.md) | Autonomous development remains separate from review/merge/release authority | Accepted |
| [0008](0008-keyverse-rp-authorization-boundary.md) | Every non-fork RP explicitly validates Keyverse identity and manages ABAC/RBAC at its own boundary | Accepted |

Expand Down
69 changes: 69 additions & 0 deletions docs/doctoring/2026-08-scim-deactivation-lock.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
# SCIM deprovisioning lock boundary

**Date:** 2026-08-21
**Status:** Implementation evidence for the active PR; not protected-main or live Keycloak acceptance

## Scope

This record documents the supported `PATCH /scim/v2/Users/{id}`
`active=false` and `DELETE /scim/v2/Users/{id}` paths joining the existing
`UserOperationLocks` boundary used by account merge and SCIM full replacement.
The change is deliberately limited to the existing mutation paths; it adds no
database schema, UI, mapper, tenant claim, or external provider behavior.

## Interpretation

- **Standards requirement:** RFC 7644 defines SCIM protocol operations,
including PATCH and DELETE for User resources.
- **HTTP behavior:** lock contention is represented as HTTP `503 Service
Unavailable`, a retryable service-boundary result under the RFC 9110 status
semantics; the response does not claim that the remote mutation started.
The wire response is a root-level SCIM error with
`Content-Type: application/scim+json`, `schemas`, `detail`, and string
`status` fields. The service does not emit `Retry-After`; callers own bounded
backoff and must re-observe before retrying.
- **Policy choice:** the request is linearized by the shared lock. If either
deprovisioning path acquires the lock first, a concurrent merge observes the
disabled duplicate and fails rather than creating a contradictory tombstone.
- **Implementation behavior:** the lock covers each existing read/deactivate
sequence. A lock timeout exits before that sequence and maps to the root-level
SCIM error shape. The deterministic regression covers the PATCH-first
ordering; independent timeout tests cover both deprovisioning paths. This
record does not claim a separate merge-first race schedule or live clustered
deployment evidence.

## Evidence

- **RED:** before the source change, a deterministic two-thread regression
allowed merge to reach duplicate deactivation while PATCH was blocked in its
deactivation call.
- **GREEN:** after the source change, the same regression completes PATCH first,
makes merge observe the inactive account, and leaves no merge tombstone.
- **HTTP regression:** `test_scim_patch_lock_timeout_is_root_scim_error`
verifies the status, SCIM media type, and root-level error fields through
`TestClient`.
- **DELETE regression:** `test_scim_delete_translates_lock_timeout` verifies
DELETE exits before deactivation and returns the same retryable status at the
direct service boundary.
- **Cross-process backend regression:**
`test_sqlite_locks_serialize_across_processes` uses the production SQLite
sidecar from an independent spawned process, proves contention becomes the
retryable timeout, and verifies acquisition after release.
- **Measured boundary:** focused and complete account-unification tests passed;
production coverage measured 2,744 statements and 738 branches at 100%;
Interrogate reported 100% docstring quality; Ruff, compileall, and `uv build`
passed on this branch.
- **Not claimed:** this is not live Keycloak, PostgreSQL, clustered deployment,
browser login, or downstream authorization acceptance evidence. The SQLite
backend remains a conservative global lock; per-account locking is a future
throughput optimization only if measured contention justifies it.

## References

Fielding, R., Nottingham, M., & Reschke, J. (Eds.). (2022). *HTTP semantics*
(RFC 9110). Internet Engineering Task Force.
https://www.rfc-editor.org/rfc/rfc9110.html

Hunt, P., Grizzle, K., Ansari, M., Wahlström, E., & Mortimore, C. (2015).
*System for Cross-domain Identity Management: Protocol* (RFC 7644). Internet
Engineering Task Force. https://www.rfc-editor.org/rfc/rfc7644.html
Loading
Loading