feat: add claims and audit

.github/ISSUE_TEMPLATE/bug-report.md

@@ -16,7 +16,7 @@ SDK version: `python -c "import indykite_sdk; print(indykite_sdk.__version__)"`
 Python version: `python --version`
 Platform: output of `uname -a` (UNIX), or Windows version and 32/64-bit
 Client: the SDK client involved (ConfigClient, CaptureClient, AuthZENClient,
-CIQClient, DataSchemaClient, EntityMatchingClient - sync or async)
+CIQClient, DataSchemaClient, EntityMatchingClient, AuditClient - sync or async)
 -->
 
 * **SDK version**:

.github/PULL_REQUEST_TEMPLATE.md

@@ -22,7 +22,7 @@ it becomes the squash-commit message that drives the release version.
 ## Affected client(s)
 
 <!-- ConfigClient, CaptureClient, AuthZENClient, CIQClient, DataSchemaClient,
-EntityMatchingClient, core (auth/transport), packaging/CI, ... -->
+EntityMatchingClient, AuditClient, core (auth/transport), packaging/CI, ... -->
 
 ## Description of change
 

.github/workflows/docs.yaml

@@ -40,6 +40,7 @@ jobs:
           pipenv run pdoc
           indykite_sdk
           indykite_sdk.errors
+          indykite_sdk.audit
           indykite_sdk.capture
           indykite_sdk.authzen
           indykite_sdk.ciq

README.md

@@ -7,7 +7,8 @@
 
 Python clients for the [IndyKite](https://www.indykite.com) platform REST APIs:
 the Identity Knowledge Graph (IKG), KBAC authorization (AuthZEN), ContX IQ
-knowledge queries, data capture, entity matching, and platform configuration.
+knowledge queries, data capture, entity matching, the tamper-proof audit
+trail, and platform configuration.
 
 - SDK API reference: <https://indykite.github.io/indykite-sdk-python/>
 - OpenAPI reference: <https://openapi.indykite.com>
@@ -30,7 +31,7 @@ The SDK uses the two standard IndyKite credential kinds, obtained from the
 
 | Credential | Used by | What it is | Environment variables |
 | --- | --- | --- | --- |
-| **Application Agent** | all data-plane clients (capture, authzen, ciq, data schema, entity matching) | the **raw credential token itself** (opaque string, sent as `X-IK-ClientKey`) | `INDYKITE_APPLICATION_CREDENTIALS` (the token) or `INDYKITE_APPLICATION_CREDENTIALS_FILE` (file with the token) |
+| **Application Agent** | all data-plane clients (capture, authzen, ciq, data schema, entity matching, audit) | the **raw credential token itself** (opaque string, sent as `X-IK-ClientKey`) | `INDYKITE_APPLICATION_CREDENTIALS` (the token) or `INDYKITE_APPLICATION_CREDENTIALS_FILE` (path) |
 | **Service Account** | `ConfigClient` | a **JSON artifact** (`serviceAccountId`, pre-issued `token`, private key), sent as `Authorization: Bearer` | `INDYKITE_SERVICE_ACCOUNT_CREDENTIALS` (inline JSON) or `INDYKITE_SERVICE_ACCOUNT_CREDENTIALS_FILE` (path) |
 
 ```sh
@@ -52,6 +53,23 @@ expires the SDK self-signs a fresh JWT from the credential's private key
 (`privateKeyJWK` or PKCS#8). The app-agent token is never a JWT the SDK mints —
 it is sent exactly as issued.
 
+Each data-plane API is guarded by one **API permission** on the application
+agent, granted when the agent is created (Hub or `ConfigClient`):
+
+| Permission | Client | Endpoints |
+| --- | --- | --- |
+| `Authorization` | `AuthZENClient` | decisions and searches (`/access/v1/evaluation*`, `/search/*`) |
+| `ReadAuthZConfigs` | `AuthZENClient.policies` | the project's active policies (`/access/v1/policies`) |
+| `Capture` | `CaptureClient` | `/capture/v1/*` |
+| `ContXIQ` | `CIQClient` | `/contx-iq/v1/*` |
+| `ReadDataSchema` | `DataSchemaClient` | `/data-schema/v1` |
+| `EntityMatching` | `EntityMatchingClient` | `/entity-matching/v1/*` |
+| `Audit` | `AuditClient` | the tamper-proof audit trail (`/audit/v1/*`) |
+
+A call to an endpoint the agent is not permitted for raises
+`AuthenticationError` (401, `insufficient API access level`). A permission
+granted later takes a moment to reach the data plane.
+
 ### Regions and environments
 
 Production defaults to `https://eu.api.indykite.com`; pass `region="us"` for
@@ -78,6 +96,32 @@ with AuthZENClient() as client:
         print(policy.tags, policy.policy["actions"])
 ```
 
+#### Token claims in policy conditions
+
+Decisions and knowledge queries accept two optional request tokens. They travel
+as headers, never in the body, and the policy condition reads their claims:
+
+| Argument | Header | Claims in the policy |
+| --- | --- | --- |
+| `user_token` | `Authorization: Bearer` | `$token`, e.g. `$token.sub` (needs a Token Introspect configuration) |
+| `delegated_token` | `X-IK-Token` | `$ik_token`, e.g. `$ik_token.act.sub` — the RFC 8693 delegation chain of a token minted by the IndyKite Token Service |
+
+```python
+with AuthZENClient() as client:
+    result = client.evaluation(
+        ("Person", "ada"),
+        "CAN_DRIVE",
+        ("Car", "kitt"),
+        user_token=end_user_access_token,  # $token.sub == "ada"
+        delegated_token=ik_delegated_token,  # $ik_token.act.sub names the acting agent
+    )
+```
+
+`token` and `ik_token` are reserved names in `input_params`: a policy never
+asks for them, a value sent under them is replaced by the real claims, and a
+token that was not sent binds an empty claim set, so a policy reading it denies
+rather than fails.
+
 ### Capture graph data
 
 ```python
@@ -119,8 +163,35 @@ with CIQClient() as client:
     user_token = "<end-user-access-token>"  # a token your Token Introspect config can validate
     me = client.whoami(user_token)
     print(me.type, me.id)  # e.g. Person ada
+
+    # Run in the user's context; the CIQ policy reads $token.sub and $ik_token.act.sub
+    client.execute("gid:my-knowledge-query-id", user_token=user_token, delegated_token="<ik-token>")
 ```
 
+### Read the tamper-proof audit trail
+
+Every audit event of a project is appended to a signed **chain**: events are
+collected into signed batches (`logs`), a signed **manifest** links each batch
+to the previous one (`prev_hash` / `head_hash`), and signed **checkpoints**
+periodically fix the chain head. The agent needs the `Audit` permission and
+can only read its own project.
+
+```python
+from indykite_sdk import AuditClient
+
+with AuditClient() as client:
+    keys = client.jwks(project_id)  # public signing keys; keep them with an export
+    for batch in client.iter_logs(project_id):  # sequence order, all pages
+        for event in batch.data:  # untyped dicts, authored by whoever triggered them
+            print(batch.sequence, event.get("type"))
+    manifests = list(client.iter_manifests(project_id))  # linkage without payloads
+    newest = client.list_checkpoints(project_id, page_size=1).items  # newest first
+```
+
+Listings return a `Page` (`items`, `has_more`, `next_cursor`); `iter_*` follows
+the cursor for you. The signing key itself is configured with
+`ConfigClient.create_audit_signing` (see below).
+
 ### Manage platform configuration
 
 ```python
@@ -131,7 +202,7 @@ with ConfigClient() as config:
     project = config.create_project("my-project", organization.id, region="europe-west1")
     app = config.create_application("my-app", project.id)
     agent = config.create_application_agent(
-        "my-agent", app.id, ["Authorization", "Capture", "ContXIQ", "ReadAuthZConfigs"]
+        "my-agent", app.id, ["Authorization", "Capture", "ContXIQ", "ReadAuthZConfigs", "Audit"]
     )
     credential = config.create_application_agent_credential(agent.id)
     agent_credentials = credential.as_credentials()  # shown once - store it securely
@@ -145,8 +216,9 @@ app = config.read_application(app_id)
 config.update_application(app_id, etag=app.etag, display_name="Renamed")
 ```
 
-Audit signing decides which key signs a project's audit records. The default
-is a platform-managed key; customer-managed providers bring their own key:
+Audit signing decides which key signs a project's audit trail (the batches,
+manifests and checkpoints `AuditClient` reads). The default is a
+platform-managed key; customer-managed providers bring their own key:
 
 ```python
 signing = config.create_audit_signing("audit-signing", project.id)  # PLATFORM_MANAGED

examples/audit_logs.py

@@ -0,0 +1,58 @@
+"""Export a project's tamper-proof audit trail via the Audit Log API.
+
+Requires INDYKITE_APPLICATION_CREDENTIALS[_FILE] for an application agent that
+holds the ``Audit`` API permission, and INDYKITE_TEST_PROJECT_ID set to the
+project that agent belongs to. Writes logs.json, manifests.json,
+checkpoints.json and jwks.json to the current directory, which together form
+a self-describing export: the manifests link the batches through
+prev_hash / head_hash, the checkpoints fix the chain head at known times, and
+the JWKS names the keys every signature was made with.
+"""
+
+import json
+import os
+
+from indykite_sdk import AuditClient
+
+
+def main() -> None:
+    """Run the example."""
+    project_id = os.environ["INDYKITE_TEST_PROJECT_ID"]
+
+    with AuditClient() as client:
+        # The public signing keys - fetch them first and keep them with the export.
+        keys = client.jwks(project_id)
+        print(f"Signing keys: {[key.kid for key in keys.keys]}")
+
+        # Manifests are small: the chain linkage without the event payloads.
+        manifests = list(client.iter_manifests(project_id))
+        print(f"Chain length: {len(manifests)} batches")
+        for manifest in manifests[-3:]:
+            prev_hash, head_hash = (manifest.prev_hash or "")[:12], (manifest.head_hash or "")[:12]
+            print(f"  #{manifest.sequence} {prev_hash}.. -> {head_hash}.. kid={manifest.kid}")
+
+        # Logs page in lockstep with manifests; each batch carries its audit events in `data`.
+        logs = list(client.iter_logs(project_id, page_size=20))
+        events = [event for batch in logs for event in batch.data]
+        print(f"Audit events: {len(events)}")
+
+        # Checkpoints come newest first; a young project may have none yet.
+        checkpoints = list(client.iter_checkpoints(project_id))
+        if checkpoints:
+            newest = checkpoints[0]
+            print(f"Newest checkpoint: sequence {newest.sequence} at {newest.created_at}")
+
+    for name, items in (
+        ("jwks.json", keys),
+        ("manifests.json", manifests),
+        ("logs.json", logs),
+        ("checkpoints.json", checkpoints),
+    ):
+        with open(name, "w", encoding="utf-8") as handle:
+            payload = items.model_dump() if hasattr(items, "model_dump") else [item.model_dump() for item in items]
+            json.dump(payload, handle, indent=2)
+        print(f"Wrote {name}")
+
+
+if __name__ == "__main__":
+    main()

indykite_sdk/__init__.py

@@ -13,11 +13,13 @@
         print(result.decision)
 
 Each platform API has a sync and an async client. Config API clients use
-service-account credentials; all others use application-agent credentials.
+service-account credentials; all others (AuthZEN, Capture, ContX IQ, Data
+Schema, Entity Matching, Audit Log) use application-agent credentials.
 """
 
 from indykite_sdk._core.credentials import Credentials
 from indykite_sdk._core.retry import RetryConfig
+from indykite_sdk.audit import AsyncAuditClient, AuditClient
 from indykite_sdk.authzen import AsyncAuthZENClient, AuthZENClient
 from indykite_sdk.capture import AsyncCaptureClient, CaptureClient
 from indykite_sdk.ciq import AsyncCIQClient, CIQClient
@@ -46,12 +48,14 @@
 __all__ = [
     "APIStatusError",
     "__version__",
+    "AsyncAuditClient",
     "AsyncAuthZENClient",
     "AsyncCIQClient",
     "AsyncCaptureClient",
     "AsyncConfigClient",
     "AsyncDataSchemaClient",
     "AsyncEntityMatchingClient",
+    "AuditClient",
     "AuthZENClient",
     "AuthenticationError",
     "BadRequestError",

indykite_sdk/_core/errors_map.py

@@ -33,8 +33,10 @@
 _HINTS: dict[int, dict[str, str]] = {
     401: {
         "app_agent": (
-            "The X-IK-ClientKey token was rejected. Ensure INDYKITE_APPLICATION_CREDENTIALS holds an "
-            "application-agent credential JSON (not a service-account one) and that it has not expired."
+            "The X-IK-ClientKey token was rejected. Ensure INDYKITE_APPLICATION_CREDENTIALS holds the "
+            "application-agent credential token (not a service-account credential) and that it has not expired. "
+            "An 'insufficient API access level' message means the agent lacks the API permission this endpoint "
+            "needs (e.g. Audit for /audit/v1, ReadAuthZConfigs for /access/v1/policies)."
         ),
         "service_account": (
             "The bearer token was rejected. Ensure INDYKITE_SERVICE_ACCOUNT_CREDENTIALS holds a service-account "
@@ -43,8 +45,9 @@
     },
     403: {
         "app_agent": (
-            "The application agent lacks the required API permission. Check its apiPermissions "
-            "(e.g. Capture, Authorization, ContXIQ, EntityMatching) in the IndyKite Hub."
+            "The application agent is not allowed to access this resource. A project_id you passed must be the "
+            "project the agent belongs to; the agent's API permissions (Audit, Authorization, Capture, ContXIQ, "
+            "EntityMatching, ReadAuthZConfigs, ReadDataSchema) are managed in the IndyKite Hub."
         ),
         "service_account": "The service account is not allowed to manage this resource.",
     },

indykite_sdk/_core/ops.py

@@ -22,12 +22,22 @@ class RequestSpec:
     headers: dict[str, str] = field(default_factory=dict)
 
 
-def user_token_headers(user_token: str | None) -> dict[str, str]:
-    """Headers for an optional end-user access token on AuthZEN/ContX IQ calls.
-
-    The end-user token rides in ``Authorization: Bearer`` *alongside* the
-    application-agent ``X-IK-ClientKey`` header.
+def user_token_headers(user_token: str | None, delegated_token: str | None = None) -> dict[str, str]:
+    """Headers for the optional request tokens on AuthZEN/ContX IQ calls.
+
+    Both ride *alongside* the application-agent ``X-IK-ClientKey`` header, and
+    the platform publishes their claim sets to policy conditions under the
+    reserved parameter names ``token`` and ``ik_token``:
+
+    - the end-user access token in ``Authorization: Bearer``; its claims are
+      readable by policies as ``$token`` (e.g. ``$token.sub``);
+    - the IndyKite delegated token in ``X-IK-Token``; its claims, including
+      the RFC 8693 ``act`` delegation chain, are readable as ``$ik_token``
+      (e.g. ``$ik_token.act.sub``).
     """
-    if not user_token:
-        return {}
-    return {"Authorization": f"Bearer {user_token}"}
+    headers: dict[str, str] = {}
+    if user_token:
+        headers["Authorization"] = f"Bearer {user_token}"
+    if delegated_token:
+        headers["X-IK-Token"] = delegated_token
+    return headers

indykite_sdk/audit/__init__.py

@@ -0,0 +1,16 @@
+"""Audit Log API - read and export a project's tamper-proof audit trail."""
+
+from indykite_sdk.audit.aio import AsyncAuditClient
+from indykite_sdk.audit.client import AuditClient
+from indykite_sdk.audit.models import Checkpoint, Key, KeySet, LogEntry, Manifest, Page
+
+__all__ = [
+    "AsyncAuditClient",
+    "AuditClient",
+    "Checkpoint",
+    "Key",
+    "KeySet",
+    "LogEntry",
+    "Manifest",
+    "Page",
+]

indykite_sdk/audit/_ops.py

@@ -0,0 +1,55 @@
+"""Sans-IO request building shared by the sync and async Audit Log clients."""
+
+from __future__ import annotations
+
+from typing import Any
+
+from indykite_sdk._core.ops import RequestSpec
+from indykite_sdk.errors import IndyKiteError, RequestValidationError
+
+LOGS_PATH = "/v1/logs"
+MANIFESTS_PATH = "/v1/manifests"
+CHECKPOINTS_PATH = "/v1/checkpoints"
+JWKS_PATH = "/.well-known/jwks.json"
+
+
+def _project_id(project_id: str) -> str:
+    project_id = (project_id or "").strip()
+    if not project_id:
+        raise RequestValidationError(
+            "project_id is required: the project GID the application agent belongs to (``gid:...``)."
+        )
+    return project_id
+
+
+def list_spec(path: str, project_id: str, cursor: str | None, page_size: int | None) -> RequestSpec:
+    """Build one page request of a listing endpoint (``project_id``, ``cursor``, ``pagesize``).
+
+    ``page_size`` must be positive; the platform caps values above 50 to 50.
+    """
+    params: dict[str, Any] = {"project_id": _project_id(project_id)}
+    if cursor:
+        params["cursor"] = cursor
+    if page_size is not None:
+        if page_size < 1:
+            raise RequestValidationError(f"page_size must be a positive integer, got {page_size}.")
+        params["pagesize"] = page_size
+    return RequestSpec("GET", path, params=params)
+
+
+def jwks_spec(project_id: str) -> RequestSpec:
+    """Build the JWKS request; ``project_id`` is required but does not select a key today."""
+    return RequestSpec("GET", JWKS_PATH, params={"project_id": _project_id(project_id)})
+
+
+def next_cursor(cursor: str | None, page_next_cursor: str | None, has_more: bool) -> str | None:
+    """The cursor of the following page, or ``None`` when this was the last one.
+
+    Guards against a server handing out the cursor it was just given, which
+    would otherwise page forever.
+    """
+    if not has_more:
+        return None
+    if not page_next_cursor or page_next_cursor == cursor:
+        raise IndyKiteError("The Audit Log API returned a page that does not advance the cursor; stopping.")
+    return page_next_cursor