Static system prompt: share the appended block across sessions via the prompt cache

README.md

@@ -159,7 +159,7 @@ Semantic search over everything — conversations, facts, preferences, events. S
 
 - Four memory types: `profile`, `event`, `knowledge`, `behavior`
 - Automatic conversation indexing on session close
-- Pre-recall: relevant memories injected into system prompt when sessions start
+- Pre-recall: relevant memories delivered with the first message when sessions start (kept out of the system prompt so it stays cacheable across sessions)
 - 3-level quality filtering prevents generic facts from polluting memory
 - Semantic deduplication (cosine similarity 0.85 threshold)
 - Full audit log of all mutations

config.example.yaml

@@ -67,6 +67,12 @@ agent:
   model_discovery: true
   max_turns: 50                  # Max agentic turns per request
   max_concurrent: 32             # Max concurrent agent sessions
+  # Keep the appended system prompt byte-identical across sessions so the
+  # exact-prefix prompt cache is shared between them: the session id and the
+  # pre-recalled memories travel in a <session-context> block at the top of
+  # each session's first message instead. false → legacy shape (id + recall
+  # inline in the system prompt; every session start re-writes the cache).
+  static_system_prompt: true
   background_agent_permissions: true  # Background sub-agents (Agent run_in_background) get the same tool permissions as foreground; false denies their Write/Edit/Bash
   # Agent teams (CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS). The CLI gates the
   # SendMessage tool behind this flag while the Agent tool advertises it

docs/architecture.md

@@ -165,7 +165,7 @@ idle-stream support, cache-TTL policy) gate engine behavior — never
 1. User sends message via Telegram
 2. `TelegramChannel` receives it, resolves active session
 3. `AgentEngine.run()` is called with the message
-4. System prompt is built (SOUL.md + IDENTITY.md + memories)
+4. System prompt is built (SOUL.md + IDENTITY.md + MEMORY.md — identical across sessions); the session id and pre-recalled memories are prepended to the first message
 5. Claude Agent SDK processes the message with tools
 6. Streaming events broadcast to Telegram (edit-in-place) and any WebSocket clients
 7. Final response stored in SQLite

docs/config.md

@@ -907,6 +907,7 @@ from any working directory:
 | `agent.max_concurrent` | int | `32` | Max concurrent agent sessions |
 | `agent.cache_ttl` | string | `"5m"` | Prompt-cache write TTL policy: `5m` (status quo), `1h` (always request the 1-hour TTL), or `auto` (per session at client-build time: sparse-cadence sessions — persistent crons, wakeup loops, spaced chats — get `1h`; dense sessions stay on `5m`). Per-cron-job override via `cache_ttl` in jobs.yaml. See `nerve/agent/cache_policy.py` |
 | `agent.cache_ttl_excluded_models` | list | `[]` | Model-name substrings that never request the 1h TTL |
+| `agent.static_system_prompt` | bool | `true` | Keep the appended system prompt byte-identical across sessions of the same workspace/source/tool set so Anthropic's exact-prefix prompt cache shares it between sessions (a cache read instead of a full re-write at every session start). The per-session parts — session id and pre-recalled memories — are delivered once, in a `<session-context>` block at the top of the session's first user message; `mcp__nerve__session_context` returns them live. `false` restores the legacy shape (id and recall inline in the system prompt) |
 | `agent.cli_max_message_bytes` | int | `67108864` (64 MiB) | Upper bound on one stream-json message read from the Claude CLI subprocess (the Agent SDK's `max_buffer_size`). The SDK's own default is 1 MiB, and a line over the bound aborts the whole turn — image tool results routinely exceed it: the Read tool ships the base64 image twice per line and re-encodes anything over 2000 px, so a 300 KB screenshot can become a 1.3 MB line. The Read image validator also refuses images whose encoded line cannot fit under this bound, so a too-big image fails as a tool error instead of killing the turn |
 | `agent.agent_teams` | bool | `true` | Set `CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS=1` for the CLI subprocess, which registers the `SendMessage` tool. The Agent tool advertises `SendMessage` for resuming a sub-agent whether or not the flag is set, so with it off the model reaches for a tool that does not exist. Nerve loads no settings files (`setting_sources=[]`), so the env dict is the flag's only route in. Teammates stay opt-in per turn and cost a full context window each; the CLI cannot restore in-process teammates when a session's client is recycled (idle timeout, restart, crash retry) |
 | `agent.prompt_rewrite.enabled` | bool | `true` | Offer the first-prompt rewrite feature in the web UI (per-user toggle lives in the composer) |

docs/memory.md

@@ -35,7 +35,7 @@ memU provides semantic search over conversations and workspace files. It runs em
 1. **Conversation indexing** — When a session closes (daily rotation, shutdown, crash recovery), all context messages are indexed into memU
 2. **Explicit memorize** — The agent can use the `memorize` tool to save specific facts on demand
 3. **Recall** — The agent uses `memory_recall` for semantic search, `conversation_history` for event-date queries, and `memory_records_by_date` for creation/update-date queries
-4. **Pre-recall** — When a new SDK client is created, relevant memories are recalled and injected into the system prompt
+4. **Pre-recall** — When a session starts, relevant memories are recalled (frozen in session metadata and reused on every client rebuild) and delivered in a `<session-context>` block at the top of the session's first message — kept out of the system prompt so its bytes stay identical across sessions and prompt-cacheable (`agent.static_system_prompt`)
 
 ### Session Lifecycle
 

nerve/agent/engine.py

@@ -47,8 +47,10 @@
     get_handler,
 )
 from nerve.agent.prompts import (
+    build_session_preamble,
     build_system_prompt,
     current_time_str,
+    prepend_session_preamble,
     set_skill_manager,
 )
 from nerve.agent.sessions import SessionManager, SessionStatus
@@ -293,6 +295,14 @@ def __init__(self, config: NerveConfig, db: Database):
             )
         )
         self._session_backends: dict[str, str] = {}
+        # Per-session context awaiting delivery (static-prompt mode). The
+        # system prompt is byte-identical across sessions, so the session
+        # id and the frozen recall are rendered separately at client build
+        # (_get_or_create_client) and prepended to the FIRST user message
+        # of the native conversation by _run_inner, which pops the entry.
+        # Keyed by session id; an entry survives a client rebuild that
+        # happens before any turn was sent (the rebuild re-renders it).
+        self._pending_preambles: dict[str, str] = {}
 
     @property
     def config(self) -> NerveConfig:
@@ -1206,13 +1216,32 @@ async def _get_or_create_client(
                 if session_meta.get("cache_ttl") != cache_ttl:
                     meta_updates["cache_ttl"] = cache_ttl
 
+            # Determine if this is a fork
+            is_fork = fork_from is not None
+
+            # Static-prompt mode (agent.static_system_prompt): the session
+            # id and the frozen recall stay OUT of the system prompt so its
+            # bytes are shared across sessions (exact-prefix cache), and
+            # are prepended to the first user message of the native
+            # conversation instead. Deliver when the transcript is fresh
+            # (no resume target), on a fork (new id; the parent's history
+            # carries the parent's), and once for a resumed session that
+            # never received one (created before this mode existed). The
+            # ``preamble_sent`` marker records the delivery so a resume
+            # doesn't repeat it; the transcript replays it on later turns.
+            static_prompt = bool(self.config.agent.static_system_prompt)
+            deliver_preamble = static_prompt and (
+                not sdk_resume_id
+                or is_fork
+                or not session_meta.get("preamble_sent")
+            )
+            if deliver_preamble and session and not session_meta.get("preamble_sent"):
+                meta_updates["preamble_sent"] = True
+
             if meta_updates and session:
                 session_meta.update(meta_updates)
                 await self.db.update_session_metadata(session_id, session_meta)
 
-            # Determine if this is a fork
-            is_fork = fork_from is not None
-
             # Create interactive hub for this session.
             # Non-web sessions (telegram, cron, hook) cannot handle
             # interactive pauses — auto-deny them to prevent deadlocks.
@@ -1226,9 +1255,10 @@ async def _get_or_create_client(
             register_handler(session_id, handler)
 
             # Render the system prompt (engine-owned: identity files,
-            # frozen recall, skills; the tool list respects the backend's
-            # exclusions so the prompt never advertises a tool this
-            # session's MCP server doesn't serve).
+            # skills; the tool list respects the backend's exclusions so
+            # the prompt never advertises a tool this session's MCP server
+            # doesn't serve). In static mode the id and the frozen recall
+            # are ignored here and go into the preamble below.
             system_prompt = build_system_prompt(
                 workspace=self.config.workspace,
                 session_id=session_id,
@@ -1237,6 +1267,7 @@ async def _get_or_create_client(
                 recalled_memories=recalled_memories or None,
                 skill_summaries=self._collect_skill_summaries(),
                 excluded_tools=backend.excluded_tools(),
+                static=static_prompt,
             )
 
             # Observer sessions of a review loop get a context block so the
@@ -1308,6 +1339,12 @@ async def _record_wakeup_cb(sid: str, tool_input: dict) -> Any:
                     session_id, {"sdk_session_id": None},
                 )
                 sdk_resume_id = None
+                # The transcript is fresh after all — it needs the preamble.
+                deliver_preamble = static_prompt
+            if deliver_preamble:
+                self._pending_preambles[session_id] = build_session_preamble(
+                    session_id, source, recalled_memories or None,
+                )
             self.sessions.set_client(session_id, client)
             self._session_backends[session_id] = backend.name
 
@@ -2665,6 +2702,23 @@ async def _run_inner(
             # rules (e.g. the Claude CLI's slash-command interception).
             turn_input = TurnInput(text=query_text, images=images)
 
+            def _with_session_preamble(base: TurnInput) -> TurnInput:
+                # Leading session-context block (static-prompt mode): the
+                # session id and the frozen recall that the shared system
+                # prompt leaves out. Rendered at client build; popped here
+                # so it ships exactly once — with the FIRST user message of
+                # the native conversation (later turns replay the
+                # transcript). Rebuilt from ``query_text`` so a retry that
+                # re-renders it never stacks two blocks. Like the time
+                # note, not persisted to the DB message (db_text above).
+                preamble = self._pending_preambles.pop(session_id, None)
+                if not preamble:
+                    return base
+                return TurnInput(
+                    text=prepend_session_preamble(query_text, preamble),
+                    images=images,
+                )
+
             # Send query + read response, with auto-retry on runtime crash.
             # The runtime may die during start_turn (TransportDiedError) or
             # during response reading (generic Exception from the stream).
@@ -2695,6 +2749,9 @@ async def _run_inner(
                 metadata=_lf_metadata,
             ):
                 for _attempt in range(2):
+                    # Re-checked per attempt: a crash-retry that rebuilt the
+                    # client on a fresh transcript re-renders the preamble.
+                    turn_input = _with_session_preamble(turn_input)
                     try:
                         await client.start_turn(turn_input)
                     except TransportDiedError as _qerr:

nerve/agent/prompts.py

@@ -1,4 +1,6 @@
-"""System prompt builder — loads SOUL.md, IDENTITY.md, and injects recalled memories."""
+"""System prompt builder — loads SOUL.md, IDENTITY.md, etc. into a
+cross-session-stable system prompt, and renders the per-session preamble
+(session id, recalled memories) that travels with the first user message."""
 
 from __future__ import annotations
 
@@ -103,6 +105,11 @@ def _format_skills_list(skill_summaries: list[dict] | None = None) -> str | None
     return "\n".join(lines)
 
 
+# Wrapper tag for the per-session preamble prepended to a conversation's
+# first user message (see ``prepend_session_preamble``).
+SESSION_CONTEXT_TAG = "session-context"
+
+
 def build_system_prompt(
     workspace: Path,
     session_id: str = "",
@@ -111,11 +118,25 @@ def build_system_prompt(
     timezone_name: str = "America/New_York",
     skill_summaries: list[dict] | None = None,
     excluded_tools: "set[str] | None" = None,
+    static: bool = True,
 ) -> str:
-    """Build the full system prompt for the agent.
-
-    Loads identity files from workspace, adds session context,
-    and appends any recalled memories from memU.
+    """Build the system prompt for the agent.
+
+    Loads identity files from the workspace, adds the session-context
+    block, the tool list and the skills summary.
+
+    ``static=True`` (default) renders a prompt that is byte-identical for
+    every session of the same workspace, source and tool set: the
+    per-session ``session_id`` and ``recalled_memories`` are accepted for
+    signature compatibility but NOT rendered — they are delivered by
+    :func:`build_session_preamble` at the top of the conversation's first
+    user message instead. Anthropic prompt caching is exact-prefix, so a
+    single differing byte (a session id, a per-session recall list) turns
+    the whole ~100K-token appended block from a cross-session cache READ
+    into a cache WRITE at every session start.
+
+    ``static=False`` restores the legacy shape: session id and recalled
+    memories inline in the system prompt.
     """
     parts: list[str] = []
 
@@ -147,7 +168,24 @@ def build_system_prompt(
     except Exception:
         today = datetime.now().strftime("%Y-%m-%d")
 
-    context = f"""# Session Context
+    if static:
+        # No per-session bytes here (see the docstring): the id and the
+        # recall land in the first user message. The pointer below is
+        # static text, so it costs nothing cache-wise.
+        context = f"""# Session Context
+- **Source:** {source}
+- **Current date:** {today}
+- **Workspace:** {workspace}
+
+Per-session details — this session's id and the memories recalled at \
+session start — are in the `<{SESSION_CONTEXT_TAG}>` block at the top of \
+this conversation's first user message. `mcp__nerve__session_context` \
+returns them live if that message is no longer in context.
+
+You have access to the following custom tools:
+{_format_tool_list(excluded_tools)}"""
+    else:
+        context = f"""# Session Context
 - **Session ID:** {session_id}
 - **Source:** {source}
 - **Current date:** {today}
@@ -162,9 +200,44 @@ def build_system_prompt(
     if skills_section:
         parts.append(skills_section)
 
-    # Recalled memories from memU
-    if recalled_memories:
-        memories_text = "\n".join(f"- {m}" for m in recalled_memories)
-        parts.append(f"# Recalled Memories\n\n{memories_text}")
+    # Recalled memories from memU — legacy (non-static) shape only; the
+    # static prompt delivers them via build_session_preamble.
+    if not static and recalled_memories:
+        parts.append(_format_recalled_memories(recalled_memories))
 
     return "\n\n---\n\n".join(parts)
+
+
+def _format_recalled_memories(recalled_memories: list[str]) -> str:
+    memories_text = "\n".join(f"- {m}" for m in recalled_memories)
+    return f"# Recalled Memories\n\n{memories_text}"
+
+
+def build_session_preamble(
+    session_id: str,
+    source: str = "web",
+    recalled_memories: list[str] | None = None,
+) -> str:
+    """Render the per-session context the static system prompt leaves out.
+
+    Returns the markdown block (session id, source, recalled memories)
+    that the engine prepends — wrapped by :func:`prepend_session_preamble`
+    — to the FIRST user message of a native conversation. Later turns
+    replay the transcript, so the block reaches the model exactly once.
+    """
+    context = (
+        "# Session Context\n"
+        f"- **Session ID:** {session_id}\n"
+        f"- **Source:** {source}"
+    )
+    parts = [context]
+    if recalled_memories:
+        parts.append(_format_recalled_memories(recalled_memories))
+    return "\n\n".join(parts)
+
+
+def prepend_session_preamble(text: str, preamble: str) -> str:
+    """Prepend ``preamble`` to a user message inside a ``<session-context>``
+    wrapper so the model can tell the injected context from the prompt."""
+    block = f"<{SESSION_CONTEXT_TAG}>\n{preamble}\n</{SESSION_CONTEXT_TAG}>"
+    return f"{block}\n\n{text}" if text else block

nerve/config.py

@@ -854,6 +854,15 @@ class AgentConfig:
     # Substrings of model names that must never request the 1h cache TTL
     # (same matching semantics as context_1m_excluded_models).
     cache_ttl_excluded_models: list[str] = field(default_factory=list)
+    # Keep the appended system prompt byte-identical across sessions of the
+    # same workspace/source/tool set so Anthropic's exact-prefix prompt cache
+    # can share it: the per-session parts (session id, pre-recalled memories)
+    # are delivered in a <session-context> block at the top of the first user
+    # message instead of in the system prompt. With per-session bytes in the
+    # prompt, every session start is a cache WRITE of the whole block (on a
+    # fleet of ~800 cron sessions/day that was measured at ~$850/day) instead
+    # of a cache READ. False restores the legacy shape (id + recall inline).
+    static_system_prompt: bool = True
     # Hung-CLI detection: max idle time between SDK messages on a single
     # turn before the engine treats the subprocess as dead and falls into
     # the existing CLI-crash retry path.  Set to 0 to disable (legacy
@@ -931,6 +940,7 @@ def from_dict(cls, d: dict) -> AgentConfig:
             cache_ttl_excluded_models=_str_list(
                 d.get("cache_ttl_excluded_models")
             ),
+            static_system_prompt=d.get("static_system_prompt", True),
             cli_idle_timeout_seconds=d.get("cli_idle_timeout_seconds", 900),
             cli_max_message_bytes=int(
                 d.get("cli_max_message_bytes", 64 * 1024 * 1024)