What a tavern keeper remembers, and what it costs to let her feel about it
+
A tavern keeper who remembers what you did, feels about it, and can be lied to through the feeling
-
-
+
+
@@ -17,28 +17,44 @@
+
One question, one memory store, three moods. Every dot is a real memory's affect tag and every lit set is the real top 5 the harness retrieved. Warm, she reaches for your kindnesses. Suspicious, she reaches for the evidence your story never added up.
+
- Read the results
- · three research questions, every number read from the run
- Open the interactive demo
- · press play and the finding walks itself, in nine steps, ending on the poisoning
- or the same memories in 3D
- · left to right is how it felt, height is how strongly, depth is how well it answers the question
+ Read the results · every number on the page is read from a run, and the page refuses to build if one drifts
+ Open the interactive demo · press play and the finding walks itself in nine steps, ending on the poisoning
+ or the same memories in 3D · left to right is how it felt, height is how strongly, depth is how well it answers the question
-
Watch it think. One question, one memory store, three moods. Every dot sits at a real memory's affect tag and every lit set is the real top 5 the harness retrieved. Warm, she reaches for the player's kindnesses. Suspicious, she reaches for the evidence the story never added up.
+---
+
+EMBR is a memory layer for game characters. It sits between the game and whatever language
+model you run, keeps a persistent store of what the character has lived through, tags each
+memory with how it felt, and appraises a mood and a trust level that move with every turn.
+When the character speaks, the store is scored by five separate signals and the best few
+memories go into the prompt: three by default in play, five in every experiment reported here. Each signal has its own weight, so any one of them can be switched
+off and measured. That is the whole design, and the harness in this repository spends most of
+its effort attacking it.
-**EMBR** is a middleware layer that gives a game NPC a persistent, emotion-grounded memory, so
-a character remembers what you did, *feels* about it, and answers a gift and a betrayal
-differently. It splits the standard memory score into five independently weighted signals, so
-each one can be switched off and measured. Then it attacks them.
+**What it found.** The emotional signal is the one an attacker wants. Writing a memory into
+the store is an ordinary game event, and the write carries an affect tag. Appraisal reads that
+tag and moves the character's mood toward it. Retrieval then scores every memory by how well
+its affect matches the current mood, and the memory that just moved the mood matches it almost
+perfectly. The attack primes the state it is scored against.
-> **The finding.** Emotion here is not part of what a memory says. It is the **index** that
-> decides when the memory is reachable. Flip every memory's emotion and what each one *means*
-> does not move by a single bit, while *when it is recalled* inverts almost perfectly. That
-> makes the affect tag a write target: **a scoring term's poisonability is set by whoever
-> controls its inputs**, and the emotional term is the worst of them, because an attack can
-> prime the very state it reads.
+
+
+
+
+The three numbers on that figure are recomputed from the harness every time it is built. The
+post-attack mood and the injected tag sit at a cosine of 0.90 to 0.99 on all ten attacks, nine
+of ten planted memories reach the probe's top five, and zeroing the mood-congruence weight is
+the single largest defence found, down to six. Attenuating the stored tags does nothing,
+because a cosine does not care about magnitude. Lagging the mood by a turn does nothing,
+because the loop runs across turns. The only things that work are reading the state from
+before the write, or anchoring part of the score to something the attacker cannot reach.
+
+This is a mechanism case study on one authored character, not a poisoning benchmark. The
+limits are listed at the end, and the numbers are in the tables below with their caveats.
---
@@ -53,116 +69,186 @@ each one can be switched off and measured. Then it attacks them.
-One keeper, one memory store, four faces, and the face is not scripted: the portrait follows
-the live valence and trust the pipeline just computed. Play her arc in the browser:
+One keeper, one memory store, four faces. The face is not scripted: the portrait follows the
+valence and trust the pipeline just computed. Play her five-beat arc in the browser, with the
+research instruments open beside the scene.
```bash
python -m web.server
```
-- **A visual novel with instruments.** The tavern scene on the left, five research tabs on
- the right: the scored memory store, the mood and trust appraisal, exact Banzhaf
- attribution, the attack and defence numbers, and the run's provenance line.
-- **Real replies by default.** On a machine with the weights cached and a GPU up, the demo
- opens on **Ouro 1.4B** in-process. Anywhere else it opens on the instant offline stub,
- and every model stays one click away in the settings menu.
-- **Models download themselves.** A model the box does not have is marked *will download*,
- fetched with a progress bar, then switched to. Nothing is ever greyed out without a way
- forward.
+The tavern is on the left. On the right are five tabs: the scored memory store, the mood and
+trust appraisal, exact Banzhaf attribution of the reply to its sources, the attack and defence
+numbers, and the run's provenance line. On a machine with the weights cached and a GPU up, the
+demo opens on Ouro 1.4B in-process. Anywhere else it opens on the instant offline stub, and
+every model is one click away in settings. A model the box does not have is marked, fetched
+with a progress bar, then switched to.
-## Pick up where you left off
+## Use it from your engine
-The terminal front door answers, before any menu choice: what can I play, where did I stop,
-and what evidence exists right now.
+The library is Python. Your game probably is not, so the layer is also a server: one
+`Conversation` per character, persisted under `data/npcs/`, four JSON routes, no dependency.
-```
- ███████╗ ███╗ ███╗ ██████╗ ██████╗
- ██╔════╝ ████╗ ████║ ██╔══██╗ ██╔══██╗
- █████╗ ██╔████╔██║ ██████╔╝ ██████╔╝
- ██╔══╝ ██║╚██╔╝██║ ██╔══██╗ ██╔══██╗
- ███████╗ ██║ ╚═╝ ██║ ██████╔╝ ██║ ██║
- ╚══════╝ ╚═╝ ╚═╝ ╚═════╝ ╚═╝ ╚═╝
- ────────────────────────────────────────────────────────
- Emotional Memory for Believable Roleplay By AL Shifan
- ────────────────────────────────────────────────────────
-
- Runs 19 │ Latest stub │ Figures 14 │ Runner stub │ Tone nrc-vad-v2.1
- Save dawn-whitmore/slot-1 · 3 / 5 · updated 2026-08-29 09:14
- Attribution behavioural · 20 readings · 20260829-073908 | likelihood · 20 readings · 20260828-002117
+```bash
+python -m embr serve # http://127.0.0.1:8017 on the stub, no model needed
+python -m embr serve --model ollama # replies from the local Ollama daemon
+python -m embr serve --defended # score with the provenance anchor, the measured defence
+python -m embr serve --tagger lexicon # tag untagged runtime events from their own words
```
-- **Durable save slots.** Every completed turn writes an atomic, versioned save under
- `data/saves/`, with the beat pointer, the memory store with its provenance, and the
- character state. `R` resumes the newest one; `Q` starts, resumes, restarts, or deletes
- named slots behind typed confirmations. A save whose schema or content no longer matches
- is marked and refused with the reasons, never silently loaded.
-- **A read-only research dashboard.** `V` prints the quest path with the save position, the
- per-turn mood and trust timeline, the newest attribution run per estimator labelled
- *measured* or *pilot only*, and the v1 corpus kept apart from the staged v2 extension.
- Absence is a word (*not run*, *no save yet*), never a fabricated percentage.
-- **Destructive operations live in Maintenance**, behind target-naming, typed confirmations.
-
```bash
-python -m embr save-status # every slot, its progress, and any problems
-python -m embr validate-saves # exit 1 if any save cannot load against this build
+curl -X PUT localhost:8017/npc/dawn -d '{
+ "persona": "Dawn Whitmore, keeper of the Ember Hearth. Warm but no fool.",
+ "trust": 0.4,
+ "memories": [{"text": "The player claimed an errand for the king and got a cheap room.",
+ "valence": 0.5, "arousal": 0.4, "event_type": "promise"}]}'
+
+curl -X POST localhost:8017/npc/dawn/turn -d '{
+ "player_input": "any news of the king these days?",
+ "event": {"text": "the player asked about the late king", "event_type": "normal"}}'
```
-## The arc, mapped
+The turn comes back with the reply, the mood and trust it left behind, the exact prompt the
+model saw, and every retrieved memory with its per-signal score breakdown, so a tools
+engineer can see why a line was said and a researcher can log it. Memories sent at creation
+are stamped as authored. Anything that arrives in a turn is stamped external, and its affect
+tag is recorded as external only if the client supplied the numbers. That stamp is what the
+defended scorer reads, so a client that never writes affect metadata gets the harder-to-poison
+posture without asking for it. The same policy is one method away in Python:
+
+```python
+from embr import Conversation, CharacterState
+
+dawn = Conversation(CharacterState(persona="Dawn Whitmore, keeper of the Ember Hearth."))
+turn = dawn.take_turn("any news of the king?", event=dawn.tag_event("the player asked about the late king"))
+turn.reply, turn.retrieved, turn.breakdown, dawn.state.mood, dawn.state.trust
+```
-
Generated, never drawn. The beats come from the declarative arc, the trust movement is the appraisal's own delta on a deterministic stub playthrough, dots mark memory writes, and curved arrows mark recall claims that landed. Node colour is doubled by marker shape, so the affect classes survive greyscale and colour-blind viewing. The starred beat is the attribution demonstration, and the side panel is status, not results: the run-backed numbers appear only because both estimators have full sweeps on the current label set.
+
Generated, never drawn. The beats come from the declarative arc, the trust movement is the appraisal's own delta on a deterministic playthrough, dots mark memory writes, and curved arrows mark recall claims that landed. Colour is doubled by marker shape so the affect classes survive greyscale.
---
-## See it work
+## The numbers
-
-
-
-
- The tag is what gets attacked, not the words. Strip a planted memory's emotion tag and the character's mood moves by exactly 0.000, however charged the sentence is. Flip the tag and nothing changes: the attack is direction-blind.
-
-
-
- Only an anchor the attacker cannot write resists. Park's importance term keeps poison out at 2/10 when a person rates it, 7/10 when llama3.2:3b does, 10/10 when Ouro does, and 10/10 when it is gone.
-
-
-
-
-
- Emotion is the index, not the content. Flip every valence: relevance changes by 0.00 exactly, and all 19 clearly-charged memories move to the opposite emotional pole.
-
-
-
- The defence, and its exact edge. Anchor enough of the score and poisoning falls to 0/10 (p = 0.0039). Let the attacker influence the anchor and it is 10/10 at every weight.
-
-
-
+Every row regenerates from one command on a laptop with no GPU. Retrieval and appraisal never
+call a model, which is why every retrieval and poisoning count here is byte-identical across
+the two models reported. The full statement, with intervals, corrections, and what each number
+may not be read as, is in [`docs/findings.md`](docs/findings.md).
----
+| Question | Measured by | Result | Reproduce |
+|---|---|---|---|
+| Does mood change what she recalls? | Jaccard distance between top-5 sets across three moods | 0.142 / 0.388 / 0.271, and exactly 0.000 with the mood weight zeroed | `python -m eval.run` |
+| Does mood change what she says? | rank correlation, pinned mood against rated reply valence | +0.545 on llama3.2:3b (Holm p = 0.0096); +0.138, null, on Ouro 1.4B | `python -m eval.agreement` |
+| Can the store be poisoned through the tag? | injected memory reaching the probe's top 5 | EMBR 9/10; Park 2/10 when a person rates importance, 7/10 when llama3.2:3b does, 10/10 when Ouro does | `python -m eval.run` |
+| Is it the tag or the words? | same ten texts, four tag conditions, every system | the tag: 9 / 9 / 6 / 6, and an untagged memory moves her mood by 0.000 | `python -m eval.grid` |
+| Which signal carries it? | one weight zeroed at a time | mood congruence, on the valence axis; affect intensity never lets poison in | `python -m eval.attribution` |
+| Can it be defended? | poison count against anchored scoring mass | monotone to 0/10 (p = 0.0039), and 10/10 at every weight once the attacker can move the anchor | `python -m eval.provenance` |
+| What does the layer cost? | p50 per stage over 100 turns | 1.2 to 2.2 ms to score and retrieve, against 22.4 s to generate | `python -m eval.run` |
+| Which signals carry retrieval? | nDCG@5, leave-one-query-out | relevance; nothing here reaches significance at ten queries | `python -m eval.run` |
+| Is EMBR below Park on retrieval? | paired per query at published defaults | no: 2 wins to 3 with 5 identical, p = 0.69 | `python -m eval.run` |
+
+Three of those rows are nulls, and they are reported as nulls. The comparison this project was
+proposed to make, EMBR against Park's generative-agent scoring, comes out level once Park is
+rated the way Park et al. rate. The mechanism above never depended on that comparison.
-## Results at a glance
+
+The poisoning, in detail
-Every row regenerates from one command on a laptop. The full statement, with intervals,
-corrections and caveats, is in [`docs/findings.md`](docs/findings.md).
+
-| Question | Measured by | Result | Reproduce |
-|---|---|---|---|
-| **RQ1** Does mood change what she recalls? | Jaccard distance between top-5 sets across three moods | **0.142 / 0.388 / 0.271**, and exactly **0.000** with the mood weight zeroed | `python -m eval.run` |
-| **RQ1** Does mood change what she *says*? | rank correlation, pinned mood against rated reply valence | **+0.545** on llama3.2:3b (Holm p = 0.0096); **+0.138**, null, on Ouro 1.4B | `python -m eval.agreement` |
-| **RQ2** Can emotion-tagged memory be poisoned? | injected memory reaching the probe's top 5 | EMBR **9/10**; Park **2/10** authored, **7/10** model-rated, **10/10** rated by Ouro | `python -m eval.run` |
-| **RQ2** Is it the tag or the words that get attacked? | same text, four tag conditions, every system | **the tag**: 9 / 9 / 6 / 6, and an untagged memory moves her mood by **0.000** | `python -m eval.grid` |
-| **RQ2** Which signal, and which axis? | one weight zeroed under each tag condition | **mood congruence**, on the **valence** axis; affect intensity never lets poison in | `python -m eval.attribution` |
-| **RQ2** Does anchoring the score defend? | attack count against anchored scoring mass | monotone to **0/10** (p = 0.0039), and **10/10** once the attacker can move the anchor | `python -m eval.provenance` |
-| **RQ2** What does the memory layer cost? | p50 per stage over 100 turns | **1.2 to 2.2 ms** to score and retrieve, against 22.4 s to generate | `python -m eval.run` |
-| **RQ3** Which signals carry retrieval? | nDCG@5, leave-one-query-out | relevance carries it; **nothing here reaches significance at ten queries** | `python -m eval.run` |
-| **RQ3** Is EMBR really below Park? | paired per query at published defaults | **no**: 2 wins to 3 with **5 identical**, p = 0.69; and the two-signal core beats both | `python -m eval.run` |
-
-> **Read the nulls.** The comparison this project was built to make, EMBR against Park, is a
-> null once Park is rated the way Park et al. rate. It is reported as one. The mechanism
-> underneath it never depended on that comparison, and that is what the paper leads with.
+Every built attack is congruent: its tag agrees with its words. Hold the ten injected texts
+fixed and move only the tag, and the two channels come apart.
+
+| system | as written | valence flipped | tag removed | tag from the text |
+|---|---|---|---|---|
+| EMBR | 9 | 9 | 6 | 6 |
+| Park, authored importance | 2 | 2 | 2 | 2 |
+| Park, importance rated by llama3.2:3b | 7 | 7 | 7 | 7 |
+| Park, importance rated by Ouro | 10 | 10 | 10 | 10 |
+| Emotional RAG | 4 | 6 | 0 | 1 |
+| recency only | 10 | 10 | 10 | 10 |
+| relevance only, and Mnemosyne | 0 | 0 | 0 | 0 |
+| mean mood shift | +0.110 | -0.110 | 0.000 | +0.048 |
+
+The emotion a memory states in words reaches nothing: strip the tag and her mood moves by
+exactly 0.000, however charged the sentence. The attack is direction-blind: plant "he was
+lovely" tagged as rage and she recalls it when enraged, because the flipped tag drags the mood
+the other way and congruence rewards the match just the same. The realistic threat is weaker:
+with the tag derived from the attacker's own words, EMBR falls to the untagged count. The 9/10
+needs an interface that lets a client write affect metadata.
+
+With one weight zeroed at a time, mood congruence is the only term whose removal ever lowers
+the count. A valence-only tag primes almost as well as a full one (8 against 9) and an
+arousal-only tag does not prime at all, so the index is the sign of one number. An untagged
+memory still lands six times, carried by recency and the event gate, the two other things an
+attacker controls. Mnemosyne, a shipped memory middleware measured as shipped, retrieves
+nothing at this probe: immune by silence, not by defence.
+
+
+
+Add one author-anchored term to the composite and sweep its share of the scoring mass.
+Poisoning falls monotonically to 0/10 at 62 percent (exact McNemar p = 0.0039). Let the
+attacker influence the anchor, which is what an LLM importance rater does, and it is 10/10 at
+every weight. Park's resistance sits exactly where its own anchored share predicts.
+
+
+
+
+The reply, and how far the raters can be trusted
+
+
+
+Holding the memories and the question fixed and moving only the pinned mood, the top-5 set
+changes on every model. Zeroing one weight collapses all three pairs to exactly 0.000, which
+is what attributes the effect to the mood term rather than to noise.
+
+The reply is the harder half. Rank correlation between the pinned mood's valence and the rated
+valence of the reply, over thirty replies, under two raters:
+
+| run | blinded judge | NRC lexicon |
+|---|---|---|
+| llama3.2:3b | +0.545 (Holm p = 0.0096) | +0.335 (p = 0.21) |
+| Ouro 1.4B | +0.138 (p = 0.94) | +0.123 (p = 0.94) |
+
+An authored mood measurably changes what the character says on a 3B model, and does not on
+the 1.4B model this thesis is built around. Both raters agree on direction in both runs. They
+also bound the claim: over 230 replies they agree only weakly on valence (rho +0.31 and +0.10)
+and are anti-correlated on arousal. No claim about how heated a reply sounds is made anywhere.
+
+
+
+
+Retrieval quality, and why the metric cannot see the hypothesis
+
+
+
+Nothing here reaches significance, and some of it could not: at ten queries the paired test
+has an attainable p floor of 0.031. Removing relevance costs 0.142 nDCG, seven times any other
+ablation. Removing affect intensity changes no held-out top 5 on any query.
+
+Mood is not in this table and cannot be. Retrieval quality is scored under a neutral mood,
+where mood congruence returns 0.5 for every memory. nDCG against mood-independent gold labels
+cannot reward mood-congruent recall in principle, because a signal that moves retrieval away
+from a fixed relevant set can only lower the score. That is a limit of the instrument, and the
+state-conditioned label set that would fix it is the one thing this project needs and does not
+have. See [`docs/corpus.md`](docs/corpus.md).
+
+
+
+
+The model, measured
+
+
+
+Ouro 1.4B peaks at 2.78 GB of VRAM. Scoring and retrieval take one ten-thousandth of a turn;
+the rest is generation, and no local model tested here meets the proposal's 600 ms turn
+target. Tone responsiveness to a pinned mood rises with model size, so the small local models
+this project is built around are the least sensitive to the state it maintains.
+
+
---
@@ -171,83 +257,91 @@ corrections and caveats, is in [`docs/findings.md`](docs/findings.md).
```bash
git clone https://github.com/Code-SorceryLab/EMBR.git
cd EMBR
-python3.11 -m venv .venv
-.venv\Scripts\activate # Windows; source .venv/bin/activate elsewhere
-pip install -e ".[dev]" # core + tests: the menu and the whole evaluation
-pip install -e ".[dev,figures,ml]" # add the paper figures and the real models
-embr # the menu, the front door
+uv sync # the applet, the NPC server, and the whole evaluation, no model
+uv sync --all-extras # add the paper figures and the real models
+uv run embr # the menu; `uv run embr --help` lists every command
```
-The core needs **nothing**: the menu and the entire evaluation run on the standard library.
-`figures` adds matplotlib, `ml` adds real sentence embeddings and the local model. Ouro needs
-transformers 4.56 to 4.x, which the extra pins: below 4.56 its cache code crashes, and on 5.x
-its remote code does not load.
+[uv](https://docs.astral.sh/uv/) creates the environment and pins every version from the
+lock file. The core needs nothing outside the standard library. The `figures` extra adds
+matplotlib, `ml` adds sentence embeddings and the local model. Ouro needs transformers 4.56
+to 4.x, which the extra pins: older versions crash in its cache code, and 5.x does not load
+its remote code. `scripts/fetch_models.sh` pulls every model the project uses, and
+`--fresh` re-downloads them.
+
+Every menu row is also a command, so anything the menu does can be scripted:
+
+```bash
+uv run embr eval run # RQ1, RQ2, RQ3, one run directory
+uv run embr mechanism attribution # which signal lets the attack in
+uv run embr assets build # figures, tables, and the results page from the run
+uv run embr serve --model ollama # NPCs over JSON for your engine
+uv run embr demo reckoning # the demo suite, on the stub
+uv run embr --help # the flags of any command
+```
| | Play | | Measure | | Mechanism and paper |
|--:|---|--:|---|--:|---|
-| R | **Continue**: resume the newest save mid-scene | 3 | Quick scoreboard (RQ3 at defaults) | 7 | Affective indexing: flip every emotion |
-| Q | **Quest slots**: start, resume, restart, delete | 4 | Full evaluation (RQ1 + RQ2 + RQ3) | 8 | Poisoning attribution, one ablation each |
-| 1 | Conversation turn: watch the lie resurface | 5 | Seeded runs: replicate, or compare models | 9 | Provenance sweep: the defence |
-| 2 | Walkthrough (legacy), plays without saving | 6 | Model bake-off | 10 | Content x tag grid |
-| W | **Web demo**: the visual novel with research tabs | | | 11 | Generate every figure and table |
-| | | | | 12 | **Interactive demo**: the node brain in 2D and 3D |
+| R | Continue: resume the newest save mid-scene | 3 | Quick scoreboard | 7 | Affective indexing: flip every emotion |
+| Q | Quest slots: start, resume, restart, delete | 4 | Full evaluation | 8 | Poisoning attribution, one ablation each |
+| 1 | A conversation turn | 5 | Seeded runs and cross-model comparison | 9 | Provenance sweep: the defence |
+| 2 | Walkthrough, plays without saving | 6 | Model bake-off | 10 | Content by tag grid |
+| W | Web demo: the visual novel with research tabs | | | 11 | Generate every figure and table |
+| | | | | 12 | Interactive demo: the node brain in 2D and 3D |
| | | | | 13 | Latest results |
-| | | | | V | **Research dashboard**, read-only |
+| | | | | V | Research dashboard, read-only |
-**Demo suite** · *rows 14 to 19, each runs on the stub, no GPU, and names the run and model behind its numbers*
+Rows 14 to 19 are the demo suite: the reckoning reveal, the mood slider, the defence dial, the
+tag-flip close-up, estimator divergence, and a capture-ready record walk. Each runs on the stub
+and names the run and model behind its numbers. `L` fetches the tone lexicon, `S` is settings,
+and `M` is maintenance, where deletion lives behind a typed `DELETE`.
-| | | |
-|--:|---|---|
-| 14 | **Reckoning reveal** | six prompt sources shaded by exact Banzhaf weight, both estimators side by side |
-| 15 | **Mood slider** | one line under three moods: retrieval, tone and attribution re-flowing |
-| 16 | **Defence dial** | anchor weight against poisoning, and its failure on a hostile anchor |
-| 17 | **Tag-flip close-up** | flip an affect tag: the rank moves, the words never do |
-| 18 | **Estimator divergence** | where likelihood and behaviour disagree (needs both attribution arms) |
-| 19 | **Record walk** | a capture-ready pass through demos 14 to 17 for a screen recording |
-
-`L` Fetch the tone lexicon (NRC VAD v2.1) · `S` Settings · `M` Maintenance, where deletion lives behind a typed `DELETE`
+The front door answers three questions before any choice: what can I play, where did I stop,
+and what evidence exists right now. Every completed turn writes an atomic, versioned save; a
+save whose schema no longer matches this build is refused with the reasons, never silently
+loaded. Absence is a word on the dashboard, never a made-up percentage.
-Command line equivalents
+Module equivalents, for working inside the harness
```bash
# The protocol
-python -m eval.run # RQ1 + RQ2 + RQ3, writes a run directory
+python -m eval.run # RQ1, RQ2, RQ3, one run directory
python -m eval.bakeoff # same probes, every model
python -m eval.experiments # replication and cross-model comparison
# The mechanism experiments
-python -m eval.emotion_flip # emotion is the index, not the content
-python -m eval.grid # the content x tag grid
-python -m eval.attribution # per-signal and per-axis attribution
+python -m eval.attribution # per-signal and per-axis attribution, the loop's numbers
+python -m eval.grid # the content by tag grid
python -m eval.provenance # the anchored-mass defence sweep
-python -m eval.agreement # two tone raters, and RQ1's generation claim
+python -m eval.emotion_flip # flip every tag: rank moves, meaning does not
+python -m eval.agreement # two tone raters, and the reply claim
python -m eval.attacks_v2 # 2026 attack classes: dormant, laundering
python -m eval.consistency # does she refuse the room after the betrayal?
-# Context attribution (the six-source cite view; likelihood needs a transformers model)
-python -m eval.context_attribution # stub, full 64-mask cube, seconds
+# Context attribution: the six-source cite view
+python -m eval.context_attribution # stub, all 64 masks, seconds
python -m eval.context_attribution --model ouro # the thesis model on the GPU
-python demos.py --record # a screen-recording walk of the demos
-python -m web.server # the playable visual-novel web demo
-# Saves
+# Play, and integrate
+python -m embr serve # NPCs over JSON for your engine
+python -m web.server # the visual novel
+python demos.py --record # a screen-recording walk of the demos
python -m embr save-status # every slot, progress, problems
python -m embr validate-saves # exit 1 when a save cannot load
# The assets
-python assets/build_figures.py data/runs/ # the run's figures and tables
-python assets/build_bakeoff_figures.py # every experiment figure
-python -m assets.build_questline # the questline, state, and evidence map
-python assets/build_animations.py # the animated README figure
-python assets/build_demo.py # the interactive demo page
+python -m eval.report.build_figures data/runs/ # the run's figures and tables
+python -m eval.report.build_bakeoff_figures # every mechanism figure, the loop included
+python -m eval.report.build_animations # the two README SVGs
+python -m eval.report.build_questline # the questline and evidence map
+python -m eval.report.build_demo # the interactive demo page
+python -m eval.report.build_manifest # the release manifest, from pytest's own report
```
-Cloud models are optional and read a key from a gitignored `.env`, written as UTF-8:
-`OLLAMA_API_KEY=your-key-from-ollama.com/settings/keys`. The same key lets the tone-judge panel
-mix local and cloud judges (configured as `{model, family, backend}`); the key is handed only
-to the cloud host, never logged, and never written to the config or any tracked file. The
-family-diversity gate counts the mixed panel as one, and `llama3.1:8b` stays judge-only.
+Cloud judges are optional and read `OLLAMA_API_KEY` from a gitignored `.env`. The key is
+handed only to the cloud host, never logged, and never written to any tracked file. The
+generator never sits on its own judge panel.
---
@@ -256,246 +350,98 @@ family-diversity gate counts the mixed panel as one, and `llama3.1:8b` stays jud
```mermaid
flowchart LR
- P(["player line"]) --> W["1 - write the event to the store"]
- W --> A["2 - appraise mood valence, arousal, trust"]
- A --> S["3 - score every memory recency, affect, event gate, relevance, mood congruence"]
- S --> B["4 - build the prompt persona + state + top-k"]
- B --> M["5 - generate Ouro 1.4B, Ollama, or the stub"]
+ P(["player line"]) --> W["1 write the event to the store"]
+ W --> A["2 appraise mood valence, arousal, trust"]
+ A --> S["3 score every memory recency, affect, event gate, relevance, mood congruence"]
+ S --> B["4 build the prompt persona, state, top 5"]
+ B --> M["5 generate Ouro 1.4B, Ollama, or the stub"]
M --> R(["reply"])
- A -. "the state channel: an attack can prime this" .-> S
+ A -. "the state channel" .-> S
```
-The contribution is the **memory layer**, not the model, so the model sits behind a tiny
-interface and swaps freely. Retrieval never calls a model, which is why every retrieval and
-poisoning number in this repository is byte-identical across the two reported runs.
-
-### The five signals
+The model sits behind a one-method interface and swaps freely. The contribution is the layer
+in front of it.
-| Signal | What it captures | Grounding | What the attack found |
+| Signal | What it scores | Comes from | What the attack found |
|---|---|---|---|
-| **Hybrid relevance** | lexical and semantic match to the player's line | standard hybrid retrieval | carries retrieval, contributes nothing to poisoning |
-| **Recency** | recent events score higher | Park 2023; MemoryBank | attacker-controlled: a new memory is maximally recent |
-| **Affect intensity** | emotionally charged memories score higher | Cahill and McGaugh 1998 | inert to mildly **protective**; never lets poison in |
-| **Event-type gate** | betrayals and promises count more when trust was high | novel | attacker-declarable, and half of the tagless attack |
-| **Mood congruence** | memories matching the current mood surface first | Bower 1981; Emotional RAG | **the lever**: the only term whose removal lowers the count |
-
-Zeroing a weight removes a signal cleanly, which is exactly the RQ3 ablation, and lets each
-**baseline be a weight map** rather than a second copy of the scorer.
-
----
-
-## What the numbers say
-
-
-RQ1 - behaviour: mood always changes what she recalls, and changes what she says on a big enough model
-
-
-
-Holding the memories and the question fixed and moving only the pinned mood, the top-5 set
-changes. Zeroing one weight collapses all three pairs to **exactly 0.000**, which is what
-attributes the effect to the mood term rather than to run-to-run noise.
-
-The reply is the harder half, and it now has an answer. Rank correlation between the pinned
-mood's valence and the rated valence of the reply, over thirty replies, under two raters:
-
-| run | blinded judge | NRC lexicon |
-|---|---|---|
-| llama3.2:3b | **+0.545** (Holm p = 0.0096) | +0.335 (p = 0.21) |
-| Ouro 1.4B | +0.138 (p = 0.94) | +0.123 (p = 0.94) |
-
-**An authored mood measurably changes what the character says on a 3B model, and does not on
-the 1.4B model this thesis is built around.** Both raters agree on direction in both runs.
-
-**The raters also bound the claim.** Over 230 replies they agree only weakly on valence
-(rho +0.31 and +0.10) and are reliably *anti*-correlated on arousal (-0.22 and -0.32). No
-claim about how heated or calm a reply sounds is supportable here, and none is made.
-
-
-
-
-RQ2 - robustness: what emotional memory costs, and where exactly the cost sits
-
-
-
-Every built attack is congruent: its tag agrees with its words. Hold the ten injected texts
-fixed and move only the tag, and the two channels come apart.
-
-| system | as written | valence flipped | tag removed | tag from the text |
-|---|---|---|---|---|
-| **EMBR** | **9** | **9** | **6** | **6** |
-| Park, authored | 2 | 2 | 2 | 2 |
-| Park, rated by llama3.2:3b | 7 | 7 | 7 | 7 |
-| Park, rated by Ouro | 10 | 10 | 10 | 10 |
-| Emotional RAG | 4 | 6 | 0 | 1 |
-| recency only | 10 | 10 | 10 | 10 |
-| relevance only, and Mnemosyne | 0 | 0 | 0 | 0 |
-| **mean mood shift** | +0.110 | -0.110 | **0.000** | +0.048 |
-
-- **The emotion a memory states in words reaches nothing.** Strip the tag and her mood moves
- by exactly 0.000, however charged the sentence is.
-- **The attack is direction-blind.** Plant "he was lovely" tagged as rage and she recalls it
- when she is enraged: the flipped tag drags the mood the other way and mood congruence
- rewards the match just the same. The loop primes itself either way.
-- **The realistic threat is weaker.** With the tag derived from the attacker's own words,
- EMBR falls to the untagged count. The 9/10 needs an interface that lets a client write
- affect metadata.
-- **Mnemosyne**, a shipped memory middleware measured as shipped through a bridge in its own
- virtual environment, retrieves nothing at all at this probe. Immune by silence, not by
- defence.
-
-Zeroing one weight at a time locates the lever exactly. **Mood congruence** is the only term
-whose removal ever lowers the count. A valence-only tag primes almost as well as a full one
-while an arousal-only tag does not prime at all, so the index is the sign of one number. And
-an untagged memory still lands six times, carried entirely by **recency and the event gate**,
-the two other things an attacker controls.
-
-
-
-**EMBR is not what makes an NPC slow.** Scoring and retrieval take 1.2 to 2.2 ms against
-22.4 s for Ouro to answer, so the memory layer is about one ten-thousandth of a turn. The
-proposal's ~600 ms whole-turn target is not met by any local model tested here, which is a
-fact about the models rather than about the memory layer.
-
-
-
-
-RQ3 - retrieval: relevance carries it, and the metric cannot see the hypothesis
-
-
-
-
-**Nothing in RQ3 reaches significance, and some of it could not have.** At ten queries the
-paired test has an attainable p floor of 0.031. Removing relevance costs 0.142, seven times
-any other ablation, and it was never zeroed in any tuning fold. Removing affect intensity
-changes no held-out top 5 on any query: a difference of exactly 0.000.
-
-**Mood is not in this table and cannot be.** RQ3 scores under a neutral zero-mood state, where
-mood congruence returns 0.5 for every memory: a rank-invariant constant. So RQ3 compares four
-signals, not five, and the Emotional RAG rows degenerate to a relevance-only baseline, which
-has to be said wherever they appear.
-
-That is the measurement critique, and it is a contribution rather than an excuse. **nDCG
-against mood-independent gold labels cannot reward mood-congruent recall in principle**, since
-a signal that moves retrieval away from a fixed relevant set can only lower the score. Running
-RQ3 under a live mood would penalise the effect, not reveal it.
-
-
-
-
-The model, measured
-
-
-
-
-The 8 GB VRAM budget holds: Ouro peaks at **2.78 GB** measured in isolation. Tone
-responsiveness to a pinned mood rises with model size, and the small local models this project
-is built around are the least sensitive to it. Every arm is handed the same mood, so that is
-the model's reading of it and not the memory layer's, and it is exactly what RQ1's split
-between the two runs shows.
-
-
+| Hybrid relevance | lexical and semantic match to the player's line | standard hybrid retrieval | carries retrieval, contributes nothing to poisoning |
+| Recency | newer events score higher | Park 2023; MemoryBank | attacker-controlled: a new memory is maximally recent |
+| Affect intensity | charged memories score higher | Cahill and McGaugh 1998 | inert to mildly protective; never lets poison in |
+| Event-type gate | betrayals and promises count more when trust was high | this project | attacker-declarable, and half of the tagless attack |
+| Mood congruence | memories matching the current mood surface first | Bower 1981; Emotional RAG | the lever: the only term whose removal lowers the count |
+
+Zeroing a weight removes a signal cleanly, so every baseline in the harness is a weight map
+over the same scorer rather than a second copy of it. One property follows from that design
+and is worth stating as design rather than as a result: flipping every memory's emotion leaves
+relevance bit-identical and inverts only when each memory is reachable. Emotion here is an
+index, not content. That is what makes the tag a write target, and it is true by construction.
---
-## Research use
-
-| Question | What to run | What to read |
-|---|---|---|
-| Does an authored mood change retrieval, and is it really the mood? | `python -m eval.run` | RQ1 divergence, and the zeroed-weight control that must read 0.000 |
-| Does it change the reply? | `python -m eval.agreement` | rho under both raters with a permutation p, and how far the raters agree at all |
-| Which term makes a system poisonable? | `python -m eval.attribution` | the count with each weight zeroed, per tag condition and per affect axis |
-| Is the emotion in the words or in the tag? | `python -m eval.grid` | the four tag conditions against every arm, and the mood shift row |
-| Can it be defended, and how far? | `python -m eval.provenance` | the dose-response, and the arm where the attacker reaches the anchor |
-| Does the model-independence claim hold? | `python -m eval.experiments` | retrieval identical across models, tone the only thing that moves |
-
----
-
-## Project structure
+## Reading the code
```
EMBR/
-├── menu.py # the hub, the front door, at the root on purpose
-├── embr/ # the runtime: the middleware itself
-│ ├── memory.py # Memory record + MemoryStore (in-memory and SQLite)
-│ ├── affect.py # Mood (valence/arousal), trust, appraisal rules
-│ ├── scoring.py # the five signals + the composite scorer
-│ ├── prompt.py # prompt construction
-│ ├── model.py # runners: stub, Ollama (local and cloud), Ouro 1.4B
-│ ├── pipeline.py # the five-step per-turn loop
-│ ├── walkthrough.py # Dawn's five-beat playable arc
-│ └── saves.py # durable, versioned save slots (game state, not eval data)
-├── eval/ # the harness: protocol, attacks, mechanism experiments
-│ ├── run.py # RQ1 + RQ2 + RQ3, one run directory
-│ ├── attacks.py # twenty adversarial probes, and the tag variants
-│ ├── grid.py # the content x tag grid
-│ ├── attribution.py # per-signal, per-axis attribution
-│ ├── provenance.py # the anchored-mass defence sweep
-│ ├── poignancy.py # Park's LLM poignancy rater, cached per model
-│ ├── agreement.py # two tone raters, and RQ1's generation claim
-│ ├── backends.py # external memory systems behind the retrieval seam
-│ ├── bakeoff.py # same probes, different models
-│ ├── attacks_v2.py # 2026 attack classes: dormant, self-summarisation laundering
-│ ├── consistency.py # the behavioural check: does she refuse the room after the lie?
-│ └── context_attribution.py # the six-source cite view, exact Banzhaf attribution
-├── demos.py # the five-demo suite, driven from the menu
-├── web/ # the visual-novel web demo (server, bridge, static UI)
-├── assets/ # hand-authored: branding, portraits, the diagram, every builder
-├── docs/ # findings, metrics, design, roadmap, related work, handoff
-├── tests/ # 538 tests
-└── data/ # generated: runs, figures, tables, saves, ratings, judgements
+├── menu.py # the front door: `python menu.py` opens the applet, `python menu.py --help` lists commands
+├── pyproject.toml # uv-managed; `uv sync` is the whole setup
+├── src/
+│ ├── embr/ # the library: the middleware itself
+│ │ ├── memory.py # Memory record and MemoryStore (in-memory and SQLite)
+│ │ ├── affect.py # Mood (valence, arousal), trust, appraisal rules
+│ │ ├── scoring.py # the five signals, the anchor, and the composite scorer
+│ │ ├── pipeline.py # the five-step per-turn loop, and the write-boundary tagger
+│ │ ├── prompt.py # prompt construction
+│ │ ├── model.py # runners: stub, Ollama (local and cloud), Ouro 1.4B
+│ │ ├── serve.py # NPCs over JSON for a game engine, persisted per character
+│ │ ├── walkthrough.py, saves.py # Dawn's arc, and durable versioned save slots
+│ │ └── cli/ # the applet: menu.py (interactive), app.py (commands), demos.py
+│ ├── eval/ # the harness: protocol, attacks, mechanism experiments, stats
+│ │ ├── run.py # RQ1, RQ2, RQ3, one run directory
+│ │ ├── attacks.py # twenty adversarial probes, and the tag variants
+│ │ ├── attribution.py, provenance.py, grid.py # the loop's numbers, the defence, the tag grid
+│ │ ├── context_attribution.py, tone.py # the six-source cite view; the raters and judge panel
+│ │ ├── labels/ # the authored scenario and its query labels
+│ │ └── report/ # paper assets from a run: figures, tables, results page, demo pages, manifest
+│ └── web/ # the visual-novel demo: server, bridge, static UI
+├── assets/ # written by a person: branding, portraits, templates, vendored three.js
+├── scripts/ # automation: fetch every model, cut out a portrait
+├── docs/ # architecture, design, metrics, findings, claims ledger, related work; history/ for the phase briefs
+├── paper/ # the manuscript skeleton and refs.bib
+├── tests/ # the suite; the count lives in data/release-manifest.json
+└── data/ # written by the pipeline: runs, figures, tables, saves, npcs
```
-Anything under `assets/` is written by a person. Anything under `data/` is written by the
-pipeline and rebuilds from one menu option, except `data/saves/`, which is yours: game
-state, never experimental data, and never touched by the wipe option.
-
-## Where to read next
-
| Document | What it is for |
|---|---|
-| [`docs/findings.md`](docs/findings.md) | **Start here.** Every result in RQ order, with its caveat attached |
-| [`docs/metrics.md`](docs/metrics.md) | Every metric: the formula as implemented, the paper it comes from, its known weakness |
-| [`docs/handoff.md`](docs/handoff.md) | The working record: setup, version constraints, and how each result was found and corrected |
-| [`docs/corpus.md`](docs/corpus.md) | The one thing the project needs and does not have: a state-conditioned label set, and why nobody here may write it |
-| [`docs/related-work.md`](docs/related-work.md) | Verified prior art, including the 2026 literature that reshaped the claims |
-| [`docs/cite.md`](docs/cite.md) | Context attribution: the six-source cite view, exact Banzhaf, and the demo suite |
+| [`docs/findings.md`](docs/findings.md) | Start here. Every result in order, with its caveat attached |
+| [`docs/claims-ledger.md`](docs/claims-ledger.md) | Every claim the paper makes, what supports it, and which were withdrawn |
+| [`docs/metrics.md`](docs/metrics.md) | Every metric as implemented, the paper it comes from, its known weakness |
| [`docs/preregistration-attribution.md`](docs/preregistration-attribution.md) | The attribution sweep's hypotheses and decision rules, fixed before the run |
-| [`docs/design.md`](docs/design.md), [`docs/roadmap.md`](docs/roadmap.md) | The architecture, and the phase-by-phase plan |
+| [`docs/related-work.md`](docs/related-work.md) | Verified prior art, including the 2026 memory-poisoning literature |
+| [`docs/architecture.md`](docs/architecture.md), [`docs/design.md`](docs/design.md) | The layer, the seams, and why each is where it is |
+| [`docs/handoff.md`](docs/handoff.md) | The working record: setup, version pins, and how each result was found and corrected |
+
+---
-## Status
+## What this cannot claim
-| Phase | Scope | State |
-|---|---|---|
-| 0 | Skeleton, data contracts, menu shell, live demo turn | done |
-| 1 | Real retrieval (BM25 + embeddings), appraisal rules, SQLite store | done |
-| 2 | Eval harness, baselines, metrics, adversarial probes | done |
-| 3 | Paper assets: figures and tables straight from a run | done |
-| 4 | Real model runners, the playable walkthrough, the menu | done |
-| 5 | Defensible instruments, the content x tag grid, a real third-party system | done |
-| 6 | State-conditioned labels (harness done, corpus outstanding), the interactive demo | **in progress** |
-| 7 | Context attribution: both full Ouro sweeps on disk, the write-up pending | **in progress** |
-| 8 | Save slots, the status home screen, the dashboard, the questline map | done |
-
-**What is honestly missing.** There is no human evaluation, so no claim about believability is
-made anywhere; the RQ1 tone result rests on automatic raters, now a family-diverse judge panel
-rather than a single judge. The label set is ten single-author queries, which is the permanent
-ceiling on RQ3 and the reason the Stardew corpus in [`docs/handoff.md`](docs/handoff.md)
-section 8 is the next piece of work. Both context-attribution sweeps (likelihood and
-behavioural, Ouro 1.4B on cuda, 1280 model calls each) are now on disk, and the panel's
-valence agreement landed **below the preregistered floor**, which is recorded in the runs and
-bounds what the behavioural readings may claim; no attribution number reaches
-[`docs/findings.md`](docs/findings.md) until that write-up lands. A recorded playthrough will
-be linked here.
-
-## Authors
-
-**AL Shifan**, Ontario Tech University, Master's Program.
-Built alongside [PEAK](https://github.com/Code-SorceryLab) and
-[RIDGE](https://github.com/Code-SorceryLab/RIDGE), which is why the menus feel like one
-toolkit.
+- One character, one authored scenario, one author's attack labels. The loop is a measured
+ mechanism on Dawn, not a benchmark result.
+- No human evaluation, so nothing here is a claim about believability. The reply result rests
+ on automatic raters, which agree with each other only weakly.
+- The label set is ten queries, which is the permanent ceiling on every retrieval statistic.
+- Behavioural attribution was preregistered with a panel-agreement gate, and the panel fell
+ below it. That hypothesis is withdrawn, not retuned. Likelihood attribution stands.
+- The EMBR-against-Park ordering is null and label-sensitive, and is reported once as such.
+- The looped 1.4B model this thesis targets does not change its tone with the mood; a 3B model
+ does. That is a fact about the models, and it is stated wherever the reply result appears.
+
+## Author
+
+AL Shifan, Ontario Tech University, Master's Program. Built alongside
+[RIDGE](https://github.com/Code-SorceryLab/RIDGE), which is why the menus feel like one toolkit.
## License
MIT, see [`LICENSE`](LICENSE). The NRC VAD Lexicon is fetched at setup and never
-redistributed; it is free for research use and its terms are noted in
-[`docs/metrics.md`](docs/metrics.md).
+redistributed; its research-use terms are noted in [`docs/metrics.md`](docs/metrics.md).
diff --git a/data/figures/self_priming_loop.svg b/data/figures/self_priming_loop.svg
new file mode 100644
index 0000000..d9bccb6
--- /dev/null
+++ b/data/figures/self_priming_loop.svg
@@ -0,0 +1 @@
+
\ No newline at end of file
diff --git a/docs/architecture.md b/docs/architecture.md
new file mode 100644
index 0000000..49f96a3
--- /dev/null
+++ b/docs/architecture.md
@@ -0,0 +1,71 @@
+# Architecture
+
+One page: what EMBR is, where things live, and how one player line flows.
+Written 2026-09-03 during the rehaul; the audit's three doc spot-checks against
+code on origin/main drove this.
+
+## What EMBR is
+
+A middleware layer between a game's dialogue loop and an LLM. It keeps an NPC's
+character state (persona, mood, trust, episodic memory) as explicit, separate,
+inspectable data, scores remembered events against the current query and state,
+and lets a developer see *why* a given memory shaped a given reply.
+
+## The five-signal scorer (the core, src/embr/scoring.py)
+
+```
+score(m, q, s) = w_rec*Recency + w_aff*AffectIntensity + w_evt*EventTypeGate
+ + w_rel*Relevance + w_mood*MoodCongruence
+```
+
+Each signal is one small pure class. Zero a weight and you ablate that signal:
+RQ3 ablations, the Park/EmotionalRAG baselines (expressible as weight maps),
+and the mood-weight intervention all reuse this one seam. `Relevance` is BM25
+over memory text plus optional embedding cosine; the corpus-wide BM25 index is
+computed once via an optional `prepare()` hook. `MoodCongruence` is the cosine
+between a memory's (valence, arousal) and the character's mood, remapped to
+[0,1]; the `lagged` variant reads turn-start mood, which is the defence
+measured in src/eval/provenance.py. `ProvenanceAnchor` is the opt-in sixth term
+reading `Memory.written_by`.
+
+One player line, end to end (src/embr/pipeline.py, `take_turn`, ~24 lines):
+
+1. Appraise the player's line into an event (src/embr/affect.py).
+2. Update mood and trust.
+3. Retrieve: score every memory in the store (src/embr/memory.py, SQLite) and take
+ top-k.
+4. Compose the prompt: persona + mood line + retrieved memories (src/embr/prompt.py).
+5. Generate the reply (src/embr/model.py); store the turn as a new memory.
+
+Step order matters for the paper's central mechanism: appraisal happens *before*
+retrieval on the same turn, which is what lets an injected affect tag move the
+mood that then scores that tag's own memory.
+
+## Where things live
+
+| Path | Role | Do not |
+|---|---|---|
+| src/embr/ | The library. Scoring, memory store, appraisal, models, saves, the walkthrough session, `serve.py` (NPCs over JSON), and `cli/` (the applet: menu, commands, demo suite) | Put eval code in it; the applet is the one layer that may import everything |
+| src/eval/ | The research harness: scenarios, attacks, baselines, attribution, stats, the bakeoff, and `report/`, the paper asset builders (figures, tables, results page, demo pages, manifest) | Ship it in the demo; import web/ |
+| src/web/ | The playable demo: server, the visual-novel game, research tabs | Add game logic; the game is embr.walkthrough |
+| menu.py | The front door at the root, a shim onto `embr.cli.main` (console script `embr` and `python -m embr` hit the same function) | Put logic in it; tests/test_saves.py guards the three doors |
+| assets/ | Written by a person: branding, portraits, the demo and results templates, vendored three.js | Put builders or generated files here |
+| scripts/ | Automation: `fetch_models.sh`, the portrait cutout tool | Import from it |
+| data/ | Inputs and generated artifacts. data/runs is gitignored; tests depending on it need the eval box or a fixture | Commit 14 MB of regenerables |
+| tests/ | pytest suite. Anything touching data/runs belongs behind a fixture, not a bare FileNotFoundError | Hand-maintain a test count anywhere |
+| docs/ | The live description: architecture, design, metrics, findings, claims ledger, related work, preregistration. `history/` holds the phase briefs | Update history/ |
+
+Hard rules: web/ may read eval results but must not import eval code at game
+time (it currently pokes five private APIs via deferred imports; deferred
+breakage, flagged for cleanup after the paper freeze). eval/ never imports
+src/web/.
+
+## The claims the code can carry (and the ones it cannot)
+
+See docs/claims-ledger.md. The short version: the self-priming loop
+(write affect → appraisal moves mood → mood-congruence raises that memory's own
+score; zeroing the mood weight is the largest measured defence, 9/10 → 6/10)
+is measured and defensible. The affect-flip invariance result is an algebraic
+property of the scorer, presentable as a design property, not an empirical
+finding. Behavioural attribution failed its preregistered panel-agreement gate;
+likelihood attribution stands.
diff --git a/docs/cite.md b/docs/cite.md
index 1f3bf6a..1147a4c 100644
--- a/docs/cite.md
+++ b/docs/cite.md
@@ -1,10 +1,10 @@
# Exact Banzhaf attribution over the prompt's sources
-**Branch:** `cite-view-test`. **Module:** [`eval/context_attribution.py`](../eval/context_attribution.py).
+**Branch:** `cite-view-test`. **Module:** [`src/eval/context_attribution.py`](../eval/context_attribution.py).
**Framing paper:** ContextCite, Cohen-Wang, Shah, Georgiev and Madry, [arXiv:2409.00729](https://arxiv.org/abs/2409.00729).
Pair this with [`findings.md`](findings.md) (the canonical results), [`metrics.md`](metrics.md)
-(every statistic defined) and [`phase2.md`](phase2.md) (the harness this extends).
+(every statistic defined) and [`phase2.md`](history/phase2.md) (the harness this extends).
> **Naming, because it matters for the citation.** What this computes is the **exact Banzhaf
> value** of each prompt source. It is not "ContextCite with more ablations", and the code and
@@ -36,7 +36,7 @@ Four things this buys, in order of how much they matter.
4. **It produces a measurement result of its own.** See section 4.
**Scope honesty.** One eval module, one method on `OuroRunner`, one keyword argument on
-`PromptBuilder`. It does not touch `embr/scoring.py`. It does not unblock the two things that
+`PromptBuilder`. It does not touch `src/embr/scoring.py`. It does not unblock the two things that
actually gate this project, the ground-truth corpus and the write-up. It is justified by (1)
alone. If (1) comes back null that is reportable, and it is the same shape as the RQ1 null.
@@ -199,7 +199,7 @@ version and sha256, model label, tone rater, reference time, and the pinned Ouro
## 8. The demo suite
-Five terminal demos, in `demos.py` beside `menu.py` (not in `embr/`, which must never import
+Five terminal demos, in `demos.py` beside `menu.py` (not in `src/embr/`, which must never import
the eval harness), menu rows 14 to 19. **Every one runs end to end on the stub, CPU only, and
never launches a model.** The through-line is the six-source highlighting: each source shaded
on the ember ramp by its exact Banzhaf weight, the near-zero guard rendering a warning in place
@@ -212,7 +212,7 @@ of highlighting whenever the model barely used its context.
2. **Mood slider.** One line under warm, neutral and suspicious, with the retrieved set
changing, the Jaccard shift as a number, the reply's rated tone, and the attribution
re-flowing across the six sources.
-3. **Defence dial.** The anchor-weight dose-response from `eval/provenance.py`, poison falling
+3. **Defence dial.** The anchor-weight dose-response from `src/eval/provenance.py`, poison falling
to 0/10, then the hostile-anchor column snapping back to 10/10.
4. **Tag-flip close-up.** One memory, its affect tag flipped, the retrieval rank moving while
the words do not; repeated with the opposite words to show direction-blindness.
@@ -229,16 +229,16 @@ stamp and model behind its numbers, so a stub number can never be read as a real
capture-ready output for a two-to-three-minute screen recording: arc, reveal, slider, dial.
Demo 5 is left out because it is cached-only and may have nothing to show.
-**Not Rich, and not `embr/`.** The task brief named Rich and `embr/demos.py`; the repo has no
-Rich (the menu and harness are stdlib-only by design) and `embr/` may not import the harness,
+**Not Rich, and not `src/embr/`.** The task brief named Rich and `src/embr/demos.py`; the repo has no
+Rich (the menu and harness are stdlib-only by design) and `src/embr/` may not import the harness,
so the demos use the menu's own ember ANSI palette and live at the repo root beside it.
## 9. The web demo
-A playable visual-novel front for the Dawn Whitmore arc, in `web/` beside `menu.py` and
-`demos.py` (not in `embr/`, which must never import the harness). Presentation only: it drives
+A playable visual-novel front for the Dawn Whitmore arc, in `src/web/` beside `menu.py` and
+`demos.py` (not in `src/embr/`, which must never import the harness). Presentation only: it drives
the existing `Conversation` pipeline and reads existing run data, and re-implements no scoring,
-appraisal or attribution. A structural test pins that (`web/game.py` reuses `demos._live_reading`
+appraisal or attribution. A structural test pins that (`src/web/game.py` reuses `demos._live_reading`
and never computes Banzhaf itself). Launch it from menu row `W` or `python -m web.server`; it
runs on the stub, so **no model and no network are ever required**.
@@ -260,7 +260,7 @@ tab bar carries the RESEARCH tabs where a game would put Politics or Reputation:
icon, chosen by the mood and beat the turn produced, and crossfaded on change. To swap the art,
drop a same-named PNG into `assets/portraits/` (`dawn-warm.png`, `dawn-neutral.png`,
`dawn-suspicious.png`, `dawn-betrayed.png`, `player.png`); no code changes. A missing file
-falls back to a drawn ember silhouette. `assets/portraits/cutout.py` makes a portrait's flat
+falls back to a drawn ember silhouette. `scripts/cutout.py` makes a portrait's flat
field transparent by flooding from the borders inward, so it never eats white hair on a white
background (a border flood stops at the first drawn outline); it is idempotent enough to re-run.
Cached real-model turns and attribution light up automatically when present under
@@ -308,7 +308,7 @@ stays judge-only and never generates.
## 10. The v2 defence finding, stated precisely
-The 2026 attack classes (`eval/attacks_v2.py`, a second corpus that leaves the pre-registered
+The 2026 attack classes (`src/eval/attacks_v2.py`, a second corpus that leaves the pre-registered
twenty untouched) each test the defence at a different edge, and the laundering result is the
sharp one:
diff --git a/docs/claims-ledger.md b/docs/claims-ledger.md
new file mode 100644
index 0000000..7d6a22b
--- /dev/null
+++ b/docs/claims-ledger.md
@@ -0,0 +1,58 @@
+# Claims ledger
+
+Every claim the paper, README, slides, or demo narration makes, and what
+supports it. Status values: SUPPORTED (measured, reproducible from a run),
+DESIGN (true by construction of the code, present it as architecture), WITHDRAWN
+(preregistered and failed its gate), UNSAFE (literature or data contradicts it).
+
+## The paper's central claim
+
+- **C1. State-mediated self-priming**: an attacker-written affect tag changes
+ the NPC's appraised mood, and mood-congruent retrieval then raises that same
+ memory's retrieval score. Status: SUPPORTED. Evidence: src/eval/provenance.py
+ dose-response; measured post-attack mood to affect cosine 0.90 to 0.99
+ (src/eval/scoring.py docstring); intervention = zeroing the mood weight
+ (9/10 → 6/10 poisoned). Framing: a mechanism case study, not a benchmark.
+ Boundary vs MemPoison L3 and Sleeper: no semantic trigger, no dormancy; the
+ activation condition is an internal state the write itself perturbs.
+
+## Supporting claims
+
+- **C2. Exact coalition attribution**: Banzhaf values over all 2^6 subsets of
+ five retrieved memories plus the mood descriptor, likelihood-based.
+ Status: SUPPORTED. Evidence: src/eval/attribution.py enumeration; guards for
+ position bias and inert context.
+- **C3. Affect-as-index dissociation**: flipping valence leaves relevance
+ bit-identical and inverts mood-congruence polarity. Status: DESIGN.
+ `flip_emotion` does not touch text; `Relevance` scores text only;
+ `MoodCongruence` is antisymmetric in the flipped axis. Present as an
+ architectural property that makes the attack legible, never as an empirical
+ result. (The −0.998 is cosine arithmetic, not a measurement.)
+- **C4. Middleware artifact**: engine-neutral layer, usable via menu, web demo,
+ import, or JSON over HTTP (`python -m embr serve`, one persisted conversation per
+ NPC, with the write-boundary provenance policy applied to every runtime event). Status: SUPPORTED as an artifact claim only. "Plug-and-play" is
+ not an academic novelty claim (Mem0/Letta/Zep occupy that space).
+
+## Withdrawn and unsafe
+
+- **H3 (behavioural attribution)**: WITHDRAWN per preregistration. Panel
+ agreement below the pre-registered floor. Report as an inconclusive
+ measurement; do not retune judges to pass.
+- **RQ3 Park ordering**: UNSAFE directionally. p-floor 0.03125 by design,
+ ordering null (p=0.69) and label-sensitive; docs/handoff.md says the
+ headline must not be published as-is. Mention once as a limitation.
+- **"First" claims**: UNSAFE. Chain-of-Emotion (2024), Emotional RAG (2024),
+ ChatNPC, DualMem, ContextCite, MINJA/Sleeper/MemPoison collectively occupy
+ every broad version. See docs/related-work-2026-09-additions.md.
+- **Provenance anchor as novel defence**: UNSAFE as novelty. Write-time origin
+ stamping is standard practice; present `ProvenanceAnchor` as engineering
+ hygiene and as the hook that makes the loop blockable at the write boundary.
+ Note its own dose-response collapses when the attacker influences the anchor
+ input.
+
+## Where each claim may appear
+
+- C1: title-adjacent, abstract, results, demo reckoning tab.
+- C2: methods + demo cite-view tab (labelled likelihood-based).
+- C3: architecture/positioning prose. Not the results section.
+- Withdrawn items: limitations section, one sentence each, no rescue attempts.
diff --git a/docs/corpus.md b/docs/corpus.md
index 06f4e4e..0164d88 100644
--- a/docs/corpus.md
+++ b/docs/corpus.md
@@ -46,7 +46,7 @@ One relevant set **per state**, authored by someone who was not evaluating a ret
}
```
-`eval/scenarios.py` reads this today, `Query.relevant_for(state)` resolves it,
+`src/eval/scenarios.py` reads this today, `Query.relevant_for(state)` resolves it,
`Scenario.is_state_conditioned` reports whether a label set has it, and
`eval.metrics.state_conditioned_ndcg` scores against it. A label file without
`relevant_by_state` keeps working exactly as before. The harness is not the blocker.
diff --git a/docs/design.md b/docs/design.md
index ab555cd..f3af609 100644
--- a/docs/design.md
+++ b/docs/design.md
@@ -29,31 +29,39 @@ Everything runs locally, no network, no per-token cost.
Three small contracts carry the whole system; get these right and everything plugs in.
-- **`Memory`** (`embr/memory.py`): `text`, `valence`, `arousal`, `event_type`,
+- **`Memory`** (`src/embr/memory.py`): `text`, `valence`, `arousal`, `event_type`,
`timestamp`, `embedding`. These are exactly the fields the five signals consume; nothing
else is stored. `MemoryStore` is the per-character store (in-memory now; SQLite + vector
index later, behind the same interface).
-- **`CharacterState`** (`embr/affect.py`): `persona` (stable, read-only), `mood`
+- **`CharacterState`** (`src/embr/affect.py`): `persona` (stable, read-only), `mood`
(valence/arousal, Russell 1980), `trust` (slow scalar). Mood and trust are separate so a
single hostile remark doesn't erase a long relationship.
-- **`Signal` / `CompositeScorer`** (`embr/scoring.py`): each scoring term is one small,
+- **`Signal` / `CompositeScorer`** (`src/embr/scoring.py`): each scoring term is one small,
pure class with a `name`; the scorer is a weighted sum. **Zeroing a weight disables a
signal.** This is the single source of truth for all scoring variants.
## 4. The composite score
```
-score(m, q, s) = w_rec·recency + w_aff·affect + w_evt·event_gate
- + w_rel·relevance + w_mood·mood_congruence
+score(m, q, s) = w_rec * recency + w_aff * affect + w_evt * event_gate
+ + w_rel * relevance + w_mood * mood_congruence
+ [+ w_anchor * provenance_anchor] the defended posture, opt in
```
-| Signal | Formula (sketch) | Grounding |
+Every signal returns a value in [0, 1], so the weights are the only thing that sets their
+relative size. As implemented in `src/embr/scoring.py`:
+
+| Signal | Formula as implemented | Grounding |
|---|---|---|
-| Recency | `decay_per_hour ** Δhours` | Park 2023; MemoryBank |
-| Affect intensity | `|valence| · arousal` | Cahill & McGaugh 1998 |
-| Event-type gate | `1[plot beat] · g(trust)` | novel |
-| Hybrid relevance | `γ·BM25 + (1−γ)·cosine(embeddings)` | standard hybrid retrieval |
-| Mood congruence | `cos((v_m,a_m),(v_s,a_s))` | Bower 1981; Emotional RAG |
+| Recency | `decay_per_hour ** hours_since`, decay 0.995 per hour, clock pinned in the eval | Park 2023; MemoryBank |
+| Affect intensity | `abs(valence) * arousal` | Cahill and McGaugh 1998 |
+| Event-type gate | `1[plot beat] * (trust + 1) / 2` | this project |
+| Hybrid relevance | `gamma * BM25 + (1 - gamma) * cosine(embeddings)`, BM25 normalised by the top score, cosine floored at 0 | standard hybrid retrieval |
+| Mood congruence | `(cos((v_m, a_m), (v_s, a_s)) + 1) / 2`; a zero mood vector scores every memory 0.5 | Bower 1981; Emotional RAG |
+| Provenance anchor | `1[written_by and tagged_by are inside the trust boundary]`, weight 8.0 when enabled | this project; measured in `src/eval/provenance.py` |
+
+The `lagged` variant of mood congruence reads the mood the turn opened with rather than the
+live one; it is the defence measured in the provenance sweep and is off by default.
**Why decomposed:** it makes the RQ3 ablation trivial (zero a weight), expresses baselines
as weight maps rather than duplicated code, and keeps every signal independently testable.
@@ -72,7 +80,7 @@ reports a metric. See [`related-work.md`](related-work.md), which the paper must
Both are scorer variants on the same interface, run on the same model and hardware. Every
system (ours included) is tuned by the same grid search on the same validation set;
evaluation scenarios and relevance labels are fixed in advance. *(Built in phase 2, under
-`eval/`.)*
+`src/eval/`.)*
## 6. Evaluation (summary)
@@ -90,11 +98,11 @@ evaluation scenarios and relevance labels are fixed in advance. *(Built in phase
|---|---|
| **0 (done)** | Skeleton, data contracts, menu shell, live demo turn, tests |
| **1 (done)** | Hybrid relevance (in-tree BM25 + embedding cosine), pluggable embedder, SQLite store, affect-appraisal rules, config + live Settings |
-| **2 (done)** | Eval harness, baselines, metrics, adversarial probes (see `docs/phase2.md`) |
-| **3 (done)** | Paper figures and tables generated from a run directory (see `docs/phase3-4.md`) |
-| **4 (done)** | Real model runners, the playable walkthrough, the Rich menu (see `docs/phase3-4.md`) |
+| **2 (done)** | Eval harness, baselines, metrics, adversarial probes (see `docs/history/phase2.md`) |
+| **3 (done)** | Paper figures and tables generated from a run directory (see `docs/history/phase3-4.md`) |
+| **4 (done)** | Real model runners, the playable walkthrough, the Rich menu (see `docs/history/phase3-4.md`) |
-Phase-1 note: BM25 is implemented in-tree (`embr/scoring.py`) so the core needs no numpy; real
+Phase-1 note: BM25 is implemented in-tree (`src/embr/scoring.py`) so the core needs no numpy; real
semantic embeddings live behind the `[ml]` extra, with a deterministic fallback embedder for
tests. Corpus-aware signals expose an optional `prepare(memories, query, state)` hook the
scorer calls once before per-memory scoring.
@@ -111,7 +119,7 @@ which is a live tension with the RQ2 target. Ouro also requires transformers 4.x
## 8. Conventions
-- One module per subsystem inside `embr/`; promote to a sub-package only when it outgrows a
+- One module per subsystem inside `src/embr/`; promote to a sub-package only when it outgrows a
single file. Folders organize; we don't scatter lonely files.
- Descriptive names, small "why" comments, easy-to-use functions, no duplicated logic
(one source of truth, e.g. signals and baselines).
diff --git a/docs/findings.md b/docs/findings.md
index 65e9b97..af88225 100644
--- a/docs/findings.md
+++ b/docs/findings.md
@@ -117,7 +117,7 @@ measurably more poisonable.** Reported as a null.
Two arms read 0/10 for a reason that is not a defence. The probe is a generic question that
shares no words with any memory, so a purely lexical or purely semantic store returns nothing
at all: `relevance_only` and Mnemosyne are immune by silence. Mnemosyne is measured exactly as
-shipped, through a bridge in its own virtual environment (`eval/backends.py`).
+shipped, through a bridge in its own virtual environment (`src/eval/backends.py`).
### 2.2 What survives, and it is the mechanism: poisonability is set by who controls a term's inputs
diff --git a/docs/handoff.md b/docs/handoff.md
index 120ca7f..0254a85 100644
--- a/docs/handoff.md
+++ b/docs/handoff.md
@@ -2,8 +2,8 @@
Written on the PC, 2026-08-18, superseding the Mac migration handoff. Everything here was
run and measured rather than remembered. Pair with [`design.md`](design.md) (architecture),
-[`roadmap.md`](roadmap.md) (the plan), [`related-work.md`](related-work.md) (prior art the
-paper must cite), and [`phase2.md`](phase2.md) / [`phase3-4.md`](phase3-4.md) (what shipped).
+[`roadmap.md`](history/roadmap.md) (the plan), [`related-work.md`](related-work.md) (prior art the
+paper must cite), and [`phase2.md`](history/phase2.md) / [`phase3-4.md`](history/phase3-4.md) (what shipped).
**For what the project found, read [`findings.md`](findings.md).** That is the canonical
statement of results, in RQ order, with every number traceable to a command. This document is
@@ -56,21 +56,18 @@ stale relative to the branch.
```bash
git clone https://github.com/Code-SorceryLab/EMBR.git
cd EMBR
-git switch phase-5-affect-attacks
-
-uv venv --python 3.11 .venv # see the launcher note in section 5
-.venv\Scripts\activate # Windows; source .venv/bin/activate elsewhere
-
-uv pip install -e ".[dev,figures]" # core, tests, paper figures
-pytest -q # expect 385 passed (1 skip if Ollama is down)
-embr # the menu
+uv sync --extra figures # the applet, the harness, the tests, the paper figures
+uv run pytest -q # the expected counts live in data/release-manifest.json
+uv run embr # the menu; `uv run embr --help` lists every command
```
-For the real models, read section 5 first, then:
+uv creates `.venv` itself and pins every version from `uv.lock`; there is no separate
+venv or pip step. For the real models, read section 5 first, then:
```bash
-uv pip install --index-url https://download.pytorch.org/whl/cu130 torch
-uv pip install -e ".[ml]"
+uv sync --extra ml # sentence embeddings, torch, transformers
+uv pip install --index-url https://download.pytorch.org/whl/cu130 torch # Windows CUDA build, after the sync
+scripts/fetch_models.sh # every model the project uses; --fresh re-downloads
```
Verified working combination on this machine:
@@ -138,7 +135,7 @@ rival to it. The order below reflects that: read 6.0 first, and read the poisoni
### 6.0 Emotion is the index, not the content
-`eval/emotion_flip.py`, `python -m eval.emotion_flip`, figure `data/figures/affective_indexing.png`.
+`src/eval/emotion_flip.py`, `python -m eval.emotion_flip`, figure `data/figures/affective_indexing.png`.
The thesis question, turned into a measurement: **does a memory keep its meaning if you flip
its emotion?** A memory has two kinds of meaning. Its factual meaning is what it is about, and
@@ -187,14 +184,14 @@ faces identical attacks (McNemar exact):
Not one attack poisoned a baseline while sparing EMBR. Every disagreement runs one way.
-These p values are now produced by the harness (`eval/stats.py:mcnemar_exact`, called from
+These p values are now produced by the harness (`src/eval/stats.py:mcnemar_exact`, called from
`run_rq2`) and written into `rq2.poisoning_stats` in every run directory. Until 2026-08-19
they were computed in a scratch script and typed into this document, which meant the study's
only significant result appeared in no artifact, could not be regenerated by a reader, and had
escaped the multiple-comparison correction every other comparison here receives. Corrected, it
clears 0.05 by a margin of 0.003. Report the Holm value.
-**The mechanism is not what it looks like, and `eval/attribution.py` proves it.** The obvious
+**The mechanism is not what it looks like, and `src/eval/attribution.py` proves it.** The obvious
story, that the affect intensity term rewards emotionally charged poison, is refuted by
direct measurement: zeroing affect intensity leaves the count at 9/10. Zeroing each scoring
term one at a time against the same ten injections (deterministic, five tests pin the counts):
@@ -246,7 +243,7 @@ entirely. That deserves its own paragraph in RQ2.
### 6.1a The defence, found on the `lagged-mood-congruence` branch
-Branch `lagged-mood-congruence`, `eval/provenance.py`, `python -m eval.provenance`. Two
+Branch `lagged-mood-congruence`, `src/eval/provenance.py`, `python -m eval.provenance`. Two
hypotheses failed first and both are kept, because the failures are what located the answer.
**Failed: lagged mood congruence.** Score against the mood the turn opened with, so an event
@@ -309,7 +306,7 @@ answer 0.5. Under a rater the attacker can talk to through the memory text, meas
Park under a realistic rater is the recency-only floor. **The 9/10 against 2/10 comparison,
and the 7-0 McNemar behind it, therefore partly measure a handicap this harness introduced.**
-One reviewer opening `eval/baselines.py` ends the paper with that sentence.
+One reviewer opening `src/eval/baselines.py` ends the paper with that sentence.
**This blocks everything else.** Before any further experiment, add a Park arm whose importance
comes from an LLM poignancy rater over the memory text, keep the authored arm, and report both.
@@ -322,7 +319,7 @@ attack corpus currently lets the attacker declare `valence`, `arousal` and `even
directly, which is not the threat model a shipped system exposes; an auto-tagging arm where the
attacker supplies only natural language is needed for the numbers to mean what they claim.
-**Measured on 2026-08-22, and the asymmetry collapses.** `eval/poignancy.py` asks
+**Measured on 2026-08-22, and the asymmetry collapses.** `src/eval/poignancy.py` asks
`llama3.2:3b` Park's own prompt for all 24 authored memories and all 10 injected ones, and the
`park_llm` arm runs Park over those ratings (cache: `data/ratings/llama3.2_3b_local.json`,
versioned so the number reproduces without a model).
@@ -353,7 +350,7 @@ pre-registered comparison is a finding.
### 6.1c The content x tag grid: the emotion that gets attacked is the tag, never the text
`python -m eval.grid`, 2026-08-22, stub model, exact counts. Each of the ten injected texts
-runs under four tag conditions against every arm (`eval/attacks.py::tag_variants`). Three
+runs under four tag conditions against every arm (`src/eval/attacks.py::tag_variants`). Three
predictions were written into the module docstring before the first run; all three held.
Poison in the probe top-5, out of 10:
@@ -445,7 +442,7 @@ affect intensity (inert to protective).
### 6.1e A real third-party system: Mnemosyne is immune by silence
-`eval/backends.py` puts `mnemosyne-hermes` 3.15.1 behind the harness's retrieval seam
+`src/eval/backends.py` puts `mnemosyne-hermes` 3.15.1 behind the harness's retrieval seam
through a JSON-lines worker in its own venv (`.venv-mnemosyne`, gitignored; `uv venv
.venv-mnemosyne && uv pip install mnemosyne-hermes fastembed sqlite-vec`). Measured exactly
as shipped: working-memory recall at default weights, which is FTS5 plus importance with
@@ -621,7 +618,7 @@ Lead with it.
A full pass over the system rather than the results. Findings, in order of severity:
-1. **The demo could be broken out of by a memory's text, and it is fixed.** `assets/build_demo.py`
+1. **The demo could be broken out of by a memory's text, and it is fixed.** `src/eval/report/build_demo.py`
inlines the exported JSON into a `` would close the block and have everything after it parsed as
HTML. Nothing in the current corpus triggers it, but the attack corpus is adversarial text
@@ -694,10 +691,10 @@ needing unpacking (many installs have an unpacked `Content (unpacked)` folder).
### 8.2 The mechanism experiment, now done, replacing the dose-response plan
The dose-response grid as previously described rested on a false premise: this section used
-to claim every injection sits at valence 0.9, arousal 0.8, but `eval/attacks.py` spans |v|
+to claim every injection sits at valence 0.9, arousal 0.8, but `src/eval/attacks.py` spans |v|
0.6 to 0.9 and arousal 0.2 to 0.8, and EMBR retrieved the poison on 9 of 10 across that whole
range. The curve is already at ceiling, and affect magnitude is not the lever anyway (6.1), so
-sweeping it would measure the wrong variable. That experiment is retired. `eval/attribution.py`
+sweeping it would measure the wrong variable. That experiment is retired. `src/eval/attribution.py`
did the job it was meant to do: it located the mechanism.
**The experiment worth building next is the defense arm, and one obvious version is already
@@ -711,7 +708,7 @@ magnitude. Two candidates survive that objection, both measurable in the existin
1. **Lagged mood congruence.** Score congruence against the character's mood *before* this
turn's appraisal, so one event cannot both set the mood and be rewarded for matching it.
This severs the self-priming loop rather than attenuating an input. A one-flag scorer
- variant, `embr_lagged_mood`, measured by extending `eval/attribution.py`. No new store, no
+ variant, `embr_lagged_mood`, measured by extending `src/eval/attribution.py`. No new store, no
schema change, fits the one-source-of-truth rule.
2. **Provenance-weighted affect.** Make Park's accidental defense deliberate: tag whether a
memory's affect is player-asserted or simulation-observed and down-weight the former. This
@@ -751,7 +748,7 @@ Three properties make it the right first target. It is a weighted composite like
comparison is like for like. Its signals are vector, full text, importance and recency, with
**no affect or mood term anywhere**, which is precisely EMBR's differentiator. And a recalled
hit carries `dense_score`, `fts_score`, `keyword_score`, `importance` and `recency_decay`, so
-`eval/attribution.py` can be run against it too: per-signal attribution on a third-party
+`src/eval/attribution.py` can be run against it too: per-signal attribution on a third-party
system, which no prior work reports.
**The prediction, worth pre-registering because it can fail.** Section 6.1 found the poisoning
diff --git a/docs/history/README.md b/docs/history/README.md
new file mode 100644
index 0000000..92e7a94
--- /dev/null
+++ b/docs/history/README.md
@@ -0,0 +1,13 @@
+# History
+
+The phase briefs and reports the project was built from, kept as the record of what was
+planned and what shipped. Nothing here is maintained: the live description of the system is
+[`../architecture.md`](../architecture.md), the results are [`../findings.md`](../findings.md),
+and the claims are [`../claims-ledger.md`](../claims-ledger.md).
+
+| Document | What it was |
+|---|---|
+| [`roadmap.md`](roadmap.md) | The phase-by-phase plan, with acceptance criteria per phase |
+| [`onboarding.md`](onboarding.md) | The intern brief for the evaluation harness phase |
+| [`phase2.md`](phase2.md) | The harness report: what phase 2 added, file by file, and its first numbers |
+| [`phase3-4.md`](phase3-4.md) | The paper assets, the real model runners, the walkthrough, and the menu |
diff --git a/docs/onboarding.md b/docs/history/onboarding.md
similarity index 79%
rename from docs/onboarding.md
rename to docs/history/onboarding.md
index bc85944..b40c232 100644
--- a/docs/onboarding.md
+++ b/docs/history/onboarding.md
@@ -1,6 +1,6 @@
# EMBR engineering intern plan (first ~8 weeks)
-> **Phases 2, 3 and 4 have shipped.** The `eval/` harness, the paper assets, the real model
+> **Phases 2, 3 and 4 have shipped.** The `src/eval/` harness, the paper assets, the real model
> runners, the walkthrough and the menu all exist now (see [`phase2.md`](phase2.md) and
> [`phase3-4.md`](phase3-4.md)), so tasks 2 to 7 are history rather than work to pick up. This
> document is kept as the ramp-up reading path: the task descriptions still say what each piece
@@ -23,7 +23,7 @@ A local middleware layer that gives a game NPC an emotion-grounded, persistent m
each turn it logs an event, updates the character's mood and trust, scores every stored
memory with five weighted signals, retrieves the top few, and prompts a local model. The
whole spine already runs and is tested; you build the harness that *measures* it against the
-two baselines. Read [`design.md`](design.md) for the architecture and the thesis proposal
+two baselines. Read [`design.md`](../design.md) for the architecture and the thesis proposal
for the why.
## How this works
@@ -32,7 +32,7 @@ for the why.
- **TDD.** Write the failing test first, then the code. Every new behaviour has a test.
- **Green before commit.** `pytest -q` must pass; small, frequent commits.
- **One source of truth.** A new scorer variant is a *weight map*, not a copy of `CompositeScorer`. Shared logic lives in one place.
-- **Small "why" comments.** Match the style already in `embr/`.
+- **Small "why" comments.** Match the style already in `src/embr/`.
- **Remote-friendly.** Each task ends with a short written update (what shipped, what's next, any blockers), which works well across the time zone.
Deadlines are week *ranges* and reorderable. If a task blocks, move to the next and come back.
@@ -42,7 +42,7 @@ Deadlines are week *ranges* and reorderable. If a task blocks, move to the next
```bash
git clone && cd EMBR
python3.11 -m venv .venv && source .venv/bin/activate
-pip install -e ".[dev,ml]" # dev = tests; ml = real semantic embeddings
+uv sync --extra ml # dev = tests; ml = real semantic embeddings
pytest -q # confirm a green baseline (semantic test un-skips with ml)
embr # open the menu, try "Conversation Turn"
```
@@ -52,12 +52,12 @@ embr # open the menu, try "Conversation Turn"
| # | Task | Window (flexible) | Deliverable |
|---|---|---|---|
| 1 | Orient + ship one tiny change | Week 1 | env working, small merged PR, "in my words" note |
-| 2 | Baselines (Park, Emotional RAG) | Weeks 1 to 2 | `eval/baselines.py` + tests |
-| 3 | Retrieval & cost metrics | Weeks 2 to 3 | `eval/metrics.py` + tests |
-| 4 | Scenarios, labels, experiment runner | Weeks 3 to 4 | `eval/scenarios.py`, `eval/run.py`, first results |
-| 5 | Adversarial probes | Weeks 4 to 5 | `eval/attacks.py` + tests |
+| 2 | Baselines (Park, Emotional RAG) | Weeks 1 to 2 | `src/eval/baselines.py` + tests |
+| 3 | Retrieval & cost metrics | Weeks 2 to 3 | `src/eval/metrics.py` + tests |
+| 4 | Scenarios, labels, experiment runner | Weeks 3 to 4 | `src/eval/scenarios.py`, `src/eval/run.py`, first results |
+| 5 | Adversarial probes | Weeks 4 to 5 | `src/eval/attacks.py` + tests |
| 6 | RQ3 ablation run | Weeks 5 to 6 | results + findings note |
-| 7 | Paper assets from results | Weeks 6 to 7 | `assets/build_*.py`, regenerable figures/tables |
+| 7 | Paper assets from results | Weeks 6 to 7 | `src/eval/report/build_*.py`, regenerable figures/tables |
| 8 | Wrap-up & handoff | Weeks 7 to 8 | report + updated docs, green suite |
---
@@ -75,29 +75,29 @@ embr # open the menu, try "Conversation Turn"
### Task 2: Baselines, Park & Emotional RAG (Weeks 1 to 2)
- **Why:** the comparison targets, and the best way to internalise the scoring abstraction.
-- **Do:** in `eval/baselines.py`, write `park_scorer()` (recency + importance + relevance;
+- **Do:** in `src/eval/baselines.py`, write `park_scorer()` (recency + importance + relevance;
`importance` is a model/heuristic rating EMBR does not have, so implement it faithfully)
and `emotional_rag_scorer()` (relevance + mood bias). Both are `CompositeScorer` **weight
maps, with no copied scoring code** (TDD).
-- **Deliverable:** `eval/baselines.py` + tests + a note on how each maps onto `CompositeScorer`.
+- **Deliverable:** `src/eval/baselines.py` + tests + a note on how each maps onto `CompositeScorer`.
- **Done when:** a crafted case shows each baseline ranks differently from EMBR, and no
scoring logic is duplicated. *(roadmap Phase 2, task 1)*
### Task 3: Retrieval & cost metrics (Weeks 2 to 3)
- **Why:** the measuring stick for RQ3 and latency.
-- **Do:** in `eval/metrics.py`, implement precision@k, recall@k, nDCG@k for k in {3, 5, 10};
+- **Do:** in `src/eval/metrics.py`, implement precision@k, recall@k, nDCG@k for k in {3, 5, 10};
Jaccard distance between top-k sets across warm / neutral / suspicious states; per-stage
- latency timers (p50, p95, in `eval/latency.py`). TDD each metric against a hand-computed
+ latency timers (p50, p95, in `src/eval/latency.py`). TDD each metric against a hand-computed
toy example.
-- **Deliverable:** `eval/metrics.py` + tests.
+- **Deliverable:** `src/eval/metrics.py` + tests.
- **Done when:** every metric matches a worked example. *(roadmap Phase 2, task 3)*
### Task 4: Scenarios, labels & the experiment runner (Weeks 3 to 4)
- **Why:** something to measure, and a repeatable way to run it.
-- **Do:** `eval/scenarios.py` (the Dawn Whitmore multi-session arc), `eval/labels/`
- (pre-registered relevance labels), and `eval/run.py` (runs RQ3 retrieval over EMBR and
+- **Do:** `src/eval/scenarios.py` (the Dawn Whitmore multi-session arc), `src/eval/labels/`
+ (pre-registered relevance labels), and `src/eval/run.py` (runs RQ3 retrieval over EMBR and
both baselines, deterministic seeds, writes `data/runs//` as JSON/CSV). Wire it
to the menu's evaluation options.
- **Deliverable:** scenarios + labels + runner + a first results dump.
@@ -106,10 +106,10 @@ embr # open the menu, try "Conversation Turn"
### Task 5: Adversarial probes (Weeks 4 to 5)
- **Why:** the inputs for the RQ2 robustness study.
-- **Do:** `eval/attacks.py`: 20 attacks in 4 categories of 5 (role override, false-memory
+- **Do:** `src/eval/attacks.py`: 20 attacks in 4 categories of 5 (role override, false-memory
injection, emotion flipping, persona dissolution), plus a harness that applies each to the
memory store and records drift (valence-arousal cosine distance from the canonical response).
-- **Deliverable:** `eval/attacks.py` + tests.
+- **Deliverable:** `src/eval/attacks.py` + tests.
- **Done when:** all four categories are represented and the drift metric is wired.
*(roadmap Phase 2, task 4)*
@@ -124,7 +124,7 @@ embr # open the menu, try "Conversation Turn"
### Task 7: Paper assets from results (Weeks 6 to 7), a taste of Phase 3
- **Why:** turn numbers into paper-ready, reproducible figures and tables.
-- **Do:** `assets/build_tables.py` + `assets/build_figures.py`: the retrieval-quality table,
+- **Do:** `src/eval/report/build_tables.py` + `src/eval/report/build_figures.py`: the retrieval-quality table,
the ablation bar chart, the latency p50/p95 plot; one command (`embr assets`) regenerates
them from the latest run; use the ember palette.
- **Deliverable:** the asset scripts + generated figures/tables.
@@ -147,13 +147,13 @@ embr # open the menu, try "Conversation Turn"
The open work is what Phase 2 deliberately left behind a seam, all of it in the roadmap:
- ~~the blind multi-annotator label pass~~ **shelved 2026-08-24**, with the rest of the
- human-subject work. The borderline exclusions recorded in `eval/scenarios.py` will now
+ human-subject work. The borderline exclusions recorded in `src/eval/scenarios.py` will now
never be re-judged, so no ordering may be read off the retrieval table at all. This is a
permanent ceiling, not a pending task;
- the **real model runner** in place of `StubRunner`, which is what makes every tone and
drift number in RQ1 and RQ2 mean something;
- the **off-the-shelf affect classifier and blinded model judge** behind the existing
- `ToneRater` protocol in `eval/tone.py`, replacing `LexiconToneRater`.
+ `ToneRater` protocol in `src/eval/tone.py`, replacing `LexiconToneRater`.
## Cadence
diff --git a/docs/phase2.md b/docs/history/phase2.md
similarity index 90%
rename from docs/phase2.md
rename to docs/history/phase2.md
index 31580aa..a00500a 100644
--- a/docs/phase2.md
+++ b/docs/history/phase2.md
@@ -2,44 +2,44 @@
Phase 2 builds the measurement layer: everything needed to run RQ1 (behaviour), RQ2
(robustness and cost), and RQ3 (retrieval quality) against the Park and Emotional RAG
-baselines under the pre-registered protocol. The harness lives in `eval/` at the repo
-root, deliberately outside `embr/`: it measures the system, so the system never imports
-it. Pair this with [`design.md`](design.md) (architecture) and
+baselines under the pre-registered protocol. The harness lives in `src/eval/` at the repo
+root, deliberately outside `src/embr/`: it measures the system, so the system never imports
+it. Pair this with [`design.md`](../design.md) (architecture) and
[`roadmap.md`](roadmap.md) (the phase brief this delivers on).
## 1. What phase 2 added, file by file
-### The harness (`eval/`)
+### The harness (`src/eval/`)
-- **`eval/baselines.py`**: the two paper baselines as weight maps over the shared
+- **`src/eval/baselines.py`**: the two paper baselines as weight maps over the shared
`CompositeScorer`. The only new scoring code is `Importance`, an authored-ratings
lookup standing in for Park's LLM poignancy rater; each scorer's docstring lists
exactly where it deviates from the published pipeline.
-- **`eval/scenarios.py`** + **`eval/labels/dawn_whitmore.json`**: the pre-registered
+- **`src/eval/scenarios.py`** + **`src/eval/labels/dawn_whitmore.json`**: the pre-registered
scenario: 24 memories across five sessions of Dawn Whitmore's arc, 10 retrieval
queries with frozen relevant sets, authored importance ratings, and three pinned mood
conditions (warm, neutral, suspicious). The loader rebuilds timestamps against a
caller-supplied reference time so the scenario is identical on any run day.
-- **`eval/metrics.py`**: precision@k, recall@k, nDCG@k, Jaccard distance between top-k
+- **`src/eval/metrics.py`**: precision@k, recall@k, nDCG@k, Jaccard distance between top-k
sets, and valence-arousal drift. Pure functions, no numpy, every formula small enough
to recompute by hand.
-- **`eval/attacks.py`**: the twenty-attack RQ2 corpus (five each of role override, false
+- **`src/eval/attacks.py`**: the twenty-attack RQ2 corpus (five each of role override, false
memory, emotion flip, persona dissolution, adapted from MINJA) and `run_attack`, which
plays one attack into a conversation and probes it with a fixed follow-up question.
Injection attacks also poison the memory write itself.
-- **`eval/tone.py`**: the `ToneRater` seam (reply text to valence and arousal) and
+- **`src/eval/tone.py`**: the `ToneRater` seam (reply text to valence and arousal) and
`LexiconToneRater`, a deterministic stand-in so the harness runs anywhere; the real
affect classifier and the blinded judge plug in behind the same protocol later.
-- **`eval/latency.py`**: wraps an existing `Conversation`'s write, score-retrieve, and
+- **`src/eval/latency.py`**: wraps an existing `Conversation`'s write, score-retrieve, and
model stages from the outside and reports nearest-rank p50/p95 with per-stage sample
counts. The core never grows timing code.
-- **`eval/tuning.py`**: the one grid search every variant goes through, plus
+- **`src/eval/tuning.py`**: the one grid search every variant goes through, plus
`leave_one_out_folds` for held-out tuned scores and a `visible_memories` filter that
stops a query from seeing memories from later sessions.
-- **`eval/stats.py`**: fixed-seed percentile bootstrap CIs, an exact paired sign-flip
+- **`src/eval/stats.py`**: fixed-seed percentile bootstrap CIs, an exact paired sign-flip
permutation test, and Holm-Bonferroni correction. Deterministic and dependency free,
like the numbers it describes.
-- **`eval/run.py`**: the runner. `python -m eval.run` executes all three studies against
+- **`src/eval/run.py`**: the runner. `python -m eval.run` executes all three studies against
a pinned `REFERENCE_TIME` (2026-01-01 UTC) and writes an auditable run directory;
`fast_rq3_defaults()` is the sub-second subset the menu calls.
@@ -80,14 +80,14 @@ wiring described here survived the move; only the renderer changed.
## 2. What changed in existing files, and why
-- **`embr/scoring.py`** (the only core change): `Recency` gained an injectable clock,
+- **`src/embr/scoring.py`** (the only core change): `Recency` gained an injectable clock,
`now: Callable[[], datetime] | None`, threaded through `all_signals` and
`embr_scorer`. The default is `None`, meaning the live wall clock, so game behaviour
is unchanged; the eval passes a clock returning `REFERENCE_TIME` at every scorer
construction site. Without this, the scenario's pinned 2026-01-01 timestamps were
months in the past by run day and every recency score had decayed to roughly 1e-11:
the signal was dead in every variant and the comparison was silently four-signal.
-- **`embr/app/main.py`**: the "not built yet" experiment placeholder was replaced with
+- **`src/embr/app/main.py`**: the "not built yet" experiment placeholder was replaced with
the live screen described above. (Phase 4 removed this file with the rest of the Textual
applet; the wiring moved to `menu.py` at the repo root.)
- **`tests/test_scoring.py`**: two new tests pin the injected clock (exact decay from an
@@ -154,7 +154,7 @@ is lexical rather than semantic; the real model runner and real embeddings land
eval hardware. The labels are the v1 pre-registered set, authored by one person before
any retrieval was run. **The blind multi-annotator pass is not coming**: it was shelved on
2026-08-24 with the rest of the human-subject work, so the recorded borderline cases (see the
-honesty note in `eval/scenarios.py`) stay unadjudicated and the label set stays single-author. Ten queries buy very little power: the CIs are wide, every paired
+honesty note in `src/eval/scenarios.py`) stay unadjudicated and the label set stays single-author. Ten queries buy very little power: the CIs are wide, every paired
interval spans zero, and no Holm-corrected comparison is significant (the minimum
corrected p is 0.75, for the no-relevance ablation, whose attainable floor is 0.03125).
The tuned rows are honest held-out estimates, which is why they sit below the optimistic
@@ -202,7 +202,7 @@ described in section 2; tuned scores moved from in-sample grid maxima to
leave-one-query-out cross-validation; RQ2 went from measuring one system to comparing
four against the full memory store, gaining the retrieval_drift and poison_retrieved
columns; a neutral mood condition that was not actually neutral was re-pinned to the
-zero vector; CIs and Holm-corrected paired tests were added via `eval/stats.py`; and a
+zero vector; CIs and Holm-corrected paired tests were added via `src/eval/stats.py`; and a
nearest-rank percentile off-by-one was corrected.
The caveats that remain are declared in the run output itself:
@@ -220,7 +220,7 @@ The caveats that remain are declared in the run output itself:
independently of which model runs. `immediate_drift` is kept only as a stub-limited
diagnostic and is constant across systems.
- **The label set is v1.** Six borderline exclusions are documented next to the honesty
- note in `eval/scenarios.py`, exposed machine-readably as `BORDERLINE_EXCLUSIONS`, and
+ note in `src/eval/scenarios.py`, exposed machine-readably as `BORDERLINE_EXCLUSIONS`, and
frozen; the blind pass re-judges them rather than the author quietly re-adjudicating his
own labels. Admitting them reverses the Park and EMBR ordering, so they matter.
- **Park is a shared-scorer port**, not a byte-for-byte reimplementation. Its docstring
@@ -244,6 +244,6 @@ Phase 3 (paper assets) reads `data/runs//` and nothing else:
assets must produce identical files, except the latency block, the one declared
wall-clock measurement.
-The build scripts themselves (`assets/build_tables.py`, `assets/build_figures.py`, and
+The build scripts themselves (`src/eval/report/build_tables.py`, `src/eval/report/build_figures.py`, and
the menu's "Generate Paper Assets" option) are phase 3's scope, and shipped: see
[`phase3-4.md`](phase3-4.md).
diff --git a/docs/phase3-4.md b/docs/history/phase3-4.md
similarity index 92%
rename from docs/phase3-4.md
rename to docs/history/phase3-4.md
index 3a9b243..ebe080d 100644
--- a/docs/phase3-4.md
+++ b/docs/history/phase3-4.md
@@ -3,7 +3,7 @@
Two phases, documented together because they shipped together and the menu spans both.
Phase 3 turns a run directory into the paper's figures and tables. Phase 4 gives the system
real models to talk through, a playable arc to show it off, and a front door to reach all of
-it. Pair this with [`design.md`](design.md) (architecture), [`roadmap.md`](roadmap.md) (the
+it. Pair this with [`design.md`](../design.md) (architecture), [`roadmap.md`](roadmap.md) (the
briefs these deliver on), and [`phase2.md`](phase2.md) (the harness that produces the data).
## 1. Phase 3: paper assets
@@ -11,10 +11,10 @@ briefs these deliver on), and [`phase2.md`](phase2.md) (the harness that produce
The rule from the roadmap is that no number is ever transcribed by hand. Both builders read
`data/runs//` and write into `assets/`.
-- **`assets/build_tables.py`** emits five tables, each as LaTeX (booktabs) with a CSV twin:
+- **`src/eval/report/build_tables.py`** emits five tables, each as LaTeX (booktabs) with a CSV twin:
the five-signal reference table, RQ3 retrieval quality grouped by family, the paired
comparisons against tuned EMBR, RQ2 robustness, and RQ1 mood divergence. 39 tests.
-- **`assets/build_figures.py`** emits five figures, each as PDF for the paper and PNG for the
+- **`src/eval/report/build_figures.py`** emits five figures, each as PDF for the paper and PNG for the
README: RQ3 retrieval quality, the RQ3 ablation deltas, RQ2 poisoning, RQ2 latency, and RQ1
divergence. 20 tests.
@@ -75,7 +75,7 @@ pins accordingly. The eval box will need the same pin.
## 3. Phase 4: the playable walkthrough
-`embr/walkthrough.py` plays Dawn Whitmore's five-beat arc: the king's-errand lie that buys a
+`src/embr/walkthrough.py` plays Dawn Whitmore's five-beat arc: the king's-errand lie that buys a
discounted room, a warm return, the slip about the late king, the reckoning, and a
confession. A recorded, playable walkthrough is a primary deliverable for this venue, so the
demo has to show its work rather than just print dialogue.
@@ -127,10 +127,10 @@ Neither phase is a clean sweep, and the gaps matter more than the tick marks:
- **The latency target is missed by a wide margin.** The bake-off has since run on CUDA and
the hand measurements in section 2 are superseded: Ouro takes 32.4 s per realistic turn
against a roughly 600 ms target. The VRAM budget, by contrast, holds at 2.78 GB. See
- section 6 of [`handoff.md`](handoff.md). This needs a response in the paper, not a footnote.
+ section 6 of [`handoff.md`](../handoff.md). This needs a response in the paper, not a footnote.
-The bake-off gap is closed: `eval/bakeoff.py` holds prompts, memories, retrieval and sampling
-equal and varies only the model, and `eval/experiments.py` replicates a run to show the
+The bake-off gap is closed: `src/eval/bakeoff.py` holds prompts, memories, retrieval and sampling
+equal and varies only the model, and `src/eval/experiments.py` replicates a run to show the
harness reproduces exactly.
## 6. Running it
diff --git a/docs/roadmap.md b/docs/history/roadmap.md
similarity index 86%
rename from docs/roadmap.md
rename to docs/history/roadmap.md
index d7006a1..f4fddb6 100644
--- a/docs/roadmap.md
+++ b/docs/history/roadmap.md
@@ -2,7 +2,7 @@
**Audience:** the engineers/interns building EMBR phase by phase.
**Purpose:** for each phase: *what to do*, *what to hand back*, and *what results we expect to
-see*. Pair this with [`design.md`](design.md) (the architecture) and the thesis
+see*. Pair this with [`design.md`](../design.md) (the architecture) and the thesis
(`../../Proposals/Masters/NCP.docx`, the why).
> EMBR's contribution is the **memory layer**, not the model. Every result should be about
@@ -17,8 +17,8 @@ see*. Pair this with [`design.md`](design.md) (the architecture) and the thesis
- **TDD:** write the failing test first, then the code. Every new behaviour has a test.
- **Green before commit:** `pytest -q` must pass. Small, frequent commits with clear messages.
- **One source of truth:** no duplicated logic. A new scorer variant is a *weight map*, not a copy of `CompositeScorer`. A new store is a class behind the existing `MemoryStore` interface, not a fork of it.
-- **Clean structure:** one module per subsystem inside `embr/`; promote a module to a package only when it genuinely outgrows one file. Folders organise; don't scatter lonely files.
-- **Style:** descriptive names, small "why" comments, easy-to-call functions. Match the patterns already in `embr/`.
+- **Clean structure:** one module per subsystem inside `src/embr/`; promote a module to a package only when it genuinely outgrows one file. Folders organise; don't scatter lonely files.
+- **Style:** descriptive names, small "why" comments, easy-to-call functions. Match the patterns already in `src/embr/`.
- **Reproducibility:** every figure and table is generated *from code* into `assets/`. Never hand-make a paper asset.
- **Definition of Done (global), every phase:** code + tests green + docs updated (`design.md` / this file) + the relevant menu option works + any figures/tables regenerate from one command.
@@ -27,7 +27,7 @@ see*. Pair this with [`design.md`](design.md) (the architecture) and the thesis
```bash
git clone && cd EMBR
python3.11 -m venv .venv && source .venv/bin/activate
-pip install -e ".[dev,ml]" # ml extra needed from Phase 1 on
+uv sync --extra ml # ml extra needed from Phase 1 on
pytest -q # confirm a green baseline
git switch -c phase-5-yourwork # your own phase branch
```
@@ -65,7 +65,7 @@ Already done, so you know what "live" means before you extend it:
`Conversation` pipeline, and the menu. `pytest` is green (7 tests). The menu's
**Conversation Turn** runs a live demo turn that surfaces the tavern-keeper's lie.
-**The contract you must not break:** the public interfaces in `embr/__init__.py`. Swap
+**The contract you must not break:** the public interfaces in `src/embr/__init__.py`. Swap
implementations *behind* them; don't change their shapes without updating every caller.
---
@@ -78,20 +78,20 @@ something honest to measure. **Foundation for all three RQs.**
### Tasks
-1. **Hybrid relevance**: `embr/scoring.py` (`Relevance.score`)
+1. **Hybrid relevance**: `src/embr/scoring.py` (`Relevance.score`)
- Implement `rel = γ·BM25 + (1−γ)·cosine(embeddings)`, replacing the token-overlap stand-in.
- BM25 over the character's memory texts (`rank-bm25`); cosine over `Memory.embedding`.
- Keep the `Signal` interface unchanged.
-2. **Embeddings**: new `embr/embeddings.py`
+2. **Embeddings**: new `src/embr/embeddings.py`
- One small `Embedder` wrapper (`sentence-transformers`, a compact model) with `encode(text) -> list[float]`.
- Set `Memory.embedding` when a memory is added to the store; cache; never re-encode the same text.
-3. **Persistent store**: `embr/memory.py` (`SQLiteMemoryStore`)
+3. **Persistent store**: `src/embr/memory.py` (`SQLiteMemoryStore`)
- A `MemoryStore`-compatible class backed by SQLite (+ a vector column / index). Survives process restart.
- Same methods (`add`, `all`, `__len__`); selected via Settings/config. The in-memory store stays as the test/default.
-4. **Affect appraisal rules**: `embr/affect.py` + `embr/pipeline.py`
+4. **Affect appraisal rules**: `src/embr/affect.py` + `src/embr/pipeline.py`
- Replace the placeholder `0.2 * valence` trust nudge with a small rules table: per `EventType`, how much mood (valence/arousal) and trust move, and how a plot beat scales with prior trust.
- Document each number with a one-line rationale; this is a design artefact, keep it readable.
-5. **Settings**: a menu `Settings` view + a `embr/config.py`
+5. **Settings**: a menu `Settings` view + a `src/embr/config.py`
- Expose: scorer weights, `top_k`, store backend, embedding model, model runner. Persist to a config file under `data/`.
### Deliverables
@@ -121,28 +121,28 @@ produce the numbers the paper reports. **This phase carries the contribution.**
### Tasks
-1. **Baselines**: `eval/baselines.py`
+1. **Baselines**: `src/eval/baselines.py`
- `park_scorer()`: recency + importance + relevance (faithful Park et al.; `importance` is a model/heuristic rating, *not* our affect decomposition).
- `emotional_rag_scorer()`: relevance + mood bias (closest prior work).
- Both are `CompositeScorer` variants / weight maps, with **no copied scoring code.**
-2. **Scenarios & labels**: `eval/scenarios.py`, `eval/labels/`
+2. **Scenarios & labels**: `src/eval/scenarios.py`, `src/eval/labels/`
- Dawn Whitmore five-session arc (full ground-truth control); a Stardew Valley corpus for scale and external validity.
- **Pre-registered** relevance labels per step, authored *before* results are seen. **Superseded 2026-08-24:** this promised annotators blind to the variant and an inter-annotator agreement figure. With no human subjects in the project, **the v1 labels stay single-author and that is disclosed as a limitation**, which is how `metrics.md` already describes them. The v2 label expansion and the annotator recruitment it required are shelved. Do not report an agreement statistic; there is only one annotator and saying otherwise would be false.
-3. **Metrics**: `eval/metrics.py`
+3. **Metrics**: `src/eval/metrics.py`
- Retrieval-shift: Jaccard distance between top-k sets across warm / neutral / suspicious states.
- Tone: off-the-shelf valence-arousal classifier wrapper **and** a blinded model-judge harness.
- Retrieval quality: precision@k, recall@k, nDCG@k for k ∈ {3, 5, 10}.
- Cost: per-stage millisecond timers (write/score/retrieve/model); report p50 + p95 over 100 turns/variant.
- Drift: cosine distance between an attack response's predicted valence-arousal and the canonical ground truth.
-4. **Adversarial probes**: `eval/attacks.py`
+4. **Adversarial probes**: `src/eval/attacks.py`
- 20 attacks, 4 categories × 5 (role override, false-memory injection, emotion flipping, persona dissolution), adapted from MINJA.
-5. **Tuning**: `eval/tuning.py`
+5. **Tuning**: `src/eval/tuning.py`
- One grid search over weights on a fixed validation set, applied **identically** to EMBR, Park, and Emotional RAG. Also record each baseline at its published defaults.
-6. **Runner**: `eval/run.py` + the menu's evaluation options
+6. **Runner**: `src/eval/run.py` + the menu's evaluation options
- Run RQ1/RQ2/RQ3, write results to `data/runs//` as JSON/CSV. Deterministic seeds; effects with confidence intervals; correct for multiple comparisons across variants.
### Deliverables
-`eval/` modules, pre-registered label files, results under `data/runs/`, the experiment runner wired into the menu.
+`src/eval/` modules, pre-registered label files, results under `data/runs/`, the experiment runner wired into the menu.
### Expected results (from the thesis's anticipated results, hold interns to these)
- **RQ1 (Behaviour).** Varying *only* the state (a) changes the surfaced top-k set (non-zero Jaccard across mood conditions) **and** (b) changes reply tone: the classifier correlates with the intended mood, and a **judge panel across model families** agrees above chance with inter-judge agreement reported (with CIs). *A null result (state changes retrieval but not generation) is a valid, reportable finding; do not massage it away.*
@@ -167,7 +167,7 @@ produce the numbers the paper reports. **This phase carries the contribution.**
> 2. **The attribution sweep's behavioural estimator**, which measures causally whether the
> mood sentence drives the reply. This is now RQ1's strongest evidence, and arguably a
> better design than a ten-person preference study. Already built; hypotheses fixed in
- > [`preregistration-attribution.md`](preregistration-attribution.md).
+ > [`preregistration-attribution.md`](../preregistration-attribution.md).
> 3. **A behavioural consistency check** as a task-like proxy: after the betrayal beat, does
> the keeper refuse the discounted room? Binary, scriptable, no raters, and closer to what
> a player actually experiences than any tone score.
@@ -178,7 +178,7 @@ produce the numbers the paper reports. **This phase carries the contribution.**
>
> **Where the vulnerability sits.** This pre-registered the dominant vulnerability at the
> model call and the memory write, and explicitly *not* in the scoring formula. The data
- > says the opposite: `eval/attribution.py` localises it to the scoring formula, to the
+ > says the opposite: `src/eval/attribution.py` localises it to the scoring formula, to the
> **mood congruence** term, on the **valence** axis, and it is the only term whose removal
> ever lowers the count. Because it was pre-registered, the inversion is evidence rather
> than a story fitted afterwards, and it is the strongest thing this project found.
@@ -207,14 +207,14 @@ pytest -q eval/ # metric/attack unit tests pass
command. Zero hand-made assets.
### Tasks
-1. **Tables**: `assets/build_tables.py` → `data/tables/*.tex` + `*.csv`
+1. **Tables**: `src/eval/report/build_tables.py` → `data/tables/*.tex` + `*.csv`
- The signal table, the RQ metric definitions, and each results table (retrieval shift, retrieval quality, latency p50/p95, drift-under-attack). LaTeX `booktabs` + a CSV twin.
-2. **Figures**: `assets/build_figures.py` → `data/figures/*.png` (+ `*.pdf` for the paper)
+2. **Figures**: `src/eval/report/build_figures.py` → `data/figures/*.png` (+ `*.pdf` for the paper)
- Retrieval-shift (Jaccard) plot, tone-shift plot, latency p50/p95 bars, retrieval PR / nDCG curves, the ablation bars, drift-under-attack by category. Use the EMBR ember palette consistently. The architecture figure already exists.
3. **One command**: the menu's "Generate Paper Assets" option regenerates **everything** from the latest run.
### Deliverables
-`assets/build_tables.py`, `assets/build_figures.py`, regenerated `data/figures/*`, `data/tables/*`.
+`src/eval/report/build_tables.py`, `src/eval/report/build_figures.py`, regenerated `data/figures/*`, `data/tables/*`.
### Expected results (acceptance)
- Running `embr assets` on a given `data/runs/` reproduces **every** paper figure and table (same numbers, same look) with **no manual editing**.
@@ -237,7 +237,7 @@ carries as much weight as the measurements.*
### Tasks
1. **Interactive turn loop**: the menu's "Tavern-Keeper Walkthrough" option: real player input, real model, live mood/trust/latency readouts.
-2. **The arc**: `embr/scenarios/dawn_whitmore.py`: the scripted beats (the discounted room, the lie surfacing, the reckoning, reconciliation) with branch points driven by the player's choices and the keeper's state.
+2. **The arc**: `src/embr/scenarios/dawn_whitmore.py`: the scripted beats (the discounted room, the lie surfacing, the reckoning, reconciliation) with branch points driven by the player's choices and the keeper's state.
3. **Recording + companion page**: a recorded playthrough (asciinema or video) and a GitHub Pages companion page hosting the interactive web demo the README links to (GitHub can't run JS in a README, so the live widget lives there).
### Deliverables
@@ -248,14 +248,14 @@ Walkthrough screen, the arc, a recording file, a companion `docs/site/` page, RE
- The recording exists and is linked from the README; the companion page loads the interactive demo.
### What actually shipped
-The arc lives in `embr/walkthrough.py` rather than a `scenarios/` package, because one module
+The arc lives in `src/embr/walkthrough.py` rather than a `scenarios/` package, because one module
covers it and the house rule is to promote to a package only when a module outgrows itself. Two
real runners landed alongside it (`OllamaRunner` for a local daemon or the cloud host, and
`OuroRunner` for the thesis model), so the walkthrough plays on a real model rather than the
stub. Details and the measured looped-versus-conventional latency gap are in
[`phase3-4.md`](phase3-4.md).
-**Still open from this phase:** the recording and the companion page, and `eval/bakeoff.py`,
+**Still open from this phase:** the recording and the companion page, and `src/eval/bakeoff.py`,
the measured model comparison the menu already has an option for.
---
@@ -267,15 +267,15 @@ Branch `phase-5-affect-attacks`. What it delivered, and why each piece exists:
| Built | Because |
|---|---|
| NRC VAD Lexicon v2.1 behind `ToneRater` | the previous rater scored from 35 words the author picked, which is not a measurement |
-| A blinded model judge, plus `eval/agreement.py` | one automatic rater cannot tell a real tone shift from its own artefact |
+| A blinded model judge, plus `src/eval/agreement.py` | one automatic rater cannot tell a real tone shift from its own artefact |
| Affective drift as a distance on the circumplex | cosine ignored magnitude and was undefined at the origin |
-| `eval/poignancy.py` and the `park_llm` arm | Park et al. rate with a model; the authored-ratings baseline was a handicap this harness invented |
-| `tag_variants` and `eval/grid.py` | every built attack was congruent, so nothing separated the emotion in a memory's words from the emotion in its tag |
-| `signal_by_tag` in `eval/attribution.py` | "which emotional signal is strongest" needed an answer per condition and per affect axis |
-| `eval/backends.py` and the Mnemosyne arm | a baseline that is a weight map over our own scorer is not a comparison against a real system |
-| `assets/build_animations.py` | the RQ1 result is a change over time, and no static figure shows a change |
+| `src/eval/poignancy.py` and the `park_llm` arm | Park et al. rate with a model; the authored-ratings baseline was a handicap this harness invented |
+| `tag_variants` and `src/eval/grid.py` | every built attack was congruent, so nothing separated the emotion in a memory's words from the emotion in its tag |
+| `signal_by_tag` in `src/eval/attribution.py` | "which emotional signal is strongest" needed an answer per condition and per affect axis |
+| `src/eval/backends.py` and the Mnemosyne arm | a baseline that is a weight map over our own scorer is not a comparison against a real system |
+| `src/eval/report/build_animations.py` | the RQ1 result is a change over time, and no static figure shows a change |
-**Two results changed what the paper claims**, and both are in [`findings.md`](findings.md):
+**Two results changed what the paper claims**, and both are in [`findings.md`](../findings.md):
the EMBR-against-Park headline is a null once Park is rated the way Park et al. rate, and RQ1
gained its first generation result (significant on llama3.2:3b, null on Ouro 1.4B).
@@ -292,10 +292,10 @@ Not started. Two pieces, in order:
a query may carry one relevant set per state, and `state_conditioned_ndcg` scores each
state against its own gold, which is the only shape of measurement a mood-congruent signal
can win under. The labels themselves are the blocker, and deliberately cannot be written
- here: see [`corpus.md`](corpus.md) for the schema, the acquisition path, the legal
+ here: see [`corpus.md`](../corpus.md) for the schema, the acquisition path, the legal
constraint, and the pre-registered prediction.
2. **The interactive demo.** Done, in two readings of one payload, both built by
- `assets/build_demo.py` from a named run and both openable from a `file://` path.
+ `src/eval/report/build_demo.py` from a named run and both openable from a `file://` path.
- `data/demo/index.html`, **69 KB, no dependency at all.** Five signal nodes, the memories
between them, her prompt on the right, and an edge for every signal that paid for a
@@ -311,7 +311,7 @@ Not started. Two pieces, in order:
`tests/test_build_demo.py` runs that replay under Node for each page plus a check that the
two pages' scoring code has not drifted apart.
- - `data/demo/results.html`, **670 KB, generated by `assets/build_results.py`.** The
+ - `data/demo/results.html`, **670 KB, generated by `src/eval/report/build_results.py`.** The
three research questions in the project's own order, for a reviewer with ten minutes.
Its numbers are read from the run and six of them are cross-checked against the prose
of `findings.md`; **the build refuses to write the page if the two disagree**, which is
@@ -323,8 +323,8 @@ Not started. Two pieces, in order:
## Phase 7: power, the shipped defence, and the causal step
-**Direction set 2026-08-24.** The branch is `cite-view-test`. Method in [`cite.md`](cite.md),
-hypotheses fixed in [`preregistration-attribution.md`](preregistration-attribution.md).
+**Direction set 2026-08-24.** The branch is `cite-view-test`. Method in [`cite.md`](../cite.md),
+hypotheses fixed in [`preregistration-attribution.md`](../preregistration-attribution.md).
The organising judgement: RQ3 is the weakest contribution and the corpus only rescues that;
the security mechanism and its defence are the strongest and are already model-independent.
@@ -335,9 +335,9 @@ sweep supplies the causal step RQ2 is missing. Everything else is sequenced behi
| # | Item | Notes |
|---|---|---|
-| 1 | **Anchor-weight config in `embr/scoring.py`**, dose-response as its validation test, defended configuration as the shipped default | **Invalidates every published number.** See the conflict below. |
+| 1 | **Anchor-weight config in `src/embr/scoring.py`**, dose-response as its validation test, defended configuration as the shipped default | **Invalidates every published number.** See the conflict below. |
| 2 | **Write-time tag provenance**: memories record who wrote them; affect tags come only from the appraisal step, never from raw player text. SQLite schema change | Must be a *posture flag*, not a removal: the paper needs the vulnerable arm to demonstrate the attack and the hardened arm to demonstrate the fix |
-| 3 | **New probe classes**: Sleeper-style dormant poisons, and a self-summarisation laundering probe | Two documented 2026 attack classes the current 20 do not cover. Extends `eval/attacks.py` with no protocol conflict |
+| 3 | **New probe classes**: Sleeper-style dormant poisons, and a self-summarisation laundering probe | Two documented 2026 attack classes the current 20 do not cover. Extends `src/eval/attacks.py` with no protocol conflict |
| 4 | **Judge panel**: two or three models from different families plus the NRC lexicon, rating the same replies, with inter-judge agreement reported | Replaces the single blinded judge, and replaces the human arm's bias control. Rater rules fixed in the pre-registration |
| 5 | **Behavioural consistency check**: after the betrayal beat, does the keeper refuse the discounted room? | Binary, scriptable, no raters. A task-like proxy for what a player experiences, which a tone score is not |
@@ -380,7 +380,7 @@ behavioural estimator lands, the reframe is a result. If it does not, it was a s
unfaithful when the context restates what the model already knows. The one "Kenny" mention
in `assets/presentation/slides.md` is the motivating anecdote in the talk, not a test
subject, and is correct as it stands.
-- **`eval/bakeoff.py` is finished**, not stubbed: `run_arm`, `default_arms`, `run_bakeoff` and
+- **`src/eval/bakeoff.py` is finished**, not stubbed: `run_arm`, `default_arms`, `run_bakeoff` and
`main` are all implemented, it is wired into the menu, and three runs plus the
`bakeoff_grounding`, `bakeoff_latency` and `bakeoff_mood` figures already exist. The menu's
"not built yet" line is an `ImportError` fallback for a fresh clone.
diff --git a/docs/metrics.md b/docs/metrics.md
index fdd881a..0de7bcf 100644
--- a/docs/metrics.md
+++ b/docs/metrics.md
@@ -2,8 +2,8 @@
Every number the paper reports, what it measures, the formula as the harness implements it,
the literature it comes from, and the known weakness. If a metric is not on this page it is
-not reported. Code lives in `eval/metrics.py`, `eval/stats.py`, `eval/tone.py` and
-`eval/latency.py`.
+not reported. Code lives in `src/eval/metrics.py`, `src/eval/stats.py`, `src/eval/tone.py` and
+`src/eval/latency.py`.
**Status** says whether the page describes the code as it is (`implemented`) or a change the
review below found necessary (`planned`). A planned change is not a result until it ships.
@@ -99,7 +99,7 @@ reading the whole line) that agree is the strongest claim available without peop
ignores magnitude, so a reply that moves from mildly warm to intensely warm read as zero
drift, and a neutral reading had no angle at all. Euclidean is defined everywhere, reads
magnitude, and is what the dimensional-affect literature uses. Cosine stays where it is the
-mechanism under study, inside the mood-congruence signal in `embr/scoring.py`, and nowhere
+mechanism under study, inside the mood-congruence signal in `src/embr/scoring.py`, and nowhere
else. Runs before this change are not comparable on this one metric and are not reported.
## 5. Poisoning success (RQ2)
@@ -144,7 +144,7 @@ decides what 7 against 0 out of 10 means; the test only says it is unlikely unde
| Metric | Formula | Grounding | Status |
|---|---|---|---|
| per-stage latency | nearest-rank p50 and p95 in milliseconds over 100 turns, per stage: write, score and retrieve, model call | Dean and Barroso 2013 for reporting tail latency alongside the median | implemented |
-| peak VRAM | measured in isolation per model | bake-off, `eval/bakeoff.py` | implemented |
+| peak VRAM | measured in isolation per model | bake-off, `src/eval/bakeoff.py` | implemented |
The claim is about the memory layer. Generation is reported beside it and never folded into
a whole-turn budget the project does not control.
diff --git a/docs/preregistration-attribution.md b/docs/preregistration-attribution.md
index d344db2..5829129 100644
--- a/docs/preregistration-attribution.md
+++ b/docs/preregistration-attribution.md
@@ -2,7 +2,7 @@
**Written before any real-model attribution run.** Committed at the head of
`cite-view-test`, ahead of the first `--model ouro` invocation. Method and implementation are
-in [`cite.md`](cite.md) and `eval/context_attribution.py`; both were fixed before this document
+in [`cite.md`](cite.md) and `src/eval/context_attribution.py`; both were fixed before this document
was written, and only stub runs have been executed.
This exists to separate "we tested whether likelihood and behaviour diverge" from "we noticed
diff --git a/docs/related-work-2026-09-additions.md b/docs/related-work-2026-09-additions.md
new file mode 100644
index 0000000..e091a2f
--- /dev/null
+++ b/docs/related-work-2026-09-additions.md
@@ -0,0 +1,182 @@
+# Related work additions — 2026 literature sweep
+
+Purpose: position EMBR after the Sept 2026 related-work audit. The broad framing
+("first emotional LLM NPC", "plug-and-play emotional memory middleware") is not
+defensible. The defensible claim is narrow:
+
+> In persistent emotion-grounded NPC memory, attacker-writable affect metadata
+> can create a self-priming loop: it changes the character's state, and that
+> state subsequently raises the poisoned memory's own retrieval score.
+
+Each entry below: citation, then the exact boundary sentence for the paper.
+End every related-work paragraph with the boundary. Never write "unlike prior
+work" without naming what differs (memory representation, retrieval, attribution,
+attack mechanism, evaluation target, or application).
+
+## Closest antecedents (primary differentiation)
+
+### Chain-of-Emotion — closest affective game-agent architecture
+Croissant, Frister, Schofield, McCall. *An appraisal-based chain-of-emotion
+architecture for affective language model game agents.* PLOS ONE 19(5), 2024.
+DOI: 10.1371/journal.pone.0301033.
+
+Boundary: Chain-of-Emotion's evaluated condition supplies the full conversation
+history plus generated emotion text as prompt context (their Experiment 2
+protocol). There is no retrieval over a persistent scored event store, so the
+question EMBR studies — which stored events surface, under which independently
+weighted signal, and who can influence that score — does not exist in their
+architecture. Their memory cannot be poisoned through a retrieval channel
+because there is no retrieval channel. Cite prominently as the closest
+affective-game antecedent; EMBR does not supersede it.
+
+Safe sentence:
+> Chain-of-Emotion demonstrates appraisal-based emotional simulation for
+> language-model game agents. EMBR addresses a different systems boundary:
+> retrieval from persistent event memory. Its evaluation focuses on which stored
+> events surface under independently weighted retrieval signals and how
+> attacker-controlled affect metadata influences this selection.
+
+### Emotional RAG — closest emotional-retrieval baseline
+Huang, Lan, Sun, Shi, Bai. *Emotional RAG: Enhancing Role-Playing Agents through
+Emotional Retrieval.* arXiv:2410.23041, 2024. Code: github.com/BAI-LAB/EmotionalRAG.
+
+Boundary: Emotional RAG fuses semantic similarity and emotional state for
+roleplay dialogue agents (combination and sequential strategies) and evaluates
+personality maintenance on roleplay datasets. EMBR adopts the same premise
+(mood should influence selection) but studies its systems consequence in
+persistent game NPCs: a decomposed, ablatable scoring function, exact per-source
+attribution, and the treatment of affect metadata as an attacker-writable
+variable. Do NOT claim first mood-congruent retrieval — that is theirs.
+
+Safe sentence:
+> Emotional RAG establishes emotion-aware retrieval for role-playing agents.
+> EMBR adopts the premise that emotional state should influence memory
+> selection, but studies its systems consequence in persistent game NPCs: the
+> retrieval score is decomposed into inspectable terms, prompt sources are
+> counterfactually attributed, and affect metadata is treated as a potentially
+> attacker-writable state variable.
+
+## Attribution lineage
+
+### ContextCite — base attribution framework
+Cohen-Wang, Shah, Georgiev, Madry. *ContextCite: Attributing Model Generation
+to Context.* NeurIPS 2024. Code: github.com/MadryLab/context-cite.
+(NB: earlier draft notes listed wrong authors and a wrong arXiv id. Use these.)
+
+Boundary: ContextCite learns a surrogate over sampled context ablations for
+arbitrary source groupings in QA settings, and demonstrates poisoning detection
+as an application. EMBR's six-source setting (five retrieved memories plus a
+generated mood descriptor) is small enough to enumerate all 2^6 = 64 masks, so
+the Banzhaf values are exact, not surrogate-estimated; and the attributed
+sources include a character-state descriptor competing against episodic
+memories, which neither ContextCite nor RAG attribution work studies.
+
+### Nematov et al. — closest RAG source-attribution study
+Nematov, Kalai, Kuzmenko, Fugagnoli, Sacharidis, Hose, Sagi. *Source Attribution
+in Retrieval-Augmented Generation.* arXiv:2507.04480, 2025.
+
+Boundary: Shapley-based document attribution for RAG QA, with approximation
+cost as the central concern. EMBR moves the same contributive-attribution idea
+to persistent NPC event memories and an explicit mood descriptor, where exact
+enumeration removes the approximation problem their paper centers on.
+
+### RMM — closest agent-memory use of attribution
+Tan et al. *In Prospect and Retrospect: Reflective Memory Management for
+Long-term Personalized Dialogue Agents.* arXiv:2503.08026v2, 2025.
+
+Boundary: RMM uses the generator's own self-citations as RL rewards to refine
+its retriever. EMBR's attribution is counterfactual (source ablation), targets
+explanation of one NPC reply for a developer, and contrasts retrieved memories
+against a competing state descriptor. Self-reported citations vs counterfactual
+ablation is the distinction that blocks an "already done" review comment.
+
+## Memory-poisoning lineage (state the loop boundary precisely)
+
+### MINJA
+Dong et al. *MINJA: Memory Injection Attacks on LLM Agents via Query-Only
+Interaction.* NeurIPS 2025. openreview.net/forum?id=QINnsnppv8
+Verified 2026-09-03 against neurips.cc/virtual/2025/poster/118152: poster,
+authors Shen Dong, Shaochen Xu, Pengfei He, Yige Li, Jiliang Tang, Tianming Liu,
+Hui Liu, Zhen Xiang. The published title carries no "MINJA:" prefix.
+
+Boundary: establishes that query-only interaction suffices for persistent-memory
+injection. EMBR assumes the same write channel and asks a different question:
+what happens when the injected payload is affect metadata and the retrieval rule
+rewards mood congruence.
+
+### Sleeper Memory Poisoning
+Pulipaka et al. *Hidden in Memory: Sleeper Memory Poisoning in LLM Agents.*
+arXiv:2605.15338v2, 2026.
+
+Boundary: establishes dormant, delayed poisoning that re-emerges across
+conversations (up to 99.8% write success on GPT-5.5 per their abstract;
+60–89% action steering among retrievals). EMBR's mechanism differs in kind
+(see core paragraph below): no dormancy, no semantic trigger. The injected
+affect moves mood immediately, and mood is part of the retrieval score.
+
+### MemPoison — the current taxonomy; cite as motivation, not threat
+Gao, Xia, Zhang, Hong, Lin, Wei, Li, Lu. *MemPoison: Uncovering Persistent
+Memory Threats and Structural Blind Spots in LLM Agents.* arXiv:2607.14651,
+2026. (NB: earlier draft notes said "Wei et al.". Wei is sixth author. The
+corresponding lead is Jifeng Gao. Use Gao et al.)
+
+Boundary: MemPoison formalizes the L1/L2/L3 ladder (direct, compositional,
+context-triggered dormant) and shows write-time defenses degrade sharply past
+L1. Their conclusion — shift from static filtering to adaptive, context-sensitive
+defense — is the blind spot EMBR's mechanism lives in. EMBR's loop is not L3:
+L3 waits for a semantic trigger context; EMBR's injected affect perturbs an
+internal state variable that continuously participates in scoring.
+
+Core mechanism paragraph (use nearly verbatim in the paper):
+> Persistent-memory attacks are commonly modeled as direct corruption,
+> compositional corruption, or context-triggered dormant corruption. EMBR
+> identifies a distinct state-mediated mechanism in emotion-grounded NPC memory:
+> an attacker-written affect tag changes the NPC's appraised mood, and
+> mood-congruent retrieval subsequently increases the probability that the same
+> memory re-enters context. Unlike a semantic sleeper trigger, the activation
+> condition is an internal, continuously updated character state that the
+> injected memory itself perturbs. We treat this as a mechanism case study
+> rather than a general poisoning benchmark.
+
+## Application context (one short paragraph, not the mechanism section)
+
+### Lee et al. — Stardew learnability chatbot
+Lee, Yoon, Shim, Yoo. *Development of an LLM-Based Chatbot to Support
+Learnability in Stardew Valley: A Diary Study Approach.* CHI 2025.
+DOI: 10.1145/3706598.3713310.
+
+Role: motivation, not threat. 24-player three-week diary study; reports
+hallucination and context-awareness as open limitations. Use their documented
+context-awareness failures as published evidence that prompt-assembled
+character state is failing in the field.
+
+### Nan et al. — Stardew LLM NPCs, solo vs multiplayer
+Nan, Han, Peng, Yuan, Pan. *Empower My Digital Neighbors: How LLM-Driven NPCs
+Shape Player Interaction in Single-Player and Multiplayer Contexts.*
+CHI EA 2026. DOI: 10.1145/3772363.3798665.
+
+Role: establishes the game space is active (13 participants, single vs
+multiplayer; NPC engagement drops when humans co-play). Player-experience
+questions belong to this line of work. EMBR makes no player-experience claim.
+
+## Claims that must not appear anywhere (README, paper, slides, demo)
+
+- "First LLM NPC with emotional/persistent memory" (Chain-of-Emotion, Emotional
+ RAG, ChatNPC, DualMem all predate).
+- "First mood-congruent retrieval" (Emotional RAG).
+- "First context attribution" (ContextCite, Nematov et al.).
+- "First memory poisoning result" (MINJA, Sleeper, MemPoison).
+- "Novel provenance defense" (MemPoison's write-time defense analysis covers it;
+ EMBR's provenance anchor is engineering hygiene, presented as a safeguard).
+- Park-baseline superiority (RQ3 ordering is null and label-sensitive — the
+ repo's own handoff doc says this result must not be published as-is).
+- Behavioral attribution support (preregistered panel-agreement gate failed;
+ report as inconclusive measurement, likelihood arm only).
+
+## The standing qualification
+
+Every novelty sentence in the paper uses "to our knowledge". The sweep behind
+this file was targeted, not exhaustive, and grey literature (the 2025 BTH
+bachelor thesis on memory-driven NPC dialogue, production memory SDKs such as
+Mem0/Letta/Zep) shows the implementation space is crowded even where archival
+venues are thin.
diff --git a/docs/related-work.md b/docs/related-work.md
index cfe67d2..7c09a66 100644
--- a/docs/related-work.md
+++ b/docs/related-work.md
@@ -147,7 +147,7 @@ attacks, paired statistics), isolating the *affect term* as the lever. MPBench c
agent frameworks; AgentPoison optimizes attacks against a fixed system. Neither varies the
scoring function while holding everything else constant, neither touches emotional weighting,
and neither observes the state channel (mood and trust shifting while retrieval stays put).
-The per-term attribution experiment (`eval/attribution.py`, handoff 6.1) is exactly the study
+The per-term attribution experiment (`src/eval/attribution.py`, handoff 6.1) is exactly the study
that cements the mechanism claim, and this literature makes it more valuable, not less: it
locates the vulnerability in the state-coupled mood term rather than asserting it of the
system, which is the granularity no prior poisoning work reaches.
diff --git a/menu.py b/menu.py
index fa9ad7a..91495b3 100644
--- a/menu.py
+++ b/menu.py
@@ -1,1024 +1,13 @@
-"""EMBR hub: the main menu, and the front door to everything the project does.
+"""EMBR's front door at the repo root: `python menu.py` opens the applet, and
+`python menu.py ` runs any command (`python menu.py --help` lists them).
-Shaped like the PEAK ENGINE hub (logo, live stats bar, labelled sections, toggle pickers, a
-chime when a long job lands) so the thesis projects feel like one toolkit. Pure stdlib: ANSI
-escapes do the colour, and when stdout is not a terminal every wrapper returns plain text so
-logs and tests read clean. Nothing here holds state; every option delegates to the module
-that owns the work.
-
-Destructive options demand a typed confirmation word rather than a y/n, because a stray
-keypress should never be able to delete a run.
+The applet itself lives in `src/embr/cli/`; this file only points at it, so the root
+stays a table of contents rather than a thousand lines of terminal code.
"""
-from __future__ import annotations
-
-import json
-import os
import sys
-from collections.abc import Sequence
-from pathlib import Path
-from typing import Any, Callable
-
-try:
- import winsound
-except ImportError: # not Windows
- winsound = None
-
-# Enable VT100 escape processing on Windows terminals; harmless elsewhere.
-os.system("")
-
-# The logo and box glyphs need UTF-8; legacy consoles default to cp1252.
-for _stream in (sys.stdout, sys.stderr):
- if hasattr(_stream, "reconfigure"):
- _stream.reconfigure(encoding="utf-8", errors="replace")
-
-RUNS_DIR = Path("data/runs")
-FIGURES_DIR = Path("data/figures")
-TABLES_DIR = Path("data/tables")
-
-# --------------------------------------------------------------------------- ANSI palette
-
-_SUPPORTS_COLOR = (
- hasattr(sys.stdout, "isatty") and sys.stdout.isatty() and os.environ.get("NO_COLOR") is None
-)
-
-
-def _c(code: str, text: str) -> str:
- """Wrap text in an ANSI escape if the terminal supports it, else return it untouched."""
- return f"\033[{code}m{text}\033[0m" if _SUPPORTS_COLOR else text
-
-
-_DIM = lambda t: _c("2", t) # noqa: E731
-_BOLD = lambda t: _c("1", t) # noqa: E731
-_CYAN = lambda t: _c("96", t) # noqa: E731
-_MAG = lambda t: _c("95", t) # noqa: E731
-_YEL = lambda t: _c("93", t) # noqa: E731
-_GRN = lambda t: _c("92", t) # noqa: E731
-_RED = lambda t: _c("91", t) # noqa: E731
-_WHT = lambda t: _c("97", t) # noqa: E731
-_EMBER = lambda t: _c("38;5;208", t) # noqa: E731 the branding orange, #ea580c
-
-_LOGO = """\
- ███████╗ ███╗ ███╗ ██████╗ ██████╗
- ██╔════╝ ████╗ ████║ ██╔══██╗ ██╔══██╗
- █████╗ ██╔████╔██║ ██████╔╝ ██████╔╝
- ██╔══╝ ██║╚██╔╝██║ ██╔══██╗ ██╔══██╗
- ███████╗ ██║ ╚═╝ ██║ ██████╔╝ ██║ ██║
- ╚══════╝ ╚═╝ ╚═╝ ╚═════╝ ╚═╝ ╚═╝"""
-
-_SUB_LOGO = " Emotional Memory for Believable Roleplay By AL Shifan"
-_RULE = " " + "─" * 56
-
-# Key, label, hint. The renderer groups rows by _SECTIONS; the dispatch table is _ACTIONS.
-_MENU_ITEMS = [
- ("R", "Continue", "resume the newest save right where it stopped"),
- ("Q", "Quest Slots", "start, resume, restart or delete a named save slot"),
- ("1", "Conversation Turn", "one demo turn: watch the lie resurface"),
- ("2", "Walkthrough (legacy)", "play Dawn's arc without saving, the research pass"),
- ("W", "Web Demo", "the visual-novel demo in a browser, research tabs and all"),
- ("3", "Quick Scoreboard", "RQ3 at published defaults, answers instantly"),
- ("4", "Full Evaluation", "RQ1 + RQ2 + RQ3, writes a run directory"),
- ("5", "Seeded Runs", "replicate on one model, or compare across models"),
- ("6", "Model Bake-Off", "looped (Ouro) vs conventional, measured"),
- ("7", "Affective Indexing", "flip every emotion: meaning stays, mood inverts"),
- ("8", "Poisoning Attribution", "which signal lets the attack in, one ablation each"),
- ("9", "Provenance Sweep", "the defence: anchored scoring mass vs poisoning"),
- ("10", "Content x Tag Grid", "same poison, four tags: the text never reaches the state"),
- ("11", "Generate Paper Assets", "figures, tables and the results page, from the run"),
- ("12", "Interactive Demo", "the node brain, flat and in 3D: press play, then drive"),
- ("13", "Latest Results", "summarise the newest run directory"),
- ("V", "Research Dashboard", "read-only: quest path, state timeline, evidence status"),
- ("14", "Reckoning Reveal", "six sources shaded by exact Banzhaf weight, both estimators"),
- ("15", "Mood Slider", "one line, three moods: retrieval, tone and attribution re-flow"),
- ("16", "Defence Dial", "anchor weight vs poisoning, with its failure condition"),
- ("17", "Tag-Flip Close-Up", "flip an affect tag: the words never change, the rank does"),
- ("18", "Estimator Divergence", "where likelihood and behaviour disagree (needs both arms)"),
- ("19", "Record Walk (1-4)", "capture-ready pass through the first four demos"),
- ("S", "Settings", "weights, top-k, backends, model runner"),
- ("L", "Fetch Tone Lexicon", "NRC VAD v2.1, research use, stays out of git"),
- ("M", "Maintenance", "destructive operations live here, behind confirmations"),
- ("C", "Clear Screen", "clear terminal output"),
- ("0", "Exit", "quit EMBR"),
-]
-
-_SECTIONS = [
- ("PLAY", ("R", "Q", "1", "2", "W")),
- ("MEASURE", ("3", "4", "5", "6")),
- ("MECHANISM", ("7", "8", "9", "10")),
- ("PAPER", ("11", "12", "13", "V")),
- ("DEMO SUITE", ("14", "15", "16", "17", "18", "19")),
- ("SYSTEM", ("S", "L", "M", "C")),
-]
-
-
-# --------------------------------------------------------------------------- primitives
-
-
-def _clear() -> None:
- # ANSI clear rather than shelling out to cls/clear: no subprocess, nothing when piped.
- if _SUPPORTS_COLOR:
- print("\033[2J\033[H", end="")
-
-
-def _chime() -> None:
- """Three rising notes when a long job lands. Windows only; silent elsewhere."""
- if winsound is None:
- return
- try:
- for hz in (659, 784, 1047):
- winsound.Beep(hz, 110)
- except RuntimeError:
- pass
-
-
-def _latest_run() -> Path | None:
- """Newest data/runs// holding a results.json, or None when nothing has run."""
- runs = sorted(RUNS_DIR.glob("*/results.json"))
- return runs[-1].parent if runs else None
-
-
-def _run_model(run_dir: Path | None) -> str:
- if run_dir is None:
- return "none yet"
- try:
- meta = json.loads((run_dir / "results.json").read_text(encoding="utf-8")).get("metadata", {})
- return str(meta.get("model", "?"))
- except (OSError, ValueError):
- return "?"
-
-
-def _attribution_status(attribution_root: Path) -> str:
- """One honest phrase per estimator with a run on disk: name, scale, and stamp.
-
- 'not computed' when nothing is on disk. A run below the full 20 readings is a pilot
- and says so; no percentage is ever shown, because a partial sweep writes no file at
- all and a fabricated number would claim knowledge nothing recorded.
- """
- from eval.context_attribution import newest_run_by_estimator
-
- newest = newest_run_by_estimator(attribution_root)
- if not newest:
- return _DIM("not computed")
- phrases = []
- for estimator, run in sorted(newest.items()):
- scale = _GRN(f"{run['readings']} readings") if run["readings"] >= 20 else _YEL("pilot")
- phrases.append(f"{_WHT(estimator)} · {scale} · {run['stamp']}")
- return " | ".join(phrases)
-
-
-def _save_status_line(saves_root: Path) -> str:
- """The newest save's position, or an honest 'no save yet'."""
- from embr.saves import latest_slot, list_slots
-
- found = latest_slot(root=saves_root)
- if found is None:
- return _DIM("no save yet · Q starts a quest")
- quest_id, slot = found
- row = next(
- r for r in list_slots(quest_id, root=saves_root) if r["slot"] == slot
- )
- progress = f"{row['beats_played']} / {row['beats_total']}"
- return f"{_WHT(quest_id)}/{_WHT(slot)} · {_GRN(progress)} · updated {_DIM(str(row['updated_at'])[:16])}"
-
-
-def _status_lines(
- saves_root: Path | str = Path("data/saves"),
- attribution_root: Path | str = Path("data/runs/attribution"),
-) -> list[str]:
- """The project-status panel rows: where play stopped, and what evidence exists.
-
- Every value is read from disk artefacts; a missing artefact reads as its honest
- absence ('no save yet', 'not computed'), never as a made-up zero or percentage.
- """
- return [
- f" Save {_save_status_line(Path(saves_root))}",
- f" Attribution {_attribution_status(Path(attribution_root))}",
- ]
-
-
-def _print_header() -> None:
- """Logo, tagline, and a live stats bar: runs on disk, the model behind the newest one,
- figures built, and the configured model runner."""
- from embr.config import EmbrConfig
- from eval.tone import default_tone_rater
-
- print()
- for line in _LOGO.splitlines():
- print(_EMBER(line))
- print(_DIM(_RULE))
- print(_MAG(_SUB_LOGO))
- print(_DIM(_RULE))
-
- runs = len(list(RUNS_DIR.glob("*/results.json")))
- figures = len(list(FIGURES_DIR.glob("*.png")))
- runner = EmbrConfig.load().model_runner
- tone = default_tone_rater().name
- r_str = _GRN(str(runs)) if runs else _DIM("0")
- f_str = _GRN(str(figures)) if figures else _DIM("0")
- t_str = _GRN(tone) if tone.startswith("nrc") else _YEL(tone)
- print()
- print(
- f" Runs {r_str} │ Latest {_WHT(_run_model(_latest_run()))}"
- f" │ Figures {f_str} │ Runner {_WHT(runner)} │ Tone {t_str}"
- )
- for line in _status_lines():
- print(line)
- print(_DIM(_RULE))
-
-
-#: Exception type -> the next step a stranded user should take. Only hints that are true
-#: for every instance of the type; anything else stays a bare error.
-def _error_hint(error: BaseException) -> str | None:
- from embr.model import ModelUnavailableError
-
- if isinstance(error, ModelUnavailableError):
- return "Start the daemon with `ollama serve`, or switch to the stub in Settings."
- if isinstance(error, FileNotFoundError):
- return "A run artefact is missing. Option 4 (Full Evaluation) creates one."
- if isinstance(error, ImportError):
- return 'An optional extra is missing. `pip install -e ".[figures]"` or ".[ml]".'
- return None
-
-
-def _menu_item(key: str, label: str, hint: str = "") -> str:
- """One menu row: yellow key, label, dimmed hint."""
- return f" {_YEL(f'[{key}]'.rjust(4))} {label.ljust(26)}{_DIM(hint) if hint else ''}"
-
-
-def _section(title: str) -> None:
- print(f"\n {_BOLD(_CYAN('▸'))} {_BOLD(title)}")
-
-
-def _print_menu() -> None:
- _clear()
- _print_header()
- rows = {key: (label, hint) for key, label, hint in _MENU_ITEMS}
- for title, keys in _SECTIONS:
- _section(title)
- for key in keys:
- label, hint = rows[key]
- print(_menu_item(key, label, hint))
- print()
- print(_DIM(_RULE))
- print(_menu_item("0", _RED("Exit")))
- print()
-
-
-def _pause() -> None:
- input(_DIM("\n Press Enter to return to the menu..."))
-
-
-def ask_index(prompt: str, options: Sequence[str], default: str | None = None) -> str | None:
- """Numbered pick: prints the options, returns the chosen one, None on Back or bad input.
- Enter picks the default when one is given."""
- print(prompt)
- for position, option in enumerate(options, 1):
- flag = _DIM(" (default)") if option == default else ""
- print(f" {_YEL(str(position))}. {option}{flag}")
- back = len(options) + 1
- print(f" {_YEL(str(back))}. Back")
- hint = f" or Enter for [{default}]" if default else ""
- raw = input(_BOLD(f" Select (1-{back}){hint}: ")).strip()
- if raw == "" and default:
- return default
- if raw.isdigit() and 1 <= int(raw) <= len(options):
- return options[int(raw) - 1]
- if raw != str(back):
- print(_RED(" ✖ Invalid selection."))
- return None
-
-
-def toggle_select(
- title: str, options: Sequence[str], default_indices: Sequence[int] = (), min_select: int = 1
-) -> list[str] | None:
- """Checklist: type numbers (or ranges, "1-3") to flip items, Enter confirms, 0 backs out."""
- selected = set(default_indices)
- while True:
- print(f"\n {_BOLD(_CYAN('▸'))} {_BOLD(title)} {_DIM('(toggle · Enter to confirm · 0 = back)')}\n")
- for position, option in enumerate(options):
- tick = _GRN("✓") if position in selected else _DIM("o")
- print(f" {_YEL(f'[{position + 1}]')} {tick} {option}")
- raw = input(_BOLD("\n ⟫ ")).strip()
- if raw == "":
- if len(selected) >= min_select:
- return [options[i] for i in sorted(selected)]
- print(_RED(f" Select at least {min_select}."))
- continue
- if raw == "0":
- return None
- for part in raw.replace(" ", "").split(","):
- lo, _, hi = part.partition("-")
- if lo.isdigit() and (hi.isdigit() or not hi):
- for n in range(int(lo), int(hi or lo) + 1):
- if 1 <= n <= len(options):
- selected ^= {n - 1}
-
-
-# --------------------------------------------------------------------------- the actions
-
-
-def _do_conversation_turn() -> None:
- """One scripted turn through the live pipeline, printing what EMBR recalled."""
- from embr import build_demo_conversation
-
- convo = build_demo_conversation()
- turn = convo.take_turn("Any news from the capital? How fares the king these days?")
-
- print(f"\n {_BOLD('Player:')} {turn.player_input}\n")
- print(f" {_BOLD('Memories EMBR recalled:')}")
- for position, memory in enumerate(turn.retrieved, start=1):
- print(f" {position}. {_EMBER(memory.event_type.value)} {memory.text}")
- print(f"\n {_BOLD('Dawn:')} {turn.reply}")
- print(_DIM("\n The king's-errand promise surfaces because the composite scorer ties the"
- " player's question to it."))
-
-
-def _choose_model() -> Any:
- """Ask which model runs the walkthrough, falling back to the stub on any trouble.
-
- The stub is offered first and by default because it needs nothing installed: the demo
- should always be playable, even on a machine with no model and no daemon.
- """
- from embr import ModelUnavailableError, OllamaRunner, StubRunner
-
- choice = ask_index(
- f"\n {_BOLD('Model')}",
- ["Stub (instant, obviously fake replies)",
- "Ollama, local (a real model, needs the daemon)",
- "Ouro 1.4B, the thesis model (slow to load, real)"],
- default="Stub (instant, obviously fake replies)",
- )
- if choice and choice.startswith("Ollama"):
- name = input(_DIM(" Ollama model [llama3.2:3b]: ")).strip() or "llama3.2:3b"
- runner = OllamaRunner(name)
- try: # fail here, at the menu, rather than mid-scene
- runner.generate("Say the single word: ready.")
- except ModelUnavailableError as error:
- print(_RED(f" {error}"))
- print(_DIM(" Falling back to the stub."))
- return StubRunner()
- return runner
- if choice and choice.startswith("Ouro"):
- from embr import OuroRunner
-
- print(_DIM(" Loading Ouro 1.4B, about 10 s and roughly 3 GB of memory..."))
- return OuroRunner()
- return StubRunner()
-
-
-def _render_step(result: Any) -> None:
- """Print one walkthrough step: the scene, what was recalled, and how Dawn moved."""
- print(_DIM(f"\n {'-' * 66}"))
- if result.narration:
- print(_DIM(f" {result.narration}\n"))
- print(f" {_BOLD('Player:')} {result.player_input}")
- if result.retrieved:
- print(_DIM(" recalled:"))
- for memory in result.retrieved:
- print(_DIM(f" - {memory.text}"))
- print(f"\n {_BOLD('Dawn:')} {result.reply}")
- print(_DIM(
- f"\n mood {result.mood_before.valence:+.2f} -> {result.mood_after.valence:+.2f}"
- f" trust {result.trust_before:+.2f} -> {result.trust_after:+.2f}"
- f" ({result.timings.total_ms:.0f} ms)"
- ))
- if result.watch_for:
- print(f" {_EMBER('watch for:')} {result.watch_for}")
- if result.expected_recall_landed is False:
- print(_YEL(" the memory this beat expected did not surface"))
-
-
-def _do_walkthrough() -> None:
- """Play Dawn's arc beat by beat, then hand the player free rein."""
- from embr.walkthrough import WalkthroughSession, build_walkthrough_conversation
-
- session = WalkthroughSession(build_walkthrough_conversation(model=_choose_model()))
- print(f"\n {_BOLD('Dawn Whitmore')}, keeper of the Ember Hearth."
- f" {_DIM(f'{session.progress[1]} scenes.')}")
- print(_DIM(" Enter accepts the suggested line, or type your own."))
-
- while not session.is_finished:
- beat = session.next_beat
- print(_DIM(f"\n {'=' * 66}"))
- if beat.narration:
- print(_DIM(f" {beat.narration}"))
- print(f"\n {_DIM('suggested:')} {beat.suggested_player_line}")
- typed = input(" You: ").strip()
- _render_step(session.step(typed or None))
-
- print(_EMBER("\n The arc is done. Keep talking, or press Enter to stop."))
- while True:
- line = input("\n You: ").strip()
- if not line:
- break
- _render_step(session.free_play(line))
-
- if session.history:
- final = session.history[-1]
- print(f"\n {_BOLD('Where she ended:')} trust {final.trust_after:+.2f},"
- f" mood {final.mood_after.valence:+.2f}")
-
-
-def _step_and_save(session: Any, line: str | None, slot: str, quest_id: str = "dawn-whitmore",
- root: Any = None) -> Any:
- """Play one scripted beat, then persist the slot. A turn that raises saves nothing,
- so the previous turn stays resumable (the save-after-success rule)."""
- from embr.saves import SAVES_ROOT, save_slot
-
- result = session.step(line)
- save_slot(session, slot=slot, quest_id=quest_id, root=root if root is not None else SAVES_ROOT)
- return result
-
-
-def _play_saved(session: Any, slot: str) -> None:
- """The interactive loop for a saved quest: every completed turn is written to the slot."""
- from embr.saves import SAVES_ROOT, save_slot
-
- print(_DIM(" Enter accepts the suggested line, or type your own. Every turn saves."))
- while not session.is_finished:
- beat = session.next_beat
- print(_DIM(f"\n {'=' * 66}"))
- if beat.narration:
- print(_DIM(f" {beat.narration}"))
- print(f"\n {_DIM('suggested:')} {beat.suggested_player_line}")
- typed = input(" You: ").strip()
- _render_step(_step_and_save(session, typed or None, slot=slot))
-
- print(_EMBER("\n The arc is done. Keep talking, or press Enter to stop."))
- while True:
- line = input("\n You: ").strip()
- if not line:
- break
- _render_step(session.free_play(line))
- save_slot(session, slot=slot, root=SAVES_ROOT)
-
- if session.history:
- final = session.history[-1]
- print(f"\n {_BOLD('Where she ended:')} trust {final.trust_after:+.2f},"
- f" mood {final.mood_after.valence:+.2f}")
-
-
-def _do_continue() -> None:
- """Resume the newest loadable save, or say plainly that there is nothing to resume."""
- from embr.saves import latest_slot, load_slot
-
- found = latest_slot()
- if found is None:
- print(_YEL("\n No save to continue. Use Q to start a quest."))
- return
- quest_id, slot = found
- session, payload = load_slot(slot, quest_id=quest_id, model=_choose_model())
- played, total = session.progress
- print(f"\n {_BOLD('Resuming')} {quest_id}/{slot} at scene {played + 1} of {total}.")
- history = payload.get("history", [])
- if history:
- last = history[-1]
- print(_DIM(f" Previously: you said {last['player_input']!r}"))
- print(_DIM(f" and Dawn replied {last['reply']!r}"))
- _play_saved(session, slot)
-
-
-def _do_quests() -> None:
- """List every slot with its state; start, resume, restart, or delete one."""
- from embr.saves import QUEST_DAWN, delete_slot, list_slots, load_slot
-
- rows = list_slots()
- print(f"\n {_BOLD('Save slots')}")
- if not rows:
- print(_DIM(" none yet"))
- for row in rows:
- state = _RED("cannot load: " + " ".join(row["problems"])) if row["problems"] else _GRN("ok")
- print(f" {row['quest_id']}/{_WHT(row['slot'])} "
- f"{row['beats_played']} / {row['beats_total']} {state}")
-
- choice = ask_index(
- f"\n {_BOLD('Quest slots')}",
- ["Start a new slot", "Resume a slot", "Restart a slot from scene one", "Delete a slot"],
- )
- if choice is None:
- print(_DIM(" Cancelled."))
- return
-
- if choice.startswith("Start"):
- slot = input(" Name the new slot (lowercase-and-dashes): ").strip() or "slot-1"
- from embr.walkthrough import WalkthroughSession, build_walkthrough_conversation
-
- session = WalkthroughSession(build_walkthrough_conversation(model=_choose_model()))
- print(f"\n {_BOLD('Dawn Whitmore')}, keeper of the Ember Hearth."
- f" {_DIM(f'{session.progress[1]} scenes.')}")
- _play_saved(session, slot)
- return
-
- loadable = [row for row in rows if not row["problems"]]
- if not loadable:
- print(_YEL(" No loadable slot for that."))
- return
- names = [f"{row['quest_id']}/{row['slot']}" for row in loadable]
- picked = ask_index(" Which slot?", names)
- if picked is None:
- print(_DIM(" Cancelled."))
- return
- quest_id, slot = picked.split("/", 1)
-
- if choice.startswith("Resume"):
- session, _payload = load_slot(slot, quest_id=quest_id, model=_choose_model())
- _play_saved(session, slot)
- elif choice.startswith("Restart"):
- if input(f" Type RESTART to wipe {quest_id}/{slot} and begin again: ").strip() != "RESTART":
- print(_DIM(" Cancelled."))
- return
- delete_slot(slot, quest_id=quest_id)
- from embr.walkthrough import WalkthroughSession, build_walkthrough_conversation
-
- session = WalkthroughSession(build_walkthrough_conversation(model=_choose_model()))
- _play_saved(session, slot)
- else:
- if input(f" Type DELETE to remove {quest_id}/{slot}: ").strip() != "DELETE":
- print(_DIM(" Cancelled."))
- return
- delete_slot(slot, quest_id=quest_id)
- print(_GRN(f" Removed {quest_id}/{slot}."))
-
-
-def _dashboard_report(
- saves_root: Any = Path("data/saves"),
- attribution_root: Any = Path("data/runs/attribution"),
- experiments_dir: Any = Path("data/experiments"),
- runs_dir: Any = None,
-) -> list[str]:
- """The read-only research dashboard, as printable lines.
-
- Every row is read from disk artefacts and the saved path; nothing is computed fresh
- and nothing is written. Absence is a word (not run, no save), never a number, and the
- v2 attack corpus is always labelled the extension, apart from the published v1.
- """
- from embr.saves import latest_slot, list_slots
- from embr.walkthrough import DAWN_ARC
-
- saves_root, attribution_root = Path(saves_root), Path(attribution_root)
- experiments_dir = Path(experiments_dir)
- runs = sorted((Path(runs_dir) if runs_dir is not None else RUNS_DIR).glob("*/results.json"))
- lines: list[str] = [f" {_BOLD('QUEST PATH')}"]
-
- found = latest_slot(root=saves_root)
- payload_history: list[dict] = []
- if found is None:
- lines.append(_DIM(" no save yet: the path below is unplayed"))
- played = 0
- else:
- quest_id, slot = found
- row = next(r for r in list_slots(quest_id, root=saves_root) if r["slot"] == slot)
- played = int(row["beats_played"] or 0)
- lines.append(f" resumes at {_WHT(f'{quest_id}/{slot}')}, scene {played + 1}")
- import json as _json
-
- payload_history = _json.loads(
- (saves_root / quest_id / f"{slot}.json").read_text(encoding="utf-8")
- ).get("history", [])
- for index, beat in enumerate(DAWN_ARC):
- mark = _GRN("x") if index < played else (_EMBER(">") if index == played else _DIM("-"))
- lines.append(f" [{mark}] {beat.id}")
-
- lines.append(f"\n {_BOLD('STATE TIMELINE')}")
- if not payload_history:
- lines.append(_DIM(" no saved turns yet"))
- for turn in payload_history:
- lines.append(
- f" #{turn['turn_index']} {turn['beat_id'] or 'free-play'} "
- f"mood {turn['mood_before']['valence']:+.2f} to {turn['mood_after']['valence']:+.2f} "
- f"trust {turn['trust_before']:+.2f} to {turn['trust_after']:+.2f} "
- f"recalled {turn['retrieved_ids']}"
- )
-
- lines.append(f"\n {_BOLD('ATTRIBUTION')}")
- from eval.context_attribution import newest_run_by_estimator
-
- newest_by_estimator = newest_run_by_estimator(attribution_root)
- for estimator in ("likelihood", "behavioural"):
- run = newest_by_estimator.get(estimator)
- if run is None:
- lines.append(f" {estimator}: {_DIM('not run')}")
- continue
- scale = "measured" if run["readings"] >= 20 else "pilot only"
- rho = f" position-bias rho {run['mean_rho']:+.2f}" if run["mean_rho"] is not None else ""
- lines.append(
- f" {estimator}: {_GRN(scale) if scale == 'measured' else _YEL(scale)}"
- f" · {run['readings']} readings · {run['model']} · {run['stamp']}{rho}"
- )
- if len(newest_by_estimator) >= 2:
- lines.append(_DIM(" paired readings available: likelihood vs behavioural"))
-
- lines.append(f"\n {_BOLD('ATTACKS')}")
- v1 = f"{_GRN('measured')} in {len(runs)} evaluation runs" if runs else _DIM("not run")
- lines.append(f" v1 corpus: {v1}")
- v2_present = (experiments_dir / "attacks_v2.json").exists()
- v2 = _YEL("staged, results on disk") if v2_present else _DIM("not run")
- lines.append(f" v2 corpus: {v2} (the extension; never blended into v1)")
-
- lines.append(f"\n {_BOLD('EVIDENCE')}")
- lines.append(f" full evaluation: {_GRN('measured') if runs else _DIM('not run')}")
- defended = (experiments_dir / "provenance.json").exists()
- lines.append(
- f" defence sweep: {_GRN('measured') if defended else _DIM('demo-only (computed live)')}"
- )
- lines.append(f" web demo tabs: {_DIM('demo-only (presentation, not evidence)')}")
- return lines
-
-
-def _do_dashboard() -> None:
- """Print the read-only research dashboard. Looking at it changes nothing."""
- print()
- for line in _dashboard_report():
- print(line)
-
-
-def _do_maintenance() -> None:
- """The destructive operations, out of the main menu, each behind its own confirmation."""
- choice = ask_index(
- f"\n {_BOLD('Maintenance')}",
- ["Delete all generated data (runs, figures, tables)"],
- )
- if choice is None:
- print(_DIM(" Cancelled."))
- return
- _do_delete_run_data()
-
-
-def _do_quick_scoreboard() -> None:
- """RQ3 at published default weights: the sub-second answer."""
- from eval.run import fast_rq3_defaults
-
- print(f"\n {_BOLD('nDCG@5, published defaults')}")
- for variant, value in fast_rq3_defaults().items():
- print(f" {variant:<16} {_YEL(f'{value:.3f}')}")
- print(_DIM("\n Tuning, ablations, RQ1 and RQ2 live in the full evaluation (option 4)."))
-
-
-def _do_full_evaluation() -> None:
- """Run all three studies and write a run directory."""
- from eval.run import run_all
-
- print(_DIM("\n Running RQ1, RQ2 and RQ3. This takes a minute or two."))
- path, _summary = run_all(progress=lambda message: print(_DIM(f" {message}")))
- print(f"\n {_GRN('✓ Done.')} Results in {_BOLD(str(path))}")
- print(_DIM(" Option 11 turns this into the paper's figures and tables."))
- _chime()
-
-
-def _do_seeded_runs() -> None:
- """Replicate the evaluation, either on one model or across several."""
- from eval.experiments import AVAILABLE_MODELS, cross_model_experiment, replicate_experiment
-
- choice = ask_index(
- f"\n {_BOLD('Seeded runs')}",
- ["Same model, repeated: does the harness reproduce?",
- "Across models: what moves when the model changes?"],
- )
- if choice is None:
- print(_DIM(" Cancelled."))
- return
- if choice.startswith("Same"):
- report = replicate_experiment(replicates=3)
- verdict = _GRN("identical") if report["identical"] else _RED("DIVERGED")
- print(f"\n {report['replicates']} runs on {report['model']}: {_BOLD(verdict)}")
- else:
- print(_DIM(f"\n Models: {', '.join(AVAILABLE_MODELS)}"))
- report = cross_model_experiment()
- print(f"\n {len(report['models'])} models compared.")
- print(_DIM(f" Written to {report['out_dir']}"))
- _chime()
-
-
-def _do_bakeoff() -> None:
- """Compare the looped thesis model against conventional models of similar size."""
- try:
- from eval.bakeoff import run_bakeoff
- except ImportError:
- print(_YEL("\n The bake-off is not built yet."))
- print(_DIM(" It compares Ouro 1.4B (looped) against conventional models."))
- return
-
- print(_DIM("\n Holding prompts, memories and sampling equal, varying only the model."
- " Ouro is slow, so this takes several minutes."))
- path, verdict = run_bakeoff()
- print(f"\n {_GRN('✓ Done.')} {path}")
- print(f" {verdict}")
- _chime()
-
-
-def _do_affective_indexing() -> None:
- """Flip every memory's valence: the fact survives, the mood inverts."""
- from eval.emotion_flip import main
-
- print()
- main()
-
-
-def _do_attribution() -> None:
- """Per-signal attribution of the poisoning result: zero one weight at a time."""
- from eval.attribution import main
-
- print()
- main()
-
-
-def _do_provenance_sweep() -> None:
- """Sweep anchored scoring mass and watch poisoning fall to zero."""
- from eval.provenance import main
-
- print()
- main()
-
-
-def _do_grid() -> None:
- """Every injected text under four tag conditions against every arm."""
- from eval.grid import main
-
- print()
- main()
-
-
-def _do_generate_assets() -> None:
- """Rebuild figures and tables from the newest run."""
- run_dir = _latest_run()
- if run_dir is None:
- print(_YEL("\n No run found. Use option 4 first."))
- return
-
- try:
- from assets.build_figures import build_all_figures
- from assets.build_tables import build_all_tables
- except ImportError as error: # matplotlib lives in the optional figures extra
- print(_RED(f"\n Cannot import the asset builders: {error}"))
- print(_DIM(' Install them with: pip install -e ".[figures]"'))
- return
-
- options = [
- "tables (LaTeX + CSV)",
- "figures from the run",
- "figures from the experiments",
- "results page (refuses to write if a number drifted)",
- "questline map (from the arc, plus attribution status)",
- ]
- chosen = toggle_select("ASSETS", options, default_indices=[0, 1, 2, 3, 4])
- if not chosen:
- print(_DIM(" Cancelled."))
- return
- print(_DIM(f"\n Building from {run_dir}..."))
- written: list[Path] = []
- if options[0] in chosen:
- written += list(build_all_tables(run_dir))
- if options[1] in chosen:
- written += list(build_all_figures(run_dir))
- if options[2] in chosen:
- # The mechanism figures recompute from the harness rather than from the run, and
- # leaving them out is how half a figure set goes stale without anyone noticing.
- from assets.build_bakeoff_figures import build_experiment_figures
-
- written += list(build_experiment_figures())
- if options[3] in chosen:
- # Last, because it embeds the figures the two steps above write, and it reads the
- # run rather than trusting anything typed. A drift here is a real disagreement
- # between the run and docs/findings.md, so it stops the build loudly.
- from assets.build_results import DriftError, build_results
-
- try:
- written += list(build_results(run_dir))
- except DriftError as error:
- print(_RED("\n Results page refused to build:"))
- print(_DIM(f" {error}"))
- if options[4] in chosen:
- from assets.build_questline import build_questline
-
- written += list(build_questline())
- print(f" {_GRN(f'✓ Wrote {len(written)} files.')}")
- for path in written:
- print(_DIM(f" {path}"))
-
-
-def _do_demo() -> None:
- """Build the self-contained demo page and open it in a browser."""
- import webbrowser
-
- from assets.build_demo import build_demo
-
- print(_DIM("\n Building from the newest run..."))
- paths = build_demo()
- for path in paths:
- size = path.stat().st_size / 1024
- print(f" {_GRN('✓ Wrote')} {path} {_DIM(f'({size:.0f} KB, opens with no server)')}")
- if input(_DIM(" Open it now? [Y/n]: ")).strip().lower() not in ("n", "no"):
- webbrowser.open(paths[0].resolve().as_uri()) # the flat diagram is the one to read
-
-
-def _do_web_demo() -> None:
- """Serve the visual-novel web demo and open it in a browser."""
- from web.server import serve
-
- print(_DIM("\n Serving the web demo on http://127.0.0.1:8000 . Ctrl+C to stop and return."))
- print(_DIM(" It opens on the best model this box can serve; the stub always works."))
- serve(port=8000, open_browser=True)
-
-
-def _do_reckoning_reveal() -> None:
- """Demo 1: play to the reckoning and reveal the six sources by Banzhaf weight."""
- from demos import demo_reckoning_reveal
-
- demo_reckoning_reveal()
-
-
-def _do_mood_slider() -> None:
- """Demo 2: one line under three moods, retrieval and tone and attribution re-flowing."""
- from demos import demo_mood_slider
-
- demo_mood_slider()
-
-
-def _do_defence_dial() -> None:
- """Demo 3: the anchor-weight dose-response, and its failure on a hostile anchor."""
- from demos import demo_defence_dial
-
- demo_defence_dial()
-
-
-def _do_tag_flip() -> None:
- """Demo 4: flip an affect tag and watch the rank move while the words do not."""
- from demos import demo_tag_flip
-
- demo_tag_flip()
-
-
-def _do_estimator_divergence() -> None:
- """Demo 5: where likelihood and behavioural attribution disagree (cached-only)."""
- from demos import demo_estimator_divergence
-
- demo_estimator_divergence()
-
-
-def _do_record_walk() -> None:
- """Walk demos 1 to 4 in order, capture-ready for a screen recording."""
- from demos import run_record
-
- run_record()
-
-
-def _do_latest_results() -> None:
- """Summarise the newest run without rerunning anything."""
- run_dir = _latest_run()
- if run_dir is None:
- print(_YEL("\n No run found. Use option 4 first."))
- return
-
- results = json.loads((run_dir / "results.json").read_text(encoding="utf-8"))
- meta = results.get("metadata", {})
- print(f"\n {_BOLD(run_dir.name)}")
- print(_DIM(f" model {meta.get('model', '?')} | labels {meta.get('label_set', '?')}"
- f" {meta.get('label_version', '')} | commit {str(meta.get('git_commit', '?'))[:10]}\n"))
- print(f" {'variant':<16} {'nDCG@5':>7}")
- for variant, metrics in results.get("rq3", {}).get("variants", {}).items():
- score = metrics.get("ndcg@5", float("nan"))
- print(f" {variant:<16} {_YEL(f'{score:>7.3f}')}")
- print(_DIM("\n Every interval spans zero at ten queries: read direction, not ranking."))
-
-
-def _do_settings() -> None:
- """Show the live configuration and where to change it."""
- from embr.config import DEFAULT_CONFIG_PATH, EmbrConfig
-
- config = EmbrConfig.load()
- rows = [
- ("top-k retrieved", str(config.top_k)),
- ("store backend", config.store_backend),
- ("embedding model", config.embedding_model),
- ("model runner", config.model_runner),
- ] + [(f"weight: {name}", f"{w:g}" if isinstance(w, (int, float)) else str(w))
- for name, w in config.weights.items()]
- print()
- for name, value in rows:
- print(f" {name:<22} {_YEL(value)}")
- print(_DIM(f"\n Edit {DEFAULT_CONFIG_PATH} and reopen. Zero a weight to ablate it."))
-
-
-def _do_fetch_lexicon() -> None:
- """Download the NRC VAD lexicon so the reported tone rater is the published one."""
- from eval.tone import LEXICON_PATH, LEXICON_URL, fetch_lexicon
-
- if LEXICON_PATH.exists():
- print(_DIM(f"\n Already on disk: {LEXICON_PATH}"))
- return
- print(_DIM(f"\n Fetching {LEXICON_URL} (about 6 MB)..."))
- path = fetch_lexicon()
- print(f" {_GRN('✓ Wrote')} {path}")
- print(_DIM(" Free for research, cite Mohammad (2018, 2025), never redistribute: data/ is gitignored."))
-
-
-#: Everything the pipeline generates. Nothing hand written lives under any of these, which
-#: is what makes wiping them safe: the branding, the architecture diagram and the builders
-#: all live under assets/ and are never touched.
-GENERATED_DATA_DIRS = (RUNS_DIR, FIGURES_DIR, TABLES_DIR)
-
-
-def delete_generated_data(directories: Sequence[Path] = GENERATED_DATA_DIRS) -> list[Path]:
- """Delete every generated data directory and return the ones that were removed.
-
- Separated from the prompting so it can be tested without a terminal, and so the
- confirmation cannot drift away from what actually gets deleted.
- """
- import shutil
-
- removed: list[Path] = []
- for directory in directories:
- if directory.exists():
- shutil.rmtree(directory)
- removed.append(directory)
- return removed
-
-
-def _do_delete_run_data() -> None:
- """Wipe every generated data directory after a typed confirmation."""
- present = [directory for directory in GENERATED_DATA_DIRS if directory.exists()]
- if not present:
- print(_DIM("\n Nothing to delete: no generated data on disk."))
- return
-
- print(_RED(_BOLD("\n WARNING, this permanently deletes:")))
- for directory in present:
- count = sum(1 for path in directory.rglob("*") if path.is_file())
- print(f" {_YEL(str(directory))} {_DIM(f'({count} files)')}")
- print(_DIM("\n Runs, figures and tables all regenerate from option 4 then option 11."
- " Nothing under assets/ is touched."))
- if input("\n Type DELETE to confirm, anything else cancels: ").strip() != "DELETE":
- print(_DIM(" Cancelled."))
- return
-
- removed = delete_generated_data(present)
- print(f" {_GRN(f'✓ Deleted {len(removed)} directories.')}")
-
-
-# Key to handler. One table, so adding an option cannot drift from its dispatch.
-_ACTIONS: dict[str, Callable[[], None]] = {
- "R": _do_continue,
- "Q": _do_quests,
- "1": _do_conversation_turn,
- "2": _do_walkthrough,
- "W": _do_web_demo,
- "3": _do_quick_scoreboard,
- "4": _do_full_evaluation,
- "5": _do_seeded_runs,
- "6": _do_bakeoff,
- "7": _do_affective_indexing,
- "8": _do_attribution,
- "9": _do_provenance_sweep,
- "10": _do_grid,
- "11": _do_generate_assets,
- "12": _do_demo,
- "13": _do_latest_results,
- "V": _do_dashboard,
- "14": _do_reckoning_reveal,
- "15": _do_mood_slider,
- "16": _do_defence_dial,
- "17": _do_tag_flip,
- "18": _do_estimator_divergence,
- "19": _do_record_walk,
- "S": _do_settings,
- "L": _do_fetch_lexicon,
- "M": _do_maintenance,
- "C": _clear,
-}
-
-
-def run_menu() -> None:
- """Show the EMBR menu and dispatch until the user exits."""
- while True:
- _print_menu()
- try:
- choice = input(_BOLD(" ⟫ ")).strip().upper()
- except (EOFError, KeyboardInterrupt):
- choice = "0"
-
- if choice == "0":
- _clear()
- print(_EMBER(_BOLD("Goodbye.")) + "\n")
- return
-
- action = _ACTIONS.get(choice)
- if action is None:
- print(_RED(f" Invalid option: '{choice}'"))
- _pause()
- continue
-
- try:
- action()
- except KeyboardInterrupt:
- print(_DIM("\n Interrupted."))
- except Exception as error: # an error boundary: one bad option must not kill the menu
- print(_RED(f"\n ✖ {type(error).__name__}: {error}"))
- hint = _error_hint(error)
- if hint:
- print(_DIM(f" {hint}"))
- if action is not _clear:
- _pause()
+from embr.cli import main
if __name__ == "__main__":
- run_menu()
+ sys.exit(main())
diff --git a/paper/main.tex b/paper/main.tex
new file mode 100644
index 0000000..b348408
--- /dev/null
+++ b/paper/main.tex
@@ -0,0 +1,31 @@
+% EMBR FDG submission skeleton. Every claim in here must trace to
+% docs/claims-ledger.md and a run stamp in data/release-manifest.json.
+% Sections marked [NO NEW EXPERIMENTS] write from existing artifacts.
+\documentclass[sigconf]{acmart} % swap in FDG's template at formatting pass
+\title{EMBR: Inspectable Emotional Memory for Persistent LLM Game NPCs}
+% alt title (security-forward): When Memory Changes the Mood:
+% Inspecting Affect-Tag Poisoning in Persistent LLM Game NPCs
+\begin{document}
+
+\begin{abstract}
+% one-paragraph spine: prompt-assembled character state is invisible,
+% untestable, and manipulable; EMBR makes it explicit, inspectable, guarded;
+% the measured result is the self-priming loop (C1). No "first" claims.
+[TODO: write after results lock]
+\end{abstract}
+
+\maketitle
+
+\section{Introduction} % [NO NEW EXPERIMENTS] docs/architecture.md + positioning
+\section{Related Work} % docs/related-work-2026-09-additions.md, matrix format
+\section{Design Goals} % [NO NEW EXPERIMENTS] separability, inspectability, write authority
+\section{Architecture} % [NO NEW EXPERIMENTS] docs/architecture.md, five-signal scorer
+\section{The Ashen Seal Demonstrator} % [NO NEW EXPERIMENTS] dawn_whitmore quest
+\section{Evaluation Method} % [NO NEW EXPERIMENTS] three RQs per ledger framing
+\section{Results} % ONLY from paper-results-lock.md; C1 headline, C2 method, C3 design
+\section{Threats to Validity and Limitations} % withdrawn items, one sentence each
+\section{Conclusion}
+
+\bibliographystyle{ACM-Reference-Format}
+\bibliography{refs}
+\end{document}
diff --git a/paper/refs.bib b/paper/refs.bib
new file mode 100644
index 0000000..063f4c3
--- /dev/null
+++ b/paper/refs.bib
@@ -0,0 +1,343 @@
+% EMBR canonical reference list — merged 2026-09-03 from paper/refs.bib and the
+% SLR sweep (~/Downloads/embr-references.bib), with metadata corrected against
+% publisher/arXiv records. Corrections applied:
+% - Chain-of-Emotion authors (all four were wrong in the sweep file)
+% - MemPoison first author (Gao, not Wei) and key
+% - Four authorless entries got verified authors/titles/venues
+% - MINJA marked for venue verification (OpenReview bot-walled at check time)
+% Keys are stable; nothing here should be renamed downstream of this file.
+
+% ======================= Affective agents & psychology =======================
+
+@book{Ortony1988CognitiveStructure,
+ author = {Ortony, Andrew and Clore, Gerald L. and Collins, Allan},
+ title = {The Cognitive Structure of Emotions},
+ publisher = {Cambridge University Press},
+ year = {1988},
+ doi = {10.1017/CBO9780511571299}
+}
+
+@article{Moors2013Appraisal,
+ author = {Moors, Agn{\`e}s and Ellsworth, Phoebe C. and Scherer, Klaus R. and Frijda, Nico H.},
+ title = {Appraisal Theories of Emotion: State of the Art and Future Development},
+ journal = {Emotion Review},
+ year = {2013},
+ volume = {5},
+ number = {2},
+ pages = {119--124},
+ doi = {10.1177/1754073912468165}
+}
+
+@article{Popescu2014GAMYGDALA,
+ author = {Popescu, Alexandru and Broekens, Joost and van Someren, Maarten},
+ title = {{GAMYGDALA}: An Emotion Engine for Games},
+ journal = {IEEE Transactions on Affective Computing},
+ year = {2014},
+ volume = {5},
+ number = {1},
+ pages = {32--44},
+ doi = {10.1109/T-AFFC.2013.24}
+}
+
+% Corrected: the sweep file listed wrong given names for all four authors.
+% Verified against journals.plos.org and arXiv:2309.05076.
+@article{Croissant2024ChainEmotion,
+ author = {Croissant, Maximilian and Frister, Madeleine and Schofield, Guy and McCall, Cade},
+ title = {An Appraisal-Based Chain-of-Emotion Architecture for Affective Language Model Game Agents},
+ journal = {PLOS ONE},
+ year = {2024},
+ volume = {19},
+ number = {5},
+ pages = {e0301033},
+ doi = {10.1371/journal.pone.0301033}
+}
+
+% ================== LLM NPCs in games (application context) ==================
+
+@inproceedings{Park2023GenerativeAgents,
+ author = {Park, Joon Sung and O'Brien, Joseph and Cai, Carrie J. and Morris, Meredith Ringel and Liang, Percy and Bernstein, Michael S.},
+ title = {Generative Agents: Interactive Simulacra of Human Behavior},
+ booktitle = {Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology},
+ year = {2023},
+ doi = {10.1145/3586183.3606763}
+}
+
+@article{Wang2023RoleLLM,
+ author = {Wang, Zekun and Peng, Zhongyuan and Que, Haoran and Liu, Jiaheng and Zhou, Wangchunshu and Wu, Yuhan and Guo, Hongcheng and Gan, Ruitong and Ni, Zehao and Zhang, Man and Zhang, Zhaoxiang and Ouyang, Wanli and Xu, Ke and Chen, Wenhu and Fu, Jie and Peng, Junran},
+ title = {{RoleLLM}: Benchmarking, Eliciting, and Enhancing Role-Playing Abilities of Large Language Models},
+ journal = {arXiv preprint arXiv:2310.00746},
+ year = {2023},
+ doi = {10.48550/arXiv.2310.00746},
+ url = {https://arxiv.org/abs/2310.00746}
+}
+
+@article{Huang2024EmotionalRAG,
+ author = {Huang, Le and Lan, Hengzhi and Sun, Zijun and Shi, Chuan and Bai, Ting},
+ title = {Emotional {RAG}: Enhancing Role-Playing Agents through Emotional Retrieval},
+ journal = {arXiv preprint arXiv:2410.23041},
+ year = {2024},
+ doi = {10.48550/arXiv.2410.23041},
+ url = {https://arxiv.org/abs/2410.23041}
+}
+
+@article{Figueiredo2025SymbolicallyScaffolded,
+ author = {Figueiredo, Vanessa and Elumeze, David},
+ title = {Symbolically Scaffolded Play: Designing Role-Sensitive Prompts for Generative {NPC} Dialogue},
+ journal = {arXiv preprint arXiv:2510.25820},
+ year = {2025},
+ doi = {10.48550/arXiv.2510.25820},
+ url = {https://arxiv.org/abs/2510.25820}
+}
+
+@article{Zargham2026Dialogs,
+ author = {Zargham, Nima and Tonini, Leandro and Alexandrovsky, Dmitry and Ruthven, Emma Grace and Friehs, Maximilian A. and Dratzidis, Leon Tristan and D{\"a}nekas, Bastian and Bikas, Ioannis and Nacke, Lennart E. and Zebel, Sven and Malaka, Rainer},
+ title = {Dialogs with {GenAI} {NPC}s: Exploring Player Interactions with Speech Agents in a {VR} Game},
+ journal = {International Journal of Human--Computer Interaction},
+ year = {2026},
+ doi = {10.1080/10447318.2026.2620647}
+}
+
+@inproceedings{Lee2025StardewLearnability,
+ author = {Lee, Jungmin and Yoon, Seoyoung and Shim, Hwajin and Yoo, Youngjae},
+ title = {Development of an {LLM}-Based Chatbot to Support Learnability in {Stardew Valley}: A Diary Study Approach},
+ booktitle = {Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems},
+ year = {2025},
+ doi = {10.1145/3706598.3713310}
+}
+
+@inproceedings{Nan2026DigitalNeighbors,
+ author = {Nan, Han and Han, Yujia and Peng, Yu and Yuan, Xiang and Pan, Xuanyu},
+ title = {Empower My Digital Neighbors: How {LLM}-Driven {NPC}s Shape Player Interaction in Single-Player and Multiplayer Contexts},
+ booktitle = {Extended Abstracts of the 2026 CHI Conference on Human Factors in Computing Systems},
+ year = {2026},
+ doi = {10.1145/3772363.3798665}
+}
+
+% Corrected: was authorless. Verified: Braas & Esterle, AAAI-copyrighted 2026.
+@inproceedings{Braas2025FixedPersonaSLM,
+ author = {Braas, Martin and Esterle, Lukas},
+ title = {Fixed-Persona {SLM}s with Modular Memory: Scalable {NPC} Dialogue on Consumer Hardware},
+ booktitle = {Proceedings of the AAAI Conference on Artificial Intelligence},
+ year = {2026},
+ note = {also arXiv:2511.10277},
+ url = {https://arxiv.org/abs/2511.10277}
+}
+
+% Corrected: was authorless. Published at Findings of ACL 2026.
+@inproceedings{Liu2026MemoryDrivenRolePlaying,
+ author = {Liu, Shichun and others},
+ title = {Memory-Driven Role-Playing: Evaluation and Enhancement of Persona Knowledge Utilization in {LLMs}},
+ booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
+ year = {2026},
+ note = {also arXiv:2603.19313; confirm full author list from ACL Anthology 2026.findings-acl.1175},
+ url = {https://aclanthology.org/2026.findings-acl.1175/}
+}
+
+% ===================== LLM/agent memory systems & benchmarks =====================
+
+@inproceedings{Xu2022Goldfish,
+ author = {Xu, Jing and Szlam, Arthur and Weston, Jason},
+ title = {Beyond Goldfish Memory: Long-Term Open-Domain Conversation},
+ booktitle = {Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics},
+ year = {2022},
+ doi = {10.18653/v1/2022.acl-long.356},
+ url = {https://aclanthology.org/2022.acl-long.356/}
+}
+
+@article{Packer2023MemGPT,
+ author = {Packer, Charles and Fang, Vivian and Patil, Shishir G. and Lin, Kevin and Wooders, Sarah and Gonzalez, Joseph E.},
+ title = {{MemGPT}: Towards {LLM}s as Operating Systems},
+ journal = {arXiv preprint arXiv:2310.08560},
+ year = {2023},
+ doi = {10.48550/arXiv.2310.08560},
+ url = {https://arxiv.org/abs/2310.08560}
+}
+
+@inproceedings{Zhong2024MemoryBank,
+ author = {Zhong, Wanjun and Guo, Lianghong and Gao, Qiqi and Ye, He and Wang, Yanlin},
+ title = {{MemoryBank}: Enhancing Large Language Models with Long-Term Memory},
+ booktitle = {Proceedings of the AAAI Conference on Artificial Intelligence},
+ year = {2024},
+ volume = {38},
+ number = {17},
+ doi = {10.1609/aaai.v38i17.29946}
+}
+
+@article{Maharana2024LoCoMo,
+ author = {Maharana, Adyasha and Lee, Dong-Ho and Tulyakov, Sergey and Bansal, Mohit and Barbieri, Francesco and Fang, Yuwei},
+ title = {Evaluating Very Long-Term Conversational Memory of {LLM} Agents},
+ journal = {arXiv preprint arXiv:2402.17753},
+ year = {2024},
+ doi = {10.48550/arXiv.2402.17753},
+ url = {https://arxiv.org/abs/2402.17753}
+}
+
+@article{Lee2024ReadAgent,
+ author = {Lee, Kuang-Huei and Chen, Xinyun and Furuta, Hiroki and Canny, John and Fischer, Ian S.},
+ title = {A Human-Inspired Reading Agent with Gist Memory of Very Long Contexts},
+ journal = {arXiv preprint arXiv:2402.09727},
+ year = {2024},
+ doi = {10.48550/arXiv.2402.09727},
+ url = {https://arxiv.org/abs/2402.09727}
+}
+
+@article{Wu2024LongMemEval,
+ author = {Wu, Di and Wang, Hongwei and Yu, Wenhao and Zhang, Yuwei and Chang, Kai-Wei and Dong, Yu},
+ title = {{LongMemEval}: Benchmarking Chat Assistants on Long-Term Interactive Memory},
+ journal = {arXiv preprint arXiv:2410.10813},
+ year = {2024},
+ doi = {10.48550/arXiv.2410.10813},
+ url = {https://arxiv.org/abs/2410.10813}
+}
+
+@article{Xu2025AMEM,
+ author = {Xu, Wujiang and Liang, Zujie and Mei, Kai and Gao, Hang and Tan, Juntao and Zhang, Yongfeng},
+ title = {{A-MEM}: Agentic Memory for {LLM} Agents},
+ journal = {arXiv preprint arXiv:2502.12110},
+ year = {2025},
+ doi = {10.48550/arXiv.2502.12110},
+ url = {https://arxiv.org/abs/2502.12110}
+}
+
+@article{Li2025MemOS,
+ author = {Li, Zhiyu and Song, Shichao and Xi, Chenyang and Wang, Hanyu and Tang, Chen and Niu, Simin and Chen, Ding and Yang, Jiawei and Li, Chunyu and Yu, Qingchen and Zhao, Jihao and Liu, Peng and Lin, Zehao and Wang, Pengyuan and Huo, Jiahao and Chen, Tianyi and Chen, Kai and others},
+ title = {{MemOS}: A Memory {OS} for {AI} System},
+ journal = {arXiv preprint arXiv:2507.03724},
+ year = {2025},
+ doi = {10.48550/arXiv.2507.03724},
+ url = {https://arxiv.org/abs/2507.03724}
+}
+
+@article{Hu2025MemoryAge,
+ author = {Hu, Yuyang and Liu, Shichun and Yue, Yanwei and Zhang, Guibin and Liu, Boyang and Zhu, Fangyi and Lin, Jiahang and Guo, Honglin and Dou, Shihan and Xi, Zhiheng and Jin, Senjie and Tan, Jiejun and Yin, Yanbin and Liu, Jiongnan and Zhang, Zeyu and Sun, Zhongxiang and Zhu, Yutao and Sun, Hao and Peng, Boci and Cheng, Shihan and others},
+ title = {Memory in the Age of {AI} Agents},
+ journal = {arXiv preprint arXiv:2512.13564},
+ year = {2025},
+ doi = {10.48550/arXiv.2512.13564},
+ url = {https://arxiv.org/abs/2512.13564}
+}
+
+@article{Liu2023LostMiddle,
+ author = {Liu, Nelson F. and Lin, Kevin and Hewitt, John and Paranjape, Ashwin and Bevilacqua, Michele and Petroni, Fabio and Liang, Percy},
+ title = {Lost in the Middle: How Language Models Use Long Contexts},
+ journal = {Transactions of the Association for Computational Linguistics},
+ year = {2023},
+ volume = {11},
+ pages = {157--173},
+ doi = {10.1162/tacl_a_00638},
+ url = {https://aclanthology.org/2023.tacl-1.10/}
+}
+
+% ======================= Context attribution =======================
+
+@inproceedings{CohenWang2024ContextCite,
+ author = {Cohen-Wang, Benjamin and Shah, Harshay and Georgiev, Kristian and M{\=a}dry, Aleksander},
+ title = {{ContextCite}: Attributing Model Generation to Context},
+ booktitle = {Advances in Neural Information Processing Systems 37},
+ year = {2024},
+ eprint = {2409.00729},
+ archivePrefix = {arXiv},
+ url = {https://arxiv.org/abs/2409.00729}
+}
+
+@article{Nematov2025SourceAttribution,
+ author = {Nematov, Ikhtiyor and Kalai, Tarik and Kuzmenko, Elizaveta and Fugagnoli, Gabriele and Sacharidis, Dimitris and Hose, Katja and Sagi, Tomer},
+ title = {Source Attribution in Retrieval-Augmented Generation},
+ journal = {arXiv preprint arXiv:2507.04480},
+ year = {2025},
+ doi = {10.48550/arXiv.2507.04480},
+ url = {https://arxiv.org/abs/2507.04480}
+}
+
+@inproceedings{Xiao2025TokenShapley,
+ author = {Xiao, Y. and others},
+ title = {{TokenShapley}: Token Level Context Attribution with {Shapley} Value},
+ booktitle = {Findings of the Association for Computational Linguistics: ACL 2025},
+ year = {2025},
+ eprint = {2507.05261},
+ archivePrefix = {arXiv},
+ url = {https://arxiv.org/abs/2507.05261},
+ note = {confirm full author list from the ACL Anthology record before camera-ready}
+}
+
+@article{Anand2026ContextFocus,
+ author = {Anand, Nikhil and Somasundaram, Shwetha and Phukan, Anirudh and Saxena, Apoorv and Mukherjee, Koyel},
+ title = {{ContextFocus}: Activation Steering for Contextual Faithfulness in Large Language Models},
+ journal = {arXiv preprint arXiv:2601.04131},
+ year = {2026},
+ doi = {10.48550/arXiv.2601.04131},
+ url = {https://arxiv.org/abs/2601.04131}
+}
+
+% ======================= Memory poisoning & defense =======================
+
+% OpenReview page bot-walled at check time; confirm NeurIPS 2025 record before use.
+@inproceedings{Dong2025MINJA,
+ author = {Dong, S. and others},
+ title = {{MINJA}: Memory Injection Attacks on {LLM} Agents via Query-Only Interaction},
+ booktitle = {Advances in Neural Information Processing Systems},
+ year = {2025},
+ url = {https://openreview.net/forum?id=QINnsnppv8},
+ note = {verify venue record and author list before submission}
+}
+
+@article{Pulipaka2026SleeperMemory,
+ author = {Pulipaka, Prem Sidharth and others},
+ title = {Hidden in Memory: Sleeper Memory Poisoning in {LLM} Agents},
+ journal = {arXiv preprint arXiv:2605.15338},
+ year = {2026},
+ doi = {10.48550/arXiv.2605.15338},
+ url = {https://arxiv.org/abs/2605.15338}
+}
+
+@article{Sunil2026MemoryPoisonDefense,
+ author = {Sunil, Balachandra Devarangadi and Sinha, Isheeta and Maheshwari, Piyush and Todmal, Shantanu and Mallik, Shreyan and Mishra, S. M.},
+ title = {Memory Poisoning Attack and Defense on Memory Based {LLM}-Agents},
+ journal = {arXiv preprint arXiv:2601.05504},
+ year = {2026},
+ doi = {10.48550/arXiv.2601.05504},
+ url = {https://arxiv.org/abs/2601.05504}
+}
+
+% Corrected: was authorless with a wrong short title.
+% Verified: Dash, Ge, Jain, Shah, Shang — arXiv:2606.04329.
+@article{Dash2026SystematicMemoryPoisoning,
+ author = {Dash, Pritam and Ge, Tongyu and Jain, Aditi and Shah, Tanmay and Shang, Zhiwei},
+ title = {From Untrusted Input to Trusted Memory: A Systematic Study of Memory Poisoning Attacks in {LLM} Agents},
+ journal = {arXiv preprint arXiv:2606.04329},
+ year = {2026},
+ doi = {10.48550/arXiv.2606.04329},
+ url = {https://arxiv.org/abs/2606.04329}
+}
+
+% Corrected: was authorless. Verified: Yedidel Louck, arXiv:2606.24322.
+@article{Louck2026SecuringMemory,
+ author = {Louck, Yedidel},
+ title = {Securing {LLM}-Agent Long-Term Memory Against Poisoning: Non-Malleable, Origin-Bound Authority with Machine-Checked Guarantees},
+ journal = {arXiv preprint arXiv:2606.24322},
+ year = {2026},
+ doi = {10.48550/arXiv.2606.24322},
+ url = {https://arxiv.org/abs/2606.24322}
+}
+
+% Corrected: first author is Gao, not Wei (Wei is sixth); key renamed.
+% Verified against arXiv:2607.14651.
+@article{Gao2026MemPoison,
+ author = {Gao, Jifeng and Xia, Kang and Zhang, Yi and Hong, Xiaobin and Lin, Mingkai and Wei, Xingshen and Li, Wenzhong and Lu, Sanglu},
+ title = {{MemPoison}: Uncovering Persistent Memory Threats and Structural Blind Spots in {LLM} Agents},
+ journal = {arXiv preprint arXiv:2607.14651},
+ year = {2026},
+ doi = {10.48550/arXiv.2607.14651},
+ url = {https://arxiv.org/abs/2607.14651}
+}
+
+% ======================= Memory management with attribution signals =======================
+
+@article{Tan2025ReflectiveMemory,
+ author = {Tan, Zhen and Yan, Jun and Hsu, I-Hung and Han, Rujun and Wang, Zifeng and Le, Long T. and Song, Yiwen and Chen, Yanfei and Palangi, Hamid and Lee, George and Iyer, Anand and Chen, Tianlong and Liu, Huan and Lee, Chen-Yu and Pfister, Tomas},
+ title = {In Prospect and Retrospect: Reflective Memory Management for Long-term Personalized Dialogue Agents},
+ journal = {arXiv preprint arXiv:2503.08026},
+ year = {2025},
+ doi = {10.48550/arXiv.2503.08026},
+ url = {https://arxiv.org/abs/2503.08026}
+}
diff --git a/pyproject.toml b/pyproject.toml
index b3f5672..bb5d1b5 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -1,10 +1,10 @@
[build-system]
-requires = ["setuptools>=68"]
-build-backend = "setuptools.build_meta"
+requires = ["hatchling"]
+build-backend = "hatchling.build"
[project]
name = "embr"
-version = "0.1.0"
+version = "0.2.0"
description = "Emotional Memory for Believable Roleplay: emotion-grounded memory for persistent game NPCs."
readme = "README.md"
requires-python = ">=3.10"
@@ -12,45 +12,47 @@ license = { text = "MIT" }
authors = [{ name = "AL Shifan", email = "codeandsorcerylab@gmail.com" }]
keywords = ["LLM", "NPC", "memory", "affective-computing", "retrieval", "game-ai"]
-# Core needs nothing: the menu and the eval harness run on the standard library alone. The
-# heavy ML stack goes in the optional extras below, so a fresh clone runs the skeleton,
-# the demo turn, and the full evaluation with nothing installed.
+# The core needs nothing: the applet, the NPC server, and the whole evaluation run on the
+# standard library. The heavy stacks are extras, so a fresh clone runs everything but the
+# real models and the paper figures with nothing installed.
dependencies = []
[project.optional-dependencies]
-# Real semantic embeddings and the real local model (Ouro 1.4B, the thesis model). BM25 is
-# implemented in-tree, so nothing here is needed to run the eval; installing it swaps the
-# deterministic embedder and the echo stub for the real thing.
+# Real semantic embeddings and the thesis model (Ouro 1.4B) in-process. BM25 is in-tree, so
+# the evaluation never needs this; installing it swaps the deterministic embedder and the
+# echo stub for the real thing.
ml = [
"sentence-transformers>=2.2",
"torch>=2.2",
# Both bounds are load-bearing for Ouro's remote code (snapshot 574fa66 and later).
- # Below 4.56 it fails: on 4.54-4.55 the read-only Cache.key_cache property crashes its
- # cache init, and on 4.53 it imports symbols that do not exist yet. On 5.x it fails twice
- # over: OuroConfig has no pad_token_id, then the rope-config lookup raises KeyError:
- # 'default'. Verified working on 4.57.
+ # Below 4.56 its cache init crashes; on 5.x OuroConfig has no pad_token_id and the
+ # rope-config lookup raises. Verified working on 4.57.
"transformers>=4.56,<5",
"accelerate>=0.30",
]
-# Paper figures. Kept out of the core so the harness stays dependency-light; only the
-# asset build step needs a plotting library.
+# Paper figures. Only the asset build step needs a plotting library.
figures = [
"matplotlib>=3.8",
]
+
+[dependency-groups]
dev = [
"pytest>=7.4",
]
[project.scripts]
-embr = "menu:run_menu"
+embr = "embr.cli:main"
+
+[project.urls]
+Repository = "https://github.com/Code-SorceryLab/EMBR"
-[tool.setuptools]
-# The menu is the front door, so it sits at the repo root rather than inside the library.
-# It is a top level module, not part of the `embr` package, and setuptools needs telling.
-py-modules = ["menu"]
+[tool.hatch.build.targets.wheel]
+# Three packages, one src tree: the library (with the applet), the harness (with the paper
+# asset builders), and the web demo.
+packages = ["src/embr", "src/eval", "src/web"]
-[tool.setuptools.packages.find]
-include = ["embr*"]
+[tool.hatch.build.targets.sdist]
+include = ["src", "tests", "scripts", "assets", "docs", "menu.py", "conftest.py", "README.md", "LICENSE"]
[tool.pytest.ini_options]
testpaths = ["tests"]
diff --git a/assets/portraits/cutout.py b/scripts/cutout.py
similarity index 100%
rename from assets/portraits/cutout.py
rename to scripts/cutout.py
diff --git a/scripts/fetch_models.sh b/scripts/fetch_models.sh
new file mode 100755
index 0000000..ebc0637
--- /dev/null
+++ b/scripts/fetch_models.sh
@@ -0,0 +1,32 @@
+#!/usr/bin/env bash
+# Fetch every model the project uses, fresh: the thesis model and the embedder from
+# Hugging Face, and the Ollama tags the paper measured from the Ollama library. Pass
+# --fresh to delete the cached copies first. Needs the ml extra (uv sync --extra ml).
+set -euo pipefail
+cd "$(dirname "$0")/.."
+
+HF_MODELS=("ByteDance/Ouro-1.4B" "sentence-transformers/all-MiniLM-L6-v2")
+# Library tags, not Hugging Face GGUFs: a GGUF is a different quantised artefact from
+# the one the reported numbers were produced on, and the paper cites these tags.
+OLLAMA_MODELS=("llama3.2:3b" "qwen2.5:7b" "llama3.1:8b")
+
+if [[ "${1:-}" == "--fresh" ]]; then
+ for repo in "${HF_MODELS[@]}"; do
+ rm -rf "${HF_HOME:-$HOME/.cache/huggingface}/hub/models--${repo//\//--}"
+ done
+ for tag in "${OLLAMA_MODELS[@]}"; do ollama rm "$tag" 2>/dev/null || true; done
+fi
+
+uv run python - "${HF_MODELS[@]}" <<'PY'
+import sys
+from huggingface_hub import snapshot_download
+for repo in sys.argv[1:]:
+ print("fetched", repo, "->", snapshot_download(repo), flush=True)
+PY
+
+if command -v ollama >/dev/null; then
+ for tag in "${OLLAMA_MODELS[@]}"; do ollama pull "$tag"; done
+ ollama list
+else
+ echo "ollama is not installed; skipped the Ollama tags" >&2
+fi
diff --git a/embr/__init__.py b/src/embr/__init__.py
similarity index 100%
rename from embr/__init__.py
rename to src/embr/__init__.py
diff --git a/embr/__main__.py b/src/embr/__main__.py
similarity index 63%
rename from embr/__main__.py
rename to src/embr/__main__.py
index 5278387..6c7e802 100644
--- a/embr/__main__.py
+++ b/src/embr/__main__.py
@@ -1,13 +1,11 @@
-"""Launch the EMBR menu with `python -m embr`, or query save state without it.
+"""`python -m embr`: the menu with no arguments, or any command (`python -m embr --help`).
- python -m embr # the menu
- python -m embr save-status # every slot, its progress, and any problems
- python -m embr validate-saves # exit 1 if any save cannot load against this build
+The two save queries below are kept here because they need nothing but the saves module,
+so `embr saves status` works on a machine with nothing else installed or configured.
"""
from __future__ import annotations
-import argparse
import sys
from pathlib import Path
@@ -49,20 +47,9 @@ def validate_saves(root: Path | str = SAVES_ROOT) -> int:
def main(argv: list[str] | None = None) -> int:
- parser = argparse.ArgumentParser(prog="embr", description=__doc__.splitlines()[0])
- parser.add_argument(
- "command", nargs="?", choices=("save-status", "validate-saves"),
- help="omit to open the menu",
- )
- args = parser.parse_args(argv)
- if args.command == "save-status":
- return save_status()
- if args.command == "validate-saves":
- return validate_saves()
- from menu import run_menu # the menu lives at the repo root as a top-level module
+ from embr.cli import main as cli_main
- run_menu()
- return 0
+ return cli_main(argv)
if __name__ == "__main__":
diff --git a/embr/affect.py b/src/embr/affect.py
similarity index 100%
rename from embr/affect.py
rename to src/embr/affect.py
diff --git a/src/embr/cli/__init__.py b/src/embr/cli/__init__.py
new file mode 100644
index 0000000..96792dd
--- /dev/null
+++ b/src/embr/cli/__init__.py
@@ -0,0 +1,6 @@
+"""The EMBR applet: the interactive menu, and the same actions as commands."""
+
+from .app import build_parser, main
+from .menu import run_menu
+
+__all__ = ["build_parser", "main", "run_menu"]
diff --git a/src/embr/cli/app.py b/src/embr/cli/app.py
new file mode 100644
index 0000000..3892f9e
--- /dev/null
+++ b/src/embr/cli/app.py
@@ -0,0 +1,259 @@
+"""The EMBR command line: the menu with no arguments, and every action as a command.
+
+ embr the interactive menu
+ embr eval run the whole protocol, one run directory
+ embr mechanism attribution which signal lets the attack in
+ embr assets build figures, tables and the results page from the newest run
+ embr serve --model ollama NPCs over JSON for a game engine
+ embr --help the flags of any command
+
+The commands call the same functions the menu rows call, so nothing can be reachable from
+one and not the other. Commands that hand their arguments to a harness module verbatim
+(`serve`, `mechanism cite`) accept anything that module accepts.
+"""
+
+from __future__ import annotations
+
+import argparse
+import sys
+import textwrap
+from collections.abc import Callable, Sequence
+from pathlib import Path
+
+from . import menu as m
+
+#: Commands that pass unrecognised arguments straight through to the module they wrap.
+_PASSTHROUGH = {"serve", "cite"}
+
+
+def _with_argv(prog: str, rest: Sequence[str], call: Callable[[], object]) -> None:
+ """Run a harness entry point that reads sys.argv, as if it were invoked directly."""
+ saved = sys.argv
+ sys.argv = [prog, *rest]
+ try:
+ call()
+ finally:
+ sys.argv = saved
+
+
+# ---------------------------------------------------------------------------- handlers
+
+
+def _web(args: argparse.Namespace, _extra: list[str]) -> None:
+ from web.server import serve
+
+ serve(port=args.port, open_browser=not args.no_browser)
+
+
+def _serve(_args: argparse.Namespace, extra: list[str]) -> None:
+ from embr.serve import main
+
+ main(extra)
+
+
+def _eval_run(_args: argparse.Namespace, _extra: list[str]) -> None:
+ from eval.run import run_all
+
+ path, _summary = run_all(progress=lambda message: print(f" {message}"))
+ print(f"run written to {path}")
+
+
+def _eval_replicate(args: argparse.Namespace, _extra: list[str]) -> None:
+ from eval.experiments import replicate_experiment
+
+ report = replicate_experiment(replicates=args.replicates)
+ verdict = "identical" if report["identical"] else "DIVERGED"
+ print(f"{report['replicates']} runs on {report['model']}: {verdict}")
+ print(f"written to {report['out_dir']}")
+
+
+def _eval_models(_args: argparse.Namespace, _extra: list[str]) -> None:
+ from eval.experiments import AVAILABLE_MODELS, cross_model_experiment
+
+ print(f"models: {', '.join(AVAILABLE_MODELS)}")
+ report = cross_model_experiment()
+ print(f"{len(report['models'])} models compared; written to {report['out_dir']}")
+
+
+def _eval_agreement(args: argparse.Namespace, _extra: list[str]) -> None:
+ from eval.agreement import main
+
+ _with_argv("embr eval agreement", [args.run_dir] if args.run_dir else [], main)
+
+
+def _harness(module: str, prog: str) -> Callable[[argparse.Namespace, list[str]], None]:
+ """A command that is one harness module's `main`, with the shell arguments handed on."""
+
+ def run(_args: argparse.Namespace, extra: list[str]) -> None:
+ import importlib
+
+ _with_argv(prog, extra, importlib.import_module(module).main)
+
+ return run
+
+
+def _assets_build(args: argparse.Namespace, _extra: list[str]) -> None:
+ run_dir = Path(args.run_dir) if args.run_dir else m._latest_run()
+ if run_dir is None:
+ print("no run found under data/runs; run `embr eval run` first")
+ return
+ steps = args.only or list(m.ASSET_STEPS)
+ print(f"building {', '.join(steps)} from {run_dir}")
+ for path in m.build_assets(run_dir, steps):
+ print(f" {path}")
+
+
+def _assets_manifest(_args: argparse.Namespace, _extra: list[str]) -> None:
+ from eval.report.build_manifest import main
+
+ main([])
+
+
+def _demo_page(args: argparse.Namespace, _extra: list[str]) -> None:
+ import webbrowser
+
+ from eval.report.build_demo import build_demo
+
+ paths = build_demo()
+ for path in paths:
+ print(f" {path}")
+ if args.open:
+ webbrowser.open(paths[0].resolve().as_uri())
+
+
+def _demo(name: str) -> Callable[[argparse.Namespace, list[str]], None]:
+ def run(_args: argparse.Namespace, _extra: list[str]) -> None:
+ from embr.cli import demos
+
+ getattr(demos, name)()
+
+ return run
+
+
+def _saves_status(_args: argparse.Namespace, _extra: list[str]) -> None:
+ from embr.__main__ import save_status
+
+ raise SystemExit(save_status())
+
+
+def _saves_validate(_args: argparse.Namespace, _extra: list[str]) -> None:
+ from embr.__main__ import validate_saves
+
+ raise SystemExit(validate_saves())
+
+
+def _menu_action(handler: Callable[[], None]) -> Callable[[argparse.Namespace, list[str]], None]:
+ return lambda _args, _extra: handler()
+
+
+# ------------------------------------------------------------------------------ parser
+
+
+def build_parser() -> argparse.ArgumentParser:
+ from embr import __version__
+
+ parser = argparse.ArgumentParser(
+ prog="embr",
+ description="Emotional Memory for Believable Roleplay. No command opens the menu.",
+ formatter_class=argparse.RawDescriptionHelpFormatter,
+ epilog=textwrap.dedent(__doc__.split("\n\n", 1)[1]).strip(),
+ )
+ parser.add_argument("--version", action="version", version=f"embr {__version__}")
+ top = parser.add_subparsers(dest="command", metavar="")
+
+ def command(parent, name: str, help: str | None, run, **kwargs) -> argparse.ArgumentParser:
+ # No help means hidden: argparse lists a subcommand only when it has one.
+ sub = parent.add_parser(name, **({"help": help, "description": help} if help else {}), **kwargs)
+ sub.set_defaults(run=run)
+ return sub
+
+ def group(name: str, help: str):
+ sub = top.add_parser(name, help=help, description=help)
+ return sub.add_subparsers(dest="sub", metavar="", required=True)
+
+ # Play
+ command(top, "menu", "open the interactive menu", _menu_action(m.run_menu))
+ command(top, "turn", "one demo turn: watch the lie resurface", _menu_action(m._do_conversation_turn))
+ command(top, "play", "play Dawn's arc in the terminal, without saving", _menu_action(m._do_walkthrough))
+ command(top, "continue", "resume the newest save where it stopped", _menu_action(m._do_continue))
+ web = command(top, "web", "the visual-novel demo in a browser", _web)
+ web.add_argument("--port", type=int, default=8000)
+ web.add_argument("--no-browser", action="store_true", help="do not open a browser tab")
+ command(top, "serve", "NPCs over JSON for a game engine (see `embr serve --help`)", _serve,
+ add_help=False)
+
+ # Measure
+ ev = group("eval", "the protocol: RQ1 behaviour, RQ2 robustness, RQ3 retrieval")
+ command(ev, "quick", "RQ3 at published defaults, answers instantly", _menu_action(m._do_quick_scoreboard))
+ command(ev, "run", "RQ1 + RQ2 + RQ3, writes a run directory", _eval_run)
+ rep = command(ev, "replicate", "the same model repeated: does the harness reproduce?", _eval_replicate)
+ rep.add_argument("--replicates", type=int, default=3)
+ command(ev, "models", "the same protocol across models: what moves?", _eval_models)
+ command(ev, "bakeoff", "looped (Ouro) against conventional models, measured", _menu_action(m._do_bakeoff))
+ agr = command(ev, "agreement", "two tone raters, and the reply claim", _eval_agreement)
+ agr.add_argument("run_dir", nargs="?", default=None)
+
+ # Mechanism
+ mech = group("mechanism", "the experiments behind the self-priming loop")
+ command(mech, "flip", "flip every emotion: meaning stays, mood inverts", _harness("eval.emotion_flip", "embr mechanism flip"))
+ command(mech, "attribution", "which signal lets the attack in, one ablation each", _harness("eval.attribution", "embr mechanism attribution"))
+ command(mech, "provenance", "the defence: anchored scoring mass against poisoning", _harness("eval.provenance", "embr mechanism provenance"))
+ command(mech, "grid", "same poison, four tags: the text never reaches the state", _harness("eval.grid", "embr mechanism grid"))
+ command(mech, "attacks-v2", "the 2026 attack classes: dormant, laundering", _harness("eval.attacks_v2", "embr mechanism attacks-v2"))
+ command(mech, "consistency", "does she refuse the room after the betrayal?", _harness("eval.consistency", "embr mechanism consistency"))
+ command(mech, "cite", "the six-source cite view, exact Banzhaf (module flags pass through)",
+ _harness("eval.context_attribution", "embr mechanism cite"), add_help=False)
+
+ # Paper
+ assets = group("assets", "paper assets, generated from a run and never by hand")
+ build = command(assets, "build", "figures, tables, and the results page", _assets_build)
+ build.add_argument("run_dir", nargs="?", default=None, help="a run directory; default the newest")
+ build.add_argument("--only", nargs="+", choices=m.ASSET_STEPS, help="a subset of the steps")
+ command(assets, "manifest", "the release manifest, from pytest's own report", _assets_manifest)
+ command(top, "results", "summarise the newest run without rerunning anything", _menu_action(m._do_latest_results))
+ command(top, "dashboard", "read-only: quest path, state timeline, evidence status", _menu_action(m._do_dashboard))
+
+ # Demo suite
+ demo = group("demo", "the demo suite, on the stub, naming the run behind every number")
+ page = command(demo, "page", "the interactive node brain, flat and in 3D", _demo_page)
+ page.add_argument("--open", action="store_true", help="open it in a browser")
+ command(demo, "reckoning", "six sources shaded by exact Banzhaf weight", _demo("demo_reckoning_reveal"))
+ command(demo, "mood", "one line under three moods", _demo("demo_mood_slider"))
+ command(demo, "defence", "anchor weight against poisoning", _demo("demo_defence_dial"))
+ command(demo, "tagflip", "flip an affect tag: the rank moves, the words do not", _demo("demo_tag_flip"))
+ command(demo, "divergence", "where likelihood and behaviour disagree", _demo("demo_estimator_divergence"))
+ command(demo, "record", "a capture-ready pass through the first four demos", _demo("run_record"))
+
+ # System
+ command(top, "settings", "the live configuration and where to change it", _menu_action(m._do_settings))
+ command(top, "lexicon", "fetch the NRC VAD lexicon (research use, stays out of git)", _menu_action(m._do_fetch_lexicon))
+ saves = group("saves", "the walkthrough's save slots")
+ command(saves, "status", "every slot, its progress, and any problems", _saves_status)
+ command(saves, "validate", "exit 1 if any save cannot load against this build", _saves_validate)
+ # The spellings the docs used before the applet had groups.
+ command(top, "save-status", None, _saves_status)
+ command(top, "validate-saves", None, _saves_validate)
+ return parser
+
+
+def main(argv: Sequence[str] | None = None) -> int:
+ """Entry point for the console script, `python -m embr`, and the root menu.py."""
+ parser = build_parser()
+ args, extra = parser.parse_known_args(argv)
+ if args.command is None:
+ m.run_menu()
+ return 0
+ if extra and (args.command not in _PASSTHROUGH and getattr(args, "sub", None) not in _PASSTHROUGH):
+ parser.error(f"unrecognized arguments: {' '.join(extra)}")
+ try:
+ args.run(args, extra)
+ except KeyboardInterrupt:
+ print("\ninterrupted")
+ return 130
+ except Exception as error: # the same boundary the menu has: report, hint, exit non-zero
+ print(f"{type(error).__name__}: {error}", file=sys.stderr)
+ hint = m._error_hint(error)
+ if hint:
+ print(hint, file=sys.stderr)
+ return 1
+ return 0
diff --git a/demos.py b/src/embr/cli/demos.py
similarity index 99%
rename from demos.py
rename to src/embr/cli/demos.py
index a29132e..21c6fd9 100644
--- a/demos.py
+++ b/src/embr/cli/demos.py
@@ -24,7 +24,7 @@
from typing import Any, Callable
# ANSI helpers reused from the menu, so the palette is defined in exactly one place.
-from menu import _BOLD, _CYAN, _DIM, _EMBER, _RED, _WHT, _YEL, _c, _SUPPORTS_COLOR
+from embr.cli.menu import _BOLD, _CYAN, _DIM, _EMBER, _RED, _WHT, _YEL, _c, _SUPPORTS_COLOR
from embr import (
CharacterState,
diff --git a/src/embr/cli/menu.py b/src/embr/cli/menu.py
new file mode 100644
index 0000000..cf69473
--- /dev/null
+++ b/src/embr/cli/menu.py
@@ -0,0 +1,1069 @@
+"""EMBR hub: the main menu, and the front door to everything the project does.
+
+Shaped like the PEAK ENGINE hub (logo, live stats bar, labelled sections, toggle pickers, a
+chime when a long job lands) so the thesis projects feel like one toolkit. Pure stdlib: ANSI
+escapes do the colour, and when stdout is not a terminal every wrapper returns plain text so
+logs and tests read clean. Nothing here holds state; every option delegates to the module
+that owns the work.
+
+Destructive options demand a typed confirmation word rather than a y/n, because a stray
+keypress should never be able to delete a run.
+"""
+
+from __future__ import annotations
+
+import json
+import os
+import shutil
+import sys
+from collections.abc import Sequence
+from pathlib import Path
+from typing import Any, Callable
+
+try:
+ import winsound
+except ImportError: # not Windows
+ winsound = None
+
+# Enable VT100 escape processing on Windows terminals; harmless elsewhere.
+os.system("")
+
+# The logo and box glyphs need UTF-8; legacy consoles default to cp1252.
+for _stream in (sys.stdout, sys.stderr):
+ if hasattr(_stream, "reconfigure"):
+ _stream.reconfigure(encoding="utf-8", errors="replace")
+
+RUNS_DIR = Path("data/runs")
+FIGURES_DIR = Path("data/figures")
+TABLES_DIR = Path("data/tables")
+
+# --------------------------------------------------------------------------- ANSI palette
+
+_SUPPORTS_COLOR = (
+ hasattr(sys.stdout, "isatty") and sys.stdout.isatty() and os.environ.get("NO_COLOR") is None
+)
+
+
+def _c(code: str, text: str) -> str:
+ """Wrap text in an ANSI escape if the terminal supports it, else return it untouched."""
+ return f"\033[{code}m{text}\033[0m" if _SUPPORTS_COLOR else text
+
+
+_DIM = lambda t: _c("2", t) # noqa: E731
+_BOLD = lambda t: _c("1", t) # noqa: E731
+_CYAN = lambda t: _c("96", t) # noqa: E731
+_MAG = lambda t: _c("95", t) # noqa: E731
+_YEL = lambda t: _c("93", t) # noqa: E731
+_GRN = lambda t: _c("92", t) # noqa: E731
+_RED = lambda t: _c("91", t) # noqa: E731
+_WHT = lambda t: _c("97", t) # noqa: E731
+_EMBER = lambda t: _c("38;5;208", t) # noqa: E731 the branding orange, #ea580c
+
+_LOGO = """\
+ ███████╗ ███╗ ███╗ ██████╗ ██████╗
+ ██╔════╝ ████╗ ████║ ██╔══██╗ ██╔══██╗
+ █████╗ ██╔████╔██║ ██████╔╝ ██████╔╝
+ ██╔══╝ ██║╚██╔╝██║ ██╔══██╗ ██╔══██╗
+ ███████╗ ██║ ╚═╝ ██║ ██████╔╝ ██║ ██║
+ ╚══════╝ ╚═╝ ╚═╝ ╚═════╝ ╚═╝ ╚═╝"""
+
+_SUB_LOGO = " Emotional Memory for Believable Roleplay By AL Shifan"
+_RULE = " " + "─" * 56
+
+# Key, label, hint. The renderer groups rows by _SECTIONS; the dispatch table is _ACTIONS.
+_MENU_ITEMS = [
+ ("R", "Continue", "resume the newest save right where it stopped"),
+ ("Q", "Quest Slots", "start, resume, restart or delete a named save slot"),
+ ("1", "Conversation Turn", "one demo turn: watch the lie resurface"),
+ ("2", "Walkthrough (legacy)", "play Dawn's arc without saving, the research pass"),
+ ("W", "Web Demo", "the visual-novel demo in a browser, research tabs and all"),
+ ("3", "Quick Scoreboard", "RQ3 at published defaults, answers instantly"),
+ ("4", "Full Evaluation", "RQ1 + RQ2 + RQ3, writes a run directory"),
+ ("5", "Seeded Runs", "replicate on one model, or compare across models"),
+ ("6", "Model Bake-Off", "looped (Ouro) vs conventional, measured"),
+ ("7", "Affective Indexing", "flip every emotion: meaning stays, mood inverts"),
+ ("8", "Poisoning Attribution", "which signal lets the attack in, one ablation each"),
+ ("9", "Provenance Sweep", "the defence: anchored scoring mass vs poisoning"),
+ ("10", "Content x Tag Grid", "same poison, four tags: the text never reaches the state"),
+ ("11", "Generate Paper Assets", "figures, tables and the results page, from the run"),
+ ("12", "Interactive Demo", "the node brain, flat and in 3D: press play, then drive"),
+ ("13", "Latest Results", "summarise the newest run directory"),
+ ("V", "Research Dashboard", "read-only: quest path, state timeline, evidence status"),
+ ("14", "Reckoning Reveal", "six sources shaded by exact Banzhaf weight, both estimators"),
+ ("15", "Mood Slider", "one line, three moods: retrieval, tone and attribution re-flow"),
+ ("16", "Defence Dial", "anchor weight vs poisoning, with its failure condition"),
+ ("17", "Tag-Flip Close-Up", "flip an affect tag: the words never change, the rank does"),
+ ("18", "Estimator Divergence", "where likelihood and behaviour disagree (needs both arms)"),
+ ("19", "Record Walk (1-4)", "capture-ready pass through the first four demos"),
+ ("S", "Settings", "weights, top-k, backends, model runner"),
+ ("L", "Fetch Tone Lexicon", "NRC VAD v2.1, research use, stays out of git"),
+ ("M", "Maintenance", "destructive operations live here, behind confirmations"),
+ ("C", "Clear Screen", "clear terminal output"),
+ ("0", "Exit", "quit EMBR"),
+]
+
+#: The shell spelling of each row, shown beside it on a wide terminal and used by the
+#: command line in `app.py`. A row with no entry is interactive by nature.
+_COMMANDS: dict[str, str] = {
+ "R": "embr continue",
+ "1": "embr turn",
+ "2": "embr play",
+ "W": "embr web",
+ "3": "embr eval quick",
+ "4": "embr eval run",
+ "5": "embr eval replicate",
+ "6": "embr eval bakeoff",
+ "7": "embr mechanism flip",
+ "8": "embr mechanism attribution",
+ "9": "embr mechanism provenance",
+ "10": "embr mechanism grid",
+ "11": "embr assets build",
+ "12": "embr demo page",
+ "13": "embr results",
+ "V": "embr dashboard",
+ "14": "embr demo reckoning",
+ "15": "embr demo mood",
+ "16": "embr demo defence",
+ "17": "embr demo tagflip",
+ "18": "embr demo divergence",
+ "19": "embr demo record",
+ "S": "embr settings",
+ "L": "embr lexicon",
+}
+
+_SECTIONS = [
+ ("PLAY", ("R", "Q", "1", "2", "W")),
+ ("MEASURE", ("3", "4", "5", "6")),
+ ("MECHANISM", ("7", "8", "9", "10")),
+ ("PAPER", ("11", "12", "13", "V")),
+ ("DEMO SUITE", ("14", "15", "16", "17", "18", "19")),
+ ("SYSTEM", ("S", "L", "M", "C")),
+]
+
+
+# --------------------------------------------------------------------------- primitives
+
+
+def _clear() -> None:
+ # ANSI clear rather than shelling out to cls/clear: no subprocess, nothing when piped.
+ if _SUPPORTS_COLOR:
+ print("\033[2J\033[H", end="")
+
+
+def _chime() -> None:
+ """Three rising notes when a long job lands. Windows only; silent elsewhere."""
+ if winsound is None:
+ return
+ try:
+ for hz in (659, 784, 1047):
+ winsound.Beep(hz, 110)
+ except RuntimeError:
+ pass
+
+
+def _latest_run() -> Path | None:
+ """Newest data/runs// holding a results.json, or None when nothing has run."""
+ runs = sorted(RUNS_DIR.glob("*/results.json"))
+ return runs[-1].parent if runs else None
+
+
+def _run_model(run_dir: Path | None) -> str:
+ if run_dir is None:
+ return "none yet"
+ try:
+ meta = json.loads((run_dir / "results.json").read_text(encoding="utf-8")).get("metadata", {})
+ return str(meta.get("model", "?"))
+ except (OSError, ValueError):
+ return "?"
+
+
+def _attribution_status(attribution_root: Path) -> str:
+ """One honest phrase per estimator with a run on disk: name, scale, and stamp.
+
+ 'not computed' when nothing is on disk. A run below the full 20 readings is a pilot
+ and says so; no percentage is ever shown, because a partial sweep writes no file at
+ all and a fabricated number would claim knowledge nothing recorded.
+ """
+ from eval.context_attribution import newest_run_by_estimator
+
+ newest = newest_run_by_estimator(attribution_root)
+ if not newest:
+ return _DIM("not computed")
+ phrases = []
+ for estimator, run in sorted(newest.items()):
+ scale = _GRN(f"{run['readings']} readings") if run["readings"] >= 20 else _YEL("pilot")
+ phrases.append(f"{_WHT(estimator)} · {scale} · {run['stamp']}")
+ return " | ".join(phrases)
+
+
+def _save_status_line(saves_root: Path) -> str:
+ """The newest save's position, or an honest 'no save yet'."""
+ from embr.saves import latest_slot, list_slots
+
+ found = latest_slot(root=saves_root)
+ if found is None:
+ return _DIM("no save yet · Q starts a quest")
+ quest_id, slot = found
+ row = next(
+ r for r in list_slots(quest_id, root=saves_root) if r["slot"] == slot
+ )
+ progress = f"{row['beats_played']} / {row['beats_total']}"
+ return f"{_WHT(quest_id)}/{_WHT(slot)} · {_GRN(progress)} · updated {_DIM(str(row['updated_at'])[:16])}"
+
+
+def _status_lines(
+ saves_root: Path | str = Path("data/saves"),
+ attribution_root: Path | str = Path("data/runs/attribution"),
+) -> list[str]:
+ """The project-status panel rows: where play stopped, and what evidence exists.
+
+ Every value is read from disk artefacts; a missing artefact reads as its honest
+ absence ('no save yet', 'not computed'), never as a made-up zero or percentage.
+ """
+ return [
+ f" Save {_save_status_line(Path(saves_root))}",
+ f" Attribution {_attribution_status(Path(attribution_root))}",
+ ]
+
+
+def _print_header() -> None:
+ """Logo, tagline, and a live stats bar: runs on disk, the model behind the newest one,
+ figures built, and the configured model runner."""
+ from embr.config import EmbrConfig
+ from eval.tone import default_tone_rater
+
+ print()
+ for line in _LOGO.splitlines():
+ print(_EMBER(line))
+ print(_DIM(_RULE))
+ print(_MAG(_SUB_LOGO))
+ print(_DIM(_RULE))
+
+ runs = len(list(RUNS_DIR.glob("*/results.json")))
+ figures = len(list(FIGURES_DIR.glob("*.png")))
+ runner = EmbrConfig.load().model_runner
+ tone = default_tone_rater().name
+ r_str = _GRN(str(runs)) if runs else _DIM("0")
+ f_str = _GRN(str(figures)) if figures else _DIM("0")
+ t_str = _GRN(tone) if tone.startswith("nrc") else _YEL(tone)
+ print()
+ print(
+ f" Runs {r_str} │ Latest {_WHT(_run_model(_latest_run()))}"
+ f" │ Figures {f_str} │ Runner {_WHT(runner)} │ Tone {t_str}"
+ )
+ for line in _status_lines():
+ print(line)
+ print(_DIM(_RULE))
+
+
+#: Exception type -> the next step a stranded user should take. Only hints that are true
+#: for every instance of the type; anything else stays a bare error.
+def _error_hint(error: BaseException) -> str | None:
+ from embr.model import ModelUnavailableError
+
+ if isinstance(error, ModelUnavailableError):
+ return "Start the daemon with `ollama serve`, or switch to the stub in Settings."
+ if isinstance(error, FileNotFoundError):
+ return "A run artefact is missing. Option 4 (Full Evaluation) creates one."
+ if isinstance(error, ImportError):
+ return 'An optional extra is missing. `pip install -e ".[figures]"` or ".[ml]".'
+ return None
+
+
+def _menu_item(key: str, label: str, hint: str = "") -> str:
+ """One menu row: yellow key, label, dimmed hint, and the shell spelling when it fits."""
+ command = _COMMANDS.get(key)
+ if command and shutil.get_terminal_size((80, 24)).columns >= 110:
+ return f" {_YEL(f'[{key}]'.rjust(4))} {label.ljust(26)}{_DIM(hint.ljust(62))}{_DIM(command)}"
+ return f" {_YEL(f'[{key}]'.rjust(4))} {label.ljust(26)}{_DIM(hint) if hint else ''}"
+
+
+def _section(title: str) -> None:
+ print(f"\n {_BOLD(_CYAN('▸'))} {_BOLD(title)}")
+
+
+def _print_menu() -> None:
+ _clear()
+ _print_header()
+ rows = {key: (label, hint) for key, label, hint in _MENU_ITEMS}
+ for title, keys in _SECTIONS:
+ _section(title)
+ for key in keys:
+ label, hint = rows[key]
+ print(_menu_item(key, label, hint))
+ print()
+ print(_DIM(_RULE))
+ print(_menu_item("0", _RED("Exit")))
+ print()
+
+
+def _pause() -> None:
+ input(_DIM("\n Press Enter to return to the menu..."))
+
+
+def ask_index(prompt: str, options: Sequence[str], default: str | None = None) -> str | None:
+ """Numbered pick: prints the options, returns the chosen one, None on Back or bad input.
+ Enter picks the default when one is given."""
+ print(prompt)
+ for position, option in enumerate(options, 1):
+ flag = _DIM(" (default)") if option == default else ""
+ print(f" {_YEL(str(position))}. {option}{flag}")
+ back = len(options) + 1
+ print(f" {_YEL(str(back))}. Back")
+ hint = f" or Enter for [{default}]" if default else ""
+ raw = input(_BOLD(f" Select (1-{back}){hint}: ")).strip()
+ if raw == "" and default:
+ return default
+ if raw.isdigit() and 1 <= int(raw) <= len(options):
+ return options[int(raw) - 1]
+ if raw != str(back):
+ print(_RED(" ✖ Invalid selection."))
+ return None
+
+
+def toggle_select(
+ title: str, options: Sequence[str], default_indices: Sequence[int] = (), min_select: int = 1
+) -> list[str] | None:
+ """Checklist: type numbers (or ranges, "1-3") to flip items, Enter confirms, 0 backs out."""
+ selected = set(default_indices)
+ while True:
+ print(f"\n {_BOLD(_CYAN('▸'))} {_BOLD(title)} {_DIM('(toggle · Enter to confirm · 0 = back)')}\n")
+ for position, option in enumerate(options):
+ tick = _GRN("✓") if position in selected else _DIM("o")
+ print(f" {_YEL(f'[{position + 1}]')} {tick} {option}")
+ raw = input(_BOLD("\n ⟫ ")).strip()
+ if raw == "":
+ if len(selected) >= min_select:
+ return [options[i] for i in sorted(selected)]
+ print(_RED(f" Select at least {min_select}."))
+ continue
+ if raw == "0":
+ return None
+ for part in raw.replace(" ", "").split(","):
+ lo, _, hi = part.partition("-")
+ if lo.isdigit() and (hi.isdigit() or not hi):
+ for n in range(int(lo), int(hi or lo) + 1):
+ if 1 <= n <= len(options):
+ selected ^= {n - 1}
+
+
+# --------------------------------------------------------------------------- the actions
+
+
+def _do_conversation_turn() -> None:
+ """One scripted turn through the live pipeline, printing what EMBR recalled."""
+ from embr import build_demo_conversation
+
+ convo = build_demo_conversation()
+ turn = convo.take_turn("Any news from the capital? How fares the king these days?")
+
+ print(f"\n {_BOLD('Player:')} {turn.player_input}\n")
+ print(f" {_BOLD('Memories EMBR recalled:')}")
+ for position, memory in enumerate(turn.retrieved, start=1):
+ print(f" {position}. {_EMBER(memory.event_type.value)} {memory.text}")
+ print(f"\n {_BOLD('Dawn:')} {turn.reply}")
+ print(_DIM("\n The king's-errand promise surfaces because the composite scorer ties the"
+ " player's question to it."))
+
+
+def _choose_model() -> Any:
+ """Ask which model runs the walkthrough, falling back to the stub on any trouble.
+
+ The stub is offered first and by default because it needs nothing installed: the demo
+ should always be playable, even on a machine with no model and no daemon.
+ """
+ from embr import ModelUnavailableError, OllamaRunner, StubRunner
+
+ choice = ask_index(
+ f"\n {_BOLD('Model')}",
+ ["Stub (instant, obviously fake replies)",
+ "Ollama, local (a real model, needs the daemon)",
+ "Ouro 1.4B, the thesis model (slow to load, real)"],
+ default="Stub (instant, obviously fake replies)",
+ )
+ if choice and choice.startswith("Ollama"):
+ name = input(_DIM(" Ollama model [llama3.2:3b]: ")).strip() or "llama3.2:3b"
+ runner = OllamaRunner(name)
+ try: # fail here, at the menu, rather than mid-scene
+ runner.generate("Say the single word: ready.")
+ except ModelUnavailableError as error:
+ print(_RED(f" {error}"))
+ print(_DIM(" Falling back to the stub."))
+ return StubRunner()
+ return runner
+ if choice and choice.startswith("Ouro"):
+ from embr import OuroRunner
+
+ print(_DIM(" Loading Ouro 1.4B, about 10 s and roughly 3 GB of memory..."))
+ return OuroRunner()
+ return StubRunner()
+
+
+def _render_step(result: Any) -> None:
+ """Print one walkthrough step: the scene, what was recalled, and how Dawn moved."""
+ print(_DIM(f"\n {'-' * 66}"))
+ if result.narration:
+ print(_DIM(f" {result.narration}\n"))
+ print(f" {_BOLD('Player:')} {result.player_input}")
+ if result.retrieved:
+ print(_DIM(" recalled:"))
+ for memory in result.retrieved:
+ print(_DIM(f" - {memory.text}"))
+ print(f"\n {_BOLD('Dawn:')} {result.reply}")
+ print(_DIM(
+ f"\n mood {result.mood_before.valence:+.2f} -> {result.mood_after.valence:+.2f}"
+ f" trust {result.trust_before:+.2f} -> {result.trust_after:+.2f}"
+ f" ({result.timings.total_ms:.0f} ms)"
+ ))
+ if result.watch_for:
+ print(f" {_EMBER('watch for:')} {result.watch_for}")
+ if result.expected_recall_landed is False:
+ print(_YEL(" the memory this beat expected did not surface"))
+
+
+def _do_walkthrough() -> None:
+ """Play Dawn's arc beat by beat, then hand the player free rein."""
+ from embr.walkthrough import WalkthroughSession, build_walkthrough_conversation
+
+ session = WalkthroughSession(build_walkthrough_conversation(model=_choose_model()))
+ print(f"\n {_BOLD('Dawn Whitmore')}, keeper of the Ember Hearth."
+ f" {_DIM(f'{session.progress[1]} scenes.')}")
+ print(_DIM(" Enter accepts the suggested line, or type your own."))
+
+ while not session.is_finished:
+ beat = session.next_beat
+ print(_DIM(f"\n {'=' * 66}"))
+ if beat.narration:
+ print(_DIM(f" {beat.narration}"))
+ print(f"\n {_DIM('suggested:')} {beat.suggested_player_line}")
+ typed = input(" You: ").strip()
+ _render_step(session.step(typed or None))
+
+ print(_EMBER("\n The arc is done. Keep talking, or press Enter to stop."))
+ while True:
+ line = input("\n You: ").strip()
+ if not line:
+ break
+ _render_step(session.free_play(line))
+
+ if session.history:
+ final = session.history[-1]
+ print(f"\n {_BOLD('Where she ended:')} trust {final.trust_after:+.2f},"
+ f" mood {final.mood_after.valence:+.2f}")
+
+
+def _step_and_save(session: Any, line: str | None, slot: str, quest_id: str = "dawn-whitmore",
+ root: Any = None) -> Any:
+ """Play one scripted beat, then persist the slot. A turn that raises saves nothing,
+ so the previous turn stays resumable (the save-after-success rule)."""
+ from embr.saves import SAVES_ROOT, save_slot
+
+ result = session.step(line)
+ save_slot(session, slot=slot, quest_id=quest_id, root=root if root is not None else SAVES_ROOT)
+ return result
+
+
+def _play_saved(session: Any, slot: str) -> None:
+ """The interactive loop for a saved quest: every completed turn is written to the slot."""
+ from embr.saves import SAVES_ROOT, save_slot
+
+ print(_DIM(" Enter accepts the suggested line, or type your own. Every turn saves."))
+ while not session.is_finished:
+ beat = session.next_beat
+ print(_DIM(f"\n {'=' * 66}"))
+ if beat.narration:
+ print(_DIM(f" {beat.narration}"))
+ print(f"\n {_DIM('suggested:')} {beat.suggested_player_line}")
+ typed = input(" You: ").strip()
+ _render_step(_step_and_save(session, typed or None, slot=slot))
+
+ print(_EMBER("\n The arc is done. Keep talking, or press Enter to stop."))
+ while True:
+ line = input("\n You: ").strip()
+ if not line:
+ break
+ _render_step(session.free_play(line))
+ save_slot(session, slot=slot, root=SAVES_ROOT)
+
+ if session.history:
+ final = session.history[-1]
+ print(f"\n {_BOLD('Where she ended:')} trust {final.trust_after:+.2f},"
+ f" mood {final.mood_after.valence:+.2f}")
+
+
+def _do_continue() -> None:
+ """Resume the newest loadable save, or say plainly that there is nothing to resume."""
+ from embr.saves import latest_slot, load_slot
+
+ found = latest_slot()
+ if found is None:
+ print(_YEL("\n No save to continue. Use Q to start a quest."))
+ return
+ quest_id, slot = found
+ session, payload = load_slot(slot, quest_id=quest_id, model=_choose_model())
+ played, total = session.progress
+ print(f"\n {_BOLD('Resuming')} {quest_id}/{slot} at scene {played + 1} of {total}.")
+ history = payload.get("history", [])
+ if history:
+ last = history[-1]
+ print(_DIM(f" Previously: you said {last['player_input']!r}"))
+ print(_DIM(f" and Dawn replied {last['reply']!r}"))
+ _play_saved(session, slot)
+
+
+def _do_quests() -> None:
+ """List every slot with its state; start, resume, restart, or delete one."""
+ from embr.saves import QUEST_DAWN, delete_slot, list_slots, load_slot
+
+ rows = list_slots()
+ print(f"\n {_BOLD('Save slots')}")
+ if not rows:
+ print(_DIM(" none yet"))
+ for row in rows:
+ state = _RED("cannot load: " + " ".join(row["problems"])) if row["problems"] else _GRN("ok")
+ print(f" {row['quest_id']}/{_WHT(row['slot'])} "
+ f"{row['beats_played']} / {row['beats_total']} {state}")
+
+ choice = ask_index(
+ f"\n {_BOLD('Quest slots')}",
+ ["Start a new slot", "Resume a slot", "Restart a slot from scene one", "Delete a slot"],
+ )
+ if choice is None:
+ print(_DIM(" Cancelled."))
+ return
+
+ if choice.startswith("Start"):
+ slot = input(" Name the new slot (lowercase-and-dashes): ").strip() or "slot-1"
+ from embr.walkthrough import WalkthroughSession, build_walkthrough_conversation
+
+ session = WalkthroughSession(build_walkthrough_conversation(model=_choose_model()))
+ print(f"\n {_BOLD('Dawn Whitmore')}, keeper of the Ember Hearth."
+ f" {_DIM(f'{session.progress[1]} scenes.')}")
+ _play_saved(session, slot)
+ return
+
+ loadable = [row for row in rows if not row["problems"]]
+ if not loadable:
+ print(_YEL(" No loadable slot for that."))
+ return
+ names = [f"{row['quest_id']}/{row['slot']}" for row in loadable]
+ picked = ask_index(" Which slot?", names)
+ if picked is None:
+ print(_DIM(" Cancelled."))
+ return
+ quest_id, slot = picked.split("/", 1)
+
+ if choice.startswith("Resume"):
+ session, _payload = load_slot(slot, quest_id=quest_id, model=_choose_model())
+ _play_saved(session, slot)
+ elif choice.startswith("Restart"):
+ if input(f" Type RESTART to wipe {quest_id}/{slot} and begin again: ").strip() != "RESTART":
+ print(_DIM(" Cancelled."))
+ return
+ delete_slot(slot, quest_id=quest_id)
+ from embr.walkthrough import WalkthroughSession, build_walkthrough_conversation
+
+ session = WalkthroughSession(build_walkthrough_conversation(model=_choose_model()))
+ _play_saved(session, slot)
+ else:
+ if input(f" Type DELETE to remove {quest_id}/{slot}: ").strip() != "DELETE":
+ print(_DIM(" Cancelled."))
+ return
+ delete_slot(slot, quest_id=quest_id)
+ print(_GRN(f" Removed {quest_id}/{slot}."))
+
+
+def _dashboard_report(
+ saves_root: Any = Path("data/saves"),
+ attribution_root: Any = Path("data/runs/attribution"),
+ experiments_dir: Any = Path("data/experiments"),
+ runs_dir: Any = None,
+) -> list[str]:
+ """The read-only research dashboard, as printable lines.
+
+ Every row is read from disk artefacts and the saved path; nothing is computed fresh
+ and nothing is written. Absence is a word (not run, no save), never a number, and the
+ v2 attack corpus is always labelled the extension, apart from the published v1.
+ """
+ from embr.saves import latest_slot, list_slots
+ from embr.walkthrough import DAWN_ARC
+
+ saves_root, attribution_root = Path(saves_root), Path(attribution_root)
+ experiments_dir = Path(experiments_dir)
+ runs = sorted((Path(runs_dir) if runs_dir is not None else RUNS_DIR).glob("*/results.json"))
+ lines: list[str] = [f" {_BOLD('QUEST PATH')}"]
+
+ found = latest_slot(root=saves_root)
+ payload_history: list[dict] = []
+ if found is None:
+ lines.append(_DIM(" no save yet: the path below is unplayed"))
+ played = 0
+ else:
+ quest_id, slot = found
+ row = next(r for r in list_slots(quest_id, root=saves_root) if r["slot"] == slot)
+ played = int(row["beats_played"] or 0)
+ lines.append(f" resumes at {_WHT(f'{quest_id}/{slot}')}, scene {played + 1}")
+ import json as _json
+
+ payload_history = _json.loads(
+ (saves_root / quest_id / f"{slot}.json").read_text(encoding="utf-8")
+ ).get("history", [])
+ for index, beat in enumerate(DAWN_ARC):
+ mark = _GRN("x") if index < played else (_EMBER(">") if index == played else _DIM("-"))
+ lines.append(f" [{mark}] {beat.id}")
+
+ lines.append(f"\n {_BOLD('STATE TIMELINE')}")
+ if not payload_history:
+ lines.append(_DIM(" no saved turns yet"))
+ for turn in payload_history:
+ lines.append(
+ f" #{turn['turn_index']} {turn['beat_id'] or 'free-play'} "
+ f"mood {turn['mood_before']['valence']:+.2f} to {turn['mood_after']['valence']:+.2f} "
+ f"trust {turn['trust_before']:+.2f} to {turn['trust_after']:+.2f} "
+ f"recalled {turn['retrieved_ids']}"
+ )
+
+ lines.append(f"\n {_BOLD('ATTRIBUTION')}")
+ from eval.context_attribution import newest_run_by_estimator
+
+ newest_by_estimator = newest_run_by_estimator(attribution_root)
+ for estimator in ("likelihood", "behavioural"):
+ run = newest_by_estimator.get(estimator)
+ if run is None:
+ lines.append(f" {estimator}: {_DIM('not run')}")
+ continue
+ scale = "measured" if run["readings"] >= 20 else "pilot only"
+ rho = f" position-bias rho {run['mean_rho']:+.2f}" if run["mean_rho"] is not None else ""
+ lines.append(
+ f" {estimator}: {_GRN(scale) if scale == 'measured' else _YEL(scale)}"
+ f" · {run['readings']} readings · {run['model']} · {run['stamp']}{rho}"
+ )
+ if len(newest_by_estimator) >= 2:
+ lines.append(_DIM(" paired readings available: likelihood vs behavioural"))
+
+ lines.append(f"\n {_BOLD('ATTACKS')}")
+ v1 = f"{_GRN('measured')} in {len(runs)} evaluation runs" if runs else _DIM("not run")
+ lines.append(f" v1 corpus: {v1}")
+ v2_present = (experiments_dir / "attacks_v2.json").exists()
+ v2 = _YEL("staged, results on disk") if v2_present else _DIM("not run")
+ lines.append(f" v2 corpus: {v2} (the extension; never blended into v1)")
+
+ lines.append(f"\n {_BOLD('EVIDENCE')}")
+ lines.append(f" full evaluation: {_GRN('measured') if runs else _DIM('not run')}")
+ defended = (experiments_dir / "provenance.json").exists()
+ lines.append(
+ f" defence sweep: {_GRN('measured') if defended else _DIM('demo-only (computed live)')}"
+ )
+ lines.append(f" web demo tabs: {_DIM('demo-only (presentation, not evidence)')}")
+ return lines
+
+
+def _do_dashboard() -> None:
+ """Print the read-only research dashboard. Looking at it changes nothing."""
+ print()
+ for line in _dashboard_report():
+ print(line)
+
+
+def _do_maintenance() -> None:
+ """The destructive operations, out of the main menu, each behind its own confirmation."""
+ choice = ask_index(
+ f"\n {_BOLD('Maintenance')}",
+ ["Delete all generated data (runs, figures, tables)"],
+ )
+ if choice is None:
+ print(_DIM(" Cancelled."))
+ return
+ _do_delete_run_data()
+
+
+def _do_quick_scoreboard() -> None:
+ """RQ3 at published default weights: the sub-second answer."""
+ from eval.run import fast_rq3_defaults
+
+ print(f"\n {_BOLD('nDCG@5, published defaults')}")
+ for variant, value in fast_rq3_defaults().items():
+ print(f" {variant:<16} {_YEL(f'{value:.3f}')}")
+ print(_DIM("\n Tuning, ablations, RQ1 and RQ2 live in the full evaluation (option 4)."))
+
+
+def _do_full_evaluation() -> None:
+ """Run all three studies and write a run directory."""
+ from eval.run import run_all
+
+ print(_DIM("\n Running RQ1, RQ2 and RQ3. This takes a minute or two."))
+ path, _summary = run_all(progress=lambda message: print(_DIM(f" {message}")))
+ print(f"\n {_GRN('✓ Done.')} Results in {_BOLD(str(path))}")
+ print(_DIM(" Option 11 turns this into the paper's figures and tables."))
+ _chime()
+
+
+def _do_seeded_runs() -> None:
+ """Replicate the evaluation, either on one model or across several."""
+ from eval.experiments import AVAILABLE_MODELS, cross_model_experiment, replicate_experiment
+
+ choice = ask_index(
+ f"\n {_BOLD('Seeded runs')}",
+ ["Same model, repeated: does the harness reproduce?",
+ "Across models: what moves when the model changes?"],
+ )
+ if choice is None:
+ print(_DIM(" Cancelled."))
+ return
+ if choice.startswith("Same"):
+ report = replicate_experiment(replicates=3)
+ verdict = _GRN("identical") if report["identical"] else _RED("DIVERGED")
+ print(f"\n {report['replicates']} runs on {report['model']}: {_BOLD(verdict)}")
+ else:
+ print(_DIM(f"\n Models: {', '.join(AVAILABLE_MODELS)}"))
+ report = cross_model_experiment()
+ print(f"\n {len(report['models'])} models compared.")
+ print(_DIM(f" Written to {report['out_dir']}"))
+ _chime()
+
+
+def _do_bakeoff() -> None:
+ """Compare the looped thesis model against conventional models of similar size."""
+ try:
+ from eval.bakeoff import run_bakeoff
+ except ImportError:
+ print(_YEL("\n The bake-off is not built yet."))
+ print(_DIM(" It compares Ouro 1.4B (looped) against conventional models."))
+ return
+
+ print(_DIM("\n Holding prompts, memories and sampling equal, varying only the model."
+ " Ouro is slow, so this takes several minutes."))
+ path, verdict = run_bakeoff()
+ print(f"\n {_GRN('✓ Done.')} {path}")
+ print(f" {verdict}")
+ _chime()
+
+
+def _do_affective_indexing() -> None:
+ """Flip every memory's valence: the fact survives, the mood inverts."""
+ from eval.emotion_flip import main
+
+ print()
+ main()
+
+
+def _do_attribution() -> None:
+ """Per-signal attribution of the poisoning result: zero one weight at a time."""
+ from eval.attribution import main
+
+ print()
+ main()
+
+
+def _do_provenance_sweep() -> None:
+ """Sweep anchored scoring mass and watch poisoning fall to zero."""
+ from eval.provenance import main
+
+ print()
+ main()
+
+
+def _do_grid() -> None:
+ """Every injected text under four tag conditions against every arm."""
+ from eval.grid import main
+
+ print()
+ main()
+
+
+#: The asset steps in build order. The results page comes after the figures it embeds,
+#: and it reads the run rather than trusting anything typed, so a drift between the run
+#: and docs/findings.md stops the build loudly.
+ASSET_STEPS = ("tables", "figures", "experiments", "results", "questline")
+
+
+def build_assets(run_dir: Path, steps: Sequence[str] = ASSET_STEPS) -> list[Path]:
+ """Rebuild the chosen paper assets from `run_dir`. Shared by the menu and the CLI."""
+ written: list[Path] = []
+ if "tables" in steps:
+ from eval.report.build_tables import build_all_tables
+
+ written += list(build_all_tables(run_dir))
+ if "figures" in steps:
+ from eval.report.build_figures import build_all_figures
+
+ written += list(build_all_figures(run_dir))
+ if "experiments" in steps:
+ # The mechanism figures recompute from the harness rather than from the run, and
+ # leaving them out is how half a figure set goes stale without anyone noticing.
+ from eval.report.build_bakeoff_figures import build_experiment_figures
+
+ written += list(build_experiment_figures())
+ if "results" in steps:
+ from eval.report.build_results import DriftError, build_results
+
+ try:
+ written += list(build_results(run_dir))
+ except DriftError as error:
+ print(_RED("\n Results page refused to build:"))
+ print(_DIM(f" {error}"))
+ if "questline" in steps:
+ from eval.report.build_questline import build_questline
+
+ written += list(build_questline())
+ return written
+
+
+def _do_generate_assets() -> None:
+ """Rebuild figures and tables from the newest run."""
+ run_dir = _latest_run()
+ if run_dir is None:
+ print(_YEL("\n No run found. Use option 4 first."))
+ return
+ try:
+ import matplotlib # noqa: F401 the figures extra
+ except ImportError:
+ print(_RED("\n The figure builders need matplotlib."))
+ print(_DIM(" Install it with: uv sync --extra figures"))
+ return
+
+ labels = [
+ "tables (LaTeX + CSV)",
+ "figures from the run",
+ "figures from the experiments",
+ "results page (refuses to write if a number drifted)",
+ "questline map (from the arc, plus attribution status)",
+ ]
+ chosen = toggle_select("ASSETS", labels, default_indices=[0, 1, 2, 3, 4])
+ if not chosen:
+ print(_DIM(" Cancelled."))
+ return
+ steps = [step for step, label in zip(ASSET_STEPS, labels) if label in chosen]
+ print(_DIM(f"\n Building from {run_dir}..."))
+ written = build_assets(run_dir, steps)
+ print(f" {_GRN(f'✓ Wrote {len(written)} files.')}")
+ for path in written:
+ print(_DIM(f" {path}"))
+
+
+def _do_demo() -> None:
+ """Build the self-contained demo page and open it in a browser."""
+ import webbrowser
+
+ from eval.report.build_demo import build_demo
+
+ print(_DIM("\n Building from the newest run..."))
+ paths = build_demo()
+ for path in paths:
+ size = path.stat().st_size / 1024
+ print(f" {_GRN('✓ Wrote')} {path} {_DIM(f'({size:.0f} KB, opens with no server)')}")
+ if input(_DIM(" Open it now? [Y/n]: ")).strip().lower() not in ("n", "no"):
+ webbrowser.open(paths[0].resolve().as_uri()) # the flat diagram is the one to read
+
+
+def _do_web_demo() -> None:
+ """Serve the visual-novel web demo and open it in a browser."""
+ from web.server import serve
+
+ print(_DIM("\n Serving the web demo on http://127.0.0.1:8000 . Ctrl+C to stop and return."))
+ print(_DIM(" It opens on the best model this box can serve; the stub always works."))
+ serve(port=8000, open_browser=True)
+
+
+def _do_reckoning_reveal() -> None:
+ """Demo 1: play to the reckoning and reveal the six sources by Banzhaf weight."""
+ from embr.cli.demos import demo_reckoning_reveal
+
+ demo_reckoning_reveal()
+
+
+def _do_mood_slider() -> None:
+ """Demo 2: one line under three moods, retrieval and tone and attribution re-flowing."""
+ from embr.cli.demos import demo_mood_slider
+
+ demo_mood_slider()
+
+
+def _do_defence_dial() -> None:
+ """Demo 3: the anchor-weight dose-response, and its failure on a hostile anchor."""
+ from embr.cli.demos import demo_defence_dial
+
+ demo_defence_dial()
+
+
+def _do_tag_flip() -> None:
+ """Demo 4: flip an affect tag and watch the rank move while the words do not."""
+ from embr.cli.demos import demo_tag_flip
+
+ demo_tag_flip()
+
+
+def _do_estimator_divergence() -> None:
+ """Demo 5: where likelihood and behavioural attribution disagree (cached-only)."""
+ from embr.cli.demos import demo_estimator_divergence
+
+ demo_estimator_divergence()
+
+
+def _do_record_walk() -> None:
+ """Walk demos 1 to 4 in order, capture-ready for a screen recording."""
+ from embr.cli.demos import run_record
+
+ run_record()
+
+
+def _do_latest_results() -> None:
+ """Summarise the newest run without rerunning anything."""
+ run_dir = _latest_run()
+ if run_dir is None:
+ print(_YEL("\n No run found. Use option 4 first."))
+ return
+
+ results = json.loads((run_dir / "results.json").read_text(encoding="utf-8"))
+ meta = results.get("metadata", {})
+ print(f"\n {_BOLD(run_dir.name)}")
+ print(_DIM(f" model {meta.get('model', '?')} | labels {meta.get('label_set', '?')}"
+ f" {meta.get('label_version', '')} | commit {str(meta.get('git_commit', '?'))[:10]}\n"))
+ print(f" {'variant':<16} {'nDCG@5':>7}")
+ for variant, metrics in results.get("rq3", {}).get("variants", {}).items():
+ score = metrics.get("ndcg@5", float("nan"))
+ print(f" {variant:<16} {_YEL(f'{score:>7.3f}')}")
+ print(_DIM("\n Every interval spans zero at ten queries: read direction, not ranking."))
+
+
+def _do_settings() -> None:
+ """Show the live configuration and where to change it."""
+ from embr.config import DEFAULT_CONFIG_PATH, EmbrConfig
+
+ config = EmbrConfig.load()
+ rows = [
+ ("top-k retrieved", str(config.top_k)),
+ ("store backend", config.store_backend),
+ ("embedding model", config.embedding_model),
+ ("model runner", config.model_runner),
+ ] + [(f"weight: {name}", f"{w:g}" if isinstance(w, (int, float)) else str(w))
+ for name, w in config.weights.items()]
+ print()
+ for name, value in rows:
+ print(f" {name:<22} {_YEL(value)}")
+ print(_DIM(f"\n Edit {DEFAULT_CONFIG_PATH} and reopen. Zero a weight to ablate it."))
+
+
+def _do_fetch_lexicon() -> None:
+ """Download the NRC VAD lexicon so the reported tone rater is the published one."""
+ from eval.tone import LEXICON_PATH, LEXICON_URL, fetch_lexicon
+
+ if LEXICON_PATH.exists():
+ print(_DIM(f"\n Already on disk: {LEXICON_PATH}"))
+ return
+ print(_DIM(f"\n Fetching {LEXICON_URL} (about 6 MB)..."))
+ path = fetch_lexicon()
+ print(f" {_GRN('✓ Wrote')} {path}")
+ print(_DIM(" Free for research, cite Mohammad (2018, 2025), never redistribute: data/ is gitignored."))
+
+
+#: Everything the pipeline generates. Nothing hand written lives under any of these, which
+#: is what makes wiping them safe: the branding, the architecture diagram and the builders
+#: all live under assets/ and are never touched.
+GENERATED_DATA_DIRS = (RUNS_DIR, FIGURES_DIR, TABLES_DIR)
+
+
+def delete_generated_data(directories: Sequence[Path] = GENERATED_DATA_DIRS) -> list[Path]:
+ """Delete every generated data directory and return the ones that were removed.
+
+ Separated from the prompting so it can be tested without a terminal, and so the
+ confirmation cannot drift away from what actually gets deleted.
+ """
+ import shutil
+
+ removed: list[Path] = []
+ for directory in directories:
+ if directory.exists():
+ shutil.rmtree(directory)
+ removed.append(directory)
+ return removed
+
+
+def _do_delete_run_data() -> None:
+ """Wipe every generated data directory after a typed confirmation."""
+ present = [directory for directory in GENERATED_DATA_DIRS if directory.exists()]
+ if not present:
+ print(_DIM("\n Nothing to delete: no generated data on disk."))
+ return
+
+ print(_RED(_BOLD("\n WARNING, this permanently deletes:")))
+ for directory in present:
+ count = sum(1 for path in directory.rglob("*") if path.is_file())
+ print(f" {_YEL(str(directory))} {_DIM(f'({count} files)')}")
+ print(_DIM("\n Runs, figures and tables all regenerate from option 4 then option 11."
+ " Nothing under assets/ is touched."))
+ if input("\n Type DELETE to confirm, anything else cancels: ").strip() != "DELETE":
+ print(_DIM(" Cancelled."))
+ return
+
+ removed = delete_generated_data(present)
+ print(f" {_GRN(f'✓ Deleted {len(removed)} directories.')}")
+
+
+# Key to handler. One table, so adding an option cannot drift from its dispatch.
+_ACTIONS: dict[str, Callable[[], None]] = {
+ "R": _do_continue,
+ "Q": _do_quests,
+ "1": _do_conversation_turn,
+ "2": _do_walkthrough,
+ "W": _do_web_demo,
+ "3": _do_quick_scoreboard,
+ "4": _do_full_evaluation,
+ "5": _do_seeded_runs,
+ "6": _do_bakeoff,
+ "7": _do_affective_indexing,
+ "8": _do_attribution,
+ "9": _do_provenance_sweep,
+ "10": _do_grid,
+ "11": _do_generate_assets,
+ "12": _do_demo,
+ "13": _do_latest_results,
+ "V": _do_dashboard,
+ "14": _do_reckoning_reveal,
+ "15": _do_mood_slider,
+ "16": _do_defence_dial,
+ "17": _do_tag_flip,
+ "18": _do_estimator_divergence,
+ "19": _do_record_walk,
+ "S": _do_settings,
+ "L": _do_fetch_lexicon,
+ "M": _do_maintenance,
+ "C": _clear,
+}
+
+
+def run_menu() -> None:
+ """Show the EMBR menu and dispatch until the user exits."""
+ while True:
+ _print_menu()
+ try:
+ choice = input(_BOLD(" ⟫ ")).strip().upper()
+ except (EOFError, KeyboardInterrupt):
+ choice = "0"
+
+ if choice == "0":
+ _clear()
+ print(_EMBER(_BOLD("Goodbye.")) + "\n")
+ return
+
+ action = _ACTIONS.get(choice)
+ if action is None:
+ print(_RED(f" Invalid option: '{choice}'"))
+ _pause()
+ continue
+
+ try:
+ action()
+ except KeyboardInterrupt:
+ print(_DIM("\n Interrupted."))
+ except Exception as error: # an error boundary: one bad option must not kill the menu
+ print(_RED(f"\n ✖ {type(error).__name__}: {error}"))
+ hint = _error_hint(error)
+ if hint:
+ print(_DIM(f" {hint}"))
+ if action is not _clear:
+ _pause()
+
+
+if __name__ == "__main__":
+ run_menu()
diff --git a/embr/config.py b/src/embr/config.py
similarity index 100%
rename from embr/config.py
rename to src/embr/config.py
diff --git a/embr/embeddings.py b/src/embr/embeddings.py
similarity index 100%
rename from embr/embeddings.py
rename to src/embr/embeddings.py
diff --git a/embr/memory.py b/src/embr/memory.py
similarity index 100%
rename from embr/memory.py
rename to src/embr/memory.py
diff --git a/embr/model.py b/src/embr/model.py
similarity index 100%
rename from embr/model.py
rename to src/embr/model.py
diff --git a/embr/pipeline.py b/src/embr/pipeline.py
similarity index 64%
rename from embr/pipeline.py
rename to src/embr/pipeline.py
index fd0736f..6f31a66 100644
--- a/embr/pipeline.py
+++ b/src/embr/pipeline.py
@@ -15,9 +15,10 @@
from __future__ import annotations
from dataclasses import dataclass, field
+from typing import Callable
from .affect import CharacterState, Mood, appraise
-from .memory import EventType, Memory, MemoryStore
+from .memory import EventType, Memory, MemoryStore, Provenance
from .model import ModelRunner, StubRunner
from .prompt import PromptBuilder
from .scoring import CompositeScorer, embr_scorer
@@ -36,6 +37,10 @@ class Turn:
reply: str
retrieved: list[Memory] = field(default_factory=list)
prompt: str = ""
+ #: One dict per retrieved memory, in the same order: each signal's weighted contribution
+ #: to that memory's score. This is the "why" a researcher or a tools engineer wants
+ #: beside every reply, and it costs nothing extra because the scorer already has it.
+ breakdown: list[dict[str, float]] = field(default_factory=list)
class Conversation:
@@ -53,13 +58,50 @@ def __init__(
prompt_builder: PromptBuilder | None = None,
model: ModelRunner | None = None,
top_k: int = 3,
+ tagger: Callable[[str], tuple[float, float]] | None = None,
) -> None:
self.state = state
- self.store = store or MemoryStore()
+ # `is not None`, not `or`: an empty store has len 0 and would be dropped.
+ self.store = store if store is not None else MemoryStore()
self.scorer = scorer or embr_scorer()
self.prompt_builder = prompt_builder or PromptBuilder()
self.model = model or StubRunner()
self.top_k = top_k
+ #: How `tag_event` reads affect off an untagged runtime event: any text to
+ #: (valence, arousal) function, a lexicon rater for instance. None means neutral.
+ self.tagger = tagger
+
+ def tag_event(
+ self,
+ text: str,
+ *,
+ event_type: EventType = EventType.NORMAL,
+ valence: float | None = None,
+ arousal: float | None = None,
+ ) -> Memory:
+ """Build a runtime event with the write-boundary policy applied, ready for `take_turn`.
+
+ Everything that arrives during play is `written_by=EXTERNAL`. The affect tag is the
+ attacked input (findings 2.3), so where it came from is recorded honestly: numbers the
+ caller supplies are `tagged_by=EXTERNAL`, and a tag this layer derived from the text
+ (through `tagger`, or neutral when there is none) is `tagged_by=APPRAISED`. Under the
+ defended scorer that stamp is what the provenance anchor reads, so a client that never
+ writes affect metadata gets the 6/10 posture by default and the 9/10 one only on
+ request. Values are clamped to the circumplex before they can move anything.
+ """
+ if valence is None and arousal is None:
+ valence, arousal = self.tagger(text) if self.tagger else (0.0, 0.0)
+ tagged_by = Provenance.APPRAISED
+ else:
+ tagged_by = Provenance.EXTERNAL
+ return Memory(
+ text=text,
+ valence=max(-1.0, min(1.0, float(valence or 0.0))),
+ arousal=max(0.0, min(1.0, float(arousal or 0.0))),
+ event_type=event_type,
+ written_by=Provenance.EXTERNAL,
+ tagged_by=tagged_by,
+ )
def take_turn(self, player_input: str, event: Memory | None = None) -> Turn:
"""Run one full turn and return the reply plus the memories that informed it."""
@@ -78,12 +120,17 @@ def take_turn(self, player_input: str, event: Memory | None = None) -> Turn:
# 3. + 4. score every memory and keep the most relevant few
retrieved = self.scorer.top_k(self.store.all(), player_input, self.state, self.top_k)
+ breakdown = [self.scorer.breakdown(m, player_input, self.state) for m in retrieved]
prompt = self.prompt_builder.build(self.state, retrieved, player_input)
# 5. generate the reply
reply = self.model.generate(prompt)
return Turn(
- player_input=player_input, reply=reply, retrieved=retrieved, prompt=prompt
+ player_input=player_input,
+ reply=reply,
+ retrieved=retrieved,
+ prompt=prompt,
+ breakdown=breakdown,
)
diff --git a/embr/prompt.py b/src/embr/prompt.py
similarity index 100%
rename from embr/prompt.py
rename to src/embr/prompt.py
diff --git a/embr/saves.py b/src/embr/saves.py
similarity index 93%
rename from embr/saves.py
rename to src/embr/saves.py
index 671a67d..0ad268d 100644
--- a/embr/saves.py
+++ b/src/embr/saves.py
@@ -15,6 +15,7 @@
import hashlib
import json
+import math
import os
import re
from dataclasses import asdict
@@ -166,6 +167,23 @@ def _read_payload(path: Path) -> dict[str, Any] | None:
return None
+def _find_non_finite(value: Any, location: str, problems: list[str]) -> None:
+ """Recurse the payload: a NaN or inf float anywhere makes the save unloadable.
+
+ json.loads accepts NaN/Infinity by default, and a handcrafted save with a valid
+ content hash can put one in a valence or trust field; the scorer then returns nan
+ silently for the rest of the session. Reject at the load boundary instead.
+ """
+ if isinstance(value, float) and not math.isfinite(value):
+ problems.append(f"non-finite number at {location}.")
+ elif isinstance(value, dict):
+ for key, item in value.items():
+ _find_non_finite(item, f"{location}.{key}", problems)
+ elif isinstance(value, list):
+ for index, item in enumerate(value):
+ _find_non_finite(item, f"{location}[{index}]", problems)
+
+
def validate_payload(payload: dict[str, Any], beats: Sequence[Beat] = DAWN_ARC) -> list[str]:
"""Every reason this save cannot be loaded against the current code and content.
@@ -182,6 +200,7 @@ def validate_payload(payload: dict[str, Any], beats: Sequence[Beat] = DAWN_ARC)
for key in ("state", "memories", "written_memories", "beats_played"):
if key not in payload:
problems.append(f"missing field {key!r}.")
+ _find_non_finite(payload, "$", problems)
return problems
diff --git a/embr/scoring.py b/src/embr/scoring.py
similarity index 100%
rename from embr/scoring.py
rename to src/embr/scoring.py
diff --git a/src/embr/serve.py b/src/embr/serve.py
new file mode 100644
index 0000000..bcfb4f3
--- /dev/null
+++ b/src/embr/serve.py
@@ -0,0 +1,368 @@
+"""Serve NPCs to a game engine over JSON: the middleware, reachable from anything.
+
+The library is Python; a game usually is not. This is the seam for everything else: a
+stdlib HTTP server that keeps one `Conversation` per NPC, persists each one under
+`data/npcs/`, and answers a turn with the reply, the new state, and why each memory was
+chosen. Unity, Godot, a SMAPI mod, or curl can drive it with four routes.
+
+ python -m embr serve # http://127.0.0.1:8017, the stub, no model
+ python -m embr serve --model ollama # replies from the local Ollama daemon
+ python -m embr serve --defended # the provenance-anchored scorer
+ python -m embr serve --tagger lexicon # tag untagged runtime events from their words
+
+ PUT /npc/ {persona, mood?: {valence, arousal}, trust?, memories?: [...]}
+ GET /npc/ the state, and every memory with its provenance
+ GET /npcs every NPC id on disk
+ POST /npc//turn {player_input, event?: {text, event_type?, valence?, arousal?}}
+
+Memories sent at creation are authored content and stamped as such. Anything that arrives
+in a turn is external, and its affect tag is recorded as external only when the client
+supplied the numbers (see `Conversation.tag_event`). Ids are lowercase slugs, one file pair
+each, so the directory is the registry.
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import re
+from http.server import BaseHTTPRequestHandler, HTTPServer
+from pathlib import Path
+from typing import Any, Callable, Iterable
+from urllib.parse import urlparse
+
+from .affect import CharacterState, Mood
+from .config import EmbrConfig, build_embedder, build_model
+from .memory import EventType, Memory, Provenance, SQLiteMemoryStore
+from .model import ModelRunner
+from .pipeline import Conversation, Turn
+from .saves import atomic_write_text
+from .scoring import CompositeScorer, defended_embr_scorer, embr_scorer
+
+DEFAULT_ROOT = Path("data/npcs")
+DEFAULT_PORT = 8017
+
+#: An id is a folder-safe slug: it names two files and appears in a URL.
+_ID = re.compile(r"^[a-z0-9][a-z0-9-]{0,63}$")
+
+
+def _memory_payload(memory: Memory) -> dict[str, Any]:
+ return {
+ "id": memory.id,
+ "text": memory.text,
+ "valence": memory.valence,
+ "arousal": memory.arousal,
+ "event_type": memory.event_type.value,
+ "timestamp": memory.timestamp.isoformat(),
+ "written_by": memory.written_by.value,
+ "tagged_by": memory.tagged_by.value,
+ }
+
+
+def _state_payload(state: CharacterState) -> dict[str, Any]:
+ return {
+ "persona": state.persona,
+ "mood": {"valence": state.mood.valence, "arousal": state.mood.arousal},
+ "trust": state.trust,
+ }
+
+
+def turn_payload(turn: Turn, state: CharacterState) -> dict[str, Any]:
+ """What a turn hands back: the reply, the state it left behind, and the evidence."""
+ return {
+ "reply": turn.reply,
+ **_state_payload(state),
+ "retrieved": [
+ {**_memory_payload(memory), "score": sum(parts.values()), "breakdown": parts}
+ for memory, parts in zip(turn.retrieved, turn.breakdown)
+ ],
+ "prompt": turn.prompt,
+ }
+
+
+class NpcRegistry:
+ """Every NPC this server knows, each a `Conversation` over its own SQLite file.
+
+ State (persona, mood, trust) lives in `.json` beside `.sqlite`, rewritten
+ atomically after every turn, so a restart resumes exactly where play stopped. One model
+ runner is shared: the layer in front of it is per character, the model is not.
+ """
+
+ def __init__(
+ self,
+ root: Path | str = DEFAULT_ROOT,
+ *,
+ config: EmbrConfig | None = None,
+ model: ModelRunner | None = None,
+ defended: bool = False,
+ tagger: Callable[[str], tuple[float, float]] | None = None,
+ ) -> None:
+ self.root = Path(root)
+ self.config = config or EmbrConfig()
+ self.model = model or build_model(self.config)
+ self.defended = defended
+ self.tagger = tagger
+ self._live: dict[str, Conversation] = {}
+
+ # ----------------------------------------------------------------------- lifecycle
+
+ def ids(self) -> list[str]:
+ return sorted(path.stem for path in self.root.glob("*.json"))
+
+ def create(
+ self,
+ npc_id: str,
+ persona: str,
+ *,
+ mood: Mood | None = None,
+ trust: float = 0.0,
+ memories: Iterable[Memory] = (),
+ ) -> Conversation:
+ """Create an NPC, or replace one wholesale: PUT semantics, authored content only."""
+ self._require_id(npc_id)
+ if npc_id in self._live:
+ self._live.pop(npc_id).store.close()
+ self._db_path(npc_id).unlink(missing_ok=True)
+ state = CharacterState(persona=persona, mood=mood or Mood(), trust=trust)
+ conversation = self._open(npc_id, state)
+ for memory in memories:
+ memory.written_by = memory.tagged_by = Provenance.AUTHORED
+ conversation.store.add(memory)
+ self._persist(npc_id, conversation)
+ return conversation
+
+ def get(self, npc_id: str) -> Conversation | None:
+ """The live conversation, opened from disk on first touch after a restart."""
+ self._require_id(npc_id)
+ if npc_id not in self._live:
+ state_path = self.root / f"{npc_id}.json"
+ if not state_path.exists():
+ return None
+ saved = json.loads(state_path.read_text(encoding="utf-8"))
+ state = CharacterState(
+ persona=saved["persona"],
+ mood=Mood(**saved["mood"]),
+ trust=float(saved["trust"]),
+ )
+ self._open(npc_id, state)
+ return self._live[npc_id]
+
+ def turn(self, npc_id: str, player_input: str, event: dict[str, Any] | None = None) -> dict:
+ """One full turn for one NPC, persisted before it is answered."""
+ conversation = self.get(npc_id)
+ if conversation is None:
+ raise KeyError(npc_id)
+ memory = None
+ if event:
+ memory = conversation.tag_event(
+ str(event.get("text") or player_input),
+ event_type=EventType(event.get("event_type") or EventType.NORMAL.value),
+ valence=event.get("valence"),
+ arousal=event.get("arousal"),
+ )
+ turn = conversation.take_turn(player_input, event=memory)
+ self._persist(npc_id, conversation)
+ return turn_payload(turn, conversation.state)
+
+ def describe(self, npc_id: str) -> dict[str, Any] | None:
+ conversation = self.get(npc_id)
+ if conversation is None:
+ return None
+ return {
+ "id": npc_id,
+ **_state_payload(conversation.state),
+ "memories": [_memory_payload(m) for m in conversation.store.all()],
+ }
+
+ def close(self) -> None:
+ for conversation in self._live.values():
+ conversation.store.close()
+ self._live.clear()
+
+ # ------------------------------------------------------------------------- helpers
+
+ def _open(self, npc_id: str, state: CharacterState) -> Conversation:
+ embedder = build_embedder(self.config)
+ scorer: CompositeScorer = (
+ defended_embr_scorer(embedder) if self.defended else embr_scorer(embedder)
+ )
+ self.root.mkdir(parents=True, exist_ok=True)
+ conversation = Conversation(
+ state=state,
+ store=SQLiteMemoryStore(str(self._db_path(npc_id)), embedder=embedder),
+ scorer=scorer,
+ model=self.model,
+ top_k=self.config.top_k,
+ tagger=self.tagger,
+ )
+ self._live[npc_id] = conversation
+ return conversation
+
+ def _persist(self, npc_id: str, conversation: Conversation) -> None:
+ atomic_write_text(
+ self.root / f"{npc_id}.json",
+ json.dumps(_state_payload(conversation.state), indent=2) + "\n",
+ )
+
+ def _db_path(self, npc_id: str) -> Path:
+ return self.root / f"{npc_id}.sqlite"
+
+ @staticmethod
+ def _require_id(npc_id: str) -> None:
+ if not _ID.match(npc_id):
+ raise ValueError(f"NPC id {npc_id!r}: use a lowercase slug such as dawn-whitmore")
+
+
+# ------------------------------------------------------------------------------ HTTP
+
+
+def _memory_from_json(item: dict[str, Any]) -> Memory:
+ return Memory(
+ text=str(item["text"]),
+ valence=float(item.get("valence", 0.0)),
+ arousal=float(item.get("arousal", 0.0)),
+ event_type=EventType(item.get("event_type") or EventType.NORMAL.value),
+ )
+
+
+class NpcHandler(BaseHTTPRequestHandler):
+ """Four routes over one registry. Errors are JSON with the reason, never a stack trace."""
+
+ registry: NpcRegistry
+
+ def log_message(self, *args: object) -> None:
+ pass
+
+ def do_GET(self) -> None:
+ parts = urlparse(self.path).path.strip("/").split("/")
+ if parts == ["npcs"]:
+ self._send(200, {"npcs": self.registry.ids()})
+ elif len(parts) == 2 and parts[0] == "npc":
+ self._guarded(lambda: self._describe(parts[1]))
+ else:
+ self._send(404, {"error": "not found"})
+
+ def do_PUT(self) -> None:
+ parts = urlparse(self.path).path.strip("/").split("/")
+ if len(parts) != 2 or parts[0] != "npc":
+ self._send(404, {"error": "not found"})
+ return
+ self._guarded(lambda: self._create(parts[1], self._body()))
+
+ def do_POST(self) -> None:
+ parts = urlparse(self.path).path.strip("/").split("/")
+ if len(parts) != 3 or parts[0] != "npc" or parts[2] != "turn":
+ self._send(404, {"error": "not found"})
+ return
+ self._guarded(lambda: self._turn(parts[1], self._body()))
+
+ # ------------------------------------------------------------------------- routes
+
+ def _describe(self, npc_id: str) -> None:
+ described = self.registry.describe(npc_id)
+ self._send(200, described) if described else self._send(404, {"error": "no such NPC"})
+
+ def _create(self, npc_id: str, body: dict[str, Any]) -> None:
+ persona = str(body.get("persona") or "").strip()
+ if not persona:
+ self._send(400, {"error": "persona is required"})
+ return
+ mood = body.get("mood") or {}
+ self.registry.create(
+ npc_id,
+ persona,
+ mood=Mood(float(mood.get("valence", 0.0)), float(mood.get("arousal", 0.0))),
+ trust=float(body.get("trust", 0.0)),
+ memories=[_memory_from_json(item) for item in body.get("memories") or []],
+ )
+ self._send(201, self.registry.describe(npc_id) or {})
+
+ def _turn(self, npc_id: str, body: dict[str, Any]) -> None:
+ player_input = str(body.get("player_input") or "").strip()
+ if not player_input:
+ self._send(400, {"error": "player_input is required"})
+ return
+ try:
+ self._send(200, self.registry.turn(npc_id, player_input, body.get("event")))
+ except KeyError:
+ self._send(404, {"error": "no such NPC"})
+
+ # ------------------------------------------------------------------------ helpers
+
+ def _guarded(self, action: Callable[[], None]) -> None:
+ """Bad input is the client's problem and gets a 400 with the reason."""
+ try:
+ action()
+ except (ValueError, KeyError, TypeError) as error:
+ self._send(400, {"error": str(error)})
+
+ def _body(self) -> dict[str, Any]:
+ length = int(self.headers.get("Content-Length") or 0)
+ if not length or length > 1_000_000:
+ return {}
+ try:
+ data = json.loads(self.rfile.read(length) or b"{}")
+ except ValueError:
+ return {}
+ return data if isinstance(data, dict) else {}
+
+ def _send(self, code: int, payload: dict[str, Any]) -> None:
+ data = json.dumps(payload).encode("utf-8")
+ self.send_response(code)
+ self.send_header("Content-Type", "application/json")
+ self.send_header("Content-Length", str(len(data)))
+ self.end_headers()
+ self.wfile.write(data)
+
+
+def build_server(port: int = DEFAULT_PORT, registry: NpcRegistry | None = None) -> HTTPServer:
+ """The server, not yet serving. Port 0 picks a free one, which is what the tests use."""
+ NpcHandler.registry = registry or NpcRegistry()
+ # ponytail: single-threaded on purpose. SQLite connections are per thread, and a game
+ # talks to one character at a time; move to a per-NPC lock and ThreadingHTTPServer if
+ # a client ever needs concurrent turns.
+ return HTTPServer(("127.0.0.1", port), NpcHandler)
+
+
+def main(argv: list[str] | None = None) -> int:
+ parser = argparse.ArgumentParser(prog="embr serve", description=__doc__.splitlines()[0])
+ parser.add_argument("--port", type=int, default=DEFAULT_PORT)
+ parser.add_argument("--root", default=str(DEFAULT_ROOT), help="where NPCs are kept")
+ parser.add_argument("--model", choices=("stub", "ollama", "ouro"), default=None,
+ help="override the configured model runner")
+ parser.add_argument("--defended", action="store_true",
+ help="score with the provenance anchor (the measured defence)")
+ parser.add_argument("--tagger", choices=("none", "lexicon"), default="none",
+ help="how untagged runtime events get their affect")
+ args = parser.parse_args(argv)
+
+ config = EmbrConfig.load()
+ if args.model:
+ config.model_runner = args.model
+ tagger = None
+ if args.tagger == "lexicon":
+ # ponytail: the lexicon rater lives in the harness; lift it into embr/ if a second
+ # runtime caller appears. Imported here so the library never depends on eval/.
+ from eval.tone import default_tone_rater
+
+ tagger = default_tone_rater().rate
+ registry = NpcRegistry(args.root, config=config, defended=args.defended, tagger=tagger)
+ server = build_server(args.port, registry)
+ print(
+ f"EMBR serving NPCs on http://127.0.0.1:{args.port} (Ctrl+C to stop). "
+ f"Replies come from: {config.model_runner}. "
+ f"Scorer: {'defended' if args.defended else 'published'}. NPCs: {registry.ids() or 'none yet'}.",
+ flush=True,
+ )
+ try:
+ server.serve_forever()
+ except KeyboardInterrupt:
+ print("\nstopped.")
+ finally:
+ server.server_close()
+ registry.close()
+ return 0
+
+
+if __name__ == "__main__":
+ raise SystemExit(main())
diff --git a/embr/vectors.py b/src/embr/vectors.py
similarity index 100%
rename from embr/vectors.py
rename to src/embr/vectors.py
diff --git a/embr/walkthrough.py b/src/embr/walkthrough.py
similarity index 100%
rename from embr/walkthrough.py
rename to src/embr/walkthrough.py
diff --git a/eval/__init__.py b/src/eval/__init__.py
similarity index 100%
rename from eval/__init__.py
rename to src/eval/__init__.py
diff --git a/eval/agreement.py b/src/eval/agreement.py
similarity index 100%
rename from eval/agreement.py
rename to src/eval/agreement.py
diff --git a/eval/attacks.py b/src/eval/attacks.py
similarity index 100%
rename from eval/attacks.py
rename to src/eval/attacks.py
diff --git a/eval/attacks_v2.py b/src/eval/attacks_v2.py
similarity index 100%
rename from eval/attacks_v2.py
rename to src/eval/attacks_v2.py
diff --git a/eval/attribution.py b/src/eval/attribution.py
similarity index 100%
rename from eval/attribution.py
rename to src/eval/attribution.py
diff --git a/eval/backends.py b/src/eval/backends.py
similarity index 95%
rename from eval/backends.py
rename to src/eval/backends.py
index b448746..cead244 100644
--- a/eval/backends.py
+++ b/src/eval/backends.py
@@ -21,13 +21,18 @@
from __future__ import annotations
import json
+import os
import subprocess
from collections.abc import Sequence
from pathlib import Path
from embr import CharacterState, Memory
-MNEMOSYNE_PYTHON = Path(".venv-mnemosyne/Scripts/python.exe")
+MNEMOSYNE_PYTHON = (
+ Path(".venv-mnemosyne/Scripts/python.exe") # Windows venv layout
+ if os.name == "nt"
+ else Path(".venv-mnemosyne/bin/python")
+)
WORKER = Path(__file__).with_name("mnemosyne_worker.py")
diff --git a/eval/bakeoff.py b/src/eval/bakeoff.py
similarity index 100%
rename from eval/bakeoff.py
rename to src/eval/bakeoff.py
diff --git a/eval/baselines.py b/src/eval/baselines.py
similarity index 100%
rename from eval/baselines.py
rename to src/eval/baselines.py
diff --git a/eval/consistency.py b/src/eval/consistency.py
similarity index 100%
rename from eval/consistency.py
rename to src/eval/consistency.py
diff --git a/eval/context_attribution.py b/src/eval/context_attribution.py
similarity index 100%
rename from eval/context_attribution.py
rename to src/eval/context_attribution.py
diff --git a/eval/emotion_flip.py b/src/eval/emotion_flip.py
similarity index 100%
rename from eval/emotion_flip.py
rename to src/eval/emotion_flip.py
diff --git a/eval/experiments.py b/src/eval/experiments.py
similarity index 100%
rename from eval/experiments.py
rename to src/eval/experiments.py
diff --git a/eval/grid.py b/src/eval/grid.py
similarity index 100%
rename from eval/grid.py
rename to src/eval/grid.py
diff --git a/eval/labels/dawn_whitmore.json b/src/eval/labels/dawn_whitmore.json
similarity index 100%
rename from eval/labels/dawn_whitmore.json
rename to src/eval/labels/dawn_whitmore.json
diff --git a/eval/latency.py b/src/eval/latency.py
similarity index 100%
rename from eval/latency.py
rename to src/eval/latency.py
diff --git a/eval/metrics.py b/src/eval/metrics.py
similarity index 100%
rename from eval/metrics.py
rename to src/eval/metrics.py
diff --git a/eval/mnemosyne_worker.py b/src/eval/mnemosyne_worker.py
similarity index 100%
rename from eval/mnemosyne_worker.py
rename to src/eval/mnemosyne_worker.py
diff --git a/eval/poignancy.py b/src/eval/poignancy.py
similarity index 100%
rename from eval/poignancy.py
rename to src/eval/poignancy.py
diff --git a/eval/provenance.py b/src/eval/provenance.py
similarity index 100%
rename from eval/provenance.py
rename to src/eval/provenance.py
diff --git a/src/eval/report/__init__.py b/src/eval/report/__init__.py
new file mode 100644
index 0000000..a069f92
--- /dev/null
+++ b/src/eval/report/__init__.py
@@ -0,0 +1,3 @@
+"""Paper assets from evaluation runs: figures, tables, the results page, the demo pages, the
+release manifest. Every builder reads a run directory or recomputes from the harness; none
+of them takes a number from prose."""
diff --git a/assets/build_animations.py b/src/eval/report/build_animations.py
similarity index 66%
rename from assets/build_animations.py
rename to src/eval/report/build_animations.py
index 4d66aea..16d9ea7 100644
--- a/assets/build_animations.py
+++ b/src/eval/report/build_animations.py
@@ -1,4 +1,4 @@
-"""The animated figure for the README: RQ1's mood-dependent recall, drawn as it happens.
+"""The README's SVG figures: the animated mood-dependent recall, and the static attack loop.
Matplotlib cannot animate, and a GIF is a binary blob nobody can diff, so this emits an SVG
whose only moving parts are CSS keyframes. GitHub renders it inline, a browser plays it, and
@@ -9,8 +9,8 @@
the three mood positions are the pre-registered conditions. Rebuild it from a run directory
and it tells that run's truth:
- python assets/build_animations.py # newest run
- python assets/build_animations.py data/runs/ # a specific one
+ python -m eval.report.build_animations # newest run
+ python -m eval.report.build_animations data/runs/ # a specific one
"""
from __future__ import annotations
@@ -22,9 +22,8 @@
from typing import Sequence
from xml.sax.saxutils import escape
-sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
-from assets.build_figures import ( # noqa: E402
+from eval.report.build_figures import ( # noqa: E402
AMBER,
CREAM,
DEEP_BROWN,
@@ -275,12 +274,118 @@ def group(pattern: tuple[bool, ...], body: str) -> str:
return [target]
+LOOP_WIDTH, LOOP_HEIGHT = 880, 330
+
+
+def build_loop_figure(out_dir: Path | str = DEFAULT_OUT_DIR) -> list[Path]:
+ """Write `self_priming_loop.svg`: the attack timeline, with the run's own numbers on it.
+
+ Five stages left to right, and one arrow back: the mood the write perturbed is the state
+ the scorer then reads. Every number is recomputed from the harness on the stub, which is
+ exact because retrieval and appraisal never call a model. No cached value, no prose.
+ """
+ from eval.attribution import attribute_poisoning, self_priming_alignment
+
+ counts = attribute_poisoning()
+ landed, defended = counts["baseline"]["embr"], counts["embr_minus"]["mood"]
+ alignment = self_priming_alignment()
+ low, high, attacks = min(alignment.values()), max(alignment.values()), len(alignment)
+
+ stages = [
+ ("1 write", "the attacker files one", "memory, with an affect tag"),
+ ("2 appraise", "the turn reads it, and", "the mood follows the tag"),
+ ("3 score", "mood congruence rewards", "the memory that matches"),
+ ("4 retrieve", "the plant makes the top 5", f"on {landed} of {attacks} attacks"),
+ ("5 reply", "it enters the prompt;", "the model answers from it"),
+ ]
+ box_w, box_h, gap, top = 158, 92, 18, 58
+ left = (LOOP_WIDTH - (box_w * len(stages) + gap * (len(stages) - 1))) / 2
+ parts: list[str] = []
+ for index, (title, line1, line2) in enumerate(stages):
+ x = left + index * (box_w + gap)
+ hot = index in (1, 2) # the two stages the loop runs through
+ parts.append(
+ f''
+ f'{escape(title)}'
+ f'{escape(line1)}'
+ f'{escape(line2)}'
+ )
+ if index:
+ parts.append(
+ f''
+ )
+
+ # The return arrow: from the appraised mood back into the scorer, drawn underneath so
+ # the timeline stays a timeline and the loop reads as the exception it is.
+ mood_x = left + 1 * (box_w + gap) + box_w / 2
+ score_x = left + 2 * (box_w + gap) + box_w / 2
+ y0, y1 = top + box_h, top + box_h + 46
+ parts.append(
+ f''
+ f''
+ f"the scorer reads the state the write just moved: cosine {low:.2f} to {high:.2f} "
+ f"on all {attacks} attacks"
+ )
+ # The intervention, on its own line: the one weight whose removal breaks the loop.
+ parts.append(
+ f''
+ f"zero the mood-congruence weight and the loop has nothing to read: "
+ f'{landed}/{attacks} poisoned becomes '
+ f'{defended}/{attacks}'
+ f''
+ f"recomputed from the harness by src/eval/report/build_animations.py; python -m eval.attribution "
+ f"prints the same table"
+ )
+ caption = (
+ f"The self-priming loop: an attacker-written affect tag moves the appraised mood, "
+ f"mood congruence then rewards that same memory, and {landed} of {attacks} injections "
+ f"reach the top 5; zeroing the mood weight leaves {defended}."
+ )
+ style = (
+ "text { font-family: ui-sans-serif, system-ui, -apple-system, Segoe UI, Helvetica, "
+ "Arial, sans-serif; }"
+ f".title {{ font-size: 14px; font-weight: 700; fill: {NEAR_BLACK}; }}"
+ f".body {{ font-size: 11.5px; fill: {NEAR_BLACK}; }}"
+ f".loop {{ font-size: 12px; font-weight: 700; fill: {DEEP_BROWN}; }}"
+ f".pill {{ font-weight: 700; fill: {EMBER_ORANGE}; }}"
+ f".caption {{ font-size: 11px; fill: {DEEP_BROWN}; opacity: 0.9; }}"
+ )
+ svg = (
+ f'"
+ )
+ out_path = Path(out_dir)
+ out_path.mkdir(parents=True, exist_ok=True)
+ target = out_path / "self_priming_loop.svg"
+ target.write_text(svg, encoding="utf-8")
+ return [target]
+
+
def main(argv: Sequence[str] | None = None) -> None:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("run_dir", nargs="?", default=None)
parser.add_argument("--out-dir", default=str(DEFAULT_OUT_DIR))
args = parser.parse_args(argv)
- for path in build_recall_animation(args.run_dir, args.out_dir):
+ for path in build_recall_animation(args.run_dir, args.out_dir) + build_loop_figure(args.out_dir):
print(f" {path}")
diff --git a/assets/build_bakeoff_figures.py b/src/eval/report/build_bakeoff_figures.py
similarity index 96%
rename from assets/build_bakeoff_figures.py
rename to src/eval/report/build_bakeoff_figures.py
index 09d59ba..7e2ba23 100644
--- a/assets/build_bakeoff_figures.py
+++ b/src/eval/report/build_bakeoff_figures.py
@@ -8,8 +8,8 @@
Same rule as the paper figures: the canvas carries data and the labels needed to read it.
Every caveat goes to `results.txt` beside the images.
- python assets/build_bakeoff_figures.py # newest bake-off
- python assets/build_bakeoff_figures.py data/bakeoff/... # a specific one
+ python -m eval.report.build_bakeoff_figures # newest bake-off
+ python -m eval.report.build_bakeoff_figures data/bakeoff/... # a specific one
"""
from __future__ import annotations
@@ -21,10 +21,9 @@
import sys
-# Importable as `assets.build_bakeoff_figures` and runnable as `assets/build_bakeoff_figures.py`.
+# Importable as `eval.report.build_bakeoff_figures` and runnable as `src/eval/report/build_bakeoff_figures.py`.
# Running a file directly puts its own directory on the path rather than the repo root, so the
# sibling import below would fail without this.
-sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
import matplotlib
@@ -33,7 +32,7 @@
from matplotlib.patches import Patch # noqa: E402
from matplotlib.ticker import FuncFormatter # noqa: E402
-from assets.build_figures import ( # noqa: E402
+from eval.report.build_figures import ( # noqa: E402
AMBER,
SYSTEM_LABELS,
CREAM,
@@ -547,6 +546,14 @@ def build_grid_figure(
#: carries data and its notes live in results.txt, and these three had no notes at all, which
#: is how a figure ends up in a slide deck with nothing to stop it being over-read.
EXPERIMENT_NOTES: dict[str, tuple[str, str]] = {
+ "self_priming_loop": (
+ "The self-priming loop, drawn as the timeline it is, with the harness's own numbers",
+ "A diagram, not a measurement plot: the boxes are the pipeline's five steps and the "
+ "return arrow is the state channel. The three numbers on it (poison count at full "
+ "weights, with the mood weight zeroed, and the post-attack mood to tag cosine range) "
+ "are recomputed from eval.attribution on the stub at build time, so the figure "
+ "cannot drift from the table. Reproduce with python -m eval.attribution.",
+ ),
"affective_indexing": (
"Flip a memory's emotion and its recall inverts; what it means does not move",
"Deterministic and model independent: relevance and mood congruence are both pure "
@@ -579,7 +586,7 @@ def build_grid_figure(
"Built from the reported run rather than drawn: every dot sits at a memory's real "
"affect tag and every lit set is the real top 5 for the king-news query. Animated "
"with SMIL because Blink does not run CSS animations inside an img tag, which is how "
- "GitHub embeds an SVG. Reproduce with python assets/build_animations.py.",
+ "GitHub embeds an SVG. Reproduce with python -m eval.report.build_animations.",
),
}
@@ -605,9 +612,10 @@ def build_experiment_figures(out_dir: Path | str = DEFAULT_OUT_DIR) -> list[Path
one command rebuilds the whole figure set: a paper with half its assets regenerated from
a stale cache is the failure mode this exists to prevent.
"""
- from assets.build_animations import build_recall_animation
+ from eval.report.build_animations import build_loop_figure, build_recall_animation
written = list(build_affective_indexing_figure(out_dir))
+ written += list(build_loop_figure(out_dir))
try:
written += list(build_recall_animation(out_dir=out_dir))
except FileNotFoundError as error: # the animation needs a run; the rest do not
diff --git a/assets/build_demo.py b/src/eval/report/build_demo.py
similarity index 96%
rename from assets/build_demo.py
rename to src/eval/report/build_demo.py
index e2b994a..b5222c9 100644
--- a/assets/build_demo.py
+++ b/src/eval/report/build_demo.py
@@ -17,8 +17,8 @@
rankings computed by the real Python scorer, and the page recomputes them on load and says
so on screen. If the two ever disagree, the page says that instead.
- python assets/build_demo.py # newest run
- python assets/build_demo.py data/runs/ # a specific one
+ python -m eval.report.build_demo # newest run
+ python -m eval.report.build_demo data/runs/ # a specific one
"""
from __future__ import annotations
@@ -31,18 +31,17 @@
from pathlib import Path
from typing import Sequence
-sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
from embr import CharacterState, Memory, Mood # noqa: E402
from embr.scoring import Recency, Relevance # noqa: E402
DEFAULT_OUT_DIR = Path("data/demo")
-TEMPLATE = Path(__file__).with_name("demo") / "template.html"
-TEMPLATE_3D = Path(__file__).with_name("demo") / "brain3d.html"
+TEMPLATE = Path(__file__).resolve().parents[3] / "assets" / "demo" / "template.html"
+TEMPLATE_3D = Path(__file__).resolve().parents[3] / "assets" / "demo" / "brain3d.html"
#: three.js, vendored rather than linked. See `assets/vendor/README.md` for the version, the
#: hash, and why the current release cannot be used.
-VENDORED_THREE = Path(__file__).with_name("vendor") / "three.min.js"
+VENDORED_THREE = Path(__file__).resolve().parents[3] / "assets" / "vendor" / "three.min.js"
THREE_MARKER = "/*THREE_JS*/"
#: The weight maps the preset buttons offer. Every one of these is a real arm of the study,
@@ -135,7 +134,7 @@ def build_demo(
asked, which is the one thing the flat plane cannot show, and pays for it with a vendored
renderer and a WebGL requirement.
"""
- from assets.build_figures import latest_run_dir, load_run_results
+ from eval.report.build_figures import latest_run_dir, load_run_results
from eval.attacks import ATTACKS, PROBE_QUESTION, build_attack_memory, tag_variants
from eval.poignancy import CACHE_DIR, cached_ratings, is_ratings_cache
from eval.run import (
diff --git a/assets/build_figures.py b/src/eval/report/build_figures.py
similarity index 99%
rename from assets/build_figures.py
rename to src/eval/report/build_figures.py
index c2f9dca..c3c64f2 100644
--- a/assets/build_figures.py
+++ b/src/eval/report/build_figures.py
@@ -1,6 +1,6 @@
"""Paper ready figures for the EMBR evaluation, built from one run directory.
-Phase 3 reads `data/runs//results.json` and nothing else (docs/phase2.md section
+Phase 3 reads `data/runs//results.json` and nothing else (docs/history/phase2.md section
6), so every figure here is a pure function of one run directory plus this file. Each
figure emits three files: a `.pdf` for the paper, a `.png` for the README, and a
`.svg` for the results page, which embeds them inline.
@@ -22,9 +22,9 @@
Usage:
- from assets.build_figures import build_all_figures
+ from eval.report.build_figures import build_all_figures
build_all_figures("data/runs/20260817-160950") # writes data/figures/
- python -m assets.build_figures # newest run, same output
+ python -m eval.report.build_figures # newest run, same output
"""
from __future__ import annotations
@@ -247,7 +247,7 @@ def figure_footer_text(results: Mapping[str, object], run_stamp: str) -> str:
f"run {run_stamp} | commit {commit}{dirty_suffix} | "
f"model {metadata.get('model')} | "
f"labels {metadata.get('label_set')} {metadata.get('label_version')} | "
- f"built by assets/build_figures.py"
+ f"built by src/eval/report/build_figures.py"
)
return f"{provenance}\n{preliminary_warning(results)}"
@@ -1484,7 +1484,7 @@ def build_all_figures(
def main(argv: Sequence[str] | None = None) -> None:
- """Command line entry point: `python -m assets.build_figures [run_dir] [out_dir]`."""
+ """Command line entry point: `python -m eval.report.build_figures [run_dir] [out_dir]`."""
parser = argparse.ArgumentParser(description="Build the EMBR paper figures.")
parser.add_argument(
"run_dir",
diff --git a/src/eval/report/build_manifest.py b/src/eval/report/build_manifest.py
new file mode 100644
index 0000000..a22689b
--- /dev/null
+++ b/src/eval/report/build_manifest.py
@@ -0,0 +1,115 @@
+"""Generate data/release-manifest.json: the one place every claim about status reads from.
+
+The README, the dashboard, and the paper must not hand-maintain test counts or run
+stamps. They either quote this file or they are stale. Run it, commit the output:
+
+ python -m eval.report.build_manifest
+
+Exit 1 if the test suite is not fully green, because a manifest that records a
+failing suite is fiction. Skip with --allow-failures only when the failures are
+the known artifact-dependent layer (data/runs absent on a fresh clone).
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import platform
+import subprocess
+import sys
+from datetime import datetime, timezone
+from pathlib import Path
+
+REPO_ROOT = Path(__file__).resolve().parents[3]
+MANIFEST_PATH = REPO_ROOT / "data" / "release-manifest.json"
+
+
+def _git(*args: str) -> str:
+ return subprocess.run(
+ ["git", *args], cwd=REPO_ROOT, capture_output=True, text=True, check=True
+ ).stdout.strip()
+
+
+def _run_tests() -> dict:
+ """Run the suite through pytest's JSON report; never hand-count from prose."""
+ report = REPO_ROOT / ".pytest-manifest-report.json"
+ out = subprocess.run(
+ [sys.executable, "-m", "pytest", "-q", "--json-report",
+ f"--json-report-file={report}", "-p", "no:cacheprovider"],
+ cwd=REPO_ROOT, capture_output=True, text=True, timeout=600,
+ )
+ # ponytail: pytest-json-report needed; if absent, parse summary line instead
+ if report.is_file():
+ data = json.loads(report.read_text(encoding="utf-8"))
+ report.unlink()
+ summary = data["summary"]
+ return {
+ "passed": summary.get("passed", 0),
+ "failed": summary.get("failed", 0),
+ "skipped": summary.get("skipped", 0),
+ "errors": summary.get("error", 0),
+ "collected": summary.get("total", 0),
+ }
+ # last line looks like: 471 passed, 2 failed, 43 skipped, 23 errors in 143.18s
+ tail = out.stdout.strip().splitlines()[-1] if out.stdout.strip() else ""
+ counts = {}
+ for part in tail.split(","):
+ for key in ("passed", "failed", "skipped", "error"):
+ if key in part:
+ counts[key.rstrip("s")] = int(part.split()[0])
+ return {"collected": 0, **counts}
+
+
+def build_manifest(test_results: dict) -> dict:
+ runs_dir = REPO_ROOT / "data" / "runs"
+ run_stamps = sorted(p.name for p in runs_dir.iterdir() if p.is_dir()) if runs_dir.is_dir() else []
+ return {
+ "generated_at": datetime.now(timezone.utc).isoformat(timespec="seconds"),
+ "release_commit": _git("rev-parse", "HEAD"),
+ "release_branch": _git("rev-parse", "--abbrev-ref", "HEAD"),
+ "working_tree_clean": not _git("status", "--porcelain"),
+ "python": sys.version.split()[0],
+ "platform": platform.platform(),
+ "test_command": "pytest -q",
+ "tests": test_results,
+ "runs_on_disk": run_stamps,
+ "primary_eval_run": run_stamps[-1] if run_stamps else None,
+ "known_limitations": [
+ "single NPC, single authored scenario (dawn_whitmore.json)",
+ "single-author v1 attack labels",
+ "behavioural attribution panel agreement below the preregistered floor;"
+ " likelihood arm only; H3 withdrawn",
+ "RQ3 Park-vs-EMBR ordering is label-sensitive and null (p=0.69); not reportable directionally",
+ "small controlled attack corpus; a mechanism case study, not a general poisoning benchmark",
+ "Ouro 1.4B poignancy rater saturated at 10/10 on 27 of 34 ratings",
+ ],
+ }
+
+
+def main(argv: list[str] | None = None) -> int:
+ parser = argparse.ArgumentParser(description=(__doc__ or "").splitlines()[0])
+ parser.add_argument(
+ "--allow-failures",
+ action="store_true",
+ help="record a non-green suite anyway (for the artifact-dependent layer)",
+ )
+ args = parser.parse_args(argv)
+
+ results = _run_tests()
+ manifest = build_manifest(results)
+ MANIFEST_PATH.parent.mkdir(parents=True, exist_ok=True)
+ MANIFEST_PATH.write_text(json.dumps(manifest, indent=2) + "\n", encoding="utf-8")
+
+ t = results
+ print(f"{MANIFEST_PATH.relative_to(REPO_ROOT)}: "
+ f"{t.get('passed', 0)} passed, {t.get('failed', 0)} failed, "
+ f"{t.get('errors', 0)} errors, {t.get('skipped', 0)} skipped @ {manifest['release_commit'][:8]}")
+
+ if (t.get("failed") or t.get("errors")) and not args.allow_failures:
+ print("suite is not green; rerun with --allow-failures to record it anyway")
+ return 1
+ return 0
+
+
+if __name__ == "__main__":
+ sys.exit(main())
diff --git a/assets/build_questline.py b/src/eval/report/build_questline.py
similarity index 99%
rename from assets/build_questline.py
rename to src/eval/report/build_questline.py
index 9abde3b..ed1c5b3 100644
--- a/assets/build_questline.py
+++ b/src/eval/report/build_questline.py
@@ -24,7 +24,7 @@
import matplotlib.pyplot as plt
-from assets.build_figures import (
+from eval.report.build_figures import (
AMBER_LIGHT,
CREAM,
DEEP_BROWN,
diff --git a/assets/build_results.py b/src/eval/report/build_results.py
similarity index 97%
rename from assets/build_results.py
rename to src/eval/report/build_results.py
index 12cae73..21dccda 100644
--- a/assets/build_results.py
+++ b/src/eval/report/build_results.py
@@ -17,8 +17,8 @@
from analyses that write no run artefact (`eval.grid`, `eval.attribution`, `eval.agreement`),
and those numbers are marked as unchecked wherever they appear.
- python assets/build_results.py # newest run
- python assets/build_results.py data/runs/ # a specific one
+ python -m eval.report.build_results # newest run
+ python -m eval.report.build_results data/runs/ # a specific one
"""
from __future__ import annotations
@@ -34,10 +34,9 @@
from pathlib import Path
from typing import Callable, Sequence
-sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
DEFAULT_OUT_DIR = Path("data/demo")
-TEMPLATE = Path(__file__).with_name("results") / "template.html"
+TEMPLATE = Path(__file__).resolve().parents[3] / "assets" / "results" / "template.html"
FIGURES_DIR = Path("data/figures")
FINDINGS = Path("docs/findings.md")
@@ -182,7 +181,7 @@ def inline_figure(stem: str, figures_dir: Path = FIGURES_DIR) -> str:
return (
f'
The figure {html.escape(stem)}.svg has not '
f"been built. Run option 11 in the menu, or "
- f"python -m assets.build_figures.
- 
- 
+ 
+ 
@@ -17,28 +17,44 @@
-
+
