ManSio · ManSio · Sep 27, 2026 · Sep 26, 2026 · Sep 26, 2026 · Sep 26, 2026
diff --git a/AGENT_DIARY.md b/AGENT_DIARY.md
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -5,6 +5,17 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [Unreleased] - 2026-09-27
+
+### Added
+- **F5 4-arm judged run** (4A unit-of-return): 16 frozen queries (8 code + 8 prose), 10 trials/arm, reader `opencode-go/longcat-2.0`, judge `opencode-go/qwen3.7-plus` (blind, disjoint from reader).
+- Overall correct: A top-k chunks 16.3% (26/160), B top-1 full-doc 34.4% (55/160), C oracle 97.5% (156/160), D closed-book 0%.
+- Code split: B 50.0% (40/80) vs A 6.3% (5/80), non-overlapping CIs; unit of return affects the reader, not gold-file hit.
+- Prose split: A 26.3% (21/80) vs B 18.8% (15/80), overlapping CIs — fragile, no claim.
+- Majority (strict >50% per query-arm): B 5/16 (F5S-13/B 5/10 tie counted out); guard: 0 `invalid` in trials=10 run (1 in t5 pilot, F5S-03/B).
+- NodeRAG refuted on the same bench: TF-IDF baseline 80% vs graph BFS 70% — graph adds cost without gain here.
+- Caveats: n=16 pilot scale; index snapshot not hard-frozen; raw answers path-normalized before commit, verdicts unchanged.
+
 ## [3.5.0] - 2026-09-22
 
 ### Added

diff --git a/EXPERIMENTS_LOG.md b/EXPERIMENTS_LOG.md
diff --git a/KNOWN_ISSUES.md b/KNOWN_ISSUES.md
diff --git a/docs/archive/KNOWN_ISSUES_2026_09.md b/docs/archive/KNOWN_ISSUES_2026_09.md
diff --git a/docs/en/CHANGELOG.md b/docs/en/CHANGELOG.md
@@ -5,6 +5,17 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [Unreleased] - 2026-09-27
+
+### Added
+- **F5 4-arm judged run** (4A unit-of-return): 16 frozen queries (8 code + 8 prose), 10 trials/arm, reader `opencode-go/longcat-2.0`, judge `opencode-go/qwen3.7-plus` (blind, disjoint from reader).
+- Overall correct: A top-k chunks 16.3% (26/160), B top-1 full-doc 34.4% (55/160), C oracle 97.5% (156/160), D closed-book 0%.
+- Code split: B 50.0% (40/80) vs A 6.3% (5/80), non-overlapping CIs; unit of return affects the reader, not gold-file hit.
+- Prose split: A 26.3% (21/80) vs B 18.8% (15/80), overlapping CIs — fragile, no claim.
+- Majority (strict >50% per query-arm): B 5/16 (F5S-13/B 5/10 tie counted out); guard: 0 `invalid` in trials=10 run (1 in t5 pilot, F5S-03/B).
+- NodeRAG refuted on the same bench: TF-IDF baseline 80% vs graph BFS 70% — graph adds cost without gain here.
+- Caveats: n=16 pilot scale; index snapshot not hard-frozen; raw answers path-normalized before commit, verdicts unchanged.
+
 ## [3.5.0] - 2026-09-22
 
 ### Added

diff --git a/docs/ru/CHANGELOG.md b/docs/ru/CHANGELOG.md
@@ -5,6 +5,17 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [Unreleased] - 2026-09-27
+
+### Added
+- **F5 4-arm judged run** (4A unit-of-return): 16 frozen queries (8 code + 8 prose), 10 trials/arm, reader `opencode-go/longcat-2.0`, judge `opencode-go/qwen3.7-plus` (blind, disjoint from reader).
+- Overall correct: A top-k chunks 16.3% (26/160), B top-1 full-doc 34.4% (55/160), C oracle 97.5% (156/160), D closed-book 0%.
+- Code split: B 50.0% (40/80) vs A 6.3% (5/80), non-overlapping CIs; unit of return affects the reader, not gold-file hit.
+- Prose split: A 26.3% (21/80) vs B 18.8% (15/80), overlapping CIs — fragile, no claim.
+- Majority (strict >50% per query-arm): B 5/16 (F5S-13/B 5/10 tie counted out); guard: 0 `invalid` in trials=10 run (1 in t5 pilot, F5S-03/B).
+- NodeRAG refuted on the same bench: TF-IDF baseline 80% vs graph BFS 70% — graph adds cost without gain here.
+- Caveats: n=16 pilot scale; index snapshot not hard-frozen; raw answers path-normalized before commit, verdicts unchanged.
+
 ## [3.5.0] - 2026-09-22
 
 ### Added

diff --git a/docs/zh/CHANGELOG.md b/docs/zh/CHANGELOG.md
@@ -5,6 +5,17 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [Unreleased] - 2026-09-27
+
+### Added
+- **F5 4-arm judged run** (4A unit-of-return): 16 frozen queries (8 code + 8 prose), 10 trials/arm, reader `opencode-go/longcat-2.0`, judge `opencode-go/qwen3.7-plus` (blind, disjoint from reader).
+- Overall correct: A top-k chunks 16.3% (26/160), B top-1 full-doc 34.4% (55/160), C oracle 97.5% (156/160), D closed-book 0%.
+- Code split: B 50.0% (40/80) vs A 6.3% (5/80), non-overlapping CIs; unit of return affects the reader, not gold-file hit.
+- Prose split: A 26.3% (21/80) vs B 18.8% (15/80), overlapping CIs — fragile, no claim.
+- Majority (strict >50% per query-arm): B 5/16 (F5S-13/B 5/10 tie counted out); guard: 0 `invalid` in trials=10 run (1 in t5 pilot, F5S-03/B).
+- NodeRAG refuted on the same bench: TF-IDF baseline 80% vs graph BFS 70% — graph adds cost without gain here.
+- Caveats: n=16 pilot scale; index snapshot not hard-frozen; raw answers path-normalized before commit, verdicts unchanged.
+
 ## [3.5.0] - 2026-09-22
 
 ### Added

diff --git a/experiments/1V_memory_contamination/burst_rename_audit.py b/experiments/1V_memory_contamination/burst_rename_audit.py
@@ -32,7 +32,7 @@
     sys.stdout.reconfigure(encoding="utf-8")
 
 ROOT = Path(r"D:\Project\MSCodeBase")
-MEM = Path(r"C:\Users\misha\AppData\Local\mscodebase\projects\bfe9644b\intelligence\project_memory.json")
+MEM = Path(r"<user>AppData\Local\mscodebase\projects\bfe9644b\intelligence\project_memory.json")
 
 _CREATE_NO_WINDOW = 0x08000000 if sys.platform == "win32" else 0
 

diff --git a/experiments/1V_memory_contamination/memory_contamination_results.json b/experiments/1V_memory_contamination/memory_contamination_results.json
@@ -4,8 +4,8 @@
     "date": "2026-08-11",
     "scan_ms_total": 91.4,
     "isolation": {
-      "store_dir": "C:\\Users\\misha\\AppData\\Local\\mscodebase\\projects\\e8d56bb9\\intelligence",
-      "real_project_store_dir": "C:\\Users\\misha\\AppData\\Local\\mscodebase\\projects\\bfe9644b\\intelligence",
+      "store_dir": "<user>AppData\\Local\\mscodebase\\projects\\e8d56bb9\\intelligence",
+      "real_project_store_dir": "<user>AppData\\Local\\mscodebase\\projects\\bfe9644b\\intelligence",
       "isolated": true
     }
   },

diff --git a/experiments/1V_memory_contamination/memory_contamination_results_v2.json b/experiments/1V_memory_contamination/memory_contamination_results_v2.json
@@ -4,8 +4,8 @@
     "date": "2026-08-11",
     "scan_ms_total": 86.4,
     "isolation": {
-      "store_dir": "C:\\Users\\misha\\AppData\\Local\\mscodebase\\projects\\f3bbb8a9\\intelligence",
-      "real_project_store_dir": "C:\\Users\\misha\\AppData\\Local\\mscodebase\\projects\\bfe9644b\\intelligence",
+      "store_dir": "<user>AppData\\Local\\mscodebase\\projects\\f3bbb8a9\\intelligence",
+      "real_project_store_dir": "<user>AppData\\Local\\mscodebase\\projects\\bfe9644b\\intelligence",
       "isolated": true
     }
   },

diff --git a/experiments/1V_memory_contamination/memory_contamination_results_v3_generated.json b/experiments/1V_memory_contamination/memory_contamination_results_v3_generated.json
@@ -6,8 +6,8 @@
     "replication": false,
     "control_group": "v3 (add-only) / 1-R (retraction)",
     "isolation": {
-      "store_dir": "C:\\Users\\misha\\AppData\\Local\\mscodebase\\projects\\ace00824\\intelligence",
-      "real": "C:\\Users\\misha\\AppData\\Local\\mscodebase\\projects\\bfe9644b\\intelligence",
+      "store_dir": "<user>AppData\\Local\\mscodebase\\projects\\ace00824\\intelligence",
+      "real": "<user>AppData\\Local\\mscodebase\\projects\\bfe9644b\\intelligence",
       "isolated": true
     },
     "verify_stats": {

diff --git a/experiments/1V_memory_contamination/memory_contamination_results_v3_retraction.json b/experiments/1V_memory_contamination/memory_contamination_results_v3_retraction.json
@@ -5,8 +5,8 @@
     "control_group": "v3 (memory_contamination_facts_v3_generated.json, same 50 facts)",
     "parity_with_v3_A_code_first_adoption": "OK",
     "isolation": {
-      "store_dir": "C:\\Users\\misha\\AppData\\Local\\mscodebase\\projects\\107b52a6\\intelligence",
-      "real_project_store_dir": "C:\\Users\\misha\\AppData\\Local\\mscodebase\\projects\\bfe9644b\\intelligence",
+      "store_dir": "<user>AppData\\Local\\mscodebase\\projects\\107b52a6\\intelligence",
+      "real_project_store_dir": "<user>AppData\\Local\\mscodebase\\projects\\bfe9644b\\intelligence",
       "isolated": true
     },
     "validation": {

diff --git a/experiments/1V_memory_contamination/memory_contamination_results_v3_verify.json b/experiments/1V_memory_contamination/memory_contamination_results_v3_verify.json
@@ -4,8 +4,8 @@
     "date": "2026-08-11",
     "control_group": "v3 (add-only) / 1-R (retraction)",
     "isolation": {
-      "store_dir": "C:\\Users\\misha\\AppData\\Local\\mscodebase\\projects\\5517d5cd\\intelligence",
-      "real": "C:\\Users\\misha\\AppData\\Local\\mscodebase\\projects\\bfe9644b\\intelligence",
+      "store_dir": "<user>AppData\\Local\\mscodebase\\projects\\5517d5cd\\intelligence",
+      "real": "<user>AppData\\Local\\mscodebase\\projects\\bfe9644b\\intelligence",
       "isolated": true
     },
     "verify_stats": {

diff --git a/experiments/1V_memory_contamination/memory_contamination_results_v4_rep.json b/experiments/1V_memory_contamination/memory_contamination_results_v4_rep.json
@@ -6,8 +6,8 @@
     "replication": true,
     "control_group": "1-V (facts v3)",
     "isolation": {
-      "store_dir": "C:\\Users\\misha\\AppData\\Local\\mscodebase\\projects\\ab604b45\\intelligence",
-      "real": "C:\\Users\\misha\\AppData\\Local\\mscodebase\\projects\\bfe9644b\\intelligence",
+      "store_dir": "<user>AppData\\Local\\mscodebase\\projects\\ab604b45\\intelligence",
+      "real": "<user>AppData\\Local\\mscodebase\\projects\\bfe9644b\\intelligence",
       "isolated": true
     },
     "verify_stats": {

diff --git a/experiments/1V_memory_contamination/redteam_burst.py b/experiments/1V_memory_contamination/redteam_burst.py
@@ -34,7 +34,7 @@
     sys.stdout.reconfigure(encoding="utf-8")
 
 ROOT = Path(r"D:\Project\MSCodeBase")
-MEM = Path(r"C:\Users\misha\AppData\Local\mscodebase\projects\bfe9644b\intelligence\project_memory.json")
+MEM = Path(r"<user>AppData\Local\mscodebase\projects\bfe9644b\intelligence\project_memory.json")
 _CREATE_NO_WINDOW = 0x08000000 if sys.platform == "win32" else 0
 
 

diff --git a/experiments/4A_unit_of_return/BASELINE.md b/experiments/4A_unit_of_return/BASELINE.md
@@ -0,0 +1,51 @@
+# 4A — BASELINE (freeze template + system snapshot)
+
+> **Правило:** НИ ОДИН прогон 4A не запускается без заполненного раздела **Freeze** ниже.
+> «После» — только **дописывается** (`## Run N`), никогда не перезаписывается (Tom: удаление =
+> тюнинг под уже увиденное). Каждое число в отчёте несёт referent (популяция / corpus / candidate
+> set / n / judge / noise).
+>
+> **Frozen inputs живут ТОЛЬКО в репо:** `experiments/4A_unit_of_return/frozen/` (git-tracked).
+> Хранить замороженный список в `%TEMP%`/`/tmp` **запрещено** — прецедент 2026-09-26: список E7/E11
+> лежал в `%TEMP%/opencode/e11` и был удалён, verbatim-регрессия стала невозможной. Guard:
+> `tests/test_frozen_inputs_tracked.py`.
+
+## System baseline — 2026-09-26 (MSCodeBase)
+
+| Поле | Значение |
+|---|---|
+| Branch | `chore/privacy-paths` (PR #49) → база для эксперимента — до мерджа ветвиться от неё |
+| HEAD | `5f158ece` (после PR #49) |
+| MCP RUN_ID | `45b0a471f48e` (PID 15624, state READY) |
+| Index | **10 106 chunks / 716 files / 14 099 symbols** |
+| Embedder / Reranker | llama.cpp 🟢 / BGE-M3 🟢 |
+| Tests | `pytest tests/`: **1866 passed, 5 skipped, 0 failed** (313s) |
+| Privacy guard | `tests/test_no_personal_paths.py` ✅ |
+
+## Freeze (заполнить ДО прогона, hash-фиксация)
+
+| Поле | Значение |
+|---|---|
+| Experiment | 4A unit-of-return |
+| git HEAD | `<sha>` |
+| Index stats до | `chunks=___ files=___ symbols=___` |
+| queries file | `<path>` sha256=`<hash>` |
+| populations file | `<path>` sha256=`<hash>` |
+| retriever / top-k | `<заморожено>` |
+| judge model + budget | `<модель> / <reasoning>` |
+| n per arm | `___` |
+| noise floor | измеренный повторным прогоном: `___` |
+| **Prediction (Tom):** | проза: whole-doc ≤ closed book; код: нет. Если обе популяции движутся одинаково → двухпопуляционная история неверна на нашем индексе. |
+
+## Run 1 — `<дата>` (append, не редактировать прошлое)
+
+| Arm | reader gets | hit@1 | hit@3 | top-1-doc-is-gold | context tokens | referent |
+|---|---|---|---|---|---|---|
+| A | top-k chunks | | | | | |
+| B | top-1 whole document | | | | | |
+| C | oracle file (chunked) | | | | | |
+| D | nothing (closed book) | | | | | |
+
+- Controls passed: `__/__`
+- Verdict: `<…>` (raw output referenced)
+- Index stats after: `chunks=___ files=___ symbols=___` (must equal "до", иначе прогон невалиден)