Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,17 @@

### Changed

- The `inferrsi.observability` package is dissolved into its owning packages. No forwarding
modules remain, so imports must use the new paths:
- `campaign_records`, `campaign_trace`, `optimization_log`, `optimization_trace`,
`profile_timeline` and `timing_clock` are under `inferrsi.harness.records`.
- `dashboard`, `dashboard_data`, `dashboard_external`, `dashboard_live`,
`dashboard_timeline`, `dashboard_workflow`, `experiment_timing`, `optimization_chart`,
`optimization_story` and both HTML templates are under `inferrsi.apps.views`.
- `pre_profile_searches` from `profile_timeline` is in
`inferrsi.apps.views.profile_searches`.
- `capture_context` and `verify_capture` from `knowledge_capture`, and `create_campaign`
with backend validation, are in `inferrsi.apps.campaign`.
- The old `run` chain is split by owner. The CLI, its exit codes 0/1/2 and the
`evaluation.json` bytes are unchanged. Root `inferrsi.evaluation`,
`inferrsi.execution` and `inferrsi.reporting` keep working as forwarders.
Expand Down
47 changes: 31 additions & 16 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ budget, attempts, feedback records, selection and independent final execution.
`harness/task_protocol.py` freezes a supplied resource directory and loads its
task-owned assessment protocol. `harness/optimization_session.py` assembles
execution and assessment; the CLI resolves task resources and injects knowledge
access and the existing Observability capture callback.
access and the capture callback from `apps/campaign.py`.
The external Agent chooses and edits candidates between round submissions.
The same controller serves ordinary projects through `campaign start/round/final`
and packaged tasks through `benchmarks session`. The
Expand Down Expand Up @@ -179,28 +179,43 @@ src/inferrsi/
claim.py Campaign verdict labels PASS/FAIL/INCONCLUSIVE; rules only
evidence_paths.py Per-campaign evidence tree: plan/results/patch/validator
summary_chart.py Stdlib SVG gain chart (chain/branch) and report figure spec
observability/
optimization_chart.py Managed optimization objective/token curves and final result
dashboard.py Offline snapshot and HTML rendering
dashboard_data.py Recorded run/campaign evidence into a view model
dashboard_live.py Read-only loopback server for live observation
dashboard_timeline.py Stage/result timeline SVG
dashboard_workflow.py Paired command events into reviewable work units
dashboard_external.py Protocol-specific external measurements
template.html Dashboard view template
optimization_log.py Campaign log write/read and reference checks
optimization_story.py Caller narrative resolved against captures and runs
optimization_trace.py Optimization trace parse, assemble and read
knowledge_capture.py Bounded knowledge context capture and verification
campaign_records.py Campaign manifest, candidate and decision records
campaign_trace.py Correlation envelope for existing agent events
apps/
campaign.py Knowledge values connected to Harness campaign records
views/
optimization_chart.py Managed optimization objective/token curves and final result
dashboard.py Offline snapshot and HTML rendering
dashboard_data.py Recorded run/campaign evidence into a view model
dashboard_live.py Read-only loopback server for live observation
dashboard_timeline.py Stage/result timeline SVG
dashboard_workflow.py Paired command events into reviewable work units
dashboard_external.py Protocol-specific external measurements
template.html Dashboard view template
optimization_story.py Caller narrative resolved against captures and runs
profile_searches.py Pre-profile search presentation
experiment_timing.py Experiment timing presentation
search_attribution.py Import attributable file-search evidence into a campaign
search_archive.py Reviewed search-archive uses
capture_delivery.py Visible capture resources in a reviewed tool return
codex_search.py Bounded attributable file-search evidence from Codex rollouts
codex_tool_returns.py Correlate Codex tool-return windows
knowledge/
context_projection.py Collected context text and item projection
harness/
api.py H public allocate/execute/read over one RunStore and CPU target
execution_service.py Ordinary-command execute: admission, binding, steps, record
input_binding.py Input capture, byte-equivalent copies and file digests
local_process.py Windows Job / POSIX process group; deadline, cancel, cleanup
run_store.py Local SQLite RunStore: claim, owner lock, fixed records
errors.py H PublicError codes and partial execution facts
records/
evidence.py Bounded evidence reads and record text rules
optimization_log.py Campaign log write/read and reference checks
optimization_trace.py Optimization trace parse, assemble and read
knowledge_capture.py Delivered context capture and byte verification
campaign_records.py Campaign manifest, candidate and decision records
campaign_trace.py Correlation envelope for existing agent events
profile_timeline.py Profile attachment records
timing_clock.py Recorded timing clock
optimization_session.py Shared start/knowledge/round/final operation assembly
project.py Freeze supplied project checks and assess retained results
task_protocol.py Freeze supplied task resources and load their protocol
Expand Down
3 changes: 3 additions & 0 deletions docs/contracts/function-owners.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,9 @@ Status: **Proposed baseline**; individually **adopted** entries are identified b
| `harness/scenarios/external` `read_metric`/`validate_metric` | B semantic validation; `execute_command` and retention → H | no whole-file ownership; offline assess executes nothing |
| `campaign_records` `STAGES`/claim capture/reconciliation | H records; dashboard/report presentation → Apps | first remove the `campaign_records → dashboard_data` reverse dependency; #333 is Proposed and no claim is made that a unified event envelope is accepted |
| `catalog.repository_pack`/`REPO_TYPES` | D020 neutral raw-data technical leaf; target interpretation stays H | K does not expose the pack reader publicly; `infra` changes only its same-leaf import; seven K bodies/root entries/tests move together |
| `apps/views/dashboard_workflow` `command_state`/`run_workflow`/`validation_progress` (added with the OBS dissolve; **adopted owner: Apps views; RSI-master architecture decision 2026-11-14 Asia/Shanghai, recorded in the [D008 wave-2 ownership decision](d008-wave2-ownership-decision.md#dashboard-workflow); independent PR review remains separate**) | Apps views: read-only presentation projection of workflow and command state | The module imports only the standard library and reads records the caller already loaded; it has no H, K or Benchmark edge. A group `state` such as `completed` is a display state for the run workflow, not a verdict, and must not be read as the legacy `Verdict`. `validation_progress` only validates counters from a snapshot `dashboard_data` has already read; it opens no file. |
| `harness/records/profile_timeline` `attach_profile`, `_compile` (builds the bounded dashboard overview stored with the record), fixed-byte verification, and digest reads (`read_profiles`/`profile_event_table`) (added with the OBS dissolve; **adopted owner: H records; RSI-master architecture decision 2026-11-14 Asia/Shanghai, recorded in the [D008 wave-2 ownership decision](d008-wave2-ownership-decision.md#profile-records-and-overview); independent PR review remains separate**) | H records: attach a reviewed profile, compile the stored overview from those bytes, and verify stored bytes | Record, overview-compile and byte-integrity paths stay in H because the overview is record content: it is computed once at attach from a bounded trace and published content-addressed, so the stored bytes cannot depend on who renders them. Its top-lane, longest-event and top-kernel selection reduces one trace for a bounded record; it ranks no candidate, run or claim and produces no verdict. UI route decoration, including `events_url`, belongs to Apps views, not H. Issue #413's broad timeline-to-Apps placement is refined here: persisted factual records and bounded overview encoding stay in H; UI projections and route decoration belong to Apps. The route decoration is removed from H in this PR, preserving stored bytes, digest verification and the external view response. |
| `apps/views/profile_searches` `pre_profile_searches` (added with the OBS dissolve; **adopted owner: Apps views; RSI-master architecture decision 2026-11-14 Asia/Shanghai, recorded in the [D008 wave-2 ownership decision](d008-wave2-ownership-decision.md#pre-profile-searches); independent PR review remains separate**) | Apps views: Dashboard-consumed presentation list relating a profile capture to named searches completed before it | It joins an H record with ledger read models, which only Apps may assemble. Its `verified`/`not_before`/`unknown_time` values are display annotations over an existing search state; they are not evidence for a claim, and using them that way needs a new owner decision. |
| `benchmark/sampling` `collect_samples` (moved out of `harness/scenarios/media` by the old-run split; **ownership disposition: B sampling; RSI-master [adopted decision, 2026-11-14 Asia/Shanghai](d008-wave2-ownership-decision.md#benchmark-sampling); independent PR review remains separate**) | B sampling: seed ordering, timing records and artifact hashes | Request transport stays with the caller's injected `request(item, path)` callback; Benchmark hashes artifacts with its own `_sha256`, pinned to `infra.artifacts.sha256` by `tests/test_legacy_run_split.py::test_sampling_sha256_matches_the_infra_hasher`, and imports no H or `infra` module. The per-group progress line on stdout is the old `run` protocol output kept for D022 and is a protocol constraint, not a presentation owner. |
| `harness/runner` `snapshot` port `(repo, exclude) -> dict` (separated from the runner by the old-run split; **ownership disposition: H port contract, Apps legacy binding, `infra/targets` implementation; RSI-master [adopted decision, 2026-11-14 Asia/Shanghai](d008-wave2-ownership-decision.md#snapshot-port); independent PR review remains separate**) | H owns the port and its contract; the Apps legacy composition owns the binding; `infra/targets` owns the Git identity implementation | H keeps the port required and keyword-only with no default and holds no copy of the implementation. `apps/legacy/run.py` binds `snapshot=repository_identity` for the old run and the root `execution.py` forwards it. A change of the binding owner needs its own review. Out-of-tree callers that used the removed H defaults are not covered by this row. |

Expand Down
2 changes: 1 addition & 1 deletion docs/dashboard.md
Original file line number Diff line number Diff line change
Expand Up @@ -767,7 +767,7 @@ may include a `decision` object with `candidate_id` and `outcome` (including
Agent trace JSONL can contain the existing `agent.*` event types and the
optional `stage` field (`diagnosis`, `knowledge`, `search`, `execution`,
`validation`, or `review`). An agent integration can pass an opt-in
`inferrsi.observability.campaign_trace.CampaignTrace` instance as its existing event
`inferrsi.harness.records.campaign_trace.CampaignTrace` instance as its existing event
callback; it appends the original event with time and caller-supplied IDs. The
caller associates the completed trace with `agent_trace_ref` using its SHA-256.
The dashboard links a pre-run event when its campaign and uniquely identified
Expand Down
2 changes: 1 addition & 1 deletion docs/knowledge-retrieval-trace.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ same immutable capture envelope used by `capture-context`:
```python
from pathlib import Path
from inferrsi.knowledge.catalog import search_knowledge
from inferrsi.observability.knowledge_capture import save_retrieval
from inferrsi.apps.campaign import save_retrieval

result = search_knowledge("decode launch", explain=True)
path, sha256 = save_retrieval(
Expand Down
11 changes: 8 additions & 3 deletions docs/module-layout.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,10 @@ Knowledge move (K01) is implemented: `src/inferrsi/knowledge/` holds the seven
moved Knowledge bodies, and the old root modules are static compatibility
shells. The engineering task contribution now implements `benchmark/registry.py`
and `benchmark/tasks/`. The current flat `cli.py` and `benchmark_operations.py`
assemble this slice.
assemble this slice. The former `observability/` package is dissolved. Its
campaign records, traces, logs and profile captures are in `harness/records/`,
its dashboard views and templates are in `apps/views/`, and the
collected-context projection is in `knowledge/context_projection.py`.
#371 remains the proposal for the Apps package path and is not accepted here.

The old `run` chain is split by owner. Its gates are in
Expand All @@ -29,8 +32,8 @@ modules are isolated.
There is no sibling `data/` resource tier. K02 moved the Knowledge corpus to
`src/inferrsi/knowledge/data/` and the `repo_packs/*/pack.json` target
descriptions to `src/inferrsi/harness/data/`. The internal corpus split is left
to a future Knowledge issue. `observability/template.html` is a view template
and follows the Apps slice; it did not belong to the corpus merely because it
to a future Knowledge issue. The dashboard templates are view templates and
live in `apps/views/`; they did not belong to the corpus merely because they
sat under `data/` before.

The move is physical only. Bundle resource keys, dependency keys and the pinned
Expand Down Expand Up @@ -82,6 +85,8 @@ Several flat modules remain shared seams until their slice starts.
| `benchmark/skill_evaluation.py`, `benchmark/skill_eval_diagnostic.py`, `benchmark/skill_eval_engineering.py`, `benchmark/improvement.py` | Skill-evaluation criteria, task definitions and acceptance rules | Pure judgments over retained facts; no Harness import and no process start |
| `apps/skill_evaluation.py`, `apps/skill_eval_engineering.py`, `apps/improvement.py` | Composed skill-evaluation entries | Join Knowledge reads, Harness execution and Benchmark rules behind the signatures that `harness/` held before the split |
| `knowledge_operations.py`, `knowledge_store.py`, `catalog.py`, `research.py`, `skill_catalog.py`, `_skill_selection.py`, `_source_snapshot.py` | Static compatibility shells since K01; bodies in `knowledge/` | Interfaces unchanged |
| `harness/records/` | Campaign manifests, traces, logs, profile captures and delivered-context captures | Imports no Knowledge or Apps module; `apps/campaign.py` supplies the backend names, the selection default and the context projection |
| `apps/views/`, `apps/campaign.py` | Dashboard views, templates and the Knowledge-to-records connection | Apps path still follows the #371 proposal |
| `harness/` | Mixed legacy tree, not target isolation | Not redefined |
| `harness/runner.py`, `harness/scenarios/external.py`, `harness/atomic_json.py` | Old run execution facts and Harness-owned JSON writes | Ports injected; no Model edge, and no Infra edge except the local process executor |
| `benchmark/legacy_run_v0.py`, `benchmark/sampling.py` | Old run gates and metric validation; image/video sample collection | Imports only `model/protocols/metrics`; owns its rule inputs and verdict labels |
Expand Down
2 changes: 1 addition & 1 deletion docs/operations.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ Effect classes used below:
| Execute one ordinary local CPU command with a fixed record | Python `inferrsi.harness.api.Harness.allocate` / `execute` / `read` | Experiment-executing / record-writing | One explicit RunStore root and one trusted local CPU target; ordinary command mode only; a record holds execution facts, so exit 0 is not a verdict; no device, model or network fallback |
| Visualize a recorded run or campaign | CLI `dashboard`; [dashboard](dashboard.md) | Report-generating / local read-only serving | Writes an offline HTML snapshot or serves an optional live view on loopback; does not run an agent or validate a claim |
| Capture a knowledge context | CLI `campaign capture-context`; [offline dashboard](dashboard.md#record-the-optimization-story-campaign-trace) | Local evidence-writing | Runs `collect` with a context budget and keeps the full envelope and resource hashes in a new `knowledge/<id>.json`; proves what was prepared, not what a model read |
| Retain an executed retrieval | Python `observability.knowledge_capture.save_retrieval`; [capture contract](knowledge-retrieval-trace.md#retain-a-standalone-result) | Local evidence-writing | Saves an existing catalog/collect result under a fresh campaign capture ID; no new search or automatic publication |
| Retain an executed retrieval | Python `apps.campaign.save_retrieval`; [capture contract](knowledge-retrieval-trace.md#retain-a-standalone-result) | Local evidence-writing | Saves an existing catalog/collect result under a fresh campaign capture ID; no new search or automatic publication |
| Import one round of knowledge searches | CLI `campaign import-search`; [round import](dashboard.md#import-one-rounds-searches) | Local evidence-writing | Reads a Codex rollout window or context captures. With `--archive-review`, retains excerpts of attributable tool returns and reviewed decisions for readers of the evidence bundle. |
| Record the optimization log | CLI `campaign log`; [offline dashboard](dashboard.md#record-the-optimization-story-campaign-trace) | Local evidence-writing | Validates and stores a new numbered campaign log version; the dashboard cross-checks it against captures and runs |
| Assemble an optimization trace | CLI `campaign assemble-trace`; [offline dashboard](dashboard.md#capture-the-optimization-trace) | Local evidence-writing | Copies an explicitly reviewed context into a new bounded campaign-root snapshot; caller owns redaction and narrative accuracy |
Expand Down
Loading
Loading