Skip to content

Genome: many genes per turn, weighted by distance; genes paged under the KV budget; the old paging stacks deleted (−6.8k) - #5367

Merged
joelteply merged 4 commits into
canaryfrom
fable/genome-many
Oct 24, 2026
Merged

joelteply merged 4 commits into
canaryfrom
fable/genome-many

Conversation

@joelteply

Copy link
Copy Markdown
Contributor

Joel: "Goal was many with weights. We need to get this on tracks." Engine half: CambrianTech/llama.cpp#72, stacked on #70.

Slice 1, genome::composer: her phenotype is many genes, weighted

  • Each gene's floor is filed at mint: the q10 of its own corpus, never below null + 2σ.
  • Weight a = clamp((sim − floor) / (1 − floor)); keep the top-k; engine scale a / max(1, Σa).
  • Hysteresis: a gene enters at its floor and leaves at 1.5× that distance.
  • The perception comes from recall's focused_query.
  • Scales stay bit-identical while her set holds. The fork clears KV on any scale change, so re-weighting every turn would re-prefill her whole context every turn.
  • compose() and Phenotype::without(g) are the examiner's hooks.
  • Replaces genes_for_turn (every gene at 1.0) and model_selection's distance rung.

Slice 2, genome::gene_residency: genes page in and out by temperature

  • It is a tenant of the one inference-lanes budget the KV uses, with no budget of its own. Pressure frees genes first, then lanes.
  • Genes in any mind's current set are pinned, and after each turn it prefetches them back off the request path.
  • Engine: fork /lora-adapters/page_out and /page_in, 2.3 ms out and 4.6 ms in (0.8B, 50 MB LoRA), with identical output after an on-demand page-in.

Deleted (Cormac agreed on airc):

  • genome/{local_manager, manager, bus, store, blob, working_set};
  • the TS-era persona/genome_paging;
  • six cognition/genome-* commands and sync-domain-classifier;
  • their TypeScript bindings.

Measured batching cost: minds with different gene sets batch at 27 tok/s, against 115 for the same set. The scheduler (goal 9) must group minds by gene set.

Measured

M5, 2026-10-24, serving down, rebased on 7cd9c97.

Check Result
genome 324 passed
cognition 1,456 passed
mind 754 passed
core check 0 errors
ratchets all fall: unjustified_unwraps 1999 → 1989, unjustified_serialization 229 → 222, upward_imports 37 → 35, unwired_machinery 43 → 42

1,270 lines added, 6,838 removed.

🤖 Generated with Claude Code

https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo

joelteply and others added 4 commits October 24, 2026 02:18
… holds (GENE-REUSE-FORK-MINT §3d)

Serves her learning (goal 8): a mind's genes follow what her mind is on, several at
once, instead of every gene at full scale on every turn (the live path) or one gene by
max_by (rung 0, a command nothing called).

- genome::selection is the one owner of APPLIED: every gene whose region the turn's
  focused line is inside, a convex mix weighted by depth in its own region
  ((sim - leave) / (1 - leave)), enter at r and leave at 1.5 r, weights held
  bit-identical while the set holds: the fork clears a slot's KV on ANY scale change
  (are_lora_equal compares scales exactly), so re-weighting per turn would re-prefill
  her context every turn. Unsigned genes run at their trained scale (as before).
- GeneSignature files its region at mint (10th-percentile similarity of its own corpus),
  floored at the space's measured null + 2 sigma; old signatures still load.
- Both turn sites ask genome::selection::live_genes with recall's focused_query (one
  owner of "what her mind is on"); the genes stay on her cycle until the next turn, so
  nothing between turns flips her slot's adapters.
- Probes: genome.select (each gene's similarity, edges, scale), genome.select.set_changed.
- Deleted: gene_trial::live_genes, GenomeRestore (restoring the pre-turn genome now
  flips her adapters twice a turn), model_selection rung 0 + NeedEmbedding + its two
  constants, the select-model command's signature join.

Tests: 6 selection contracts + 1 region test; genome 387 pass, model_selection 14 pass,
source-hygiene 47 pass; continuum-core --tests checks clean.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo
…a / max(1, sum a); the receipt carries it

Per the coordinator's learning-loop design: genome::selection becomes genome::composer, the ONE
composer of a turn's phenotype (gene_trial::genes_for_turn is now her_genes, the candidate set
at trained scale; the composer sets the scale).

- weight a = clamp((sim - f) / (1 - f), 0, 1) from each gene's floor; the `capacity` deepest
  kept; engine scale a / max(1, sum a); hysteresis and held-while-the-set-holds unchanged.
- compose(persona, perception) -> Phenotype; Phenotype::without(g) is the examiner's P minus g
  (g off, every other scale unchanged).
- GenerationReceipt carries gene_scales aligned with genes (PartialEq only, f64); turn sites
  probe persona.genome.turn_genes with each gene's scale.

Tests: composer 8 (adds capacity and P minus g), genome 389, provenance 3, mind 38,
source-hygiene 47; continuum-core --tests checks clean.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo
…de the KV's budget; the unwired working-set stack and the TS-era LRU are deleted

Serves her learning on any hardware (goal 8, SERVING-ENGINE §3 one budget): a node's genes
hold device memory only while minds run them, without a second budget beside the KV.

- genome::gene_residency: ONE node-wide owner. A DeviceTenant of the lanes' pool
  (inference-lanes): its device bytes count there and the PressureBroker relieves it first
  (a gene pages in ms, a lane costs a mind her KV); LRU-2 temperature (second-last use by any
  mind, no decay constant); genes in a mind's current phenotype are pinned; after each turn a
  coalesced reconcile pages pinned host genes back in, off the request path. It holds each
  mind's phenotype, which is the composer's previous (one copy). Probes genome.page,
  genome.page.failed.
- CoordinatorResourcePool::with_tenant: usage = lanes + tenants, evict tenants before lanes.
- llama_server::gene_pages: the engine's /lora-adapters catalog with residency and the
  page_out / page_in verbs (CambrianTech/llama.cpp#72); unknown bytes stay None.
- gene_trial::Standing (Pending / Adopted) on her_genes; Standing::may_run is the one rule
  the examiner's verdict owns (Cormac's condition); the composer asks it.
- Deleted (Cormac agreed on airc): genome/{local_manager (+848-line tests), manager, bus,
  store, blob, working_set}, tier.rs down to TierRole (recall uses it); persona/genome_paging
  (+tests) and PersonaCognition::with_budget; the six cognition/genome-* commands and
  sync-domain-classifier (no caller in any client; COMMAND-CENSUS marks it for deletion);
  the channel tick's enrollment step (read an engine nothing ever filled);
  DomainClassifier::sync_from_adapters; their ts-rs bindings; the dead stress-tests feature.

Tests: residency 3, tenant 1, gene_pages 1; genome 323, engine 17, mind 68, core cognition
104, source-hygiene 49; continuum-core --tests checks clean with and without stress-tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo
…es + Standing::may_run), at trained scale

gene_trial::live_genes was replaced by genome::composer; the exam holds the genome fixed so
the arms differ only by the gene.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo
@joelteply
joelteply merged commit 4ac5e64 into canary Oct 24, 2026
5 checks passed
@joelteply
joelteply deleted the fable/genome-many branch October 24, 2026 07:28
joelteply added a commit that referenced this pull request Oct 24, 2026
#5357's core job 120814431211 at bd3a1c1: module_wiring_guard listed PageFaultOnly
(genome bus) and RecorderModule (genome local_manager), and #5367 (4ac5e64) deleted
both with the old paging stacks. The two entries are deleted.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FRQWzgSfo79JtnwZywHKrE
joelteply added a commit that referenced this pull request Oct 24, 2026
#5357's core job 120814431211 at bd3a1c1: module_wiring_guard listed PageFaultOnly
(genome bus) and RecorderModule (genome local_manager), and #5367 (4ac5e64) deleted
both with the old paging stacks. The two entries are deleted.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FRQWzgSfo79JtnwZywHKrE
(cherry picked from commit 84505ee)
joelteply added a commit that referenced this pull request Oct 24, 2026
… on her next cold turn (card 6b28a445) (#5357)

* A resident page discarded by an engine relaunch is recorded and named on her next cold turn (card 6b28a445)

Line 6, every fast-path miss is loud, on top of #5348. A replacement now saves
residents first, but three paths cannot, and they lost the KV with no record:
checkpoint_before_replacement returned Ok when the generation had exited or paging
was quarantined, and retire_adopted skipped the checkpoint when paging was
quarantined and returned Ok when the adopted engine was already dead. The cold turn
that followed said no_page or page_on_disk_unconfirmed, blaming the page store for a
relaunch.

Each of those paths now calls EndpointSlots::record_relaunch_discard with its
reason (engine_exited, paging_uncertain, adopted_engine_dead). That records the minds
held in a slot without a confirmed save, and probes
inference.kv_page.discarded_by_relaunch with the count. The record lives on the
endpoint because the pool is replaced with the engine. Every admission takes its own
page's record, so only the first turn after the relaunch is blamed on it. A cold
turn then reports cause=discarded_by_relaunch with discard_reason. The record is one
entry per mind, replaced and never grown; forks die with their core and are not
recorded, as in the ledger.

New file slots/relaunch_discard.rs; test
slots::tests::a_relaunch_records_only_unsaved_residents_and_their_next_turn_takes_it_once.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FRQWzgSfo79JtnwZywHKrE

* A relaunch discard is captured early and recorded only once the engine is gone; retention is one pool of slots (card 6b28a445)

Codex's review of #5357 at ab7788b:
(1) The record was not bounded: minds discarded by one relaunch that never returned
stayed forever, across generations. DiscardLog now keeps at most the latest
discarding pool's slot count, oldest evicted first. One relaunch cannot discard more
minds than it had slots, so the latest is kept whole, and an empty capture evicts
nothing.
(2) A loss was recorded before the retirement re-checked current(), re-verified the
engine's identity or signalled it, so a superseded or refused retirement could
report a loss that never happened. The residents are now captured as a
PendingDiscard (must_use) while the old pool is installed, and committed only once
the engine is actually gone. checkpoint_before_replacement returns the capture, and
launch commits it after kill_child. retire_adopted commits after signal_if_same
succeeds, or as adopted_engine_dead when the engine is seen dead. Every early return
drops the capture unrecorded.

Tests: the lifecycle path through checkpoint_before_replacement (a dropped capture
records nothing, a committed one is taken once and survives the pool's replacement),
and retention across three generations.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FRQWzgSfo79JtnwZywHKrE

* The retention test defines 'oldest' with one mind per relaunch (card 6b28a445)

CI job 120805073062 at 769d5d7: discard_records_keep_one_pool_of_slots_across_generations
asserted minds 1 and 2 were evicted after relaunches [1,2,3] then [4,5]. But the
residents of one relaunch come from a HashMap, so their order within that relaunch is
arbitrary, and which two were oldest was not defined. The test now evicts with one
mind per relaunch ([1],[2],[3],[4] at three slots: 1 evicted), then checks that a
whole pool's relaunch ([7,8,9]) is kept whole over an older one ([5,6]) and that an
empty relaunch evicts nothing. The code is unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FRQWzgSfo79JtnwZywHKrE

* A relaunch's loss is recorded at the engine's observed death, never when the kill is sent (card 6b28a445)

Codex's re-review of #5357 at 5d8e4c2: launch committed the discard right after
kill_child (a void call that may only log a failed signal), and retire_adopted
committed it right after signal_if_same, both before any exit was observed. A sent
signal is not a death.

- Owned engine: checkpoint_before_replacement hands its capture to the endpoint
  (PendingDiscard::on_exit_of), held for that generation (DiscardLog's watch) and
  recorded by EngineGeneration::observed_exit, or at once if the exit was already
  observed. A generation already watched keeps its first capture, so re-entry never
  records twice. launch.rs is back to canary's.
- Adopted engine: the capture rides the retirement custody beside retiring_adopted
  (LlamaServerProcess::retiring_discard). await_adopted_exit records it only when the
  exact process is seen gone, so a timed-out, cancelled or re-entered wait records
  nothing until then. A failed signal returns before custody, so its capture drops
  unrecorded. An engine already seen dead is recorded at once.

Tests: through checkpoint_before_replacement, nothing is recorded while quarantined
(two attempts), the observed exit records it once, and a re-entry after exit adds
nothing. adopted_exit_wait_retains_custody_until_observed_exit now checks that no loss
is recorded after a cancelled poll or a zero-budget timeout, and that the real child's
exit records it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FRQWzgSfo79JtnwZywHKrE

* Two unwired-fixture entries named modules #5367 deleted (card 6b28a445)

#5357's core job 120814431211 at bd3a1c1: module_wiring_guard listed PageFaultOnly
(genome bus) and RecorderModule (genome local_manager), and #5367 (4ac5e64) deleted
both with the old paging stacks. The two entries are deleted.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FRQWzgSfo79JtnwZywHKrE

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
joelteply added a commit that referenced this pull request Oct 24, 2026
* Restage incomplete held checkouts instead of exposing them to Hands

* fix(workspace): embed transactional AIRC clone allocator

* test(workspace): give bound activity a real checked-out repository

* Two unwired-fixture entries named modules #5367 deleted (card 6b28a445)

#5357's core job 120814431211 at bd3a1c1: module_wiring_guard listed PageFaultOnly
(genome bus) and RecorderModule (genome local_manager), and #5367 (4ac5e64) deleted
both with the old paging stacks. The two entries are deleted.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FRQWzgSfo79JtnwZywHKrE
(cherry picked from commit 84505ee)

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant