Repository navigation
Conversation
…or is what it embeds Joel, 2026-11-10: "It depends on where it's resident... We just can't assume any specific machine." The embed lane leased VRAM on every node, even for a context in host memory (a backend offloading no layer, or a build with no GPU), and its standing floor was a fixed 1,792 MiB VRAM bound reserved by the resolver, which knows neither the context's residence nor her texts. - LlamaCppBackend::context_pool: device memory when llama offloads to a GPU (llama::supports_gpu_offload) and the backend offloads any layer, else host memory. - inference/embed_lane_memory.rs owns the lane's memory: each lease in that pool, and the floor re-stated at each lease at the largest context the lane has embedded (the resolver's probe and calibration passes set it first). EMBED_LANE_VRAM_BYTES and the resolver's reservation are deleted; the lease code leaves llamacpp_adapter.rs (938 -> 848 lines). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo
…reclaim it blocks The reclaim of a mind's grant takes her turn lock (MindTurn::reclaim), so a turn that waits for the reclaim inside itself waits on itself: on the M5 (d0abbdd, 2026-11-10) Aris parked to the 1,500 s tick deadline five times and acted 3 times in 132 minutes. Reclaiming refuses again (her turn ends, the reclaim proceeds, she is regranted); queued and an engine relaunching still wait. Each wait now probes its reason (delib.window.waiting), which the loop had dropped. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo
…es idle cores and yields On the M5 (2026-11-10) a resident's `cargo test` of the whole workspace ran at the engine's own priority beside serving: load 36, 4.5 GB swapped, decode at 3.4 tokens/s against 56 at rest, and the residents' acts fell from 39 to 10 per 15 minutes. Her build is never refused or capped for it; it is scheduled like an OS background job. - continuum-host process_class: Unix (macOS, Linux) lowers the child to the OS's lowest ordinary priority (PRIO_MAX), which still runs on every idle core; Windows gives it IDLE_PRIORITY_CLASS; elsewhere the parent's class. - The kernel's command_for, the one way a caller's child is spawned, applies it to every child, confined or not. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo
…th a floor per pool Codex on #5910: one context's bytes are in two pools (KV and device compute on the device; the CPU's compute share and the output buffer on the host), a configured placement is not a measured one, and one process-wide floor was reused across pools. - Placement (configured, never measured: card 9bddda93) is Device, Host or Split. A split is not measured, so each pool is charged its upper bound. - ContextBytes { device, host } comes from rates written as receipts (MiB llama reported over the tokens measured), now including the host-resident context: CPU KV 224 and compute 1,200.97 MiB at 2,048 tokens (engine-c, -ngl 0, logs in evidence/replicate-2026-11-10). - embed_lane_memory leases each non-empty part in its pool, with a floor per pool. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LoTjvf5j3Ez13g6k8mRkFo
…/embedding-admission-consumes-backend-mea # Conflicts: # core/continuum-engine/src/inference/backends/llamacpp/embedding_batch.rs # core/continuum-engine/src/inference/backends/llamacpp/llama_cpp_backend.rs # core/continuum-engine/src/inference/embed_lane_memory.rs # core/continuum-engine/src/inference/llamacpp_adapter.rs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Embedding contexts were charged using Metal-derived constants and configured placement, which could not represent CUDA allocations split across host and device memory. This replaces that estimate with a typed allocation plan measured by the native backend on the already loaded model. The same plan governs both ResourceDaemon admission and context materialization; partial pool grants are released before retry.
Depends on native allocator API CambrianTech/llama.cpp#103 (pinned commit 23ac1168a). Incorporates PR #5910; removes its superseded embedding estimates while retaining its reclaim and process classification changes.
Validation: isolated native CPU and CUDA SM120 builds pass; three focused Rust allocation-plan tests pass. Engine embed tests: 41 passed, 0 failed, 3 model-dependent tests ignored. Independent Intel Mac build/materialization verification is in progress. No deployment or live recovery acceptance claimed.