Repository navigation
Conversation
Collaborator
|
|
Discover, list, search, view, export, and convert from dsh sessions under $DSH_HOME/sessions (default ~/.dsh/sessions). The store is read-only: dsh has no documented session import command and its persistence seam does not delete logs. Logs are append-only JSONL, Zstandard-framed by default. The first line is a type:session header; the Common projection rebuilds the ordered surface (user/message, assistant/message, tool/result) and leaves packed chunk rows and unknown events in the native body.
…ption Four review findings on the dsh harness, all confirmed against the official `@deepseek-ai/dsh-session` sources: - `surfaceOp` replace endpoints are event seqs, not surface-vector indexes. Log-only events sit between surface nodes, so the two diverge immediately and compaction shadowed the wrong range. The surface now tracks each node's seq and resolves both endpoints by seq, matching `replacementRange` in dsh's own `surface.ts`. The previous test masked this by using seqs that happened to equal vector indexes. - Durable `assistant/message` `usage` and `interrupted` were dropped. Map `interrupted: true` to `StopReason::Aborted` and translate the `TokenUsage` counts, in both directions. `reasoningTokens` has no Common counterpart and stays in the native body. - `std::io::Read` was imported unconditionally but only used by the native `decode_zstd`, failing the wasm32 job under `-D warnings`. - Zstandard frames were split by scanning for frame magic, which also occurs inside compressed payloads and checksums. The streaming decoder already walks frame structure and continues across boundaries, so decode in one pass. Also fixes an invariant violation in the writer: `tool/result` emitted `sourceEventSeqs: []`, which dsh's `assertProvenance` rejects — empty arrays are legal only on `assistant/message`. Verified by feeding a regenerated log from a real session to the official `foldSurface`, which now passes.
The dsh store pulled zstd in as an unconditional non-wasm dependency. Codex now declares the same crate as optional behind `codex_zstd`, and Cargo merges the two declarations, so the unconditional one would turn zstd on for every native build and make that feature meaningless. Follow the Codex pattern instead: `dsh_zstd = ["dep:zstd"]`, on by default. Without it — including the wasm build — `session.jsonl.zstd` logs fail to load and drop out of discovery, while `session.jsonl` logs read as before.
zzyking
force-pushed
the
feat/dsh-writable
branch
from
October 15, 2026 11:57
96c04f6 to
7d2ce86
Compare
dsh 0.2 publishes each format generation as its own file (`session[.vN].jsonl[.zstd]`) and keeps older generations beside it. Discovery now picks the highest generation, and the reader understands v0 through v4 directly: `startSeq`/`endSeq` replace endpoints, the `system/message` and `developer/message` surface nodes, and tool-role `tool/result` messages. Environment context dsh injects as user messages (runtime snapshot, skill catalog, instruction files, clocks, ...) is no longer projected into Common. `parentSession` becomes a Fork lineage, or Spawn for a subagent run. Checked against sessions recorded by dsh 0.2.0-rc.2, and against a v0 session read both as-is and after the official catalog migrated it to v4.
dsh ships no session import command, but it does not need one: it finds sessions by walking its root, so writing one is a matter of reproducing the layout it scans for. What makes that exact rather than approximate is dsh's validation. Three of its checks fail the *entire* listing rather than skipping the one bad session, which makes each a hard requirement on the writer: - `assertZstdHeaderFrame` — the first Zstandard frame must decode to exactly one line. Compressing the whole log as a single frame still round-trips through txcript's own reader, which decodes across frame boundaries, so only the official reader rejects it. - `assertStoredIdentity` — the header's `id` and `cwd` must name the path the log was found at. A copy given a new identity would otherwise keep pointing at the original's, and take the listing down with it. - `checkRootEncoding` — one root must not mix `.jsonl` with `.jsonl.zstd`. Writing the default encoding into a plaintext root breaks that root under *both* configurations, so the root decides the encoding and only an empty one falls back to dsh's `zstd` default. A duplicate id across two project directories is rejected the same way, so re-saving a session whose cwd changed retires the copy under the old project directory. Path derivation follows the official `encodeSegment` and `projectKey`: separators collapse to `-`, `[A-Za-z0-9._-]` survives, every other UTF-16 code unit becomes `~XXXX`, and the project key is wrapped in `--`. Escaping over code units rather than scalars is what keeps it injective over lone surrogates. Escaping the separators is also what contains a traversing id — `../../evil` becomes the literal directory `..~002F..~002Fevil`, matching dsh, which encodes rather than rejects. Verified by running the official persistence backend's own `listArtifacts` and `loadStored` against a txcript-written root: it lists both a normal session and a `_no-cwd` one, and decodes all 1257 event records of the former. Re-introducing the single-frame bug makes that same check fail with dsh's own "first frame is not exactly one header line". The derived project-directory names reproduce those of the real local session store exactly, including its non-ASCII path. `delete` removes the session directory. dsh's persistence seam has no delete API, but it keeps no index either — an absent directory is simply not scanned.
`save` retired copies of the session under other project directories before writing the replacement, so a write that failed afterwards left no copy at all. It now encodes the log first, writes it to a temporary file and renames it into place, and only then removes the stale copies. `delete` called `remove_dir_all` on whatever path it was handed. It now requires a session log, resolves the path with `canonicalize`, and refuses anything that isn't exactly `<root>/<project>/<session>` — the same guard the grok store applies. Tests cover a failed save under a new cwd, a session-shaped directory outside the root, a project directory, a directory with no log, and a symlink escaping the root.
dsh 0.2 migrates older logs when it opens them, but it reads only the current format with full validation, and a log it has to migrate is kept beside its successor. Write v4 directly from Common: - the turn/step lifecycle the official reader checks, with each tool call started by `tool/call` and answered by a tool-role `tool/result`; - an empty `system/message` heading the surface, without which the log dsh extends on resume fails its next load; - dsh's own not-started repair for a call with no result; - a provider/model on every model source and an empty embedded stream. A native body keeps its version, and `save` names the file for it (`session[.vN].jsonl[.zstd]`), retiring other generations beside it. Converted Claude Code, Codex, and OpenCode sessions pass the official catalog's strict current-format validation, and a converted session resumed twice with `dsh --profile headless --session-id`.
zzyking
force-pushed
the
feat/dsh-writable
branch
from
October 15, 2026 12:59
7d2ce86 to
a764222
Compare
Contributor
Author
|
Both fixed in c5b491d:
Separately, dsh moved on while this was open: 0.2.0-rc.2 writes session format v4, so I've updated the writer to match (a764222). This doesn't depend on our reader. Every log is checked with dsh's own format catalog under its strictest settings (
The details and provenance are in |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #19 — until that merges, the diff here includes its commits. The
new commits are
feat: make the DeepSeek Harness store writable,fix(dsh): keep the previous copy until a save succeeds, confine delete, andfeat(dsh): write session format v4.Companion to #32, which does the same for Kimi. Both follow from the read-only
question raised on #18/#19: the bar I set there (a harness needs an official
import command before txcript writes its storage) was stricter than the one the
rest of the tree holds — of the twelve writable harnesses, only
opencodegoesthrough a vendor import command.
Why it works
dsh discovers sessions by walking
<root>/<project>/<session>/. No index, noimport command — writing the layout it scans for is enough to be found.
What makes it exact rather than approximate
dsh validates what it finds, and three of its checks fail the entire
listing rather than skipping the one bad session. Each is a hard requirement on
the writer:
assertZstdHeaderFrameassertStoredIdentityidandcwdname the path the log was found atcheckRootEncoding.jsonlwith.jsonl.zstdThe first is the interesting one. Compressing the whole log as a single frame
still round-trips through txcript's own reader, because that decodes across
frame boundaries — so no test written against txcript catches it. Only the
official reader does.
The third is the most destructive: writing the default
zstdencoding into aroot that dsh configured for plaintext breaks that root under both
configurations, taking down the user's whole session list. So the root decides
the encoding, and only an empty root falls back to dsh's own default.
A duplicate id across two project directories is rejected the same way, so
re-saving a session whose cwd changed retires the copy filed under the old one.
Writing format v4
dsh 0.2.0-rc.2 reads the current format (v4) with full validation, so Common
sessions are written as v4 directly, laid out as the turn/step lifecycle that
validation enforces: each tool call is started by
tool/calland answered by atool-role
tool/resultbefore its step ends. A few events exist only becausedsh requires them:
system/messageheads the surface. dsh appends itssystem prompt when it resumes a session and accepts that only behind a system
head; without one, the first resume works and the log dsh wrote fails its
next load.
TOOL_NOT_STARTEDrepair, andthat turn ends
interrupted.txcript) andmodel; its embedded stream is empty, the form dsh skips when replaying.
A native dsh body keeps its version, and
savenames the file for it(
session[.vN].jsonl[.zstd]), since dsh rejects a file name and header thatdisagree. Other generations in the directory are removed.
Path derivation
Follows the official
encodeSegmentandprojectKey: separators collapse to-,[A-Za-z0-9._-]survives, every other UTF-16 code unit becomes~XXXX,and the project key is wrapped in
--. Escaping over code units rather thanUnicode scalars is what keeps it injective over lone surrogates.
Escaping the separators is also what contains a traversing id:
../../evilbecomes the literal directory
..~002F..~002Fevil. That matches dsh, whichencodes rather than rejects — only a segment that is entirely
.or..isspecial-cased.
Verification
dsh's own format catalog under its strictest settings (
recovery: 'strict',validation: 'current') accepts sessions converted from Claude Code, Codex,and OpenCode (the largest about 14k events). A converted session was resumed
twice with
dsh --profile headless --session-id; dsh answered from theimported history, and the log it extended passed the same check after each
turn.
The official persistence backend's own
listArtifactsandloadStored, runagainst a txcript-written root:
_no-cwdone;decodeStorageRecord.Re-introducing the single-frame bug makes that same check fail with dsh's own
corrupt Zstandard session log: first frame is not exactly one header line,which is what confirms the oracle has teeth.
The derived project-directory names also reproduce those of the real local
session store exactly, including its non-ASCII path. Real dsh data was never
written to.
savewrites to a temporary file and renames it into place, retiring othercopies only afterwards;
deleteonly accepts a path that resolves (symlinksincluded) to
<root>/<project>/<session>.dsh joins the
path_safetyandstore_deletesuites, and the CLI's refusal tocontinue a dsh session in place is removed.
Full matrix green locally: fmt, clippy
-D warnings, workspace tests,no-default-features, both wasm32 invocations, MSRV 1.96.