Skip to content

fix(cli): include size + extras in rootfs cache key - #280

Open
jrimmer wants to merge 1 commit into
deeplethe:mainfrom
jrimmer:fix/rootfs-cache-key-include-size-extras
Open

fix(cli): include size + extras in rootfs cache key#280
jrimmer wants to merge 1 commit into
deeplethe:mainfrom
jrimmer:fix/rootfs-cache-key-include-size-extras

Conversation

@jrimmer

@jrimmer jrimmer commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

Related issue: #284


The rootfs cache key was derived only from the image name (slug), so forkd from-image elixir:1.17 (default 1536 MiB) and forkd from-image elixir:1.17 --size-mib 3072 both mapped to the same elixir-1-17.ext4. A rebuild with a different --size-mib (or --extra) silently reused the stale rootfs, producing a truncated/corrupt ext4 that kernel-panics on mount (VFS: Unable to mount root fs).

rootfs_cache_key() hashes size_mib + extra packages into the cache filename → elixir-1-17-3072-<sha8>.ext4. Applied in both from_image_cmd and run_cmd. Adds 4 unit tests (size, extras, determinism, slug sanitization).

Verified: cargo test -p forkd-cli → 36 passed; re-baked elixir:1.17 --size-mib 3072 after the fix → snapshot builds and boots, elixir --version → 1.17.3 in a live sandbox.

@jrimmer
jrimmer force-pushed the fix/rootfs-cache-key-include-size-extras branch from 3d345c1 to 5f7fb0d Compare August 10, 2026 16:45
@jrimmer
jrimmer force-pushed the fix/rootfs-cache-key-include-size-extras branch from 5f7fb0d to f56e569 Compare August 10, 2026 17:01
@jrimmer

jrimmer commented Aug 10, 2026

Copy link
Copy Markdown
Contributor Author

Per the contributing guidelines, I ran the local gate on this branch:

  • cargo fmt --all -- --checkpass
  • cargo clippy --all-targets --all-features -- -D warningspass
  • cargo test --all → 1 failure: chain::tests::assemble_chain_memory_produces_correct_bytes (crates/forkd-vmm/src/chain.rs:511)

The failing test is pre-existing and unrelated to this PR: it panics with FICLONE on base memory → Operation not permitted (os error 1) — a filesystem/reflink limitation of the host I ran it on. I verified it fails identically on a clean checkout of deeplethe/forkd@main (same panic, same crate), and this PR touches no forkd-vmm code. The other 40 forkd-vmm tests pass.

Commits are signed off per the DCO requirement.

@jrimmer

jrimmer commented Aug 10, 2026

Copy link
Copy Markdown
Contributor Author

Note on the force-push: the branch history was rewritten after the PR was opened, to align with the contributing guidelines:

  1. DCO sign-off. The contributing guidelines require commits to be signed off (git commit -s). The original commits did not carry a Signed-off-by trailer, so they were amended to add one.

  2. Correct authorship. The original commits were authored by the agent tooling that implemented the fix on our behalf rather than by the repository owner. The amendment corrected the author/committer attribution to the actual maintainer of the contribution.

  3. Formatting. Running the guideline's cargo fmt --all -- --check locally flagged two formatting deviations in this branch's changed lines; cargo fmt --all was applied and the amendment includes the formatted result.
    No functional changes were introduced by the rewrite; the diff against main is identical to the original PR (verified before push).

@WaylandYang WaylandYang left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix and the added tests. One cache-correctness blocker remains: rootfs_cache_key feeds only size_mib and extra into SHA-256; the image reference is represented only by the lossy slug. Distinct valid image references such as foo/bar:1 and foo-bar-1 both normalize to foo-bar-1, so identical size/extras will still reuse the same rootfs and can boot content from the wrong source image.

Please include the raw image reference in the hashed payload (with unambiguous framing, e.g. length prefixes or separators) and add a regression test covering two image names that collide after slug sanitization.

jrimmer pushed a commit to jrimmer/forkd that referenced this pull request Aug 11, 2026
The digest previously covered only size_mib + extra packages; the image
reference appeared solely in the lossy slug filename, so two distinct
refs that slug identically (e.g. "foo/bar:1" vs "foo-bar-1") with the
same size/extras produced the SAME cache key and could boot the wrong
rootfs. Hash the raw reference (length-framed) into the digest; the slug
remains for filename readability only.

Regression test: cache_key_differentiates_refs_with_same_slug
jrimmer added a commit to jrimmer/forkd that referenced this pull request Aug 11, 2026
The digest previously covered only size_mib + extra packages; the image
reference appeared solely in the lossy slug filename, so two distinct
refs that slug identically (e.g. "foo/bar:1" vs "foo-bar-1") with the
same size/extras produced the SAME cache key and could boot the wrong
rootfs. Hash the raw reference (length-framed) into the digest; the slug
remains for filename readability only.

Regression test: cache_key_differentiates_refs_with_same_slug
@jrimmer
jrimmer force-pushed the fix/rootfs-cache-key-include-size-extras branch from 8836afe to 733ffa9 Compare August 11, 2026 16:52
@jrimmer

jrimmer commented Aug 11, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the review — confirmed, and fixed in commit 733ffa9.

The digest previously covered only size_mib + extra packages; the image reference appeared only in the lossy slug filename, so two distinct refs that slug identically (e.g. foo/bar:1 vs foo-bar-1) with the same size/extras produced the same cache key and could boot the wrong rootfs.

Fix: hash the raw image reference into the digest, length-framed (len(image) || image), before the existing size/extras. The slug stays in the filename purely for human readability; the digest is the collision-proof part of the key.

Regression test: cache_key_differentiates_refs_with_same_slug asserts foo/bar:1 and foo-bar-1 (same size/extras) produce different keys, and that the same ref stays deterministic.

Tradeoffs: the length prefix means two refs that are byte-different but share a prefix can't alias (e.g. ab + c vs a + bc); SHA-256 truncation to 8 bytes for the filename keeps keys short — collision odds there are negligible for a cache namespace, and full-digest verification still applies to the rootfs content when it's mounted.

fmt/clippy/tests green.

The digest previously covered only size_mib + extra packages; the image
reference appeared solely in the lossy slug filename, so two distinct
refs that slug identically (e.g. "foo/bar:1" vs "foo-bar-1") with the
same size/extras produced the SAME cache key and could boot the wrong
rootfs.

- Hash size_mib, extra packages, and the raw image reference
  (length-framed) into the digest; the slug remains for readability
- Regression test: cache_key_differentiates_refs_with_same_slug
@jrimmer
jrimmer force-pushed the fix/rootfs-cache-key-include-size-extras branch from 733ffa9 to 588cf68 Compare August 11, 2026 17:14
@jrimmer

jrimmer commented Aug 11, 2026

Copy link
Copy Markdown
Contributor Author

History rebuilt: the branch is now a single clean commit on top of current main (5e457c4) — no merged-upstream history in the diff. Same content as before: raw image ref hashed (length-framed) into the rootfs cache key + slug-collision regression test. Force-pushed; the old commits are gone from the branch.

@WaylandYang WaylandYang left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed the cleaned-up head. The cache key now covers the raw image reference, size, and NUL-delimited extras; the slug-collision regression is correct, the diff is scoped again, and CI is green. Approved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants