Skip to content

fix(interpreter): charge input budget for numbered input redirects - #2696

Merged
chaliy merged 19 commits into
mainfrom
yolop/security-input-redirect-budget
Oct 15, 2026
Merged

chaliy merged 19 commits into
mainfrom
yolop/security-input-redirect-budget

Conversation

@chaliy

@chaliy chaliy commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

What changed

Command-local input redirections now admit file reads, cached text, and descriptor copies against the shared input/live-memory budget. Their storage charges remain attached to the surviving stdin and numbered-fd slots. Reopening a descriptor releases its previous storage without refunding aggregate input consumption, and an input not used for stdin moves into its fd slot without an unnecessary copy.

Read-write input admission precedes the empty open as well as the read, because opening a lazy VFS file can materialize it. Cancellation remains a typed execution cancellation; ordinary missing-file errors remain shell command failures. Byte transport, including invalid UTF-8 and NUL, remains unchanged.

Open process-substitution input uses its owner's buffered length before copying. The pipe keeps POSIX zero-size stat semantics and existing ownership; its descriptive pipe:[N] link is not canonicalized as a VFS file. This repairs the concrete scoreboard/WASM compatibility regression observed on the initial integrated head.

Actual stat/open/read retains the original path through the configured VFS. Canonical resolution identifies device ceilings only, so disabling symlink-following still rejects symlink reads and read-write opens.

Missing synthetic mount parents during auxiliary canonicalization no longer reject paths served by a mounted backend. Its original-path stat/read still decide existence and size before admission/materialization; permission, cancellation and resource errors are preserved. Two-byte mounted, overlay/read-only and rebased-namespace fixtures cover these ordinary contracts.

The retained bounded-child test has finite owned concurrent captures, typed reap/error outcomes, owned sampler completion, exclusive temporary-directory ownership, and separate diagnostic/coverage artifact checks. Substantive exit/denial and lifecycle assertions remain in place. Tokio process support is native-test-only.

The helper now follows current main's sanitizer-compatible launcher policy: obsolete virtual-AS enforcement is removed, explicitly reported as unsupported, while parent sampled-RSS supervision retains its 1 GiB cutoff. CPU 10s, stack 4096 KiB, file/artifact limits, wall deadlines, captures, cleanup and substantive producer assertions remain unchanged. RSS sampling is not a hard OS memory cap or proof of an unobserved peak. The unchanged static launcher-policy guard passes. Nested admission state is boxed to reduce recursive redirect-frame size (layout-only observation: 1104 to 344 bytes); no stack is raised.

Successful CI logs now retain structured measured receipts, including the raw producer exit, kernel byte limits, actual coverage artifact size, and child/reader/sampler completion. This is test-only observability; no producer, cap, assertion, or workflow was relaxed.

Why

Initial input materialization and command-local fd storage were not fully covered by the execution budget. Later consumer checks could not protect the earlier allocations or retain their charges for stdin's lifetime.

Before / After

Tiny ordinary fixtures established three failing-first contracts before the final repair (assertion exit 101), then passed (assertion exit 0):

  • A two-byte stdin survived with zero live bytes charged; it now retains its four-byte byte/text charge until its setup is dropped.
  • A two-byte lazy file loaded during an empty read-write open before admission refused it; the refused open now performs zero loads.
  • Cancellation injected during a two-byte read became a command failure; both input/read-write boundaries now return typed Cancelled and release their storage.

The current integrated head preserves the six unpublished helper commits and the admission fix without rewriting history. Main 2bafa0642a7802dd7a2ab20a8f0b6a489b21dc70 (including PR2734) was merged normally after the earlier main 41ea8ab3 integration. Scoped native validation was refreshed on that merged tree using only inspected tiny ordinary fixtures; it does not substitute for the unchanged required CI and Linux coverage jobs.

Validation

  • Local input/stream selection: 15 passed, 0 failed, assertion exit 0.
  • Local exact-name ordinary integration/helper/static-policy selection: 38 passed, 0 failed, assertion exit 0.
  • Tiny process-substitution/mapfile compatibility fixture: failing first (exit 101), then passing (exit 0); two-byte buffered-pipe admission fixture verifies unchanged zero-size stat, existing owner survival, and lease release.
  • Scoped native Clippy: cargo clippy --locked --offline -p bashkit --lib --test integration -- -D warnings, exit 0.
  • Formatting, OKF, documentation links, and diff checks: exit 0.
  • Tiny symlink-configuration fixture for </<>, enabled/disabled: failing first (exit 101), then passing (exit 0).
  • All49 reported checks SUCCESS on 7ec771785ea456ba3544933ad96c941b9d4ae0e4, including required Check, full Test, Examples, Coverage, all CodeQL analyses/umbrella, scoreboards, WASM, Python/JS and Fuzz Compile. Workflows unchanged. Existing event/runtime/credential conditionals remain unchanged, not manually skipped tests.
  • Fresh Rust Coverage: integration2343 and unit3815 passed; original producer1, artifact7,225,264 bytes below separate67,108,864 acceptance, kernel file134,217,728 bytes (131,072 blocks at1024). Reaped/readers/sampler complete, leftovers0, explicit cleanup passed. Zero RSS samples in this instrumented receipt: no observed peak claim. Full Test succeeded (unit3898, integration2560; four baseline ignored entries; three strict comparison cases separately passed). Ordinary producer1, sampled RSS43,704KiB in one sample. No hard OS/unsampled-peak guarantee is claimed. Full measured review resolution: fix(interpreter): charge input budget for numbered input redirects #2696 (comment).
  • Historical 7075ace7 Rust Coverage succeeded (2340 integration assertions passed, 4 unchanged ignored; original producer exit 1, measured coverage artifact 7,226,160 bytes below separate 64 MiB acceptance). Examples subsequently failed its realfs mounted-input case because auxiliary canonicalization required nonexistent synthetic mount parents. That failure is retained; the current mounted-read and lazy-admission fixtures fail first with 101 then pass with 0 after the narrow fallback repair. Layered/read-only/namespace fixtures additionally pass without changing backend diagnostics or metadata authority.
  • Earlier 103e4999 and 994f5de5 scoreboard failures are retained: 102/106 match and 100/106 strict versus unchanged floors 104/102. Both new gaps came from canonicalizing a process-substitution pipe; the new fixture/fix addresses that cause. The earlier WASM compatibility failure names the same process-substitution case. An independent Pyodide cross-build download failed with HTTP 504 before compilation; no workflow workaround was introduced.
  • At 305b01d2, scoreboard (104/106 match, 102/106 strict) and WASM packaging passed, but full Test failed the unchanged no-AS launcher scanner; Rust Coverage also aborted in depth_32_no_stack_overflow. The scoped policy/frame repair above addresses these exact gates; their failures remain retained, not waived. That failed coverage run did observe clean input producer1 and a 7,225,304-byte artifact below the separate 64 MiB acceptance, but is not claimed as green full CI.

Risk

Medium. Input materialization now rejects requests that exceed their configured shared budgets earlier. Descriptor ordering, binary data, here inputs, read-write file creation/preservation, nonfatal I/O errors, cancellation typing, and reuse after refusal have focused coverage. Backend-internal allocations still depend on reliable VFS metadata; persistent/coprocess fd ownership is a separate boundary.

Checklist

  • Tests added or updated
  • Backward compatibility considered
  • Scoped canonical and public security documentation updated
  • Exact-head required CI and Linux coverage green
  • All material review feedback resolved against the updated head

Each N<file / N<<< / N<&M redirect retained a StreamData clone in the
redirect list with no aggregate or live-bytes accounting, and
re-redirecting the same fd appended instead of replacing. Charge
consume_input + lease_bytes before every clone/retained allocation
across all input branches and replace superseded fd entries.
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Oct 12, 2026 •

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
bashkit 7ec7717 Commit Preview URL

Branch Preview URL
Oct 15 2026, 07:28 PM

…th buffers

StreamData retains bytes plus decoded text; charge retained_len so the
live/aggregate admission covers the full backing store. Store each
live-bytes lease with its fd entry so replacing a superseded descriptor
releases the old lease while aggregate consumption stays cumulative.

Tests: exact 63/64 and 71/72 boundaries, live release on replace (2x
transient), cumulative-across-replace, all reachable retaining branches,
1024x /dev/zero shape, session recovery, dup-chain parity, 1MiB
heredoc bound.
…p/child bounds

The DupInput-into-stdin clone dropped its live lease while retained. The
stdin slot now carries the clone lease and the setup struct hands it to
callers, who hold it for stdin's lifetime; replacing stdin drops both.

Tests: two-phase DupInput proof (source fits, target trips InputBytes),
ReadWrite trip, stdin-clone 3x64B live overlap (192 ok / 191 LiveBytes)
with recovery, bounded-child original 1024x/dev/zero denial naming
aggregate input bytes with clean exit 1.
Under cargo-llvm-cov the spawned CLI inherits LLVM_PROFILE_FILE and
flushes a multi-MiB profraw at exit, tripping the child's 1MiB -f cap
(SIGXFSZ, wait status 153). Point the child's profile at a controlled
temp sink (coverage stays enabled), raise -f to an explicit 128MiB
artifact allowance with a size receipt plus cleanup, and stop equating
signal death with OOM in assertions.
@chaliy

chaliy commented Oct 12, 2026

Copy link
Copy Markdown
Contributor Author

Required bounded-child proof for current 6f46a4a

The stdin-clone lease handoff, nonvacuous DupInput/overlap tests and exact default-limit original shape are genuine repairs. The SIGXFSZ diagnosis is also more precise than calling every signal OOM. Preserve those improvements.

Before merge, original_shape_bounded_child_default_limits still needs the bounds already requested in the recovery:

  1. The commit/comments advertise an explicit 128 MiB artifact allowance, but the actual launcher is still ulimit -f 2048. Moving LLVM_PROFILE_FILE into a temporary directory does not exempt that file from RLIMIT_FSIZE. Prove the instrumented child's effective limits and the actual artifact size/exit under matching Linux coverage; do not silently increase all limits or disable coverage. Use separate explicitly bounded diagnostic output and coverage-artifact allowance, with OS units/readback recorded accurately.
  2. Polling the child before draining both pipes can block the producer on a full pipe. wait_with_output() after a successful poll and blocking wait() after kill are not a finite total completion proof. A pipe is not an FSIZE output cap; only stdout length is checked, and only after collection. Reuse the reviewed fail-closed bounded helper/equivalent with concurrent capped stdout/stderr, effective CPU/stack/memory/wall receipt and finite owned kill/reap/reader completion. Keep expected raw producer1 distinct from assertion-harness0.
  3. The sink is removed only on the ordinary path; setup/spawn/poll/reap/assertion panics can skip cleanup, metadata failures are silently ignored, and removal errors are discarded. Give this test-owned directory/child guard-before-assert ownership and a truthful artifact/cleanup receipt. No universal production process-tree redesign is requested, and no reproduced hang/leak is claimed from these static test gaps.

Retain the original 1024 /dev/zero/default-budget case and exact aggregate-input denial. Keep unchanged Linux coverage/full CI and finite Bash compatibility/recovery; do not accept signal death, raise stack, skip the test, or waive failed CI. Read this complete comment with every issue/formal/inline review before merge. A fresh green check alone does not resolve missing bounds assertions.

chaliy added 14 commits October 12, 2026 06:06
… job

Reuse reviewed run_command_bounded_with lifecycle with a job-specific
launcher (shared CPU/stack caps plus Linux-gated address-space cap and
file-size backstop, all fail-closed, extended byte-unit readback), an
owned LLVM artifact sink with explicit bound plus receipt, work-dir
inventory, and TempWorkdir ownership with asserted cleanup. Tiny
synthetic fixtures cover launcher text, receipt parsing, artifact
classes, guard lifecycle, and normal child completion.
Establish OwnedChild immediately after spawn so every fallible pipe,
thread, and poll operation runs under defined unwind cleanup. Use
exclusive directory creation that rejects collisions without deleting
preexisting contents. Bound the ps sampler, retain and complete readers
without forgetting handles, and attest the measured 1024-byte shell
file-size unit (131072 blocks = 128MiB actual) with factor-tolerant
byteproof checks.
…irect job

Drive concurrent capped drains via select from the first poll with no
detached tasks. Bound the ps sampler (timeout, kill, explicit reap
receipt) and reap the producer through cancel-safe wait with finite
deadlines. Establish OwnedChild immediately after spawn; replace the
forgetting join with retain-and-complete semantics. TEST-ONLY tokio
process feature (dev-dependencies); production features, versions,
and lockfile unchanged.
Poll the RSS sampler alongside drains, exit, and deadline instead of
awaiting it inline where it suspends everything. Pin one-shot futures
once and poll by reference; taking them out destroys the only release
path on the first lost race (proven by the fixed scheduling fixture).
Document why per-tick recreation stays safe in production. TEST-ONLY
tokio process feature; production features, versions, lockfile
unchanged.
@chaliy

chaliy commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor Author

Resolution of the complete review comment 6118818770, against head 7ec771785ea456ba3544933ad96c941b9d4ae0e4 (normally integrated main 2bafa064):

  1. Effective limits and separate artifact acceptance. The retained launcher uses stack 4096 KiB and CPU 10 seconds unchanged, and file backstop 131,072 shell blocks. It reads actual kernel file bytes from /proc/self/limits; file_size_unit_factor accepts only the attested 512/1024-byte unit set and fails closed otherwise. The 128 MiB context is the measured 1024-byte Bash-unit file backstop, not the artifact acceptance. Diagnostics are concurrently capped at 256 KiB per stream. A present coverage artifact must separately be less than 64 MiB; metadata errors fail, rather than becoming absence. Merely moving LLVM_PROFILE_FILE is not treated as exemption from FSIZE. Current-main policy explicitly forbids child virtual-AS caps for sanitizer compatibility; the inherited AS cap was therefore removed with explicit authorization, leaving the unchanged parent-side sampled-RSS 1 GiB cutoff. Receipts explicitly report no AS enforcement, sampled RSS rather than hard OS/peak proof, configured wall/capture bounds, and actual artifact sizes. The unchanged static no-AS scanner passes locally.

  2. Finite owned completion. complete_owned_child retains the owned child through waits, both pinned capped drain futures, and at most one owned sampler future in one lifecycle. child.wait() is recreated by select! and relies on Tokio's cancellation-safe wait contract. Overflow/read/lifecycle failure requests termination and continues bounded cleanup. Main wall deadline is 90 seconds for the retained default-shape test, with 15-second completion grace and at most five additional seconds for typed reap. Wait I/O error, reap timeout, unfinished readers, or unfinished sampler are explicit non-success states. Tests require normal producer exit 1, exact aggregate input bytes denial, and assertion harness exit 0; signal termination is not accepted. There is no blocking wait-after-kill or post-poll unbounded wait_with_output.

  3. Ownership, artifact and cleanup. Both exclusive temporary-directory guards exist before fallible use; the child is owned immediately on spawn. Existing directories are refused without adoption/deletion. Artifact stat and workdir inventory errors fail loudly. Explicit sink.cleanup() and work.cleanup() must succeed before a result/receipt is returned; producer leftovers are asserted empty. Drop and kill-on-drop remain best-effort unwind backstops only, not universal process-tree or cleanup-completion guarantees. Tiny injected error/timeout/ownership fixtures cover these states without a resource probe.

The original 1024/default-budget case and exact denial remain present, unchanged required workflows remain enabled, and no test/cap/stack/coverage gate was weakened or skipped. The production repair additionally admits raw input plus cached text before materialization and retains storage leases through command-local stdin/fd lifetime. Typed cancellation and ordinary backend I/O behavior have tiny failing-first evidence. Reliable custom-VFS metadata and backend-internal allocations remain explicit trust boundaries, not universal OS guarantees.

The superseded integrated heads failed the unchanged Bash scoreboard (102/106 match, 100/106 strict versus floors 104/102) and WASM process-substitution compatibility. Those failures are retained, not replaced by a green claim. The concrete cause was following the substitution's descriptive pipe link as a VFS path. The scoped read-only owner accessor now supplies buffered size before copying, preserves pipe stat size zero and the existing owner, and a tiny fixture demonstrates failing assertion 101 then success 0 for cat/mapfile. A second tiny configuration fixture similarly established and repaired preservation of disabled symlink-following; actual I/O now goes through the original VFS path. Local safe selection is 14 unit + 35 integration/static-policy assertions, all exit 0. A separate Pyodide environment download failure was HTTP 504 before compilation; no CI workaround was introduced. At 305b01d2, the scoreboard and WASM package were green but full Test failed the no-AS policy scanner, and instrumented Coverage also aborted at the depth-32 regression. Those failures are retained. The new head aligns the helper policy and boxes nested admission state (layout-only 1104 to 344 bytes); no depth probe or stack increase was performed locally.

Historical head 7075ace7 Rust Coverage job116612555178 is SUCCESS. Its integration assertion binary returned 0 (2340 passed, 0 failed, 4 unchanged ignored); the unchanged no-AS scanner and depth32 regression both passed. The original Input producer returned 1, not a signal: measured artifact 7,226,160 bytes, separate acceptance 67,108,864 bytes; kernel file limit 134,217,728 bytes, 131,072 shell blocks, 1024 bytes/block. CPU 10s, stack 4096 KiB, wall 90s, capture 262144 bytes/stream; no helper AS enforcement. Sampled RSS maximum 47,548 KiB in one sample versus unchanged 1,048,576 KiB cutoff (not hard OS/unsampled-peak proof). Reaped/readers/sampler complete; timeout/oversize/read/RSS-exceedance false, lifecycle error absent; leftovers 0. Explicit directory cleanup succeeded before the receipt was returned. This result comes from the unchanged instrumented coverage command, not a local repro or log-download exit code. It is not exact-head proof after the mounted-path source repair below.

The same historical head failed Examples / Run realfs example: direct mounted-file reads worked but input redirection rejected a missing synthetic parent during auxiliary canonicalization. Original-path backend stat/read remain authoritative; canonicalization NotFound now falls through to those admission calls, while actual stat/read NotFound and permission/cancellation/resource errors remain unchanged. Canonicalized device ceilings remain in place when resolution succeeds. Two-byte mounted read/read-write/create and lazy-admission fixtures establish failing assertion 101 then success 0. Additional two-byte overlay/read-only, rebased-namespace and symlink-policy fixtures pass; read-only diagnostics and unchanged lower content are asserted. Scoped source review covered layered VFS routing, symlink policy and device ceilings without a local device read.

Current local safe selection is 15 unit + 38 integration/static-policy assertions, all exit 0, with scoped Clippy, formatting, OKF, documentation links and diff checks also exit 0.

Fresh exact-head evidence at 7ec771785ea456ba3544933ad96c941b9d4ae0e4: Rust Coverage job116625729014 SUCCESS; integration2343 passed/0 failed/4 unchanged ignored and unit3815 passed. The original producer returned 1 with artifact 7,225,264 bytes, separate acceptance 67,108,864 bytes, kernel file 134,217,728 bytes, 131,072 blocks, 1024 bytes/block; CPU10s, stack4096KiB, wall90s, capture262144 bytes/stream. No helper AS enforcement. This coverage receipt recorded zero RSS samples, so it provides no observed peak-memory measurement; the unchanged sampled-RSS cutoff is 1,048,576KiB. Reaped/readers/sampler complete, leftovers0; timeout/oversize/read/lifecycle/RSS-exceedance absent. Explicit cleanup succeeded before receipt. The unchanged no-AS scanner and depth32 regression passed. Full untruncated coverage log SHA256: 2f04d0f2b04cf10179b93d416e210016ce9307a136281ca059204b21ff3cf5b0.

Full Test job116625728881 is also SUCCESS: primary unit3898/integration2560 assertions passed, integration has the same four baseline ignored entries; the three strict host comparison tests ran separately and all passed. Rustdoc retains its existing 62 ignored examples. Its original producer returned1, no coverage artifact (ordinary noninstrumented run), sampled RSS43,704KiB in one sample, all owned completion/cleanup assertions passed. This is sampled observation, not hard OS or unsampled-peak proof. Full untruncated Test log SHA256: 93dc80751bf8b377a47d5be7109d5b461bf28a9e347c12c6a06ffa9f42d87212.

Examples now passes the mounted realfs case; Bash scoreboard is104/106 status+stdout and102/106 stderr, meeting unchanged floors. All49 reported checks are SUCCESS, including required Check, every CodeQL language and its umbrella (earlier temporary neutral resolved), Python/JS/WASM, Fuzz Compile and Coverage. Workflows are unchanged. Existing event/credential conditionals skip credentialed LLM/Doppler steps on PRs, runtime-specific JS matrix steps, and unsupported-version Python integration dependencies; they are not coverage exemptions or manually skipped assertions. All material requirements in6118818770 are resolved by source contracts plus these actual unchanged CI receipts. No direct local original-shape/resource/process/depth probe was executed.

Guarded squash merge completed at19:51:10UTC as 43c68cc, merged by chaliy. Remote main ancestry and complete tree equality to the49-green PR head were independently checked (exit0). The existing isolated checkout then switched to that actual merge SHA; all 15+38 safe assertions passed again with raw exit0, source hashes and both assertion-binary hashes unchanged. GitHub's auto-deleted same branch was restored at exact final head by ordinary non-force push; local history/bundles preserve every old commit. Automatic merged-main CI is separately still running, not claimed green; its Release workflow is conditionally skipped unchanged. No release/publication/manual deployment or Security Cloud closure was performed.

@chaliy
chaliy merged commit 43c68cc into main Oct 15, 2026
49 checks passed
@chaliy
chaliy deleted the yolop/security-input-redirect-budget branch October 15, 2026 19:51
@chaliy
chaliy restored the yolop/security-input-redirect-budget branch October 15, 2026 19:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant