Skip to content

Add command builders, stream analytics, packages and development tools - #225

Merged
konard merged 8 commits into
mainfrom
issue-224-7434d5c60324
Oct 11, 2026
Merged

konard merged 8 commits into
mainfrom
issue-224-7434d5c60324

Conversation

@konard

@konard konard commented Oct 11, 2026 •

Copy link
Copy Markdown
Member

This adds the command builders, terminal buffer, command package ecosystem, stream processing, and development tools requested by all six child issues of #224. Existing ESM, CommonJS, lightweight ProcessRunner, zx, ShellJS, and Execa entry points remain covered by their compatibility suites.

Issue Delivered behavior
#1 $fy, command-stream fy, and fy(source) produce executable ES modules that preserve shell arguments, stdin, heredocs, control flow, and status. Rust provides the translator and CLI too.
#9 Independent writable terminal buffers, changed-frame subscriptions, resizing, and plain/colored text snapshots using the existing xterm/headless and vt100 emulators.
#30 Typed command packages; atomic install/reload/uninstall; composition, inheritance, middleware, wrapped generators, npm installation/search; ten installable example packages plus Git, deployment, ls-color, and enterprise workflows.
#31 Pull-driven map/filter/reduce/forEach, UTF-8 lines, bounded split/merge, windows/batches, live error-rate/latency/throughput statistics, monitoring/alerts, and measured buffered comparison. Native and generator pumps now respect stream capacity.
#32 Watch/restart and package reload; JavaScript/shell REPL and completion; browser pipeline debugger with stepping, data previews, flow, profiling, memory and timeline; packageable VS Code extension; command mocks, pipeline assertions and output snapshots.
#34 Immutable lazy exact-argv builders with option transforms, run/await/stream, exit-code selection and POSIX string rendering, with a Rust builder counterpart.

The complete requirement inventory, alternatives, primary-source research, and acceptance plan covers every issue and comment. None of the six issues was already completely resolved. Portable core APIs have Node/Bun/Rust feature examples; JavaScript module packages, the JavaScript evaluator and IDE/browser tools are documented as JavaScript integrations.

Reproduction and regression coverage:

  • The focused acceptance tests failed before implementing the missing APIs. See js/tests/advanced-command.test.mjs, advanced-stream.test.mjs, command-packages.test.mjs, development-tools.test.mjs, terminal-buffer.test.mjs, and the matching Rust integration tests.
  • experiments/issue-224/pipeline-pressure.mjs reproduces eager production through the old native pipeline: all 200 chunks were produced while downstream was paused. The integration test now verifies bounded production and preservation of all 3,276,800 bytes in Node and Bun.
  • advanced-integration.test.mjs verifies slow consumers retain output after child exit, cancellation interrupts an idle merged command, inherited stdin forwards EOF and stops waiting after child exit, invalid package reloads preserve working handlers, watcher shutdown/restarts, REPL behavior, debugger controls, extension providers, and all ten package examples.
  • Finite Node/Bun cancellation probes reproduce a native stage surviving mixed-pipeline cancellation; shared process-tree termination and forceful escalation now stop it. A worker capped at 16 MiB reproduces the editor regex stall; disjoint escape alternatives complete the same input.
  • A symlinked watch root reproduces macOS retaining old package handlers. Watch snapshots and package entry paths now use canonical identities; the regressions and independent Node/Bun probes verify reload through linked directories, including Node with --preserve-symlinks.
  • Preloading a disabled virtual registry reproduces the CI REPL/mock failures. All five new suites now use the existing cleanup hooks, and watched fixtures save atomically. The full Bun suite and the preload regression both pass. CI findings and exact log locations are preserved.
  • The finite memory probe processes 100,000 records (25.6 MB) in separate Node processes with a 128 MiB heap limit. Observed RSS growth was 13,369,344 bytes for streaming and 84,529,152 for buffering. These are single-run measurements, not a universal performance guarantee; methodology and results are preserved.

Local validation passed:

  • Full Bun suite: 2,688 passed, 10 skipped, 0 failed.
  • Rust cargo test --all-features: 838 passed, 5 ignored, 0 failed; formatting, Clippy across all targets/features with warnings denied, and rustdoc checks passed. Builds used one job and reduced debug data within the container memory budget.
  • Node runner/terminal/CJS regressions: 104 passed. Node zx/Execa compatibility and production artifact tests: 524 passed, 2 skipped.
  • ESLint, repository Prettier, strict ESM/CJS declarations, duplication gate (2.57%), frozen lockfile installation, npm packaging, and VSIX packaging passed.
  • All 34 Node/Bun/Rust executable feature examples passed parity; generated documentation is current. All 211 tracked-documentation checks passed.
  • Browser verification exercised Pause/Step/Resume and error inspection; the browser was closed afterward. The new debugger has no previous UI to compare against.

Pipeline debugger showing stream flow, inspected data and profiling

Final verification on 417df1d: all eight GitHub workflows pass, including JavaScript, Rust, and Security/CodeQL. The separate CodeQL findings check passes, and both earlier alerts are fixed. The matrices cover Node 22/24/26, Bun on Linux/macOS/Windows, and Rust on Linux/macOS/Windows. Conditional release jobs skip for pull requests.

  • Reviewed the final diff and all comment types; verified fresh CI timestamps and the exact head SHA; preserved and fixed the failing-log findings.
  • Verified the latest main is included, the working tree is clean, the complete scope and documentation agree, and PR 225 is ready for review.

Behavior and release notes: translation delegates shell semantics to the original interpreter and requires that shell plus command-stream; it is not an AST-to-JavaScript compiler. Existing result-returning virtual handlers and explicit capture/composition retain complete input/output strings; use native stages, generators and capture:false for bounded processing. Package entry reload preserves relative imports while imported dependencies retain their runtime cache; a process restart reloads the whole graph. Ten example packages and the extension are ready for independent publication; published-package counts, extension downloads, adoption and user feedback are post-release success metrics and are not claimed as achieved. A JavaScript minor changeset and Rust minor release fragment prepare the library releases without manually changing release-managed versions.

Fixes #224
Fixes #1
Fixes #9
Fixes #30
Fixes #31
Fixes #32
Fixes #34

Adding .gitkeep for PR creation (default mode).
This file will be removed when the task is complete.

Issue: #224
@konard konard self-assigned this Oct 11, 2026
@konard konard changed the title [WIP] Address all 6 open issues in link-foundation/command-stream Add command builders, stream analytics, packages and development tools Oct 11, 2026
Comment thread extensions/vscode-command-stream/src/extension.cjs Fixed
Comment thread js/tests/advanced-integration.test.mjs Fixed
@konard
konard marked this pull request as ready for review October 11, 2026 07:53
@konard

konard commented Oct 11, 2026

Copy link
Copy Markdown
Member Author

Working session summary

Implemented shell translation, terminal buffers, command packages, stream analytics, development tools, and safe command builders in PR #225, now ready for review.

The complete analysis lists every requirement, alternatives, implementation plans, library research, and codebase coverage.

Validation passed: 2,688 Bun tests, 838 Rust tests, 104 Node core checks, and 524 compatibility tests. All eight CI workflows and CodeQL pass. The branch is clean and includes latest main.

$fy uses the original shell interpreter. Package publication and adoption remain post-release outcomes.


This summary was automatically extracted from the AI working session output.

@konard

konard commented Oct 11, 2026

Copy link
Copy Markdown
Member Author

🤖 Solution Draft Log

This log file contains the complete execution trace of the AI solution draft process.

💰 Cost estimation:

  • Model: GPT-6.1 Sol
  • Provider: OpenAI
  • Public pricing estimate: $11.344620

📊 Context and tokens usage:

  • 839.3K input tokens across requests (cumulative, larger than the 272K context window, so not one request's context), 193.9K / 128K (151%) output tokens

Total: (839.3K + 25.4M cached) input tokens, 193.9K output tokens, $11.344620 cost

🤖 Models used:

  • Tool: OpenAI Codex
  • Requested: gpt-6.1-sol
  • Thinking level: xhigh (~31999 tokens)
  • Model: GPT-6.1 Sol (gpt-6.1-sol)

📎 Log file uploaded as Gist (17000KB)


Now working session is ended, feel free to review and add any feedback on the solution draft.

@konard
konard merged commit a64c64c into main Oct 11, 2026
55 checks passed
@konard

konard commented Oct 11, 2026

Copy link
Copy Markdown
Member Author

🎉 Auto-merged

This pull request has been automatically merged by hive-mind.

  • All CI checks have passed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants