refactor(workflow): one file boundary, one batcher, shared readers, engine types in the designer

[huyplb]

The structural items left over from the workflow review (#530): one way to touch the filesystem, one batcher, one reader per format shared by run and preview, and designer types that come from the engine.

File boundary

• Before this change, every file pipe resolved its configured path inside the files root itself and then called node:fs directly. The rule that keeps a workflow editor from reading /etc/passwd only held while each new pipe remembered to make that one call, and forgetting it failed open.
• openWorkflowFile (pipes/utility/workflow-files.ts) now resolves the path once and returns the only handle that can read or write it. CSV, text, JSON, the CSV sink and the preview route all use it.
• pipes-fs-boundary.test.ts fails when anything else under pipes/ imports node:fs.

One batcher

• CSV, text and JSON each had their own reject/onInvalid batching, and they had drifted on:
  • where batches cut
  • what the final cursor named
  • how error messages read
• BatchEmitter in delimited.ts now does it once:
  • record cuts for CSV and JSON.
  • boundary cuts for text, so a line never straddles two batches.
  • The final cursor names the last position read. Rows refused at the end of a file are no longer read again on resume. JSON gets this fix too.

Preview and run read the same way

• The designer preview had its own copy of each source's read loop, and the copies had drifted. CSV numbered rows as preamble + header + index, which is wrong after any quoted field that spans lines.
• csvRows and TextRowReader are now shared by the run and the preview. CSV rows carry the physical line they start on.
• preview-run-agreement.test.ts checks that the two agree.

Designer types from the engine

• These now come from @foxschema/workflow-engine/definitions as type-only imports, using z.input where the designer writes something the engine parses:
  • the HTTP request model
  • column rules
  • credential kinds
  • saved pipes
• The swap exposed two more things lost on save, both fixed:
  • Retry policy: a pipe's retry settings beyond attempts (backoff, maxDelayMs, jitter, on) were dropped.
  • Column type: the field was typed as any string.

Tests

• batch-emitter.test.ts, workflow-files.test.ts (outside-root, symlink, write/append/read, head), pipes-fs-boundary.test.ts, CSV line numbers with multi-line quoted fields, preview CSV lines.
• The e2e round trip now covers retry, plus a new test that the trigger dialog stays on its section while you type.

Verification

• Full npx vitest run: three timing tests in files this PR doesn't touch fail under parallel load and pass when run alone.
• apps/web tsc: clean.
• eslint and security lint: clean.
• npm run build:app: passes.
• Bundle: 146.2 KB gzip.
• The e2e workflow tests pass in a real browser.
• Every new test was checked to fail against the code before this change.

🤖 Generated with Claude Code

---

Note

Medium Risk
Touches core file-source parsing, batching/checkpoint cursors, and workflow save serialization; filesystem boundary changes are security-sensitive but centralized and tested.

Overview
Unifies workflow file I/O, source parsing, and designer save behavior so preview, runs, and the canvas agree and stop stripping config.

Engine: File pipes must use openWorkflowFile (root-confined reads/writes); a boundary test blocks other pipes/ code from importing node:fs. CSV/text/JSON sources share BatchEmitter (record vs line-boundary batching, shared reject/cursor semantics) and shared readers csvRows / TextRowReader—CSV lines are now physical start lines (fixes multi-line quoted fields); preview uses the same paths as runs.

Designer: Saved pipes use engine pipeSchema / pipelineSchema so full retry policies (not only attempts) round-trip; HTTP models are engine types with token-refresh headers/body defaulted and carried through; delimited editors use engine column types.

Tests: New unit/integration coverage (batch emitter, workflow files, preview↔run agreement, CSV line numbers) and e2e extends designer round-trip for retry plus trigger-dialog section stability while typing.

^{Reviewed by Cursor Bugbot for commit 7746bb2. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cursor]

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: b4c56e57-ad40-4ecc-b78a-e3fc7f45eef1)