diff --git a/CHANGELOG.md b/CHANGELOG.md index 3a67481..f78a51a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,26 @@ # Changelog +## [Unreleased] + +### Added + +- **`prepare_inputs` takes the method three ways.** Beside inline `files`, it accepts a `method_ref` address (resolved by the runner) or a stored `method_id` (resolved by the hosted platform) — exactly one per call, all three server-resolved, nothing expanded client-side. Empty is absent (`files=[]`, a blank `method_ref` / `method_id`) but the wrong type is not: none, several, or a non-string selector raises `InputPreparationError` before any request leaves the process — reading a mistyped selector as absent would let the exclusivity check pass and prepare against the wrong method. A non-string `pipe_ref` is refused on the same boundary, rather than being absorbed by the pipe defaulting. A method addressed by URL that declares a file input now has an input-preparation path; previously it had none, even though the request beneath accepted the address. +- **`PipelexValidationReport.default_pipe_ref`** — the qualified `pipe_ref` a caller gets by omitting the pipe selector, or `null` when the closure declares none or several. Optional and read leniently: a runner that predates the field sends nothing, and `prepare_inputs` falls back to the opaque `bundle_blueprint.main_pipe`. + +### Changed + +- **Breaking: `prepare_inputs` reads its signature from the input-form descriptor, not the inputs template.** It composes one `POST /v1/validate` with `views: ["input_form"]` and `allow_signatures=True`, and walks the standard's `InputForm` artifact — `document` / `image` mark a file position, `object` recurses through `fields`, `list` through `item`, everything else passes through. Source-compatible for every caller passing `files`; the SDK no longer calls `/v1/build/inputs` at runtime. A valid report carrying no descriptor is an error naming the pipelex-api floor, never a silent degrade to "no uploads". +- **Breaking: `build_inputs` and its models are removed.** `client.build_inputs`, `BuildInputsRequest`, `BuildInputsValidReport`, `BuildInputsResponse`, `BuildInputsResponseAdapter` and `InputsTemplateFormat` are gone — the route wrapper existed only to be the signature source `prepare_inputs` read, and nothing calls it now. This is the Python SDK's step of the workspace program retiring `/v1/build/*`; a caller that still needs a fill-in template projects one from the descriptor with `mthds.protocol.inputs_template` (`render_inputs_template` / `project_inputs_template`, in both the compact and explicit shapes, as JSON or TOML), which is also where `InputsTemplateFormat` now lives. The wrapper is not a capability lost but one relocated to the standard's own package — and projected client-side, so a method reached by `method_ref` or `method_id` gets a template with no server round-trip at all. +- **Breaking: the shared crate envelope moved to `pipelex_sdk.crate_models`.** `MthdsFileItem`, `CrateRequestBase` and `CrateInvalidReport` now live beside the routes that use them (`/v1/resolve`, `/v1/codegen`) and `pipelex_sdk/build_models.py` is deleted — a module named for the build routes could not go on holding the envelope after they left. The models themselves are unchanged; update the import path. +- **Breaking: a canonical file dict nested inside a `Dynamic` input is no longer uploaded.** Such an input is `kind: "unknown"` in the descriptor — the standard's escape hatch — and the walk does not enter it. Uploading on the strength of a `url` key is the value-shape guess this change removes; a caller with a Dynamic input uploads with `upload_file` first and passes the storage URI, which `docs/input-preparation.md` has always prescribed. +- **`prepare_inputs` accepts the explicit `{concept, content}` input envelope**, not only compact values, closing a parity gap with the JS SDK. An agent that fills an explicit template — the shape the hosted console and MCP hand out — can now hand it straight back; previously every file-bearing envelope position raised `InputPreparationError: Unsupported value at a file input … got dict`. The envelope's `content` is interpreted identically and preserved on output, so the concept annotation rides through to the run. +- **The documentation is rewritten around the descriptor.** `docs/input-preparation.md` now describes the three call shapes, the signature call, pipe selection and its manifest-only `main_pipe` gap, the envelope, and why the template was the wrong signature source; `docs/architecture.md` follows the removal. It also catches up with v0.9.0, which shipped `output_form` and the `mthds` 0.13.0 bump without touching the docs: the views list is no longer described as having one token, the report's typed fields now include `output_form` and `default_pipe_ref`, and the pipe I/O contracts no longer claim an output carries no schema — 0.13.0 made `json_schema` required there, reversing the reasoning the doc still quoted. + +### Security + +- **An optional nested file field is now uploaded.** The required-only inputs template never rendered one, so its file position was invisible and the caller's local path travelled to the runner as a literal string. The descriptor states `required: false` and the walk enters it. +- **A text field merely *named* `url` is no longer read from disk.** The template marked a file position by rendering a `url`-bearing dict — a side effect of the field's *name*, not of its concept — so a path-shaped text value was uploaded. `kind: "text"` ends that. + ## [v0.9.0] - 2026-09-02 ### Added diff --git a/docs/architecture.md b/docs/architecture.md index 3f8231c..02be059 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -31,7 +31,7 @@ MTHDS is the brand of the open standard (the language, the protocol). Pipelex is The Pipelex narrowing of the `/v1/validate` verdict union is one such implementation envelope and lives here (`pipelex_sdk.validation_models`): `PipelexValidationReport` / `PipelexInvalidReport` / the `PipelexValidationResult` union, plus the supporting `ValidationErrorItem` / `ValidationErrorCategory` / `ValidatedPipeEntry` / `DryRunStatus` / `LiftablePipeEntry` / `SuggestedFix` / the `FixOp` variants / `FixOpKind` / `FixSafety`. They narrow the neutral `ValidationReport` / `InvalidValidationReport` / `ValidationResult` bases that `mthds` keeps (in `mthds.protocol.models`). The report/union types carry the `Pipelex` prefix; the supporting types stay neutrally named — branding the envelope, not the fields inside it. The brand-neutral `Dict*` wire concretes (`DictRunResultExecute` and friends) stay in `mthds` — they are a shared wire contract the `pipelex` runtime itself builds on — and this SDK reuses them by inheritance rather than redefining them (a deliberate divergence from `pipelex-sdk-js`, which duplicates both the `Dict*` and the `Pipelex*` types in its own `models.ts`). -The same boundary decides two members *inside* the Pipelex envelope. The input-form descriptor and the pipe I/O contracts are MTHDS artifacts — the standard's own recommended extension fields of the validate report, each with a normative page — so this SDK types them by importing `mthds.protocol.input_form` and `mthds.protocol.pipe_io_contracts` rather than declaring them, and does not re-export them under its own name. A Pipelex-branded envelope may carry a neutral artifact; it may not adopt it. See "Typed by import" below. +The same boundary decides several members *inside* the Pipelex envelope. The input- and output-form descriptors and the pipe I/O contracts are MTHDS artifacts — the standard's own recommended extension fields of the validate report, each with a normative page — so this SDK types them by importing `mthds.protocol.input_form`, `mthds.protocol.output_form` and `mthds.protocol.pipe_io_contracts` rather than declaring them, and does not re-export them under its own name. A Pipelex-branded envelope may carry a neutral artifact; it may not adopt it. See "Typed by import" below. ## Credentials & configuration @@ -143,25 +143,26 @@ The protocol `validate` is **overridden** (not inherited) to add the Pipelex-API - **Markdown render is always injected.** `validate(...)` adds `"markdown"` to the `render` list (de-duplicated, caller tokens first) so both a valid `PipelexValidationReport` and a produced `PipelexInvalidReport` carry `rendered_markdown`. Unknown render tokens are server-side lenient-ignored. - **`mthds_sources`** is a named parameter (parallel to `mthds_contents`) threaded onto each diagnostic's `source`; sent only when provided. -- **`views`** is the opt-in for the server's structured views, on both `validate` and `validate_files`. `input_form` — named by the `VALIDATION_VIEW_INPUT_FORM` constant — is the only token today. Unlike `render`, the list travels **verbatim**: nothing is injected, nothing is de-duplicated, and an explicit `[]` is sent as `[]`, because the server resolves the tokens as a set and lenient-ignores the ones it does not know (never a `422`), so client-side normalization would only hide what the caller asked for. Left at `None` the key is not sent at all, which is what keeps the default response byte-identical for the consumers that discard views (hook pipelines, CI gates, agent loops). The constant is a constant rather than a closed enum on purpose: the request boundary is deliberately open, so a stale token never fails a call. +- **`views`** is the opt-in for the server's structured views, on both `validate` and `validate_files`. The tokens are named by constants — `VALIDATION_VIEW_INPUT_FORM` for `input_form` and `VALIDATION_VIEW_OUTPUT_FORM` for `output_form`, the descriptor of what the pipe produces. Unlike `render`, the list travels **verbatim**: nothing is injected, nothing is de-duplicated, and an explicit `[]` is sent as `[]`, because the server resolves the tokens as a set and lenient-ignores the ones it does not know (never a `422`), so client-side normalization would only hide what the caller asked for. Left at `None` the key is not sent at all, which is what keeps the default response byte-identical for the consumers that discard views (hook pipelines, CI gates, agent loops). The constant is a constant rather than a closed enum on purpose: the request boundary is deliberately open, so a stale token never fails a call. - **`validate_files(files, …)`** takes `MthdsFile(content, uri?)` records. When any file carries a URI, every content gets a parallel source label — the named file's URI, or a deterministic `inline://file-N.mthds` for an unnamed sibling — so the server never sees a length-mismatched `mthds_sources`. The override reuses the inherited base transport seam `_post_validate` (which builds the body — passing `render` / `mthds_sources` through the protocol's `extra` extension passthrough — sends the request, and raises on a no-verdict non-2xx), then parses the raw 200 body into this SDK's own `PipelexValidationResult` via `PipelexValidationResultAdapter`. Body-building and transport stay shared with the base; only the Pipelex presentation/sources concerns and the branded narrowing live here. The validation models (`PipelexValidationResult` = `PipelexValidationReport | PipelexInvalidReport`, with `rendered_markdown`) are **owned by this SDK** (`pipelex_sdk.validation_models`); they narrow `mthds`'s neutral verdict bases, completing the brand boundary (the resolved follow-up #9). The base `MthdsAPIClient.validate()` returns the neutral `ValidationResult` instead. **Checkpoint-5 decision (validate error regime):** the delegation keeps the inherited `httpx.HTTPStatusError` regime on a *no-verdict* non-2xx, where the JS `validate` raises `ApiResponseError`. Kept as-is (deferred parity), because in both SDKs `validate`'s error regime matches the *other* protocol routes of that SDK — JS routes all raise `ApiResponseError`, Python protocol routes all inherit `httpx.HTTPStatusError` (decision #5). Making Python's `validate` alone raise `ApiResponseError` would make it inconsistent with `execute`/`start`/`models`/`version`, which is worse than the JS divergence. The verdict itself (valid/invalid) is always a 200 either way — only the no-verdict failure *presentation* differs. -**What the report carries.** A valid `PipelexValidationReport` adds three typed fields beyond the protocol base. `warnings: list[ValidationErrorItem]` are advisory lints on a bundle that is nonetheless valid — the same item type as `validation_errors[]`, so one parser serves both channels, but they never flip `is_valid` (this is where the `hint_*` error types ride). `liftable_pipes: list[LiftablePipeEntry]` inventories the pipes the runtime may skip when an optional slot resolves absent. `input_form: InputForm | None` carries the per-pipe input-form descriptors, keyed exactly like `pipe_io_contracts`, and is present only when the request named the `input_form` view. The two lists default empty and `input_form` defaults `None`, so a body from an older runner still parses; the empty default is also what a clean bundle yields, so no caller can tell the two apart. `PipelexInvalidReport` gains none of them: `warnings` and `input_form` derive from a crate that was never assembled. +**What the report carries.** A valid `PipelexValidationReport` adds typed fields beyond the protocol base. `warnings: list[ValidationErrorItem]` are advisory lints on a bundle that is nonetheless valid — the same item type as `validation_errors[]`, so one parser serves both channels, but they never flip `is_valid` (this is where the `hint_*` error types ride). `liftable_pipes: list[LiftablePipeEntry]` inventories the pipes the runtime may skip when an optional slot resolves absent. `input_form: InputForm | None` and `output_form: OutputForm | None` carry the per-pipe input- and output-form descriptors, keyed exactly like `pipe_io_contracts`, each present only when the request named that view. `default_pipe_ref: str | None` is the qualified `pipe_ref` a caller gets by omitting the pipe selector, `None` when the closure declares none or several — manifest-aware for a fetched package, which is what makes it outrank a `bundle_blueprint` read; `docs/input-preparation.md` walks the fallback ladder `prepare_inputs` runs when a runner predates it. The lists default empty and the optionals default `None`, so a body from an older runner still parses; an empty list is also what a clean bundle yields, so no caller can tell the two apart. On the view fields the same default says something stronger: an opt-in view's absence means the request did not ask for it, never that the method has nothing to describe. `PipelexInvalidReport` gains none of them — they all derive from a crate that was never assembled. `ValidationErrorItem` gains `missing_pipe_code` (symmetrical with `missing_concept_code`) and `suggested_fix: SuggestedFix | None` — a deterministic repair proposal with a `fix_code`, a `description`, a `FixSafety` (`safe` / `unsafe`, with an `is_safe` property), an optional `source`, and `ops`: a list discriminated on `kind` over the closed `FixOpKind` vocabulary (`set_key`, `ensure_table`, `delete_key`, `delete_table`, `rename_table_key`, `move_key`, `remap_value`), narrowed with an exhaustive `match op: case SetKeyOp(): …`. The ops are **reader** models here: `extra="allow"`, no `frozen`, none of the runtime's wildcard-refusing validators, because this SDK only reads fixes where the runtime plans them. A `kind` this SDK does not know fails the whole verdict parse, deliberately and consistently with `ValidationErrorCategory`. `error_type` stays an open `str`: the runtime union keeps gaining advisory members, and closing it here would turn every runtime addition into an SDK break. -### Typed by import: the descriptor and the pipe I/O contracts +### Typed by import: the descriptors and the pipe I/O contracts -Two members of the valid arm are the **standard's** artifacts rather than Pipelex's, and this SDK narrows them by importing the standard's own client models instead of restating their shape: +Several members of the valid arm are the **standard's** artifacts rather than Pipelex's, and this SDK narrows them by importing the standard's own client models instead of restating their shape: -- **`pipe_io_contracts: PipeIOContracts`** — `dict[pipe_ref, PipeIOContract]` from `mthds.protocol.pipe_io_contracts`. An input slot reads as typed members: `concept_ref`, a three-valued `presence` (`PresenceMarker`, so an authored `!` is not flattened into a boolean), a `multiplicity` (`IOMultiplicity`), the `item_count` that is non-null exactly on the fixed arm, and the slot's `json_schema`. The output side is deliberately asymmetric — a two-valued `optional` and no schema — because `!` is rejected on an output and the payload a run produces is the run's own result. +- **`pipe_io_contracts: PipeIOContracts`** — `dict[pipe_ref, PipeIOContract]` from `mthds.protocol.pipe_io_contracts`. An input slot reads as typed members: `concept_ref`, a three-valued `presence` (`PresenceMarker`, so an authored `!` is not flattened into a boolean), a `multiplicity` (`IOMultiplicity`), the `item_count` that is non-null exactly on the fixed arm, and the slot's `json_schema`. The output side stays asymmetric in exactly one place: a two-valued `optional`, because `!` is a use-site assertion about an input and is rejected on an output. It is no longer asymmetric on the schema — since `mthds` 0.13.0 an output carries a required `json_schema` too, and the two answer different questions. An input's describes what a caller **sends**, so a plural slot's is a bare array; an output's describes what **comes back**, which is the concept's content model, so a plural output's is that model's list envelope. Both are declared facts, knowable before any run happens. - **`input_form: InputForm | None`** — `dict[pipe_ref, PipeInputFormDescriptor]` from `mthds.protocol.input_form`. A descriptor's `fields` are the recursive `InputFormField` union discriminated on `kind`; narrow a node with `match node: case ListField(): …` or an `isinstance` check, importing the per-kind models from `mthds.protocol.input_form`. An `object` node recurses through `fields`, a `list` node through `item` — and the item changes layer. Since `mthds` v0.10.0 the union is split by whether a node names itself: a top-level field is the named union (`TextField`, `DocumentField`, …, each requiring `name: str`), a `ListField.item` is the nameless one (`TextItem`, `DocumentItem`, …), which refuses a `name` at the parse. Narrow a list's item to `DocumentItem`, never `DocumentField`; because each `*Field` derives from its `*Item`, the item layer is the only safe narrowing target in that position. +- **`output_form: OutputForm | None`** — `dict[pipe_ref, PipeOutputFormDescriptor]` from `mthds.protocol.output_form`, the twin of the above on the other side of the pipe. It carries a single `field` rather than a `fields` list, since a pipe has exactly one output, and reuses the same `InputFormField` node union verbatim rather than declaring a second one that could drift — so a renderer walks it with the patterns it already has. It states no `presence` and no `gating`: those are facts of a slot a caller fills, and a result is not one; plurality is stated by wrapping the field in a `list` node. Read it together with that pipe's `output.json_schema` off `pipe_io_contracts` — the descriptor says what the result IS, the schema names the property its payload arrives under, and a consumer holding one but not the other is back to inferring the other from the value. -**Why import rather than declare.** These artifacts belong to MTHDS: they describe a method's inputs, which is a language-level fact, and any engine derives them from a resolved library with no Pipelex API in the loop. They were carried opaquely until now for a reason that has since expired — when that call was made, no published Python package declared them, so "type it here" could only mean "copy it here", and a copy is free to drift from the runtime that emits it. Since `mthds` 0.9.0 the standard's own client declares both, so typing them means importing them: one declaration per language, nothing to drift from. The principle the opaque ruling was protecting — this SDK is transport and does not own these types — is what an import preserves and a restatement would have broken. +**Why import rather than declare.** These artifacts belong to MTHDS: they describe what a method takes and what it produces, which is a language-level fact, and any engine derives them from a resolved library with no Pipelex API in the loop. They were carried opaquely until now for a reason that has since expired — when that call was made, no published Python package declared them, so "type it here" could only mean "copy it here", and a copy is free to drift from the runtime that emits it. Since `mthds` 0.9.0 the standard's own client declares them, so typing them means importing them: one declaration per language, nothing to drift from. The principle the opaque ruling was protecting — this SDK is transport and does not own these types — is what an import preserves and a restatement would have broken. The types are imported and used, never re-exported from `pipelex_sdk`. Re-exporting would put this package's name on a vocabulary it does not own and hand consumers a second import path to drift against; import them from `mthds.protocol` directly. @@ -175,9 +176,9 @@ The types are imported and used, never re-exported from `pipelex_sdk`. Re-export `resolve` and `codegen` (`pipelex_sdk/crate_models.py` + the two client methods) are the second crate-family surface, mirroring the JS SDK's v0.12.0 routes: `POST /v1/resolve` emits the normalized library crate (the MTHDS Library Crate Format — typed as opaque transport, `dict[str, Any]`, because the crate schema is owned by the standard and a restatement here would be free to drift), and `POST /v1/codegen` projects that crate through the `kind` × `target` axes into stamped typed artifacts plus their `codegen.lock` (write both verbatim and the offline `pipelex codegen check` passes on the tree). Both follow the build routes' 200-verdict discipline, sharing `CrateInvalidReport`, and ride `_request_product`, so a no-verdict condition raises the typed `ApiResponseError`. -Their closure selector is the strict three-way XOR — inline `files`, an address-form `method_ref` (server-resolved; the registry form keeps its `501`), or the hosted `method_id` pass-through — enforced at request construction (`CrateToolingRequest._exactly_one_selector`) as well as by the server. `BuildInputsRequest` shares the `CrateRequestBase` envelope (`files` XOR `method_ref`) but takes NO `method_id`: the `/v1/build/*` projections are deliberately excluded from the hosted tooling selector, and a `method_id` handed to it raises a teaching error naming the migration (fetch the method with `get_method` and pass its source as `files`) rather than letting pydantic silently ignore the key. +Their closure selector is the strict three-way XOR — inline `files`, an address-form `method_ref` (server-resolved; the registry form keeps its `501`), or the hosted `method_id` pass-through — enforced at request construction (`CrateToolingRequest._exactly_one_selector`) as well as by the server. The shared envelope itself (`MthdsFileItem`, `CrateRequestBase`, `CrateInvalidReport`) lives in `crate_models.py` beside the routes that use it; it sat in a `build_models.py` module until `prepare_inputs` moved onto the input-form descriptor and the `/v1/build/inputs` wrapper was removed with it. -**A `method_ref` closure gets a fetch-sized budget.** Resolving an address can make the server clone a repository before it answers, and the server-side clone timeout runs well past the 30s management budget on a cold cache — an abort there would report a healthy, still-cloning server as unreachable. So a `method_ref`-carrying `build_inputs` / `resolve` / `codegen` uses an internal 3-minute budget (`_METHOD_REF_FETCH_TIMEOUT_SECONDS`, threaded through `_request_product`'s `request_timeout` override); it is internal (no new caller-facing parameter) and inert behind the hosted gateway's own cap. Mirrors the JS SDK's fetch budget; the run routes and `validate` need none because they already ride the 20-min blocking ceiling. +**A `method_ref` closure gets a fetch-sized budget.** Resolving an address can make the server clone a repository before it answers, and the server-side clone timeout runs well past the 30s management budget on a cold cache — an abort there would report a healthy, still-cloning server as unreachable. So a `method_ref`-carrying `resolve` / `codegen` uses an internal 3-minute budget (`_METHOD_REF_FETCH_TIMEOUT_SECONDS`, threaded through `_request_product`'s `request_timeout` override); it is internal (no new caller-facing parameter) and inert behind the hosted gateway's own cap. Mirrors the JS SDK's fetch budget; the run routes and `validate` need none because they already ride the 20-min blocking ceiling. ## Pipelex product surface (hosted management routes) @@ -215,7 +216,7 @@ The wire models are snake_case Pydantic v2. Response models are extension-open ( ## Out of scope -- The remaining `/v1/build/*` authoring helpers — `build_output`, `build_runner`, `concept`, `pipe_spec`. `build_inputs` shipped in 0.5.0 and is no longer deferred. +- The `/v1/build/*` helpers — `build_output`, `build_runner`, `concept`, `pipe_spec`. `build_inputs` shipped in 0.5.0 and was removed again once `prepare_inputs`, its only caller, moved onto `validate` + the input-form descriptor: this SDK no longer touches `/v1/build/*`, which the workspace is retiring (`wip/build-retirement/`). - Organization *switch* (a WorkOS session operation, not a `/v1` route). - A `~/.pipelex/config` file reader (env-only for now, matching the JS SDK). - A synchronous client facade. @@ -229,12 +230,12 @@ The wire models are snake_case Pydantic v2. Response models are extension-open ( This SDK is a port of the TypeScript `@pipelex/sdk` (`PipelexApiClient`) and tracks it closely, but it is **not surface-complete, and this section is where the gaps are named.** The Checkpoint-5 parity audit walked the JS `src/client.ts`, `src/index.ts` (the public barrel), and `docs/architecture.md`, plus a field-by-field sweep of `runs.ts` / `product-models.ts` / `models.ts`; it concluded surface-completeness, and that conclusion went stale as the JS SDK grew. The honest list of what has no Python counterpart today: - **Tooling routes** — `lint`, `format` (`resolve` and `codegen` shipped in 0.8.0 with the method selectors and are **not** gaps). -- **Authoring helpers** — `build_output`, `build_runner`, `concept`, `pipe_spec` (`build_inputs` shipped in 0.5.0 and is **not** a gap). +- **Authoring helpers** — `build_output`, `build_runner`, `concept`, `pipe_spec`. JS still exports its `buildInputs` wrapper; Python's was **deleted** rather than left unused, because `prepare_inputs` was its only caller. A deliberate divergence, not a gap: the JS wrappers retire together as their own step of the same program. - **Offline helpers** — `run_codegen_check` (the codegen drift check) and `get_method_closure` (client-side sugar that parses the polymorphic `mthds` source into a run-ready closure — in the JS SDK it is the documented migration target for the deleted by-id expansion legs; this SDK never had such legs, so the utility stays deferred rather than required). Each stays deferred rather than silently missing. Everything else — the protocol routes, the durable lifecycle, the whole product surface, and the errors — does have a Python equivalent. -**Methods** — everything outside the gap list above has a counterpart: protocol (`execute`, `start`, `validate`, `validate_files`, `models`, `version`), durable lifecycle (`get_run_status`, `get_run_result`, `wait_for_result`, `start_and_wait`, the private `_supports_run_lifecycle` / `_execute_blocking`), the whole product surface (profile, methods CRUD with paged listing and the two iterators, organizations, billing, Pipelex API keys, gateway key, onboarding, storage, run records with `get_run_detail`), `build_inputs`, the crate routes (`resolve`, `codegen`), the input-preparation surface (`upload_file` / `prepare_inputs`), and `health`. The method selectors (`method_ref` / `method_id`) match the JS v0.16.0 surface across the run and tooling methods, with one signature-shape divergence: JS `validate` takes a `ValidateMethodSelector` object in place of its first argument, while Python takes `method_ref=` / `method_id=` keyword parameters — same wire, same XOR, idiomatic per language. +**Methods** — everything outside the gap list above has a counterpart: protocol (`execute`, `start`, `validate`, `validate_files`, `models`, `version`), durable lifecycle (`get_run_status`, `get_run_result`, `wait_for_result`, `start_and_wait`, the private `_supports_run_lifecycle` / `_execute_blocking`), the whole product surface (profile, methods CRUD with paged listing and the two iterators, organizations, billing, Pipelex API keys, gateway key, onboarding, storage, run records with `get_run_detail`), the crate routes (`resolve`, `codegen`), the input-preparation surface (`upload_file` / `prepare_inputs`, taking all three method selectors and reading its signature from the input-form descriptor), and `health`. The method selectors (`method_ref` / `method_id`) match the JS v0.16.0 surface across the run and tooling methods, with one signature-shape divergence: JS `validate` takes a `ValidateMethodSelector` object in place of its first argument, while Python takes `method_ref=` / `method_id=` keyword parameters — same wire, same XOR, idiomatic per language. **Models** — field-for-field across the run-lifecycle types and the product wire models. Deliberate idiomatic ports (not gaps): milliseconds → seconds (`interval_seconds` / `timeout_seconds` / `elapsed_seconds`); the JS `AbortSignal` → Python `asyncio` cancellation (no `signal` field); JS inline string-unions promoted to `StrEnum`s (`OrgRole`, `PipeStatus`, the onboarding fields) with identical wire values; response models are `extra="allow"` for forward-compat. The Pipelex validation narrowing is **owned here** (`pipelex_sdk.validation_models`), narrowing `mthds`'s neutral verdict bases (the resolved follow-up #9); the brand-neutral `Dict*` wire concretes (`DictRunResultExecute`) are reused from `mthds` by inheritance — they are a shared wire contract the `pipelex` runtime also builds on — rather than duplicated as `pipelex-sdk-js` does. Two divergences worth naming: the page envelopes keep the wire's snake_case `next_cursor`, where the JS mirror renamed it `nextCursor` for its own consumers; and the method-files catalog converter (`parse_method_files` / `serialize_method_files`) lives in this package, where the JS pair lives in `mthds-js` because `pipelex-mcp` consumes the same format and wanted one owner. There is no second Python consumer, and the catalog serialization is a Pipelex product concern rather than an MTHDS protocol one, so this SDK is a proper home for it. If `mthds-python` ever grows an owner for the format, this SDK adopts it then. diff --git a/docs/input-preparation.md b/docs/input-preparation.md index 6d65eed..43929c4 100644 --- a/docs/input-preparation.md +++ b/docs/input-preparation.md @@ -1,8 +1,8 @@ # Input preparation (`upload_file` / `prepare_inputs`) -> **Status: implemented** (`pipelex_sdk/upload.py`, `pipelex_sdk/prepare_inputs.py`, `pipelex_sdk/build_models.py`). This document records the contract (design source: `wip/upload/README.md` in the workspace, tracked in `TODOS.md`). `upload_file` and `prepare_inputs` are the Python counterpart of `@pipelex/sdk`'s `uploadFile` / `prepareInputs`, built on the raw `upload()` wire call. This work also added the `build_inputs` route (the signature source), which the Python SDK previously lacked. +> **Status: implemented** (`pipelex_sdk/upload.py`, `pipelex_sdk/prepare_inputs.py`). `upload_file` and `prepare_inputs` are the Python counterpart of `@pipelex/sdk`'s `uploadFile` / `prepareInputs`, built on the raw `upload()` wire call. The design of record for the current shape is `pipelex-sdk-js/wip/prepare-inputs-selectors/design.md` in the sibling repo; the two SDKs are kept semantically identical. > -> **Current scope.** `prepare_inputs` takes the method closure as inline `files` (the signature source). Two pieces are deliberately deferred and additive (they do not change this contract): resolving a closure from a catalog `method_id`, and the opt-in ingest of `http(s)` URLs into storage — for now an `http(s)` URL at a file position always passes through unchanged. Kept in parity with `@pipelex/sdk`. +> **Current scope.** `prepare_inputs` names the method three ways — inline `files`, a `method_ref` address, or a stored `method_id` — and reads the target pipe's signature from the standard's input-form descriptor. One piece is deliberately deferred and additive (it does not change this contract): the opt-in ingest of `http(s)` URLs into storage — for now an `http(s)` URL at a file position always passes through unchanged. ## Why this exists @@ -14,8 +14,6 @@ Preparation is **explicit and separate from running.** `execute` / `start` never This is the Python side of one cross-language contract. The behavior matrix, pass-through rules, `Dynamic` handling, dedup, and failure categories are identical to the JS SDK; only the accepted source types differ per language. The two SDKs must agree semantically. See the JS counterpart's `docs/input-preparation.md` for the mirror. -The Python SDK previously had **no `/v1/build/*` coverage** — this change added the `build_inputs` counterpart `prepare_inputs` needs to resolve the declared signature (the JS SDK already exposes `buildInputs`). - ## The two operations ### `upload_file` — single-asset convenience @@ -40,29 +38,96 @@ The MIME type and size are known client-side, so the record is assembled without ### `prepare_inputs` — signature-driven input preparation ``` -prepare_inputs(method_ref, pipe, inputs) → PreparedInputs +client.prepare_inputs(files=…, pipe_ref=…, inputs=…) → PreparedInputs +client.prepare_inputs(method_ref=…, pipe_ref=…, inputs=…) → PreparedInputs +client.prepare_inputs(method_id=…, pipe_ref=…, inputs=…) → PreparedInputs ``` -Takes the **method reference** (bundle files or catalog `method_id`) plus the target **pipe**, resolves the pipe's declared input signature, interprets the caller's compact `inputs` top-down against that signature, uploads the file-bearing values, and returns `PreparedInputs`: +Takes the **method** as exactly one of three selectors, the optional target **pipe**, and the caller's `inputs`; resolves the pipe's declared input signature; interprets the inputs top-down against it; uploads the file-bearing values; and returns `PreparedInputs`. Per input, the caller may submit **either** the compact value **or** the explicit `{concept, content}` envelope — see "[Compact or explicit-envelope inputs](#compact-or-explicit-envelope-inputs)" below: - `inputs` — a **copy** of the caller's inputs with each asset reference replaced by the canonical content shape carrying `pipelex-storage://` in its `url` field (see "Rewritten-input shape" below). Copy-on-write: the caller's original object is never mutated. -- `uploads` — one upload record per prepared asset (the `upload_file` record shape), exposing `uri` so callers can log which source became which reference without reverse-engineering the rewritten object. +- `uploads` — one `UploadRecord` per prepared asset, exposing `uri` so callers can log which source became which reference without reverse-engineering the rewritten object. The prepared `inputs` are passed to the existing run lifecycle unchanged. +#### The three selectors + +Exactly one per call. **Empty is absent** — `files=[]`, `method_ref=""`, `method_id=" "` — mirroring the run options' rule, so an empty selector may sit beside a real one without tripping the exclusivity check. None or several raises `InputPreparationError` naming the three forms, before any request leaves the process. + +**Empty is absent; the wrong type is not.** A non-string `method_ref` / `method_id` / `pipe_ref` raises `InputPreparationError` naming the argument and the type it got, on that same pre-request boundary. Reading it as absent instead is what would make the exclusivity check unsound — `method_ref=123` beside a real `files` would pass the check and silently prepare against the wrong method — and would let a mistyped `pipe_ref` be absorbed by the defaulting below. + +| Selector | What it is | Who resolves it | +| --- | --- | --- | +| `files` | the inline MTHDS closure (`MthdsFileItem` entries: `content` plus an optional `source` label) | nobody — inline | +| `method_ref` | a published method's address, `github.com//[/][@]` | the runner, server-side (pipelex-api >= 0.21.0 fetches the repository at the tag) | +| `method_id` | a stored method's catalog id (`mt_…`) | the hosted platform, which injects the stored source before the runner sees the request | + +Nothing is expanded client-side: `method_id` here is a **pass-through**, the same rule every other id-taking operation in this SDK follows. + +#### Where the signature comes from + +One `POST /v1/validate` per call, whatever the selector, asking for the **input-form descriptor**: + +```python +await client.validate(, True, …, views=[VALIDATION_VIEW_INPUT_FORM]) +``` + +`allow_signatures=True` is deliberate. Preparation needs a pipe's *declared* inputs, and a bundle mid-authoring with an unresolved signature somewhere else must not be refused inputs for a pipe whose inputs are declared — whether the bundle runs is the run's verdict, not preparation's. An `is_valid: false` verdict still means the closure does not load, which is a preparation failure. + +A `method_ref` makes the server clone a repository first; `validate` needs no special budget for it, because the route already defaults to the 20-minute execute ceiling. + +**A valid report that carries no descriptor is an error, never a silent "no uploads".** The descriptor rides `views: ["input_form"]` on pipelex-api >= 0.18.0; pointed at an older runner, `prepare_inputs` says so rather than returning inputs whose local paths would travel to the runner verbatim. + +#### Pipe selection + +`validate` has no pipe selector — its report describes every pipe, keyed by qualified `pipe_ref` — so the helper picks one, in this order: + +1. **`pipe_ref` when given.** Qualified-only: `domain.pipe_code`. A bare code, a non-string, or a ref the method does not declare, is an `InputPreparationError` listing the qualified refs — one step to fix. The helper never grows a searched `pipe_code`: search is a run-route affordance, and the descriptor is keyed by qualified refs. +2. **The report's typed resolved default** (`default_pipe_ref`), once the runner serves it: the ref a caller gets by omitting the selector, manifest-aware for a fetched package. Read when present; a server that predates it sends nothing. +3. **The bundle's declared `main_pipe`**, read defensively from the opaque `bundle_blueprint` and qualified by its `domain`. +4. **The single pipe**, when the method declares exactly one. +5. Otherwise an `InputPreparationError` naming the candidates and asking for `pipe_ref`. + +> **The manifest-only `main_pipe` gap.** A published package may name its entry pipe in `METHODS.toml` alone — `github.com/Pipelex/methods/documents` and `.../image_generation` do — and the validate report never carries a manifest. Until step 2's field ships, such a package needs an explicit `pipe_ref`; the error lists the candidates, so the fix is one line. + +## Compact or explicit-envelope inputs + +Each input may be submitted in **either** of two shapes, and preparation treats them equivalently: + +- **Compact** — the bare value: a source string / `bytes` / `Path` / canonical `{"url": …}` content (e.g. `photo="…/p.png"`). +- **Explicit envelope** — the `{"concept", "content"}` shape (e.g. `photo={"concept": "native.Image", "content": {"url": "…"}}`). This is the template shape the hosted console and MCP hand agents to fill, so an agent that fills a template can hand it straight back. + +When a value is an envelope (a dict whose keys are **exactly** `concept` and `content`, matching the runtime's `_is_explicit` in `input_shaper.py`), preparation unwraps `content`, interprets it exactly as the compact value would be, and **re-wraps** the result — so the concept annotation rides through to the run. The envelope's `content` may itself be a scalar, canonical file content, a list, or a structured object nesting file fields; the same top-down walk applies underneath. + ## Signature-driven asset identification -The SDK **must not** guess that every string resembling a path is an asset — that would make ordinary text inputs environment-dependent and could upload unintended files. Interpretation comes from the method's **declared signature**, never from a value's shape alone. This mirrors the runtime's own top-down interpretation (`pipelex/pipelex/core/memory/input_shaper.py`, `InputShaper`) combined with the file-reference resolution of `pipelex/pipelex/pipeline/input_normalizer.py`, so local and hosted execution read the same compact inputs the same way. +The SDK **must not** guess that every string resembling a path is an asset — that would make ordinary text inputs environment-dependent and could upload unintended files. Interpretation comes from the method's **declared signature**, never from a value's shape alone. This mirrors the runtime's own top-down interpretation (`pipelex`'s `InputShaper`) combined with the file-reference resolution of `input_normalizer`, so local and hosted execution read the same compact inputs the same way. + +The signature is the **input-form descriptor** (`InputForm` from `mthds.protocol.input_form`), and the walk is discriminated on each node's declared `kind`: + +| Node kind | What the walk does | +| --- | --- | +| `document`, `image` | a **file position**, whatever the value's shape — resolved per the pass-through rules below | +| `object` | walks the declared `fields` by name against a dict value; keys the descriptor does not name are copied through untouched | +| `list` | walks `item` against each element of a list value | +| `text`, `prose`, `date`, `number`, `boolean`, `enum`, `unknown` | passes through at any depth | + +An **optional** field (`required: false`) is walked when the caller supplies it. A caller value whose shape disagrees with the node — a scalar at an `object`, a non-list at a `list` — passes through for the run to reject; preparation never second-guesses the signature. + +`unknown` is the standard's escape hatch for a `Dynamic` or `Composite` input, and it is **not** entered: the signature declares no file there. A caller with such an input uploads with `upload_file` first and passes the resulting `pipelex-storage://` URI. + +### Why the descriptor, and why not the inputs template + +Earlier releases read the signature from the explicit inputs template (`POST /v1/build/inputs`), which marked a file position by rendering a `{"url": …}` dict. That is a side effect of a field being **named** `url`, not of its concept being an Image or a Document, and two positions were misread as a result: + +- an **optional nested file field** was never rendered by the required-only template, so its position was invisible and the caller's local path travelled to the runner as a literal string; +- a **text field merely named `url`** was read from disk and uploaded. -The declared signature is resolved via the explicit inputs template (`build_inputs` with `explicit=True`), which carries concept identity, canonical content shape, and multiplicity per input. +The descriptor states the resolved kind at every depth and includes optional fields, so both are gone. It is also the standard's own artifact, derived from authored facts rather than from a rendered shape, and `/v1/validate` resolves all three method selectors server-side — which is what made the uniform selector surface possible at no server cost. -Interpretation per declared input: +**If you actually wanted the template.** `prepare_inputs` no longer needs one, and this SDK's `build_inputs` wrapper went with it, but the template itself did not disappear — `mthds.protocol.inputs_template` projects one from the same descriptor, client-side: `render_inputs_template(descriptor=…, explicit=…, output_format=…)` for the JSON or TOML text, `project_inputs_template(descriptor=…, explicit=…)` for the dict. Ask `validate` for the `input_form` view, hand the pipe's descriptor to either, and the round-trip the removed route used to cost is gone too — which is what makes a template available for a method named only by `method_ref` or `method_id`. -- A bare string, `Path`, or `bytes` value at an **Image/Document-declared** input is a **file reference**: local paths, data URLs, and bytes are uploaded and rewritten to `pipelex-storage://` URIs; HTTP(S) URLs and existing `pipelex-storage://` URIs pass through unchanged. -- The **identical** bare string at a **Text-declared** input is text and is never touched. -- **Canonical image/document content structures** are recognized by their URL-bearing fields wherever they appear, including nested in structured objects and lists — exactly as the runtime normalizer walks them. The refining case matters: a concept refining `Image`/`PDF` is classified by the **canonical content shape**, not by the concept ref alone. -- Inputs declared **`Dynamic`** are not path-interpreted (the signature genuinely cannot guide them); they accept canonical content structures or already-prepared references only. -- A repeated reference to the **same source** within one preparation is uploaded once and rewritten consistently (within-preparation dedup by source identity). +**Known limit.** A class-backed concept (`structure = "SomeClass"`) whose reflection cannot map a field annotation collapses to `kind: "unknown"` in the descriptor, so a file field beneath one is invisible to this walk. That is a fidelity bug in the runtime's `build_input_form`, tracked separately; pass such a value as an already-uploaded storage URI until it is fixed. ### Pass-through rules diff --git a/pipelex_sdk/build_models.py b/pipelex_sdk/build_models.py deleted file mode 100644 index 2e32765..0000000 --- a/pipelex_sdk/build_models.py +++ /dev/null @@ -1,171 +0,0 @@ -"""Wire models for the `/v1/build/inputs` route — the signature source `prepare_inputs` -reads to resolve a pipe's declared inputs. - -The Python SDK had no `/v1/build/*` coverage; `prepare_inputs` needs the explicit -inputs template, so this adds the `build_inputs` counterpart of `pipelex-sdk-js`'s -`buildInputs` (only this route — the other build projections are not needed here). -The crate routes (`/v1/resolve`, `/v1/codegen`) share this module's `CrateRequestBase` -envelope and `CrateInvalidReport` arm through `crate_models.py`. -A produced verdict is a `200` discriminated on `is_valid`; a no-verdict condition -(unknown `pipe_ref`, auth, server fault) throws `ApiResponseError`. -""" - -from __future__ import annotations - -from typing import Annotated, Any, Literal, Self, TypeAlias - -from pydantic import BaseModel, ConfigDict, Field, TypeAdapter, field_validator, model_validator - -from pipelex_sdk.validation_models import ValidationErrorItem - -InputsTemplateFormat = Literal["json", "toml"] - - -class MthdsFileItem(BaseModel): - """One MTHDS file in a build closure. `source` is an optional provenance label the - server threads onto diagnostics raised from this file. - """ - - content: str - source: str | None = None - - -class CrateRequestBase(BaseModel): - """The closure selector every crate-family route shares — `/v1/resolve`, - `/v1/codegen`, and `/v1/build/*` (mirror of the server's `MthdsFilesRequest` and - of `pipelex-sdk-js`'s `CrateRequestBase`). - - Supply the closure EITHER as inline `files` OR as a `method_ref` — never both, and - never neither. An **address-form** `method_ref` - (`github.com//[/][@]`) is resolved by the server - (pipelex-api >= 0.21.0): the repository is fetched at the tag, the package is - located by manifest identity, and its `.mthds` files feed the closure with their - real relative paths as per-file sources. The **registry form** (any non-address - reference) stays reserved and answers `501` until a method registry exists. - - An EMPTY selector is normalized to absent before the exclusivity check — `files=[]` - selects no closure and `method_ref=""` (or whitespace-only) no address, the same - empty-as-absent rule the run routes apply — so an unusable value never counts as the - sole selector and never reaches the wire. - - The subclasses own the exclusivity validator, because the crate routes add a third - selector (the hosted `method_id`) that the build projections deliberately refuse. - """ - - files: list[MthdsFileItem] | None = None - method_ref: str | None = None - - @field_validator("files") - @classmethod - def _empty_files_are_absent(cls, value: list[MthdsFileItem] | None) -> list[MthdsFileItem] | None: - # `files=[]` is not a closure — normalize to absent so the XOR counts real selectors only. - return value or None - - @field_validator("method_ref") - @classmethod - def _blank_method_ref_is_absent(cls, value: str | None) -> str | None: - # A blank address selects nothing — same empty-as-absent rule as the run routes' - # `_normalized_selector` boundary. A real value is passed through untouched. - if value is None or not value.strip(): - return None - return value - - -class BuildInputsRequest(CrateRequestBase): - """Request for `POST /v1/build/inputs`. The closure is inline `files` XOR a - `method_ref` address; there is NO by-id form — the `/v1/build/*` projections are - deliberately excluded from the hosted tooling selector (`method_id` covers - `validate` / `resolve` / `codegen` only), so a stored method is expanded first - (fetch it with `get_method` and pass its source as `files`). - """ - - pipe_ref: str | None = None - format: InputsTemplateFormat = "json" - explicit: bool = False - - @model_validator(mode="before") - @classmethod - def _refuse_method_id(cls, data: Any) -> Any: - # A teaching error beats pydantic's default extra="ignore" silently dropping the - # key: a caller migrating from the by-id habit must learn the build routes have - # no by-id form (mirrors the JS `method_id: never` pin + runtime guard). - raw: Any = data - if isinstance(data, dict) and "method_id" in data: - msg = ( - "build_inputs takes no method_id — the /v1/build/* projections are excluded from the " - "hosted tooling selector (it covers validate/resolve/codegen only). Expand the stored " - "method first: fetch it with get_method and pass its MTHDS source as files." - ) - raise ValueError(msg) - return raw - - @model_validator(mode="after") - def _exactly_one_closure_selector(self) -> Self: - # Mirrors the server's own XOR so an illegal shape fails at construction, before - # anything hits the wire. - if (self.files is None) == (self.method_ref is None): - msg = "provide exactly one of `files` or `method_ref`" - raise ValueError(msg) - return self - - -class BuildInputsValidReport(BaseModel): - """The `is_valid: true` arm. The template rides `inputs` (json) or `inputs_toml` (toml).""" - - model_config = ConfigDict(extra="allow") - - is_valid: Literal[True] - pipe_ref: str - requested_pipe_ref: str | None = None - message: str - format: InputsTemplateFormat - explicit: bool - inputs: dict[str, Any] | None = None - inputs_toml: str | None = None - - @model_validator(mode="after") - def _template_matches_format(self) -> Self: - # Honor the adapter's malformed-200 guarantee for the template shape too: a valid - # verdict must carry the template field its `format` selects (and not the other). - # Without this, an `is_valid: true` body missing both templates would parse as a valid - # report and only fail one layer down in `prepare_inputs`. - match self.format: - case "json": - if self.inputs is None: - msg = "inputs is required when format is 'json'" - raise ValueError(msg) - if self.inputs_toml is not None: - msg = "inputs_toml must be absent when format is 'json'" - raise ValueError(msg) - case "toml": - if self.inputs_toml is None: - msg = "inputs_toml is required when format is 'toml'" - raise ValueError(msg) - if self.inputs is not None: - msg = "inputs must be absent when format is 'toml'" - raise ValueError(msg) - return self - - -class CrateInvalidReport(BaseModel): - """The `is_valid: false` arm shared by the build routes — an unresolvable closure is a - produced verdict on a `200`, never a thrown error. Branch on `is_valid`, not transport. - """ - - model_config = ConfigDict(extra="allow") - - is_valid: Literal[False] - validation_errors: list[ValidationErrorItem] - message: str - - -BuildInputsResponse: TypeAlias = Annotated[ - BuildInputsValidReport | CrateInvalidReport, - Field(discriminator="is_valid"), -] - -# The single parse path for a 200 `/build/inputs` body — discriminated on `is_valid`, built once at -# import (TypeAdapter construction is expensive), mirroring `PipelexValidationResultAdapter`. A -# malformed 200 (or an empty body) raises a clean `pydantic.ValidationError` rather than being -# mistaken for a valid verdict. -BuildInputsResponseAdapter: TypeAdapter[BuildInputsResponse] = TypeAdapter(BuildInputsResponse) # pylint: disable=invalid-name diff --git a/pipelex_sdk/client.py b/pipelex_sdk/client.py index e601af9..1f51018 100644 --- a/pipelex_sdk/client.py +++ b/pipelex_sdk/client.py @@ -32,11 +32,11 @@ from pydantic_core import to_json from typing_extensions import override -from pipelex_sdk.build_models import BuildInputsRequest, BuildInputsResponse, BuildInputsResponseAdapter, MthdsFileItem from pipelex_sdk.crate_models import ( CodegenRequest, CodegenResponse, CodegenResponseAdapter, + MthdsFileItem, ResolveRequest, ResolveResponse, ResolveResponseAdapter, @@ -173,7 +173,7 @@ # `method_ref` resolution can make the server CLONE a repository before it answers, and the # server-side clone timeout runs well past the 30s management budget on a cold cache — an # abort there would report a healthy, still-cloning server as unreachable. So a -# `method_ref`-carrying crate/build request gets this internal fetch-sized budget instead of +# `method_ref`-carrying crate request gets this internal fetch-sized budget instead of # `_POLL_REQUEST_TIMEOUT_SECONDS` (no new caller-facing parameter, and inert behind the # hosted gateway's own cap). The run routes and `validate` need no such override: they # already ride `request_timeout_seconds` (the 20-min blocking-execute ceiling), which clears @@ -1117,25 +1117,6 @@ async def upload(self, upload_input: UploadInput) -> UploadedFile: body = upload_input.model_dump(mode="json", exclude_none=True) return UploadedFile.model_validate(await self._request_product("POST", "upload", body=body)) - async def build_inputs(self, request: BuildInputsRequest) -> BuildInputsResponse: - """Project a pipe's declared inputs as a fill-in template — `POST /v1/build/inputs`. - - The closure is inline `files` XOR a `method_ref` address (server-resolved, - pipelex-api >= 0.21.0; the registry form stays a `501`). There is NO by-id form: the - `/v1/build/*` projections take no `method_id` (the hosted tooling selector covers - `validate`/`resolve`/`codegen` only) — expand a stored method's source into `files` - yourself (fetch it with `get_method`). - - Returns a 200 verdict: branch on `is_valid` before reading the arm — an unresolvable - closure comes back as `is_valid: false` with `validation_errors`, not a thrown error. - A no-verdict condition (unknown `pipe_ref`, a selector-resolution failure, auth, server - fault) raises `ApiResponseError`. This is the signature source `prepare_inputs` reads - (with `explicit=True`). - """ - body = request.model_dump(mode="json", exclude_none=True) - raw = await self._request_product("POST", "build/inputs", body=body, request_timeout=_crate_request_timeout_seconds(request.method_ref)) - return BuildInputsResponseAdapter.validate_python(raw) - # ── Crate extensions (Pipelex API — `/v1/resolve`, `/v1/codegen`) ───── # # The second crate-family surface, mirroring the JS SDK: `/v1/resolve` emits the @@ -1203,7 +1184,9 @@ async def upload_file( async def prepare_inputs( self, *, - files: list[MthdsFileItem], + files: list[MthdsFileItem] | None = None, + method_ref: str | None = None, + method_id: str | None = None, pipe_ref: str | None = None, inputs: dict[str, Any], ) -> PreparedInputs: @@ -1211,10 +1194,25 @@ async def prepare_inputs( assets, and return copy-on-write rewritten inputs (canonical content carrying `pipelex-storage://` in `url`) plus one upload record per prepared asset. HTTP(S) URLs and existing `pipelex-storage://` URIs pass through unchanged; all failures are raised - before any run is created. The caller supplies the method closure as inline `files`. - See `docs/input-preparation.md`. + before any run is created. + + The method is named exactly one of three ways — inline `files`, a `method_ref` address + (runner-resolved) or a stored `method_id` (platform-resolved) — all server-resolved, + with nothing expanded client-side. An empty selector is treated as absent. The + signature comes from one `POST /v1/validate` asking for the `input_form` view, so the + walk is guided by each input's DECLARED kind rather than by the shape of its value. + + `pipe_ref` is qualified-only (`domain.pipe_code`); omit it to default. See + `docs/input-preparation.md`. """ - return await _prepare_inputs_impl(self, files=files, pipe_ref=pipe_ref, inputs=inputs) + return await _prepare_inputs_impl( + self, + files=files, + method_ref=method_ref, + method_id=method_id, + pipe_ref=pipe_ref, + inputs=inputs, + ) async def list_runs( self, @@ -1420,9 +1418,9 @@ def _assert_method_ref_pairs_with_nothing(*, mthds_contents: list[str] | None, m def _crate_request_timeout_seconds(method_ref: str | None) -> float: - """The request budget for a call carrying a crate closure (the crate routes and the build - projections alike): the management default, unless the closure is a `method_ref` the - server may have to fetch first — see `_METHOD_REF_FETCH_TIMEOUT_SECONDS`. + """The request budget for a call carrying a crate closure (`/v1/resolve`, `/v1/codegen`): + the management default, unless the closure is a `method_ref` the server may have to fetch + first — see `_METHOD_REF_FETCH_TIMEOUT_SECONDS`. """ return _METHOD_REF_FETCH_TIMEOUT_SECONDS if method_ref else _POLL_REQUEST_TIMEOUT_SECONDS diff --git a/pipelex_sdk/crate_models.py b/pipelex_sdk/crate_models.py index 5048e7e..167ab4c 100644 --- a/pipelex_sdk/crate_models.py +++ b/pipelex_sdk/crate_models.py @@ -1,13 +1,18 @@ -"""Wire models for the crate routes — `POST /v1/resolve` and `POST /v1/codegen`. - -The second crate-family surface, mirroring `pipelex-sdk-js`: `/v1/resolve` emits the -normalized library crate, `/v1/codegen` projects that crate into stamped typed artifacts -plus their lock. Both are Pipelex API extensions (NOT MTHDS Protocol routes) over the -standard-owned artifact, so their wire fields stay brand-neutral. Same envelope and same -verdict discipline as the build routes: a produced verdict is a `200` discriminated on -`is_valid`, with `CrateInvalidReport` (from `build_models`) as the shared invalid arm; a -no-verdict condition (a malformed selector, a selector-resolution failure, auth, a server -fault) raises `ApiResponseError`. +"""Wire models for the crate routes — `POST /v1/resolve` and `POST /v1/codegen` — and the +shared crate envelope they are built on. + +The envelope lives here because these are the routes that still use it. `MthdsFileItem`, +`CrateRequestBase` and `CrateInvalidReport` used to sit in a `build_models` module beside the +`/v1/build/inputs` wire models; those went when `prepare_inputs` moved its signature source to +the input-form descriptor and this SDK stopped calling `/v1/build/*` (workspace campaign +`wip/build-retirement/`). Nothing about the envelope changed in the move. + +`/v1/resolve` emits the normalized library crate, `/v1/codegen` projects that crate into stamped +typed artifacts plus their lock. Both are Pipelex API extensions (NOT MTHDS Protocol routes) over +the standard-owned artifact, so their wire fields stay brand-neutral. A produced verdict is a +`200` discriminated on `is_valid`, with `CrateInvalidReport` as the shared invalid arm; a +no-verdict condition (a malformed selector, a selector-resolution failure, auth, a server fault) +raises `ApiResponseError`. The closure arrives in exactly one of three forms — the tooling routes' strict three-way XOR: inline `files`, an address-form `method_ref` (server-resolved, pipelex-api >= 0.21.0; @@ -23,7 +28,69 @@ from pydantic import BaseModel, ConfigDict, Field, TypeAdapter, field_validator, model_validator -from pipelex_sdk.build_models import CrateInvalidReport, CrateRequestBase +from pipelex_sdk.validation_models import ValidationErrorItem + + +class MthdsFileItem(BaseModel): + """One MTHDS file in a crate closure. `source` is an optional provenance label the + server threads onto diagnostics raised from this file. + """ + + content: str + source: str | None = None + + +class CrateRequestBase(BaseModel): + """The closure selector every crate-family route shares — `/v1/resolve` and + `/v1/codegen` (mirror of the server's `MthdsFilesRequest` and of + `pipelex-sdk-js`'s `CrateRequestBase`). + + Supply the closure EITHER as inline `files` OR as a `method_ref` — never both, and + never neither. An **address-form** `method_ref` + (`github.com//[/][@]`) is resolved by the server + (pipelex-api >= 0.21.0): the repository is fetched at the tag, the package is + located by manifest identity, and its `.mthds` files feed the closure with their + real relative paths as per-file sources. The **registry form** (any non-address + reference) stays reserved and answers `501` until a method registry exists. + + An EMPTY selector is normalized to absent before the exclusivity check — `files=[]` + selects no closure and `method_ref=""` (or whitespace-only) no address, the same + empty-as-absent rule the run routes apply — so an unusable value never counts as the + sole selector and never reaches the wire. + + The subclass owns the exclusivity validator, because the crate routes add a third + selector (the hosted `method_id`) this base does not know about. + """ + + files: list[MthdsFileItem] | None = None + method_ref: str | None = None + + @field_validator("files") + @classmethod + def _empty_files_are_absent(cls, value: list[MthdsFileItem] | None) -> list[MthdsFileItem] | None: + # `files=[]` is not a closure — normalize to absent so the XOR counts real selectors only. + return value or None + + @field_validator("method_ref") + @classmethod + def _blank_method_ref_is_absent(cls, value: str | None) -> str | None: + # A blank address selects nothing — same empty-as-absent rule as the run routes' + # `_normalized_selector` boundary. A real value is passed through untouched. + if value is None or not value.strip(): + return None + return value + + +class CrateInvalidReport(BaseModel): + """The `is_valid: false` arm shared by the crate routes — an unresolvable closure is a + produced verdict on a `200`, never a thrown error. Branch on `is_valid`, not transport. + """ + + model_config = ConfigDict(extra="allow") + + is_valid: Literal[False] + validation_errors: list[ValidationErrorItem] + message: str class CrateToolingRequest(CrateRequestBase): @@ -95,7 +162,7 @@ class ResolveValidReport(BaseModel): ] # The single parse path for a 200 `/resolve` body — discriminated on `is_valid`, built once -# at import (TypeAdapter construction is expensive), mirroring `BuildInputsResponseAdapter`. +# at import (TypeAdapter construction is expensive), mirroring `PipelexValidationResultAdapter`. ResolveResponseAdapter: TypeAdapter[ResolveResponse] = TypeAdapter(ResolveResponse) # pylint: disable=invalid-name diff --git a/pipelex_sdk/prepare_inputs.py b/pipelex_sdk/prepare_inputs.py index 7857ad8..ec79e5b 100644 --- a/pipelex_sdk/prepare_inputs.py +++ b/pipelex_sdk/prepare_inputs.py @@ -1,16 +1,21 @@ -"""`prepare_inputs` — signature-driven input preparation. Resolves the target pipe's -declared inputs via the explicit inputs template, interprets the caller's compact inputs -top-down against it, uploads the file-bearing values, and returns rewritten inputs -(canonical content carrying `pipelex-storage://` in `url`) plus one upload record per -prepared asset. Python counterpart of `pipelex-sdk-js`'s `prepareInputs`. - -The classification mirrors the runtime: `pipelex`'s `input_normalizer` walks -Image/Document contents (recognized by their `url`-bearing shape, incl. nested in -structured content) and `resolve_uri` decides upload vs pass-through. The declared -signature comes from the explicit template (`build_inputs`, `explicit=True`), whose -canonical content shape is the classifier — the file signal is a value that is a dict -containing a `url` key. See the shared behavior matrix (`wip/upload/behavior-matrix.md`) -and `docs/input-preparation.md`. +"""`prepare_inputs` — signature-driven input preparation. Names the method three ways, +resolves the target pipe's declared inputs from the standard's input-form descriptor, +interprets the caller's inputs top-down against it, uploads the file-bearing values, and +returns rewritten inputs (canonical content carrying `pipelex-storage://` in `url`) plus one +upload record per prepared asset. Python counterpart of `pipelex-sdk-js`'s `prepareInputs`. + +The signature comes from ONE `POST /v1/validate` asking for `views: ["input_form"]`, and the +walk is discriminated on each descriptor node's declared `kind` — never on the shape of a +value. That is the whole point: the previous source, the explicit inputs template, marked a +file position by rendering a `{"url": …}` dict, which is a side effect of a field being NAMED +`url` rather than of its concept being an Image or a Document. Two positions were misread as a +result — an OPTIONAL nested file field, which the required-only template never rendered, was +left un-uploaded and its local path travelled to the runner as a literal string; and a text +field merely named `url` was read from disk and uploaded. The descriptor states the resolved +kind at every depth and includes optional fields, so both are gone. + +See `docs/input-preparation.md`, and the design of record in +`pipelex-sdk-js/wip/prepare-inputs-selectors/design.md`. """ from __future__ import annotations @@ -22,13 +27,29 @@ from typing import TYPE_CHECKING, Any, Protocol, cast from urllib.parse import unquote_to_bytes +from mthds.protocol.input_form import ( + BooleanItem, + DateItem, + DocumentItem, + EnumItem, + ImageItem, + InputForm, + InputFormItem, + ListItem, + NumberItem, + ObjectItem, + ProseItem, + TextItem, + UnknownItem, +) from pydantic import BaseModel -from pipelex_sdk.build_models import BuildInputsRequest, BuildInputsResponse, CrateInvalidReport, MthdsFileItem from pipelex_sdk.errors import InputPreparationError from pipelex_sdk.upload import UploadRecord, UploadSource, upload_file +from pipelex_sdk.validation_models import VALIDATION_VIEW_INPUT_FORM, PipelexInvalidReport, PipelexValidationReport, PipelexValidationResult if TYPE_CHECKING: + from pipelex_sdk.crate_models import MthdsFileItem from pipelex_sdk.product_models import UploadedFile, UploadInput PIPELEX_STORAGE_SCHEME = "pipelex-storage://" @@ -48,11 +69,24 @@ class PreparedInputs(BaseModel): class _PrepareClient(Protocol): - """The client surface `prepare_inputs` needs: raw `upload` plus the `build_inputs` signature source.""" + """The client surface `prepare_inputs` needs: raw `upload` plus `validate` as the + signature source. Typed as `PipelexAPIClient.validate`'s own signature so the client + satisfies it structurally. + """ async def upload(self, upload_input: UploadInput) -> UploadedFile: ... - async def build_inputs(self, request: BuildInputsRequest) -> BuildInputsResponse: ... + async def validate( + self, + mthds_contents: list[str] | None = None, + allow_signatures: bool = False, + mthds_sources: list[str] | None = None, + render: list[str] | None = None, + views: list[str] | None = None, + *, + method_ref: str | None = None, + method_id: str | None = None, + ) -> PipelexValidationResult: ... class _PrepareContext: @@ -65,11 +99,60 @@ def __init__(self, client: _PrepareClient) -> None: self.dedup: dict[UploadSource, str] = {} +def _non_empty_string(value: object) -> str | None: + """A trimmed non-empty string, or `None` — the "empty is absent" rule. + + Lenient on purpose, because what it reads is OPAQUE server payload — `bundle_blueprint`, + whose schema is the runtime's, not ours — where a shape that does not match is genuinely + an absent value to fall through on. A CALLER-supplied selector goes through + `_caller_selector` instead, which refuses a non-string rather than reading it as absent. + + Deliberately local rather than reusing `client.py`'s `_normalized_selector`: that helper + is private to the client boundary and raises `PipelineRequestError`, where every failure + of this module owes an `InputPreparationError`. + """ + if not isinstance(value, str): + return None + trimmed = value.strip() + return trimmed or None + + +def _caller_selector(value: object, *, argument: str) -> str | None: + """A caller-supplied selector, trimmed — `None` when absent, refused when not a string. + + The "empty is absent" rule of `_non_empty_string`, plus the boundary check that helper + must not make. Coercing a non-string to `None` here would read `method_ref=123` as an + absent selector and let it fall through to another one — defeating the exactly-one check + this whole surface rests on — and would let a non-string `pipe_ref` silently take the + default pipe instead of the one the caller named. Both are caller mistakes, and a caller + mistake owes an `InputPreparationError` raised before any request. + """ + if value is None or isinstance(value, str): + return _non_empty_string(value) + msg = f"Cannot prepare inputs: `{argument}` must be a string, got {type(value).__name__}." + raise InputPreparationError(msg) + + def _is_file_content(node: Any) -> bool: - """A canonical Image/Document content is a dict carrying a `url` key.""" + """A canonical Image/Document content is a dict carrying a `url` key. + + A value-shape helper only, consulted at a position the DESCRIPTOR already declared a + file. It is no longer a classifier: reading it as one is the defect this module removed. + """ return isinstance(node, dict) and "url" in node +def _is_explicit_envelope(value: Any) -> bool: + """The explicit `{concept, content}` input envelope — keys EXACTLY `concept` and `content`. + + Matches the runtime's `_is_explicit` (`input_shaper.py`), so an agent that filled an + explicit template can hand it straight back. Anything else is a compact value. + """ + if not isinstance(value, dict): + return False + return set(cast("dict[str, Any]", value)) == {"concept", "content"} + + def _decode_data_url(data_url: str) -> tuple[bytes, str]: """Decode a `data:` URL into bytes plus its MIME type. @@ -140,7 +223,7 @@ async def _resolve_source(ctx: _PrepareContext, source: Any) -> str: async def _resolve_file_position(ctx: _PrepareContext, caller_value: Any) -> Any: """Resolve a value known to sit at a file position into canonical content with a rewritten `url`.""" - if isinstance(caller_value, dict) and "url" in caller_value: + if _is_file_content(caller_value): content = cast("dict[str, Any]", caller_value) resolved = await _resolve_source(ctx, content["url"]) return {**content, "url": resolved} @@ -148,61 +231,266 @@ async def _resolve_file_position(ctx: _PrepareContext, caller_value: Any) -> Any return {"url": resolved} -async def _resolve_node(ctx: _PrepareContext, template_node: Any, caller_value: Any) -> Any: - """Template-guided walk: a template node that is canonical file content marks a file position.""" - if _is_file_content(template_node): - return await _resolve_file_position(ctx, caller_value) - if isinstance(template_node, list) and template_node: - element_template = cast("list[Any]", template_node)[0] - if isinstance(caller_value, list): - items = cast("list[Any]", caller_value) - return [await _resolve_node(ctx, element_template, item) for item in items] - return caller_value # shape mismatch — leave it for the run to reject - if isinstance(template_node, dict) and isinstance(caller_value, dict): - template_dict = cast("dict[str, Any]", template_node) - caller_dict = cast("dict[str, Any]", caller_value) - result: dict[str, Any] = dict(caller_dict) - for key in template_dict: - if key in caller_dict: - result[key] = await _resolve_node(ctx, template_dict[key], caller_dict[key]) - return result - return caller_value # scalar (text/number/…) or shape mismatch — pass through +async def _resolve_node(ctx: _PrepareContext, node: InputFormItem, caller_value: Any) -> Any: + """Descriptor-guided walk, discriminated on the node's declared kind. + + - `document` / `image` — a file position, whatever the value's shape; + - `object` — walk the declared `fields` by name; keys the descriptor does not name are + copied through untouched. An OPTIONAL field is walked when present, which is what + makes an optional nested file reachable at all; + - `list` — walk `item` against each element; + - every other kind — pass through at any depth. `unknown` is the standard's escape hatch + for a `Dynamic` / `Composite` input and is deliberately NOT entered: the signature + declares no file there, and uploading on the strength of a `url` key is the value-shape + guess this walk removes. Such a caller uploads with `upload_file` first and passes the + storage URI. + + A caller value whose shape disagrees with the node (a scalar at an `object`, a non-list at + a `list`) passes through for the run to reject — preparation never second-guesses the + signature. The match is over the item classes rather than over `kind`, because each + per-kind `*Field` derives from its `*Item`: one set of patterns covers both the named + layer (top level, `object.fields`) and the nameless one (`list.item`), and it narrows the + node for the type checker where matching on `node.kind` would not. + """ + match node: + case DocumentItem() | ImageItem(): + return await _resolve_file_position(ctx, caller_value) + case ObjectItem(): + if not isinstance(caller_value, dict): + return caller_value + caller_dict = cast("dict[str, Any]", caller_value) + result: dict[str, Any] = dict(caller_dict) + for field in node.fields: + if field.name in caller_dict: + result[field.name] = await _resolve_node(ctx, field, caller_dict[field.name]) + return result + case ListItem(): + if not isinstance(caller_value, list): + return caller_value + elements = cast("list[Any]", caller_value) + return [await _resolve_node(ctx, node.item, element) for element in elements] + case TextItem() | ProseItem() | DateItem() | NumberItem() | BooleanItem() | EnumItem() | UnknownItem(): + return caller_value + + +def _resolve_selector( + *, + files: list[MthdsFileItem] | None, + method_ref: str | None, + method_id: str | None, +) -> tuple[list[MthdsFileItem] | None, str | None, str | None]: + """Normalize the three selectors and check that exactly one remains. + + Empty is absent — `files=[]`, `method_ref=""`, `method_id=" "` — mirroring the run + options' rule and the `CrateRequestBase` normalizers, so an empty selector may sit beside + a real one without tripping the XOR. A non-string `method_ref` / `method_id` is NOT absent + but refused, so a mistyped selector cannot slip past the XOR as a silent `None`. The check + lives here because this module is what composes the `validate` call, and it runs BEFORE + any request. + """ + selected_files = files or None + selected_method_ref = _caller_selector(method_ref, argument="method_ref") + selected_method_id = _caller_selector(method_id, argument="method_id") + + given: list[str] = [] + if selected_files is not None: + given.append("`files`") + if selected_method_ref is not None: + given.append("`method_ref`") + if selected_method_id is not None: + given.append("`method_id`") + + if not given: + msg = ( + "Cannot prepare inputs: no method selector. Supply exactly one of `files` (an inline MTHDS " + "closure), `method_ref` (a published method's address) or `method_id` (a stored method's " + "catalog id)." + ) + raise InputPreparationError(msg) + if len(given) > 1: + msg = ( + f"Cannot prepare inputs: {' and '.join(given)} were both given. Supply exactly one method " + "selector — `files`, `method_ref` or `method_id`." + ) + raise InputPreparationError(msg) + return selected_files, selected_method_ref, selected_method_id + + +async def _fetch_signature( + client: _PrepareClient, + *, + files: list[MthdsFileItem] | None, + method_ref: str | None, + method_id: str | None, +) -> PipelexValidationReport: + """Ask `validate` for the signature, whatever the selector, and hand back the valid report. + + `allow_signatures=True` on purpose: preparation needs a pipe's DECLARED inputs, and a + bundle mid-authoring with an unresolved signature elsewhere must not be refused inputs for + a pipe whose inputs are declared — whether the bundle runs is the run's verdict, not + preparation's. An `is_valid: false` arm still means the closure does not load, which IS a + preparation failure. + + No timeout override for a `method_ref`: `validate` already rides the 20-minute blocking + ceiling, and the internal 3-minute fetch budget exists to RAISE the ~30s poll-ceiling + routes, not to lower this one. + """ + views = [VALIDATION_VIEW_INPUT_FORM] + result: PipelexValidationResult + if files is not None: + contents = [file_item.content for file_item in files] + # `validate_files`' rule: label every content once any file names a source, so the + # server never sees a length-mismatched `mthds_sources` array. + sources: list[str] | None + if any(file_item.source is not None for file_item in files): + sources = [file_item.source or f"inline://file-{index + 1}.mthds" for index, file_item in enumerate(files)] + else: + sources = None + result = await client.validate(contents, True, sources, None, views) + else: + result = await client.validate(None, True, None, None, views, method_ref=method_ref, method_id=method_id) + + if isinstance(result, PipelexInvalidReport): + first = result.validation_errors[0].message if result.validation_errors else result.message + msg = f"Cannot prepare inputs: the method signature did not resolve — {first}" + raise InputPreparationError(msg) + return result + + +def _blueprint_main_pipe_ref(blueprint: dict[str, Any]) -> str | None: + """The bundle blueprint's declared `main_pipe`, qualified by its `domain` when authored bare. + + Every read is defensive: `bundle_blueprint` is carried opaquely by this SDK on purpose — + its schema is the runtime's, not ours — so a shape that does not match falls through + rather than raising. + """ + main_pipe = _non_empty_string(blueprint.get("main_pipe")) + if main_pipe is None: + return None + if "." in main_pipe: + return main_pipe + domain = _non_empty_string(blueprint.get("domain")) + return f"{domain}.{main_pipe}" if domain is not None else None + + +def _select_pipe_ref(report: PipelexValidationReport, input_form: InputForm, requested: str | None) -> str: + """Pick the pipe whose descriptor guides the walk. + + `validate` has no pipe selector — its report describes every pipe, keyed by qualified + `pipe_ref` — so the choice is made here, in the order `docs/input-preparation.md` + documents: an explicit qualified `pipe_ref`, then the report's typed resolved default, + then the bundle's declared `main_pipe`, then the single pipe, else an error naming the + candidates. + """ + refs = list(input_form) + candidates = ", ".join(refs) if refs else "(none — the closure declares no pipes)" + + if requested is not None: + if "." not in requested: + msg = ( + "Cannot prepare inputs: `pipe_ref` must be qualified (`domain.pipe_code`), got the bare " + f'"{requested}". The method declares: {candidates}.' + ) + raise InputPreparationError(msg) + if requested not in input_form: + msg = f'Cannot prepare inputs: the method declares no pipe "{requested}". It declares: {candidates}.' + raise InputPreparationError(msg) + return requested + + # The typed resolved default, when the runner serves it (manifest-aware for a `method_ref` + # package, which is why it outranks the blueprint read below). + typed_default = _non_empty_string(report.default_pipe_ref) + if typed_default is not None and typed_default in input_form: + return typed_default + + blueprint_default = _blueprint_main_pipe_ref(report.bundle_blueprint) + if blueprint_default is not None and blueprint_default in input_form: + return blueprint_default + + if len(refs) == 1: + return refs[0] + + msg = f"Cannot prepare inputs: the method declares no single default pipe, so `pipe_ref` is required. It declares: {candidates}." + raise InputPreparationError(msg) async def prepare_inputs( client: _PrepareClient, *, - files: list[MthdsFileItem], + files: list[MthdsFileItem] | None = None, + method_ref: str | None = None, + method_id: str | None = None, pipe_ref: str | None = None, inputs: dict[str, Any], ) -> PreparedInputs: """Prepare a pipe's inputs: upload local/byte/data-URL assets at the signature's file-bearing positions and return copy-on-write rewritten inputs plus upload records. - HTTP(S) URLs and existing `pipelex-storage://` URIs pass through unchanged. All failures - are raised before any run is created. The declared signature is resolved from the inline - `files` closure; a closure that does not resolve raises `InputPreparationError`. No-verdict - conditions from the signature route (unknown `pipe_ref`, auth, server fault) surface as the - build route's `ApiResponseError`. + Args: + client: The client supplying `upload` and `validate`. + files: The method closure inline. Exactly one of `files` / `method_ref` / `method_id`. + method_ref: A published method's address — + `github.com//[/][@]` — resolved by the runner. + method_id: A stored method's hosted catalog id (`mt_…`), resolved by the platform. + A pure pass-through: nothing is expanded client-side. + pipe_ref: The target pipe as a QUALIFIED `domain.pipe_code`. Omit it to default — + see "Pipe selection" in `docs/input-preparation.md`. A bare `pipe_code` is + refused: the descriptor is keyed by qualified refs, and search is a run-route + affordance this helper deliberately does not grow. + inputs: The caller's inputs (variable name → value), compact or explicit-envelope + per input. + + Returns: + `PreparedInputs` — a copy of `inputs` with each file-bearing value rewritten to + canonical content carrying `pipelex-storage://` in `url`, plus one `UploadRecord` + per uploaded asset. + + Raises: + InputPreparationError: No selector or several; a selector that is not a string; the + closure did not resolve; the report carries no descriptor; the pipe could not be + selected; or a value at a file position is unusable. HTTP(S) URLs and existing + `pipelex-storage://` URIs pass through unchanged, and every failure is raised + BEFORE any run is created. + httpx.HTTPStatusError: A no-verdict condition from `/v1/validate` — a malformed + selector, an unknown or foreign-org `method_id` (`404`), a stored method with no + source, a fetch failure at the address. `validate` is 200-diagnostic and stays on + the inherited protocol error regime, so a no-verdict failure arrives as the raw + status error rather than the product routes' `ApiResponseError`. """ - report = await client.build_inputs(BuildInputsRequest(files=files, pipe_ref=pipe_ref, format="json", explicit=True)) - if isinstance(report, CrateInvalidReport): - first = report.validation_errors[0].message if report.validation_errors else report.message - msg = f"Cannot prepare inputs: the method signature did not resolve — {first}" + selected_files, selected_method_ref, selected_method_id = _resolve_selector(files=files, method_ref=method_ref, method_id=method_id) + # Normalized here rather than at its use below, so a mistyped `pipe_ref` is refused on the + # same pre-request boundary as a mistyped selector — before the `validate` round-trip. + requested_pipe_ref = _caller_selector(pipe_ref, argument="pipe_ref") + report = await _fetch_signature(client, files=selected_files, method_ref=selected_method_ref, method_id=selected_method_id) + + input_form = report.input_form + if input_form is None: + # Never a silent degrade to "no uploads": without the descriptor there is no + # signature to prepare against. + msg = ( + "Cannot prepare inputs: the validate report carries no `input_form` descriptor — the signature " + 'preparation reads. The descriptor rides `views: ["input_form"]` on pipelex-api >= 0.18.0; ' + "point the client at a runner that serves it." + ) raise InputPreparationError(msg) - if report.format != "json" or report.inputs is None: - msg = f'Cannot prepare inputs: expected a JSON inputs template, got "{report.format}".' - raise InputPreparationError(msg) - template = report.inputs + + selected_pipe_ref = _select_pipe_ref(report, input_form, requested_pipe_ref) + declared = {field.name: field for field in input_form[selected_pipe_ref].fields} ctx = _PrepareContext(client) rewritten = dict(inputs) for name, caller_value in inputs.items(): - entry = template.get(name) - if not isinstance(entry, dict) or "content" not in entry: - # Not a declared input (or an unexpected envelope) — pass through untouched. + field = declared.get(name) + if field is None: + # Not a declared input — pass through untouched. continue - content = cast("dict[str, Any]", entry)["content"] - rewritten[name] = await _resolve_node(ctx, content, caller_value) + if _is_explicit_envelope(caller_value): + # Unwrap, walk the content against the same node, re-wrap: the concept annotation + # rides through to the run, which accepts the envelope as an input. + envelope = cast("dict[str, Any]", caller_value) + walked = await _resolve_node(ctx, field, envelope["content"]) + rewritten[name] = {**envelope, "content": walked} + else: + rewritten[name] = await _resolve_node(ctx, field, caller_value) return PreparedInputs(inputs=rewritten, uploads=ctx.uploads) diff --git a/pipelex_sdk/product_models.py b/pipelex_sdk/product_models.py index 287a5ba..cbd4339 100644 --- a/pipelex_sdk/product_models.py +++ b/pipelex_sdk/product_models.py @@ -58,8 +58,8 @@ class MethodFile(BaseModel): This is the shape the hosted platform persists for a method's custom PipeFunc Python: a JSON `[{name, content}]` array in one wire string. It is deliberately distinct from two neighbours that look similar and are not: `MthdsFile` (`client.py`) is the *validate* - input, content plus an optional provenance URI; `MthdsFileItem` (`build_models.py`) is - the *build* closure entry. Three shapes for three surfaces — do not merge them. + input, content plus an optional provenance URI; `MthdsFileItem` (`crate_models.py`) is + the *crate* closure entry. Three shapes for three surfaces — do not merge them. """ model_config = ConfigDict(extra="allow") diff --git a/pipelex_sdk/validation_models.py b/pipelex_sdk/validation_models.py index e5947b7..a5bcf1a 100644 --- a/pipelex_sdk/validation_models.py +++ b/pipelex_sdk/validation_models.py @@ -306,6 +306,15 @@ class PipelexValidationReport(ValidationReport): """The parsed bundle, carried opaquely: no published package declares its shape, so a type here could only be a copy free to drift from the runtime that emits it.""" + default_pipe_ref: str | None = None + """The qualified `pipe_ref` a caller gets by omitting the pipe selector, or `None` when the + closure declares none or several. + + Manifest-aware for a fetched package, which is what makes it outrank a `bundle_blueprint` read: + a published package may name its entry pipe in `METHODS.toml` alone, and the blueprint never + carries a manifest. Optional and read leniently — a runner that predates the field simply sends + nothing, so a consumer falls back (`prepare_inputs` reads the blueprint's `main_pipe` next).""" + pipe_io_contracts: PipeIOContracts = Field(default_factory=dict) """The per-pipe I/O contracts, typed by importing the standard's own client models. @@ -313,9 +322,12 @@ class PipelexValidationReport(ValidationReport): declared input slot reads as typed members — `concept_ref`, a three-valued `presence` (`PresenceMarker`), a `multiplicity` (`IOMultiplicity`), the `item_count` that is non-null exactly on the fixed arm, and its `json_schema` — and the output side reads its own asymmetric shape - (a two-valued `optional`, because `!` is rejected on an output). The artifact belongs to the - standard, so it is imported rather than restated: one declaration per language is what makes - drift impossible, which is precisely what keeping it opaque used to buy. + (a two-valued `optional`, because `!` is rejected on an output). Its `json_schema` is required + too since `mthds` 0.13.0, but states the concept's CONTENT MODEL rather than a caller's + argument: where a plural input's schema is a bare array, a plural output's is that model's list + envelope. The artifact belongs to the standard, so it is imported rather than restated: one + declaration per language is what makes drift impossible, which is precisely what keeping it + opaque used to buy. Contracts are **closed** shapes: a member this `mthds` version does not define is version drift and fails the parse. That closure is scoped to the artifact — the report around it stays diff --git a/pyproject.toml b/pyproject.toml index 6fbe71d..c72f56f 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -337,7 +337,7 @@ convention = "google" "implicit-namespace-package", # Allow test files to not have __init__.py in their directories (avoids namespace collisions) "private-member-access", # Unit tests legitimately probe private transport/error helpers (e.g. _request_product, _request_json) "import-private-name", # Unit tests legitimately import private module helpers under test (e.g. _parse_error_body) - "unused-method-argument", # Test-double methods match a Protocol signature; an unused param (e.g. a fake build_inputs ignoring `request`) is intentional + "unused-method-argument", # Test-double methods match a Protocol signature; an unused param (e.g. a fake validate ignoring `render`) is intentional "float-equality-comparison", # Tests assert exact float literals that round-trip exactly; `pytest.approx` would only add noise ] "examples/**/*.py" = [ diff --git a/tests/unit/test_build_inputs.py b/tests/unit/test_build_inputs.py deleted file mode 100644 index c262a00..0000000 --- a/tests/unit/test_build_inputs.py +++ /dev/null @@ -1,191 +0,0 @@ -"""The `build_inputs` route — the signature source `prepare_inputs` reads. Pins the verb + -path + body, the 200-verdict discipline (branch on `is_valid`), and the no-verdict throw. - -Ports the relevant slice of `pipelex-sdk-js/tests/build-routes.test.ts` for `/v1/build/inputs`. -`_send` is mocked; a produced verdict is a 200 discriminated on `is_valid`, a no-verdict -condition throws `ApiResponseError`. -""" - -import asyncio -import json - -import httpx -import pytest -from pydantic import ValidationError -from pytest_mock import MockerFixture, MockType - -from pipelex_sdk.build_models import BuildInputsRequest, BuildInputsResponseAdapter, BuildInputsValidReport, CrateInvalidReport, MthdsFileItem -from pipelex_sdk.client import PipelexAPIClient -from pipelex_sdk.errors import ApiResponseError - -_BASE_URL = "http://localhost:8081" - - -def _response(status_code: int, *, json_body: object | None = None) -> httpx.Response: - request = httpx.Request("POST", f"{_BASE_URL}/x") - if json_body is not None: - return httpx.Response(status_code, json=json_body, request=request) - return httpx.Response(status_code, request=request) - - -class TestBuildInputs: - def _client(self) -> PipelexAPIClient: - return PipelexAPIClient(api_key="test-token", base_url=_BASE_URL) - - def _mock_send(self, mocker: MockerFixture, client: PipelexAPIClient, response: httpx.Response) -> MockType: - return mocker.patch.object(client, "_send", mocker.AsyncMock(return_value=response)) - - def test_posts_files_and_flags_to_build_inputs(self, mocker: MockerFixture) -> None: - client = self._client() - valid = { - "is_valid": True, - "pipe_ref": "demo.main", - "message": "ok", - "format": "json", - "explicit": True, - "inputs": {"photo": {"concept": "demo.Photo", "content": {"url": "https://mock/p.png"}}}, - } - send = self._mock_send(mocker, client, _response(200, json_body=valid)) - - report = asyncio.run( - client.build_inputs(BuildInputsRequest(files=[MthdsFileItem(content='domain = "demo"', source="b.mthds")], format="json", explicit=True)) - ) - - call = send.call_args - assert call.args[0] == "POST" - assert call.args[1] == f"{_BASE_URL}/v1/build/inputs" - body = json.loads(call.kwargs["content"]) - assert body == {"files": [{"content": 'domain = "demo"', "source": "b.mthds"}], "format": "json", "explicit": True} - assert isinstance(report, BuildInputsValidReport) - assert report.pipe_ref == "demo.main" - assert report.inputs is not None - - def test_invalid_closure_is_a_200_verdict(self, mocker: MockerFixture) -> None: - client = self._client() - invalid = { - "is_valid": False, - "message": "closure did not validate", - "validation_errors": [{"category": "blueprint_validation", "message": "unknown pipe type"}], - } - self._mock_send(mocker, client, _response(200, json_body=invalid)) - - report = asyncio.run(client.build_inputs(BuildInputsRequest(files=[MthdsFileItem(content="x")]))) - - assert isinstance(report, CrateInvalidReport) - assert report.validation_errors[0].message == "unknown pipe type" - - @pytest.mark.parametrize( - "body", - [ - # format=json but carrying no template at all — the flagged malformed-200 shape. - {"is_valid": True, "pipe_ref": "demo.main", "message": "ok", "format": "json", "explicit": True}, - # format=json but carrying the toml template (mismatched/opposite shape). - {"is_valid": True, "pipe_ref": "demo.main", "message": "ok", "format": "json", "explicit": True, "inputs_toml": "x = 1"}, - # format=toml but carrying no toml template. - {"is_valid": True, "pipe_ref": "demo.main", "message": "ok", "format": "toml", "explicit": True}, - ], - ) - def test_valid_report_without_matching_template_is_rejected(self, body: dict[str, object]) -> None: - # A valid verdict must carry the template its `format` selects — the adapter's - # malformed-200 guarantee, now honored for the template shape too. - with pytest.raises(ValidationError): - BuildInputsResponseAdapter.validate_python(body) - - def test_valid_toml_report_is_accepted(self) -> None: - body = {"is_valid": True, "pipe_ref": "demo.main", "message": "ok", "format": "toml", "explicit": True, "inputs_toml": "photo = 1"} - report = BuildInputsResponseAdapter.validate_python(body) - assert isinstance(report, BuildInputsValidReport) - assert report.inputs_toml == "photo = 1" - - def test_no_verdict_422_raises_api_response_error(self, mocker: MockerFixture) -> None: - client = self._client() - problem = {"detail": "Unknown pipe_ref", "error_type": "PipeNotFound"} - self._mock_send(mocker, client, _response(422, json_body=problem)) - - with pytest.raises(ApiResponseError) as exc_info: - asyncio.run(client.build_inputs(BuildInputsRequest(files=[MthdsFileItem(content="x")], pipe_ref="demo.nope"))) - assert exc_info.value.status == 422 - - # ── the closure selector (files XOR method_ref, NO method_id) ──── - - def test_method_ref_closure_rides_the_body_with_the_fetch_budget(self, mocker: MockerFixture) -> None: - """An address closure may make the server clone before answering, so the request gets - the fetch-sized budget instead of the 30s management one. - """ - client = self._client() - valid: dict[str, object] = { - "is_valid": True, - "pipe_ref": "documents.summarize", - "message": "ok", - "format": "json", - "explicit": False, - "inputs": {}, - } - send = self._mock_send(mocker, client, _response(200, json_body=valid)) - - report = asyncio.run(client.build_inputs(BuildInputsRequest(method_ref="github.com/Pipelex/methods/documents@v0.1.0"))) - - call = send.call_args - body = json.loads(call.kwargs["content"]) - assert body == {"method_ref": "github.com/Pipelex/methods/documents@v0.1.0", "format": "json", "explicit": False} - assert call.kwargs["request_timeout"] == 180.0 - assert isinstance(report, BuildInputsValidReport) - - def test_inline_files_keep_the_management_budget(self, mocker: MockerFixture) -> None: - client = self._client() - valid: dict[str, object] = {"is_valid": True, "pipe_ref": "demo.main", "message": "ok", "format": "json", "explicit": False, "inputs": {}} - send = self._mock_send(mocker, client, _response(200, json_body=valid)) - - asyncio.run(client.build_inputs(BuildInputsRequest(files=[MthdsFileItem(content="x")]))) - - assert send.call_args.kwargs["request_timeout"] == 30.0 - - @pytest.mark.parametrize( - "kwargs", - [ - {}, - {"files": [MthdsFileItem(content="x")], "method_ref": "github.com/x/y@v1"}, - ], - ) - def test_request_construction_enforces_files_xor_method_ref(self, kwargs: dict[str, object]) -> None: - with pytest.raises(ValidationError, match="exactly one"): - BuildInputsRequest.model_validate(kwargs) - - @pytest.mark.parametrize( - "kwargs", - [ - {"files": []}, - {"method_ref": ""}, - {"method_ref": " "}, - {"files": [], "method_ref": " "}, - ], - ) - def test_empty_closure_selectors_are_absent_and_fail_the_xor(self, kwargs: dict[str, object]) -> None: - """`files=[]` and a blank `method_ref` select nothing — normalized to absent before the - XOR, so an unusable value is refused at construction instead of reaching the wire. - """ - with pytest.raises(ValidationError, match="exactly one"): - BuildInputsRequest.model_validate(kwargs) - - def test_blank_method_ref_beside_files_is_simply_absent(self, mocker: MockerFixture) -> None: - """Same rule as the run boundary: an empty selector beside a real one is absent, not a - conflict — the closure is `files`, the empty key is not sent, the budget stays 30s. - """ - request = BuildInputsRequest(files=[MthdsFileItem(content="x")], method_ref="") - assert request.method_ref is None - - client = self._client() - valid: dict[str, object] = {"is_valid": True, "pipe_ref": "demo.main", "message": "ok", "format": "json", "explicit": False, "inputs": {}} - send = self._mock_send(mocker, client, _response(200, json_body=valid)) - asyncio.run(client.build_inputs(request)) - - body = json.loads(send.call_args.kwargs["content"]) - assert "method_ref" not in body - assert send.call_args.kwargs["request_timeout"] == 30.0 - - def test_method_id_is_refused_with_a_teaching_error(self) -> None: - """The `/v1/build/*` projections take no `method_id` — a teaching error beats pydantic - silently ignoring the unknown key for a caller migrating off the by-id habit. - """ - with pytest.raises(ValidationError, match="build_inputs takes no method_id"): - BuildInputsRequest.model_validate({"method_id": "mt_1"}) diff --git a/tests/unit/test_crate_routes.py b/tests/unit/test_crate_routes.py index 6d0e88e..9f620b9 100644 --- a/tests/unit/test_crate_routes.py +++ b/tests/unit/test_crate_routes.py @@ -14,9 +14,8 @@ from pydantic import ValidationError from pytest_mock import MockerFixture, MockType -from pipelex_sdk.build_models import CrateInvalidReport, MthdsFileItem from pipelex_sdk.client import PipelexAPIClient -from pipelex_sdk.crate_models import CodegenRequest, CodegenValidReport, ResolveRequest, ResolveValidReport +from pipelex_sdk.crate_models import CodegenRequest, CodegenValidReport, CrateInvalidReport, MthdsFileItem, ResolveRequest, ResolveValidReport from pipelex_sdk.errors import ApiResponseError _BASE_URL = "http://localhost:8081" diff --git a/tests/unit/test_prepare_inputs.py b/tests/unit/test_prepare_inputs.py index 18c79a8..69fcb4f 100644 --- a/tests/unit/test_prepare_inputs.py +++ b/tests/unit/test_prepare_inputs.py @@ -1,50 +1,109 @@ -"""`prepare_inputs` — signature-driven input preparation. Cases derive from the shared -behavior matrix (`wip/upload/behavior-matrix.md`): file-bearing positions are found from the -explicit template's canonical content shape (a `{"url": …}` dict), assets are uploaded and -rewritten to `pipelex-storage://` in `url`, http(s)/storage references pass through, dedup -keys on source identity, and the call is copy-on-write. - -Ports `pipelex-sdk-js/tests/prepare-inputs.test.ts`. The fake client returns a canned explicit -template from `build_inputs` and a counting `upload`; one wiring test drives the real client. +"""`prepare_inputs` — signature-driven input preparation over the input-form descriptor. + +Cases derive from the shared behavior matrix (`wip/upload/behavior-matrix.md`) and port +`pipelex-sdk-js/tests/prepare-inputs.test.ts`: file-bearing positions come from the DESCRIPTOR's +declared kind (`document` / `image`), assets are uploaded and rewritten to `pipelex-storage://` +in `url`, http(s)/storage references pass through, dedup keys on source identity, and the call +is copy-on-write. + +The fake client returns a canned `PipelexValidationReport` from `validate` and records the call, +so the request shape is asserted and not just the outcome; one wiring test drives the real client. """ import asyncio import base64 +import json from pathlib import Path -from typing import Any +from typing import Any, cast import httpx import pytest +from mthds.protocol.input_form import ( + DocumentField, + DocumentItem, + ImageField, + InputFormField, + ListField, + ObjectField, + PipeInputFormDescriptor, + TextField, + UnknownField, +) +from mthds.protocol.pipe_io_contracts import PresenceMarker from pytest_mock import MockerFixture -from pipelex_sdk.build_models import BuildInputsRequest, BuildInputsResponse, BuildInputsValidReport, CrateInvalidReport, MthdsFileItem from pipelex_sdk.client import PipelexAPIClient +from pipelex_sdk.crate_models import MthdsFileItem from pipelex_sdk.errors import ApiResponseError, InputPreparationError, RejectedAssetError from pipelex_sdk.prepare_inputs import prepare_inputs from pipelex_sdk.product_models import UploadedFile, UploadInput +from pipelex_sdk.validation_models import PipelexInvalidReport, PipelexValidationReport, PipelexValidationResult _BASE_URL = "http://localhost:8081" _FILES = [MthdsFileItem(content='domain = "demo"')] +_PIPE_REF = "demo.main" + + +def _required(**kwargs: Any) -> dict[str, Any]: + """The pipe-slot facts every TOP-LEVEL field must state (`required` restates `presence`).""" + return {"required": True, "presence": PresenceMarker.PLAIN, "gating": True, **kwargs} + +def _optional(**kwargs: Any) -> dict[str, Any]: + """An optional slot: `required: false`, `presence: optional`, and it never gates.""" + return {"required": False, "presence": PresenceMarker.OPTIONAL, "gating": False, **kwargs} -def _entry(concept: str, content: Any) -> dict[str, Any]: - return {"concept": concept, "content": content} + +def _form(*fields: InputFormField, pipe_ref: str = _PIPE_REF) -> dict[str, PipeInputFormDescriptor]: + return {pipe_ref: PipeInputFormDescriptor(fields=list(fields))} + + +def _report( + input_form: dict[str, PipeInputFormDescriptor] | None, + *, + bundle_blueprint: dict[str, Any] | None = None, + default_pipe_ref: str | None = None, +) -> PipelexValidationReport: + return PipelexValidationReport( + is_valid=True, + bundle_blueprint=bundle_blueprint if bundle_blueprint is not None else {}, + default_pipe_ref=default_pipe_ref, + input_form=input_form, + ) class _FakePrepareClient: - """Fake client: `build_inputs` returns the given envelope template; `upload` counts calls.""" + """Fake client: `validate` returns the given report and records the call; `upload` counts calls.""" - def __init__(self, template: dict[str, Any], *, report: BuildInputsResponse | None = None, upload_error: Exception | None = None) -> None: - self._template = template - self._report = report + def __init__(self, result: PipelexValidationResult, *, upload_error: Exception | None = None) -> None: + self._result = result self._upload_error = upload_error self.upload_calls: list[UploadInput] = [] + self.validate_calls: list[dict[str, Any]] = [] self._counter = 0 - async def build_inputs(self, request: BuildInputsRequest) -> BuildInputsResponse: - if self._report is not None: - return self._report - return BuildInputsValidReport(is_valid=True, pipe_ref="demo.main", message="ok", format="json", explicit=True, inputs=self._template) + async def validate( + self, + mthds_contents: list[str] | None = None, + allow_signatures: bool = False, + mthds_sources: list[str] | None = None, + render: list[str] | None = None, + views: list[str] | None = None, + *, + method_ref: str | None = None, + method_id: str | None = None, + ) -> PipelexValidationResult: + self.validate_calls.append( + { + "mthds_contents": mthds_contents, + "allow_signatures": allow_signatures, + "mthds_sources": mthds_sources, + "views": views, + "method_ref": method_ref, + "method_id": method_id, + } + ) + return self._result async def upload(self, upload_input: UploadInput) -> UploadedFile: if self._upload_error is not None: @@ -54,9 +113,199 @@ async def upload(self, upload_input: UploadInput) -> UploadedFile: return UploadedFile(uri=f"pipelex-storage://user/assets/{self._counter}.bin", filename=upload_input.filename) +def _image_client(name: str = "photo", **upload_error: Any) -> _FakePrepareClient: + return _FakePrepareClient(_report(_form(ImageField(name=name, **_required()))), **upload_error) + + class TestPrepareInputs: + # ── The signature call ──────────────────────────────────────────────── + + def test_asks_validate_for_the_input_form_view(self) -> None: + client = _image_client() + + asyncio.run(prepare_inputs(client, files=_FILES, inputs={})) + + call = client.validate_calls[0] + assert call["views"] == ["input_form"] + assert call["allow_signatures"] is True + assert call["mthds_contents"] == ['domain = "demo"'] + # No file names a source, so none is synthesized — the server never sees a + # length-mismatched `mthds_sources` array. + assert call["mthds_sources"] is None + + def test_labels_every_content_once_any_file_names_a_source(self) -> None: + client = _image_client() + files = [MthdsFileItem(content="a"), MthdsFileItem(content="b", source="b.mthds")] + + asyncio.run(prepare_inputs(client, files=files, inputs={})) + + assert client.validate_calls[0]["mthds_sources"] == ["inline://file-1.mthds", "b.mthds"] + + def test_method_ref_is_a_server_side_pass_through(self) -> None: + client = _image_client() + + asyncio.run(prepare_inputs(client, method_ref="github.com/Pipelex/methods/documents", inputs={})) + + call = client.validate_calls[0] + assert call["method_ref"] == "github.com/Pipelex/methods/documents" + assert call["mthds_contents"] is None + assert call["views"] == ["input_form"] + + def test_method_id_is_a_server_side_pass_through(self) -> None: + client = _image_client() + + asyncio.run(prepare_inputs(client, method_id="mt_abc123", inputs={})) + + call = client.validate_calls[0] + assert call["method_id"] == "mt_abc123" + assert call["mthds_contents"] is None + + # ── The three selectors ─────────────────────────────────────────────── + + def test_no_selector_is_refused_before_any_request(self) -> None: + client = _image_client() + + with pytest.raises(InputPreparationError, match="no method selector"): + asyncio.run(prepare_inputs(client, inputs={"photo": bytes([1])})) + assert client.validate_calls == [] + assert client.upload_calls == [] + + @pytest.mark.parametrize( + "kwargs", + [ + {"files": _FILES, "method_ref": "github.com/o/r"}, + {"files": _FILES, "method_id": "mt_1"}, + {"method_ref": "github.com/o/r", "method_id": "mt_1"}, + ], + ) + def test_several_selectors_are_refused_before_any_request(self, kwargs: dict[str, Any]) -> None: + client = _image_client() + + with pytest.raises(InputPreparationError, match="exactly one method selector"): + asyncio.run(prepare_inputs(client, inputs={}, **kwargs)) + assert client.validate_calls == [] + + def test_empty_selectors_are_absent_beside_a_real_one(self) -> None: + # `files=[]` and a blank `method_id` select nothing, so they may sit beside a real + # `method_ref` without tripping the XOR — the run options' empty-as-absent rule. + client = _image_client() + + asyncio.run(prepare_inputs(client, files=[], method_ref="github.com/o/r", method_id=" ", inputs={})) + + assert client.validate_calls[0]["method_ref"] == "github.com/o/r" + + def test_only_empty_selectors_is_no_selector(self) -> None: + client = _image_client() + + with pytest.raises(InputPreparationError, match="no method selector"): + asyncio.run(prepare_inputs(client, files=[], method_ref="", inputs={})) + + @pytest.mark.parametrize( + ("kwargs", "argument", "type_name"), + [ + ({"method_ref": 123}, "method_ref", "int"), + ({"method_id": ["mt_1"]}, "method_id", "list"), + ({"method_ref": True}, "method_ref", "bool"), + ], + ) + def test_a_non_string_selector_is_refused_rather_than_read_as_absent(self, kwargs: dict[str, Any], argument: str, type_name: str) -> None: + # Empty is absent, but a WRONG TYPE is not: coercing it to `None` would let the XOR + # pass on `files` alone and prepare against a method the caller did not name. + client = _image_client() + + with pytest.raises(InputPreparationError) as exc_info: + asyncio.run(prepare_inputs(client, files=_FILES, inputs={}, **kwargs)) + + assert str(exc_info.value) == f"Cannot prepare inputs: `{argument}` must be a string, got {type_name}." + assert client.validate_calls == [] + assert client.upload_calls == [] + + def test_a_non_string_pipe_ref_is_refused_rather_than_silently_defaulted(self) -> None: + # Read as absent, it would be absorbed by the single-declared-pipe default: the pipe + # the caller named would vanish without a word. Refused on the pre-request boundary. + client = _image_client() + + with pytest.raises(InputPreparationError) as exc_info: + asyncio.run(prepare_inputs(client, files=_FILES, pipe_ref=cast("str", 123), inputs={})) + + assert str(exc_info.value) == "Cannot prepare inputs: `pipe_ref` must be a string, got int." + assert client.validate_calls == [] + + # ── Pipe selection ──────────────────────────────────────────────────── + + def test_uses_the_single_declared_pipe_when_no_ref_is_given(self) -> None: + client = _image_client() + + prepared = asyncio.run(prepare_inputs(client, files=_FILES, inputs={"photo": bytes([1])})) + + assert prepared.inputs == {"photo": {"url": "pipelex-storage://user/assets/1.bin"}} + + def test_typed_default_pipe_ref_outranks_the_blueprint(self) -> None: + input_form = { + "demo.first": PipeInputFormDescriptor(fields=[TextField(name="photo", **_required())]), + "demo.second": PipeInputFormDescriptor(fields=[ImageField(name="photo", **_required())]), + } + client = _FakePrepareClient(_report(input_form, bundle_blueprint={"domain": "demo", "main_pipe": "first"}, default_pipe_ref="demo.second")) + + prepared = asyncio.run(prepare_inputs(client, files=_FILES, inputs={"photo": bytes([1])})) + + # `demo.second` declares `photo` as an image; `demo.first` declares it as text. + assert prepared.inputs == {"photo": {"url": "pipelex-storage://user/assets/1.bin"}} + + def test_falls_back_to_the_blueprint_main_pipe_qualified_by_its_domain(self) -> None: + input_form = { + "demo.first": PipeInputFormDescriptor(fields=[ImageField(name="photo", **_required())]), + "demo.second": PipeInputFormDescriptor(fields=[TextField(name="photo", **_required())]), + } + client = _FakePrepareClient(_report(input_form, bundle_blueprint={"domain": "demo", "main_pipe": "first"})) + + prepared = asyncio.run(prepare_inputs(client, files=_FILES, inputs={"photo": bytes([1])})) + + assert prepared.inputs == {"photo": {"url": "pipelex-storage://user/assets/1.bin"}} + + def test_explicit_pipe_ref_wins(self) -> None: + input_form = { + "demo.first": PipeInputFormDescriptor(fields=[TextField(name="photo", **_required())]), + "demo.second": PipeInputFormDescriptor(fields=[ImageField(name="photo", **_required())]), + } + client = _FakePrepareClient(_report(input_form, default_pipe_ref="demo.first")) + + prepared = asyncio.run(prepare_inputs(client, files=_FILES, pipe_ref="demo.second", inputs={"photo": bytes([1])})) + + assert prepared.inputs == {"photo": {"url": "pipelex-storage://user/assets/1.bin"}} + + def test_bare_pipe_ref_is_refused_naming_the_qualified_candidates(self) -> None: + client = _image_client() + + with pytest.raises(InputPreparationError, match="must be qualified") as exc_info: + asyncio.run(prepare_inputs(client, files=_FILES, pipe_ref="main", inputs={})) + assert _PIPE_REF in str(exc_info.value) + + def test_unknown_pipe_ref_is_refused_naming_the_candidates(self) -> None: + client = _image_client() + + with pytest.raises(InputPreparationError, match="declares no pipe") as exc_info: + asyncio.run(prepare_inputs(client, files=_FILES, pipe_ref="demo.absent", inputs={})) + assert _PIPE_REF in str(exc_info.value) + + def test_several_pipes_and_no_default_is_an_honest_refusal(self) -> None: + # The manifest-only `main_pipe` gap: a fetched package may name its entry pipe in + # METHODS.toml alone, which the report never carries. The error lists the candidates + # so the caller's fix is one line. + input_form = { + "demo.first": PipeInputFormDescriptor(fields=[]), + "demo.second": PipeInputFormDescriptor(fields=[]), + } + client = _FakePrepareClient(_report(input_form)) + + with pytest.raises(InputPreparationError, match="no single default pipe") as exc_info: + asyncio.run(prepare_inputs(client, method_ref="github.com/Pipelex/methods/documents", inputs={})) + assert "demo.first, demo.second" in str(exc_info.value) + + # ── The descriptor-guided walk ──────────────────────────────────────── + def test_uploads_top_level_image_bytes(self) -> None: - client = _FakePrepareClient({"photo": _entry("demo.Photo", {"url": "https://mock/p.png"})}) + client = _image_client() prepared = asyncio.run(prepare_inputs(client, files=_FILES, inputs={"photo": bytes([1, 2, 3])})) @@ -65,7 +314,7 @@ def test_uploads_top_level_image_bytes(self) -> None: assert prepared.uploads[0].uri == "pipelex-storage://user/assets/1.bin" def test_passes_http_url_through(self) -> None: - client = _FakePrepareClient({"photo": _entry("demo.Photo", {"url": "https://mock/p.png"})}) + client = _image_client() prepared = asyncio.run(prepare_inputs(client, files=_FILES, inputs={"photo": "https://example.com/real.png"})) @@ -74,7 +323,7 @@ def test_passes_http_url_through(self) -> None: assert client.upload_calls == [] def test_passes_existing_storage_uri_through(self) -> None: - client = _FakePrepareClient({"photo": _entry("demo.Photo", {"url": "https://mock/p.png"})}) + client = _image_client() prepared = asyncio.run(prepare_inputs(client, files=_FILES, inputs={"photo": "pipelex-storage://user/assets/already.png"})) @@ -82,7 +331,7 @@ def test_passes_existing_storage_uri_through(self) -> None: assert prepared.uploads == [] def test_decodes_and_uploads_data_url(self) -> None: - client = _FakePrepareClient({"photo": _entry("demo.Photo", {"url": "https://mock/p.png"})}) + client = _image_client() prepared = asyncio.run(prepare_inputs(client, files=_FILES, inputs={"photo": "data:image/png;base64,AQIDBA=="})) @@ -100,7 +349,7 @@ def test_decodes_and_uploads_data_url(self) -> None: def test_malformed_base64_data_url_raises_typed_error(self, data_url: str) -> None: # A malformed base64 data URL must surface as the typed `InputPreparationError` # (never a raw binascii.Error), and must never upload silently-corrupted bytes. - client = _FakePrepareClient({"photo": _entry("demo.Photo", {"url": "https://mock/p.png"})}) + client = _image_client() with pytest.raises(InputPreparationError): asyncio.run(prepare_inputs(client, files=_FILES, inputs={"photo": data_url})) @@ -109,14 +358,14 @@ def test_malformed_base64_data_url_raises_typed_error(self, data_url: str) -> No def test_percent_encoded_binary_data_url_keeps_exact_bytes(self) -> None: # A non-base64 data URL carrying percent-encoded binary must upload its exact bytes; # decoding as UTF-8 text first would corrupt any byte >= 0x80 (e.g. %FF). - client = _FakePrepareClient({"photo": _entry("demo.Photo", {"url": "https://mock/p.png"})}) + client = _image_client() asyncio.run(prepare_inputs(client, files=_FILES, inputs={"photo": "data:application/octet-stream,%00%ff%01"})) assert base64.b64decode(client.upload_calls[0].data) == bytes([0x00, 0xFF, 0x01]) - def test_uploads_each_element_of_declared_multiple(self) -> None: - client = _FakePrepareClient({"exhibits": _entry("demo.Exhibit", [{"url": "https://mock/d.pdf"}])}) + def test_uploads_each_element_of_a_declared_list(self) -> None: + client = _FakePrepareClient(_report(_form(ListField(name="exhibits", item=DocumentItem(required=True), **_required())))) prepared = asyncio.run(prepare_inputs(client, files=_FILES, inputs={"exhibits": [bytes([1]), bytes([2])]})) @@ -124,41 +373,43 @@ def test_uploads_each_element_of_declared_multiple(self) -> None: assert len(prepared.uploads) == 2 def test_leaves_text_input_untouched(self) -> None: - client = _FakePrepareClient({"question": _entry("demo.Question", {"text": "text_value"})}) + client = _FakePrepareClient(_report(_form(TextField(name="question", **_required())))) prepared = asyncio.run(prepare_inputs(client, files=_FILES, inputs={"question": "notes/summary.txt"})) assert prepared.inputs == {"question": "notes/summary.txt"} assert client.upload_calls == [] - def test_uploads_only_nested_image_of_structured_input(self) -> None: - client = _FakePrepareClient( - {"dossier": _entry("demo.Dossier", {"title": "title_value", "cover": {"url": "https://mock/c.png", "mime_type": "image/png"}})} + def test_uploads_only_the_nested_image_of_a_structured_input(self) -> None: + dossier = ObjectField( + name="dossier", + fields=[TextField(name="title", required=True), ImageField(name="cover", required=True)], + **_required(), ) + client = _FakePrepareClient(_report(_form(dossier))) prepared = asyncio.run(prepare_inputs(client, files=_FILES, inputs={"dossier": {"title": "Q3 report", "cover": bytes([7, 7])}})) assert prepared.inputs == {"dossier": {"title": "Q3 report", "cover": {"url": "pipelex-storage://user/assets/1.bin"}}} assert len(prepared.uploads) == 1 - def test_does_not_path_interpret_bare_string_at_dynamic_input(self) -> None: - client = _FakePrepareClient({"freeform": _entry("native.Anything", {"whatever": "value"})}) + def test_preserves_sibling_keys_of_canonical_file_content(self) -> None: + client = _image_client() - prepared = asyncio.run(prepare_inputs(client, files=_FILES, inputs={"freeform": "resembles/a/path"})) + prepared = asyncio.run(prepare_inputs(client, files=_FILES, inputs={"photo": {"url": bytes([1]), "mime_type": "image/png"}})) - assert prepared.inputs == {"freeform": "resembles/a/path"} - assert client.upload_calls == [] + assert prepared.inputs == {"photo": {"url": "pipelex-storage://user/assets/1.bin", "mime_type": "image/png"}} - def test_uploads_canonical_image_nested_in_dynamic(self) -> None: - client = _FakePrepareClient({"data": _entry("native.Composite", {"text": "t", "images": [{"url": "https://mock/i.png"}]})}) + def test_copies_through_object_keys_the_descriptor_does_not_name(self) -> None: + dossier = ObjectField(name="dossier", fields=[ImageField(name="cover", required=True)], **_required()) + client = _FakePrepareClient(_report(_form(dossier))) - prepared = asyncio.run(prepare_inputs(client, files=_FILES, inputs={"data": {"text": "hi", "images": [bytes([5])]}})) + prepared = asyncio.run(prepare_inputs(client, files=_FILES, inputs={"dossier": {"cover": bytes([1]), "note": "kept"}})) - assert prepared.inputs == {"data": {"text": "hi", "images": [{"url": "pipelex-storage://user/assets/1.bin"}]}} - assert len(prepared.uploads) == 1 + assert prepared.inputs["dossier"]["note"] == "kept" def test_dedups_by_source_identity(self) -> None: - client = _FakePrepareClient({"exhibits": _entry("demo.Exhibit", [{"url": "https://mock/d.pdf"}])}) + client = _FakePrepareClient(_report(_form(ListField(name="exhibits", item=DocumentItem(required=True), **_required())))) shared = bytes([9, 9, 9]) prepared = asyncio.run(prepare_inputs(client, files=_FILES, inputs={"exhibits": [shared, shared]})) @@ -168,7 +419,7 @@ def test_dedups_by_source_identity(self) -> None: assert exhibits[0]["url"] == exhibits[1]["url"] def test_is_copy_on_write(self) -> None: - client = _FakePrepareClient({"photo": _entry("demo.Photo", {"url": "https://mock/p.png"})}) + client = _image_client() original = {"photo": bytes([1, 2, 3])} asyncio.run(prepare_inputs(client, files=_FILES, inputs=original)) @@ -176,14 +427,14 @@ def test_is_copy_on_write(self) -> None: assert original["photo"] == bytes([1, 2, 3]) def test_passes_through_undeclared_input(self) -> None: - client = _FakePrepareClient({"photo": _entry("demo.Photo", {"url": "https://mock/p.png"})}) + client = _image_client() prepared = asyncio.run(prepare_inputs(client, files=_FILES, inputs={"photo": "https://example.com/p.png", "stray": "left alone"})) assert prepared.inputs["stray"] == "left alone" def test_uploads_real_local_path(self, tmp_path: Path) -> None: - client = _FakePrepareClient({"photo": _entry("demo.Photo", {"url": "https://mock/p.png"})}) + client = _image_client() path = tmp_path / "shot.png" path.write_bytes(bytes([1, 2, 3, 4])) @@ -192,8 +443,110 @@ def test_uploads_real_local_path(self, tmp_path: Path) -> None: assert prepared.inputs == {"photo": {"url": "pipelex-storage://user/assets/1.bin"}} assert client.upload_calls[0].content_type == "image/png" + def test_a_shape_mismatch_passes_through_for_the_run_to_reject(self) -> None: + # A scalar where the descriptor declares an object: preparation never second-guesses + # the signature, so the value rides through and the run answers for it. + dossier = ObjectField(name="dossier", fields=[ImageField(name="cover", required=True)], **_required()) + client = _FakePrepareClient(_report(_form(dossier))) + + prepared = asyncio.run(prepare_inputs(client, files=_FILES, inputs={"dossier": "not an object"})) + + assert prepared.inputs == {"dossier": "not an object"} + assert client.upload_calls == [] + + # ── The two misclassifications of L-260826-ddd843 ───────────────────── + + def test_uploads_an_optional_top_level_file_field_when_supplied(self) -> None: + client = _FakePrepareClient(_report(_form(DocumentField(name="appendix", **_optional())))) + + prepared = asyncio.run(prepare_inputs(client, files=_FILES, inputs={"appendix": bytes([3])})) + + assert prepared.inputs == {"appendix": {"url": "pipelex-storage://user/assets/1.bin"}} + + def test_uploads_an_optional_nested_file_field(self) -> None: + # First edge: the required-only inputs template never rendered an optional nested + # file field, so its position was invisible and the caller's local path travelled to + # the runner as a literal string. The descriptor states `required: false` and the + # walk enters it. + dossier = ObjectField( + name="dossier", + fields=[TextField(name="title", required=True), ImageField(name="cover", required=False)], + **_required(), + ) + client = _FakePrepareClient(_report(_form(dossier))) + + prepared = asyncio.run(prepare_inputs(client, files=_FILES, inputs={"dossier": {"title": "t", "cover": bytes([7])}})) + + assert prepared.inputs["dossier"]["cover"] == {"url": "pipelex-storage://user/assets/1.bin"} + + def test_does_not_read_a_text_field_merely_named_url_from_disk(self) -> None: + # Second edge: the template marked a file position by rendering a `url`-bearing dict — + # a side effect of the field's NAME, not of its concept — so a path-shaped text value + # was uploaded. `kind: "text"` ends that. + client = _FakePrepareClient(_report(_form(TextField(name="url", **_required())))) + + prepared = asyncio.run(prepare_inputs(client, files=_FILES, inputs={"url": "notes/summary.txt"})) + + assert prepared.inputs == {"url": "notes/summary.txt"} + assert client.upload_calls == [] + + def test_does_not_enter_a_dynamic_input(self) -> None: + # A `Dynamic` / `Composite` input is `kind: "unknown"` — the standard's escape hatch — + # and the walk does not enter it, so a canonical file dict nested inside is NOT + # uploaded. Uploading on the strength of a `url` key is the value-shape guess this + # walk removes; such a caller uses `upload_file` first and passes the storage URI. + client = _FakePrepareClient(_report(_form(UnknownField(name="data", **_required())))) + nested = {"text": "hi", "images": [{"url": "https://mock/i.png"}]} + + prepared = asyncio.run(prepare_inputs(client, files=_FILES, inputs={"data": nested})) + + assert prepared.inputs == {"data": nested} + assert client.upload_calls == [] + + def test_does_not_path_interpret_a_bare_string_at_a_dynamic_input(self) -> None: + client = _FakePrepareClient(_report(_form(UnknownField(name="freeform", **_required())))) + + prepared = asyncio.run(prepare_inputs(client, files=_FILES, inputs={"freeform": "resembles/a/path"})) + + assert prepared.inputs == {"freeform": "resembles/a/path"} + assert client.upload_calls == [] + + # ── The explicit `{concept, content}` envelope ──────────────────────── + + def test_unwraps_and_rewraps_the_explicit_envelope(self) -> None: + client = _image_client() + + prepared = asyncio.run(prepare_inputs(client, files=_FILES, inputs={"photo": {"concept": "native.Image", "content": bytes([1])}})) + + # The concept annotation rides through to the run; only `content` is rewritten. + assert prepared.inputs == {"photo": {"concept": "native.Image", "content": {"url": "pipelex-storage://user/assets/1.bin"}}} + + def test_walks_inside_an_envelope_carrying_a_structured_content(self) -> None: + dossier = ObjectField( + name="dossier", + fields=[TextField(name="title", required=True), ImageField(name="cover", required=True)], + **_required(), + ) + client = _FakePrepareClient(_report(_form(dossier))) + envelope = {"concept": "demo.Dossier", "content": {"title": "t", "cover": bytes([7])}} + + prepared = asyncio.run(prepare_inputs(client, files=_FILES, inputs={"dossier": envelope})) + + assert prepared.inputs["dossier"]["concept"] == "demo.Dossier" + assert prepared.inputs["dossier"]["content"]["cover"] == {"url": "pipelex-storage://user/assets/1.bin"} + + def test_a_dict_that_is_not_exactly_concept_and_content_is_not_an_envelope(self) -> None: + # The envelope test matches the runtime's `_is_explicit`: keys EXACTLY `concept` and + # `content`. A third key means it is ordinary content, not an envelope. + client = _image_client() + + with pytest.raises(InputPreparationError, match="Unsupported value at a file input"): + asyncio.run(prepare_inputs(client, files=_FILES, inputs={"photo": {"concept": "x", "content": bytes([1]), "extra": 1}})) + + # ── Failures, all raised before any run exists ──────────────────────── + def test_raises_for_unrecognized_value_at_file_position(self) -> None: - client = _FakePrepareClient({"photo": _entry("demo.Photo", {"url": "https://mock/p.png"})}) + client = _image_client() # A plain object that is neither a canonical {url} content nor bytes — a realistic # caller typo — must surface as a typed error, not pass through unresolved. @@ -201,40 +554,48 @@ def test_raises_for_unrecognized_value_at_file_position(self) -> None: asyncio.run(prepare_inputs(client, files=_FILES, inputs={"photo": {"mimeType": "image/png", "bytes": [1, 2, 3]}})) assert client.upload_calls == [] - def test_raises_when_signature_does_not_resolve(self) -> None: - report = CrateInvalidReport(is_valid=False, message="closure did not validate", validation_errors=[]) - client = _FakePrepareClient({}, report=report) + def test_raises_when_the_signature_does_not_resolve(self) -> None: + invalid = PipelexInvalidReport(is_valid=False, message="closure did not validate", validation_errors=[]) + client = _FakePrepareClient(invalid) - with pytest.raises(InputPreparationError): + with pytest.raises(InputPreparationError, match="the method signature did not resolve"): asyncio.run(prepare_inputs(client, files=_FILES, inputs={"photo": bytes([1])})) + def test_a_report_without_the_descriptor_is_an_error_not_a_silent_no_op(self) -> None: + # Never a silent degrade to "no uploads": without the descriptor there is no signature + # to prepare against, and the caller's local path would travel to the runner verbatim. + client = _FakePrepareClient(_report(None)) + + with pytest.raises(InputPreparationError, match="carries no `input_form` descriptor"): + asyncio.run(prepare_inputs(client, files=_FILES, inputs={"photo": bytes([1])})) + assert client.upload_calls == [] + def test_surfaces_rejected_asset_before_returning(self) -> None: error = ApiResponseError( "HTTP 413", api_url=f"{_BASE_URL}/v1/upload", status=413, status_text="Payload Too Large", response_body="", server_message="too big" ) - client = _FakePrepareClient({"photo": _entry("demo.Photo", {"url": "https://mock/p.png"})}, upload_error=error) + client = _image_client(upload_error=error) with pytest.raises(RejectedAssetError): asyncio.run(prepare_inputs(client, files=_FILES, inputs={"photo": bytes([1])})) + # ── Wiring ──────────────────────────────────────────────────────────── + def test_wires_through_the_real_client(self, mocker: MockerFixture) -> None: client = PipelexAPIClient(api_key="test-token", base_url=_BASE_URL) - build_body = { + validate_body = { "is_valid": True, - "pipe_ref": "demo.main", - "message": "ok", - "format": "json", - "explicit": True, - "inputs": {"photo": {"concept": "demo.Photo", "content": {"url": "https://mock/p.png"}}}, + "bundle_blueprint": {}, + "input_form": {_PIPE_REF: {"fields": [{"kind": "image", "name": "photo", "required": True, "presence": "plain", "gating": True}]}}, } upload_body = {"uri": "pipelex-storage://user/assets/1.bin", "filename": "upload.bin"} request = httpx.Request("POST", f"{_BASE_URL}/x") - mocker.patch.object( + send = mocker.patch.object( client, "_send", mocker.AsyncMock( side_effect=[ - httpx.Response(200, json=build_body, request=request), + httpx.Response(200, json=validate_body, request=request), httpx.Response(200, json=upload_body, request=request), ] ), @@ -244,3 +605,22 @@ def test_wires_through_the_real_client(self, mocker: MockerFixture) -> None: assert prepared.inputs == {"photo": {"url": "pipelex-storage://user/assets/1.bin"}} assert len(prepared.uploads) == 1 + assert send.await_args_list[0].args[1] == f"{_BASE_URL}/v1/validate" + + def test_wires_a_method_ref_through_the_real_client(self, mocker: MockerFixture) -> None: + client = PipelexAPIClient(api_key="test-token", base_url=_BASE_URL) + validate_body = { + "is_valid": True, + "bundle_blueprint": {}, + "input_form": {_PIPE_REF: {"fields": [{"kind": "text", "name": "question", "required": True, "presence": "plain", "gating": True}]}}, + } + request = httpx.Request("POST", f"{_BASE_URL}/x") + send = mocker.patch.object(client, "_send", mocker.AsyncMock(return_value=httpx.Response(200, json=validate_body, request=request))) + + asyncio.run(client.prepare_inputs(method_ref="github.com/o/r", inputs={"question": "hi"})) + + body = json.loads(send.await_args_list[0].kwargs["content"]) + assert body["method_ref"] == "github.com/o/r" + assert body["views"] == ["input_form"] + assert body["allow_signatures"] is True + assert "mthds_contents" not in body diff --git a/tests/unit/test_validation_contract.py b/tests/unit/test_validation_contract.py index 0044989..55624b4 100644 --- a/tests/unit/test_validation_contract.py +++ b/tests/unit/test_validation_contract.py @@ -231,13 +231,25 @@ def _body_with_contracts(input_contract: dict[str, Any]) -> dict[str, Any]: - """A valid body whose one pipe declares exactly `input_contract` as its single input slot.""" + """A valid body whose one pipe declares exactly `input_contract` as its single input slot. + + The output block states a `json_schema` for the same reason `VALID_BODY` does — required on + the contract since the output side gained a payload schema. It matters more here: every body + this helper builds feeds a test asserting the parse FAILS, so an incomplete output would make + each of them fail on the output rather than on the input drift the test names. + """ return { **VALID_BODY, "pipe_io_contracts": { "legal_contracts.summarize": { "inputs": {"contract": input_contract}, - "output": {"concept_ref": "legal_contracts.Summary", "multiplicity": "single", "item_count": None, "optional": False}, + "output": { + "concept_ref": "legal_contracts.Summary", + "multiplicity": "single", + "item_count": None, + "optional": False, + "json_schema": {}, + }, } }, } @@ -383,6 +395,25 @@ def test_valid_arm_carries_warnings_liftable_pipes_and_input_form(self) -> None: # Keyed exactly like `pipe_io_contracts` — the same `pipe_ref` set addresses both artifacts. assert set(report.input_form) == set(report.pipe_io_contracts) + def test_default_pipe_ref_is_absent_by_default(self) -> None: + """A runner that predates the field simply sends nothing — the report still parses.""" + report = _parse(VALID_BODY_WITH_VIEWS) + assert isinstance(report, PipelexValidationReport) + assert report.default_pipe_ref is None + + def test_default_pipe_ref_reads_as_the_qualified_ref_when_served(self) -> None: + body = {**VALID_BODY_WITH_VIEWS, "default_pipe_ref": "legal_contracts.summarize"} + report = _parse(body) + assert isinstance(report, PipelexValidationReport) + assert report.default_pipe_ref == "legal_contracts.summarize" + + def test_default_pipe_ref_reads_an_explicit_null(self) -> None: + """`null` is how the runner says the closure declares no single default — not a parse failure.""" + body = {**VALID_BODY_WITH_VIEWS, "default_pipe_ref": None} + report = _parse(body) + assert isinstance(report, PipelexValidationReport) + assert report.default_pipe_ref is None + def test_input_form_reads_as_the_standards_models(self) -> None: """The descriptor is typed by import: nodes narrow on `kind`, and the recursion is typed through.""" report = _parse(VALID_BODY_WITH_VIEWS) diff --git a/wip/prepare-inputs-selectors/plan.md b/wip/prepare-inputs-selectors/plan.md new file mode 100644 index 0000000..e54abe2 --- /dev/null +++ b/wip/prepare-inputs-selectors/plan.md @@ -0,0 +1,50 @@ +--- +status: active +item: L-260829-8a25d5 +--- + +# `prepare_inputs`: three selectors, signature from the input-form descriptor + +The Python half of the workspace campaign retiring `/v1/build/*` (epic `L-260829-848001`, `wip/build-retirement/` at the workspace root). + +## The design of record is the JS one + +This repo writes no second design. `pipelex-sdk-js/wip/prepare-inputs-selectors/design.md` holds the investigation, Louis's ruling of 2026-08-29, the surface, the walk, the pipe-selection ladder, the error wordings, the alternatives rejected and the known limits — and it names this item's mandate explicitly: `prepare_inputs` lands the same surface, and because `build_inputs` and `BuildInputsRequest` exist here only to back it, this item deletes them. + +The JS twin (`L-260829-300c50`) landed as `pipelex-sdk-js` PR #42 (`bea4632`) and is the reference implementation. Divergence from it is a bug unless recorded below. + +## What this repo did + +- `prepare_inputs(client, *, files=None, method_ref=None, method_id=None, pipe_ref=None, inputs)` — keyword parameters rather than JS's `never`-pinned discriminated union, matching how `validate` already takes its selectors here. Empty-as-absent and the exactly-one check run before any request, raising `InputPreparationError`. +- One `validate(..., allow_signatures=True, views=["input_form"])` per call; the walk is a `match` over the descriptor's item classes rather than over `kind`, because each `*Field` derives from its `*Item` — one set of patterns covers the named layer (top level, `object.fields`) and the nameless one (`list.item`), and it narrows for pyright where matching on `node.kind` would not. +- `PipelexValidationReport.default_pipe_ref` added ahead of the server (`L-260829-0208c7`), as JS did. +- `build_inputs` and the `BuildInputs*` models deleted. `build_models.py` deleted with them: the three survivors it also held — `MthdsFileItem`, `CrateRequestBase`, `CrateInvalidReport` — moved to `crate_models.py`, beside the routes that still use them. +- The explicit `{concept, content}` envelope is now accepted. This was a **pre-existing parity gap**, not part of the item's letter: JS gained it in an earlier release and Python never did, so the two SDKs would not have been identical after the fix. Ruled in scope with the user on 2026-08-30. + +## Decisions taken here + +| Decision | Why | +|---|---| +| Keyword selectors, not a request model | The repo's own `validate` idiom; `architecture.md` already records the JS-vs-Python signature-shape divergence as idiomatic per language. | +| `build_inputs` deleted, where JS kept `buildInputs` | JS has a wrapper family (`buildOutput`, `buildRunner`, `concept`, `pipeSpec`) retiring together under `L-260829-eefc3f`. Python only ever had this one, added in 0.5.0 solely to back `prepare_inputs`. | +| `build_models.py` folded into `crate_models.py` | A module named for the build routes cannot go on owning the crate envelope after those routes leave. | +| A local `_non_empty_string`, not `client._normalized_selector` | That helper is private to the client boundary and raises `PipelineRequestError`; every failure of this module owes an `InputPreparationError`. | +| Two helpers, not one: `_caller_selector` beside `_non_empty_string` | Review round 1. The single lenient helper read both a CALLER's selector and the OPAQUE `bundle_blueprint`, and those want opposite answers for a non-string: absent for the payload whose schema is the runtime's, refused for the argument. Raising inside the shared helper — the suggested fix — would have made the defensive blueprint reads throw on a shape they exist to tolerate. | +| No fetch budget on the signature call | `validate` already rides the 20-minute ceiling; the 3-minute budget exists to *raise* the ~30s poll-ceiling routes. JS implemented this and reverted it — do not re-add. | + +## What this supersedes + +`wip/pr-11-review-notes.md` recorded a nested-file limitation of the old template walk: a top-level `url` key caused an early return, so a sibling file field went un-uploaded, and the note explained that shape refinement was ambiguous because the walk dropped the envelope's `concept`. The descriptor walk removes that class of problem structurally — position and kind are stated, never inferred — so the note is history, not open work. + +## Review + +Round 1 (2026-09-07) confirmed one defect in two threads and one wrong docstring, both fixed on the branch: + +- **A non-string selector was read as absent.** `_non_empty_string` coerced any non-string to `None`, so `method_ref=123` beside a real `files` passed the exactly-one check and prepared against a method the caller never named, and a non-string `pipe_ref` was absorbed by the pipe defaulting. Split into `_caller_selector` (refuses, per the decision row above) and the unchanged lenient reader, with `pipe_ref` normalization hoisted so both refusals land on the same pre-request boundary. +- **The `Raises:` section named the wrong exception.** `validate` is 200-diagnostic and stays on the inherited `httpx.HTTPStatusError` regime, not the product routes' `ApiResponseError`, so a caller following the docstring would have missed exactly the no-verdict failures it listed. + +Nothing else was raised. + +## Release + +None from this item directly. The change lands on `dev` and records its warrant under `## [Unreleased]`; `/release` cuts the version. `L-260826-ddd843` (the two misclassifications) closes only when **both** SDKs have shipped a release carrying the fix — the JS half was still unreleased when this landed.