Skip to content

feat(ENG-12409): implement support for the Nix format & upstreams - #365

Draft
apoclyps wants to merge 7 commits into
masterfrom
kyleharrison/eng-12409/implement-nix-support
Draft

feat(ENG-12409): implement support for the Nix format & upstreams#365
apoclyps wants to merge 7 commits into
masterfrom
kyleharrison/eng-12409/implement-nix-support

Conversation

@apoclyps

Copy link
Copy Markdown
Contributor

Description

Adds Nix as a supported package format, following the same pattern used to add generic support in #256.

  • Bumped the cloudsmith-api dependency from >=2.0.29,<3.0 to >=2.0.30,<3.0 (and regenerated uv.lock) to pick up the generated Nix SDK models (NixPackageUploadRequest, NixUpstream, NixUpstreamRequest, NixUpstreamRequestPatch).
  • Added "nix" to UPSTREAM_FORMATS in cloudsmith_cli/cli/commands/upstream.py, enabling cloudsmith upstream nix {create,list,update,delete}.
  • cloudsmith push nix required no code change — push.py derives its per-format subcommands and options dynamically via introspection of cloudsmith_api.models for *PackageUploadRequest classes, so it picked up the new NixPackageUploadRequest model (and its --narinfo, --version, --republish, --tags options) automatically once the SDK dependency was bumped.
  • Added a CHANGELOG.md entry under [Unreleased].

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation update
  • Refactoring
  • Other (please describe)

Additional Notes

Example usage:

$ cloudsmith push nix --help
Usage: cloudsmith push nix [OPTIONS] OWNER/REPO PACKAGE_FILE

  Push/upload a new Nix package upstream.
...
  --narinfo TEXT                The narinfo sidecar metadata file (<storeHash>.narinfo).
                                 Optional — the package can be completed later by
                                 publishing the matching narinfo via the native HTTP PUT path.
  --republish / --no-republish  If true, the uploaded package will overwrite any others
                                 with the same attributes (e.g. same version); otherwise,
                                 it will be flagged as a duplicate.
  --tags TEXT                   A comma-separated values list of tags to add to the package.
  --version TEXT                The raw version for this package.

$ cloudsmith upstream nix --help
Usage: cloudsmith upstream nix [OPTIONS] COMMAND [ARGS]...

  Manage nix upstreams for a repository.

Commands:
  create|new  Create a nix upstream for a repository.
  delete|rm   Delete a nix upstream for a repository.
  list|ls     List nix upstreams for a repository.
  update      Update a nix upstream for a repository.

Verified locally with the above --help output, the full unit test suite (699 passed), and pre-commit run --all-files.

Copilot AI lite review requested due to automatic review settings August 19, 2026 11:52
@apoclyps
apoclyps requested a review from a team as a code owner August 19, 2026 11:52

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds first-class CLI support for the Nix package format by updating the generated Cloudsmith SDK dependency and enabling Nix in the upstream command group list, aligning with the existing dynamic format-introspection patterns used by push and upstream.

Changes:

  • Bump cloudsmith-api dependency to >=2.0.30,<3.0 (and update uv.lock) to pick up Nix SDK models.
  • Add "nix" to UPSTREAM_FORMATS, enabling cloudsmith upstream nix {create,list,update,delete}.
  • Document the new support in CHANGELOG.md under [Unreleased].

Reviewed changes

Copilot reviewed 3 out of 4 changed files in this pull request and generated 1 comment.

File Description
uv.lock Locks cloudsmith-api to 2.0.30 with updated sdist/wheel hashes.
pyproject.toml Updates the runtime dependency constraint to cloudsmith-api>=2.0.30,<3.0.
cloudsmith_cli/cli/commands/upstream.py Enables nix as an upstream format via UPSTREAM_FORMATS.
CHANGELOG.md Adds an [Unreleased] entry announcing Nix support and the SDK bump.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread pyproject.toml Outdated
@apoclyps
apoclyps marked this pull request as draft August 19, 2026 13:27
@apoclyps apoclyps self-assigned this Aug 19, 2026
apoclyps and others added 6 commits August 20, 2026 15:32
Bumps the `cloudsmith-api` dependency to >=2.0.30,<3.0 to pick up the
generated Nix SDK models (NixPackageUploadRequest, NixUpstream*), and
adds "nix" to the supported upstream formats. `cloudsmith push nix` is
already registered automatically by the existing package-format
introspection in push.py.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The generic push pipeline only treated package-format fields ending in
`_file` as uploadable files needing a file-upload round trip before being
sent to package-create as an identifier. Nix's SDK model names its sidecar
metadata field `narinfo` (matching the real <storeHash>.narinfo filename),
not `narinfo_file`, so it fell through as a plain string: the CLI forwarded
the raw local path straight into the package-create payload, and the API
rejected it with a 422 ("Narinfo Field: Could not find specified package
file") since it expected an upload identifier.

Add a small allowlist of non-_file-suffixed fields that still carry a
single uploadable file (currently just narinfo) and route them through the
same validate/upload path as package_file. --narinfo is now typed as a real
path (client-side existence check) and correctly gets uploaded first.

Also adds a bug report (NIX_SCAN_BUG_REPORT.md) for a separate,
non-CLI issue found while testing this against a live sandbox: nix packages
consistently fail sync at a "Scanning Package" stage server-side with no
reason given, regardless of this fix.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…ver names it narinfo_file

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…rename

The SDK renamed NixPackageUploadRequest's narinfo field to narinfo_file,
matching the CLI's existing _file-suffix upload convention (the special
case for the old name was dropped in 77a2fbc). Update the docs/bug-report
references accordingly.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…upload

upload_file()/multi_part_upload_file() raised ApiException without a
detail, since they POST/PUT straight to the pre-signed storage URL rather
than through the Cloudsmith API client that normally populates detail from
the JSON error body. The CLI could therefore only ever show the generic
HTTP status phrase (e.g. "Bad Request") for a failed upload, no matter what
the storage backend actually said - this is what made an expired S3 upload
token on a nix test sandbox look like an unexplained failure until
reproduced by hand.

Extract the <Message> from the storage backend's XML error body via a
small regex rather than a full XML parser, since this is response content
from wherever upload_url points and a real parser would add an
XXE/entity-expansion surface for no benefit over pulling out one text
field.

Also updates NIX_SCAN_BUG_REPORT.md with the actual root cause (an expired
upload token, unrelated to nix or narinfo) now that it's been confirmed
resolved end-to-end against a live sandbox.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@apoclyps
apoclyps force-pushed the kyleharrison/eng-12409/implement-nix-support branch from f7a050d to 9afdebc Compare August 20, 2026 14:32
…o checksum sync-loop report

NIX_SCAN_BUG_REPORT.md was unexpectedly deleted (217 -0 diff) in
9afdebc/f7a050d - reconstructed from the pre-edit content at cef2b81 and
re-applied its Resolution section.

Adds NIX_NARINFO_CHECKSUM_SYNC_LOOP.md documenting a distinct, more
serious bug found reproducing on staging and production (not the sandbox):
nix packages loop indefinitely between sync stages rather than reaching a
terminal state, because the .narinfo file's checksum is never computed
server-side. Client-side timing, upload ordering, concurrency, and
cross-repo content-addressable-storage collisions were all tested
directly against the raw API and ruled out - a guaranteed-fresh synthetic
nar/narinfo pair reproduces the exact same loop on staging with completely
normal sequential timing, while identical stress tests (reversed order,
concurrent uploads, racing package-create against the uploads) stay clean
on the sandbox. This is an unconditional backend behavior difference
between the sandbox and staging/production, not a race or a data
collision.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

2 participants