Skip to content

Add the registrar provisioning subcommand (#989) - #990

Merged
AcoPiper merged 9 commits into
mainfrom
AcoPiper/issue-989
Sep 8, 2026
Merged

Add the registrar provisioning subcommand (#989)#990
AcoPiper merged 9 commits into
mainfrom
AcoPiper/issue-989

Conversation

@AcoPiper

@AcoPiper AcoPiper commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Adds bootroot registrar, the provisioning surface a deployment tool drives before the host-local registrar endpoint is of any use. The endpoint itself already ships; what did not was any way for an installer to learn that it exists, or to obtain the client credential that reaches it — so an install could only defer the registrar, and roxyd's certificate could not be minted on a real host.

Both of the caller's pinned expectations that contradict contracts this repository owns are resolved in this repository's favour, because the identity rule and the socket are bootroot's to define.

Changes

registrar capabilities [--json] [--socket-unit <path>] — read-only, reporting api_version (bootroot.registrar.v1), socket_path and verbs (registrar.issue, registrar.mint, registrar.deregister, in that order). It describes the surface rather than the runtime, so it answers with the endpoint disabled and with nothing running. The pathname is the installed socket unit's effective ListenStream=, resolved the way systemd resolves it: the unit directories are searched in systemd's own precedence order so an operator-installed unit outranks a packaged one, the first unit file found masks the ones below it, and every drop-in systemd would apply to that unit is merged on top, from every unit directory — the unit's own bootroot-registrar.socket.d/*.conf, where systemctl edit writes its override; the dash-truncated bootroot-.socket.d/*.conf systemd derives from the unit's name; and the type-wide socket.d/*.conf that alters every socket unit on the host. Drop-ins of the same filename are one drop-in, the more specific directory winning and a name-specific one outranking the type-wide one, and what is left applies sorted by filename wherever it came from. A drop-in symlinked to /dev/null is systemd's own mask: it takes the name out of the merge without contributing a setting, which is how a type-wide drop-in is cancelled for this unit alone. Reading less than this would report the unit-local path while systemd bound another, and the caller would connect to a socket that is not there. With no unit installed anywhere it falls back to the unit this repository ships, embedded at compile time so the reported value cannot drift from the file an operator installs. A unit named through --socket-unit is authoritative and is not extended by drop-ins: one that cannot be read, or that binds nothing, is a refusal rather than a fall-through, as is an installed unit that binds nothing once its drop-ins are merged.

registrar issue --host <label> --domain <domain> --cert-path <p> --key-path <p> [--secrets-dir <p>] [--json] — issues the registrar's client leaf into the caller's paths with the CA bundle as the certificate path's sibling, at the modes service add establishes, and answers api_version, identity and not_after.

  • The identity is composed here from REGISTRAR_CLIENT_LABEL at the v1 instance label. No flag on the surface accepts a composed name, so the certificate and recognize_registrar_client cannot disagree; the response reports what was composed, so the caller learns the identity rather than asserting it.
  • The ACME inputs come from the rendered bootroot-internal registrar configuration under --secrets-dir, which keeps the verb credential-free — a provisioning tool drives it as root with no OpenBao token in hand. A host bootroot init has never provisioned is refused by naming the file that is not there, distinctly from a configuration that is present and malformed.
  • The three output destinations — --cert-path, --key-path and the ca-bundle.pem derived beside the certificate — are held to being distinct files before anything is issued. Two of them naming one file would publish one and then overwrite it with another while reporting success; two spellings of one file are one destination. A directory that does not exist yet is resolved as well, because the publication creates it — every missing level, through whatever symlink the path traverses on the way: the nearest ancestor that does exist is resolved and the components below it are applied to that, so with /srv/link a symlink to /srv/real and neither carrying a new/, --cert-path /srv/link/new/registrar.pem and --key-path /srv/real/new/registrar.pem are the one file they would become and are refused rather than compared as typed. Applying the unborn components without following them is sound precisely because they are not there: a directory that does not exist is not a symlink.
  • Material is issued into a staging directory below the secrets tree and published only once every byte is in hand, so a run that fails part-way leaves no half-written pair. That directory is named per process: two invocations racing below one secrets tree would otherwise read back each other's material.
  • Publication itself is reversible. The three destinations are read back before the first is replaced — their bytes, their permission bits and their ownership — and a failure part-way through puts every one that was already replaced back as it was: the contents it held, at the permission bits it carried and under the uid and gid that owned it, removing, on a first provisioning, the files the failed run created. What was there is restored rather than republished, so a certificate an operator had tightened to 0640 does not come back world-readable because that is the mode this verb writes at, and a key another account owned does not come back owned by the run that failed. setuid, setgid and the sticky bit ride the restore too, applied after the ownership rather than with the rename, because chown(2) clears them — restoring the bits first and re-owning the file afterwards loses them silently. Without any of it, a key write failing after the certificate write succeeded would leave the new leaf beside the previous key: a pair that is complete, readable and useless, with nothing on disk saying so. A rollback that cannot itself complete — including one whose chown an unprivileged run is refused — names the files to check by hand and does not replace the reason the run failed.
  • The rollback answers for a failure of this run, and two writers that both succeed have nothing to put back, so publication is serialised across processes: a publication holds an exclusive flock(2) on .bootroot-registrar-publish.lock in each directory it writes into, from before it reads the destinations back until after its last write or restore. Without it, one writer publishing the certificate while another publishes the certificate and key, before the first publishes its key, leaves one leaf beside the other's key with both reporting success. The lock is taken on the resolved directory, so two spellings of one directory are one lock, and the directories are taken in sorted order, so two writers naming the same ones in different roles cannot each hold what the other waits for. The lock file is empty, created 0600, and left in place — unlinking it would hand the next two runs an inode each — and the kernel releases it when the descriptor closes, so a run killed mid-publication strands nothing. A destination named after it is refused. The issuance ahead of the publication stays unlocked: it writes only into the run's own staging directory, and holding a lock across an ACME round trip would make one writer wait out another's network exchange.
  • That lock is not the CLI's own. Three writers replace this material and they are not all in one process: this verb at the caller's paths, the daemon's start-time issuance of the surface pairs at [registrar_endpoint] client_cert_path/client_key_path, and the daemon's renewal of that same leaf. A deployment points this verb at the paths the daemon is configured with — that is the point of provisioning the credential the daemon then maintains — so a CLI-only lock leaves the daemon free to publish between this run's certificate and its key, and both report success over a pair that matches neither. The lock therefore lives in the library (src/publication_lock.rs) and all three take it, at the destinations each writes. The daemon's start-time issuance reaches the ACME exchange and the publication as separate steps so the lock can cover the second and not the first; its renewal takes the lock outside ca_bundle_lock, which is the one order every writer takes the two in, so no pair of them can each hold what the other waits for.

Neither verb changes what the daemon does. It still learns its own socket from the descriptor systemd hands it, no configuration key names one, and no part of mint, deregister or renewal is reimplemented here — the daemon's only change is that its two publications of this same material now take the shared publication lock, which is what stops a run of this verb interleaving with one of theirs.

Documentation lands as the cross-repository contract in docs/reference/registrar-provisioning-surface.md, plus the operator-facing entries in the mirrored docs/en/cli.md and docs/ko/cli.md.

Closes #989

Test plan

  • cargo fmt -- --check --config group_imports=StdExternalCrate and cargo clippy --all-targets -- -D warnings are clean, on macOS and in a Linux container (where the Linux-only endpoint code compiles).
  • capabilities --json answers the three fields, with socket_path equal to a fixture socket unit's ListenStream= rather than a literal in the test.
  • The unit search follows systemd's precedence order, an installed unit outranks the shipped one, and the shipped fallback is the checked-in file.
  • A systemctl edit-style drop-in override is merged; drop-ins below the unit still apply; they apply in filename order across directories; a same-named drop-in higher up masks a lower one; a non-.conf entry is ignored; a named unit is not extended by drop-ins; an installed unit binding nothing masks the ones below it; a drop-in that clears the list is a refusal.
  • The type-wide socket.d/ and the dash-truncated bootroot-.socket.d/ are resolved too: a type-wide drop-in that resets and rebinds ListenStream= is reported, one below the unit still applies, it is ordered among the rest by filename, and the more specific directory wins an equally named file. The derived directory names follow systemd's prefix hierarchy — including the trailing-dash and leading-dash cases — and the full search order is asserted against systemd's own. A drop-in symlinked to /dev/null masks the copy below it, both against a type-wide file and against a lower-precedence unit-specific one. Six of these fail against the previous implementation.
  • A --socket-unit that cannot be read, and one binding nothing, are refusals.
  • capabilities answers with the endpoint disabled and no daemon running, and independently of configuration.
  • verbs lists the three wire names in the fixed order, and a surface missing one fails the test rather than answering a shorter list.
  • issue writes the certificate (0644), the key (0600) and the sibling CA bundle (0644) at the caller's paths.
  • The issued leaf is accepted by recognize_registrar_client for that domain and refused for another.
  • identity is the composed name; not_after parses as RFC 3339 and lies in the future.
  • No caller-supplied composed identity is accepted: no such flag exists, and a composed name in --host is refused.
  • A re-invocation leaves a usable pair, and a failed issuance publishes nothing and sweeps its staging directory.
  • Overlapping output destinations are refused before the ACME path is reached: one path for both halves of the pair, either half landing on the derived ca-bundle.pem, and two spellings of one file.
  • Two destinations that alias only through a directory neither has yet — <link>/new/registrar.pem against <target>/new/registrar.pem — are refused, as is a --key-path landing on the derived bundle the same way; two genuinely distinct destinations below unborn directories are still accepted, and the unborn tail is normalised (.. included) onto the resolved ancestor. The two aliasing tests fail against the previous implementation.
  • A publication failure induced after the real writers have run puts all three destinations back at the contents and modes they had, and removes the files a first provisioning had created; a failure in the middle of the pair puts the certificate and the bundle back.
  • The rollback restores the mode each destination actually carried rather than the one this verb writes at, starting from a 0640 certificate, a 0400 key and a 0600 bundle; the snapshot captures the ownership beside the mode and the restore re-establishes it, skipping the chown that would change nothing and reporting the one it is refused.
  • The rollback puts back a mode carrying a special bit: a 04600 file restored under a gid the process can grant comes back setuid, where restoring the bits with the rename and re-owning afterwards loses it. The test fails against the previous implementation.
  • The publication holds its lock across every step, asserted from inside the publication by a step that tries the lock on a descriptor of its own — the same refusal a second process gets — and fails if it is granted. One destination set is one lock, two spellings of a directory are one lock, a disjoint set is another, the lock is released with the guard, and a publication that had to wait publishes the whole set. A destination named after the lock file is refused before anything is issued.
  • The daemon takes the same lock at the same destinations, asserted the same way and with no timing in any of it. Its start-time publication is probed from inside for every path it writes, including the merged bundle, and with no bundle configured it locks the pair and leaves an unrelated directory free. Its renewal is probed through the live-write seam across the bundle write, the pair write and all three restores of a rolled-back attempt. All four fail against the unfixed code, and a daemon-shaped destination set and a CLI-shaped one sharing a directory are shown to be one lock.
  • The Linux-only renewal module — which this Mac cannot build — was compiled, linted and tested in a rust:latest container: cargo clippy --all-targets -- -D warnings clean, and every registrar_renewal test plus the new publication_lock and registrar_certs ones pass.
  • An unprovisioned host is refused on the missing file, distinctly from a malformed configuration.
  • A configured EAB binds the issuance's account; an unconfigured one binds nothing.
  • An unknown verb and a missing required flag exit with clap's usage code.
  • Nothing gives the daemon a configured socket path.
  • ./scripts/check-docs.sh builds the manual with --strict, and markdownlint-cli2 is clean.

A deployment tool has to know that bootroot's registrar surface exists,
and has to obtain the credential that reaches it, before the host-local
endpoint is of any use. Neither was possible: the endpoint shipped, but
nothing on the CLI reported it and nothing minted the registrar's own
client leaf, so an install could only defer the registrar and roxyd's
certificate could not be issued on a real host.

Two of the caller's pinned expectations contradict contracts this
repository owns, and both are resolved in this repository's favour,
because the identity rule and the socket are bootroot's to define.

The identity is composed here from the host label and the domain, at the
v1 instance label and under REGISTRAR_CLIENT_LABEL. No flag on the
surface accepts a composed name: a caller-supplied name is a name that
can disagree with what recognize_registrar_client accepts, and there is
then no single place that decides what the identity is. The response
reports what was composed, so the caller learns the identity rather than
asserting it.

The socket pathname is read from the installed socket unit, searched in
systemd's own unit-directory precedence order, and falls back to the
unit this repository ships — embedded at compile time, so the reported
value cannot drift from the file an operator installs. No configuration
key names the endpoint's path and none is added; the daemon still learns
its own from the descriptor systemd hands it, and the answer has to
stand while nothing is running.

Issuance runs into a staging directory below the secrets tree and
publishes only once every byte is in hand, so a run that fails part-way
leaves no half-written pair at the caller's paths. That directory is
named per process rather than at one fixed path: two invocations racing
below the same secrets tree would otherwise read back each other's
material, and one run's leaf published beside another's key is a pair
that is neither half-written nor usable, so nothing downstream would
say so.

Its ACME inputs come from the rendered bootroot-internal registrar
configuration, which keeps the verb credential-free: a provisioning tool
drives it as root on the bootroot host with no OpenBao token in hand.

Closes #989
`registrar issue` reads its ACME inputs through
`load_internal_config`, which cannot tell an absent configuration from a
malformed one: a config source that is not there deserializes as an
empty one, so a host that `bootroot init` has never provisioned was
reported as a file that "expected exactly one profile, found 0" — a
diagnostic about the contents of a file that does not exist.

That is the state a provisioning tool finds when it probes before the
endpoint has been enabled, which makes it the one refusal on this
surface a caller most needs to act on. Absence is checked here, ahead of
the load, so the two stay distinguishable without changing a loader that
the daemon, rotate and status also read through.

Cover the EAB the issuance registers its account with, too. The verb
drops `[eab]` from the settings it builds and passes the credentials as
an argument, and nothing asserted the argument arrived: an account
registration against an EAB-requiring step-ca would have been rejected
on a real host while every existing assertion still passed.

Part of #989
Comment thread src/commands/registrar.rs Dismissed
CodeQL's `rust/cleartext-logging` heuristic follows any identifier
containing `secret` into a formatting sink, so the sweep's warning
tripped it on the staging path derived from `--secrets-dir`. The
path is the point of that warning — it names the directory whose
unpublished key an operator has to remove by hand — so record why
the sink is safe rather than dropping it, in the same form the
other suppressed sinks in this crate use.

Part of #989
@AcoPiper

AcoPiper commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

[Reviewer Round 1]

Changes requested — I found three correctness blockers.

  • [P1] capabilities does not resolve the effective systemd socket configuration. src/commands/registrar.rs:223 reads only a raw unit file, while the documented contract explicitly excludes .d drop-ins. A normal systemctl edit bootroot-registrar.socket override that resets ListenStream= will make systemd bind a different path while this command reports the packaged path, so the provisioning client connects to the wrong socket. It should evaluate relevant drop-ins (and stop treating an invalid higher-precedence unit as absent).

  • [P1] The output paths are not validated as distinct. src/commands/registrar.rs:514 writes the derived bundle, then writes the certificate and key sequentially. Passing the same path for --cert-path and --key-path exits successfully but replaces the certificate with the private key. Collisions with the derived ca-bundle.pem are similarly destructive. Reject all overlapping output destinations before ACME issuance.

  • [P1] Staging does not make final publication failure-safe. src/commands/registrar.rs:353 correctly stages issuance, but staging is removed before the three independent destination writes. If the bundle or certificate publish succeeds and the key publish then fails, the prior usable pair is replaced by a mismatched or incomplete one. This contradicts the issue’s “never leaves a half-written pair” requirement. Publish must roll back already-replaced files (including the bundle) on failure, with a regression test that induces a late write failure.

PR hygiene is otherwise sound: PR #990 closes #989, includes a test plan, and its visible discussion does not show an author-created issue.

@AcoPiper

AcoPiper commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

[Review Verdict Round 1: NOT_APPROVED]

`capabilities` read `ListenStream=` off one unit file and passed over a
higher-precedence unit that bound nothing. Both diverge from what the
host actually binds: a `systemctl edit` override resets the value in a
`.d` drop-in the search never opened, and systemd loads exactly one unit
file rather than falling through to a packaged one. A provisioning tool
that trusted the answer would connect to a socket that is not there.

The unit directories are now searched for the one unit file systemd
would load, the `*.conf` drop-ins from every unit directory are merged
on top of it in systemd's own dedup-and-sort order, and an installed
unit that binds nothing once merged is a refusal. Only a host carrying
no unit file at all still falls back to the shipped one; a unit named
through `--socket-unit` is still the whole answer, since it is not a
unit systemd has loaded.

`issue` also held the three output destinations to nothing. Passing one
path for `--cert-path` and `--key-path` published the certificate and
then overwrote it with the private key, and exited zero; a `--key-path`
on the derived `ca-bundle.pem` published a private key `0644` into the
deployment's trust store. They are now held to being distinct files,
compared on the absolute path with the containing directory resolved,
before an order is placed.

Publication was three independent writes with staging in front of them,
which kept a failed *issuance* from reaching the caller's paths but not
a failed *publication*: a key write failing after the certificate write
succeeded left the new leaf beside the previous key, which the issue's
"never leaves a half-written pair" forbids. Every destination is now
read back before the first is replaced and put back at the mode it had
when a step fails, with a first provisioning's files removed instead.

Part of #989
@AcoPiper

AcoPiper commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

[Author Round 1]

CI is fully green on e8f5373: 22 pass, 1 skipping (Registrar Redteam (docs-only), skipped by the change filter). That includes Quality Check, Unit & CLI Smoke, all twelve Docker E2E arms, and CodeQL — so the E2E matrix I could not run locally has now run in CI and passed.

Round 1 disposition

All three findings were valid; all three are Fixed.

[P1] capabilities does not resolve the effective systemd socket configuration — Fixed.
Both halves were right. systemctl edit bootroot-registrar.socket writes to a .d/override.conf the old search never opened, and the old loop also fell through to a lower-precedence unit when the higher one bound nothing, where systemd loads exactly one unit file. Now: the first unit file found masks the ones below it and a present-but-unreadable one is a refusal (installed_unit); bootroot-registrar.socket.d/*.conf from every unit directory is merged on top in systemd's own dedup-by-filename-then-sort order (dropin_paths), with the bare-ListenStream= reset carrying across files (listen_stream_merged); an installed unit binding nothing once merged is a refusal. Two limits are deliberate and documented: --socket-unit names a file systemd has not loaded, so nothing merges onto it, and the shipped fallback takes no drop-ins either — a drop-in with no unit to extend is inert in systemd too.

[P1] Output paths are not validated as distinct — Fixed.
ensure_distinct_outputs runs before the ACME order, covering all three destinations including the derived ca-bundle.pem, comparing absolute paths with the containing directory resolved through symlinks.

[P1] Staging does not make final publication failure-safe — Fixed.
Staging protected a failed issuance, not a failed publication. All three destinations are now snapshotted before the first is replaced and rolled back through the same writers on failure, so a restored file returns at the mode it had; a first provisioning's files are removed instead. A rollback that cannot complete names the stranded files without displacing the original error. For the requested regression test, publish_with_steps takes its steps as data, so a test can append a step that fails after the real writers have run — a_late_publication_failure_puts_every_destination_back asserts contents and modes, with ..._removes_what_the_run_created and a_failure_writing_the_key_puts_the_certificate_back (a real mid-pair failure, no injection) alongside.

Docs updated: docs/reference/registrar-provisioning-surface.md §2.1 plus new §3.3/§3.4, the mirrored docs/en/cli.md and docs/ko/cli.md, and the existing unreleased ### Added changelog entry amended in place — no new entry, since the feature has never shipped. The PR description was updated to match. 17 new tests.

@AcoPiper

AcoPiper commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

[Reviewer Round 2]

Review verdict: not approved.

Round 1 status:

  • Socket-unit/drop-in resolution: resolved.

  • Distinct output validation: resolved.

  • Failure-safe publication: still incomplete.

  • [P1] Rollback does not restore previous modes (or ownership), despite promising to leave destinations “as they were.” snapshot_destination saves only file contents (src/commands/registrar.rs:854), while restore_bundle, restore_cert, and restore_key rewrite with the normal fixed-mode writers (src/commands/registrar.rs:831). Thus a failed re-issue changes an existing certificate/bundle to 0644 and key to 0600, even if they had different modes beforehand; ownership can likewise change to the invoking process. This contradicts the documented rollback guarantee (docs/reference/registrar-provisioning-surface.md:186). Snapshot and restore the relevant metadata, and add a regression test beginning with non-default modes before inducing the late failure. The current test only starts from files produced by the same fixed-mode writers, so it cannot expose this regression.

PR linkage and test-plan hygiene are otherwise sound.

@AcoPiper

AcoPiper commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

[Review Verdict Round 2: NOT_APPROVED]

A failed publication put the previous bytes back through this surface's
own writers, so a certificate an operator had tightened to 0640 came
back world-readable and a key another account owned came back owned by
the process whose run failed. That is a change, not a restoration, and
the surface documents a restoration.

The snapshot now captures the permission bits and the uid and gid
beside the contents, and the rollback republishes at the captured mode
— which lands with the rename, so the destination is never observable
wider than the mode it is going back to — and then re-establishes the
ownership, skipping a chown that would change nothing. A chown that is
refused strands the path in the diagnostic that already names the files
to check by hand.

Part of #989
@AcoPiper

AcoPiper commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

[Author Round 2]

CI is fully green on a22101a: 22 pass, 1 skipping (Registrar Redteam (docs-only), skipped by the change filter). That covers Quality Check, Unit & CLI Smoke, all thirteen Docker E2E arms, CodeQL and the instructions drift check.

The Round 2 disposition stands as reported: the single P1 was valid and is Fixed — the rollback snapshot now captures each destination's mode and ownership beside its bytes, restores the captured mode with the rename (never observable wider than the mode it returns to) and re-establishes the uid/gid afterwards, with a refused chown stranding the path in the existing "check by hand" diagnostic. Two tests were added, including the requested regression test starting from 0640/0400/0600 files that fails against the previous implementation. Docs, changelog entry (amended in place) and the PR description were updated to match.

@AcoPiper

AcoPiper commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

[Reviewer Round 3]

Review verdict: not approved.

Round 2’s prior rollback finding is resolved: the updated snapshot restores normal modes and uid/gid.

  • [P1] Concurrent registrar issue invocations can still report success while leaving a mismatched certificate/key pair. Each process snapshots and publishes independently in registrar.rs, while the pair writer replaces the certificate and key in separate operations (fs_util.rs). For example, process A can write cert A, process B can write cert B and key B, then A can write key A; both commands succeed, but the final pair is cert B/key A. The rollback only handles an error within one process and cannot repair this successful interleaving. Serialize snapshot → publish → rollback for a destination set across processes, and add a contention regression test.

  • [P2] The new rollback does not restore all captured permission bits. It records mode & 0o7777, but restore_prior applies that mode before chown; on Unix, a subsequent ownership-changing chown clears setuid/setgid bits. This contradicts the documented promise to restore the prior permission bits. Restore ownership before the final chmod (while keeping the staged file private until then), and test a mode containing a special bit.

@AcoPiper

AcoPiper commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

[Review Verdict Round 3: NOT_APPROVED]

The rollback a failed publication runs answers only for the run that
owns it. Two `registrar issue` runs against the same paths can each
succeed and still leave one run's certificate beside the other's key:
A publishes its certificate, B publishes its certificate and key, A
publishes its key. Neither failed, so neither has anything to put
back, and nothing on disk records it.

They are separate processes, so the process-wide bundle lock does not
reach them. A publication now holds an exclusive flock on a lock file
in each directory it writes into, from before the first snapshot until
after the last write or restore, taken on the resolved directory and
in sorted order so two runs cannot each hold what the other waits for.
The issuance ahead of it stays unlocked: it writes only into its own
per-process staging directory, and holding a lock across an ACME round
trip would make one run wait out another's network exchange.

The rollback also lost setuid, setgid and the sticky bit. It carried
the whole mode through the rename and re-owned the file afterwards,
and chown(2) clears those three, so a file that carried one came back
without it while the restore reported success. They are re-applied
after the ownership instead, which also keeps a setuid bit off the
file while it is still owned by the run that failed.

Part of #989
@AcoPiper

AcoPiper commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

[Author Round 3]

CI is fully green on b185b11: 22 pass, 1 skipping (Registrar Redteam (docs-only), skipped by the change filter). That covers Quality Check, Unit & CLI Smoke, all thirteen Docker E2E arms, CodeQL and the instructions drift check.

Round 3 disposition

Both findings were valid; both are Fixed.

[P1] Concurrent runs can interleave into a mismatched pair — Fixed.
The rollback answers only for a failure of its own run, so two runs that both succeed have nothing to put back, and ca_bundle_lock is process-wide and does not reach a second process. publish_with_steps now holds an exclusive flock(2) on .bootroot-registrar-publish.lock in each directory it writes into, across the whole snapshot → publish → rollback span (src/commands/registrar/publication_lock.rs). Locks are taken on the resolved directory, so two spellings of one directory are one lock, and on at most two directories in sorted order, so two runs naming them in opposite roles cannot each hold what the other waits for. The lock file is empty, 0600, and left in place — unlinking it would hand the next two runs an inode each — and the kernel releases it on process death, so nothing goes stale. A destination named after it is refused before issuance. The issuance stays unlocked deliberately: it writes only into its per-process staging directory, and a lock held across an ACME round trip would make one run wait out another's network exchange.

The requested contention test is a_publication_holds_the_lock_across_every_step, asserted from inside the publication by a step that tries the lock on a descriptor of its own — flock binds to the open file description, so one process gets exactly the refusal a second one does. It fails against the previous implementation. The first version I wrote was the "hold the lock, assert the other run hasn't written yet" shape, and it passed against the unfixed code because yields don't wait on the blocking pool; the LOCK_NB form needs no timing at all. Four more tests cover exclusion, the two-spellings case, release on drop, a waiter publishing the whole set, and the CLI-level refusal.

[P2] The rollback loses setuid/setgid/sticky — Fixed.
The rename carries mode & 0o777 and the special bits are re-applied after the chown that would clear them, which also keeps a setuid bit off the file while it is still owned by the run that failed; it is a no-op for a file carrying none. a_rollback_puts_back_a_mode_carrying_a_special_bit starts from 04600 and restores under a gid the process can actually grant, so the chown really runs — it fails pre-fix with left: 384, right: 2432.

Docs updated: reference §3.4 and a new §3.5, the mirrored docs/en/cli.md and docs/ko/cli.md behaviour and failure lists, the module docs, and the unreleased ### Added changelog entry amended in place — no new entry, since the feature has never shipped. The PR description was updated to match. 6 new tests.

@AcoPiper

AcoPiper commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

[Reviewer Round 4]

Changes requested — one remaining blocker.

  • [P1] The new publication lock serializes only two registrar issue CLI processes, not the daemon that owns renewal of this same client leaf. The CLI holds publication_lock in registrar.rs, but daemon renewal writes the configured client pair through separate certificate/key renames without acquiring it in registrar_renewal.rs and registrar_renewal.rs. Startup issuance has the same gap in registrar_certs.rs. Thus a manual/re-invoked CLI issuance can still interleave with daemon renewal and leave certificate A beside key B while both succeed—the exact broken state the Round 3 lock aimed to prevent. Use one shared cross-process publication coordinator for the CLI, startup issuer, and renewal path, and add a regression test covering the CLI-versus-daemon writer case.

The prior Round 3 items themselves are resolved: CLI-to-CLI publication is now serialized, and rollback correctly reapplies special permission bits after ownership restoration. PR linkage and test-plan hygiene are sound.

@AcoPiper

AcoPiper commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

[Review Verdict Round 4: NOT_APPROVED]

The lock a publication of the registrar client leaf takes reached two
runs of `registrar issue` and nothing else, but the CLI is not the only
writer of that leaf. The daemon issues the same pair at start-up and
renews it afterwards, at the `[registrar_endpoint]` paths a
provisioning tool points this verb at -- pointing them at one file is
the whole point of provisioning a credential the daemon then
maintains. So a CLI run could still publish its certificate between
the daemon's certificate and its key: both report success, the pair on
disk matches neither, and neither rollback has anything to put back,
because neither run failed.

The lock moves into the library and all three writers take it, at the
destinations each of them replaces. The daemon's start-time issuance
reaches the ACME exchange and the publication as two steps rather than
one call so the lock can cover the second and not the first, which is
the rule the CLI already followed and the reason a stalled exchange
cannot hold the lock. Its renewal takes the lock outside the bundle
lock, and that is the one order every writer takes the two in, so no
pair of them can each hold what the other waits for.

Part of #989
@AcoPiper

AcoPiper commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

[Author Round 4]

Waiting on both. Here's where things stand while they run:

Round 4 disposition — the single P1 was valid and is Fixed.

The reviewer was right, and the finding was specific: the Round 3 lock served two registrar issue runs and nothing else, but the CLI is not the only writer of the registrar client leaf. The daemon issues the same pair at start-up (registrar_certs::issue_surface_pair) and renews it (registrar_renewal::publish_candidate), at the [registrar_endpoint] paths a provisioning tool points the verb at — pointing them at one file being the whole point of provisioning a credential the daemon then maintains. A CLI run could still publish its certificate between the daemon's certificate and its key, both succeed, and neither rollback have anything to put back.

What changed:

  • The lock moved from src/commands/registrar/publication_lock.rs (bin-crate-private) to src/publication_lock.rs in the library, and all three writers take it at the destinations each replaces.
  • The daemon's start-time issuance now reaches the ACME exchange and the publication as two steps (acme::issue_certificate_material then acme::publish_issued_material) rather than one issue_certificate_with_bootstrap call, so the lock covers every write and none of the network exchange — the rule the CLI already followed.
  • The renewal takes the lock outside ca_bundle_lock. That is the one order every writer takes the two in, so no pair can each hold what the other waits for.

Four regression tests, all asserted from inside the publication with a non-blocking flock attempt on a descriptor of its own (no timing anywhere), and all four fail against the unfixed code:

  • a_start_time_publication_holds_the_lock_a_cli_issuance_takes and a_publication_with_no_bundle_locks_only_the_pair
  • a_renewal_publication_holds_the_lock_a_cli_issuance_takes and a_rolled_back_publication_holds_the_lock_through_the_restore (probed through the LivePaths seam across the bundle write, the pair write, and all three restores)
  • plus a_daemon_publication_and_a_cli_publication_share_one_lock in the lock module itself.

Docs updated: reference §3.5 rewritten around three writers rather than two runs, the mirrored docs/en/cli.md / docs/ko/cli.md entries, module docs, and the unreleased ### Added changelog entry amended in place — no new entry, since none of this has shipped. The PR description was updated to match, and the branch is pushed (792f943).

One verification note I'll confirm when the runs land: the Linux-only renewal module cannot build on this Mac, so I compiled, linted (clippy --all-targets -D warnings) and tested it in a rust:latest container — all registrar_renewal tests pass there. Locally: cargo fmt --check, clippy, rustdoc, the full test suite (1081 lib + 1333 bin + integration), markdownlint-cli2, and check-docs.sh with mkdocs --strict are all clean.

@AcoPiper

AcoPiper commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

[Reviewer Round 5]

[Reviewer Round 5]

Changes requested — one blocker.

  • [P1] capabilities still does not always report the socket systemd actually binds. It calls dropin_paths for only bootroot-registrar.socket.d directories (registrar.rs), explicitly excluding systemd’s applicable type-wide socket.d drop-ins. A socket.d/*.conf that resets and rebinds ListenStream= leaves systemd serving the custom path while this command returns the unit-local path, defeating the provisioning client’s discovery contract. Resolve the complete applicable systemd drop-in hierarchy and add a reset/rebind regression test.

The prior Round 4 publication-lock finding is resolved: startup issuance and renewal now acquire the shared cross-process lock, and the tests verify it remains held throughout publication and rollback. PR linkage and test-plan hygiene are sound.

@AcoPiper

AcoPiper commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

[Review Verdict Round 5: NOT_APPROVED]

capabilities read only the unit's own bootroot-registrar.socket.d,
which is not the whole set systemd merges. A top-level socket.d/*.conf
that resets and rebinds ListenStream= moves this host's socket while
the verb kept answering the unit-local path, and a provisioning tool
that trusts the answer connects to a socket nothing is listening on.
The dash-truncated bootroot-.socket.d, which systemd derives from the
unit's name, was missed for the same reason.

Both are now resolved in systemd's own order: each unit directory
contributes the unit's name and then its dash-truncated prefixes, and
the type-wide directories come last across all of them, because that
one is the most generic and is meant to lose to anything named after
the unit. The prefix hierarchy is derived from the unit name rather
than spelled out, so it stays right if the unit is ever renamed.

A drop-in is no longer filtered by file type either. systemd's own way
of cancelling one for a single unit is a symlink to /dev/null of the
same name higher up, which masks by occupying the name; skipping it
let the copy below apply instead. It now claims its name in the merge
and contributes nothing, which is what makes a type-wide drop-in
cancellable for this unit alone.

Part of #989
@AcoPiper

AcoPiper commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

[Author Round 5]

CI is running; I'll report back when it lands. Meanwhile, the round 5 disposition:

[P1] capabilities excludes systemd's type-wide socket.d/ drop-ins — Fixed.

The reviewer was right, and the previous rationale ("a ListenStream= there would bind every socket unit to one path, so it is not a configuration this reports for") was a policy claim about a configuration systemd nonetheless honours. The verb's contract is to report what this host binds, so if an operator resets and rebinds in socket.d/, that is the answer.

I checked the behaviour against systemd's own sources rather than from memory (man/systemd.unit.xml, src/shared/dropin.c, src/basic/conf-files.c), which turned up a second missing directory and one wrong filter:

  • Type-wide socket.d/ is now resolved, ordered as systemd orders it: the type directories are appended after every unit directory's name-specific ones, so a name-specific drop-in outranks a type-wide one of the same filename, while files with different names still apply in lexicographic order wherever they came from.
  • Dash-truncated bootroot-.socket.d/ was missing for the same reason — systemd derives it from any unit name containing a dash. unit_dropin_names derives the hierarchy from the unit name (mirroring systemd's trailing-dash chop and leading-dash stop) instead of spelling it out, so a rename of the unit keeps it right.
  • /dev/null masks. The old code filtered drop-in entries by is_file(), so the symlink systemd documents for cancelling a drop-in was dropped from the list and the copy below it applied instead — the exact opposite of masking. A .conf entry now claims its name in the merge regardless of file type and contributes no setting, which is what makes a type-wide drop-in cancellable for this unit alone.

Seven tests added; six fail against the previous implementation (the type-wide rebind, one below the unit, filename ordering across the two, and both /dev/null mask cases, plus the asserted search order). Two are pure derivation tests: the prefix hierarchy (including foo--.service and -foo.service) and the full directory list against systemd's order.

Docs updated: reference §2 (the paragraph that declared socket.d/ unread is gone, replaced by the full merge rule), the mirrored docs/en/cli.md and docs/ko/cli.md, the module and function docs, and the unreleased ### Added changelog entry amended in place — no new entry, since none of this has shipped. PR description updated to match.

Local verification: cargo fmt --check, clippy --all-targets -D warnings, the full test suite (1081 lib + 1342 bin + integration), markdownlint-cli2, and check-docs.sh with mkdocs --strict are all clean. One bootroot_cli test failed in the whole-suite run and passed on a rerun of that binary — the known ephemeral-port race between sibling tests, not a regression. No E2E script exercises registrar capabilities, and this change touches only its drop-in resolution and docs, so the matrix is not implicated.

@AcoPiper

AcoPiper commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

[Reviewer Round 6]

[Reviewer Round 6]

Changes requested — one blocker.

  • [P1] Distinct-output validation misses aliases through a symlinked ancestor that does not yet exist. output_identity only canonicalizes the immediate parent; when that parent is absent, it falls back to its lexical path. Thus --cert-path /link/new/registrar.pem and --key-path /real/new/registrar.pem, where /link -> /real and new/ does not yet exist, pass ensure_distinct_outputs. Publication then creates new/ through the symlink and writes both destinations to the same actual file, overwriting the certificate with the key while reporting success. Resolve the nearest existing ancestor before appending the unresolved suffix (and retain lexical normalization), then add a regression test using this missing nested directory case.

The Round 5 systemd finding is resolved: the updated resolver includes both socket.d and dash-prefix directories, preserves systemd’s precedence/filename merge behavior, and correctly handles /dev/null masking.

@AcoPiper

AcoPiper commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

[Review Verdict Round 6: NOT_APPROVED]

The distinctness check canonicalised only the destination's immediate
parent, so two paths that alias solely through a directory neither
carries yet were compared as typed. With a symlinked ancestor, that
lets `--cert-path <link>/new/registrar.pem` and `--key-path
<target>/new/registrar.pem` past the check; the publication then
creates `new/` through the link and the key overwrites the
certificate at the one file they always shared, with the run
reporting success.

Resolve the nearest ancestor that does exist and apply the remaining
components to it. Applying them lexically is sound because they are
not there: a directory that does not exist is not a symlink, so a
`..` past it can only mean the level above.

Part of #989
@AcoPiper

AcoPiper commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

[Author Round 6]

CI is fully green on be575b7 — 22 pass, 2 skipping (CodeQL gated by the change filter, and Registrar Redteam (docs-only)). That covers Quality Check, Unit & CLI Smoke, all thirteen Docker E2E arms, CodeQL and the instructions drift check.

Round 6 disposition

[P1] Distinct-output validation misses aliases through a symlinked ancestor that does not yet exist — Fixed.

The reviewer was right and the reproduction was exact. output_identity canonicalised only the destination's immediate parent and fell back to the lexical path when it was absent, so --cert-path /link/new/registrar.pem and --key-path /real/new/registrar.pem (with /link -> /real, no new/ on either side) compared as two different paths. write_cert_and_key then creates the missing levels through the link (cert_group::ensure_key_parent_dir), and the key lands on the certificate at the one file they always shared, with the run reporting success.

output_identity now resolves the containing directory through a new resolve_existing_ancestor: it walks up to the nearest ancestor that does exist, canonicalises that — following every symlink on the way to it — and applies the components below it to the result, normalising .. and . lexically. That normalisation is sound precisely because those components are absent: a directory that does not exist is not a symlink, so a .. past it can only mean the level above. The walk stops at the root or a prefix and returns the path as typed, so nothing can loop or panic on an unresolvable root. The existing certs/../certs/leaf.pem case is unaffected — that directory exists, and canonicalize still answers on the first iteration.

I checked the sibling resolution in publication_lock::hold_blocking for the same gap and it does not have one: it create_dir_alls each directory before canonicalising it, so the lock is already taken on a resolved path even on a first provisioning.

Four tests added, in src/commands/registrar/tests.rs:

  • two_spellings_through_a_link_to_a_directory_not_yet_created_are_refused — the reviewer's case verbatim;
  • a_key_path_on_the_bundle_through_a_link_is_refused — the same aliasing reaching the derived bundle, which is the 0644 private-key outcome;
  • distinct_destinations_below_unborn_directories_are_accepted — the guard that resolving more does not collapse two genuinely different destinations that share only an ancestor;
  • the_unborn_tail_is_normalised_onto_the_resolved_ancestor — the derivation itself, .. included.

The first two fail against the previous implementation (expect_err panics: one file below a link, the key on the bundle below a link); the other two are guards on the new behaviour and pass either way.

Docs updated: reference §3.3, and the mirrored docs/en/cli.md / docs/ko/cli.md behaviour bullets, all three now stating that a directory the publication is about to create is resolved too and why the unborn tail is applied lexically. The function's own rustdoc carries the rationale. No changelog change — the unreleased ### Added entry already says the three destinations must be distinct files, checked before anything is issued, which remains accurate; nothing here has shipped, so there is no ### Fixed line to add. The PR description was updated in both the behaviour bullet and the test plan.

Local verification: cargo fmt --check, clippy --all-targets -D warnings, rustdoc, the full suite (1081 lib + 1346 bin + integration, all green on the first run this time), markdownlint-cli2, and check-docs.sh with mkdocs --strict are all clean. No E2E script exercises registrar issue or registrar capabilities (grep over scripts/ finds none), and this change touches only the destination comparison and docs, so the preflight matrix is not implicated.

@AcoPiper

AcoPiper commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

[Reviewer Round 7]

[Reviewer Round 7]

Approved — no unresolved findings.

The Round 6 P1 is resolved: resolve_existing_ancestor now resolves the nearest existing ancestor before normalizing the unborn tail, and the symlinked-missing-directory regression is covered in tests.rs.

The updated implementation meets the issue contract, including the composed identity, socket-unit discovery, atomic/reversible publication, shared daemon/CLI publication locking, and meaningful integration coverage. PR linkage and test plan are sound.

@AcoPiper

AcoPiper commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

[Review Verdict Round 7: APPROVED]

@AcoPiper

AcoPiper commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

Suggested squash commit

Title

Add the registrar provisioning subcommand

Body

The registrar endpoint ships, but nothing provisions it. An installer
had no way to learn that the surface exists, nor to obtain the client
credential that reaches it, so an install could only defer the
registrar and roxyd's certificate could not be minted on a real host.

Two of the consumer's pinned expectations contradict contracts this
repository owns, and both are resolved in its favour, because the
identity rule and the socket are bootroot's to define. `issue`
composes the identity from the host label and the domain rather than
accepting a composed name: a name the caller supplies is a name that
can disagree with `recognize_registrar_client`, leaving a registrar
that authenticates to nothing. `capabilities` reports what this host
binds rather than a value derived from the caller's layout.

That pathname is the installed socket unit's effective ListenStream=,
resolved the way systemd resolves it — the unit directories in
systemd's own precedence order, the first unit file masking the ones
below it, and every drop-in systemd would apply merged on top, the
unit's own, the dash-truncated one and the type-wide one. Reading less
would report the unit-local path while systemd had bound another, and
send the caller to a socket that is not there.

Issuance stages its material and publishes only once every byte is in
hand, so a run that fails part-way leaves no half-written pair. The
publication is reversible: each destination is captured with its
bytes, its permission bits and its ownership before it is replaced, so
a failure does not leave a new leaf beside the previous key, nor
reopen at this surface's mode a file an operator had tightened. That
rollback answers only for a failure of its own run, so publications
are serialised on a lock per destination directory. The lock lives in
the library rather than the CLI because three writers replace this
material across two processes -- this verb, the daemon's start-time
issuance and its renewal -- and a CLI-only lock would leave the daemon
free to publish between this run's certificate and its key, with both
reporting success over a pair that matches neither.

The daemon is otherwise unchanged. It still learns its own socket from
the descriptor systemd hands it, no configuration key names one, and
no part of mint, deregister or renewal is reimplemented here.

Closes #989

@AcoPiper
AcoPiper merged commit 415f5f1 into main Sep 8, 2026
23 checks passed
@AcoPiper
AcoPiper deleted the AcoPiper/issue-989 branch September 8, 2026 11:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add the registrar provisioning subcommand

2 participants