Support Mounts and Secrets in Compose Provider Extensions #14163

Activity

1. phr34k

   added

   kind/feature

   status/0-triage

   on Sep 2, 2026
2. ndeloof

   removed

   status/0-triage

   on Sep 2, 2026
3. ndeloof commented on Sep 16, 2026

   

   ndeloof

   on Sep 16, 2026

   Contributor

   More actions

   Following up on this now that #14230 is proposing the mount/secret injection mechanism itself. One thing I'd like to explore before we settle on that as the shape: right now it relies purely on depends_on + implicit injection (a provider's plugin message gets appended to the Volumes/Secrets of any service that depends on it), which has two downsides — it's invisible from the consuming service's own compose.yaml entry, and nothing disambiguates which provider fills which secret/volume when a service depends on several.

   Alternative: let a top-level secrets:/volumes: entry declare who populates it:

   secrets:
     db_password:
       populated_by: database   # a service with a `provider:` definition
   
   services:
     database:
       provider:
         type: managed-db
     app:
       secrets:
         - db_password           # no depends_on needed — derived automatically

   populated_by alone would carry three responsibilities, each close to something the codebase already does:

   1. Skip normal validation. loader/validate.go currently requires a non-external secret to declare file or environment. populated_by becomes a second exemption alongside External — a secret filled at runtime by a named provider doesn't need any of the usual sources declared upfront.

   2. Derive the dependency edge automatically. Compose already does exactly this for build.additional_contexts — addBuildDependencies (pkg/compose/build.go) scans for the service: prefix and injects an implicit build dependency. Same pattern here: scan every service's secrets:/volumes: references, and if the resolved definition declares populated_by: X, inject an implicit (required) depends_on: X.

   3. Resolution — already free, for secrets. resolveFileContent (pkg/compose/secrets.go) already treats Content as the primary resolution source, and already treats an empty Content (with no Environment either) as "nothing to inject, skip silently" — no error. That's exactly the behavior needed for an unpopulated dynamic secret. The provider side just needs to end up setting Content on project.Secrets[name], and the existing injection path takes it from there with no new code.

   For volumes there's no Content, but the same signal exists: the classic way to make a named Docker volume point at a location a provider owns is a local-driver bind — driver_opts: {type: none, o: bind, device: <path>}. A provider populating that needs no new creation code path either: createVolume (pkg/compose/create.go) already passes Driver/DriverOpts straight through to VolumeCreate. The "still empty" signal for a volume becomes "no driver/opts set", the same role Content == "" plays for a secret.

   Ordering falls out for free too: secrets/volumes are injected per-container at ContainerStart time, inside the existing dependency-ordered startup loop — not a global upfront pass — so once the derived edge exists, the provider is guaranteed to have already run by the time the consumer's container starts.

   One gap to close either way: since empty Content/unset driver opts are silently treated as "nothing to inject" today, a provider that fails to actually populate a secret/volume it's declared populated_by for would currently produce a container silently missing it. This should fail the dependent service's startup explicitly instead — symmetric to service_completed_successfully.

   @glours curious what you think of this shape vs. the pure depends_on-based injection in #14230 — happy to write it up more formally if it seems worth pursuing alongside (or instead of) that PR.