feat(platform): D2: spec deployment-scoped artefacts - #529
Conversation
dannash100
changed the title
passcod
left a comment
passcod
left a comment
There was a problem hiding this comment.
good in thrust, i have notes tho
| An artefact may belong to a group, and one that does is offered only to that group. | ||
| An artefact belonging to no group is offered to every group, which is what a version's installers, migrations, and manifests are: properties of the version and of nothing narrower. | ||
|
|
||
| Belonging to a group is therefore an addition rather than a requirement, and the artefacts published today are unaffected. |
There was a problem hiding this comment.
this specifies "in time" which is disallowed by spec styleguide: it is true in our today but as soon as this ships makes no sense, since a reader in the future will already have "artefacts that belong to a group" in their today
| id: ART | ||
| --- | ||
|
|
||
| # Version artefacts |
There was a problem hiding this comment.
this file's filename and i believe the code spells it "artifact" so we probably should keep to that in the spec
| An artefact belonging to no group is offered to every group, which is what a version's installers, migrations, and manifests are: properties of the version and of nothing narrower. | ||
|
|
||
| Belonging to a group is therefore an addition rather than a requirement, and the artefacts published today are unaffected. | ||
| Resolution keys on type, platform, and group together, so a group-scoped artefact and an unscoped one of the same type do not displace each other and each group is offered its own. |
There was a problem hiding this comment.
This reads like an ungrouped viewer will query (version, type, platform) and see zero or one answer, and a grouped viewed will query (version, type, platform) and potentially see multiple answers?
| A group-scoped artefact's existence is disclosed only to a caller it is offered to: a caller that names or guesses one it is not offered is answered as though it did not exist, so which groups hold one is not enumerable through the artefact surface. | ||
|
|
||
| A group scope exists because some artefacts are derived from a group's own data and are wrong for anyone else, a reporting schema being the case that motivates it (see [RPT](../public-server/reporting-schemas.md)). | ||
| Such an artefact is published into that group's own object storage, over a credential Canopy issues the publisher for the run, and is read back through Canopy, which passes it only to a caller the artefact is offered to. |
There was a problem hiding this comment.
What is a group's own object storage? This might be over- or under-specified: if this is introducing a new way to refer to objects (e.g. s3:// URLs, exclusively proxied) then this is underspecified, a new concept not introduced before, with no information in the spec about how auth to the object backend is done; if this is talking about the usual mechanism of an HTTP URL, then there's no guarantee that the URL is pointing to object storage only for the group, and putting it into this spec is over-reaching).
|
|
||
| ## Registration | ||
|
|
||
| A releaser device registers artefacts against a version over its own credential, which is what a product's release automation uses when it publishes. |
There was a problem hiding this comment.
There's a lot of "which is what {existing thing} does {when/for}" phrasing; in this one example and elsewhere in this draft. This again documents things that happen to be true today, but may not be in the future, so it's ambiguous whether the spec is over-reaching (specifying things that are not its business) or under-specified (defining hard requirements only by happenstance examples).
| ## Registration | ||
|
|
||
| A releaser device registers artefacts against a version over its own credential, which is what a product's release automation uses when it publishes. | ||
| Registering a group-scoped artefact requires being authorised for that group, and the group is named on the registration rather than inferred from the caller. |
There was a problem hiding this comment.
That means we'd need N releaser devices for N groups?
|
|
||
| A releaser device registers artefacts against a version over its own credential, which is what a product's release automation uses when it publishes. | ||
| Registering a group-scoped artefact requires being authorised for that group, and the group is named on the registration rather than inferred from the caller. | ||
| A restore consumer registers the artefacts its builds produce for the groups its own declarations authorise it to read, so a build needs no releaser credential to publish what it made. |
There was a problem hiding this comment.
That feels like a significant capability to be buried here, and also like something that should probably be gated to only some restores / restore-consumers. For our practical purposes, PGRO would be allowed, and only for those workflows that need it (ie the ones producing artifacts), but "someone restoring a backup" should not be.
Further, very slight over-reach in that this is naming "its builds" where one could plausibly imagine such a process wanting to register an artifact that isn't technically something it built.
| An operator may also record one directly. | ||
| Canopy records which device registered an artefact and, where the registration names one, the run that produced it, so an artefact that arrived by automation is distinguishable from one entered by hand and traceable to what made it. | ||
|
|
||
| ## What Canopy does not know |
There was a problem hiding this comment.
This and the "out of scope" section is running afoul of a pretty subtle spec-writing principle and should be carefully reconsidered: "documenting absences" vs "documenting anti-requirements". That is, are we specifying things that must not be implemented now and in the future, because doing so violates a property we must uphold (if so: which)? Or are we specifying things we're not doing in this effort but could be done tomorrow? (in which case, they shouldn't be disclaimed at all and should not be present in the spec, because a future effort could read this spec and refuse to proceed with a design on the grounds that it is verboten)
|
|
||
| ## Group-scoped artefacts | ||
| A group-scoped artifact rests as an object in its own group's storage, under a prefix of its own apart from the group's backup repo, and a registration placing one anywhere else is refused. | ||
| Canopy reads it on a caller's behalf by assuming the group's storage role confined to reading that prefix (see [BAK](../public-server/backup.md)), and streams the bytes to the caller, so the file rests only in the group's storage and is readable only through Canopy. |
There was a problem hiding this comment.
Yeah, see, the problem with this is that now we're putting artifact data into a backup bucket. That's a very different access, retention, and audit pattern we're overloading. It also means that we're billing artifacts as backups, or alternatively not billing backups as backups anymore.
There was a problem hiding this comment.
Also this means these artifacts are no longer "canopy is just an index", they're "canopy stores these artifacts". That might justify being a whole separate spec and mechanism (even if it's exposed in the same way UI/API wise).
There was a problem hiding this comment.
By which point the language and API surface can change:
- not registration, but upload
- uploader has to provide a digest for integrity, but canopy verifies that on upload, and then generates its own suite of digests (across different algorithms) for downloaders to use
- etc
2 participants
Spec only, proposed.
Some artefacts are derived from one deployment's data and are wrong for anyone else; today every artefact is fleet-wide and the artefact reads are unauthenticated.
Base for #525.