Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view

This file was deleted.

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -22,20 +22,40 @@ profiles:
- "quality-observations-*"
# branch: "main"

# For a local source:
# For a local source that reads a canonical file already on disk:
# - change `transport` above to `"local-folder"`
# - remove both `auth` and `github`
# - add this block:
# local_folder:
# path: "artifacts/quality"

# For results the reading application fetches itself — a local test report,
# a platform that already holds the run — use the host transport. The
# provider name is resolved by whoever reads this repo, so which providers
# exist depends on the application, not on Quality:
# - change `transport` above to `"host"`
# - remove `observation_path`, `auth`, and `github`
# - add this block:
# host:
# provider: "local-reports"
# options:
# path: "playwright-report/report.json"
# report: "playwright-report/index.html"

# Authoring rules:
# - One profile represents one workflow or local folder.
# - observation_path is relative to the downloaded artifact or local_folder.
# - One profile represents one workflow, local folder, or host provider.
# - observation_path is relative to the downloaded artifact or local_folder. A
# host profile omits it: it addresses no file.
# - GitHub artifact_names may select several uploaded archives from one run.
# Every matching observation_path uses the same canonical contract; Quality
# merges their observations.
# - A local-folder profile reads exactly one observation_path.
# - A host profile names `host.provider`. An application that does not register
# that provider reports it as a diagnostic rather than reading nothing, so a
# profile written for one reader stays legible to another.
# - `local-reports` is the provider Quality ships: it reads a Playwright JSON
# report from the working tree and points each result at the HTML report a
# reviewer opens. It records no commit unless `options.commit` pins one.
# - Raw JUnit, Playwright, telemetry, or custom reports are producer inputs.
# Convert them in the workflow with `quality-tools observations`; source
# configuration contains no parser or format selection.
Expand Down
59 changes: 41 additions & 18 deletions agent-skills/quality/references/improve/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,8 @@ Classify each gap before editing:
3. **Evidence strength:** a mapped method cannot establish the full claim or lacks
the required execution context/gate.
4. **Source acquisition:** credentials, repository/workflow selection, artifact
names, or local-folder path prevent results from loading.
names, local-folder path, or an unregistered host provider prevent results
from loading.
5. **Artifact emission:** the workflow emits no canonical observation file or
emits it at the wrong path.
6. **Producer format:** the canonical file has an invalid version, envelope,
Expand Down Expand Up @@ -179,7 +180,7 @@ acquisition and resolution have been ruled out.
<quality-observations.json>
```

- Observation config: compare with the configuration schemas in `assets/`, then
- Observation config: assess the project and read its `INVALID_*` diagnostics, then
run the relevant assessment. Engine diagnostics verify acquisition and graph
joins.
- Implementation or verification-method changes: run their owning verification command before
Expand Down Expand Up @@ -209,23 +210,43 @@ or requires a human decision.
- `.quality/config/observation-sets.yaml`
- `.quality/config/views.yaml`

Use the configuration templates and schemas under `assets/`. Use
`quality-observations.template.json` as the canonical output example; obtain its
current schema from `quality-tools observations schema`, not a bundled copy.
Use the configuration templates under `assets/`. Never vendor a copy of a
schema: a copy cannot be checked against the contract and drifts the moment the
contract moves.

### Sources
For the observation manifest, obtain the current schema from
`quality-tools observations schema`.

One profile represents one acquisition integration, such as one GitHub Actions
workflow or one local result folder. It answers only:
Configuration files — sources, sets, views — are validated by the engine itself
when you assess the project: an invalid profile, set, or view reports an
`INVALID_*` diagnostic naming the exact `yamlPath`. That is the authority, since
it is the parser that actually runs. Read the diagnostics rather than
pre-validating against a schema.

Use `quality-observations.template.json` as the canonical output example.

- which transport fetches results: `github-actions` or `local-folder`
- which `observation_path` contains canonical `quality-observations.json`
content
### Sources

A source never selects a parser. Raw JUnit, Playwright, telemetry, or custom
gate output must be converted by its producer before the source reads it. Do
not create a source profile until the canonical file exists or its emit step is
being added in the same authorized change.
One profile represents one acquisition integration, such as one GitHub Actions
workflow, one local result folder, or one provider the reading application
supplies. It answers only:

- which transport fetches results: `github-actions`, `local-folder`, or `host`
- for the two file transports, which `observation_path` contains canonical
`quality-observations.json` content
- for `host`, which `host.provider` the reading application resolves

A file-based source never selects a parser. Raw JUnit, Playwright, telemetry,
or custom gate output must be converted by its producer before the source reads
it. Do not create a source profile until the canonical file exists or its emit
step is being added in the same authorized change.

A `host` profile is the exception, and only because the reading application —
not this configuration — owns the fetch. Its provider may read a native report
directly. The engine still normalizes, resolves, and diagnoses every record it
returns, so a host provider gets no record past a check a canonical file must
pass. Which providers resolve depends on who reads the repo; one that is not
registered is reported as a diagnostic rather than read as nothing.

A local-folder profile reads one file. A GitHub Actions profile may select
several uploaded artifacts from one workflow run; every matching
Expand Down Expand Up @@ -329,6 +350,7 @@ Follow this sequence. Do not ask the user to choose a parser or config shape.
GitHub Actions metadata comes from `GITHUB_SHA`, `GITHUB_REF_NAME`, and
`GITHUB_RUN_ID`. Outside GitHub Actions, supply `--commit`; `--branch`,
`--run-id`, `--run-url`, and `--observed-at` are optional.

4. **Schema-validate before upload.**

```bash
Expand All @@ -346,9 +368,10 @@ Follow this sequence. Do not ask the user to choose a parser or config shape.
every one uses the same contract. Raw native reports may remain alongside
the canonical file for diagnosis; the quality engine never parses them.
6. **Configure the transport.** Copy
`assets/observation-sources.template.yaml`. Set `transport`,
`observation_path`, and either `github` or `local_folder`. Source
configuration contains no parser list or format selection.
`assets/observation-sources.template.yaml`. Set `transport`, then either
`observation_path` plus `github` or `local_folder` for a file transport, or
`host.provider` for a host transport. File-transport configuration contains
no parser list or format selection.
7. **Add the profile to an observation set.**
8. **Run `assess`.** Verify source acquisition first, then verify every
observation resolves to the intended evidence identity. Use the engine's
Expand Down
Loading