Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 52 additions & 0 deletions .agents/skills/accuracy-check.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# Accuracy check

Go through a page and check that everything it claims is actually true in the
code.

## Where to look

- Anything about the web app, its screens, settings, or API: `OpenFn/lightning`.
- Anything about the CLI, deploying, `project.yaml`, or how job code is
compiled: `OpenFn/kit`.
- Anything about a specific adaptor's functions or credentials:
`OpenFn/adaptors`, under `packages/<adaptor name>/`.

Use the main branch unless the page names a version. Write down which commit
you checked against.

## Steps

1. **List the claims.** Read the page and number every statement a reader
could act on: code samples, function names and arguments, button and menu
names, settings and their defaults, version numbers, file paths, limits.
Skip the motivational text and the "in v1 this used to be" notes.
2. **Check each one.** Find the code that backs it up and note the file and
line. For job code samples, run `openfn compile` to make sure they at least
compile. Only run commands that read; never deploy, delete, or hit a live
OpenFn instance. If a name is missing from the code, look at the git
history in case it was renamed.
3. **Mark each claim** pass, fail, or uncertain. Uncertain covers things you
cannot verify from code: values set per deployment, anything that needs a
running system, pricing, and policy. Those belong to the product team.
4. **Decide what to do about failures.** Fix it if the right value drops
straight into the sentence (a flag name, a default, a label). Suggest it if
the paragraph would need rewriting. Ask if you cannot tell whether the docs
or the code is the one that is wrong, and say which you suspect.

## Generated adaptor pages

The pages under `adaptors/packages/` are built from code comments in
`OpenFn/adaptors`. Do not edit them here. Instead, find the comment in
`packages/<name>/src/` and write up an issue: the page URL, the source file
and line, what the docs say, what the code does, and the corrected comment.
Put it in the PR under "Upstream issues". Only file it if the user asked you
to.

If a hand-written overview page (`adaptors/<name>.md`) repeats the same
mistake, fix that one normally.

## What to report

For each page: the commits you checked, how many claims passed, failed, or
were uncertain, a table of the claims with their evidence, and the findings
in the standard format. Apply your fixes, run Prettier, and run `yarn build`.
59 changes: 59 additions & 0 deletions .agents/skills/fresh-user-eval.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
# Fresh-user evaluation

Read one page as if you had never heard of OpenFn, then try to do what it
says. The point is to find where a newcomer would get lost. Run this after the
accuracy check, so the facts are already right.

## Steps

1. **Read it cold.** Pretend you know what an API, a webhook, JSON, and a
terminal are, and nothing else. Do not follow links. Do not look at the
page's history. When you finish, write one sentence saying what the page
teaches and who it is for. If you cannot, that is your first finding.

2. **Try to do it.**
- If it is a how-to, follow the steps. For anything in the web app, check
the button or screen really exists in the Lightning code. For CLI steps,
run the commands.
- If it explains a concept, explain it back in two sentences. Then think of
three questions a newcomer would ask and see if the page answers them.
- If it is a reference table, pick three rows and see if you could use each
one from its description alone.

Every time you have to guess what a word means, cannot find what the page
points at, read a sentence two ways, need something the page assumed you
had, find the steps in the wrong order, or cannot tell whether you
succeeded, write it down.

3. **Check your guesses** against the code or nearby pages. If you guessed
wrong, the page definitely needs that information. If you guessed right, it
probably still does.

## Sorting the findings

- **Fix**: one missing fact that fits in one sentence and that you have
verified. Add it.
- **Suggestion**: anything bigger, such as a new subsection, an example, or a
rewrite. Propose the text but do not add it.
- **Question**: you could not find the answer, or it depends on who the page
is for.

Do not touch the tone or voice.

## Scores

Give two scores from 1 to 5, each with a one-sentence reason.

- **Readability.** 5: understood everything on one read. 3: got the gist but
had to work at it. 1: could not follow it without outside knowledge.
- **Completeness.** 5: could finish the task from this page alone and knew
when I was done. 3: got there, but only by following links or guessing.
1: the page does not actually say how to do it.

## What to report

What the page teaches, who it is for, what you tried, your findings in the
standard format, and the two scores.

Do not evaluate the generated adaptor pages. Do not write a new page; if the
page a user needs does not exist, that is a job for gap analysis.
69 changes: 69 additions & 0 deletions .agents/skills/gap-analysis.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
# Gap analysis

Work out what a section of the docs should cover but does not. Produce a
ranked list. Do not write the missing pages unless you are asked to.

## Steps

1. **List what the docs cover.** For every page in the section, note what it
explains, using the headings and tables. Also note where it links out to
other sections, and check those pages exist.

2. **List what the product has.** Look at the part of the code that matches
the section. For the web app, that is the routes and screens in
`OpenFn/lightning`. For the CLI, read `packages/cli/src/cli.ts` and
`commands.ts` in `OpenFn/kit`; that is where commands are registered, and
it shows things `--help` hides. Use `openfn --help` only as a fallback if
the CLI happens to be installed. For job writing, look at what
`packages/common` exports in `OpenFn/adaptors`. For deployment, read
`DEPLOYMENT.md` and the runtime config in Lightning.

If you cannot reach the repo you need, say so in the report and skip that
part. Do not fill the gap from memory.

3. **Compare the two lists.** Before you call anything a gap, search the
whole docs folder, the articles, and the adaptor overviews. It might be
documented somewhere else. Search for the thing as a user would type it,
not the bare noun: "openfn metadata" settles the question in one hit,
while "metadata" matches twenty pages of ordinary prose. Label each gap
as one of:
- **Missing page**: nothing in the docs mentions it.
- **Partial page**: the right page exists but does not cover this.
- **Misplaced**: it is documented, but not where a user would look.
- **Stale**: it describes an old version. Hand these to the accuracy check.

Also note anything the docs describe that the product no longer has.

4. **Look for user evidence**, if you have access to it: issues on the docs
repo, the community forum, support channels, search analytics. Count how
often a topic comes up. Do not quote anyone. If you have no access, say so,
and do not make up demand.

5. **Rank.** Score each gap from 1 to 5 on three things: how many users it
affects, how bad it is to be without it, and how much evidence you have
that people want it. Add those three up; that is the impact score, and
you sort by it. Separately, score how easy it is to write (5 means a
paragraph, 1 means a whole tutorial) and show it in its own column. Use
ease only to break ties. Folding it into the sum lets a trivial gap
nobody asked for outrank a genuinely undocumented feature.

## What to report

For each gap: its label, its impact score and ease score, what is missing,
your evidence, where it should go (which file, and where in the sidebar or
which heading), and a rough outline. Put the top ten in the PR description
and collapse the rest.

The report is the deliverable. A normal run of this skill opens a PR with an
empty or near-empty diff and a long description. That is expected.

## What you may change

Only one thing: add a single sentence linking to a page that clearly already
covers the topic. Everything else is a suggestion.

Thin generated adaptor pages are not gaps here; they are code-comment issues
for `OpenFn/adaptors`. If a feature is behind a feature flag, or a command is
deliberately hidden in code (for example a CLI command with `describe:
false`), it is not a gap. Do not propose documenting it. Note it and ask the
product team if it looks like it should be public.
55 changes: 55 additions & 0 deletions .agents/skills/lint.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# Lint

Run this first on any section. These are mechanical checks. Most of what you
find, you can fix on the spot.

Before you start, read `glossary.yml` (the approved terms) and
`style-exceptions.yml` (things humans have told us not to flag). Skip any
finding that matches an exception.

Do not lint the generated adaptor pages, the old v1 docs, or translations
marked human-reviewed.

## What to check

Only look at prose. Ignore code blocks, URLs, and front matter.

1. **Terminology.** If a page uses a spelling listed under `variants` in the
glossary, replace it with the approved term. Always change "adapter" to
"adaptor". Fix. Multi-word terms can be split across a line break by
Prettier's wrapping, so match against text with newlines collapsed to
spaces.
2. **Headings.** No `#` headings in the body (the title comes from front
matter). No jumping from `##` to `####`. No two headings with the same
text. Fix. If a page mixes Title Case and sentence case, suggest a change
rather than making it, because headings double as link anchors.
3. **Internal links.** Every link to another docs page must work. The easiest
way to check is `yarn build`, which fails on broken links. Fix it if the
right target is obvious. Otherwise ask.
4. **External links.** Try each one twice. A 404 or 410 means it is dead.
Replace it if there is a clear replacement; otherwise suggest removing it.
A 403 or 429 does not mean dead, so leave those alone and note them.
5. **Orphan pages.** A page that is not in any sidebar and not linked from
anywhere. Suggest adding or removing it. Do not decide yourself; someone
might be drafting it.
6. **Front matter.** Must be valid YAML and must have a `title`. If the title
is missing, take it from the page's first heading. Fix.
7. **Code blocks.** Every fenced block needs a language: `js`, `json`,
`yaml`, `bash`, or `text`. Fix.
8. **Alt text.** Every image needs alt text that says what the image shows.
"Screenshot" does not count. Write it from the surrounding paragraph. Fix.
9. **Callouts.** `:::tip` and similar blocks need a blank line after the
opening and before the closing. Fix.

## How to fix

Change only the line with the problem. When you are done, run Prettier on the
files you touched, then `yarn build`. Do not commit until both are clean.

## What to report

Give the counts (files checked, fixes, suggestions, questions), then one line
per finding in the standard format.

If a reviewer undoes one of your fixes, do not push back. Suggest they add an
entry to `style-exceptions.yml` so it is not flagged again.
80 changes: 80 additions & 0 deletions .agents/skills/release-review.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
# Release review

Look at what the product shipped recently and make sure the docs caught up.
With no arguments, review every release in `OpenFn/lightning`, `OpenFn/kit`,
and `OpenFn/adaptors` from the last month. Someone can narrow it to one repo,
a date range, a release tag, or a single PR.

This is the inverse of the accuracy check. That skill starts from a docs page
and looks for the code. This one starts from what changed in the code and
looks for the docs.

## Two repos are involved

The releases live in the product repos. The docs live here. You need both.

- **Running in the docs repo** (the usual case): clone each product repo you
need into a scratch directory outside this one. Fetch tags. Do not modify
the product repos.
- **Running inside a product repo** (someone asks "did the docs keep up with
this release?"): clone `OpenFn/docs` into a scratch directory and do the
same work. Docs changes always go in a branch and PR on the docs repo.

Say at the top of your report which repos, tags, and dates you covered.

## Steps

1. **Start from the changelogs, not the diffs.** Each repo keeps one:
Lightning has a single `CHANGELOG.md`; kit and adaptors have one per
package under `packages/<name>/CHANGELOG.md`. Read every entry released in
the period. These are already a curated list of user-facing changes, so
they are cheaper and more reliable than reading every PR. Skip the
Unreleased section; docs describe what has shipped.

2. **Turn each entry into a plain statement of what changed for a user**: a
new feature, a renamed button, a new CLI flag, a changed default, a removed
option, a new setting. Drop entries that are internal (refactors,
dependency bumps, test changes). If an entry is too vague to act on, open
the PR it links to and read the diff. Only then.

3. **Find the docs that talk about it.** For each change, search `docs/`,
`articles/`, and `adaptors/*.md` for the feature, the old and new names,
the flag, or the setting. Search for things the way a user would type
them ("openfn pull --beta", not "beta"). Note every page and line.

4. **Decide what each page needs.**
- **Now wrong.** The docs describe the old behaviour. Fix it if the new
behaviour is clear and slots into the existing sentence. Otherwise
suggest wording.
- **Now incomplete.** The page is still right but does not mention the new
thing. Suggest where the paragraph or table row goes and draft it.
- **Nothing in the docs.** A new feature with no home. Suggest a page or
section and a short outline. Do not write the page unless asked.
- **Screenshot affected.** The change alters a screen that appears in an
image. List the image for the retake list. Never retake it.

A feature behind a feature flag, or a command hidden in code, is not a
docs gap. Note it and ask the product team if it looks like it should be
public.

## Adaptors are different

Changes to adaptor functions flow into the docs automatically through the
generated reference pages, so you do not need to chase those. For adaptors,
ask only two questions: is there a new adaptor with no overview page in
`adaptors/`, and did a change break something a tutorial or guide relies on.

## What to report

Start with one paragraph: the repos and period covered, how many releases,
how many user-facing changes, and how many docs pages are affected. Then the
findings in the standard format, grouped by page. Finish with images that
need retaking and new pages that are needed.

Apply the fixes, run Prettier and `yarn build`, and open a docs PR that links
to the releases it covers. Suggestions and questions go in the PR
description. If the period had no user-facing changes, say so and do not open
a PR.

This skill suits a monthly schedule with default arguments, plus a manual run
after any large release.
74 changes: 74 additions & 0 deletions .agents/skills/screenshot-triage.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
# Screenshot triage

Find the screenshots most likely to be out of date and rank them so a human
can retake them. You never retake, edit, or delete an image yourself.

By default, scan the whole repo. A ranking only means something across the
whole site. If the user names a page or section, limit the scan to the images
those pages use.

You need a clone of `OpenFn/lightning` with full history, because the whole
method is about comparing dates.

## Remembering what each image shows

Keep a file called `screenshot-map.yml` at the repo root. For each image it
records what the image shows, which UI area that maps to, and how confident
you were. Read it at the start of every run. Only classify images that are
new, renamed, or missing from the file, then add them. Dates are always
recomputed; classifications are not. This makes a repeat scan of the whole
repo cheap.

Humans can edit this file to correct a classification, and the correction
sticks. Put a short comment at the top explaining the format.

## Steps

1. **List the images.** For the whole repo, everything in `static/img/`,
noting any image no page uses. For a page or section, only the images
those pages link as `/img/...`.

2. **Find out how old each image is** from its last commit in this repo. If
the last commit was a bulk optimisation that touched lots of images, look
at the one before it.

3. **Work out what each image shows**, for images not already in
`screenshot-map.yml`. Use the file name, the alt text, and the paragraph
around it. Say how confident you are. Then match it to the part of the
Lightning code that draws that screen. Roughly: the workflow
canvas is under `assets/js/workflow-diagram`; the step editor, runs,
credentials, and project settings each have their own folder under
`lib/lightning_web/live/`; global styling is in `assets/css` and
`lib/lightning_web/components`. Screenshots of other products (Kobo,
DHIS2) have no matching code; mark them "external".

4. **Find out when that part of the UI last changed.** Take the newest commit
touching the matching code, or the global styling, whichever is later. Skim
the commit messages since the image was taken and keep the ones that sound
visible to users (renamed, moved, redesigned, added a button).

5. **Flag and rank.** An image is a suspect if the UI changed after it was
taken. Sort by the size of the gap, then by how many visible changes
happened in it, and give pages in Get Started and Tutorials a nudge up the
list. If you were not confident about what an image shows and it lands
near the top, say so.

## What to report

A table with: image, page and line, image date, what it shows and your
confidence, date of the last UI change, the gap in days, and what probably
changed. List external, unused, and diagram images separately. Put the top
fifteen in the PR and collapse the rest.

Whole-repo scans are report-only apart from updating `screenshot-map.yml`.
When scoped to a page or section, you may also correct alt text that describes
an image wrongly.

## Later: taking screenshots automatically

Not built yet. Lightning already has Playwright browser tests under
`assets/test/e2e/specs/`. When those can produce screenshots, add a final
step: a mapping file that says which test reaches which screenshot, run the
test with a capture flag against a local Lightning, save the result over the
old image, and present the before-and-after in the PR as a suggestion for a
human to approve.
Loading