Skip to content

docs(flaky-tests): publish the v2 API and split the reference by scope - #329

Draft
TylerJang27 wants to merge 1 commit into
test-collections/dashboardfrom
test-collections/api-reference
Draft

docs(flaky-tests): publish the v2 API and split the reference by scope#329
TylerJang27 wants to merge 1 commit into
test-collections/dashboardfrom
test-collections/api-reference

Conversation

@TylerJang27

@TylerJang27 TylerJang27 commented Sep 9, 2026

Copy link
Copy Markdown
Contributor
Screenshot 2026-09-09 at 3 08 47 PM

Important

Do not land until the default test_collection_status enum is flipped. This page is live, and it presents collection endpoints as the primary scope.

Overview

PR 5 of the Test Collections docs stack: publishes the v2 OpenAPI spec and splits the Flaky Tests API reference by scope, so a reader on collections can find the endpoints that apply to them.

Details

Stacked on #325#324#323. Review those first.

Three changes:

  • openapi-v2.json — generated by pnpm emit:openapi in ts/apps/public-api, copied verbatim (240 KB).
  • docs.json — a sibling API reference (v2) nav group declaring the seven flaky-tests operations.
  • api-reference.mdx — rewritten to lead with the scope split, with a mapping table between the two.

Considerations

  • The spec carries 35 operations; only 7 are Flaky Tests. The rest are merge-queue, dynamic-ci and repositories. The nav declares the seven explicitly rather than pointing the whole file at this section, so nothing unrelated appears under Flaky Tests. If merge-queue ever wants its v2 endpoints documented, it can declare its own group against the same file.
  • The v1 endpoint pages keep their URLs. My first attempt nested the v1 group under a new parent, which would have moved live pages; this adds a sibling group instead and leaves the v1 group untouched.
  • The mapping table is the point of the page. The two scopes are separate APIs, not two versions of one, so "what's the collection equivalent of X?" has no general answer — and for linking a ticket to a test case there is currently no equivalent, which the table says outright rather than leaving to be inferred from absence.
  • A test case's ID is not the same value in both scopes, so an ID read from one cannot be passed to the other. Called out in a note, since it is the kind of thing that produces a confusing 404 rather than a clear error.
  • The spec is a copy, not a live pull, so it will drift as endpoints change. Regenerate with pnpm emit:openapi in ts/apps/public-api and re-copy. Worth a follow-up to automate.
  • Endpoint page URLs include the OpenAPI tag as a path segment (…/v2/testcollections/list-test-collections), which the v1 pages do not have because their operations are untagged. Anything linking to a v2 endpoint page needs that segment.

Testing

Ran the Mintlify CLI locally, which had not been done anywhere in this stack:

  • mint validate (strict — fails on warnings) passes.
  • mint broken-links reports none, across the whole stack rather than just this PR.
  • mint dev served the site, and I confirmed by HTTP status that the seven new v2 endpoint pages render, the five v1 endpoint pages still render at their original URLs, and the stack's other pages (dashboard, migrate-to-test-collections, get-started/test-collections) render.

That last step is also how the endpoint URLs were determined — I had guessed them from the operation summaries and all seven 404'd; the real hrefs came from the built navigation.

Follow-up Work

  • PR 6: the flip — unhide the concept and migration pages and add their nav entries.
  • Automating the v2 spec copy, so the published spec cannot drift from the service.

References

🤖 Generated with Claude Code

The Flaky Tests API reference documented only the repository-scoped v1
endpoints, and said nothing about scope at all — so a reader on
collections had no way to find the endpoints that apply to them.

Adds `openapi-v2.json`, generated by `pnpm emit:openapi` in
`ts/apps/public-api` and copied verbatim. The nav declares only the seven
flaky-tests operations from it, so the merge-queue and dynamic-ci
operations the same file carries do not appear under Flaky Tests.

The page now leads with the split — collection endpoints first, then
repository endpoints — and closes with a mapping table, because the two
are separate APIs rather than two versions of one and not every
repository endpoint has a collection counterpart. The one that does not
says so. It also records that a test case's ID differs between the two
scopes, so an ID from one cannot be passed to the other.

The v1 endpoint pages keep their existing URLs; only a sibling nav group
is added beside them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant