docs(flaky-tests): replace the dashboard's landing page for collections - #325
docs(flaky-tests): replace the dashboard's landing page for collections#325TylerJang27 wants to merge 2 commits into
Conversation
`/<org>/flaky-tests` lands on test collections, so the page opened on a premise that is false for any organization past the migration — it described a repositories overview at a URL that no longer serves one. The collections list is documented in its place, column by column, including the two columns repositories have no equivalent for: the Collection ID that CI passes, and the Setup count. Two deliberate absences are stated rather than left to be filed as bugs — there is no total row, because a flaky count summed across unrelated collections is not actionable, and side-by-side comparison is not available yet. The collection dashboard replaces "Key repository metrics": the range toggle, the three impact cards, the three daily charts — including that Test Status Changes counts transitions rather than totals — and the filter bar with its three top-ten tables, whose repository filter is the one place a collection exceeds repository parity. The repository view keeps a subsection, since it stays reachable behind **Legacy view** during a migration, and it retains the quarantine icons and inactive-repository behaviour. Everything below the fold is scope-agnostic already — test cases, test details, the Summary and Test History tabs — and is untouched. Reuses the collection overview screenshot from the concept page rather than adding a second copy; the collections list still needs one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
62a5566 to
722ddd0
Compare
| ## Collections overview | ||
|
|
||
| When you navigate to `/<your-org>/flaky-tests`, you land on a repositories overview showing all monitored repositories at a glance. | ||
| When you navigate to `/<your-org>/flaky-tests`, you land on your [test collections](./get-started/test-collections). The page exists to answer two questions — which collection is unhealthy, and which is not finished being set up — so every row carries both. |
There was a problem hiding this comment.
| When you navigate to `/<your-org>/flaky-tests`, you land on your [test collections](./get-started/test-collections). The page exists to answer two questions — which collection is unhealthy, and which is not finished being set up — so every row carries both. | |
| When you navigate to `/<your-org>/flaky-tests`, you land on your [test collections](./get-started/test-collections). The page exists to answer two questions — what collections have flaky tests, and which are not finished being set up. |
| <Info> | ||
| **Setup** is a count, never a percentage. The number of steps grows as features ship, so `3/4` and `3/5` are honestly different where 75% and 60% would invite a comparison that means nothing. It is the same number the collection's own setup checklist shows. | ||
| </Info> |
There was a problem hiding this comment.
remove this block
| **Setup** is a count, never a percentage. The number of steps grows as features ship, so `3/4` and `3/5` are honestly different where 75% and 60% would invite a comparison that means nothing. It is the same number the collection's own setup checklist shows. | ||
| </Info> | ||
|
|
||
| There is deliberately no total row: a flaky count summed across unrelated collections is not a number anyone acts on, and the page's job is to point at one collection. Comparing two collections side by side is not available yet. |
There was a problem hiding this comment.
remove this block too
Applies the suggested opening, and drops the two blocks called out — the Setup-is-a-count callout and the paragraph about there being no total row. The screenshot filled a gap the prose had wrong: quarantine shield icons appear on **collection** rows too, not only repository ones. They were documented under Repositories alone. The icon table moves up to the collections list where it belongs, and the repositories subsection now refers back to it rather than restating it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
All three applied in 6e81b91, and the collections-list screenshot is in — so this PR has no TODOs left. The screenshot caught a real error in the prose. Quarantine shield icons appear on collection rows too, not only repository ones — The shot is a good one for this page for a reason worth noting: it happens to show Setup at Both files landed as |
Important
Do not land until the default
test_collection_statusenum is flipped. This page is live, and it would otherwise tell every reader that/<org>/flaky-testslands on collections before it does.Overview
PR 4 of the Test Collections docs stack:
dashboard.mdxdocuments the collections list and the collection dashboard, replacing the repositories overview it opened on.Details
Stacked on #324 → #323 → #322. Review those first.
The page opened with "When you navigate to
/<your-org>/flaky-tests, you land on a repositories overview" — false for any organization past the migration, since that URL serves collections. This is a replacement of the two landing sections, not an addition beside them.Everything below the fold — test cases overview, test case details, code owners, the Summary and Test History tabs — is already scope-agnostic and is untouched.
Considerations
3/4and3/5are honestly different where 75% and 60% invite a meaningless comparison.Top 10 Most Unstable Tests,Top 10 Longest-Running Tests,Top 10 Longest-Manually-Quarantined Tests, fromoverview-tables-section.tsx), and the status-changes chart documents all three series — Newly Broken included, which the screenshot's legend omits because that window had no broken transitions. Documenting only what the screenshot showed would have left a series out.<Info>callouts further down the page (the framework caveats) is left alone — confirmed it pre-dates the stack.Testing
key-metrics-*images dropped from this page are still referenced byoverview.mdx, so nothing is orphaned.mint dev.Follow-up Work
References
🤖 Generated with Claude Code